1. 2/2/2009
E2E QoE: User-Centric Approach
Closing event E2E QoE @IMEC, Leuven, 29/01/09
Quality of Experience: user-centric approach
WP3.1 User settings
Definition and conceptualisation
Measurement of QoE
Environmental scan
…
WP3.2 Business perspectives
Innovation policy
Business models
…
WP3.3 Legal and regulatory issues
Challenges in frequency management
New technologies and ex ante regulation
…
1

2. 2/2/2009
Presentation overview
Evaluating Quality of Experience: bridging the gap between
technical parameters and human experience factors
Katrien De Moor - Lieven De Marez (IBBT-MICT-UGent)
Situating Net Neutrality in a broader context. Towards an
analytical framework for the distribution of content on the
Internet
Leo Van Audenhove - Luciano Morganti (IBBT-SMIT-VUB)
The European Response to Network Neutrality in the Context
of the Electronic Communications Reform
Liyang Hou (IBBT - ICRI - KU Leuven)
UGent
Evaluating Quality of Experience:
bridging the gap between technical parameters
and human experience factors
Katrien De Moor
Lieven De Marez
Closing event E2E QoE @IMEC, Leuven, 29/01/09
2

3. 2/2/2009
Introduction
WP3 User settings
Context: current ICT environment
Changed role of the user
Bottom-up approaches, user-driven innovation, …
UGent
5
Context: ‘Experience’ as USP
Quality of Experience, User Experience, … is omnipresent!
UGent
3

4. 2/2/2009
Focus on two challenges
1. Conceptualisation and definition:
Positioning QoE – UX – QoS
Which factors are affecting (mobile) QoE?
2. Measuring QoE:
Delivering high QoE: how?
UGent
Relevance?
Good QoE users
Find ways to enhance user’s QoE with existing
applications
li ti
Anticipate future killer applications
UGent
http://mobchina.blogspot.com
4

5. 2/2/2009
Challenge 1: conceptualisation of QoE
QoE definition and conceptualisation
Lack of clear definition/conceptualisation
Literature is scattered
Inconsistency and confusion
Scope usually too narrow (~QoS, ~Usability)
Creation of conceptual model
Input from desk research and expert panel
5 main building blocks, many subdimensions
g y
Emphasis on multidimensional character of QoE
Objective and subjective dimensions
Expectations
Context
… UGent
Expectations
Context Usability
QoE
DIMENSIONS
Quality of Quality of
Efficienty Effectiveness
5

6. 2/2/2009
… not only what the technology
QoE is … can do …
… but also what people (can) do with the
technology…
… what people want/expect
from it …
…in a certain context…
in context
Challenge 2: Measuring QoE
Currently: emphasis on post-development measurements (3),
usability and QoS measurements (2)
1
2
Prior-to-development & 3
Prior-to-launch (Post)Development
& Prior-to-launch Post development &
Post-launch
Prototypes, demo’s,
???? ….
Expectations?
E t ti ? Usability testing User research:
QoS optimisation Satisfaction
measurements
Good experience?
UGent
12
6

7. 2/2/2009
Measuring and enhancing QoE: how?
Measuring all dimensions: objective ánd subjective
Interdisciplinary and more anticipatory approach
QoE-expectations before usage / development
Q E t ti b f d l t
Actual experience during and after usage
QoE measurement as a ‘flow’
“QoE is a journey rather than a destination”
j y
(Enterprise Management Associates, 2001)
Empirical case-studies
4 common scenarios:
Video-on-Demand
Videoconferencing
3D-Gaming
Networked video sensors
7

8. 2/2/2009
Development of new methodology (MICT-WiCa)
Focus on:
Relation between technical parameters and human
experience factors
p
WHAT is happening on the network?
WHY is the user behaving in a certain way?
Pre- during and post-usage
First tests:
Mobile wineguide
Wapedia-application: case-study (N=10) in
controlled research setting
Results relate network QoS in a mobile setting to
perception of the user
Illustration of five-step methodology
Pre-usage user research
Step 1 - detection of relevant user experience dimensions and expectations
- multi-method approach
Pre-usage translation workshops
- find optimal match between ‘user-indicated’ QoE dimensions and
Step 2
‘measurable QoS parameters’ (e.g. Simulation exercises)
- social scientists + engineers
Monitoring during usage
g g g
- usage scenarios for test users
Step 3 - e.g. case-study: different reception levels + monitoring of ‘signal
strength’
- software probe model (cfr. Deryckere, Joseph et al, 2008)
8

9. 2/2/2009
Illustration of five-step methodology
Post-usage questions on device
Step 4 - after completion of usage scenario questions on device (general
experience, frustration, speed, …)
Post-usage Comparison (expectations vs.
Step 5 experience)
- user experience gaps? Multi-method approach cfr. phase 1
reduction in speed (lower
[dBm]
general experience drops
Illustration of five-step methodology
9

10. 2/2/2009
What’s next?
IBBT GR@SP project
Refinement of relevant dimensions and QoE metrics
Large-scale QoE-research in real-life contexts / Living Lab
Creation of interdisciplinary short- and long-term evaluation
tools
Combine objective monitoring of various parameters with
subjective human experience factors
Extend with contextual information (e.g. context-aware
context aware
experience sampling)
…
Thank you!
Q&A?
KatrienR.DeMoor@Ugent.be
Lieven.DeMarez@Ugent.be
www.mict.be
www mict be
10