Click here to watch the full webinar: http://www.spirent.com/OnDemand/Servi...
This webinar discusses the increasing importance of user experience measurement and analysis in live networks and reviews best practices and actual data from early VoLTE deployments.
Service providers are more focused on user experience than ever before, actively measuring and comparing the experience of voice over LTE (VoLTE) and IR.94 services to over-the-top (OTT) and legacy services to determine if VoLTE devices and services are ready to launch. Their goal is to deliver a user experience that's better than all alternative options. This webinar discusses the increasing importance of user experience measurement and analysis in live networks and reviews best practices and actual data from early VoLTE deployments
Key Topics of discussion in this webinar are:
• Why user experience is of increasing importance to service providers rolling out VoLTE
• The general status of VoLTE deployments, including the current availability of VoLTE-enabled devices and services
• Launching IR.94 video calling and VoLTE services, either in tandem or close together
• The competitive dynamic between VoLTE and OTT and legacy voice services: why it matters
• Technical advantages of VoLTE services vs. OTT when the mobile network is congested
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Spirent: Improving the VoLTE Experience
1. Improving the VoLTE Experience:
Best Practices from Early Launches
An Infonetics Research Webinar
Co-produced with Spirent Communications
#VoLTE
2. Today’s Speakers
2
JoAnne Emery
Event Director
Infonetics Research
(Moderator)
Rich McNally
Director
Service Experience Programs
Spirent Communications
Stéphane Téral
Principal Analyst,
Mobile Infrastructure and
Carrier Economics
Infonetics Research
Eric Sinclair
Manager
Service Experience Programs
Spirent Communications
#VoLTE
5. ‣ 1.5 billion active users
worldwide of OTT mVoIP
in 2013…
• Mobile operators using OTT for
subscribers travelling out
of country
• Activity continues to expand in
Japan, China, and South Korea
‣ …that will taper as users
whittle choices down and
over-penetration settles out
‣ Proliferation of smartphones
fueling growth
• We expect OTT mVoIP subscribers to more than double from 2013 to 2018, to 3.1 billion
0
500
1,000
1,500
2,000
2,500
3,000
3,500
CY11 CY12 CY13 CY14 CY15 CY16 CY17 CY18
Subscribers(Millions)
OTT mVoIP VoLTE
5Infonetics Research: 2G, 3G, LTE Mobile Services and Subscribers Market Size and Forecasts, June 2014
Although OTT Dominates mVoIP…
7. 0
20
40
60
80
100
120
140
2009 2010 2011 2012 2013 2014
Cumulative W-AMR HD Voice Networks
HSPA commercial VoLTE commercial
‣ 119 mobile operators offer HD
voice with W-AMR* as key
HD voice enabler
• 95 on HSPA
• 12 on LTE
(VoLTE + IR.94 video calling)
• 8 on GSM/HSPA
• 2 on HSPA/LTE
• 2 on GSM
‣ 92 smartphones (including
carrier and frequency variants)
support VoLTE
• Including products by Asus, Huawei, LG, Pantech, Samsung, and Sony Mobile
• The new Apple iPhone 6 & 6 Plus models support VoLTE
Source: GSA, September 23, 2014
7*W-AMR: Wideband Adaptive Multi-Rate codec
VoLTE Taking Off, Marketed as HD Voice…
9. The Feedback Is Extremely Positive so Far
‣ Connection is super fast, ringtone sound quality exceptional
‣ Virtually no background noise; voice is crystal clear
‣ Video call quality is crisp and sound
• Superior to Skype and other OTT apps
9
The ultimate goal is to keep every subscriber on your network
with the carrier-grade service features that beat OTTs
10. There Is Little Surprise!
‣ VoLTE is reliable
• Offers guaranteed QoS
‣ VoLTE provides high quality
• Uses W-AMR codecs
(12.65kbps or 23.85kpbs)
and QoS Class Identifier
(QCI) SIP signaling
‣ VoLTE is spectrally efficient
• Due to LTE’s all-IP architecture and new features in 3GPP releases —
such as MIMO antenna technology — voice requires less bandwidth in
LTE spectrum than it does in 2G/3G networks
10
Source: Nokia Networks, 2012
13. The Experience Matters
13
‣ VoLTE needs to deliver a better user
experience than OTT and legacy services
• There is only one way to truly know if new devices
and services are ready to launch
• You need to measure the experience at the
device, using actual consumer devices
‣ Spirent has been in the trenches, rolling out
VoLTE with leading operators for 2 years
• Following are some best practices for evaluating
and improving VoLTE…
• …to ensure a successful launch of new devices
and services
15. Practice #1: Measure What Matters
15
‣ Our experience:
• Functional testing is increasingly insufficient for
assuring user experience
• Combined live network and lab evaluation of user
experience (UX) is a better approach
• If you don’t measure everything that matters to
users, problems will emerge
‣ Best practices:
• Focus on objective assessment of what the
user experiences
• Measure all key factors impacting user experience
• Measure quality and consistency
MOS (wideband
mobile-to-mobile and
narrowband landline)
Mobile Originated and
Terminated Block/Drop
Rates
Conversational Audio
Delay
Video Delivery
Audio / Video Sync
Battery Life
VoLTE UX Metrics
16. Practice #2: Stress the Device Like a User
16
‣ Our experience:
• Measurements of UX are only valid if they
reflect real user behavior
• Multi-service voice and data usage
stresses the device
• Dropped calls and other UX elements are
impacted by multi-service usage
‣ Best practices:
• Implement multi-service use cases for
pre-launch device evaluation
• For example, set up continuous push
emails during test calls
1.1%
0.9%
1.1%
Manufacturer 1 Manufacturer 2 Manufacturer 3
Voice only Multi-service
Dropped Call Rate by
Device Manufacturer
Voice Only Vs. Multi-Service
1.3%
1.4%
1.9%
17. Practice #3: Rank Devices (and Services)
17
‣ Our experience:
• There is a wide variance in UX across
device models
• It’s a competitive marketplace - showing
rank by UX category is a powerful motivator
‣ Best practices:
• Consider the relative performance of
devices and not just the absolute
• Rank and compare all pre-launch devices
by UX category
• Set thresholds based on population
performance (and raise over time) No one wants to be in this range
Speech Quality
Downlink Uplink
Device A
Device A
Device B Device B
Device D
Device D
Device C
Device C
Device E
Device E
18. Practice #4: Assess from the Top Down
18
‣ Our experience:
• You can spend a lot of time and money
gathering stack-loads of data you don’t use
• Broad statistical assessments with focused
drilldown into problems saves time
• RTP and RF tracing can provide key insight into
root causes of poor experience
‣ Best practices:
• Only collect RTP/IP and RF DM logs at UE
(and IMS) where problems are identified or in
focused pre-planned instances
• Use the data to accelerate triage/isolation of
UX issues to network/device/service 0
10
20
30
40
50
60
200 300 400 500 600
Packets/s
Received Packets/s Missing Packets/s
HO HO
Isolating the Root Causes of
Poor VoLTE Speech Quality
0
1
2
3
4
5
0 100 200 300 400 500 600 700 800
DownlinkMOS
RTP Metrics Help Identify Cell-Specific Issue
20. Case Study: VoLTE Speech Quality
20
‣ Drive test of live US VoLTE
network with varying RF
‣ MOS very flat across good, fair
and poor RF environments
• Average speech quality mean
opinion score (MOS) of 3.5
• In very poor RF conditions, MOS
tends to degrade rapidly
‣ RTP jitter and delay also flat
across varying RF conditions
• In very poor RF, RTP performance
drops off rapidly
22.1
17.2 16.8 16.2 16.6 16.2 15.9
-105 -100 -95 -90 -85 -80 -75
Average RTP Jitter (ms) by RSRP (dBm)
3.55 3.53 3.463.53 3.48 3.52
Good Fair Poor
Device 1
Device 2
VoLTE MOS (POLQA) by RF Environment
21. Case Study: Typical Causes of Poor MOS
21
‣ Week-long drive test of VoLTE in
major US city
‣ RF issues are most common and
fairly easy to detect
• Distant or no dominant server
• Interference / weak server
• Dragging handover
‣ IP (RTP) issues also common
• Configuration / IP connectivity
‣ Other issues
• No apparent RF or IP issues
• Device issue related to buffering,
packet re-assembly or codec
RF
79%
IP
16%
Other
5%
Root cause of VoLTE MOS < 2.5 by category
for week long drive test of major US city
22. ‣ Are problems due to:
device, network, or interaction?
‣ Comparative scorecards help
identify the source of issues
‣ Scorecard on right comes from
tests on a live VoLTE network
• Network RF conditions are good
• Device 2 is less efficient at using
air-interface resources, but…
• RTP performance is okay
• Ideal MOS from RTP stream shows
speech quality should be good
• Device 2 jitter buffer or codec is
not working properly!
Case Study: Isolating Root Causes
22
VoLTE Analytics Scorecard
Overall Rank Device 1 Device 2
User Experience
Speech Quality
Network – RF Conditions
Received Power (RSRP)
Signal Quality (RSRQ)
Interference (CINR)
Device-Network Interaction
Resource Allocation
Modulation Efficiency
RTP – Packet Jitter
RTP – Packet Delay
RTP – Packet Loss
Device Buffering and Decoding
Ideal MOS from RTP
Actual - Ideal MOS
23. Case Study: Isolating Root Causes
23
‣ Used RTP analysis to isolate issue
with service infrastructure / IP layer
‣ Collected IP sniffer data via device and
server-side logs in area with low MOS
• RF is good
• RTP jitter and delay is okay
• RTP packet loss rate very high
‣ Diagnosis
• The network is working and packets are
getting through but…
• There is a major IP connectivity issue
leading to dropped packets
-86.3
-98.7
Serving Cell Neighbor Cell
RSRP
One location: VoLTE MOS < 2.5 with
good RF but packet loss rate > 9%!
RTP Jitter: 14 ms
RTP Delay: 20 ms
RTP PLR: 9.3%
RF
RTP
RSRQ: -12.3
CINR: 4.6
24. Case Study: Making Improvements
24
‣ Results from live network testing
of VoLTE and IR.94 video
‣ Key factors that matter to users:
• Speech quality (MOS)
• Frozen or impaired frame rate
‣ Prior to optimization, VoLTE
performs poorly
‣ After optimization VoLTE is 1st
or 2nd best in all UX categories
3.4%
13.2%
19.5%
25.3%
Frozen/Impaired Frames
IR.94 (Optimized)
OTT 2
OTT 1
IR.94 (Pre-
optimization)
Frozen/Impaired Frames for IR.94 Video vs. OTT
2.4
3.0
3.3
3.6
3.8
Speech Quality (POLQA MOS)
OTT 1
VoLTE (Pre-
optimization)
Circuit-Switched
VoLTE (Optimized)
OTT 2
VoLTE vs. OTT and CS Speech Quality (HD Codec)
Optimization
improvement
Optimization
improvement
26. 26
Partnering with service providers and device manufacturers to
improve the user experience of devices and services
Over 2 years F4L program
led to decreased variance
between devices, more
consistent speech quality
F4L program also led to
better average speech
quality across portfolio
Effect of Fit4Launch on Portfolio Speech
Quality over first 2 years
Fit4Launch programs for
evaluating user experience
of pre-launch devices are
deployed at 3 US operators
Improving User Experience
27. 27
Inventing systems and methodologies for measuring and analyzing
the user experience of mobile devices and services
Inventing Systems and Methods
Location
2003 2008 2010 2012 2013 2014
Live Lab
Spirent User Experience Analytics timeline
Speech
File Transfer &
Web Browsing Call Battery Life
Voice & Video
Calling
2011
Video
29. ‣ With the RF in order, beat the OTT with voice and video calling
over LTE!
• Video calling over LTE is very challenging because it requires low
latency, low error rate and high bit rate
• With the GSMA toolbox, IR.92 and IR.94, when the LTE network is
congested, VoLTE and video calling services can use LTE QoS
features to prioritize packets to ensure service quality
- Pure OTT can’t
‣ With 71 operators currently investing in VoLTE studies, trials or
deployments, it’s only a matter of time for mobile operators to
retake their destiny
29
The VoLTE Advantage
30. Thank you for your interest!
30
Watch the full webinar on the link given below:
http://www.spirent.com/OnDemand/Service_Exp
erience/Improving-the-VoLTE-Experience