SlideShare una empresa de Scribd logo
1 de 30
Descargar para leer sin conexión
1Spirent Communications PROPRIETARY AND CONFIDENTIAL
VoWi-Fi Testing
Challenges
Measuring the
Quality of Experience
in the field and lab
2Spirent Communications PROPRIETARY AND CONFIDENTIAL
Agenda
Part 1: VoWi-Fi technology landscape
 Drivers for Wi-Fi adoption
 VoWi-Fi QoE Challenges
Part 2: SRG VoWi-Fi field-based study results
 Wi-Fi study background and motivation
 Study setup and measurement methodology
 VoWi-Fi Field test results
Part 3: Evaluating VoWi-Fi performance in the lab
 Advantages of lab-based testing
 Evaluating Key Performance Indicators (KPIs) in the lab
 Future deployment and test considerations
 Audience Q&A
3Spirent Communications PROPRIETARY AND CONFIDENTIAL
Drivers for Wi-Fi adoption
Why Wi-Fi is an attractive choice as a technology
4Spirent Communications PROPRIETARY AND CONFIDENTIAL
VoWi-Fi QoE Challenges
Offloading Algorithms
• When and why?
• When to go back?
Assessing Signal Strength and Integrity
• RSSI measurements before and
during offload
IP Layer
• Best effort delivery
• Access Point variations
Authorization
• ePDG authentication for
unsecured Wi-Fi access
5Spirent Communications PROPRIETARY AND CONFIDENTIAL
KPIs used to measure VoWi-Fi Quality of Experience (QoE)
 Voice Quality:
 MOS Score – Uplink and Downlink
 Packet (Audio) loss percentage
 Conversational (Ear-to-mouth) delay
 Call reliability:
 Percentage of successful handovers in both directions under “expected” circumstances
 Percentage of call drop under various ePDG adversarial scenarios (lab-based simulation)
 Percentage of successful handovers when access point is loaded
 Time taken for call setup (for Wi-Fi originated calls)
 Time taken for LTE<-> Wi-Fi handover
6Spirent Communications PROPRIETARY AND CONFIDENTIAL
Part 1: VoWi-Fi technology landscape
 Drivers for Wi-Fi adoption
 VoWi-Fi QoE Challenges
Part 2: SRG VoWi-Fi field-based study results
 Wi-Fi study background and motivation
 Study setup and measurement methodology
 VoWi-Fi Field test results
Part 3: Evaluating VoWi-Fi performance in the lab
 Advantages of lab-based testing
 Evaluating Key Performance Indicators (KPIs) in the lab
 Future deployment and test considerations
 Audience Q&A
7Spirent Communications PROPRIETARY AND CONFIDENTIAL
Background and rationale for the study
 Spirent and SRG have collaborated on all-things-VoLTE for the last 15 months
 VoWi-Fi is the rage
 Although operators are rushing to embrace the technology – sometimes in
advance of VoLTE – there isn’t a clear understanding how the technology
performs
 VoWi-Fi is a natural extension of VoLTE, given the interoperability that should
exist between the two voice solutions – also doesn’t hurt that HD Voice is
supported across all 3 RAN components of the operator’s network
8Spirent Communications PROPRIETARY AND CONFIDENTIAL
Study setup and test methodology
 Spirent Communications provided Nomad User Experience system
 Accuver X-Cal M data collection tool and Accuver XCAP post-processing software
for logging all lower layer parameters
 The testing relied on mobile-to-mobile calls – implies that the uplink MOS of one
device can affect the downlink MOS of the other
 Testing included VoLTE-VoWi-Fi handovers and vice versa, as well as call quality
with different device configurations (3G only, Wi-Fi- only, etc.)
 VoWi-Fi voice quality measured under realistic traffic loading at residential and
public Wi-Fi access points
9Spirent Communications PROPRIETARY AND CONFIDENTIAL
Procedures:
 Introduced additional background traffic onto same Wi-Fi access point while performing voice quality test
Observations:
 Streaming HD Video (2x) had very little impact on voice quality when the downlink consisted of two video
streams
Field test results: residential Wi-Fi access point
How does the
performance look
when a device using
VoWi-Fi calling
on a residential Wi-Fi
access point is loaded
on the downlink
10Spirent Communications PROPRIETARY AND CONFIDENTIAL
Field test results: residential Wi-Fi access point
How does the
performance look
when a device using
VoWi-Fi calling
on a residential Wi-Fi
access point is loaded
on the uplink
Procedures:
 Transferred emails with ~10 MB files attached over same Wi-Fi AP being use to support voice calls
Observations:
 Relatively modest amounts of uplink traffic can impact call quality, including no voice detection
 As implemented, VoWi-Fi does not support QoS so all traffic is treated as best effort
11Spirent Communications PROPRIETARY AND CONFIDENTIAL
Field test results: public Wi-Fi access point
How does the
performance look
when two
smartphones are
using VoWi-Fi
in a public Wi-Fi
access point
Procedures:
 Stationary testing at airport with both smartphones using VoWi-Fi and the same SSID
– network seemed to be lightly loaded
Observations:
 Bigger variances in the results and lower scores, despite lightly-loaded network
– VoWi-Fi on both ends increases impact
 Quality of network has a major influence on call quality
12Spirent Communications PROPRIETARY AND CONFIDENTIAL
Field test results: public Wi-Fi access point
How does the
performance look
when a device
using VoWiFi calls
another device on
VoWiFi on a public
Wi-Fi Access point
Procedures:
 Comparable to airport test but done at San Diego Convention Center – 18,000 attendees for a conference
Observations:
 Less variance in the results compared with the airport test
 Quality of network has a major influence on call quality – in this case a good thing
13Spirent Communications PROPRIETARY AND CONFIDENTIAL
Field test results: VoWi-Fi to VoLTE handovers
How does the
performance look
when a device using
VoWi-Fi hands over
from a residential
Wi-Fi Access Point
to VoLTE
Procedures:
 Stationary device, which is placed near Wi-Fi AP, uses VoLTE; Second device was mobile – started on VoWi-Fi
 Walked down the street pushing the “little red wagon” until the call switched to VoLTE
 Returned back to Wi-Fi AP and waited for handover to Wi-Fi before walking again of Wi-Fi coverage
Observations:
 No impact to MOS Score during handover; also hard to detect handover by listening to the call
 Handovers back into VoWi-Fi take longer (no surprise) while outbound handovers aren’t predictable
14Spirent Communications PROPRIETARY AND CONFIDENTIAL
Field test results: VoWi-Fi to VoLTE handovers
How does the network
and device activity
look when a device
switches between
VoLTE and VoWi-Fi
Procedures:
 One device configured for 3G voice only; second device configured for VoLTE / VoWi-Fi
 Originate call within Wi-Fi AP coverage and then proceed to walk out of coverage with both phones in hand
 Continue until handover occurs and then walk back into coverage; repeat as necessary
Observations:
 Everything consistent with expectations; lack of QoS with VoWi-Fi is evident
15Spirent Communications PROPRIETARY AND CONFIDENTIAL
Field test results: VoWi-Fi to VoLTE handovers, II
How does the
network and device
activity look when a
device switches
between VoLTE and
VoWi-Fi
Procedures:
 One device (S5) configured for 3G voice only; second device (S6) configured for VoLTE / VoWi-Fi
 Originate call within Wi-Fi AP coverage and then proceed to walk out of coverage with both phones in hand
 Continue until handover occurs and then walk back into coverage; repeat as necessary
Observations:
 S5 used UMTS as intended but S6 didn’t use VoWi-Fi, instead using a combination of UMTS and 2G (NB-AMR)
 Call drops when we returned to Wi-Fi coverage (repeated with same results)
16Spirent Communications PROPRIETARY AND CONFIDENTIAL
Key observations
 The benefits of VoWi-Fi are clear but it isn’t a panacea
 VoWi-Fi is as good or as bad as the underlying Wi-Fi network
 VoWi-Fi handovers with the LTE network generally worked but there were
some exceptions
 Problems, which we attribute to network/IMS registration and device
interoperability, exist
17Spirent Communications PROPRIETARY AND CONFIDENTIAL
Part 1: VoWi-Fi technology landscape
 Drivers for Wi-Fi adoption
 VoWi-Fi QoE Challenges
Part 2: SRG VoWi-Fi field-based study results
 Wi-Fi study background and motivation
 Study setup and measurement methodology
 VoWi-Fi Field test results
Part 3: Evaluating VoWi-Fi performance in the lab
 Advantages of lab-based testing
 Evaluating Key Performance Indicators (KPIs) in the lab
 Future deployment and test considerations
 Audience Q&A
18Spirent Communications PROPRIETARY AND CONFIDENTIAL
Advantages of evaluating VoWi-Fi in the lab
 Audio quality issues related to low MOS scores, audio (packet) loss and codec
(AMR-NB/AMR-WB/EVS) negotiation failure
 SIP registration issues
 ePDG authentication failure
 Handover at power levels higher/lower than that mandated by operator test plans
 Poor QoE resulting from UE toggling between LTE – Wi-Fi networks
 Does not require a wagon
19Spirent Communications PROPRIETARY AND CONFIDENTIAL
Advantages of evaluating VoWi-Fi in the lab
VoLTE
VoWi-FieAP AKA
IPSec tunnel
VoWi-Fi
VoWi-Fi
20Spirent Communications PROPRIETARY AND CONFIDENTIAL
Evaluating VoWi-FI KPIs in the lab
Test scenario Measurable KPIs QoE impact
Voicequality
Test voice quality when the Wi-Fi
power is in the edge condition of
making a decision on handover
from LTE-Wi-Fi or vice-versa and
is toggling between the two
 MOS Score
 Packet loss percentage
 Network latency
 Jitter
 Lower MOS Score => lower
voice quality
 High packet loss =>
Intermittent drops in speech
 High network latency =>
Variable delays in speech
Test voice quality in the
presence of loading on the Wi-Fi
access point and tolerance of
UE to larger network delays
Test voice quality of two test
devices against a reference
“soft” client, both on the Uplink
and the Downlink
21Spirent Communications PROPRIETARY AND CONFIDENTIAL
Evaluating VoWi-Fi KPIs in the lab
Test scenario Measurable KPIs QoE impact
Callreliability
Test the reliability of the call
through the handover process
and time for handover by control
of Wi-Fi power over multiple
iterations
 Percentage of successful call
completion
 Handover time from
LTE – Wi-Fi
 Larger handover time =>
periods of voice discontinuity
 Lower percentage of call
completion => Increased
dropped calls during handover
 Improper response to ePDG
adversarial scenarios => call
drops during handover
Test the reliability of the call
when the UE is sent adversarial
responses by ePDG
Test the reliability of the call
through the handover process
and time for handover by control
of Wi-Fi power over multiple
iterations
22Spirent Communications PROPRIETARY AND CONFIDENTIAL
Test case 1
Test voice quality
between two
devices Device A
and Device B and
measure MOS
score on both
Uplink and
Downlink
Result
Low MOS
score
measured
on Uplink of
Device A
Test Case 1: Voice Quality
Low MOS score on Downlink of Device B in device-to-device call
traced to low Uplink MOS score on Device A
23Spirent Communications PROPRIETARY AND CONFIDENTIAL
Test case 2
ePDG
adversarial
testing scenario
Result
LTE-Wi-Fi
Handover
failure
UE fails to handover and drops call when ePDG issues error code
Test Case 2: ePDG responses
24Spirent Communications PROPRIETARY AND CONFIDENTIAL
Test case 3
Test voice quality when the UE is
toggling between Wi-Fi and
LTE network
Result
Lower MOS Score
Test Case 3: Handover
Packet loss
Root cause identified: Packet loss as a result of device “toggling” between LTE-Wi-Fi
25Spirent Communications PROPRIETARY AND CONFIDENTIAL
Test case 4
Call initiation on Wi-Fi
originated call
Result
Low MOS score
Test Case 4: Codec negotiation
UE1 supporting AMR-WB UE2 supporting only AMR-NB
Root cause identified: Disagreement of codec between UE1 and UE2
26Spirent Communications PROPRIETARY AND CONFIDENTIAL
VoWi-Fi test capabilities and requirements
27Spirent Communications PROPRIETARY AND CONFIDENTIAL
Bringing field study into the lab
SRG and Spirent partner on evaluating VoWi-Fi QoE in the lab
 LTE EPC Configuration
 IMS registration call flow
 Wi-Fi authentication
 ePDG Adversarial test condition
 IMS test agent simulation
28Spirent Communications PROPRIETARY AND CONFIDENTIAL
Additional VoWi-Fi QoE considerations
 Service request type
 Wi-Fi / LTE power level
 Capacity/loading on Wi-Fi radio link
 Measuring impact on voice quality while introducing fading and noise on radio
interface of Wi-Fi
 Testing audio quality and call performance with different classes of access points
(Home, Enterprise, public Wi-Fi hotspots)
 Testing performance in the presence of introducing IP impairments on the IP
backhaul
 Testing the impact of performance in the presence of interference from other
adjacent bands such as LTE and/or adjacent Wi-Fi hotspots
 ANDSF, Hotspot 2.0…
29Spirent Communications PROPRIETARY AND CONFIDENTIAL
Part 1: VoWi-Fi technology landscape
 Drivers for Wi-Fi adoption
 VoWi-Fi QoE Challenges
Part 2: SRG VoWi-Fi field-based study results
 Wi-Fi study background and motivation
 Study setup and measurement methodology
 VoWi-Fi Field test results
Part 3: Evaluating VoWi-Fi performance in the lab
 Advantages of lab-based testing
 Evaluating Key Performance Indicators (KPIs) in the lab
 Future deployment and test considerations
 Audience Q&A
30Spirent Communications PROPRIETARY AND CONFIDENTIAL
Questions?

Más contenido relacionado

La actualidad más candente

Lessons Learned: Implementing VoLTE Roaming APAC
Lessons Learned: Implementing VoLTE Roaming APAC Lessons Learned: Implementing VoLTE Roaming APAC
Lessons Learned: Implementing VoLTE Roaming APAC Syniverse
 
VoLTE Flows and CS network
VoLTE Flows and CS networkVoLTE Flows and CS network
VoLTE Flows and CS networkKarel Berkovec
 
AIRCOM LTE Webinar 2 - Air Interface
AIRCOM LTE Webinar 2 - Air InterfaceAIRCOM LTE Webinar 2 - Air Interface
AIRCOM LTE Webinar 2 - Air InterfaceAIRCOM International
 
5G_NR_Overview_Architecture_and_Operating_Modes
5G_NR_Overview_Architecture_and_Operating_Modes5G_NR_Overview_Architecture_and_Operating_Modes
5G_NR_Overview_Architecture_and_Operating_ModesAalekh Jain
 
IMS Core Elements
IMS Core ElementsIMS Core Elements
IMS Core ElementsKent Loh
 
5G Services Story
5G Services Story5G Services Story
5G Services StoryEricsson
 
Different Types of Backhaul
Different Types of BackhaulDifferent Types of Backhaul
Different Types of Backhaul3G4G
 
RAN - Intro, I&C & Basic Troubleshooting (3).pptx
RAN - Intro, I&C & Basic Troubleshooting (3).pptxRAN - Intro, I&C & Basic Troubleshooting (3).pptx
RAN - Intro, I&C & Basic Troubleshooting (3).pptxFelix Franco
 
5G RAN fundamentals
5G RAN fundamentals5G RAN fundamentals
5G RAN fundamentalsRavi Sharma
 
Presentation on Vowifi
Presentation on VowifiPresentation on Vowifi
Presentation on Vowifisrishti jain
 
Beginners: 5G Terminology (Updated - Feb 2019)
Beginners: 5G Terminology (Updated - Feb 2019)Beginners: 5G Terminology (Updated - Feb 2019)
Beginners: 5G Terminology (Updated - Feb 2019)3G4G
 
5G Basic Call Flows.pdf
5G Basic Call Flows.pdf5G Basic Call Flows.pdf
5G Basic Call Flows.pdfIbrahimSayed61
 
VoLTE Voice over LTE Explained - Complete End to End VoLTE Overview - What is...
VoLTE Voice over LTE Explained - Complete End to End VoLTE Overview - What is...VoLTE Voice over LTE Explained - Complete End to End VoLTE Overview - What is...
VoLTE Voice over LTE Explained - Complete End to End VoLTE Overview - What is...Vikas Shokeen
 
5G NR: Key features and enhancements
5G NR: Key features and enhancements5G NR: Key features and enhancements
5G NR: Key features and enhancements3G4G
 
SRVCC (Single Radio Voice Call Continuity) in VoLTE & Comparison with CSFB
SRVCC (Single Radio Voice Call Continuity) in VoLTE & Comparison with CSFBSRVCC (Single Radio Voice Call Continuity) in VoLTE & Comparison with CSFB
SRVCC (Single Radio Voice Call Continuity) in VoLTE & Comparison with CSFBVikas Shokeen
 
volte ims network architecture
volte ims network architecturevolte ims network architecture
volte ims network architectureVikas Shokeen
 
Core Network Optimization: The Control Plane, Data Plane & Beyond
Core Network Optimization: The Control Plane, Data Plane & BeyondCore Network Optimization: The Control Plane, Data Plane & Beyond
Core Network Optimization: The Control Plane, Data Plane & BeyondRadisys Corporation
 
volte ims network architecture tutorial - Explained
volte ims network architecture tutorial - Explained volte ims network architecture tutorial - Explained
volte ims network architecture tutorial - Explained Vikas Shokeen
 

La actualidad más candente (20)

Lessons Learned: Implementing VoLTE Roaming APAC
Lessons Learned: Implementing VoLTE Roaming APAC Lessons Learned: Implementing VoLTE Roaming APAC
Lessons Learned: Implementing VoLTE Roaming APAC
 
VoLTE Flows and CS network
VoLTE Flows and CS networkVoLTE Flows and CS network
VoLTE Flows and CS network
 
AIRCOM LTE Webinar 2 - Air Interface
AIRCOM LTE Webinar 2 - Air InterfaceAIRCOM LTE Webinar 2 - Air Interface
AIRCOM LTE Webinar 2 - Air Interface
 
5G_NR_Overview_Architecture_and_Operating_Modes
5G_NR_Overview_Architecture_and_Operating_Modes5G_NR_Overview_Architecture_and_Operating_Modes
5G_NR_Overview_Architecture_and_Operating_Modes
 
IMS Core Elements
IMS Core ElementsIMS Core Elements
IMS Core Elements
 
5G Services Story
5G Services Story5G Services Story
5G Services Story
 
5G Network Slicing
5G Network Slicing5G Network Slicing
5G Network Slicing
 
Different Types of Backhaul
Different Types of BackhaulDifferent Types of Backhaul
Different Types of Backhaul
 
RAN - Intro, I&C & Basic Troubleshooting (3).pptx
RAN - Intro, I&C & Basic Troubleshooting (3).pptxRAN - Intro, I&C & Basic Troubleshooting (3).pptx
RAN - Intro, I&C & Basic Troubleshooting (3).pptx
 
5G RAN fundamentals
5G RAN fundamentals5G RAN fundamentals
5G RAN fundamentals
 
Presentation on Vowifi
Presentation on VowifiPresentation on Vowifi
Presentation on Vowifi
 
Beginners: 5G Terminology (Updated - Feb 2019)
Beginners: 5G Terminology (Updated - Feb 2019)Beginners: 5G Terminology (Updated - Feb 2019)
Beginners: 5G Terminology (Updated - Feb 2019)
 
5G Basic Call Flows.pdf
5G Basic Call Flows.pdf5G Basic Call Flows.pdf
5G Basic Call Flows.pdf
 
VoLTE Voice over LTE Explained - Complete End to End VoLTE Overview - What is...
VoLTE Voice over LTE Explained - Complete End to End VoLTE Overview - What is...VoLTE Voice over LTE Explained - Complete End to End VoLTE Overview - What is...
VoLTE Voice over LTE Explained - Complete End to End VoLTE Overview - What is...
 
5G NR: Key features and enhancements
5G NR: Key features and enhancements5G NR: Key features and enhancements
5G NR: Key features and enhancements
 
SRVCC (Single Radio Voice Call Continuity) in VoLTE & Comparison with CSFB
SRVCC (Single Radio Voice Call Continuity) in VoLTE & Comparison with CSFBSRVCC (Single Radio Voice Call Continuity) in VoLTE & Comparison with CSFB
SRVCC (Single Radio Voice Call Continuity) in VoLTE & Comparison with CSFB
 
volte ims network architecture
volte ims network architecturevolte ims network architecture
volte ims network architecture
 
IMS Registration Flow
IMS Registration FlowIMS Registration Flow
IMS Registration Flow
 
Core Network Optimization: The Control Plane, Data Plane & Beyond
Core Network Optimization: The Control Plane, Data Plane & BeyondCore Network Optimization: The Control Plane, Data Plane & Beyond
Core Network Optimization: The Control Plane, Data Plane & Beyond
 
volte ims network architecture tutorial - Explained
volte ims network architecture tutorial - Explained volte ims network architecture tutorial - Explained
volte ims network architecture tutorial - Explained
 

Similar a VoWiFi testing challenges

gopi-4h2.pptx
gopi-4h2.pptxgopi-4h2.pptx
gopi-4h2.pptxKGANESH34
 
Next Generation Service Platforms for Multimedia and Value Added Services
Next Generation Service Platforms for Multimedia and Value Added ServicesNext Generation Service Platforms for Multimedia and Value Added Services
Next Generation Service Platforms for Multimedia and Value Added ServicesAli Saghaeian
 
Multi Vendor Wireless Channel Interference.docx
Multi Vendor Wireless Channel Interference.docxMulti Vendor Wireless Channel Interference.docx
Multi Vendor Wireless Channel Interference.docxKunal Sharma
 
OIF Interop: The Key to Unlocking the Benefits of SDN
OIF Interop: The Key to Unlocking the Benefits of SDNOIF Interop: The Key to Unlocking the Benefits of SDN
OIF Interop: The Key to Unlocking the Benefits of SDNDeborah Porchivina
 
Cellular Services over WiFi
Cellular Services over WiFiCellular Services over WiFi
Cellular Services over WiFi3G4G
 
Revolutionary Approach to Cell Site Activation
Revolutionary Approach to Cell Site ActivationRevolutionary Approach to Cell Site Activation
Revolutionary Approach to Cell Site ActivationADVA
 
UK Spectrum Policy Forum – Sami Susiaho, BSkyB - Current and future spectrum ...
UK Spectrum Policy Forum – Sami Susiaho, BSkyB - Current and future spectrum ...UK Spectrum Policy Forum – Sami Susiaho, BSkyB - Current and future spectrum ...
UK Spectrum Policy Forum – Sami Susiaho, BSkyB - Current and future spectrum ...techUK
 
THE FUTURE DIRECTIONS IN EVOLVING WI-FI: TECHNOLOGIES, APPLICATIONS AND SERVICES
THE FUTURE DIRECTIONS IN EVOLVING WI-FI: TECHNOLOGIES, APPLICATIONS AND SERVICESTHE FUTURE DIRECTIONS IN EVOLVING WI-FI: TECHNOLOGIES, APPLICATIONS AND SERVICES
THE FUTURE DIRECTIONS IN EVOLVING WI-FI: TECHNOLOGIES, APPLICATIONS AND SERVICESijngnjournal
 
OneZone: Toronto metro-WiFi network overview
OneZone: Toronto metro-WiFi network overviewOneZone: Toronto metro-WiFi network overview
OneZone: Toronto metro-WiFi network overviewPhil Vlach
 
Wifi offload-through-eap-authentication
Wifi offload-through-eap-authenticationWifi offload-through-eap-authentication
Wifi offload-through-eap-authenticationJustus @GreenPacket
 
Wi-Fi Offload Authentication & Security through EAP based approach - White P...
 Wi-Fi Offload Authentication & Security through EAP based approach - White P... Wi-Fi Offload Authentication & Security through EAP based approach - White P...
Wi-Fi Offload Authentication & Security through EAP based approach - White P...Green Packet
 
Unit 1-VoWiFi1.ppt
Unit 1-VoWiFi1.pptUnit 1-VoWiFi1.ppt
Unit 1-VoWiFi1.pptAbyThomas54
 
Unit 1-VoWiFi1.ppt
Unit 1-VoWiFi1.pptUnit 1-VoWiFi1.ppt
Unit 1-VoWiFi1.pptAbyThomas54
 
Wifi Challenges
Wifi ChallengesWifi Challenges
Wifi Challengesnone
 
ITU-T Study Group 12 Introduction
ITU-T Study Group 12 IntroductionITU-T Study Group 12 Introduction
ITU-T Study Group 12 IntroductionITU
 

Similar a VoWiFi testing challenges (20)

gopi-4h2.pptx
gopi-4h2.pptxgopi-4h2.pptx
gopi-4h2.pptx
 
Next Generation Service Platforms for Multimedia and Value Added Services
Next Generation Service Platforms for Multimedia and Value Added ServicesNext Generation Service Platforms for Multimedia and Value Added Services
Next Generation Service Platforms for Multimedia and Value Added Services
 
Voip on Wimax
Voip on WimaxVoip on Wimax
Voip on Wimax
 
Multi Vendor Wireless Channel Interference.docx
Multi Vendor Wireless Channel Interference.docxMulti Vendor Wireless Channel Interference.docx
Multi Vendor Wireless Channel Interference.docx
 
OIF Interop: The Key to Unlocking the Benefits of SDN
OIF Interop: The Key to Unlocking the Benefits of SDNOIF Interop: The Key to Unlocking the Benefits of SDN
OIF Interop: The Key to Unlocking the Benefits of SDN
 
Cellular Services over WiFi
Cellular Services over WiFiCellular Services over WiFi
Cellular Services over WiFi
 
Revolutionary Approach to Cell Site Activation
Revolutionary Approach to Cell Site ActivationRevolutionary Approach to Cell Site Activation
Revolutionary Approach to Cell Site Activation
 
VoLTE Testing Explained
VoLTE Testing ExplainedVoLTE Testing Explained
VoLTE Testing Explained
 
Questions for VoIP Installers.docx
Questions for VoIP Installers.docxQuestions for VoIP Installers.docx
Questions for VoIP Installers.docx
 
UK Spectrum Policy Forum – Sami Susiaho, BSkyB - Current and future spectrum ...
UK Spectrum Policy Forum – Sami Susiaho, BSkyB - Current and future spectrum ...UK Spectrum Policy Forum – Sami Susiaho, BSkyB - Current and future spectrum ...
UK Spectrum Policy Forum – Sami Susiaho, BSkyB - Current and future spectrum ...
 
THE FUTURE DIRECTIONS IN EVOLVING WI-FI: TECHNOLOGIES, APPLICATIONS AND SERVICES
THE FUTURE DIRECTIONS IN EVOLVING WI-FI: TECHNOLOGIES, APPLICATIONS AND SERVICESTHE FUTURE DIRECTIONS IN EVOLVING WI-FI: TECHNOLOGIES, APPLICATIONS AND SERVICES
THE FUTURE DIRECTIONS IN EVOLVING WI-FI: TECHNOLOGIES, APPLICATIONS AND SERVICES
 
WiMAX_CPE_VoIP
WiMAX_CPE_VoIPWiMAX_CPE_VoIP
WiMAX_CPE_VoIP
 
2020 wifi Ready
2020 wifi Ready2020 wifi Ready
2020 wifi Ready
 
OneZone: Toronto metro-WiFi network overview
OneZone: Toronto metro-WiFi network overviewOneZone: Toronto metro-WiFi network overview
OneZone: Toronto metro-WiFi network overview
 
Wifi offload-through-eap-authentication
Wifi offload-through-eap-authenticationWifi offload-through-eap-authentication
Wifi offload-through-eap-authentication
 
Wi-Fi Offload Authentication & Security through EAP based approach - White P...
 Wi-Fi Offload Authentication & Security through EAP based approach - White P... Wi-Fi Offload Authentication & Security through EAP based approach - White P...
Wi-Fi Offload Authentication & Security through EAP based approach - White P...
 
Unit 1-VoWiFi1.ppt
Unit 1-VoWiFi1.pptUnit 1-VoWiFi1.ppt
Unit 1-VoWiFi1.ppt
 
Unit 1-VoWiFi1.ppt
Unit 1-VoWiFi1.pptUnit 1-VoWiFi1.ppt
Unit 1-VoWiFi1.ppt
 
Wifi Challenges
Wifi ChallengesWifi Challenges
Wifi Challenges
 
ITU-T Study Group 12 Introduction
ITU-T Study Group 12 IntroductionITU-T Study Group 12 Introduction
ITU-T Study Group 12 Introduction
 

Último

Satara Call girl escort *74796//13122* Call me punam call girls 24*7hour avai...
Satara Call girl escort *74796//13122* Call me punam call girls 24*7hour avai...Satara Call girl escort *74796//13122* Call me punam call girls 24*7hour avai...
Satara Call girl escort *74796//13122* Call me punam call girls 24*7hour avai...nishasame66
 
Mobile App Penetration Testing Bsides312
Mobile App Penetration Testing Bsides312Mobile App Penetration Testing Bsides312
Mobile App Penetration Testing Bsides312wphillips114
 
Mobile Application Development-Components and Layouts
Mobile Application Development-Components and LayoutsMobile Application Development-Components and Layouts
Mobile Application Development-Components and LayoutsChandrakantDivate1
 
Bromazolam CAS 71368-80-4 high quality opiates, Safe transportation, 99% pure
Bromazolam CAS 71368-80-4 high quality opiates, Safe transportation, 99% pureBromazolam CAS 71368-80-4 high quality opiates, Safe transportation, 99% pure
Bromazolam CAS 71368-80-4 high quality opiates, Safe transportation, 99% pureamy56318795
 
Android Application Components with Implementation & Examples
Android Application Components with Implementation & ExamplesAndroid Application Components with Implementation & Examples
Android Application Components with Implementation & ExamplesChandrakantDivate1
 
Mobile Application Development-Android and It’s Tools
Mobile Application Development-Android and It’s ToolsMobile Application Development-Android and It’s Tools
Mobile Application Development-Android and It’s ToolsChandrakantDivate1
 

Último (7)

Satara Call girl escort *74796//13122* Call me punam call girls 24*7hour avai...
Satara Call girl escort *74796//13122* Call me punam call girls 24*7hour avai...Satara Call girl escort *74796//13122* Call me punam call girls 24*7hour avai...
Satara Call girl escort *74796//13122* Call me punam call girls 24*7hour avai...
 
Mobile App Penetration Testing Bsides312
Mobile App Penetration Testing Bsides312Mobile App Penetration Testing Bsides312
Mobile App Penetration Testing Bsides312
 
Obat Penggugur Kandungan Di Apotik Kimia Farma (087776558899)
Obat Penggugur Kandungan Di Apotik Kimia Farma (087776558899)Obat Penggugur Kandungan Di Apotik Kimia Farma (087776558899)
Obat Penggugur Kandungan Di Apotik Kimia Farma (087776558899)
 
Mobile Application Development-Components and Layouts
Mobile Application Development-Components and LayoutsMobile Application Development-Components and Layouts
Mobile Application Development-Components and Layouts
 
Bromazolam CAS 71368-80-4 high quality opiates, Safe transportation, 99% pure
Bromazolam CAS 71368-80-4 high quality opiates, Safe transportation, 99% pureBromazolam CAS 71368-80-4 high quality opiates, Safe transportation, 99% pure
Bromazolam CAS 71368-80-4 high quality opiates, Safe transportation, 99% pure
 
Android Application Components with Implementation & Examples
Android Application Components with Implementation & ExamplesAndroid Application Components with Implementation & Examples
Android Application Components with Implementation & Examples
 
Mobile Application Development-Android and It’s Tools
Mobile Application Development-Android and It’s ToolsMobile Application Development-Android and It’s Tools
Mobile Application Development-Android and It’s Tools
 

VoWiFi testing challenges

  • 1. 1Spirent Communications PROPRIETARY AND CONFIDENTIAL VoWi-Fi Testing Challenges Measuring the Quality of Experience in the field and lab
  • 2. 2Spirent Communications PROPRIETARY AND CONFIDENTIAL Agenda Part 1: VoWi-Fi technology landscape  Drivers for Wi-Fi adoption  VoWi-Fi QoE Challenges Part 2: SRG VoWi-Fi field-based study results  Wi-Fi study background and motivation  Study setup and measurement methodology  VoWi-Fi Field test results Part 3: Evaluating VoWi-Fi performance in the lab  Advantages of lab-based testing  Evaluating Key Performance Indicators (KPIs) in the lab  Future deployment and test considerations  Audience Q&A
  • 3. 3Spirent Communications PROPRIETARY AND CONFIDENTIAL Drivers for Wi-Fi adoption Why Wi-Fi is an attractive choice as a technology
  • 4. 4Spirent Communications PROPRIETARY AND CONFIDENTIAL VoWi-Fi QoE Challenges Offloading Algorithms • When and why? • When to go back? Assessing Signal Strength and Integrity • RSSI measurements before and during offload IP Layer • Best effort delivery • Access Point variations Authorization • ePDG authentication for unsecured Wi-Fi access
  • 5. 5Spirent Communications PROPRIETARY AND CONFIDENTIAL KPIs used to measure VoWi-Fi Quality of Experience (QoE)  Voice Quality:  MOS Score – Uplink and Downlink  Packet (Audio) loss percentage  Conversational (Ear-to-mouth) delay  Call reliability:  Percentage of successful handovers in both directions under “expected” circumstances  Percentage of call drop under various ePDG adversarial scenarios (lab-based simulation)  Percentage of successful handovers when access point is loaded  Time taken for call setup (for Wi-Fi originated calls)  Time taken for LTE<-> Wi-Fi handover
  • 6. 6Spirent Communications PROPRIETARY AND CONFIDENTIAL Part 1: VoWi-Fi technology landscape  Drivers for Wi-Fi adoption  VoWi-Fi QoE Challenges Part 2: SRG VoWi-Fi field-based study results  Wi-Fi study background and motivation  Study setup and measurement methodology  VoWi-Fi Field test results Part 3: Evaluating VoWi-Fi performance in the lab  Advantages of lab-based testing  Evaluating Key Performance Indicators (KPIs) in the lab  Future deployment and test considerations  Audience Q&A
  • 7. 7Spirent Communications PROPRIETARY AND CONFIDENTIAL Background and rationale for the study  Spirent and SRG have collaborated on all-things-VoLTE for the last 15 months  VoWi-Fi is the rage  Although operators are rushing to embrace the technology – sometimes in advance of VoLTE – there isn’t a clear understanding how the technology performs  VoWi-Fi is a natural extension of VoLTE, given the interoperability that should exist between the two voice solutions – also doesn’t hurt that HD Voice is supported across all 3 RAN components of the operator’s network
  • 8. 8Spirent Communications PROPRIETARY AND CONFIDENTIAL Study setup and test methodology  Spirent Communications provided Nomad User Experience system  Accuver X-Cal M data collection tool and Accuver XCAP post-processing software for logging all lower layer parameters  The testing relied on mobile-to-mobile calls – implies that the uplink MOS of one device can affect the downlink MOS of the other  Testing included VoLTE-VoWi-Fi handovers and vice versa, as well as call quality with different device configurations (3G only, Wi-Fi- only, etc.)  VoWi-Fi voice quality measured under realistic traffic loading at residential and public Wi-Fi access points
  • 9. 9Spirent Communications PROPRIETARY AND CONFIDENTIAL Procedures:  Introduced additional background traffic onto same Wi-Fi access point while performing voice quality test Observations:  Streaming HD Video (2x) had very little impact on voice quality when the downlink consisted of two video streams Field test results: residential Wi-Fi access point How does the performance look when a device using VoWi-Fi calling on a residential Wi-Fi access point is loaded on the downlink
  • 10. 10Spirent Communications PROPRIETARY AND CONFIDENTIAL Field test results: residential Wi-Fi access point How does the performance look when a device using VoWi-Fi calling on a residential Wi-Fi access point is loaded on the uplink Procedures:  Transferred emails with ~10 MB files attached over same Wi-Fi AP being use to support voice calls Observations:  Relatively modest amounts of uplink traffic can impact call quality, including no voice detection  As implemented, VoWi-Fi does not support QoS so all traffic is treated as best effort
  • 11. 11Spirent Communications PROPRIETARY AND CONFIDENTIAL Field test results: public Wi-Fi access point How does the performance look when two smartphones are using VoWi-Fi in a public Wi-Fi access point Procedures:  Stationary testing at airport with both smartphones using VoWi-Fi and the same SSID – network seemed to be lightly loaded Observations:  Bigger variances in the results and lower scores, despite lightly-loaded network – VoWi-Fi on both ends increases impact  Quality of network has a major influence on call quality
  • 12. 12Spirent Communications PROPRIETARY AND CONFIDENTIAL Field test results: public Wi-Fi access point How does the performance look when a device using VoWiFi calls another device on VoWiFi on a public Wi-Fi Access point Procedures:  Comparable to airport test but done at San Diego Convention Center – 18,000 attendees for a conference Observations:  Less variance in the results compared with the airport test  Quality of network has a major influence on call quality – in this case a good thing
  • 13. 13Spirent Communications PROPRIETARY AND CONFIDENTIAL Field test results: VoWi-Fi to VoLTE handovers How does the performance look when a device using VoWi-Fi hands over from a residential Wi-Fi Access Point to VoLTE Procedures:  Stationary device, which is placed near Wi-Fi AP, uses VoLTE; Second device was mobile – started on VoWi-Fi  Walked down the street pushing the “little red wagon” until the call switched to VoLTE  Returned back to Wi-Fi AP and waited for handover to Wi-Fi before walking again of Wi-Fi coverage Observations:  No impact to MOS Score during handover; also hard to detect handover by listening to the call  Handovers back into VoWi-Fi take longer (no surprise) while outbound handovers aren’t predictable
  • 14. 14Spirent Communications PROPRIETARY AND CONFIDENTIAL Field test results: VoWi-Fi to VoLTE handovers How does the network and device activity look when a device switches between VoLTE and VoWi-Fi Procedures:  One device configured for 3G voice only; second device configured for VoLTE / VoWi-Fi  Originate call within Wi-Fi AP coverage and then proceed to walk out of coverage with both phones in hand  Continue until handover occurs and then walk back into coverage; repeat as necessary Observations:  Everything consistent with expectations; lack of QoS with VoWi-Fi is evident
  • 15. 15Spirent Communications PROPRIETARY AND CONFIDENTIAL Field test results: VoWi-Fi to VoLTE handovers, II How does the network and device activity look when a device switches between VoLTE and VoWi-Fi Procedures:  One device (S5) configured for 3G voice only; second device (S6) configured for VoLTE / VoWi-Fi  Originate call within Wi-Fi AP coverage and then proceed to walk out of coverage with both phones in hand  Continue until handover occurs and then walk back into coverage; repeat as necessary Observations:  S5 used UMTS as intended but S6 didn’t use VoWi-Fi, instead using a combination of UMTS and 2G (NB-AMR)  Call drops when we returned to Wi-Fi coverage (repeated with same results)
  • 16. 16Spirent Communications PROPRIETARY AND CONFIDENTIAL Key observations  The benefits of VoWi-Fi are clear but it isn’t a panacea  VoWi-Fi is as good or as bad as the underlying Wi-Fi network  VoWi-Fi handovers with the LTE network generally worked but there were some exceptions  Problems, which we attribute to network/IMS registration and device interoperability, exist
  • 17. 17Spirent Communications PROPRIETARY AND CONFIDENTIAL Part 1: VoWi-Fi technology landscape  Drivers for Wi-Fi adoption  VoWi-Fi QoE Challenges Part 2: SRG VoWi-Fi field-based study results  Wi-Fi study background and motivation  Study setup and measurement methodology  VoWi-Fi Field test results Part 3: Evaluating VoWi-Fi performance in the lab  Advantages of lab-based testing  Evaluating Key Performance Indicators (KPIs) in the lab  Future deployment and test considerations  Audience Q&A
  • 18. 18Spirent Communications PROPRIETARY AND CONFIDENTIAL Advantages of evaluating VoWi-Fi in the lab  Audio quality issues related to low MOS scores, audio (packet) loss and codec (AMR-NB/AMR-WB/EVS) negotiation failure  SIP registration issues  ePDG authentication failure  Handover at power levels higher/lower than that mandated by operator test plans  Poor QoE resulting from UE toggling between LTE – Wi-Fi networks  Does not require a wagon
  • 19. 19Spirent Communications PROPRIETARY AND CONFIDENTIAL Advantages of evaluating VoWi-Fi in the lab VoLTE VoWi-FieAP AKA IPSec tunnel VoWi-Fi VoWi-Fi
  • 20. 20Spirent Communications PROPRIETARY AND CONFIDENTIAL Evaluating VoWi-FI KPIs in the lab Test scenario Measurable KPIs QoE impact Voicequality Test voice quality when the Wi-Fi power is in the edge condition of making a decision on handover from LTE-Wi-Fi or vice-versa and is toggling between the two  MOS Score  Packet loss percentage  Network latency  Jitter  Lower MOS Score => lower voice quality  High packet loss => Intermittent drops in speech  High network latency => Variable delays in speech Test voice quality in the presence of loading on the Wi-Fi access point and tolerance of UE to larger network delays Test voice quality of two test devices against a reference “soft” client, both on the Uplink and the Downlink
  • 21. 21Spirent Communications PROPRIETARY AND CONFIDENTIAL Evaluating VoWi-Fi KPIs in the lab Test scenario Measurable KPIs QoE impact Callreliability Test the reliability of the call through the handover process and time for handover by control of Wi-Fi power over multiple iterations  Percentage of successful call completion  Handover time from LTE – Wi-Fi  Larger handover time => periods of voice discontinuity  Lower percentage of call completion => Increased dropped calls during handover  Improper response to ePDG adversarial scenarios => call drops during handover Test the reliability of the call when the UE is sent adversarial responses by ePDG Test the reliability of the call through the handover process and time for handover by control of Wi-Fi power over multiple iterations
  • 22. 22Spirent Communications PROPRIETARY AND CONFIDENTIAL Test case 1 Test voice quality between two devices Device A and Device B and measure MOS score on both Uplink and Downlink Result Low MOS score measured on Uplink of Device A Test Case 1: Voice Quality Low MOS score on Downlink of Device B in device-to-device call traced to low Uplink MOS score on Device A
  • 23. 23Spirent Communications PROPRIETARY AND CONFIDENTIAL Test case 2 ePDG adversarial testing scenario Result LTE-Wi-Fi Handover failure UE fails to handover and drops call when ePDG issues error code Test Case 2: ePDG responses
  • 24. 24Spirent Communications PROPRIETARY AND CONFIDENTIAL Test case 3 Test voice quality when the UE is toggling between Wi-Fi and LTE network Result Lower MOS Score Test Case 3: Handover Packet loss Root cause identified: Packet loss as a result of device “toggling” between LTE-Wi-Fi
  • 25. 25Spirent Communications PROPRIETARY AND CONFIDENTIAL Test case 4 Call initiation on Wi-Fi originated call Result Low MOS score Test Case 4: Codec negotiation UE1 supporting AMR-WB UE2 supporting only AMR-NB Root cause identified: Disagreement of codec between UE1 and UE2
  • 26. 26Spirent Communications PROPRIETARY AND CONFIDENTIAL VoWi-Fi test capabilities and requirements
  • 27. 27Spirent Communications PROPRIETARY AND CONFIDENTIAL Bringing field study into the lab SRG and Spirent partner on evaluating VoWi-Fi QoE in the lab  LTE EPC Configuration  IMS registration call flow  Wi-Fi authentication  ePDG Adversarial test condition  IMS test agent simulation
  • 28. 28Spirent Communications PROPRIETARY AND CONFIDENTIAL Additional VoWi-Fi QoE considerations  Service request type  Wi-Fi / LTE power level  Capacity/loading on Wi-Fi radio link  Measuring impact on voice quality while introducing fading and noise on radio interface of Wi-Fi  Testing audio quality and call performance with different classes of access points (Home, Enterprise, public Wi-Fi hotspots)  Testing performance in the presence of introducing IP impairments on the IP backhaul  Testing the impact of performance in the presence of interference from other adjacent bands such as LTE and/or adjacent Wi-Fi hotspots  ANDSF, Hotspot 2.0…
  • 29. 29Spirent Communications PROPRIETARY AND CONFIDENTIAL Part 1: VoWi-Fi technology landscape  Drivers for Wi-Fi adoption  VoWi-Fi QoE Challenges Part 2: SRG VoWi-Fi field-based study results  Wi-Fi study background and motivation  Study setup and measurement methodology  VoWi-Fi Field test results Part 3: Evaluating VoWi-Fi performance in the lab  Advantages of lab-based testing  Evaluating Key Performance Indicators (KPIs) in the lab  Future deployment and test considerations  Audience Q&A
  • 30. 30Spirent Communications PROPRIETARY AND CONFIDENTIAL Questions?