ABR Algorithms Explained (from Streaming Media East 2016)
•Download as PPTX, PDF•
17 likes•9,692 views
Adaptive bitrate algorithms have become paramount in ensuring quality video delivery on every device and across varying network conditions. This presentation looks at the design goals and the inner workings of ABR logic, how it is used in the open-source players hls.js and dash.js, and what broadcasters can do to improve and optimize their own stack.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to ABR Algorithms Explained (from Streaming Media East 2016)
Similar to ABR Algorithms Explained (from Streaming Media East 2016) (20)
More from Erica Beavers
More from Erica Beavers (12)
Recently uploaded
Recently uploaded (20)
ABR Algorithms Explained (from Streaming Media East 2016)
- 1. ADAPTIVE BITRATE ALGORITHMS: HOW THEY WORK AND HOW TO OPTIMIZE YOUR STACK Streaming Media East – Track D Tuesday, May 10, 2016 1:45 to 2:30 pm CLIENT-ACCELERATED STREAMING
- 2. Streamroot: Who are we? PARTNERS INFINITE POSSIBILITIES, LIMITLESS DELIVERY Streamroot combines the best of a controlled, centralized network with the resilience and scalability of a widely distributed delivery architecture.
- 3. Presentation Outline I. Introduction: What are we trying to accomplish? Why does this matter? II. The Basics of how ABR algorithms work: constraints & parameters, process Example: hls.js III. Possible improvements to basic ABR algorithms: smoothing, quantizing, scheduling Example: dash.js IV. Going further Another Approach: buffer levels The key to improving: testing and iterating
- 4. I. Why ABR? Multiplicity of network conditions and devices need to dynamically select resolution HTTP / TCP stack removal from the transport protocol congestion logic client-level estimation & decisions Source: FESTIVE diagram of HTTP streaming
- 5. I. Design Goals 1. Maximize efficiency – stream at the highest bitrate possible 2. Minimize rebuffering – avoid underrun and playback stalls 3. Encourage stability – switch only when necessary (4. Promote fairness across network bottlenecks)
- 6. I. Why this Matters Views 24 min longer when buffer ratio is < 0.2% for live content View time drops 40% when > 0.4% buffer ratio mark Buffer ratio vs. play time Source: NPAW aggregated data for a set of European live broadcasters
- 7. II. The Basics: Constraints and Parameters CONSTRAINTS TRADEOFF PARAMETERS Screen size / Player size Buffer size CPU & Dropped frame threshold Bandwidth & possible bitrate Startup time / Rebuffering recovery (Bonus: P2P Bandwidth)
- 8. II. The Basics: Constraints 1. Screen & Player Size Bitrate should never be larger than the actual size of the video player 2. CPU & Dropped frame rate Downgrade when too many dropped frames per second 3. Startup time Always fetch the lowest quality first whenever the buffer is empty
- 9. II. The Basics: Tradeoff parameters 1. Maximize bitrate available bandwidth estimation Estimate the available bandwidth based on prior segment(s) Available bandwidth = size of chunk / time taken to download 2. Minimize rebuffering ratio buffer size Buffer ratio = buffering time / (buffering time + playback time) Abandon strategy Source: BOLA
- 10. Example: HLS.js HTML5 (MSE-based) media engine open-sourced by Dailymotion https://github.com/dailymotion/hls.js Very modular, so you can change the rules without even forking the media engine!
- 11. Example: HLS.js player size level capping https://github.com/dailymotion/hls.js/blob/master/src/controller/cap-level-controller.js#L68 Checks the max CapLevel corresponding to current player size Frequency: every 1000 ms
- 12. Example: HLS.js dropped frame rule https://github.com/dailymotion/hls.js/blob/master/src/controller/fps-controller.js#L33 Calculates the dropped frames per second ratio. If > 0.2, bans the level forever goes into restricted capping levels fpsDroppedMonitoringThreshold fpsDroppedMonitoringPeriod
- 13. Example: HLS.js startup strategy https://github.com/dailymotion/hls.js/blob/master/src/controller/stream-controller.js#L131 First segment is loaded from the first level in the playlist, then continues with normal ABR rule.
- 14. Example: HLS.js bandwidth-based ABR controller https://github.com/dailymotion/hls.js/blob/master/src/controller/abr-controller.js Simple algorithm, inspired by Android’s AVController’s ABR algo
- 15. Example: HLS.js P2P bandwidth estimation
- 16. Example: HLS.js bandwidth fragmentLoad abort rule https://github.com/dailymotion/hls.js/blob/master/src/controller/abr-controller.js#L51
- 17. STRONG POINTS COULD BE IMPROVED Very simple and understandable Add history parameter to BW estimation and adjustment Handles CPU & player size constraints Startup time constraint could be improved to get the lowest level first Conservative BW adjustment to avoid oscillation Sound emergency abort mechanism Example: HLS.js sum-up Simple algorithm with better performances in practice compared to native implementations.
- 18. 1. Tweak the parameters https://github.com/dailymotion/hls.js/blob/master/API.md#fine-tuning Dropped FPS: capLevelOnFPSDrop: false, fpsDroppedMonitoringPeriod: 5000, fpsDroppedMonitoringThreshold: 0.2 PlayerSize: capLevelToPlayerSize: false, 2. Write your own rules! AbrController: AbrController capLevelController: CapLevelController, fpsController: fpsController Example: HLS.js how to improve
- 19. III. Improvements: the pitfalls of bandwidth estimation • Not resilient to sudden network fluctuations • Often leads to bitrate oscillations • Biased by HTTP/TCP calls on the same device/network
- 20. III. Improvements: better bandwidth estimation A new 4-step approach: 1. Estimation 2. Smoothing 3. Quantizing 4. Scheduling Source: Block diagram for PANDA
- 21. III. Improvements: estimation & smoothing Estimation: take history into account! Smoothing: Apply a smoothing function to the range of values obtained. Possible functions: average, median, EMWA, harmonic mean How many segments? 3? 10? 20?
- 22. III. Improvements: quantizing Quantizing: quantize the smoothed bandwidth to a discrete bitrate Additive increase multiplicative decrease conservative when switching up, more aggressive when down. Source: FESTIVE
- 23. III. Improvements: scheduling (bonus) Continuous & periodic download scheduling oscillation, over- or underused resources Randomize target buffer level to avoid startup bias and increase stability. Also extremely useful for promoting fairness! Source: FESTIVE
- 24. Example 2: DASH.JS Dash.js is the reference DASH player developed by DASH-IF. https://github.com/Dash-Industry-Forum/dash.js/wiki 4 different rules: 2 Main: ThroughputRule AbandonRequestsRule 2 secondary: BufferOccupancyRule InsufficientBufferRule
- 25. Example 2: DASH.JS main rules Source: DASH-IF, Maxdome
- 26. Example 2: DASH.JS, sum-up STRONG POINTS COULD BE IMPROVED Smoothes bandwidth No quantization of bitrates Segment abort mechanism to avoid buffering during network drops Doesn’t handle CPU & Player size constraints Rich buffer threshold to avoid BW oscillations
- 27. Example 2: DASH.JS how to improve 1. Tweak the Parameters ThroughputRule: AVERAGE_THROUGHPUT_SAMPLE_AMOUNT_LIVE = 2; AVERAGE_THROUGHPUT_SAMPLE_AMOUNT_VOD = 3; AbandonRequestRule: GRACE_TIME_THRESHOLD = 500; ABANDON_MULTIPLIER = 1.5; 2. Write your own rules https://github.com/Dash-Industry-Forum/dash.js/wiki/Migration-2.0#extending-dashjs https://github.com/Dash-Industry-Forum/dash.js/blob/development/src/streaming/rules/abr/ABRRulesCollection.js BufferOccupancyRule: RICH_BUFFER_THRESHOLD = 20
- 28. Buffer size based ONLY no more bandwidth estimations Uses utility theory to make decisions: configurable tradeoff between rebuffering potential & bitrate maximization: Maximize Vn + y Sn Where: Vn is the bitrate utility Sn is the playback Smoothness y is the tradeoff weight parameter IV. Going further: DASH.js BOLA, another approach
- 29. IV. Going further: test and iterate! Tweaking algorithms is easy, creating your forks too. You’ve got the power! - Know what is important to you (buffering, max bitrate, bandwidth savings…) - Compare and cross with QoS analytics to understand your audiences - Test and iterate: AB testing allows you to compare changes in real-time Significant improvements without even changing your workflow!
- 30. QUESTIONS?
- 31. Further Reading / Contact Us Probe and Adapt: Rate Adaptation for HTTP Video Streaming At Scale. Zhi Li, Xiaoqing Zhu, Josh Gahm, Rong Pan, Hao Hu, Ali C. Begen, Dave Oran, Cisco Systems, 7 Jul 2013. Improving Fairness, Efficiency, and Stability in HTTP-based Adaptive Video Streaming with FESTIVE, Junchen Jiang, Carnegie Mellon University, Vyas Sekar, Stony Brook University, Hui Zhang, Carnegie Mellon, University/Conviva Inc. 2012. ELASTIC: a Client-side Controller for Dynamic Adaptive Streaming over HTTP (DASH). Luca De Cicco, Member, IEEE, Vito Caldaralo, Vittorio Palmisano, and Saverio Mascolo, Senior Member, IEEE. BOLA: Near-Optimal Bitrate Adaptation for Online Videos. Kevin Spiteri, Rahul Urgaonkar , Ramesh K. Sitaraman, University of Massachusetts Amherst, Amazon Inc., Akamai Technologies Inc. Contact us at: Nikolay Rodionov, Co-Founder and CPO, nikolay@streamroot.io Erica Beavers, Head of Partnerships, erica@streamroot.io
Editor's Notes
- Explain what HLS.js is. Also say it’s quite simple to extend, as the different controllers are actually option parameters, and so can be easily replaced.
- Checks the max CapLevel corresponding to current player size Every 1000ms. You can also add up manual level caps on initialization. If the cap level is bigger that the last one (which means the player size has grown, like in Fullscreen for exemple), then you flush the current buffer and ask for a new quality right away (force the buffer)
- Calculates the dropped frames per second ratio. If it is > 0.2, bans the level for ever => goes into restricated levels Not activated in production! fpsDroppedMonitoringThreshold fpsDroppedMonitoringPeriod
- First segment always from the lowest quality, then it continues with normal rule (very simple simple rule in practice!) Another optimization is just to load this level (and playlist), and don’t wait for the other levels to have been loaded
- Simple algorithm,
- Here talk about Streamroot, and the fact having the sources from different buffers is even more difficult! Code from us?x Basically a onProgress & bandwidth estimation too (coming from CDN & P2P network!) Request.onProgress Request.onLoad => classic estimation With P2P estimation! Don’t wanna infinite speed, and thus includes a P2P bandwidth metric. Not the same for different peers, so averaged and smoothed Code from us?x Basically a onProgress & bandwidth estimation too (coming from CDN & P2P network!) Shema => a P2P cache and a CDN buffer => and time = 0
- One of the most important ones here What happens if you started a request and then BW drops ? Especially important when you ahve long fragments, this can very easily lead to a buffer underrun! After Half of the needed time, compare the estimate time of arrival to time of buffer underrun. And then see if there is another level that could solve the issue?
- Pros: Simple implementation, taking into account a lot of different params Works as good as the other implementation at Dailymotion! (alshls, android, iPhone… etc) Cons: Still Naive bandwidth estimation => possible overestimation, and possible oscillation around bitrates? We can do a lot of improvements on bandwidth estimation! difficult to correlate a unique segment download time to the real device’s available bandwidth, for several reasons: You can have very quick bandwidth changes, especially on a mobile network, as well as unexpected bandwidth drops The requests can be living in parallel with other TCP request (HTTP or any other on the user’s device) This can lead to frequent estimation oscillations!
- The different static constants more for you use-case? You can play with them You can also easily build your own rule! Here is an example on Github? First explain how to do that?
- difficult to correlate a unique segment download time to the real device’s available bandwidth, for several reasons: You can have very quick bandwidth changes, especially on a mobile network, as well as unexpected bandwidth drops The requests can be living in parallel with other TCP request (HTTP or any other on the user’s device) This can lead to frequent estimation oscillations!
- difficult to correlate a unique segment download time to the real device’s available bandwidth, for several reasons: You can have very quick bandwidth changes, especially on a mobile network, as well as unexpected bandwidth drops The requests can be living in parallel with other TCP request (HTTP or any other on the user’s device) This can lead to frequent estimation oscillations!
- Good to minimize the oscillations! Can have a different switch when UP or DOWN: Conservative when UP, less conservative when DOWN You can also scale taking into account the bitrate (and it’s utility)
- DASH.Js has 4 different Rules ThroughputRule calculates bandwidth with some smoothing! No real quantizing (have a real estimate and no other values) AbandonRequestsRule cancels if takes more than 1.5x of donwload BufferOccupancyRule to now go down if buffer large enough (RICH BUFFER TRESHOLD) InsufficientBufferRule au tas
- You can easily take the best out of hls.js here! Write a player size rule, a FPS drop rule… change the Abandonrate rule! It’s all very easy to do!
- BOLA stuff ? The approach is quite difficult to explain… based on utility theory, and supposed to be a lot more efficient because there are no need to estimate the bandiwdth. BUT Not fully implemented in dash.js, and there are some optimisation constants that depend a lot on the use-case (target buffer, live, vod…) Today not working great for small segment sizes AND small buffer size ( but good for 1+ min apparently?) Still work in progress, but an interesting approach!
- We can give a lot of tips, but most of the use-cases are spcific (segment size, playlist size, latency… and also which parameter is most important to you (buffer rate? Best bitrate ? Best bitrate no so useful if you KNOW that most of your user have a better bandwidth anyway? Number of switches) So what’s important is to have a way to iterate and improve ? The best is to have AB testing on 50/50 of population, to be able to quickly see results and compare them! What happens if you just tweak one parameter ? The results can be quite stunning!
- We can give a lot of tips, but most of the use-cases are spcific (segment size, playlist size, latency… and also which parameter is most important to you (buffer rate? Best bitrate ? Best bitrate no so useful if you KNOW that most of your user have a better bandwidth anyway? Number of switches) So what’s important is to have a way to iterate and improve ? The best is to have AB testing on 50/50 of population, to be able to quickly see results and compare them! What happens if you just tweak one parameter ? The results can be quite stunning!