Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

WebRTC: Mostly Video Bits

Quality of Service is a difficult problem in WebRTC. You spend hours perfecting
your design in the lab. You test in different network conditions and try your best.
A week later the reports trickle in of different customers having unreproducable issues.
Then when you go to research you can't even find consensus on how to solve these problems.
Most companies don’t discuss how they do it, and those that do don’t agree.

This talk is about my time building media experiences with WebRTC. The lessons I
have learned, and mistakes along the way. This talk doesn’t discuss signaling, connectivity
or security. If you are interested in those topics check out WebRTC for the Curious.

  • Sé el primero en comentar

WebRTC: Mostly Video Bits

  1. 1. WebRTC: Mostly the video bits
  2. 2. MY BACKGROUND
  3. 3. 1 Pion Collection of Go RTC libraries - Building blocks to build your vision - Designed to be flexible - Community Owned/Non-Commercial PeerConnection API in Pure Go - CreateOffer/AddTrack/…. - User controls media pipeline - SettingEngine - Interceptors
  4. 4. 2 Pion TURN as an API in Pure Go - Callbacks for authentication - Bring your own logger - Run TURN and HTTPS on same port - Embed TURN in application Ion - Cluster based system for building RTC - SFU Instance - AVP Instance (Processing real-time media) - LIVE Instance (SIP/RTMP/HLS Endpoints)
  5. 5. 3 WebRTC for the Curious Book on how WebRTC really works - Not just about the public APIs! - Deep dive on protocols History of WebRTC - Interviews with RFC authors WebRTC in practice - Debugging every piece - Teach the sharp edges
  6. 6. REAL WORLD CONSTRAINTS
  7. 7. 4 Everyone notices quality of service Good UI, Easy signup everything looks good! Then people get frustrated - Video freezes or is corrupted - Audio drops out - Audio/Video becomes desynchronized - Quality fluctuates rapidly - Disconnected…. RTC is the eternal struggle of quality vs latency
  8. 8. 5 The real world is cruel Video is huge - Uncompressed 1280x720 8-Bit needs ~80 MB/s Codecs - Compressed video needs careful transport - Support differs everywhere Network - Congestion - Errors
  9. 9. 7 Video Compression Compression can be grouped into two categories - Intraframe (Compression in single frame) - Interframe (Compression across many frames) Interframe types - Intra-code Frame (Keyframe) - Predicted Frame - Bi-Directional Predicted Frame Lots of knobs here - Disable bframes - Keyframe interval
  10. 10. 8
  11. 11. 9 Real World Networks Congestion - Can manifest as loss or latency - Every network hop can have different behaviors Jitter Networks are dynamic Errors can happen at every step - Wifi/Cellular - Gateways
  12. 12. HOW DOES RTC SOLVE IT
  13. 13. 10 WebRTC Media Transport Over UDP RTP (Media Transport) - SSRC (Session Identifier) - Sequence Number (Unique ID, rolls over) - Timestamp (Relative timestamp, random start) - Extensions RTCP (Control Messages) - Flexible, but a common subset is agreed upon
  14. 14. 11 Fixing Congestion Determine available bandwidth and adjust - Bitrate - Resolution - Framerate Common Congestion Controller Algorithms - Google Congestion Controller - NADA - SCreAM All this is being actively discussed in RMCAT[0] [0] https://datatracker.ietf.org/wg/rmcat/documents/
  15. 15. 12 Congestion Controller Flow
  16. 16. 13 Fixing Jitter If we play as they arrive we will have stuttering JitterBuffer - Trade-off between latency and quality - This concept will be re-used in the future
  17. 17. 14 Fixing Audio/Video Sync Buffering/Network messes up your timing Sender Reports to the rescue! - Map each SSRC Relative Timestamp to NTP - Then you can join each Relative Timestamp - Also contains other useful data….
  18. 18. 15 Fixing Loss Some loss is unavoidable, even without congestion - Wifi - Cellular Not all loss is equal - Do you care about a P-Frame before a I-Frame?
  19. 19. 16 Fixing Loss By Reacting RTCP Negative Acknowledgments (NACK) - SequenceNumber `I` of SSRC `J` was lost - Recover small part of video frame RTCP Picture Loss Indication (PLI) - SSRC `J` is broken - In unrecoverable state, request new keyframe Re-use that JitterBuffer and add a delay to wait
  20. 20. 17 Fixing Loss preemptively Forward Error Correction (FEC) - Redundant Data is sent in another stream - Is dynamic/protects the important bits Less RTT then NACK, but wasting bandwidth Still in flux, not available everywhere
  21. 21. 18 Fixing Loss After Decode More common with audio - Play the frame before the missing one - Use Model-Based methods to extrapolate Never done this myself!
  22. 22. 19 WebRTC Servers Simulcast - Send multiple video feeds (high, medium, low) - Server then distributes the feed to each viewer Scalable Video Coding - Layered coding - Not as common/Browser Support not great
  23. 23. 20 My Time with RTP It's possible to ship without all this - Don't let perfect be enemy of the good - Do a high keyframe interval and ship Always something you can improve Moving any knob can effect the others - Congestion equals more NACKs, - More NACKs equals more congestion.... Lab/Demos are fun, but not very useful - Network simulators are deterministic - User metrics are what really matter
  24. 24. OPEN SOURCE SHOWCASE
  25. 25. ns-remote 21
  26. 26. kerberos.io 22
  27. 27. CloudRetro.io 23
  28. 28. 24 Neko github.com/nurdism/neko
  29. 29. 25 TelloGo
  30. 30. 26 ascii
  31. 31. 27 cloud-morph
  32. 32. 28 ssh-p2p
  33. 33. 29 Snowflake
  34. 34. webwormhole.io 30
  35. 35. s4y/space 31
  36. 36. Project Lightspeed 32
  37. 37. 33 github.com/pion pion.ly/slack twitter.com/_pion sean@pion.ly

    Sé el primero en comentar

  • charlenopires

    Feb. 10, 2021

Quality of Service is a difficult problem in WebRTC. You spend hours perfecting your design in the lab. You test in different network conditions and try your best. A week later the reports trickle in of different customers having unreproducable issues. Then when you go to research you can't even find consensus on how to solve these problems. Most companies don’t discuss how they do it, and those that do don’t agree. This talk is about my time building media experiences with WebRTC. The lessons I have learned, and mistakes along the way. This talk doesn’t discuss signaling, connectivity or security. If you are interested in those topics check out WebRTC for the Curious.

Vistas

Total de vistas

209

En Slideshare

0

De embebidos

0

Número de embebidos

0

Acciones

Descargas

2

Compartidos

0

Comentarios

0

Me gusta

1

×