2016 February - WebRTC Conference japan - English

WebRTC Evolution
Dr. Alex Gouaillard
© Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-NC 4.0

Overview
• Early 2015 WebRTC
• Object Models APIs
• Glue (Transceiver)
• Faster ICE
• Ports optimizations
• Simulcast
• Others in webrtc 1.0

Early 2015, peer-to-peer webRTC Model
http://www.html5rocks.com/en/tutorials/webrtc/basics/

Signaling
server
PeerConnection
JavaScript APP(Sender) JavaScript APP (Receiver)
PeerConnection
Media
Stream
Data
Channel
CreateOffer
SetLocal
SetRemote
SetLocal
SetRemote
Offer (SDP) Answer (SDP)
Candidates
(SDP)
Media
Stream
Data
Channel
OnCandidate
AddCandidate
CreateAnswer
ICE
Bindings + tests

Signaling
server
OmniPotent PeerConnection
PeerConnection
JavaScript APP (Sender) JavaScript APP (Receiver)
PeerConnectionMedia
Stream
Data
Channel
ICE UADTLS
RTP
Code
c
CreateOffer
Gather
Transport
SetLocal
SetRemote
ICE UA DTLS
RTP
Code
c
Gather
Transport
SetLocal
SetRemote
Offer (SDP) Answer (SDP)
Candidates
(SDP)
Media
Stream
Data
Channel
OnCandidate
AddCandidate
CreateAnswer
ICE
Bindings + tests

DONEFROZENINCALLCONNECTIONHANDSHAKE(SDPO/A).
new
checking
connected
disconnected
failed
Completed
close
new
gathering
complete
CALLER SIG-SERVER CALLEE
stable
have-local-offer
stable
Close
OFFER
ANSWER
Create PC
Add local stream(s)
Create offer
<modify sdp>
SetLocal(offer)
Sending offer Create PC
SetRemote(offer)
addRemoteStream(s)
Add local stream(s)
Create answer
<modify sdp>
SetLocal(answer)
Send answer
stable
Have-remote-
offer
stable
new
gathering
complete
onIceCandidate
<Filter
candidates>
Send candidate
<Filter
candidates>
addIceCandidate
onIceCandidate
<Filter candidates>
Send candidate
<Filter candidates>
addIceCandidate
new
checking
connected
disconnected
failed
Completed
close Close
© Temasys Communications, pvt, ltd, 2014
Document provided under CC BY-NC 4.0
CANDIDATES
Peer
Connection
ICE
Connection
ICE
Gathering
SetRemote(answer)
addRemoteStream
BYE

Open Questions
1. How do I handle bandwidth preferences?
2. How do I check fingerprint (MITM attacks prevention)?
3. How do I handle codecs and their parameters?
=> Hum, ……. Just modify the SDP and pray it works.
4. How to know which pair of candidate is being used?
5. How do I restrict the ICE gathering (not to expose my IPs)
6. How do I get more info about ICE failures?
7. How do I make the (ICE) handshake faster?
8. How do I get rid of JSEP, SDP, O/A, …..?
9. Can I switch front / self view without renegotiation?
=> No can’t do.

Q2 2015, New W3C WebRTC charter
• => ORTC – WebRTC merge
• Notion of Object Model API (ORTC inspired)
http://ortc.org/wp-content/uploads/2014/08/ortc.html

Object Models?
• Each Object Does only one thing!
• Each Object can be probed for capabilities (JS)
• Parameters can be get/set dynamically (JS)
#8 (Edge)
http://ortc.org/wp-content/uploads/2014/08/ortc.html

Late 2015, Object Model API
Open Questions
Have a final API.
• How do I handle bandwidth preferences
• How do I check fingerprint (MITM attacks prevention)
• How to know which pair of candidate is being used?
• How do I get more info about ICE failures?
• How do I get rid of JSEP, SDP, O/A, …..?
• Can I switch front / self view without renegotiation?
Have some API, can do better or more.
• How do I handle codecs and their parameters
• How do I restrict the ICE gathering (not to expose my IPs)
Not addressed yet
• How do I make the (ICE) handshake faster?

WebRTC 1.0 = Object Model + JSEP
PLACEHOLDER FOR DRAWING

WebRTC 1.0 = Object Model + JSEP
Problems:
1. m-lines in SDP are intrinsically bi-directional
2. m-lines in SDP have attributes that fit neither
sender nor receiver object model (mid), or
that would be duplicated.
How to model the equivalent to a m-line on top
of the Object Model (so we can throw it later if
we decide to get rid of SDP, JSEP, ….) ?

Rtp
Sender
Rtp
Receiver
Dtls
Transport
Ice
Transport
Local
Track
NET
Ice
Transport
Dtls
Transport
Rtp
Sender
WebRTC / Object Model
Example 1: Audio only, 1:1
Dtls
Transport
Ice
Transport
Rtp
Receiver
Remote
Track
Dtls
Transport
Ice
Transport
Rtp
Transceiver
Remote
Track
Local
Track
Rtp
Transceiver
© Dr. Alex Gouaillard @ Cosmo Consulting., 2015. Document provided under CC BY-NC 4.0

What’s your flavor?
Full, Trickle, Lite
Feature flavor Full Trickle Lite
Make connectivity checks Yes Yes NO
Answer connectivity checks Yes Yes Yes
Controlling or controled Both Both Controlled
Candidates send in the Offer/Answer Yes No ?
Flag Ice-option:
trickle
a=ice-lite
https://tools.ietf.org/html/draft-ietf-mmusic-trickle-ice-02
https://webrtchacks.com/trickle-ice/
Trickle: From 10s to 1S
Q: How to get 100ms?

Faster ICE => more async, less waiting
- Start even before the call!
- ICE pool
- Pre-gathering
- Warmup:
- ICE, DTLS, media ready before signaling is.
- Early Media:
- Start sending media with answer
- Simplify & fasten the pairing / checking
- Default TURN server

RTP/RTCP within 1 PeerConnection
Use case: 2 media tracks (audio and video), how
many ports do I need? Less ports would useful
for
- Better connection rate esp. over mobile net.
- Optimize turn/media server resources
- Client side ports constraints

RTP/RTCP within a PeerConnection
Default
4
A
V
RTP RTCP
http://tools.ietf.org/html/draft-ietf-mmusic-sdp-bundle-negotiation
http://datatracker.ietf.org/doc/rfc5761/

RTP/RTCP
Default
4
Bundle
2
A
V
RTP RTCP
A
V
RTP RTCP
a=group:BUNDLE

RTP/RTCP
Default
4
Bundle
2
Rtcp-mux
2
A
V
RTP RTCP
A
V
RTP RTCP
A
V
RTP RTCP
a=rtcp-muxa=group:BUNDLE

RTP/RTCP
Default
4
Bundle
2
Rtcp-mux
2
Both
1
A
V
RTP RTCP
A
V
RTP RTCP
A
V
RTP RTCP
A
V
RTP RTCP

RTP/RTCP – extrapolation to n tracks
Default
2n
Bundle
n
Rtcp-mux
n
Both
1
A
V
RTP RTCP
A
V
RTP RTCP
A
V
RTP RTCP
A
V
RTP RTCP

Rtp
Sender
Rtp
Receiver
Dtls
Transport
Ice
Transport
Local
Track
NET
Ice
Transport
Dtls
Transport
Rtp
Sender
Dtls
Transport
Ice
Transport
Rtp
Receiver
Remote
Track
Dtls
Transport
Ice
Transport
Rtp
Transceiver
Remote
Track
Local
Track
Rtp
Transceiver
RTP
RTCP
Default
2n

Rtp
Sender
Rtp
Receiver
Local
Track
NET
Remote
Track
Rtp
Sender
Local
Track
rtcp-mux
Dtls
Transport
Ice
Transport
Rtp
Receiver
Remote
Track
Dtls
Transport
Ice
Transport
Rtp
Transceiver
Rtp
Transceiver
Rtcp-mux
n

Example 2: Audio + Video, 1:1
rtcp-mux
Rtp
Sender
Rtp
Receiver
Local
Track
NET
Remote
Track
Rtp
Sender
Local
Track
Dtls
Transport
Ice
Transport
Rtp
Receiver
Remote
Track
Dtls
Transport
Ice
Transport
Rtp
Transceiver
Rtp
Transceiver
Rtp
Sender
Rtp
Receiver
Local
Track
NET
Remote
Track
Rtp
Sender
Local
Track
Dtls
Transport
Ice
Transport
Rtp
Receiver
Remote
Track
Dtls
Transport
Ice
Transport
Rtp
Transceiver
Rtp
Transceiver
AUDIO
VIDEO
Rtcp-mux
n

RtpTransceiver
RtpTransceiver
Rtp
Sender
Rtp
Sender
Rtp
Sender
Rtp
ReceiverTrack
Rtp
Sender
Rtp
Receiver
Dtls
Transport
Ice
Transport
Local
Track
NET Ice
Transport
Dtls
Transport
Track
Remote
Track
Rtp
ReceiverTrack
Remote
Track
Rtp
Sender Track
Local
Track
Example 2: Audio + Video, 1:1
rtcp-mux + BUNDLE
RtpTransceiver
RtpTransceiver
Both
1

Data
Channel
Data
Channel
Generic & Default !
RtpTransceiver
RtpTransceiver
Rtp
Sender
Rtp
Sender
Rtp
Sender
Rtp
ReceiverTrack
Rtp
Sender
Rtp
Receiver
Dtls
Transport
Ice
Transport
Local
Track
NET Ice
Transport
Dtls
Transport
Track
Remote
Track
Rtp
ReceiverTrack
Remote
Track
Rtp
Sender Track
Local
Track
RtpTransceiver
RtpTransceiver
Sctp
Transport
Data
Channel
Sctp
Transport
Data
Channel
Both
1

Overview
• Glue (Transceiver)
• ICE and Optimizations
• Simulcast

Reminder
Multiparty vs Simulcast vs SVC
A
B
C
Multiparty
• Several tracks
• Decodable separately
• Bandwidth management separated
HD
SD
QVGA
Simulcast
• Several tracks
• Coming from the same source
• Smart bandwidth management possible
1SSRC
Δ2
Δ1
QVGA
SVC Encoding
• Several tracks
• Not Decodable separately (Except base layer)
• Smart bandwidth management mandatory
• Less bandwidth, more resilience.
1SSRC
E
E
E
E
E
E
SVCEnc.
D
D
D
HD
SD
QVGA

Reminder
State of Multiparty signaling
From the beginning of webrtc, one have been able to
send multiple media streams in separate peer
connections, but it consumes much more ports, and
there is no sync between those tracks (lip sync, ..).
A
B
C
E
E
E
Multicast
• Several tracks

Reminder
State of Multiparty signaling
For multiple streams in a single peer connection!

Reminder
Multiparty vs Simulcast vs SVC
A
B
C
Multicast
• Several tracks
HD
SD
QVGA
Simulcast
• Several tracks
• Smart bandwidth management possible
1SSRC
Δ2
Δ1
QVGA
SVC Encoding
• Several tracks
• Not Decodable separately (Except base layer)
• Smart bandwidth management mandatory
• Less bandwidth, more resilience.
1SSRC
E
E
E
E
E
E
SVCEnc.
D
D
D
HD
SD
QVGA

Simulcast: Use case for webRTC 1.0
- Use Case for webrtc 1.0: SFU
- Browser send simulcast,
does not receive simulcast
(in webrtc 1.0)
Desk.
HD
SFU
SD
QVGA
Cr.
Book
Mobile
Desk.
HD
SD
QVGA

Simulcast (and SVC later):
3 layers, 3 working groups, 2 standard bodies
- Separate the
- “on-the-wire” info (1),
- JS API (2),
- and signaling (3)
Signaling
server
Web
server
Bob
(device)
Alice
(browser)
Media
(3)
(2)
(1)

Overview
• Glue (Tranceiver)
• ICE and Optimizations
• Simulcast

Screen Capture API simplification
- No more distinction between screen, windows
and apps. => no more constraints, flat list.
• Screen Capture
http://w3c.github.io/mediacapture-screen-share/

Record your streams (local), but not all streams (remote).
http://www.w3.org/TR/mediastream-recording/
Media Stream Recording (Client side)

Use <AUDIO> or <VIDEO> as a source !
http://w3c.github.io/mediacapture-fromelement/
Media Capture from DOM Elements
&& Image Capture (a.k.a. snapshot)

Send different streams to different outputs
http://w3c.github.io/mediacapture-output/
Audio output devices API

http://www.w3.org/TR/webrtc-stats/
Identifiers for WebRTC's Statistics API
Full stats about data channel, codecs, FEC, …

Transferable DataChannels
a.k.a. DataChannels in workers

SECURITY: IP Address Leakage
4 levels for candidate gathering
- All [with consent]
- Restricted I (host + 1918) [default]
- Restricted II (no host) [pref/extension]
- Only proxy/VPN [pref/extension]

Conclusion
WebRTC 1.0 is Awesome [Evil Grin]

2016 February - WebRTC Conference japan - English

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a 2016 February - WebRTC Conference japan - English

Similar a 2016 February - WebRTC Conference japan - English (20)

Más de Alexandre Gouaillard

Más de Alexandre Gouaillard (20)

Último

Último (20)

2016 February - WebRTC Conference japan - English

Notas del editor