A live Q&A session about WebRTC in general and deep dive into WebRTC Screen Sharing and Screen Capture. Session by Alex Gouailard, Dan Burnett and Amir Zmora

1. WebRTC Standards &
Implementation Q&A
Amir
Zmora
TheNewDialTone
Dan
Burne3
StandardsPlay
Alex
Gouaillard
WebRTC
by
Dr
Alex
/
Citrix

2. Session sponsored by
WebRTC.ventures
is
a
custom
design
and
development
shop
dedicated
to
building
WebRTC
based
applicaFons
for
web
and
mobile.
We
have
built
end-­‐to-­‐end
broadcast
soluFons
for
events
and
entertainment
clients,
telehealth
soluFons
for
mulFple
clients,
live
support
tools,
as
well
as
communicaFon
tools
for
a
variety
of
other
applicaFons.
WebRTC.ventures
is
a
recognized
development
partner
of
TokBox
and
has
also
built
naFve
WebRTC
soluFons

3. We use CrowdCast….It’s WebRTC

4. WebRTCStandards.info

5. About Us
• Amir Zmora • Dan Burnett • Alex Gouaillard

6. Screen Capture & Screen Sharing with WebRTC

7. Screen Sharing in WebRTC
• Is WebRTC plus Screen Capture
• Screen capture gives you MediaStreamTrack
• WebRTC lets you send it
• We will talk about the Screen Capture piece

8. Security in native apps
• If you install it, the app has complete access to your device
• So, choosing not to install is the first level of security

9. Security in the Web model
• Visiting a site is the "install"
• But visiting a site needs to be safe
• So, the Web uses site origin as security
• By default, limited access to the device browser runs on
• Also, page has access to JS it loads but no access to JS from other tabs/windows

10. Problem - API keys in stupid sites

11. Screen capture breaks web model
• Browser controls allow Site A to do a user View-Source on Site B
• Normally, user can see B's popped up source but A can't read
• But with screen capture, A can read

12. Nasty scenario
• Site A uses WebRTC with user permission to access camera, screen
• Site A scrapes screen image to see what other tabs/windows user has open in browser
• Site A tracks user's eyes with camera
• When user looks away, Site A does view-source on a tab, scrapes the screen, closes
view-source window

13. WebRTC Screen Capture standard
• http://w3c.github.io/mediacapture-screen-share/
• Still very new
navigator.mediaDevices.getDisplayMedia({ video: true })
.then(stream => {
// we have a stream, attach it to a feedback video element
videoElement.srcObject = stream;
}, error => {
console.log("Unable to acquire screen capture", error);
});

14. Protections in the standard
• By default no viewing of other tabs or other browser windows, even in other browser apps
(e.g., Chrome app can't see FF browser)
• Requirement for explicit, elevated permissions in order to view these since one app could
control what is presented on the others
• In practice,
• Permissions will probably be a form of whitelist similar to what FF uses today
• Likely no way for WebRTC apps to get exemptions in advance

15. Screen Sharing with Chrome

16. Chrome Specific WebRTC Bits
Media
Stream
Manager
(singleton@browser)
GUM
JS
API
(tab/sandbox)
1.
Send
request

17. Chrome Specific WebRTC Bits
Media
Stream
Manager
(singleton@browser)
GUM
JS
API
(tab/sandbox)
2.
Check
if
MST
is
already
available

18. Chrome Specific WebRTC Bits
Media
Stream
Manager
(singleton@browser)
GUM
JS
API
(tab/sandbox)
Security
Manager
(source,
origin)
3.
Check
rights
2.
Check
if
MST
is
already
available
-­‐
NO

19. Chrome Specific WebRTC Bits
Media
Stream
Manager
(singleton@browser)
Audio
Capturer
Video
Capturer
Security
Manager
(source,
origin)
GUM
JS
API
(tab/sandbox)
4.
Ask
Corresponding
capturer
type
to
start
capturing
3.
Check
rights
-­‐
OK

20. Chrome Specific WebRTC Bits
Media
Stream
Manager
(singleton@browser)
Audio
Capturer
Video
Capturer
Security
Manager
(source,
origin)
A
GUM
JS
API
(tab/sandbox)
4.
Ask
Corresponding
capturer
type
to
create
one
-­‐
OK
V
5.
Store
the
MST

21. Chrome Specific WebRTC Bits
Media
Stream
Manager
(singleton@browser)
Audio
Capturer
Video
Capturer
Security
Manager
(source,
origin)
A
GUM
JS
API
(tab/sandbox)
V
6.
Trigger
callback
Keep
feeding
frames

22. Chrome Specific WebRTC Bits
Media
Stream
Manager
(singleton@browser)
Audio
Capturer
Video
Capturer
Security
Manager
(source,
origin)
A
GUM
JS
API
(tab/sandbox)
V
NOTE
1:
second
call
for
same
device
with
same
constraints
will
directly
return
the
MST,
that
allows
to
share
streams
across
tabs
without
blocking

23. Chrome Specific WebRTC Bits
Media
Stream
Manager
(singleton@browser)
Audio
Capturer
Video
Capturer
Security
Manager
(source,
origin)
A
GUM
JS
API
(tab/sandbox)
V
NOTE
2:
Recently,
a
second
call
for
the
same
device
but
with
different
constraints
(think
simulcast)
will
indeed
return
a
different
resoluFon.
Before
it
would
return
the
first
resoluFon
asked.

24. Chrome Specific WebRTC Bits
Media
Stream
Manager
(singleton@browser)
Audio
Capturer
Video
Capturer
Security
Manager
(source,
origin)
A
GUM
JS
API
(tab/sandbox)
V
NOTE
3:
Not
only
this
allow
to
share
cams
across
processes,
it
allows
for
global
echo
cancellaFon
(yes,
including
the
key
strokes).
Before
tabs
could
cross
feed.

25. Chrome Screensharing 2 steps (1)
Media
Stream
Manager
(singleton@browser)
Screen/Windows/Tab
Capturer
Security
Manager
(source,
origin)
Screensharing
(extension)
1
2
3
4
S
5

26. Chrome Screensharing 2 steps (2)
Media
Stream
Manager
(singleton@browser)
Screen/Windows/Tab
Capturer
S
GUM
JS
API
(tab/sandbox)
With
ID

27. Screen Sharing with Firefox

28. Firefox
• Whitelisting (wiki.mozilla.org/Screensharing)
• Manual
• Hardcoded
• Extension

29. Firefox
• Whitelisting - Manual
• Manual
• Hardcoded
• Extension

30. Firefox
• Whitelisting - Manual
• Manual
• Hardcoded
• Extension

31. Firefox
• Whitelisting (wiki.mozilla.org/Screensharing)
• Hardcoded
⇒ open a bug!
⇒ Attack surface?

32. Firefox• webex.com,*.webex.com,ciscospark.com,*.ciscospark.com,projectsquared.com,*.projectsquared.com,
• *.room.co,room.co,
• beta.talky.io,talky.io,
• *.clearslide.com,
• appear.in,*.appear.in,
• tokbox.com,*.tokbox.com, *.opentok.com,
• *.sso.francetelecom.fr,*.si.francetelecom.fr,*.sso.infra.ftgroup,*.multimedia-conference.orange-business.com,*.espacecollaboration.orange-business.com,
• example.com,
• *.mypurecloud.com,*.mypurecloud.com.au,
• spreed.me,*.spreed.me,*.spreed.com,
• air.mozilla.org,
• *.circuit.com,*.yourcircuit.com,circuit.siemens.com,yourcircuit.siemens.com,circuitsandbox.net,*.unify.com,tandi.circuitsandbox.net,
• *.ericsson.net,*.cct.ericsson.net,
• *.conf.meetecho.com,
• meet.jit.si,*.meet.jit.si,
• web.stage.speakeasyapp.net,web.speakeasyapp.net,
• *.hipchat.me,
• *.beta-wspbx.com,*.wspbx.com,
• *.unifiedcloudit.com,
• *.smartboxuc.com,
• *.smartbox-uc.com,
• *.panterranetworks.com,
• pexipdemo.com,
• *.pexipdemo.com,pex.me,*.pex.me,*.rd.pexip.com,
• 1click.io,*.1click.io,
• *.fuze.com,*.fuzemeeting.com,
• *.thinkingphones.com,
• free.gotomeeting.com,g2m.me,*.g2m.me,gotomeeting.com,*.gotomeeting.com,gotowebinar.com,*.gotowebinar.com,gototraining.com,*.gototraining.com,citrix.com,*.citrix.com,expertcity.com,*.expertcity.com,citrixonline.com,*.citrixonline.com,g2m.me,*.g2m.me,gotomeet.me,*.gotomeet.me,gotomeet.at,*.gotomeet.at

33. Both Firefox and Chrome
• FF-Whitelisting – Extension / addOn
• Cr – Extension
See e.g. here:
Blog - https://tokbox.com/developer/guides/screen-sharing/js/
Code - https://github.com/opentok/screensharing-extensions

34. ?

35. Thank You
Amir
Zmora
TheNewDialTone
Dan
Burne3
StandardsPlay
Alex
Gouaillard
WebRTC
by
Dr
Alex