Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Janus + NDI @ ClueCon 2021
1. Bringing WebRTC to producers using NDI
Lorenzo Miniero
ClueCon TGI2021
February 17th 2021
2. Who am I?
Lorenzo Miniero
• Ph.D @ UniNA
• Chairman @ Meetecho
• Main author of Janus®
Contacts and info
• lorenzo@meetecho.com
• https://twitter.com/elminiero
• https://www.slideshare.net/LorenzoMiniero
• https://soundcloud.com/lminiero
3. Just a few words on Meetecho
• Co-founded in 2009 as an academic spin-off
• University research efforts brought to the market
• Completely independent from the University
• Focus on real-time multimedia applications
• Strong perspective on standardization and open source
• Several activities
• Consulting services
• Commercial support and Janus licenses
• Streaming of live events (IETF, ACM, etc.)
• Proudly brewed in sunny Napoli, Italy
7. What is NDI?
• Network Device Interface (NDI)
• Royalty-free software standard developed by NewTek
• https://www.ndi.tv/
• Live exchange of multimedia streams within the same LAN
• Multichannel and uncompressed media streams (high quality)
• mDNS used for service discovery
• Easy to use (and integrate) native SDK
• Available on Windows, Linux, MacOS, Android, etc.
• https://www.ndi.tv/sdk/
• VLC team working on an alternative implementation
• https://code.videolan.org/jbk/libndi
• Plenty of applications for different use cases
8. What is NDI?
• Network Device Interface (NDI)
• Royalty-free software standard developed by NewTek
• https://www.ndi.tv/
• Live exchange of multimedia streams within the same LAN
• Multichannel and uncompressed media streams (high quality)
• mDNS used for service discovery
• Easy to use (and integrate) native SDK
• Available on Windows, Linux, MacOS, Android, etc.
• https://www.ndi.tv/sdk/
• VLC team working on an alternative implementation
• https://code.videolan.org/jbk/libndi
• Plenty of applications for different use cases
9. What is NDI?
• Network Device Interface (NDI)
• Royalty-free software standard developed by NewTek
• https://www.ndi.tv/
• Live exchange of multimedia streams within the same LAN
• Multichannel and uncompressed media streams (high quality)
• mDNS used for service discovery
• Easy to use (and integrate) native SDK
• Available on Windows, Linux, MacOS, Android, etc.
• https://www.ndi.tv/sdk/
• VLC team working on an alternative implementation
• https://code.videolan.org/jbk/libndi
• Plenty of applications for different use cases
10. What is NDI?
• Network Device Interface (NDI)
• Royalty-free software standard developed by NewTek
• https://www.ndi.tv/
• Live exchange of multimedia streams within the same LAN
• Multichannel and uncompressed media streams (high quality)
• mDNS used for service discovery
• Easy to use (and integrate) native SDK
• Available on Windows, Linux, MacOS, Android, etc.
• https://www.ndi.tv/sdk/
• VLC team working on an alternative implementation
• https://code.videolan.org/jbk/libndi
• Plenty of applications for different use cases
18. A real use case: CommCon Virtual 2020
• Due to the pandemic, CommCon was done virtually in 2020
• https://2020.commcon.xyz/
• Dan Jenkins, the organizer, wanted pre-recorded talks for more quality
• Speakers would present via WebRTC
• Media production team would record and process
• Per Dan’s words, a bit of a painful process, though
• Mix of different applications for different things
• Jitsi and vMix both used at the same time
• NDI, required by media team, hard to capture
• Manual capture of the Jitsi window for live content
• vMix only supports mixed streams, apparently?
• “Life would have been WAY easier with direct WebRTC-to-NDI...”
19. A real use case: CommCon Virtual 2020
• Due to the pandemic, CommCon was done virtually in 2020
• https://2020.commcon.xyz/
• Dan Jenkins, the organizer, wanted pre-recorded talks for more quality
• Speakers would present via WebRTC
• Media production team would record and process
• Per Dan’s words, a bit of a painful process, though
• Mix of different applications for different things
• Jitsi and vMix both used at the same time
• NDI, required by media team, hard to capture
• Manual capture of the Jitsi window for live content
• vMix only supports mixed streams, apparently?
• “Life would have been WAY easier with direct WebRTC-to-NDI...”
20. A real use case: CommCon Virtual 2020
• Due to the pandemic, CommCon was done virtually in 2020
• https://2020.commcon.xyz/
• Dan Jenkins, the organizer, wanted pre-recorded talks for more quality
• Speakers would present via WebRTC
• Media production team would record and process
• Per Dan’s words, a bit of a painful process, though
• Mix of different applications for different things
• Jitsi and vMix both used at the same time
• NDI, required by media team, hard to capture
• Manual capture of the Jitsi window for live content
• vMix only supports mixed streams, apparently?
• “Life would have been WAY easier with direct WebRTC-to-NDI...”
21. A real use case: CommCon Virtual 2020
• Due to the pandemic, CommCon was done virtually in 2020
• https://2020.commcon.xyz/
• Dan Jenkins, the organizer, wanted pre-recorded talks for more quality
• Speakers would present via WebRTC
• Media production team would record and process
• Per Dan’s words, a bit of a painful process, though
• Mix of different applications for different things
• Jitsi and vMix both used at the same time
• NDI, required by media team, hard to capture
• Manual capture of the Jitsi window for live content
• vMix only supports mixed streams, apparently?
• “Life would have been WAY easier with direct WebRTC-to-NDI...”
22. A real use case: CommCon Virtual 2020
• Due to the pandemic, CommCon was done virtually in 2020
• https://2020.commcon.xyz/
• Dan Jenkins, the organizer, wanted pre-recorded talks for more quality
• Speakers would present via WebRTC
• Media production team would record and process
• Per Dan’s words, a bit of a painful process, though
• Mix of different applications for different things
• Jitsi and vMix both used at the same time
• NDI, required by media team, hard to capture
• Manual capture of the Jitsi window for live content
• vMix only supports mixed streams, apparently?
• “Life would have been WAY easier with direct WebRTC-to-NDI...”
23. A real use case: CommCon Virtual 2020
• Due to the pandemic, CommCon was done virtually in 2020
• https://2020.commcon.xyz/
• Dan Jenkins, the organizer, wanted pre-recorded talks for more quality
• Speakers would present via WebRTC
• Media production team would record and process
• Per Dan’s words, a bit of a painful process, though
• Mix of different applications for different things
• Jitsi and vMix both used at the same time
• NDI, required by media team, hard to capture
• Manual capture of the Jitsi window for live content
• vMix only supports mixed streams, apparently?
• “Life would have been WAY easier with direct WebRTC-to-NDI...”
33. How to make that WebRTC compliant?
• First very simple step may be RTP forwarders1
• This prototype supports RTP ingestion already
• VideoRoom + RTP forwarders could easily fill the gap
• Why not something more integrated, though?
• RTP forwarders do require a bit of orchestration (e.g., ports exchange)
• Custom plugin to receive WebRTC streams, maybe?
• Audio/video decoding + NDI translation in plugin itself (lightweight enough)
• Quite a straightforward process
• Janus has its own RTP stack, no need for the one in rtp2ndi
• Everything else can work pretty much the same way
1
Don’t miss my workshop, in a couple of hours
34. How to make that WebRTC compliant?
• First very simple step may be RTP forwarders1
• This prototype supports RTP ingestion already
• VideoRoom + RTP forwarders could easily fill the gap
• Why not something more integrated, though?
• RTP forwarders do require a bit of orchestration (e.g., ports exchange)
• Custom plugin to receive WebRTC streams, maybe?
• Audio/video decoding + NDI translation in plugin itself (lightweight enough)
• Quite a straightforward process
• Janus has its own RTP stack, no need for the one in rtp2ndi
• Everything else can work pretty much the same way
1
Don’t miss my workshop, in a couple of hours
35. How to make that WebRTC compliant?
• First very simple step may be RTP forwarders1
• This prototype supports RTP ingestion already
• VideoRoom + RTP forwarders could easily fill the gap
• Why not something more integrated, though?
• RTP forwarders do require a bit of orchestration (e.g., ports exchange)
• Custom plugin to receive WebRTC streams, maybe?
• Audio/video decoding + NDI translation in plugin itself (lightweight enough)
• Quite a straightforward process
• Janus has its own RTP stack, no need for the one in rtp2ndi
• Everything else can work pretty much the same way
1
Don’t miss my workshop, in a couple of hours
41. Users will not be in your own LAN, though...
• As it is, the plugin works great for local users
• Browsers can connect to the Janus + NDI plugin directly
• The WebRTC-to-NDI feature works seamlessly
• What if you need to bring in someone remote, though?
• Janus + NDI plugin needs to be in the LAN (or NDI won’t work)
• Hard to make Janus server reachable, if it’s behind NAT/firewall
• (and you’ll definitely want the server to be publicly reachable anyway)
• A couple of options, here
1 Transform the code (again) to a “client” that can talk to the public instance
2 Leave Janus as it is, and “use” it as a programmable client instead!
42. Users will not be in your own LAN, though...
• As it is, the plugin works great for local users
• Browsers can connect to the Janus + NDI plugin directly
• The WebRTC-to-NDI feature works seamlessly
• What if you need to bring in someone remote, though?
• Janus + NDI plugin needs to be in the LAN (or NDI won’t work)
• Hard to make Janus server reachable, if it’s behind NAT/firewall
• (and you’ll definitely want the server to be publicly reachable anyway)
• A couple of options, here
1 Transform the code (again) to a “client” that can talk to the public instance
2 Leave Janus as it is, and “use” it as a programmable client instead!
43. Users will not be in your own LAN, though...
• As it is, the plugin works great for local users
• Browsers can connect to the Janus + NDI plugin directly
• The WebRTC-to-NDI feature works seamlessly
• What if you need to bring in someone remote, though?
• Janus + NDI plugin needs to be in the LAN (or NDI won’t work)
• Hard to make Janus server reachable, if it’s behind NAT/firewall
• (and you’ll definitely want the server to be publicly reachable anyway)
• A couple of options, here
1 Transform the code (again) to a “client” that can talk to the public instance
2 Leave Janus as it is, and “use” it as a programmable client instead!
44. Users will not be in your own LAN, though...
• As it is, the plugin works great for local users
• Browsers can connect to the Janus + NDI plugin directly
• The WebRTC-to-NDI feature works seamlessly
• What if you need to bring in someone remote, though?
• Janus + NDI plugin needs to be in the LAN (or NDI won’t work)
• Hard to make Janus server reachable, if it’s behind NAT/firewall
• (and you’ll definitely want the server to be publicly reachable anyway)
• A couple of options, here
1 Transform the code (again) to a “client” that can talk to the public instance
2 Leave Janus as it is, and “use” it as a programmable client instead!
45. Users will not be in your own LAN, though...
• As it is, the plugin works great for local users
• Browsers can connect to the Janus + NDI plugin directly
• The WebRTC-to-NDI feature works seamlessly
• What if you need to bring in someone remote, though?
• Janus + NDI plugin needs to be in the LAN (or NDI won’t work)
• Hard to make Janus server reachable, if it’s behind NAT/firewall
• (and you’ll definitely want the server to be publicly reachable anyway)
• A couple of options, here
1 Transform the code (again) to a “client” that can talk to the public instance
2 Leave Janus as it is, and “use” it as a programmable client instead!
50. Now that it works, why not turn it into a “service”?
51. Now that it works, why not turn it into a “service”?
• Collaboration between Dan Jenkins (Nimble Ape) and Meetecho
• https://broadcaster.vc/
• Approach pretty much the one we’ve seen in the previous slides
• Public Janus instances for conversations (SaaS, hosted by Nimble Ape)
• Docker images with { Janus + Meetecho NDI plugin } to launch in your LAN
• Orchestration between the two done “automagically”
• Local Janus instance automatically subscribes (WebRTC) to active participants
• NDI feeds automatically generated for who joins your room
Give it a try!
We had to improve a few things first (see next slides), but in Beta now
52. Now that it works, why not turn it into a “service”?
• Collaboration between Dan Jenkins (Nimble Ape) and Meetecho
• https://broadcaster.vc/
• Approach pretty much the one we’ve seen in the previous slides
• Public Janus instances for conversations (SaaS, hosted by Nimble Ape)
• Docker images with { Janus + Meetecho NDI plugin } to launch in your LAN
• Orchestration between the two done “automagically”
• Local Janus instance automatically subscribes (WebRTC) to active participants
• NDI feeds automatically generated for who joins your room
Give it a try!
We had to improve a few things first (see next slides), but in Beta now
53. Now that it works, why not turn it into a “service”?
• Collaboration between Dan Jenkins (Nimble Ape) and Meetecho
• https://broadcaster.vc/
• Approach pretty much the one we’ve seen in the previous slides
• Public Janus instances for conversations (SaaS, hosted by Nimble Ape)
• Docker images with { Janus + Meetecho NDI plugin } to launch in your LAN
• Orchestration between the two done “automagically”
• Local Janus instance automatically subscribes (WebRTC) to active participants
• NDI feeds automatically generated for who joins your room
Give it a try!
We had to improve a few things first (see next slides), but in Beta now
54. Now that it works, why not turn it into a “service”?
• Collaboration between Dan Jenkins (Nimble Ape) and Meetecho
• https://broadcaster.vc/
• Approach pretty much the one we’ve seen in the previous slides
• Public Janus instances for conversations (SaaS, hosted by Nimble Ape)
• Docker images with { Janus + Meetecho NDI plugin } to launch in your LAN
• Orchestration between the two done “automagically”
• Local Janus instance automatically subscribes (WebRTC) to active participants
• NDI feeds automatically generated for who joins your room
Give it a try!
We had to improve a few things first (see next slides), but in Beta now
