Global Data Stream Network for Internet of Things
•
4 recomendaciones•956 vistas
This document summarizes a presentation about data stream networks and how they help the IoT ecosystem. It defines a data stream network as a secure, global network for two-way communication and real-time data push. It discusses how data stream networks provide services like publish/subscribe messaging, global replication and reliability, presence detection, storage and playback, and access control. Finally, it outlines how data stream networks help solve IoT pain points like presence detection, scalability, bandwidth usage, and security by enabling real-time communication, analytics, message routing and more between mobile/IoT/web applications.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Global Data Stream Network for Internet of Things
Similar a Global Data Stream Network for Internet of Things (20)
Último
Último (20)
Global Data Stream Network for Internet of Things
- 1. Introduction • Developer Evangelist • PubNub • @bhavana1110 • bhavana@pubnub.com
- 2. Agenda 1. What is a Data Stream Network? 2. Why do you need one? 3. How does it help the IoT ecosystem?
- 3. What is a Data Stream Network?
- 4. Global Scale for Data Streams Content Delivery Networks (CDNs) Quickly serve static content to websites & mobile apps worldwide. 2000s Internet Backbone Networks (ISPs) Connect all the world’s computers together. 1990s Data Stream Network (DSN) Secure two-way communication & data push in realtime. Today Content Delivery Networks (CDNs) Services Protocols Caching Security QoS Redundancy VPN HTTP HTTPS SPDY Internet Backbone Networks (ISPs) Services Protocols Peering Routing PoPs TCP UDP IP Data Stream Network (DSN) Services Protocols Data Streams Presence Storage Visualization Gateways Syndication Websockets COMET/BOSH HTTP 2.0 MQTT
- 5. Minimizing Time to Market PubNub Confidential © 2015 Data at Rest: Well understood Data in Motion: Difficult to implement Mature software stacks Developers trained & experienced Scaling & security understood Custom Development No reliable, mature standards Exposure to security risks Unreliable connectivity at scale
- 6. Why does a Data Stream Network Matter? PubNub Confidential © 2015
- 7. The two challenges of Realtime
- 8. Global Data Stream Network Secure Scale Manage Connect
- 9. PubNub is a Data Stream Network providing PubNub Confidential © 2015 two-way communication to/from every device in the world. Realtime Reliable Secure
- 10. API & SDKs … and over 50 more
- 11. Different Use case verticals • Social/Entertainment • Connected Vehicles • Smart Home • Telecom • Online collaboration • Audience Participation • Financial services • Gaming
- 12. What do you get with a Data Stream Network?
- 13. Publish/Subscribe Messaging • Protocol-independent pub/sub core messaging service • Messages up to 32kb w/ TLS encryption • Lightweight channel handling: no limit on number of channels & no cost impact • Publish 1-to-1, 1-to-many, many-to-many // publish pubnub.publish({ channel : “hw_test”, message : “Hello World!” }); // subscribe pubnub.subscribe({ channel : "hw_test", message : receiver })
- 14. Global Replication & Reliability • 14 data centers located on 4 continents • 99.999% uptime guaranteed • All messages replicated to all 14 data centers by default • Automatically manages dropped connections & reconnects • Message caching catches devices up on missed data
- 15. PubNub Presence • Built into API for all channels • Automatically detect join/leave or device status change events • Trigger events/behaviors based on presence status changes • API method for retrieving list of device in a given state & device location • Set custom status changes, i.e. GPS location, or any custom name/value pairs // presence pubnub.subscribe({ channel : “hw_test", presence : pres_reciever, message : receiver });
- 16. Storage & Playback • Default 24 hour storage of all messages; unlimited retention available • Catch users up on message history when they come online • Load a configuration onto a device when it is booted up • Audit past messages for compliance • Play back messages at same rate they were received // storage and playback pubnub.history({ channel : “hw_test", callback : sp_reciever })
- 17. Access Manager • Token-based access control scheme (similar to OAuth) • Grant access to individual channels or groups of channels • Read / write controlled at token level • Tokens can auto-expire based on time (TTL) • Grant/revoke tokens manually // Key Authorization var pubnub = PUBNUB.init({ publish_key : "pam", subscribe_key : "pam", auth_key : "3y8uiaj" })
- 18. Data Stream Network for IoT
- 19. The Things
- 20. The Internet
- 21. The different components of IoT • Data stream network does the heavy lifting • Smart network that manages “things” and the data that flows between them. Things Data Stream Network Mobile applications
- 22. IoT pain points Presence detection Decreased time to market Storing offline messages Minimum bandwidth usage Cross platform support security Read/write access Bidirectional data flow Power drain
- 24. IoT pain points • Detect machine connections • Monitor user and machine presence • Global scale Presence Detection • Storage of realtime data • History to retrieve messages • Replay messages in order received Message History
- 25. • Sensors • Always on devices • Protocol independent approach Bandwidth/power usage • Millions of devices • Plug and play configuration • Decreased time to market Scalability
- 26. Devices Vulnerable to Hacking IoT + Security
- 27. Enterprise-Grade Security Cross-platform AES & TLS Message envelope meta- data for smart routing & analytics Fine-grain token-based access control Auto-expiration of tokens & manual revocation No inbound open ports removes attack vector Can route to different data centers to thwart regional attacks. Safe Harbor Compliant Safe for HIPAA and PII Intelligent Data Routing Access Control Attack Prevention LegislativeEncryption
- 28. Why a Data Stream Network matters for IoT? • Realtime communication • Analytics • Message Routing • Scalability Mobile/IoT/web
- 29. IoT Use cases • Control and monitor devices • Stream vehicle data • Trigger automatic actions • Monitor device metadata • http://www.pubnub.com/solutions/ Home Automation Connected/Smart cars Wearables/ Connected devices Coffe pot monitor : https://vimeo.com/134363392 PubNub security camera : https://vimeo.com/131005832 PubNub IoT house : https://vimeo.com/114900127 Motion controlled servos with leap motion and the Pi : https://vimeo.com/136779399
- 30. Thank you
Notas del editor
- Hi everybody, thank you for coming. My name is Bhavana srinivas and I work as a developer evangelist for PubNub. PubNub is a global DSN that provides easy to use API for real time , reliable signaling between sensors, mobile devices, embedded chips, or software applications, anywhere around the globe in a secure manner. We have 70+ SDKs that we support, to let you build and scale real time applications for mobile, web and IoT. As I go through my presentation, you will see what a data stream network means and how it will benefit IoT. My twitter handle is @bhavana1110 and you can reach me at bhavana@pubnub.com. I have some cards and data sheets and frisbees with me as well in case you are interested.
- I will walk you through what a data stream network is, what you can do with it and why it is criticial for IoT.
- Lets start of by understanding what this even means and then move on to how we can use it for IoT.
- BUILD SLIDE More and more companies/software's needed the ability to control and work with real time data. Whether you needed stock updates, multi player game movements, screen sharing, voice over IP, telephony etc, you had this new data in motion, as compared to the traditional data in rest, which can be pulled out of a database every now and then. So data is being constantly emitted be it social streams or financial streams, and there was no way to route it, or manage it or any kind of infrastructure to get the data to the devices that needed it in real time. more and more companies were spending all their time and effort to build this kind of infrastructure than focussing on their application logic. So just to make this really clear where PubNub sits, this is the evolution of the Internet. MAKE NOTES FROM TODDS PRESENTATION. Lets go back 25 years or so. People were building the core infrastructure for the internet. So you had companies like Verizon, Comcast and level3 that were offering peering, routing or PoPs that speak TCP/UDP. Basically, connecting you to the internet. Fast forward 10/15 years, you had all these webisites with tonnes of informtaion , all trying to scale. So you had CDNS like Akamai, cogent etc, that made copies of the information on these sites, and put them all over the world. If you were trying to access yahoo weather info from Japan, you would probably be connected to a data center that’s diff from someone accessing that information from California. These guys were speaking difffernet protocols like HTTP to do caching services etc. A data stream network is similar to this, but meamt for two way communication to and from these data centers. U still have data centers all over the world, but you are publishing and subscribing data from them, and also speak other protocols like websockets,, MQTT, which are constantly evolving.
- The current infrastructure for data at rest is pretty mature and well understood..They have been used for the past so many years, and they make it very easy to build mobile and web apps. For data in motion on the other hand, its first of all not compatible with the infrastructure available for data at rest applications. The software or technologies themselves are relatively new. There are several open source technologies that can be used, but when you have to scale it becomes a problem. They are not secure, since they are new as well, and hasn’t been documented well. So what we’ve just seen is that pretty much the whole internet is architected to handle data at rest. We’re interested in data in motion – streaming data. Similar to how you wouldn’t build your own ISP or CDN, what we’re suggesting is you shouldn’t try to architect your own DSN either. You can roll your own, but… Time to build is costly and increases time to market. Hidden costs of scaling up are disproportionately expensive. Ongoing maintenance and orchestration requires a dedicated team of experts. Flexibility and expansion potential constrained by limited feature sets. Security is an ongoing risk.
- When you are devloping an application on a small scale , it is easy. So does it make sense for everyone building apps, build hteir own dsn? No. why? But in the real world, you have a lot of problems like unrealiable connectiviity’ different networks, - devices are always moving from one network to another. Within tunnels or losing powerr anf hence connectivity. diff devices to be supported - Latency scaling security and many more.
- Add notes from Todds presentation on what a global DSN is. This is why using a Data stream network helps,, since it provides a way to connect, scale and secure real time data streams for rhe web, mobile and IoT. Massive numbers of connections across any device Extremely low latency Unmatched reliability / uptime End to end security over our network Connect Support several platforms Powered by publish/subscribe messaging Storage & Playback functionality Secure End to end security: TLS/SSL & AES included by default PubNub Access Manager for token-based access control No open ports to the web EU SafeHarbor, HIPAA compliant Scale Global data replication to 14 data centers Intelligent routing automatically connects users to closest PoP Message & bandwidth efficiency with PubNub Stream Controller Add scale easily, automatically, and economically Audience simulation service for smooth rollouts & QA Manage Developer console & debug tools Stream analytics shows data stream activity in real time Steam syndication design pattern makes it easy to monetize data streams Business model scales with your project’s success No year-long contracts
- PubNub excels because we deliver:
- If you’ve had a chance to align on required capabilities before the call, list them here. If not, think about taking the time to fill this in with the prospect during the presentation. Real-time data transfer speeds Real-time location tracking Extremely high reliability (works in the real world) Easy, efficient scalability Robust security built in Developer tools to build in preferred environment Business model that scales with your success Add-on features to enable your use case
- Explain the reasoning behind each of these features as per Todd’s video. A data stream network has the ability to manage bi-directional communication between the devuces connected to them. This can be 1-1, 1-many or many-to-many, basucally the ability to publish and subscribe to messages. For several applicattions, you want to be able to know who is part of the conversastion, or who has joined that channel of communicaationEssentially you wajnt to know the presence on that channel. ThisThis is similar to ,the green dot you see in a chat app like skype, when a person is online. Or a grey when they leave. This is being used a lot in IoT, where you want to know when your remote controlled thermostat comes online, or also know when that connection goes off. Now that being said, say you have several users on a chat channel, or several devices that are sending and receiving information using the data stream network. . You are able to receive messages when you are connected to the network. But say your devuce went offline for a day, month or year. You want to be able to see the history of the streams that you missed on that channel and played back at the same rate at which it was received. Also, in order to secutre the deivices that send/receive messages through the network, you want to be able to give fine grained read/ write access on a channel and token level. This way, only devices that have access can publish or subscribe.
- What is the current state of IoT. What are the different verticals What are some of the common problems – no bi directional comm/ no uniformity ex all devices cant talk to each other/ memory/bandwidth/security/ How does a data stream network/pubnub solve it
- So, as is often described, there are many machine to machine communications within the category of IoT. You have smart home appliances to wearables that may or may not be plugged into medical diagnostics to connected cars and transportation. These, along with many other categories are the “Things” in IoT.
- But when describing the “Internet” part of IoT, most presentations simply had some sort of amorphous cloud that connected everything magically. The point is that everyone seems to talk about the big data and the devices that make up the IoT, but not the lines to the “cloud” that connects it all. People are working on different verticals. And a very common application is when a sensor or a device is sending information back to the user. But you want to be able to do more than just receiving info from your thermostat. You want to be able to take some actions based on that value. You want to be able to control the devices you own, remotely, irrrespective of where you are in the world. You want to be able to do this in real time, cos noone wants their garage door to open five min after you press a button. You want it to happen instantly. And that’s where a data stream network like pubnub fits in. It is the communication layer that connects the IoT.
- What is the best way to build an IoT application that handles everything from controlling home appliances remotely to inform an aero-engine manufacturer that one of its engines need servicing in the opposite side of the world to gathering information from sensors destributed in the wild? The starting point for Internet of Things applications are the things themselves. These edge devices typically have no screen (although that's not always the case), a low-power processor, some sort of embedded operating system and a way of communicating (usually wirelessly) using one or more communication protocols. The things may connect directly to the Internet, to neighboring things or to an Internet gateway device – typically a plastic box with blinking lights. The next tier of the system, is a software and infrastructure that runs in a corporate data center or in the cloud and receives and organizes the streams of data coming from the things. Software running in the ingestion tier is usually also responsible for managing things and updating their firmware when necessary. After this comes the analytics tier; this takes the organized data and processes it. Finally, there's the end-user tier, the application that the end user actually sees and interacts with. This may be an enterprise application, a Web app or, perhaps, a mobile app.
- So from the previous image, you can see how there are so many problems to solve startting from the “tjings in the internet of things, the network or infrastructure trat will support them, and the devices that run the application. Lets start with the things – there are so many of them, from MCUs to MPUs, all runninf diff Oses, diff platforms etc. A samsung device doesn’t talk to an LG one which doesn’t talk to Apple devices., Whehn they are so varied, you need cross platform support. Let me go through some of the obstacles in building an IoT application. power drain : PubNub’s publish / subscribe paradigm reduces “chattiness” to keep power drain to a minimum bandwidth : Protocol-independent approach uses the minimum bandwidth necessary to monitor and control IoT devices Firmware: Use secure, private channels to remotely upgrade firmware with no user action needed time to market : Don’t have to waste months of your team’s time architecting and maintaining a realtime infrastructure in-house device provisioning: Make your IoT device plug-and-play, even through home and business firewalls The way we do security is that the device opens a connection to Pn, and we keep that connection opne, so that now data canonly flow back through the pn nextwork to the device. Remote Firmwarae upgrades – we don’t provide the FTP server that people usually need for this. We have a design practise to do this. The server first lets its devices know that a new firmware update is available by broadcasting an alert message on a channel that all devices can read. Then the devices that are online (and we know which ones they are through the use of Presence) download the update immediately. For those devices that are not online, they can download the update from a cache on the channel when they are booted back up and available.
- For some applications, its all about collecting data from the device like a sensor . For those, uni directional communication is more that enough Data collection is: * Easy * Everyone is focusing on it today [CLICK FOR ANIMATION] Device Control is: * Harder * With smart home, connected cars, industrial valve controls, robots, etc. we don’t just want to read their status, we also want to control the devices The promise of IoT is with device control, it’s not just about receiving the data from the devices
- Built into API for all channels Automatically detect join/leave or device status change events Trigger events/behaviors based on presence status changesstate & device location Set custom status changes, i.e. GPS location, or any custom name/value pairs We have spotty connections everywhrre, when a handoff happens. In remote areas and when you go under tunnerls. This is when devices go offline. There is a meed for the device to automatically try to reconnect, and come back online as soon as possible. When it does, it needs to access all the information that might be lost, at the same rate that it was sent out,. Default 24 hour storage of all messages; unlimited retention available Catch users up on message history when they come online Load a configuration onto a device when it is booted up Audit past messages for compliance Play back messages at same rate they were received
- These sensors and hardare devices are becoming very small and cheap., but the constraint is power and memory. They have low program memory, and also cant last too long on, unless they conserve battery. The device must be able to send and receive messahes , by not spending too much energy . When building an application, it is usually targeted to a large number of users. Once these devuces are deployed, it is necessary to ensure that every one of them can communicate with the DSN. When bought, these devices will prove ineffective if they need to be configured to work with the house/office firewalls. People expect it to be plug and play. You bring a new speaker at hime, that connects to the internet, you expect to be able to set it up really easily.
- Already, too many manufacturers worry more about getting their IoT product to market quickly than about securing it. In some cases, the devices are so small that it’s hard to build in the right protection. how do you go about managing the username/passwords for your ever increasing number of connected devices and appliances? What about the privacy of your information? Take as an example various Internet connected video cameras with easily defeated security controls or baby monitors. So offloading security to the network is the right answer for many developers. Jeep Hack (August 2015) KQED.org (August 2015): http://ww2.kqed.org/futureofyou/2015/08/03/millions-of-americans-use-medical-devices-that-are-vulnerable-to-hacking/ Deep brain neurotransmitters Gastric stimulators Foot drop implants Cochlear implants Cardiac defibrillators / pacemakers Insulin pumps
- Security has to be a forethought more than anything else. The race to market for new devices means that security is not included in the design, or it is severely limited or poorly implemented. ’ You need to be able to encrypt the message using maybe TLS to secure the message from the device to the network, and trhen use something like AES that does the encryption even when it goes through the networlk. OPEN PORTS: most devices on the internet, are also listening to any messages that come to them throufh the network. They always have open ports. This is asking for trouble since anyone can hack into that and sned them the wrong messages. You want to be able to open a connection to your device tham the other way rounf to be more secure. TLS + AES: ACCESS CONTROL – giving different users diff read/write permissions.
- That's why it usually makes more sense to build an application on top of a ready-made "Internet of Things platform," Gillett adds. These platforms usually include an ingestion tier that carries out time-series archiving for incoming data, as well as an analytics tier, thin provisioning, activation and management capabilities, a real-time message bus, and an API to allow communication between the platform and applications built on top of it. Build vs buy argument. How scale also helps.
- A data stream network lets you manage, connect, scale your realitme applications for mobile web and IoT. At pubnub as well, we have seen several customers , over 2000, in a lot of verticals like chat, social apps, screen share, collaborative apps and IoT. With IoT, we have seen people using us for home automatrion, devcice contril, connected cars and connected devuces, We have seen and built a lot of demos from hobbysists like raspberry pi and arduino community. They involve bidirectional messaging to and from the device.