Develop a portal to manage your IoT Hub solution
•Download as PPTX, PDF•
1 like•690 views
Azure IoT Hub allows you to transform your embedded devices into the IoT devices. Azure with portal and CLI allows you to manage the lifetime of your devices, but only if you are an expert and experienced in Azure. How can your customer, for which you have developed the solution, manage the solution independently? In this session, we will start from the principles of the IoT Hub about the configuration and management of device lifetime. We will look into the APIs and tools for managing IoT Hub. We'll talk a little bit about telemetry and its meaning. Then we will start building a portal in ASP.NET to manage everything. And in the end we will talk about multitenancy and what it means to sell (from the technical side) IoT solutions to several customers.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Develop a portal to manage your IoT Hub solution
Similar to Develop a portal to manage your IoT Hub solution (20)
More from Marco Parenzan
More from Marco Parenzan (20)
Recently uploaded
Recently uploaded (20)
Develop a portal to manage your IoT Hub solution
- 1. DEV031 - Sviluppare un portale per gestire la tua soluzione IoT Hub MARCO PARENZAN WPC2017 1
- 2. @marco_parenzan .NET, IoT and Azure Developer, Trainer and Consultant Microsoft MVP 2017 for Azure Book Author @ Packt, LSWR Community Lead 1nn0va facebook.com/1nn0va
- 3. DEV031 - Sviluppare un portale per gestire la tua soluzione IoT Hub MARCO PARENZAN WPC2017 3
- 4. Agenda •Azure for the Internet of Things •Using CLI •Managing Device Identities with Azure IoT Hub •Managing Devices with Azure IoT Hub •Handling Events with Azure Serverless •Conclusions WPC2017 4
- 5. Azure for the Internet of Things
- 6. Things IoT Pattern + Edge Insights ActionsInsights Actions Cloud Gateway
- 7. Things Azure “Insights & Actions” Insights ActionsInsights Actions Cloud Gateway
- 8. Comprehensive set of offerings for IoT Azure Time Series Insights Azure Machine Learning Azure Stream Analytics Cosmos DB Azure Data Lake Azure Data Lake Analytics Azure HD Insight Spark, Storm, Kafka Azure Event Hubs Microsoft Flow Azure Logic Apps Notification Hubs Azure Websites Microsoft Power BI Azure Active Directory Azure IoT Hub Azure IoT Hub Device Provisioning Service Azure IoT Edge Azure Monitor PaaSServices& DeviceSupport Edge Support Device Support Azure IoT Device SDK Certified Devices Azure Certified for IoT Security Program for Azure IoT IoT Services Data & Analytics Services Visualization & Integration Services IoTSolutions (PaaS) IoTSolutions (SaaS) Microsoft IoT Central IoT SaaS Microsoft Connected Field Service Field Service SaaS Remote Monitoring Predictive Maintenance Connected factory Windows 10 IoT Core Azure IoT Suite
- 9. Things The cloud gateway Insights ActionsInsights Actions Cloud Gateway
- 10. Azure IoT Hub Bi-directional communication Enterprise scale & integration End-to-End Security
- 11. IoT Hub messaging IoT Hub
- 13. Telemetry sending, acquiring and measuring data Event State (or state event) Message
- 15. Receiving a message as a service guarantees reliability and durability handling messages. handles intermittent connectivity on the device side. at least once 1day (default) to 7 days Custom Processor, Stream Analytics, Azure Func
- 16. Limitations and Guidance Transient storage 50 messages 48 hours 64Kb
- 17. IoT Hub messaging pricing and scaling Service instance made of units Device messages (limit 256Kb) are billed in 4Kb chunks (0,5Kb for Free tier) Twins messages (limit 8Kb) are billed in 0,5Kb chunks
- 18. Things Developing things Insights ActionsInsights Actions Cloud Gateway
- 19. Developing things Comprehensive set of SDK Supporting languages Supporting hardware Test with FakeDevices
- 20. DEMO 20
- 21. Device has a «functional» lifetime
- 22. Device has a «non functional» lifetime
- 23. Using CLI
- 24. Azure Resource Manager Infrastructure as Code Declarative Approach Reproducibility Consistency Repeatability Disposability Service continuity Self-testing systems Small changes Version all the things
- 25. PowerShell Born in 2004 with Monad project Most complete and powerful command line experience Not simple to learn, but not too hard Most powerful Windows devops experience Most powerful Azure devops experience Now available on Linux and Mac with .NET Core
- 26. IoTHub Explorer Command line experience to manage IoT Hub npm install -g iothub-explorer (requires Node.js) https://github.com/Azure/iothub-explorer Pro - Specific commands: • monitor-* (feedback/events/uploads/ops) • Work with twins/jobs/direct-methods
- 27. Azure CLI 1.0 (a.k.a. Xplat CLI) Command line experience to manage Azure Subscription npm install -g azure-cli (requires Node.js) https://github.com/Azure/azure-xplat-cli Pro – ASM Support Pro – First multiplatform devops tool Cons – will die in favour of Azure CLI 2.0 as ASM «disappear»
- 28. Azure CLI 2.0 Command line experience to manage Azure Subscription Complete rewrite curl -L https://aka.ms/InstallAzureCli | bash (MacOsX/Linux) pip install --user azure-cli (Windows) (requires Python) https://github.com/Azure/azure-cli Pro – consistent commandline experience Cons – only ARM support, just released, miss many services
- 29. DEMO 29
- 30. Managing Device Indentities with Azure IoT Hub
- 32. What is a Shared Access Policy? Used to authorize services Is a permission to services or devices to access some endpoints Uses symmetric key encryption technology for token authorization Good practice: 1 policy, 1 service
- 36. Device Registry Informations deviceId is the name you assign to the device. generationId is a property used to distinguish devices with the same deviceId, but that are deleted and recreated. So the real key should be deviceId+generationid auth contains authentication information such as the symmetric keys, that are the couple of primary and secondary keys shared with IoTHUb used to secure each message. Those keys are stored in BASE64 format Status, statusReason and statusUpdateTime are used to disable or enable the device and trace why device was disabled and when. If disabled, the device cannot use its identity to access to IoTHub. connectionState and connectionStateUpdatedTime shows if the device is connected or not. This property is available only if you use AMQP or MQTT protocols and it is updated only every 5 minutes. So it can contain false positives and should be used only for debugging and testing purposes. You need to implement some sort of heartbeat functionality on your device to have a real feedback on connection from your device. lastActivityTime tracks the last operation on that device
- 37. Registry Read/Registry Write Permission
- 38. Transport Level Security TCP based protocols (HTTPS, MQTT, AMQP) Endpoints exposes certificates with public key Automatically handled by TCP/IP stack DEVICE IOT HUB (TCP endpoint) TCP Connection
- 39. Asymmetric Key encryption Used to receive secure data by the parties Couple of keys. Private keys, kept safe by the generator of the keys, decrypt what is encrypted by the public key, Either self signed certificates or CA certificates (preview) Pro • single point of failure • long keys (2^10 bit+) difficult to decrypt Cons • can encrypt small chunks of data Used to encrypt a symmetric key at each communication Encrypt( , , )= Decrypt( , , )=
- 40. Symmetric Key encryption Used to exchange secure data by the parties Single Key shared by the parties Pro • Can encrypt big blocks of data Cons • Unsecure if one of the parties loose the key, multiple point of failures Encrypt( , , )= Decrypt( , , )=
- 41. Generate the symmetric key DEVICE IOT HUB C:>az iot device create –hub-name <hubname> --device-id <deviceId>
- 42. All devices have different keys Device Registry
- 43. Authorization with encryption Encrypt( , , )=
- 44. DEMO 44
- 45. Managing Devices with Azure IoT Hub
- 46. IoT Hub Device Management Device Twin Queries Methods Jobs
- 47. Device Twin Twin is the logical representation of the device. It is a JSON document that stores device state information. Information is properties that you can distinguish as tags, desired properties, and reported properties. The document is stored in IoT Hub, in a CosmosDb-like eventually consistent container. In general, all properties as just JSON properties so you can write anything you what respecting JSON rules. It can be patched
- 48. Device Twin Limitations Properties can have a maximum depth of 5. The size of the property values cannot be bigger that 8Kb max . JSON types supported: boolean, number, string, object. Arrays are not allowed The document is updated and synchronized with device handling optimistic concurrency. 8Kb (billing size: 16 messages)
- 49. Desired properties We need to configure the device. It is not cost effective to perform locally. Desired property is a kind of property that is configured on the twin. IoT Hub handles the change queuing the update on the device endpoint. So when it reconnects, it will update its state. The maximum size of desired properties is 8Kb.
- 50. Reported properties The device has a local state. That state changes because device runs some tasks. You want to know that. The device can send updates on these when they change. IoTHub receives a message from the device endpoint about the update, and that is changed on the twin. The maximum size of reported properties is 8Kb.
- 51. Tags Desired properties and reported properties are functional for the device. Some properties are useful only for the service and not the devices. It’s a key/value data dictionary.
- 52. Querying Device Twins FROM WHERE SELECT GROUP BY
- 53. Sample queries Devices located in the US configured to send telemetry less often that every minute Devices which have wifi o wired connectivity Devices where reported and desired properties do not match Devices group by status
- 54. Device Management Lifetime Operations Reboot Factory Reset Firmware Update Configuration Reporting progress and status
- 55. Direct Methods Immediate confirmation Two-way data flow MQTT 8Kb request - 8Kb response MQTT
- 56. Demo
- 57. Handling Events with Azure Serverless
- 58. Where storing a message Azure SQL Database Azure DocumentDb Azure EventHub Azure Queue Azure Service Bus Azure Storage
- 59. Alternatives to process events Stream Analytics Event Processor Functions
- 60. Comparing different ways Data streaming Event correlation High scalability Single event Performance not critical Custom coding Flexible coding Special hosting requirements Special performance requirements
- 61. Serverless manifesto Function are the unit of deployment and scaling. Scales per request Users cannot over- or under-provision capacity. Never pay for idle (no cold servers/containers or their costs) Trigger-based invocation code run because of an event happened and has to be handled
- 62. What is Azure Serverless https://www.geekwire.com/2017/serverless-nirvana-microsoft-azure-cto-mark-russinovich-future-cloud/ https://www.geekwire.com/2017/interview-microsofts-mark-russinovich-intersection-serverless-edge-computing/
- 63. "Ho i dati e ora devo creare una dashboard. Posso usare Power BI? No grazie, fanne una tu, così ne abbiamo completamente il controllo"
- 64. Demo
- 65. Conclusioni
- 66. IoT Hub Availability Highly available (Service Fabric-based) Single Region Registry Import/Export (manual)
- 67. Group devices and control access according to your organization's needs Securely authenticate devices, on-board for management and provision for service Provide updates, configuration & applications to assign the purpose of each device Monitor device inventory, health & security while providing proactive remediation of issues Replace or decommission devices after failure, upgrade cycle or service lifetime IoT Device Lifecycle PLAN PROVISI ON CONFIG URE MONI TOR RETIRE
- 68. IoT Hub Device Provisioning Devices are automatically and securely connected to the IoT Hub service and provisioned with initial configuration A single device provisioning tenant can provide service for multiple IoT hubs (in multiple regions) Customers provide rules and logic to assure the right device is attached to the right IoT solution (and associated IoT Hub endpoint) Device provisioning ability is extensible with support for several types of identity attestation patterns
- 69. Device Provisioning Service Setup Devices already have the URI for the device provisioning service at first boot Device provisioning service already knows about the IoT hubs to which it can connect Device provisioning service already knows the type of identity attestation it is using, including connection info for each IoT Hub Device registry (ID/key and metadata) Device IoT Hub device provisioning service Distribution rules IoT Hub Device registry (ID/key and metadata) Enrollment list
- 70. 70
- 72. Benefici attesi dall’Industria 4.0
- 73. Q&A Domande e risposte WPC2017 73 Marco Parenzan – 1nn0va marco [dot] parenzan [at] 1nn0va [dot] it Twitter: @marco_parenzan AZURE MVP, 1nn0va Community Lead, Trainer and Consultant on .NET, Azure and IT
- 74. Corsi consigliati • MOC535 (replaces 534) WPC2017 74
- 75. Contatti OverNet Education Info@OverNetEducation.it www.OverNetEducation.it Rozzano (MI) +39 02 365738 Bologna +39 051 269911 www.wpc-overneteducation.it ROZZANO (MI) – BOLOGNA ROMA – NAPOLI – GENOVA TORINO WPC2017 75
Editor's Notes
- 12/1/2017 7:37 AM
- IoT Hub is all about messaging. IoT Hub is a messaging gateway. All the messages enter in IoT Hub or exit from IoTHub endpoints, that is an abstraction of some queuing technology. Infact we speak about queues, EventHub compatible-endpoints, or custom endpoints like a real EventHub, ServiceBus queue or topic. IoTHub is a messaging broker, that supports devices implementing resilience, availability and scalability for them. That is what we already called «service assisted communication»
- Here you can have some problems. You are a software guy, you don’t know what a device is, it is not a PC. It is a situation where device capabilities are not a requirements, but often a constraint. You have no video or keyboard or mouse to interact. You can have intermittent connections. You can talk a plethora of protocols, not just you friend IP protocol. This happens only on modern devices. But you can also have legacy protocols, because old devices cannot be dismissed or you have low powered devices and you need to use less expensive protocols from the energy consumption point off view. You can’t also reach the device because it can be in places difficult or too expensive to reach. So you have two topics here. How do you develop on a device and how you communicate with the devices.
- Telemetry is the process of sending, acquiring and measuring data from devices. The data can be an event. The event is the trace about something that is happened at a definite time. It contains some values that describe that event. The data can be also state. State is a set of values that gives you some measures from some observed properties in the device at a definite time. And you can consider it also an event if you say “I take a snapshot at that time of the state and I notify about that.” In Azure IoTHub, either state or event are implemented with the message artifact.
- The message is a sort of envelope. It takes your original telemetry and carries that as opaque. Its content is not interesting for IoTHub. It is only a payload that has to be wrapped in a message, unwrapped at the destination. In can also be rewrapped into another message, sometime, if it needs to be forwarded. If your process requires to perform some tasks on some values that are external to the message but not regarding IoTHub properties, you can add your information to the Application Properties data dictionary, adding a literal name. System properties are properties related to the IoTHub infrastructure, which are known by IoTHub and are used to customize some processes, like message routing.
- So the message is received by IoTHub. IoTHub guarantees reliability and durability handling messages to help during situations when there can be intermittent connectivity on the device side. IoTHub does not acknowledge the device if the message is not received. But it can happen that it receive the message, but it does not completes notifying the device it received the message. IoT Hub implements “at least once” delivery guarantees about messaging on this side. When the messages arrive, is kept in IoTHub with a transient approach. You can configure how long IoTHub should keep into it, from one day (the default) to up seven days. Then it is treated as a dead message. So you need to decide how much time you have to handle the message, from a failure point of view. The messages are then available through an EventHub compatible interface. You have many options to handle that. First one is writing an EventHub’s event processor. You can do that with.NET SDK for example. It’s the most flexible but also more complicated way to do that. Then you can also use Azure Stream Analytics, which gives you the expressive and comfortable power of an SQL-Like language to manage events with a time window-based approach. Using Stream Analytics is overall great if you want to make some consideration about multiple events correlation in the same time window. Or you can use Azure WebJobs SDK or now Azure Function, the serverless approach, to handle a single event based approach.
- You need to handle the messages. IoTHub is not designed to keep messages. It’s a transient storage. You have to balance the number of messages the cloud sends to the device. The device queue can hold a maximum of 50 messages. After those, sending more messages returns an error. So this measure You have a retention time, a maximum of 48 hours. After that, the message is becomes a «dead letter» message. Remember that you can send messages down to the device not greater than 64Kb.
- IoT Hub can scale to million of devices with IoT Hub. But when you scale, you don't think about resource allocation, CPU or memory. You just think about messages. How many messages and devices you need. That's why with IoT Hub you talk about a "unit," that is the scaling element. Each unit has a different capacity in different tier. Each differs in the number of messages a unit can handle per day. But you have to be careful. During the course, you will often meet the notions of message and size. Well, that size is a variable size that depends on by the functionality, for example, 8Kb or 256Kb) The "unit" message size is different. A "unit" message size is a billable message and is a fixed size 4Kb chunk of data in standard tier, 0.5Kb in the free tier or if the message is about device configuration with twins. For instance, if a device sends 16 kilobytes message, in S1, S2, and S3 tiers, it will be billed as four messages. So when you need to configure service to run or scaling, you have to measure you messaging performances. You need to count your devices, your messages, calculate the average size and estimage the number of billable messages. Then you count the number of units you need, and you can change that number during configuration. Or you can deploy other instances of IoT Hub, which is a good practice regarding scalability but also resilience in case of failure. For this last uses, IoT Hub supports you importing and exporting device registry if you need to backup and restore for recovery.
- Device has a «functional» lifetime. Functional means that a device performs some tasks such as measuring and those tasks consume data or produce data. Besides that, it needs an identity. So it can be distinguished from the others. And with an identity, it receives some credentials to communicate in a secure way.
- Device has also a «non functional» lifetime «Nonfunctional» means that a device needs to run some tasks to manage its lifetime, not really the core device functions. For example, it needs to be configured because it is probably headless, so we have to configure it remotely after initial configuration and deployment. We need also to query devices to perform some analysis, either on some classification information or its state. And we need to perform some actions on devices because conditions change through their lifetime, so we need to run some tasks
- IoTHub contains an identity registry to stores all the information about devices. On this registry, you can perform some operations. Create device identity Update device identity Retrieve device identity Delete device identity List all the stored identities Export all identities to Azure blob storage Import identities from Azure blob storage All the operations are available by a REST API so you can build your device registration tool inside your required device management workflow. You can also use one of the SDKs available for the major programming languages and platforms.
- Each device identity contains some standard informations. deviceId is the name you assign to the device. generationId is a property used to distinguish devices with the same deviceId, but that are deleted and recreated. So the real key should be deviceId+generationid auth contains authentication information such as the symmetric keys, that are the couple of primary and secondary keys shared with IoTHUb used to secure each message. Those keys are stored in BASE64 format Status, statusReason and statusUpdateTime are used to disable or enable the device and trace why device was disabled and when. If disabled, the device cannot use its identity to access to IoTHub. connectionState and connectionStateUpdatedTime shows if the device is connected or not. This property is available only if you use AMQP or MQTT protocols and it is updated only every 5 minutes. So it can contain false positives and should be used only for debugging and testing purposes. You need to implement some sort of heartbeat functionality on your device to have a real feedback on connection from your device. lastActivityTime tracks the last operation on that device
- So when you have an identity, you can interact with the device and perform some non functional tasks, for maintenance. There are some operations that you can perform with devices. Device Twin Synchronize the device condition and configuration between cloud and device Queries Dynamic reporting across device twin and jobs to attest device status and health Methods Perform interactive actions on devices Then there are Jobs, but we don’t cover that in this lecture.
- So, what is a Device Twin. Twin is the logical representation of the device. It is a JSON document that stores device state information. Information is properties that you can distinguish as tags, desired properties, and reported properties. The document is stored in IoT Hub, not in the device itself. So you can query that information also when the device is not running, and you don’t need to reach all of them with related latencies. In general, all properties as just JSON properties so you can write anything you what respecting JSON rules. There some limitations. Properties can have a maximum depth of 5. The size of the property values cannot be bigger that 8Kb max . All values in JSON objects can be of the following JSON types: boolean, number, string, object. Arrays are not allowed The document is updated and synchronized with device handling optimistic concurrency. In fact in the document contains also timestamps to handle versioning, at a property level. The timestamps are in UTC and encoded in the ISO8601 format YYYY-MM-DDTHH:MM:SS.mmmZ
- So, what is a Device Twin. Twin is the logical representation of the device. It is a JSON document that stores device state information. Information is properties that you can distinguish as tags, desired properties, and reported properties. The document is stored in IoT Hub, not in the device itself. So you can query that information also when the device is not running, and you don’t need to reach all of them with related latencies. In general, all properties as just JSON properties so you can write anything you what respecting JSON rules. There some limitations. Properties can have a maximum depth of 5. The size of the property values cannot be bigger that 8Kb max . All values in JSON objects can be of the following JSON types: boolean, number, string, object. Arrays are not allowed The document is updated and synchronized with device handling optimistic concurrency. In fact in the document contains also timestamps to handle versioning, at a property level. The timestamps are in UTC and encoded in the ISO8601 format YYYY-MM-DDTHH:MM:SS.mmmZ
- We need to configure the device. We need to configure remotely. It is not cost effective to perform locally, because the device can be headless, but also because you cannot reach it or it can be not cost effective. So desired property is a kind of property that is configured on the twin. IoT Hub handles the change queuing the update on the device endpoint. So when it reconnects, it will update its state. The feature is great to handle situations where the connection is not available or because the device is not always available. The device receives a notification in the form of name/value tuple that device will use to update its local state. The maximum size of desired properties is 8Kb.
- The device has a local state. That state changes because device runs some tasks. Think about a temperature level, a switch state, open or closed. You want to know that. The device can have a big state made of lots of values. And you need to know about some of that. So the device can send updates on these when they change. That’s the role of reported properties. From the service point of view, IoTHub receives a message from the device endpoint about the update, and that is changed on the twin. The maximum size of reported properties is 8Kb.
- Desired properties and reported properties are functional for the device. Some properties are useful only for the service and not the devices. You need to organize devices with information about the role, location, dates, and so on. For example, you can save the geographical information where the device is deployed or the date when the device was deployed to plan maintenance. It’s a key/value data dictionary. It can be enough for some businesses, and not enough for some others. So you can just save a foreign key for a configuration document in some other external storage.
- All the device twins are persisted in the registry, in a sort of NoSQL database. All data in the twins can be queried with an SQL-like language to perform any listing or summary query for statistics or deep dive evaluations. It contains extensions for the IoTHub requirements. FROM clause allow selecting documents allowed, at the moment the only allowed keywords are devices or jobs. WHERE clause allows expressing filter predicated on the twin document structure. Root properties are tags, properties.Reported, properties.desired and you can. You can fully write predicates to include or exclude devices for the result of the query. You can also use functions to manipulate data such as math functions, type testing functions, string manipulation functions. SELECT allow to project twin property values or tags, so to represent the correct data set minimizing data or to evaluate some summarizing values with aggregate functions GROUP BY allow aggregating values by some values
- Apply supplied firmware image on a device Initiate a restored on a device Revert device to initial factory image and configuration Use of device twin properties to configure behavior on a device
- Direct methods represent a request-reply interaction with a device similar to an HTTP call in that they succeed or fail immediately (after a timeout). You use Direct methods for operations that require immediate confirmation A device receives direct methods through a device-specific MQTT topic. In fact, it is only supported MQTT. If you need to use AMQP, you cannot use Direct methods. It’s a two-way data flow. There is a limitation with data exchanged as you can exchange 8Kb in the request and 8Kb in the response. When the device is disconnected, IoTHub cannot reach it. The backend receives a notification when the device is disconnected. You can set the timeout you require to understand that the device is offline, in a range from 30sec to 3600sec.
- Quante volte succede? Beh, più di quante possiate immaginare. Non tutti i clienti accettano di fare Business Intelligence e Data Visualization con uno strumento esterno come Power BI. Quindi con .NET, JavaScript e Azure alla mano, ci mettiamo a sviluppare una soluzione. Vedremo quali servizi, librerie e pattern possiamo o dobbiamo usare per farci fare un lavoro quanto più semplice possibile, ma efficace e di effetto. Non sarà mai PowerBI, ma insomma, cerchiamo di non sentirne la mancanza!