Merging Realities
Using the Web to Bring the Internet of Things to High End Augmented Reality
FullStack Conference
#FullStackCon, London UK <2019-07-10>
Philippe Coval
Fabien Benetou
URL: http://purl.org/aframe-webthing
1. #FullStackCon 2019 1 http://purl.org/aframe-webthing
Philippe Coval
Fabien Benetou
Merging Realities
Using the Web to Bring the Internet of
Things to High End Augmented Reality
FullStack Conference
#FullStackCon, London UK <2019-07-10>
2. #FullStackCon 2019 2 http://purl.org/aframe-webthing
$ who are we ?
●
Philippe Coval <https://social.samsunginter.net/@rzr>
– Software engineer for Samsung Open Source Group
– Interest: Web of Things with “Privacy by Design”
●
Fabien Benetou <https://twitter.com/@utopiah>
– WebXR developer consulting for the European Parliament / UNICEF
– Mozilla Tech Speaker
●
Met In 2019 at FOSDEM and mutual inspiration
– Check @MozHacks
3. #FullStackCon 2019 3 http://purl.org/aframe-webthing
Agenda
1) AR and VR on the web
2) WebThings
3) Binding WebThings and XR
4. Samsung Open Source Group 4 https://fosdem.org/2018/schedule/event/tizen_rt/
“Simplicity
is the ultimate sophistication.”
~Leonardo da Vinci
22. #FullStackCon 2019 22 http://purl.org/aframe-webthing
Web of Thing aka WoT
●
W3C working group
– Standardization
●
Interaction with devices
– using Web technologies:
●
JSON, RESTful,
●
HTTP, WebSockets, CoAP…
●
Things description
– JSON-TD / JSON-LD
– Semantics: (iot.schema.org)
Simplified Thing Description
{
"name":"Switch",
"id":"urn:dev:wot:…:switch",
"base":"https://…/switch",
"properties": {
"on": {
"type":"boolean",
// ...
"href":"/on"
}
}
// …
"actions": // …
"events": // …
}…
23. Samsung Open Source Group 23 https://social.samsunginter.net/@rzr
Mozilla WebThing project
●
FLOSS Platform for IoT
– using Web
– with Privacy By Design
●
WebThing API and Schemas
– Can be implemented
– using native (C/C++), Python,
– JS (node.js or IoT.js)
●
Things gateway
– to connect, control (UI)
– and automate webthings
G
D
P
R
G
D
PR
A
rt. 25
A
rt.
25
24. Samsung Open Source Group 24 https://social.samsunginter.net/@rzr
Javascript the language of Web (of Twins)
●
IoT.js: an alternative runtime inspired by Node.js:
– Powered by JerryScript engine designed for micro-controllers
– Feature: Networking (HTTP, MQTT...) & IO (I2C, GPIO, ADC, PWM)
●
Things can be build using webthing-iotjs module:
– Standalone HTTP servers exposing Mozilla Things API:
●
RESTful architecture: read, update operations
– Devices can be connected to MozIoT webthing gateway
●
ACL (JWT), Remote access using pkgkite tunneling
– Or Web services (OpenData, Social Media etc)
25. Samsung Open Source Group 25 https://fosdem.org/2018/schedule/event/tizen_rt/
“The secret to getting ahead
is getting started.”
~Mark Twain
26. #FullStackCon 2019 26 http://purl.org/aframe-webthing
Example: A Robotic arm idea
●
From concept:
– Top level properties: Angles:
●
Torso [-180, +180]
●
Shoulder [0, +90]
●
Arm [0, +90]
●
Hand [0, +45]
●
To early specifications:
– Design Model CAD→ VR/AR
●
Simulation
●
→ Identify integration issues:
– (Location, constraints)
– Implement embedded system
●
Sourcing hardware
●
Controller / Controllee
●
Adjust and iterate
– Thing description+API stay up
27. #FullStackCon 2019 27 http://purl.org/aframe-webthing
WebThing to XR Integration
●
A-Frame components can be accessed from DOM API
●
So attributes can be updated from JS
– Bind Webthing from URL
●
Using HTTP polling (and/or WebSockets if supported)
– If shared through gateway (server):
●
Set URL + JWT token to requests (client app)
– authorization issued from WebUI (devel menu)
●
It works both way:
– Sensors: (Real to XR) & Actuators (XR to Real)
●
https://www.npmjs.com/package/aframe-webthing
28. Samsung Open Source Group 28 https://fosdem.org/2018/schedule/event/tizen_rt/
“Any sufficiently
advanced technology
is indistinguishable
from magic.”
~ Arthur C. Clarke
29. #FullStackCon 2019 29 http://purl.org/aframe-webthing
Digital Twins with WebThing-IoTjs (on STM32)
https://youtu.be/sUayRsjV1Ys
30. #FullStackCon 2019 30 http://purl.org/aframe-webthing
Live control in 3D using A-Frame on GearVR:
https://youtu.be/s3r8pQtzhAU#wotxr-20190320rzr
31. #FullStackCon 2019 31 http://purl.org/aframe-webthing
Lets build the Web of Twins ?
●
Digital twins
– Introduced by Dr M. Grieves
(2002)
●
Real time (or deferred) connectivity:
– Between the physical component
●
and its digital counterpart
●
“Devices as service” concept:
●
Applies to many industries:
– City, manufacturing, health,
transport…
●
Useful for:
– Re/Co/Design
– Monitoring, Quality tracking
– Impact analysis:
●
Dependency, process,
lifecycle, financial...
– Digital traces for analytic
– Simulation, AI/ML etc
– Improve decision making
33. #FullStackCon 2019 33 http://purl.org/aframe-webthing
Community rules @ #FullStackCon
34. #FullStackCon 2019 34 http://purl.org/aframe-webthing
Summary
●
3D in browser has never been so easy
– Using A-frame framework supported by browsers and devices
●
Ready for JavaScript developers
– Develop for low end microcontrollers using IoT.js
– And support WebThing API to be interacted from the web
●
Direct or connected to gateway (shared securely to apps)
●
Let’s develop the Web of Twins:
– built on FLOSS tools with privacy by design
35. #FullStackCon 2019 35 http://purl.org/aframe-webthing
Summary
1) AR and VR on the web ✓
36. #FullStackCon 2019 36 http://purl.org/aframe-webthing
Summary
1) AR and VR on the web ✓
2) WebThings connectivity ✓
37. #FullStackCon 2019 37 http://purl.org/aframe-webthing
Summary
1) AR and VR on the web ✓
2) WebThings connectivity ✓
3) Binding WebThings and XR ✓