Nous savons que l’animation est essentielle pour l'interaction avec nos utilisateurs. Mais aujourd'hui l'interactivité ne peut plus se résumer à l’UX. Twitter, Waze, Slack … toutes les applications à succès utilisent le temps réel pour rendre leurs utilisateurs addicts. Mais comment faire pour transformer votre API statique en un flux de données temps-réel ? Du pull ? Du push ? Des Webhooks ? Vous connaissez sûrement les Web-Sockets, mais avez vous déjà entendu parler de Server-Sent Events ? Dans ce talk nous comparerons ces technologies pour comprendre dans quel cas elles s'appliquent le mieux avant de voir comment on peut aller encore plus loin en réduisant la quantité de données échangées. Et parce qu'il n'y a pas que le web qui a besoin de temps réel (et parce que c'est bien plus fun), je vous montrerai comment faire danser des drones sur des APIs streamées.

1. #DevoxxFR @Audrey_Neveu
The End of Polling
Why and how to transform a REST API
into a Data Streaming API

2. #DevoxxFR @Audrey_Neveu
ABOUT ME
Developer Relations Co-Leader France
@Audrey_Neveu

3. #DevoxxFR @Audrey_Neveu
MODERN TIMES

4. #DevoxxFR @Audrey_Neveu
OUR GOAL

5. #DevoxxFR @Audrey_Neveu
ANIMATION IS THE KEY... BUT WHY?

6. #DevoxxFR @Audrey_Neveu
BECAUSE OF EVOLUTION

7. #DevoxxFR @Audrey_Neveu
BECAUSE OF EVOLUTION

8. #DevoxxFR @Audrey_Neveu
USER INTERFACE

9. #DevoxxFR @Audrey_Neveu
KEEP CALM ...

10. #DevoxxFR @Audrey_Neveu
KEEP CALM ...

11. #DevoxxFR @Audrey_Neveu
KEEP CALM ... AND LEAVE.

12. #DevoxxFR @Audrey_Neveu
REFRESH BUTTON IS EVIL

13. #DevoxxFR @Audrey_Neveu
REAL-TIME USER EXPERIENCE

14. #DevoxxFR @Audrey_Neveu
F.O.M.O (FEAR OF MISSING OUT)

15. #DevoxxFR @Audrey_Neveu
API STREAMING

16. #DevoxxFR @Audrey_Neveu
SOLUTIONS FOR REAL-TIME APPLICATIONS
✓ WebHooks
✓ (Long) Polling
✓ Web-Sockets
✓ Pub/Sub
✓ Server-Sent Events

17. #DevoxxFR @Audrey_Neveu
IT’S ALL ABOUT FREQUENCY ...

18. #DevoxxFR @Audrey_Neveu
… AND USAGE

19. #DevoxxFR @Audrey_Neveu
(Long) Polling

20. #DevoxxFR @Audrey_Neveu
POLLING

21. #DevoxxFR @Audrey_Neveu
98.5% of polls are
wasted
Source:
http://resthooks.org/
POLLING MADNESS ™

22. #DevoxxFR @Audrey_Neveu
98.5% of polls are
wasted
Source:
http://resthooks.org/
POLLING IS NOT A SOLUTIONMADNESS ™
(and neither is long polling)

23. #DevoxxFR @Audrey_Neveu
WebHooks

24. #DevoxxFR @Audrey_Neveu
✓ webhooks.org
✓ aka User Defined HTTP Callback
✓ concept / method
WEBHOOKS
✓

25. HTTP/1.1 201 Created
Link:<http://subscription-serve
r.com/subscription>;
rel="subscription"
@StreamdataIOstreamdataio
WEBHOOKS
POST /eventsource HTTP/1.1
Host: subscription-server.com
Pragma: subscribe
Callback:
<http://example.com/callback>;
method="POST"
rel="subscriber"
Consumer API Provider

26. HTTP/1.1 201 Created
Link:<http://subscription-serve
r.com/subscription>;
rel="subscription"
@StreamdataIOstreamdataio
WEBHOOKS
POST /eventsource HTTP/1.1
Host: subscription-server.com
Pragma: subscribe
Callback:
<http://example.com/callback>;
method="POST"
rel="subscriber"
POST /callback HTTP/1.1
Host: example.com
Link:
<http://subscription-server.com
/subscription>;
rel="subscription"
Content-HMAC: sha1
C+7Hteo/D9vJXQ3UfzxbwnXaijM=
Content-Length: 21
Content-Type:
application/x-www-form-urlencod
ed
payload=Hello%20world
Consumer API Provider

27. #DevoxxFR @Audrey_Neveu
✓ Define a callback URL
TODO LIST
Consumer API Provider
✓ Implement your webhook queue
- inline HTTP Requests
- SQL-based queue
- AMQP broker
- batch

28. #DevoxxFR @Audrey_Neveu
✓ Define a callback URL ✓ Create a subscription endpoint
- GET /webhook
- POST /webhook
- GET /webhook/{id}
- PUT /webhook/{id}
- DELETE /webhook/{id}
TODO LIST
Consumer API Provider

29. #DevoxxFR @Audrey_Neveu
CONSUMER SETUP

30. #DevoxxFR @Audrey_Neveu
EVENT RECEIVED
POST /callback HTTP/1.1
Host: www.example.com
X-Github-Delivery:
72d3162e-cc78-11e3-81ab-4c9367
dc0958
User-Agent:
GitHub-Hookshot/044aadd
Content-Type: application/json
Content-Length: 6615
X-GitHub-Event: issues
Payload=
{
"action": "opened",
"issue": {
"url": "https://api.github.com/repos/octocat/Hello-World/issues/1347",
"number": 1347,
...
},
"repository" : {
"id": 1296269,
"full_name": "octocat/Hello-World",
"owner": {
"login": "octocat",
"id": 1,
...
},
...
},
"sender": {
"login": "octocat",
"id": 1,
...
}
}

31. #DevoxxFR @Audrey_Neveu
PROS AND CONS
✓ Real-Time updates
✓ Easily consumed
✓ Without dedicated resources
■ Poor user experience
■ Does not work with all clients
■ Manual setup✓ Easily integrated
■ Debugging

32. #DevoxxFR @Audrey_Neveu
PROS AND CONS
✓ Easily consumed
✓ Without dedicated resources
■ Poor user experience
■ Does not work with all clients
■ Manual setup✓ Easily integrated
■ Debugging
✓ (almost) Real-Time updates

33. #DevoxxFR @Audrey_Neveu
KNOWN ISSUES
Consumer API Provider
■ DDoS Attack
■ Missed notification
■ DDoS Attack
■ Deduplication

34. #DevoxxFR @Audrey_Neveu
CHECK LIST
Consumer API Provider
✓ Implement authentication
✓ Expected answer?
✓ Monitor payload size
✓ Handle request number
✓ Handle duplicates
✓ One callback per webhook

35. #DevoxxFR @Audrey_Neveu
DYNAMIC SUBSCRIPTIONS
✓ Restful Webhooks
✓ REST Hooks
✓ PubSubHubbub

36. #DevoxxFR @Audrey_Neveu
PubSubHubbub
aka PubSub
aka PuSH

37. #DevoxxFR @Audrey_Neveu
RSS / Atom feeds
pubsubhubbub/
Open Protocol
Based on Publish / Subscribe Pattern …
PUBSUB
… and on topics

38. #DevoxxFR @Audrey_Neveu
SubscribersHub
PUBSUB
Publishers

39. #DevoxxFR @Audrey_Neveu
Discovery
SUBSCRIBE - PUBLISHERS
Link: <https://my-hub.com/>; rel="hub"
Link: <https://my.resource.com>; rel="self"
<link rel="self" href="https://my-resource.com/">
<link rel="hub" href="https://my-hub.com/">
Publishers

40. #DevoxxFR @Audrey_Neveu
SUBSCRIBE - SUBSCRIBERS 1/2
POST https://my-hub.com/
…
hub.mode="subscribe"
hub.topic="https://my-resource.com/"
hub.callback="http://example.com/callback"
hub.secret="my-token"
Subscribers Send subscription request to the Hub

41. #DevoxxFR @Audrey_Neveu
SUBSCRIBE - HUB
GET http://example.com/callback
…
hub.mode="subscribe"
hub.topic="https://my-resource.com/"
hub.challenge="a random string"
hub.lease_seconds=86400
Hub Verify intent of the subscribers

42. #DevoxxFR @Audrey_Neveu
SUBSCRIBE - SUBSCRIBERS 2/2
HTTP/1.1 200 OK
Body:{
hub.challenge: "a random string"
}
Subscribers Answer verification request

43. #DevoxxFR @Audrey_Neveu
CONSUMER SETUP

44. #DevoxxFR @Audrey_Neveu
PUBLISH - PUBLISHERS
POST https://my-hub.com/
…
hub.mode=publish
hub.url=https://my.updated-resource.com
Publishers Ping the Hub

45. #DevoxxFR @Audrey_Neveu
PUBLISH - HUB
POST http://example.com/callback
Link: <https://my-hub.com/>; rel="hub"
Link: <https://my.updated-resource.com>; rel="self"
Hub Content Distribution

46. #DevoxxFR @Audrey_Neveu
PUBLISH - HUB
POST http://example.com/callback
Link: <https://my-hub.com/>; rel="hub"
Link: <https://my.updated-resource.com>; rel="self"
X-Hub-Signature="my-sha1-signature"
Hub (Authenticated) Content Distribution

47. #DevoxxFR @Audrey_Neveu
PUBLISH - SUBSCRIBERS
GET https://my.updated-resource.com
Subscribers Pull the updated resource

48. #DevoxxFR @Audrey_Neveu
{
"object":"page",
"entry":[
{
"id":"51044240199134611",
"time":1447342027,
"changes":[
{
"field":"leadgen",
"value":{
"adgroup_id":0,
"ad_id":0,
"created_time":1447342026,
"leadgen_id":55459717045641545,
"page_id":516540199134611,
"form_id":551111744595541
}
}
]
}
]
}
EVENT RECEIVED

49. #DevoxxFR @Audrey_Neveu
UNSUBSCRIBE - SUBSCRIBERS 1/2
POST https://my-hub.com/
…
hub.mode="unsubscribe"
hub.topic="https://my-resource.com/"
hub.callback="http://example.com/callback"
Subscribers Send unsubscription request to the Hub

50. #DevoxxFR @Audrey_Neveu
UNSUBSCRIBE - HUB
GET http://example.com/callback
…
hub.mode="unsubscribe"
hub.topic="https://my-resource.com/"
hub.challenge="a random string"
Hub Verify intent of the subscribers

51. #DevoxxFR @Audrey_Neveu
UNSUBSCRIBE - SUBSCRIBERS 2/2
HTTP/1.1 200 OK
Body:{
hub.challenge: "a random string"
}
Subscribers Answer verification request

52. #DevoxxFR @Audrey_Neveu
WEBHOOKS PROS AND CONS
✓ (almost) Real-Time updates
✓ Easily consumed
✓ Without dedicated resources
■ Poor user experience
■ Does not work with all clients
■ Manual setup✓ Easily integrated
■ Debugging

53. #DevoxxFR @Audrey_Neveu
WEBHOOKS PROS AND CONS
✓ (almost) Real-Time updates
✓ Easily consumed
✓ Without dedicated resources
■ Does not work with all clients
✓ Easily integrated
■ Debugging
■ Need another call to get data

54. #DevoxxFR @Audrey_Neveu
KNOWN ISSUES
Consumer API Provider
■ DDoS Attack
■ Missed notification
■ DDoS Attack
■ Deduplication

55. #DevoxxFR @Audrey_Neveu
WEBHOOK VS PUBSUB

56. #DevoxxFR @Audrey_Neveu
Push Technologies

57. #DevoxxFR @Audrey_Neveu
PUSH TECHNOLOGIES
Server-Sent Events
2008 2006
W3C Specification
Web-Sockets

58. #DevoxxFR @Audrey_Neveu
PUSH TECHNOLOGIES
Text + Binary Text
Server-Sent EventsWeb-Sockets

59. #DevoxxFR @Audrey_Neveu
PROTOCOLS
GET /chat HTTP/1.1
Host: example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Protocol: chat
Sec-WebSocket-Version: 13
GET /stream HTTP/1.1 1
Host: example.com
Accept: text/event-stream
Web-Sockets
(RFC-6455)
Server-Sent Events
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=
Sec-WebSocket-Protocol: chat
Sec-WebSocket-Version: 13

60. #DevoxxFR @Audrey_Neveu
CONFIGURATION
Server-Sent EventsWeb-Sockets

61. #DevoxxFR @Audrey_Neveu
MESSAGES FORMAT
var msg = {
type: "message",
text: "Hello Devoxx
France!",
id: 12345,
date: Date.now()
};
data: Hello Devoxx France!
data: Hello Devoxx France!
data: with two lines
data: {"time": "16:34:36", "text": "Hello Devoxx
France!"}
id: 12345
event: foo
data: Hello Devoxx France!
retry: 10000
Server-Sent EventsWeb-Sockets

62. #DevoxxFR @Audrey_Neveu
IMPLEMENTATION
var websocket =
new WebSocket
('ws://websocketserver/echo');
var eventSource =
new EventSource
('http://sseserver/echo');
websocket.onOpen = function(){
...
};
eventSource.onopen = function(){
...
};
websocket.onMessage = function(e){
var data = e.data;
var message = data.msg;
...
};
eventSource.onMessage = function(e){
var message = e.data;
...
};
websocket.onError = function(e){
...
};
eventSource.onError = function(e){
...
};
Server-Sent EventsWeb-Sockets

63. #DevoxxFR @Audrey_Neveu
IMPLEMENTATION
var websocket =
new WebSocket
('ws://websocketserver/echo');
var eventSource =
new EventSource
('http://sseserver/echo');
websocket.onOpen = function(){
...
};
eventSource.onopen = function(){
...
};
websocket.onMessage = function(e){
var data = e.data;
var message = data.msg;
...
};
eventSource.onMessage = function(e){
var message = e.data;
...
};
websocket.onError = function(e){
...
};
eventSource.onError = function(e){
...
};
...
eventSource
.addEventListener('foo',function(e){
// do something
}, false);
eventSource
.addEventListener('bar',function(e){
// do something else
}, false);
Server-Sent EventsWeb-Sockets

64. #DevoxxFR @Audrey_Neveu
LOST IN CONNECTION
Server-Sent EventsWeb-Sockets

65. #DevoxxFR @Audrey_Neveu
BROWSER SUPPORT
57. 52. 10. 44. 11. 14. 57. 52. 10. 44. 11. UC.
source : http://caniuse.com/
Server-Sent EventsWeb-Sockets

66. #DevoxxFR @Audrey_Neveu
BROWSER SUPPORT
57. 52. 10. 44. 11. 14. 57. 52. 10. 44. 11. UC.
source : http://caniuse.com/
Server-Sent EventsWeb-Sockets

67. #DevoxxFR @Audrey_Neveu
BROWSER SUPPORT
57. 52. 10. 44. 11. 14. 57. 52. 10. 44. 11. UC.
source : http://caniuse.com/
Server-Sent EventsWeb-Sockets

68. #DevoxxFR @Audrey_Neveu
MOBILE BROWSER SUPPORT
source : http://caniuse.com/
57. 51. 10.3. all. 56. 57. 51. 10.3. all. 56.
Server-Sent EventsWeb-Sockets

69. #DevoxxFR @Audrey_Neveu
PERFORMANCES
8s 5s x1.6
8s 6s x1.3
16s 7s x.2.2
source: http://matthiasnehlsen.com/blog/2013/05/01/server-sent-events-vs-websockets/
SSEWeb-Sockets Diff

70. #DevoxxFR @Audrey_Neveu
CHOOSE WISELY

71. #DevoxxFR @Audrey_Neveu
Proxy-as-a-Service
✓ works with any JSON API
✓ streaming based on Server-Sent Events
✓ dynamic cache
✓ incremental updates
STREAMDATA.IO

72. #DevoxxFR @Audrey_Neveu
JSON-PATCH (RFC-6902)
[{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"},
{"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"},
{"title":"Value 2","price":85,"param1":"1","param2":"22","param3":"33"},
{"title":"Value 3","price":21,"param1":"31","param2":"12","param3":"4"},
{"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"},
{"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"},
{"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}]
[{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"},
{"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"},
{"title":"Value 2","price":5,"param1":"1","param2":"22","param3":"33"},
{"title":"Value 3","price":21,"param1":"31","param2":"32","param3":"4"},
{"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"},
{"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"},
{"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}]
[{"op":"replace","path":"/2/price","value":5},
{"op":"replace","path":"/3/param2","value":"32"}]

73. #DevoxxFR @Audrey_Neveu
DEMO

74. #DevoxxFR @Audrey_Neveu
VOTE SERVER-SENT EVENTS!

75. #DevoxxFR @Audrey_Neveu
THANKS!
Q&A