1. Background & Motivation Handling Overload Evaluation Summary & Future Work 1 of 22 slides
Highly Available Publish/Subscribe
DEBS 2006
Zbigniew Jerzak, Christof Fetzer
Dresden University of Technology
Monday 3rd July, 2006
{Zbigniew.Jerzak, Christof.Fetzer}@tu-dresden.de
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
2. Background & Motivation Handling Overload Evaluation Summary & Future Work 2 of 22 slides
Motivation
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
3. Background & Motivation Handling Overload Evaluation Summary & Future Work 2 of 22 slides
Motivation
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
4. Background & Motivation Handling Overload Evaluation Summary & Future Work 2 of 22 slides
Motivation
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
5. Background & Motivation Handling Overload Evaluation Summary & Future Work 2 of 22 slides
Motivation
Priorities?
Congestion?
Self-optimization?
Flexibility?
Fairness?
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
6. Background & Motivation Handling Overload Evaluation Summary & Future Work 3 of 22 slides
Transient Congestion Effect
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
7. Background & Motivation Handling Overload Evaluation Summary & Future Work 4 of 22 slides
Vicious Cycle
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
8. Background & Motivation Handling Overload Evaluation Summary & Future Work 4 of 22 slides
Vicious Cycle
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
9. Background & Motivation Handling Overload Evaluation Summary & Future Work 4 of 22 slides
Vicious Cycle
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
10. Background & Motivation Handling Overload Evaluation Summary & Future Work 4 of 22 slides
Vicious Cycle
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
11. Background & Motivation Handling Overload Evaluation Summary & Future Work 5 of 22 slides
Pub/Sub Invariants [EFGK03]
Space Decoupling
Time Decoupling
Synchronisation Decoupling
Acyclic/Cyclic graph network topology
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
12. Background & Motivation Handling Overload Evaluation Summary & Future Work 6 of 22 slides
Solution Sketch
Priorities — value based approach
Congestion — load shedding (spanning tree)
Self-optimization — dynamically adapt to load
Flexibility — no fixed classes/priorities
Fairness — approach based on market economy
No violation of decoupling properties
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
13. Background & Motivation Handling Overload Evaluation Summary & Future Work 6 of 22 slides
Solution Sketch
Priorities — value based approach
Congestion — load shedding (spanning tree)
Self-optimization — dynamically adapt to load
Flexibility — no fixed classes/priorities
Fairness — approach based on market economy
No violation of decoupling properties
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
14. Background & Motivation Handling Overload Evaluation Summary & Future Work 7 of 22 slides
Subscription Routing
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
15. Background & Motivation Handling Overload Evaluation Summary & Future Work 7 of 22 slides
Subscription Routing
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
16. Background & Motivation Handling Overload Evaluation Summary & Future Work 7 of 22 slides
Subscription Routing
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
17. Background & Motivation Handling Overload Evaluation Summary & Future Work 8 of 22 slides
Subscription Routing summary
Each subscription piggybacks client assigned value (Priorities,
Flexibility)
Values are aggregated along with subscriptions
Small memory footprint and minimal bandwidth overhead
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
18. Background & Motivation Handling Overload Evaluation Summary & Future Work 9 of 22 slides
Publication Forwarding — No Congestion/Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
19. Background & Motivation Handling Overload Evaluation Summary & Future Work 10 of 22 slides
Publication Forwarding — Link Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
20. Background & Motivation Handling Overload Evaluation Summary & Future Work 10 of 22 slides
Publication Forwarding — Link Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
21. Background & Motivation Handling Overload Evaluation Summary & Future Work 10 of 22 slides
Publication Forwarding — Link Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
22. Background & Motivation Handling Overload Evaluation Summary & Future Work 10 of 22 slides
Publication Forwarding — Link Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
23. Background & Motivation Handling Overload Evaluation Summary & Future Work 11 of 22 slides
Publication Forwarding — Router Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
24. Background & Motivation Handling Overload Evaluation Summary & Future Work 11 of 22 slides
Publication Forwarding — Router Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
25. Background & Motivation Handling Overload Evaluation Summary & Future Work 11 of 22 slides
Publication Forwarding — Router Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
26. Background & Motivation Handling Overload Evaluation Summary & Future Work 11 of 22 slides
Publication Forwarding — Router Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
27. Background & Motivation Handling Overload Evaluation Summary & Future Work 12 of 22 slides
Publication Forwarding Summary
Kill two birds with one stone (Congestion):
router overload
link overload
Re-integrate shedded links when load decreases (Flexibility)
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
28. Background & Motivation Handling Overload Evaluation Summary & Future Work 13 of 22 slides
Why Coverage?
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
29. Background & Motivation Handling Overload Evaluation Summary & Future Work 13 of 22 slides
Why Coverage?
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
30. Background & Motivation Handling Overload Evaluation Summary & Future Work 13 of 22 slides
Why Coverage?
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
31. Background & Motivation Handling Overload Evaluation Summary & Future Work 13 of 22 slides
Why Coverage?
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
32. Background & Motivation Handling Overload Evaluation Summary & Future Work 13 of 22 slides
Why Coverage?
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
33. Background & Motivation Handling Overload Evaluation Summary & Future Work 14 of 22 slides
Coverage Summary
Shed only as much as needed and only when needed
(Self-optimization)
Charge for message delivery ⇒ prevent selfish behaviour
(Fairness)
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
34. Background & Motivation Handling Overload Evaluation Summary & Future Work 15 of 22 slides
Evaluation Environment
OMNeT++
20 node network – acyclic graph topology
Each run – 2 · 106 messages
Number of publications > subscriptions ⇒ system stabilises
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
35. Background & Motivation Handling Overload Evaluation Summary & Future Work 16 of 22 slides
Figure 1: Link marking and resulting matching times in the HAPS router
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
36. Background & Motivation Handling Overload Evaluation Summary & Future Work 17 of 22 slides
Figure 2: Router revenue for different interface marking algorithms
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
37. Background & Motivation Handling Overload Evaluation Summary & Future Work 18 of 22 slides
Figure 3: Backup link activation and resulting transmission times using the message
shedding
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
38. Background & Motivation Handling Overload Evaluation Summary & Future Work 19 of 22 slides
Summary
Decoupled approach
Symmetrical algorithms
Fine grained approach
Self-optimizing strategy
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
39. Background & Motivation Handling Overload Evaluation Summary & Future Work 20 of 22 slides
Current & Future Work
Formalise the market approach
Integration with FADS [FC99]
upper bound on message transmission delay
additional parameter for overload manager
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
40. Background & Motivation Handling Overload Evaluation Summary & Future Work 21 of 22 slides
Thank You for Your Attention
This presentation and other related resources:
http://wwwse.inf.tu-dresden.de/haps
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
41. Background & Motivation Handling Overload Evaluation Summary & Future Work 22 of 22 slides
References
Antonio Carzaniga, Matthew J. Rutherford, and Alexander L. Wolf.
A routing scheme for content-based networking.
In Proceedings of IEEE INFOCOM 2004, Hong Kong, China, March 2004.
Antonio Carzaniga and Alexander L. Wolf.
Forwarding in a content-based network.
In Proceedings of ACM SIGCOMM 2003, pages 163–174, Karlsruhe, Germany,
August 2003.
Patrick Th. Eugster, Pascal A. Felber, Rachid Guerraoui, and Anne-Marie
Kermarrec.
The many faces of publish/subscribe.
ACM Comput. Surv., 35(2):114–131, 2003.
Christof Fetzer and Flaviu Christian.
A fail-aware datagram service.
In Iain Bate and Alan Burns, editors, IEE Proceedings - Software Engineering,
volume 146, pages 58–74. IEE, April 1999.
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer