Nasdanika WebTest - Modular functional testing of Web and Mobile Applications
Using Real-Time Scheduling Principles in Web Service Clusters to Achieve Predictability of Execution
1. Using Real-Time Scheduling Principles in Web
Service Clusters to Achieve Predictability of
Service Execution
Vidura Gamini Abhaya
Prof. Zahir Tari and Assoc. Prof. Peter Bertok
Distributed Systems and Networking Group
School of Computer Science and IT
RMIT University
Melbourne, Australia
December 8, 2010
2. Intro. Prev. Work Solution Results Conclusion
Presentation Structure
1 Introduction
2 Previous Work
3 Solution
4 Empirical Results
5 Conclusion
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
3. Intro. Prev. Work Solution Results Conclusion Motivation
Motivation
Web Services enable communication between Distributed
Heterogeneous Systems.
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
4. Intro. Prev. Work Solution Results Conclusion Motivation
Motivation
Web Services enable communication between Distributed
Heterogeneous Systems. It is the foundation for...
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
5. Intro. Prev. Work Solution Results Conclusion Motivation
Motivation
Web Services enable communication between Distributed
Heterogeneous Systems. It is the foundation for...
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
6. Intro. Prev. Work Solution Results Conclusion Motivation
Motivation
Web Services enable communication between Distributed
Heterogeneous Systems. It is the foundation for...
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
7. Intro. Prev. Work Solution Results Conclusion Motivation
Motivation
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
8. Intro. Prev. Work Solution Results Conclusion Motivation
Motivation
Scalability Issues...
Quality of Service ?
Performance ?
Predictability ?
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
9. Intro. Prev. Work Solution Results Conclusion Single Host
1 Introduction
2 Previous Work
3 Solution
4 Empirical Results
5 Conclusion
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
10. Intro. Prev. Work Solution Results Conclusion Single Host
Our Previous Work [5,6,*]...
Achievement
Achieve Predictability and Service Differentiation in WS execution
(in a single server)
* Building Web Service Middleware with Predictable Service Execution, WISE 2010, LNCS, vol. 6488 pp. 23-37.
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
11. Intro. Prev. Work Solution Results Conclusion Single Host
Our Previous Work [5,6,*]...
Achievement
Achieve Predictability and Service Differentiation in WS execution
(in a single server)
Highlights...
Introduced the notion of a deadline
Use Real-time scheduling principles
Select tasks with achievable deadlines, reject the rest
Selected tasks scheduled in the order of increasing deadlines
Implementation based on Apache Axis2
Tested for various request size, arrival rate combinations
* Building Web Service Middleware with Predictable Service Execution, WISE 2010, LNCS, vol. 6488 pp. 23-37.
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
12. Intro. Prev. Work Solution Results Conclusion Single Host
Task Execution with a Deadline
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
13. Intro. Prev. Work Solution Results Conclusion Single Host
Task Execution with a Deadline
Laxity of a Task
Deadline
Laxity =
ExecutionTime
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
14. Intro. Prev. Work Solution Results Conclusion Single Host
Single Host Implementation
Task selection based on a ‘Schedulability Check’
Deadline based scheduling
Implementation on a single server
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
15. Intro. Prev. Work Solution Results Conclusion Single Host
Deadline based Task Schedule
EDF based task schedule
Larger laxities allow requests to be executed in staggered
manner
Enable timely completion of requests with earlier deadlines
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
16. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
1 Introduction
2 Previous Work
3 Solution
4 Empirical Results
5 Conclusion
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
17. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
Solution Overview
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
18. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
Solution Overview
Sun Real-time Java Specification used as
development platform
Sun Solaris 10 08/05 used as RT-OS
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
19. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
Solution Overview
Sun Real-time Java Specification used as
development platform
Sun Solaris 10 08/05 used as RT-OS
Apache Synapse based dispatcher implementation
Apache Axis2 based executor implementation
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
20. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
Solution Overview
Sun Real-time Java Specification used as
development platform
Sun Solaris 10 08/05 used as RT-OS
Apache Synapse based dispatcher implementation
Apache Axis2 based executor implementation
All Thread-pools replaced with custom-designed
Real-time Thread pools
Introduction of Real-time scheduler component
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
21. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
Cluster Design
Request processing at the dispatcher
Requests are dispatched based on algorithm used
Requests are ‘mostly’ queued at dispatcher
Results are sent back to clients via dispatcher
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
22. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
Dispatching Algorithms
Objectives
*Ensures deadline requirement of tasks could be met
*Distribute requests evenly among executors based on a condition
*Avoids executors going into overload conditions
4 algorithms used
RT-RoundRobin
RT-Sequential
RT-ClassBased
RT-LaxityBased
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
23. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
RT-RoundRobin
Details
Requests assignment cycles
through all executors in RR
fashion
Sched. check is done only once
per request
If sched. check fails on selected
server → request rejected
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
24. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
RT-RoundRobin
Details
Requests assignment cycles
through all executors in RR
fashion
Sched. check is done only once
per request
If sched. check fails on selected
server → request rejected
Hightlights
POC for how a simple dispatching algorithm could be made real-time ready
Processing overhead is minimum
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
25. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
RT-Sequential
Details
Requests are assigned in a
sequential manner
Req’s sent to one executor till
sched. check fails, then assigned
to the second and so on
Sched. check happens against
multiple executors till a request
can be assigned
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
26. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
RT-Sequential
Details
Requests are assigned in a
sequential manner
Req’s sent to one executor till
sched. check fails, then assigned
to the second and so on
Sched. check happens against
multiple executors till a request
can be assigned
Highlights
If it’s possible to schedule a job within the cluster, it will be guaranteed
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
27. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
RT-ClassBased
Details
Requests are divided into classes
based on a priority scheme
Mapping of requests to executors
is pre-determined and defined
offline
Sched. check is only considered
with the assigned executor
Reference implementation uses
priorities based on task sizes
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
28. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
RT-ClassBased
Details
Requests are divided into classes
based on a priority scheme
Mapping of requests to executors
is pre-determined and defined
offline
Sched. check is only considered
with the assigned executor
Reference implementation uses
priorities based on task sizes
Highlights
Results in the reduction of variance of task sizes at each executor
Pre-defined mapping of request sizes to executors could be done using
pre-profiled execution times or execution time history
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
29. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
RT-LaxityBased
Details
Laxity = Ability of delaying a
request whilst still meeting its
deadline
Higher the laxity the more
requests an executor could service
Sched. check is done only against
the assigned executor
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
30. Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details
RT-LaxityBased
Details
Laxity = Ability of delaying a
request whilst still meeting its
deadline
Higher the laxity the more
requests an executor could service
Sched. check is done only against
the assigned executor
Highlights
Distribution of requests results in a broad range of laxities at each executor
Keeps track of the last two laxities assigned and prevents them being assigned
for the same executor consecutively
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
31. Intro. Prev. Work Solution Results Conclusion RT-RR RT-CB Others
1 Introduction
2 Previous Work
3 Solution
4 Empirical Results
5 Conclusion
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
32. Intro. Prev. Work Solution Results Conclusion RT-RR RT-CB Others
RT-RoundRobin vs Round-Robin
Comparison of Round-robin vs. RT-RoundRobin - Deadline Achievement Rates
100
RR - 2 Execs.
RR - 3 Execs.
90 RR - 4 Execs.
RT-RR - 2 Execs.
RT-RR - 3 Execs.
80 RT-RR - 4 Execs.
Percentage of deadlines met
70
60
50
40
30
20
10
0.05s 0.1s 0.1s 0.25s 0.1s - 0.5s 0.25s - 1s
Inter-arrival times
RT-RR accepts between 20.5% (2 Exec. - fastest) and 99.9% (4 Exec. -
slowest) of the requests.
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
36. Intro. Prev. Work Solution Results Conclusion RT-RR RT-CB Others
RT-Sequential and RT-Laxity
Comparison Deadline Achievement Rates - Averaged by Algorithm
100
90
80
Averaged percentage of deadlines met
70
60
50
40
30
20
RT-RR
10 RT-CB
RT-Seq
RT-Lax
0.05s 0.1s 0.1s 0.25s 0.1s - 0.5s 0.25s - 1s
Inter-arrival times
Highest acceptance rate for any algorithm for 2 executors with highest arrival
rate is 38.5% for RT-LaxityBased
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
37. Intro. Prev. Work Solution Results Conclusion RT-RR RT-CB Others
RT-Sequential and RT-Laxity
CPU utilisation at each Executor - (1 - 5000000 (Uniform); 0.25sec - 1sec (Uniform) laxitybased 3 exec)
60
Executor 1
Executor 2
Executor 3
50
40
CPU Utilization %
30
20
Comparison Deadline Achievement Rates - Averaged by Algorithm
100
90
80
10
Averaged percentage of deadlines met
70
60
50
40
30
20
0
10
RT-RR
RT-CB
RT-Seq
0 200 400 600 800 1000 1200
RT-Lax
0.05s 0.1s 0.1s 0.25s
Inter-arrival times
0.1s - 0.5s 0.25s - 1s
Sample #
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
38. Intro. Prev. Work Solution Results Conclusion Future Work Recap
1 Introduction
2 Previous Work
3 Solution
4 Empirical Results
5 Conclusion
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
39. Intro. Prev. Work Solution Results Conclusion Future Work Recap
Future Work
Further analysis of cluster behaviour with mix of tasks
Advanced dispatching and re-dispatching models
Predictability in the networking layer
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
40. Intro. Prev. Work Solution Results Conclusion Future Work Recap
Conclusion
Previous work - We achieved predictability of execution on a
single server
We extend it to achieve predicability in a cluster of servers
Presented four task dispatching algorithms
Evaluated their performance empirically
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings
41. Intro. Prev. Work Solution Results Conclusion Future Work Recap
Thank You !
Questions or Comments ?
V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings