A presentation from SEAMS 2017 on formalization of hybrid planning. Lead author: https://www.cs.cmu.edu/~ashutosp/ Full paper: http://www.cs.cmu.edu/~iruchkin/docs/pandey17-towards.pdf Abstract: "Decision-making approaches in self-adaptation face a fundamental trade-off between quality and timeliness of adaptation plans. Due to this trade-off, designers often have to make an offline compromise between finding adaptation plans quickly and finding closer-to-optimal plans that demand longer computation times. Recent work has proposed that hybrid planning can resolve this trade-off dynamically, achieving higher utility than either fast or slow approaches individually. The promise of hybrid planning is to combine multiple decision-making approaches at run time to produce adaptation plans of the high quality within given time constraints. However, the diversity of decision-making approaches makes the problem of hybrid planning complex and multi-faceted. This paper advances the theory of hybrid planning by formalizing the central concepts and four sub-problems of hybrid planning. This formalization can serve as a foundation for creating and evaluating hybrid planners in the future."

1. TOWARDS A FORMAL
FRAMEWORK FOR HYBRID
PLANNING IN SELF-ADAPTATION
Ashutosh Pandey Ivan Ruchkin
Bradley Schmerl Javier Cámara
SEAMS 2017 (The 12th International Symposium on Software Engineering for Adaptive and
Self-Managing Systems)Buenos Aires, Argentina · 22-23 May 2017
1

2. KEY REQUIREMENTS: QUALITY AND TIMELINESS OF DECISION-MAKING
➤ Although these systems are designed for balancing multiple
objectives to provide long term quality
➤ Amazon web services primarily cares about availability1
➤ Netflix primarily cares about response-time perceived by
clients2
1.https://aws.amazon.com/ec2/sla/
2.http://techblog.netflix.com/2010/12/5-lessons-weve-learned-using-
aws.html 2

3. CONFLICTING REQUIREMENTS: QUALITY AND TIMELINESS OF DECISION-MAKING
Desired
Region
Probabilistic Planning
Ex: MDP, POMDP
Case-based Reasoning
Ex: Rainbow framework
Heuristic Planning
Ex: FF-Replan
Reactive Planning
Ex: Condition-action rules
Decision-making Time
DecisionQuality
3
*WOLPERT, David H., and William G. MACREADY, 1995. No free lunch theorems for search.
Technical Report SFI-TR-95-02-010. Sante Fe, NM, USA:
*WOLPERT, David H., and William G. MACREADY, 1997. No free lunch theorems for optimization.
IEEE Transactions on Evolutionary Computation, 1(1), 67–82.

4. HYBRID PLANNING
4
𝛒 Fast
𝛒 slow
𝛱Fast
t
𝛱Slow
t’
Planning Process
𝛒 ➜ Planner
𝛱 ➜ Plan
Timeline
Planning
Problem

5. Slow Planner ⇒ Planner based on Markov Decision Processes (MDP)
Fast Planner ⇒ Deterministic planner
HYBRID PLANNING LOOKS PROMISING!!
5
0
1
0.5
-0.5
-1.0
-1.5
1.5
Fast Planner
Slow Planner
Hybrid Planner
Normalized Aggregate Utility
A. Pandey et. al. Hybrid planning for decision making in self- adaptive systems. International
Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016, pp. 12-16

6. THE PROBLEM OF
HYBRID PLANNING
6

7. THE PROBLEM OF HYBRID PLANNING
“Given a planning problem, and a set of planners, find
a hybrid plan that maximizes a posteriori utility.”
7

8. THE PROBLEM OF HYBRID PLANNING
“Given a planning problem, and a set of planners, find
a hybrid plan that maximizes a posteriori utility.”
8

9. “Given a planning problem, and a set of planners, find
a hybrid plan that maximizes the a posteriori utility.”
A PLANNING PROBLEM
9
S, si,
A, o, T, Ue
Set of states
Initial state Transition function T : S x A x Z → S
Environment o : S → Z
A posteriori utility function
Set of controllable
actions
Set of
uncontrollable
actions

10. A POSTERIORI UTILITY
10
High
Load
Add-Server
H
igh-Load
Low
Load
Medium
Load
Low
-Load
Add-Server
Low-Load
High-Load
Add-Server
=
High
Load
Medium
Load
High
Load
Low
Load
U
U’
A Priori A Posteriori

11. 11
THE PROBLEM OF HYBRID PLANNING
“Given a planning problem, and a set of planners, find
a hybrid plan that maximizes a posteriori utility.”

12. 12
REACHABILITY GRAPH
Pb
U1
E1
Pb2 Pl2
Pb Pl1
Pb Planning problem
Pl Planner
E Execution
U A posteriori utility
d Deadline to trigger a planner
Problem-Planner node
Execution edge

13. 13
REACHABILITY GRAPH
Pb Pl1Pb
U1
E1
Pb2 Pl2
E1
U1
Pb2 Pl3
Pb Planning problem
Pl Planner
E Execution
U A posteriori utility
d Deadline to trigger a planner
Problem-Planner node
Execution edge

14. 14
REACHABILITY GRAPH
Pb Pl1Pb
U1
E1
Pb2 Pl2
E1
U1
Pb2 Pl3
E2
U2
Pb3 Pl4
E2
U2
Pb3 Pl5
Pb Planning problem
Pl Planner
E Execution
U A posteriori utility
d Deadline to trigger a planner
Problem-Planner node
Execution edge

15. 15
REACHABILITY GRAPH
Pb Pl1Pb
U1
E1
Pb2 Pl2
E1
U1
Pb2 Pl3
E2
U2
Pb3 Pl4
E2
U2
Pb3 Pl5
E3
U3
En
Un
Pbi Plj
Pb Planning problem
Pl Planner
E Execution
U A posteriori utility
d Deadline to trigger a planner
Problem-Planner node
Execution edge

16. 16
REACHABILITY GRAPH
Pb Pl1Pb
U1
E1
Pb2 Pl2
E1
U1
Pb2 Pl3
E2
U2
Pb3 Pl4
E2
U2
Pb3 Pl5
E3
U3
En
Un
Pbi Plj
Pb Planning problem
Pl Planner
E Execution
U A posteriori utility
d Deadline to trigger a planner
Problem-Planner node
Execution edge

17. 17
REACHABILITY GRAPH WHEN PLANNING TIME IS NOT ZERO
Pb Pl1
dl1
Pb
U1
E1
Pb2 Pl2
dl2
E1
U1
Pb2 Pl3
dl3
E2
U2 Pb3 Pl4
dl4E2
U2
Pb3 Pl5
dl5
E3
U3
En
Un
Pbi Plj
dlk
Pb Planning problem
Pl Planner
E Execution
U A posteriori utility
dl Deadline to trigger a planner
Problem-Planner node
Execution edge

18. 18
SOLVING THE HYBRID PLANNING PROBLEM
Pb Pl1
dl1
Pb
U1
E1
Pb2 Pl2
dl2
E1
U1
Pb2 Pl3
dl3
E2
U2 Pb3 Pl4
dl4E2
U2
Pb3 Pl5
dl5
E3
U3
En
Un
Pbi Plj
dlk
Pb Planning problem
Pl Planner
E Execution
U A posteriori utility
dl Deadline to trigger a planner
Problem-Planner node
Execution edge

19. 19
DECOMPOSITION OF THE PROBLEM OF HYBRID PLANNING
Assessment of
planners
w.r.t.to problems
Set of
planning
problems
Planning
problem
Sequence of
nodes from the
reachability graph
Reachability
Graph
Set of
planning
problems
Set of planners
and
Problem-Planner
Compatibility
Relationship
GPHCON
PTHSELPRBSEL
PLRAST

20. 20
SUB-PROBLEM: THE PATH SELECTION (PTHSEL)
Assessment of
planners
w.r.t.to problems
Set of
planning
problems
Planning
problem
Sequence of
nodes from the
reachability graph
Reachability
Graph
Set of
planning
problems
Set of planners
and
Problem-Planner
Compatibility
Relationship
GPHCON
PTHSELPRBSEL
PLRAST

21. 21
SUB-PROBLEM: THE GRAPH CONSTRUCTION (GPHCON)
Assessment of
planners
w.r.t.to problems
Set of
planning
problems
Planning
problem
Sequence of
nodes from the
reachability graph
Reachability
Graph
Set of
planning
problems
Set of planners
and
Problem-Planner
Compatibility
Relationship
GPHCON
PTHSELPRBSEL
PLRAST

22. 22
SUB-PROBLEM: THE PLANNER ASSESSMENT (PLRAST)
Assessment of
planners
w.r.t.to problems
Set of
planning
problems
Planning
problem
Sequence of
nodes from the
reachability graph
Reachability
Graph
Set of
planning
problems
Set of planners
and
Problem-Planner
Compatibility
Relationship
GPHCON
PTHSELPRBSEL
PLRAST

23. 23
SUB-PROBLEM: THE PLANNING-PROBLEM SELECTION (PRBSEL)
Assessment of
planners
w.r.t.to problems
Set of
planning
problems
Planning
problem
Sequence of
nodes from the
reachability graph
Reachability
Graph
Set of
planning
problems
Set of planners
and
Problem-Planner
Compatibility
Relationship
GPHCON
PTHSELPRBSEL
PLRAST

24. 24
ASSUMPTIONS FOR THE FORMAL FRAMEWORK
➤ No instantaneous actions
➤ Markovian domain
➤ Instantaneous solution to sub-problems
➤ Known worst-case planning time
➤ Known utility of executions

25. CONCLUSION
➤ Hybrid Planning approach is useful but a non-trivial problem
to solve
➤ This formalism could help in understanding and
approximating a more general solution to the problem
25

26. ACKNOWLEDGEMENT
•This work is supported in part by awards
•N000141310401 and N000141310171 from the Office of
Naval Research (ONR)
•FA87501620042 from the Air Force Research Laboratory
(AFRL)
26

27. 27