1. James McGalliard, FEDSIM
CMG Southern Region
Raleigh - April 11, 2014
Richmond – April 17, 2014
1

2. Agenda
 Background
 Why We Model
 Multiple Objective Dynamic Prioritization
 Game Theory
 Comparison of Dynamic Prioritization and
Game Theory Methods
 Conclusions
2

3. Background
 Current generation High Performance
Computers are typically clusters of commodity
microprocessors that can execute multiple jobs of
assorted sizes (number of processors, run time)
simultaneously
 There are many workload scheduling alternatives
 2013 Dynamic Prioritization CMG presentation
& paper focused on the MapReduce framework
3

4. Background, cont’d.
 My coauthor has proposed an extension of the
2013 results using game theory
 Game theory-based workload scheduling has
been studied extensively
4

5. Some Terminology
 Multiple Objective
 Dynamic Prioritization
 Game Theory
 Agent
 Strategy
 Nash Equilibrium
 Price of Anarchy
5

6. Why We Model
 Represent a subset of the attributes of some
phenomenon of interest…
 Using a set of symbols that convey meaning,
such as significant elements of a system’s
structure and dynamics
 To gain insight by focusing on that subset
 To test a hypothesis
 To validate experience, live test results, etc.
6

7. Why We Model, cont’d.
 Choice of attributes & symbols impacts what
is seen
 Analytical modeling using queueing theory
has historically dominated computer
performance evaluation modeling at CMG
 Queuing models are computationally easy
but forces assumptions that may not be
realistic
7

8. Why We Model, cont’d.
 FEDSIM historically favored simulation over
analytical modeling
 Simulation is more computationally
demanding but needs fewer constraining
assumptions
 Is a more general purpose tool
 Can have its own issues, such as spin up
 Computation is cheaper than it used to be
8

9. Why We Model, cont’d.
 Game theory and multiple objective
dynamic optimization can both be studied
using simulation, but with different
attributes, symbols, and assumptions, e.g.,
single agent vs. multiple agents
9

10. Multiple Objective Dynamic
Prioritization
 Presented in 2013 at Raleigh and Richmond and at the
annual national conference in La Jolla
 Simulation of scheduling alternatives with a defined
objective function across the known workload
 Improved performance compared to the default FCFS
workload scheduler
 Multiple objectives evaluated from the perspective of
the central scheduler/system administrator
10

11. Multiple Objective Dynamic
Prioritization, cont’d.
 These objectives could include sys admin’s – e.g.,
maximize hardware utilization…
 Or users’ – e.g., minimize turnaround time; expansion
factor…
 Or any objective that can be calculated
 A single agent - the central scheduler - but multiple
perspectives
11

12. Multiple Objective Dynamic
Prioritization, cont’d.
 Assumed fractional knapsack allocation
 Workload scheduling considerations included:
 Wait Time
 Run Time
 Number of CPUs
 Queue
 Composite priorities
 Dynamic priorities
12

13. Multiple Objective Dynamic
Prioritization, cont’d.
 Workload scheduling considerations included:
 Resource awareness
 Phase Based
 Delay Timing
 Pre-emption & Interruption
 Social Scheduling
 Variable Budget Scheduling
 Complex workload structures (e.g., copy/compute)
13

14. Multiple Objective Dynamic
Prioritization, cont’d.
 Some new considerations:
 Power consumption – based on number of
cores, CPU time
 Power consumption can also reflect resource
awareness – locality
 Reliability – modeled as a random process,
included in the simulation
14

15. Game Theory
 Many applications in applied mathematics
 Assumes multiple agents as opposed to a single
agent
 Agents can act independently and are assumed
to act in their own best interest
15

16. Game Theory, cont’d.
 For example, the prisoner’s dilemma…
16

17. Game Theory, cont’d.
 Active area of research, including study of
machine scheduling
 E.g., grid computing, with multiple independent
local schedulers that cooperate in some way to
distribute the workload
 Or in systems with multiple users or users vs. the
system admin
 The latter is proposed by my coauthor
17

18. Game Theory, cont’d.
 Some considerations in Game Theory studies
of workload scheduling:
 Distributed Scheduling
 Hierarchical Scheduling
 Cooperative vs. Non-cooperative
 Complete vs. Incomplete Information
 “Truth Telling”
18

19. Game Theory, cont’d.
 More considerations in Game Theory studies of
workload scheduling:
 Bidding, Auctioning, Pricing, Bartering,
Commodity Market
 “Friendship”
 Complex workload structures (e.g., phased
& distributed)
19

20. Nash Equilibrium
 Object of inquiry is often the distinction between the
globally optimal solution and solutions where each
independent agent strives for its own optimum
 When no agent changes their strategy from one
iteration to the next, the system is in equilbrium
 When there exists a set of locally optimal solutions,
such that no individual agent can improve their own
objective by changing their strategy, this is called a
Nash Equilibrium
20

21. Nash Equilibrium, cont’d.
 Difference between global and local optima is called
the “Price of Anarchy,” how much less optimal
solution is with competing independent agents vs.
global optimum
 Global optimum is often too complex to calculate
(“NP-complete”)
 It has been shown that a Nash Equilbrium exists,
provided that agents can use mixed strategies, where
each agent selects from several choices based on a
probability distribution
21

22. Dynamic Prioritization Vs. Game
Theory Methods
 In dynamic prioritization, strategy changes over
time based on analysis of the workload using
simulation
 In game theory, strategies change over time based
on a probability distribution
 Results of each alternative are solved using
simulation
 The simulation uses a known historical or
synthetic workload
22

23. Dynamic Prioritization Vs. Game
Theory Methods, cont’d.
 The Nash Equilibrium is rarely optimum
 Dynamic prioritization can find the optimum
solution (subject to parameter constraints) using
brute force and should beat Nash
 Nash generally entails probabilistic mixed
strategies
23

24. Dynamic Prioritization Vs. Game
Theory Methods, cont’d.
 Dynamic prioritization is deterministic over its
parameter constraints
 Dynamic prioritization can simulate multiple
agents’ priorities and in that sense have a game
theoretic perspective
 Dynamic prioritization will incorporate agents’
actions in the simulation once each job has been
submitted to the queue – probability has become
reality
24

25. Dynamic Prioritization Vs. Game
Theory Methods, cont’d.
 Dynamic prioritization is deterministic based on
the currently submitted workload – does not
forecast the future
 This is feasible because repeated simulation has
become computationally cheap
 Game theory deals with future probabilities
25

26. New Simulation: Set Up
 All users are considered collectively as one agent, all
using the same strategy
 The two agents are the User group and the System
administrator
 Users are unaware of the Sys admin’s strategy
 User objectives: minimize run time & minimize
expansion factor
 Sys admin objectives: minimize power use; maximize
system utilization; maximize reliability & maximize
throughput
26

27. New Simulation: Results
 Solve using both dynamic prioritization and
game theory methods and compare…
 Results are pending
27

28. Conclusions
 As a practical matter, independent users/agents
will in fact tend to behave in their own self-
interests
 Users are clever and their specific behavior is hard
to predict
 Often this will lead to mixed strategy behavior
 Generally, there will be a Nash Equilibrium among
the agents, with agents using mixed strategies and
less than globally optimal performance
28

29. Conclusions
29
 System Administrators have reason to consider the
expected selfish behavior of users
 Because of brute-force effectiveness, simulation
should find optimal workload schedules in the
presence of active, selfish user/agents
 Studies using game theory provide new insights,
test new hypotheses, and can help validate
experience and live test results