A presentation given to Data Science London in May 2013

1. Thomas French
29th May 2013
www.sandtable.com
Technical Challenges of Real-
World Agent-Based Modelling
Thursday, 30 May 13

2. Outline
• What is ABM?
• Why use ABM?
• Classic ABM example
• Real World ABM
• Three Key Technical Challenges
Thursday, 30 May 13

3. “Essentially, all models are
wrong, but some are useful”
G.E. Box (1987)
Thursday, 30 May 13

4. ABM in a nutshell
AGENT
ENVIRONMENT
SENSORS
MESSAGES
ACTIONS
PERCEPTS
OBJECT
ACTUATORS
Based&on&Bordini&et&al&&(2007)
Thursday, 30 May 13

5. Why are we talking about ABM?
• It shows promise for
understanding complex
systems:
– heterogeneous and
adaptive actors
– complex interactions:
interdependencies;
feedback loops
– dynamic environment
• It provides an accessible
metaphor for modelling
– modelling individuals
• More and more data is
available for our models
- Finer levels of
granularity
• Computing power is
available on-demand
- Costs continue to
reduce
Thursday, 30 May 13

6. Classic ABM: Schelling Segregation Model
• Developed by Thomas Schelling in 1970s.
• Study racial segregation of populations emerging
from individual discriminatory behaviours.
Thursday, 30 May 13

7. Source: Eric Fisher
Thursday, 30 May 13

8. Schelling Segregation Model
Thursday, 30 May 13

9. Schelling Segregation Model
Thursday, 30 May 13

10. Schelling Behaviour Tree
Thursday, 30 May 13

11. Real World ABM
Thursday, 30 May 13

12. QuitSIM Behaviour Tree
-
QUIT SIM 2
QS Tree in Colour Censor
Thu May 30 2013
Thursday, 30 May 13

13. QuitSIM Behaviour Tree
Take up
smoking?
Never smoker = 1
Become smoker
Age, gender
Do nothing
Cut down
attempt length, route,
age, dependency
Consume
media / ingest
experience
Smoker
Smoker = 1
Never Smoker
Never Smoker = 1
Consume
media / ingest
experience
Get support?
Set support flag
Planned or
Unplanned?
Do something
about
smoking?
motivation, events,
price, GP, social,
pregnant, media,
random
8#2013
Thursday, 30 May 13

14. Technical Challenges
BUILD VALIDATE EXPERIMENT
Designing*and
building*models
Building
Confidence*
in*Models
Conducting
Large?Scale
Experiments
HARD VERY,*VERY*HARD VERY*HARD
Thursday, 30 May 13

15. Building Models
BUILD
VALIDATE
EXPERIMENT
Behavioural+
Data
Survey
Data
Assumptions
Intuition
Analyse Build
Individual+Agent+
Attributes
Behaviour+Tree
Environment
(e.g.+Media)
Representative+
Population
Data+Sources
Simulation
Components
Thursday, 30 May 13

16. Validation - Building Confidence
VALIDATE
EXPERIMENT
Does the implemented
model reflect the
real-world system?
Thursday, 30 May 13

17. Validation – Establishing Criteria
A framework for evaluating state of validity of models
for on-going monitoring.
VALIDATE
EXPERIMENT
VALIDATION
INTERNAL
VALIDATION
EXTERNAL
VALIDATION
Model&
implemented&
correctly
Behaviours&
predicted&make&
sense&/&are&logical
Model&stands&up&
to&comparison&
with&external&data
Thursday, 30 May 13

18. Validation - Examples
Represented in a formal logic
• linear-time temporal logic with extensions
Internal:
(s_Att.gender = f) => (G (s_Att.gender = f) )
G (!((s_Att.smoker = 1) && (s_Att.takeUp = 1)))
G (!((s_Att.smoker = 1) && (s_Att.age < 11)))
External:
n_MSE (s_Val1.prevalence, r_Val1.prevalence)
n_MSE (s_Val2.quit_atts, r_Val2.quit_atts)
VALIDATE
EXPERIMENT
Thursday, 30 May 13

19. Validation –
Solving Multi-Criteria Problems
VALIDATE
EXPERIMENT
Thursday, 30 May 13

20. Validation - Workflow
VALIDATE
EXPERIMENT
Select&Model
Select&Tests
Select&
Reference&Data
Configure&Test&
Suite
Execute&
Replications
Summarise&
Individual&Tests
Summarise&Test&
Suite
Thursday, 30 May 13

21. Experimentation -
Approaches
• Empirical Calibration
• Sensitivity Analysis
• Scenario Exploration
• Goal-Directed Search
EXPERIMENT
Thursday, 30 May 13

22. Experimentation –
Exploring Parameter Spaces
EXPERIMENT
Small Large
Explore Exhaustive+Search
Simple+Random+Sampling,+
Latin+Hypercube+Sampling
e.g.+7+vars,+10/100+values+=+
1+Trillion+parameter+sets
Seek Exhaustive+Search
Noisy,+MultiEObjective+
Evolutionary+Algorithms
Parameter+Space
Search+Type
Thursday, 30 May 13

23. Experimentation -
Handling Noise
EXPERIMENT
Thursday, 30 May 13

24. Experimentation –
Handling Output Data
EXPERIMENT
Thursday, 30 May 13

25. Experimentation –
Platform Architecture
EXPERIMENT
CATALOG
REST API
WORKFLOW
SCENARIOS
ANALYSIS
VALIDATION
OPTIMISATION
SERVICES
mongoDB
MANAGER
WORKER 1
PLATFORM
RabbitMQ
MESSAGING
http://
sandtable.com
Sandtable Simulation Platform
CLIENT
simulation
analysis
validation
1
2
k
2
3
N
S3
Sandtable)Simulation)Platform
Thursday, 30 May 13

26. Experimentation -
Managing Workflow
EXPERIMENT
Thursday, 30 May 13

27. Thursday, 30 May 13

28. “Nothing is built on stone;
all is built on sand. But we must
build as if sand were stone.”
J.L. Borges
Thursday, 30 May 13

29. Thanks for listening!
thomas@sandtable.com
www.sandtable.com
Thursday, 30 May 13

30. Further study
Book:
• John Miller and Scott Page: 'Complex Adaptive
Systems: An Introduction to Computational Models
of Social Life' (2007)
Coursera:
• Scott Page: 'Model Thinking'
• https://www.coursera.org/course/modelthinking
Thursday, 30 May 13