1. Communicating Uncertainty
Andy Hart
Food and Environment Research Agency, York
andy.hart@fera.gsi.gov.uk
BSA Science Communication Conference, 1 May 2014

2. Why communicate uncertainty?
Red River Flood, Grand Forks USA, 1997
• Levee height: 51 feet
• River height prediction: 49 feet
• Uncertainty: plus or minus 9 feet
51
49
±9
After: Nate Silver, The Signal and the Noise, 2012

3. Why communicate uncertainty?
Red River Flood, Grand Forks USA, 1997
• Levee height: 51 feet
• River height prediction: 49 feet
• Uncertainty: plus or minus 9 feet
• Actual flood height: 54 feet
51
49
±9
After: Nate Silver, The Signal and the Noise, 2012

4. Why communicate uncertainty?
Red River Flood, Grand Forks USA, 1997
• Estimated damages: ca. $3.5 billion
51
49
±9

5. Why communicate uncertainty?
‘Credibility requires trust…
…trust requires openness…
…openness requires
recognition of uncertainty’
(Phillips Report on BSE, 2000)

6. What to communicate?
• What information is needed to communicate
uncertainty?

7. Reasons for uncertainty
• Red River Flood height affected by:
− Temperature (snow melt)
− Rainfall
− Bridges, sandbags etc. (retard flow)
− Volume of water downstream
− Extrapolation beyond previous data
Reasons are important, but not enough
51
49

8. Confidence in predictions?
• 12th April 2013 – US Intelligence reports
‘moderate confidence’ that North Korea
has nuclear weapons
• They say: ‘Moderate confidence generally means
the information is interpreted in various ways, we
have alternative views, or the information is
credible and plausible but not corroborated
sufficiently to warrant a higher level of confidence’
Qualitative expressions of confidence are
not enough
51
49

9. The range of plausible outcomes
• Tells you what might happen
• But not how likely that is
• Okay if all the outcomes are okay
• Not enough if bad outcomes are possible
but costly to avoid or mitigate
 Also need information on the relative
likelihoods of different outcomes
51
49
±9

10. The range of plausible outcomes
• Also need information on the relative likelihoods
of different outcomes
TV forecasts for
Hurricane Irene,
2011

11. 299600
299700
299800
299900
300000
1870 1890 1910 1930 1950
Measuredspeedoflight(km/sec)
Confidence intervals
Include information on likelihood but
only for uncertainties that have been quantified
Historical
measurements of
the speed of light,
± measurement
error (1 standard
deviation)
After Henrion and Fischoff, 1986
Generally
accepted
value (1986)

12. Confidence intervals
• Confidence intervals from data or modelling only
express uncertainty quantified in the data or model
 Need to:
− say which uncertainties have been quantified
− identify any additional uncertainties
− indicate their potential impact on the range and
likelihood of alternative outcomes
− acknowledge any deep uncertainties for which
this is not possible

13. Additional uncertainties
• Impact of additional uncertainties
e.g. Bank of England fan charts take account of
additional uncertainties when forecasting inflation
% increase on
prices a year
earlier
past future

14. Qualitative expressions of likelihood?
• The same word
means different
things to different
people
• Words are not
enough
51
49
±9
Macleod & Pietravalle

15. Subjective probabilities
“Some of our intelligence
officers thought that it was
only a 40 or 30% chance
that Bin Laden was in the
compound. Others thought
that it was as high as 80 or
90%. At the conclusion of a
fairly lengthy discussion
where everybody gave their
assessments I said: this is
basically 50-50.” Barack Obama speaking on ‘Bin Laden:
Shoot to Kill’, Channel 4, 7 September 2011

16. Words with quantitative meaning
• E.g. IPCC Likelihood scale:
• Verbal definitions just transfer the problem
51
49
±9

17. Words with quantitative meaning
‘The UNEP report puts the average
global cooling that could be achieved
with black-carbon control measures at
0.3 watts per square metre…
... the range of possible effects runs
from twice that much to nothing at all
...there is even an outside chance that
the proposed action could lead to a little
warming...
...though if that were the case the
actions on ozone would still provide a
net cooling for the actions as a whole…’
NASA GSFC
The Economist, 19 Feb. 2011

18. Deep uncertainty
• If it’s not possible to quantify likelihood
even approximately then it is misleading to
use undefined verbal expressions
• Indicates presence of deep uncertainties
− describe them
− explain the prediction is just one
possibility
− if nothing can be said about the range
and/or likelihood of other outcomes then
say so
51
49

19. What is being done about it
• Who are the relevant authorities?
• What are they doing to decrease the
chance of adverse outcomes?
• What are they doing to monitor the
situation?
• What contingency plans do they have?
• What options can others consider?
51
49
±9

20. Outcomes are always uncertain
express confidence in the chosen strategy
(if warranted)
but
acknowledge the outcome is still uncertain
51
49
±9

21. Summary
Information needed for communicating uncertainty:
• Reasons for uncertainty
• Range of plausible outcomes
• Likelihood of adverse outcomes
− As quantitative as possible
• Nature of deep uncertainties, if present
• What is being done about it?
If this information isn’t provided, ask for it!
51
49
±9