HIS 2015

1. Murphy vs Satan
Why programming secure
systems is still hard
Roderick Chapman,
Director, Protean Code Limited
Honorary Visiting Professor, Dept. of Computer Science, University of York

2. Contents
• Murphy vs Satan?
• The bad news…
• Some good news…
• A future…

3. Murphy’s computer…
• …fails randomly (and silently with any luck…)

4. Satan’s computer…
• …fails maliciously…
• …at the worst possible time…
• …in the worst possible way…

5. Developing secure systems…
…is “Programming Satan’s Computer”
(Anderson and Needham 2005)

6. Contents
• Murphy vs Satan?
• The bad news…
• Some good news…
• A future…

7. Developing Secure Systems is really hard
• Why?
• Here’s an (incomplete) list of things to think
about…

8. 1. The game is rigged
• Attacker is smarter than you, and has more
time and money than you…

9. 2. Asymmetry of efforts
• As a developer, you have the obligation to
prevent all classes of defect (that you know
about…)
• Attacker needs to find just one defect…

10. 3. Asymmetry of knowledge
• As a developer, you need to prevent the
“vulnerabilities”…
• …that we all know about…
• …that the attackers know about, but you
don’t… 
• …that no-one knows about (yet…)  

11. 3. Asymmetry of knowledge
Paul Kocher et al.,
Differential Power Analysis,
1998

12. 4. Limits of Testing…
• Satan’s Computer defies Testing…
• No matter how hard you try, the attackers will
always find a combination of state and inputs
that you haven’t tested…

13. 5. A market for citrus fruit?
• Secure? Insecure? How can you tell the
difference?
My software’s really secure.
Honest. We’ve tested it lots...

14. Contents
• Murphy vs Satan?
• The bad news…
• Some good news…
• A future…

15. 1. Maths to the rescue…
• Analytical software verification
• Really works…
• Can find all the bugs (of particular classes)…
• Prevents defects earlier in the life-cycle, so saves
money as well…
• Don’t be afraid of “Formal Methods” – it’s a feature of
all mature engineering disciplines…

16. 1. Maths to the rescue…

17. There must
be a catch!

18. OK…The Catch…
• “Soundness” of verification really matters, then?
• For security, “preventing all the vulnerabilities” implies an obligation to
use sound verification methods.
• Yup…but it’s hard to achieve…
• Does the analysis make unverifiable (or just plain wrong) assumptions?
• No free lunch once you get past the “dumb bugs…”
No Verification without Specification

19. 2. Better can be Cheaper
• Myth: really high-quality software must be really
expensive.
• Most software development spends most money on
waste – inserting, then finding and correcting
defects.
• Therefore, a lean, “zero-tolerance” approach to
quality gets you a better product and saves money.

20. 2. Better can be Cheaper
• Myth: really high-quality software must be really
expensive.
• Most software development spends most money on
waste – inserting, then finding and correcting
defects.
• Therefore, a lean, “zero-tolerance” approach to
quality gets you a better product and saves money.

21. 3. How to avoid Lemons…
• Q. How do you avoid getting a lemon?
• A. Ask for a warranty.
• A. Ask for data on defect density, productivity
etc.
• A. Assess the product as well as the process
that produced it…

22. In other words…
• Get the source code…
• Require specific, provable
security properties…
• Put these things in the
contract, with penalty
clauses for non-
compliance.
• If it doesn’t work or is
insecure, then get your
money back…
Talk is cheap.
Show me the
code!

23. Contents
• Murphy vs Satan?
• The bad news…
• Some good news…
• A future…

24. A future…
• We have the technology and know-how right
now to produce software with remarkably low
defect density.
• We must persuade procurers and developers
that it can be done at reasonable cost, and that
there is economic benefit in doing so…
• Then we can go back to thinking about the
really hard problems…

25. Homework…
• For the next piece of software that you
procure…
Get a warranty…
or
Get a different supplier…