1. Closing the TLS Authentication Gap Marsh Ray Steve Dispensa PhoneFactor www.phonefactor.com

2. Who are we, anyway?

3. We’re going to tell a story Finding the flaw Deciding how to address it Private disclosure Public disclosure Post-disclosure work Lessons learned

4. Finding the problem August 11, 2009

5. So how did this happen? It’s Microsoft’s fault! Answered a question in a forum… Which turned into a series of interesting discussions over the summer about MitM Eventually, Marsh got fed up and went spelunking in mod_ssl

6. /* To allow authentication to complete in this auth hook, the * solution used here is to fill a (bounded) buffer with the * request body, and then to reinject that request body later. */ if (renegotiate && !renegotiate_quick && (apr_table_get(r->headers_in, "transfer-encoding") || (apr_table_get(r->headers_in, "content-length") && strcmp(apr_table_get(r->headers_in, "content-length"), "0"))) && !r->expecting_100) { intrv; /* Fill the I/O buffer with the request body if possible. */ rv = ssl_io_buffer_fill(r); ...

7. Apache 2.2 mod_ssl documentation

8. That can’t be right… Buffering and replaying a request seemed… scary So, I decided to make sure authentication continuity was maintained across the renegotiation. Imagine my surprise when I couldn’t find a clear answer.

9. 7.4.1. Hello Messages ... compression algorithms are initialized to null. The current connection state is used for renegotiation messages. 7.4.1.2. Client Hello When this message will be sent: When a client first connects to a server, it is required to send the ClientHello as its first message. The client can also send a ClientHello in response to a HelloRequest or on its own initiative in order to renegotiate the security parameters in an existing connection. RFC 5246: Strangely quiet on renegotiation

10. So, Marsh disappeared for a couple of weeks, and out came:

12. Demo!

13. We immediately understood the significance.

15. Disclosure without disclosure, on September 8

16. Three attacks Client certificate-based attack Client certificates can trigger renegotiation Upgrade attack Different-strength crypto requirements can lead to renegotiation Client-initiated attack In theory, a client could start a renegotiation at any time

17. But wait, there’s more! Browsers don’t always validate the server cert before handing out the client cert! Therefore, a client cert can effectively be forwarded to any server on the net that accepts it Browsers don’t always prompt for client certificates when they make can make an “intelligent” choice Victim never knows what hit him

18. OK, does it matter? We struggled to assess scope and impact… Client certificate – first finding; mitigation painful Upgrade attack – cute, but… meh… Client-initiated – almost an afterthought at this point Not absolutely sure it would even work

19. Is Renegotiation worth saving? Disabling renegotiation completely would be: Easy Effective Solve about 95% of the problem Will it ever come back? IP Source Routing, anyone?

20. Some uses of TLS renegotiation

21. Some uses of TLS renegotiation Wikipedia

22. Wild speculation!

23. DoD Common Access Card System 3.5M active cards 1M card readers Primarily based on client certificates! Wikipedia

27. Doesn’t this make you want a National ID card?

28. We realized: Needed a coordinated effort to fix Huge leak potential We wanted near-simultaneous disclosure Wanted a solution before the bug leaked

29. Project Mogul September 14, 2009

30. Oh, by the way, it’s not this guy:

31. Disclosure plan Decided on a phased disclosure plan: Disclose a few respected security gurus Disclose to SSL code owners and start a fix Widen the circle carefully over time Hope for a controlled public disclosure

32. The NDA Everyone told us we were insane to want an NDA, until… …they heard about the flaw! We wanted pressure on vendors Intentionally written to expire on 1/31/2010.

34. "Both the insider and his friend were active members of the hacking group, and regularly attended the organization’s meetings. They used IRC channels to communicate back and forth with one another and relay information under assumed hacker names in an attempt to mask their identities."

35. First, we asked Frank Heidt of Leviathan Security, Who confirmed our intuition about the impact of this vuln:

37. Extraordinary claims require extraordinary proof Ponytail immediately began flapping wildly. Took up smoking again. Frank referred us to lots of helpful people, including: Jon Callas, as an independent security review Ben Laurie, for obvious reasons Dan Geer, Kerberos Jennifer Granick @ EFF

38. We thought we needed a plan. Turns out, we needed several. Plan A: Get code owners together and tell them all at once, under NDAs Drawback: Needed people besides coders

39. Plan B: Get programmers and limited support people to Mountain View and disclose all at once under NDA Drawbacks: Vendors needed to know the bug before committing Vendors needed some time to assess impact in order to figure out who needed to be involved

40. Plan C: Disclose bug to code owners and limited support personnel under NDA and then go to Mountain View to work out the details Drawback: Had trouble getting some companies under NDA

41. Plan D: Disclose in advance to the fewest people possible (coders, PSIRT managers, …) under group NDAs such as ICASI and Google then get people to Mountain View to work out the details in a week or two

42. Disclosure All disclosures were completed within about a week Disclosed to Ben Laurie Reproduced across the Internet, tempting the demo gods Tried to disclose to IBM: NDA fail Disclosed to Microsoft next

43. ICASI Pointed to ICASI by Frank, IBM, and Microsoft Disclosed to Steve Manzuik of Juniper/ICASI, leading to: Microsoft Intel Cisco Juniper Nokia IBM

44. Exceptions There were a few notable exceptions: Red Hat lawyers worked the weekend! Sun : “Type your vuln here and hit submit ok thx bye” Apple: We didn’t realize they had their own TLS code Others, due to an attempt to limit scope

45. Mogul meeting: September 28, 2009 About 45 people representing about a dozen organizations Description and captures, again Severity and impact Lots of time spent on client-initiated renegotiation Solution discussion Rescorla, Oskov, and Dispensa/Ray had identical proposals

46. Proposed solution The obvious solution was to bind the cryptographic state from the previous handshake to the current one This is easy: Resend the verify_data from the previous Finished message Already cryptographically secure Already under consideration as a “channel binding” Not a perfect solution, however: Requires a TLS extension Requires additional storage (bad for silicon?)

47. Post-conference work Turns out it’s hard to organize a private, cross-vendor, ad-hoc team! Manzuik requisitioned help from Paul Vixie / OpSecTrust Manzuik set up [mogul-private] and a PGP keyring We set up a private SILC channel Good initial discussion on the lists, but vendor engagement dropped off quickly No data!

48. Initial implementations of safe renegotiation Nasko Oskov from Microsoft had a working implementation quickly Eric Rescorla provided code for OpenSSL Dispensa worked up a patch for GNUTLS We suspect others were making progress

49. TLA [this page intentionally left blank.]

50. Timeline tension Work was going really slowly January 31 “couldn’t possibly work” Not a Patch Tuesday Not a weekday BlackHat / ShmooCon By late October, Steve was on repeated ICASI calls Insisting that we postpone publishing Our position was unchanged officially Meanwhile, Marsh and Steve started to argue about scenarios

51. Public Disclosure November 4, 2009

52. So, we were all minding our own business, when…

53. To: tls at ietf.org Subject: [TLS] MITM attack on delayed TLS-client auth through renegotiation From: Martin Rex <Martin.Rex at sap.com> Date: Wed, 4 Nov 2009 18:28:00 +0100 (MET) After elaborating so much about the client cert authentication through renegotiation with Microsoft IIS, I'm beginning to believe that there is a potential security problem with that scheme, because it is susceptible to a MITM attack. ... [TLS] turns into Full-Disclosure

54. Hilarity ensues Dispensa calls Martin Rex Project Mogul is notified in a very tense call Vendors Hope It Goes Away™ Vendors end by insisting that it would be “extremely irresponsible” to publish After all, “nobody will notice.”

55. So, did anyone notice?

56. Three hours after Martin Rex’s e-mail…

57. …it was re-tweeted a few times, too.

58. Steve gets bored and decides to do something else.

59. Any guesses as to how long it took for working exploit code to be posted to [full-disclosure]?

60. 18 hours. 34 minutes.

61. Initial reactions were… mixed. “The sky is not falling” –Moxie Marlinspike It’s just like CSRF! (Whew! … Whew?) “Most, if not all, major web applications have implementation level protections against CSRF… Those protection measures are effective against this new SSL man in the middle attack. Therefore, this vulnerability has minimal security impact for most websites and Internet users.” –Tom Cross, IBM ISS

62. A couple of days later…

65. Coming to terms with the bug Yeah, but who cares? The 41% of users who use the same password for Twitter as they use for… everything! More importantly - you can’t tell what will be broken In the end, the confusion was our fault

66. Post-disclosure Work

67. IETF ID ready day 1 Flawed: undercounted SSLv3 No extensions! Post-disclosure, Benn Bollay from F5 shared data: 22%! Tons of e-mails on the IETF list Practically a full-time job for Marsh Finally, we added SCSV to address old servers RFC 5746 very soon!

70. OPERA FTW!!!

71. Not everyone has rolled out a fix

72. Some lessons learned

73. Security bugs are a no-win situation Traumatic for vendors Not great for researchers Worst, of course, for the users This was a really hard process – hard to balance lots of competing interests

74. There are other (bigger?) problems with SSL PKI is great in theory, but: ~200 trusted root certificates in Firefox – do you trust them all? There will never be a solution to the dancing bunnies problem Applies to Business Bunnies too! Sometimes, root CA’s do this:

76. We needed hard data We had no success getting vendors to contribute data Would have been extremely helpful to know about SSLv3 prevalence before the IETF process Does client-initiated renegotiation ever happen?

77. IETF security process? It has been suggested that the IETF security review process is broken. If it is, this bug isn’t why: SSL was a Netscape creation SSLv3 was utterly ownerless for years The IETF did find it, a few months after us The IETF could have done a better job adopting SSL

78. One last lesson This one goes out to the slow/no-disclosure crowd with our compliments:

80. So Marsh emailed Pavel Kankovsky and: "I had some free time during the last days of 2006 and wrote the PoC exploit to carry out an experimental verification of the vulnerability. It was easier than I had expected because I found a clever way to makeOpenSSLcooperate. “The exploit was finished on January 3, 2007."

81. One last question for you Did we achieve our goal of minimizing the world’s exposure to the bug?

82. Questions? marsh@extendedsubset.com dispensa@phonefactor.com www.phonefactor.com mogul on silc.hick.org mogul-open@lists.links.org