HashiTLS Demystifying Security Certs

HashiTLS
Demystifying Security Certs
HashiTalks 2020
Alan Scherger
Cloud Janitor

Microsoft Teams - 3hr Outage
Feb 3, 2020

Nomad - CVE-2020-7956
Jan 28th, 2020

Kubernetes - CVE-2016-7075
Oct 10, 2016

So what?
1. Managing certiﬁcates in 2020 is still hard.
2. We keep repeating the same errors in our software.
3. It all starts with education, and a foundation in the basics.

bit.ly/learn-mtls-the-hard-way
A future workshop that will be much more in-depth than these slides.

Agenda
Hands-on Practice
Use practical tools like mkcert
and HashiCorp Consul to get
familiar with how certiﬁcates
can be managed and used.
First Principles
Learn what makes up a
certiﬁcate and how those
pieces come together to for a
complete system.
Learn the Code
Examine a few code bases
which utilize mTLS to get work
done.

SSL Certificate
TLS Certificate
HTTPS Certificate
X.509 v3 Certificate

X.509 v3 Certiﬁcate
RFC 5280

Certificate Basic Fields - 1/3
▪ Version Number
– The version of the encoded certificate.
▪ Serial Number
– A positive integer unique to the Certificate Authority.
▪ Signature Algorithm
– Contains the identifier for the cryptographic algorithm used by the CA to
sign the certificate.

▪ Issuer
– Identifies the entity that has signed and issued the certificate.
▪ Validity
– The time interval during which the CA warrants that it will maintain
information about the status of the certificate.
▪ Subject
– Identifies the entity associate with the public key stored in the subject
public key field.

▪ Subject Public Key Info
– Used to carry the public key and identify the algorithm with which the key
is used.
▪ Extensions
– Provide methods for associating additional attributes to certiﬁcates.

Let’s Look at a Certiﬁcate
TERMINAL
> openssl x509 -in ~/.local/share/mkcert/rootCA.pem -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
ba:26:56:af:26:bd:3c:1a:e5:05:9d:fa:0b:83:40:26
Signature Algorithm: sha256WithRSAEncryption
Issuer: O = mkcert development CA,
OU = ascherger@incontrol (Alan Scherger),
CN = mkcert ascherger@incontrol (Alan Scherger)
Validity
Not Before: Feb 18 22:34:27 2020 GMT
Not After : Feb 18 22:34:27 2030 GMT
...

TERMINAL
> openssl version
LibreSSL 2.8.3
>macOS
Catalina
10.15.3
LibreSSL since 10.13 -
September 2017

TERMINAL
> docker run --rm ubuntu:18.04 /bin/bash -c "apt-get
update && apt-get install openssl -y && openssl
version"
…
Get:1 http://archive.ubuntu.com/ubuntu
bionic-updates/main amd64 libssl1.1 amd64
1.1.1-1ubuntu2.1~18.04.5 [1300 kB]
Get:2 http://archive.ubuntu.com/ubuntu
bionic-updates/main amd64 openssl amd64
1.1.1-1ubuntu2.1~18.04.5 [613 kB]
…
Setting up openssl (1.1.1-1ubuntu2.1~18.04.5) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
OpenSSL 1.1.1 11 Sep 2018
Ubuntu
Bionic
Beaver
18.04
OpenSSL 1.1.1

OpenSSL Command
TERMINAL
This is the command which is used for displaying and signing X509 formatted
certificates.

OpenSSL Command
TERMINAL
This specifies the input filename to read a certificate from.

OpenSSL Command
TERMINAL
-text : Prints out the certificate in text form.
-noout : Prevents output of the encoded version of the certificate.

Version
The version of the encoded certiﬁcate.
TERMINAL
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
ba:26:56:af:26:bd:3c:1a:e5:05:9d:fa:0b:83:40:26
Validity
Not After : Feb 18 22:34:27 2030 GMT
...

Serial Number
A positive integer unique to the Certiﬁcate Authority.
TERMINAL
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
ba:26:56:af:26:bd:3c:1a:e5:05:9d:fa:0b:83:40:26
Validity
Not After : Feb 18 22:34:27 2030 GMT
...

Signature Algorithm
Contains the identiﬁer for the cryptographic algorithm used by the CA to sign this certiﬁcate.
TERMINAL
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
ba:26:56:af:26:bd:3c:1a:e5:05:9d:fa:0b:83:40:26
Validity
Not After : Feb 18 22:34:27 2030 GMT
...

Issuer
Identiﬁes the entity that has signed and issued the certiﬁcate.
TERMINAL
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
ba:26:56:af:26:bd:3c:1a:e5:05:9d:fa:0b:83:40:26
Validity
Not After : Feb 18 22:34:27 2030 GMT
...

Validity
The time interval during which the CA warrants that it will maintain information about the status of the certiﬁcate.
TERMINAL
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
ba:26:56:af:26:bd:3c:1a:e5:05:9d:fa:0b:83:40:26
Validity
Not After : Feb 18 22:34:27 2030 GMT
...

Subject
Identiﬁes the entity associate with the public key stored in the subject public key ﬁeld.
TERMINAL
Subject: O = mkcert development CA,
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (3072 bit)
Modulus: [large integer represented in colon-hexadecimal notation]
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Key Usage: critical
Certificate Sign
X509v3 Basic Constraints: critical
CA:TRUE, pathlen:0
X509v3 Subject Key Identifier:
B9:D5:B3:06:55:B4:E6:CE:CB:CB:56:B3:4A:35:96:A3:AA:5F:2D:C4

Subject Public Key Info
Used to carry the public key and identify the algorithm with which the key is used.
TERMINAL
Exponent: 65537 (0x10001)
X509v3 extensions:
Certificate Sign
CA:TRUE, pathlen:0

Extensions: Key Usage
Deﬁnes the purpose of the key contained in the certiﬁcate.
TERMINAL
Exponent: 65537 (0x10001)
X509v3 extensions:
Certificate Sign
CA:TRUE, pathlen:0

Extensions: Basic Constraints
Identifies whether the subject of the certificate is a CA and
the maximum depth of valid certification paths that include this certificate.
TERMINAL
Exponent: 65537 (0x10001)
X509v3 extensions:
Certificate Sign
CA:TRUE, pathlen:0

Extensions: Subject Key Identiﬁer
Provides a means of identifying certiﬁcates that contain a particular public key.
TERMINAL
Exponent: 65537 (0x10001)
X509v3 extensions:
Certificate Sign
CA:TRUE, pathlen:0

Blue Book
Volume VIII - Fascicle VIII.8
Data Communication Networks Directory
Recommendations X.500-X.521
🔥 236 glorious pages of explanation 🔥

Certiﬁcate Authority
Is an entity that issues digital certiﬁcates.

Certificate Chain of Trust
A list of certificates (usually starting with an end-entity
certificate) followed by one or more CA certificates (usually the
last one being a self-signed certificate), with the following
properties:
1. The Issuer of each certificate (except the last one)
matches the Subject of the next certificate in the list.
2. Each certificate (except the last one) is supposed to be
signed by the secret key corresponding to the next
certificate in the chain (i.e. the signature of one
certificate can be verified using the public key contained
in the following certificate).
3. The last certificate in the list is a trust anchor: a
certificate that you trust because it was delivered to you
by some trustworthy procedure.

Cryptographic Protocols
The ﬁnal chapter in me rambling!

Cryptographic Protocols
Protocol Document Deprecated
SSL Version 2.0 Draft
March 2011
RFC 6176
SSL Version 3.0 RFC 6101
June 2015
RFC 7568
TLS 1.0 RFC 2246
May 2015
RFC 7525
---
June 2018
PCI DSS
TLS 1.1 RFC 4346
First ½ 2020
Apple, Microsoft, Google
TLS 1.2 RFC 5246
TLS 1.3 RFC 8446

FiloSottile/mkcert
A simple tool for making locally-trusted development certificates.

iOS 13 and macOS 10.15 - Gotchya
https://support.apple.com/en-us/HT210176

Install
Download the latest
Release
Use the -install ﬂag to
generate an install
certiﬁcates.
TERMINAL
> chmod +x ~/Downloads/mkcert-v1.4.1-darwin-amd64
> ~/Downloads/mkcert-v1.4.1-darwin-amd64 -install
Created a new local CA at
"/Users/ascherger/Library/Application Support/mkcert" 💥
Sudo password:
The local CA is now installed in the system trust store!
⚡
Warning: "certutil" is not available, so the CA can't be
automatically installed in Firefox! ⚠
Install "certutil" with "brew install nss" and re-run
"mkcert -install" 👈
>

TERMINAL
> ls -al /Users/ascherger/Library/Application Support/mkcert
total 16
drwxr-xr-x 4 ascherger staff 128 Feb 16 22:09 .
drwx------+ 121 ascherger staff 3872 Feb 16 22:09 ..
-r-------- 1 ascherger staff 2488 Feb 16 22:09 rootCA-key.pem
-rw-r--r-- 1 ascherger staff 1728 Feb 16 22:09 rootCA.pem
>
Look at our
certiﬁcates.
rootCA-key.pem is the private key.
rootCA.pem is the public certiﬁcate.

Inspect the
Root CA
Private Key
RFC 3447
Section 3.2
TERMINAL
> openssl rsa -in rootCA-key.pem -text -noout
Private-Key: (3072 bit)
modulus:
00:e0:70:56:33:aa:83:d5:ed:0f:46:f1:99:d5:81:
...
publicExponent: 65537 (0x10001)
privateExponent:
00:aa:f4:e6:b4:74:2d:f7:c5:9a:dd:6f:2a:be:77:
...
prime1:
00:f2:6d:61:68:e0:6d:18:72:ef:7e:86:17:7a:b7:
...
prime2:
00:ed:01:22:6b:5b:88:ae:c1:21:e5:be:91:58:c4:
...
exponent1:
00:a4:82:77:5c:cd:17:1b:45:3b:a2:57:6a:6d:7b:
...
exponent2:
00:9f:f6:39:0d:ee:bc:c4:fe:33:6b:c6:00:81:c2:
...
coefficient:
10:0e:1d:ad:75:e5:b9:fc:fe:47:72:c6:86:88:dc:
...

TERMINAL
> openssl x509 -in rootCA.pem -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
84:4a:83:84:72:ad:27:89:09:7e:48:44:b9:f6:30:6e
Issuer: O=mkcert development CA,
OU=ascherger@Alans-MacBook-Pro.local,
CN=mkcert ascherger@Alans-MacBook-Pro.local
Validity
Not After : Feb 17 04:09:41 2030 GMT
Subject: O=mkcert development CA,
Public-Key: (3072 bit)
Modulus:
00:e0:70:56:33:aa:83:d5:ed:0f:46:f1:99:d5:81:
...
Exponent: 65537 (0x10001)
X509v3 extensions:
Certificate Sign
CA:TRUE, pathlen:0
93:EF:36:51:3D:94:46:01:8F:01:F7:B9:22:09:75:F9:E7:63:93:F9
bb:0e:80:b4:35:b8:2a:58:9e:36:f3:4a:ce:87:5c:0b:86:54:
...
Inspect the
Root CA
Public
Certiﬁcate

Make a new
localhost
certiﬁcate.
TERMINAL
> mkcert localhost 127.0.0.1 ::1
Using the local CA at
"/Users/ascherger/Library/Application Support/mkcert" ✨
Created a new certificate valid for the following names
📜
- "localhost"
- "127.0.0.1"
- "::1"
The certificate is at "./localhost+2.pem" and the key at
"./localhost+2-key.pem" ✅
>

Serial Number
Remember “18:f1” we’ll see that again soon.
TERMINAL
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
18:f1:88:4e:19:56:0f:7a:ae:11:75:eb:e9:67:8d:57
Validity
Not Before: Jun 1 00:00:00 2019 GMT
Not After : Feb 19 07:05:58 2030 GMT
Subject: O=mkcert development certificate,
OU=ascherger@Alans-MacBook-Pro.local

Issuer
Matches our Root CA public key information.
TERMINAL
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
18:f1:88:4e:19:56:0f:7a:ae:11:75:eb:e9:67:8d:57
Validity
Not After : Feb 19 07:05:58 2030 GMT

Validity
Patched to get around the macOS hiccup.
TERMINAL
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
18:f1:88:4e:19:56:0f:7a:ae:11:75:eb:e9:67:8d:57
Validity
Not After : Feb 19 07:05:58 2030 GMT

Subject
No Common Name (CN), and diﬀerent than CA information.
TERMINAL
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
18:f1:88:4e:19:56:0f:7a:ae:11:75:eb:e9:67:8d:57
Validity
Not After : Feb 19 07:05:58 2030 GMT

Key Usage
Explicitly allows for Web Server usage, but not Client usage.
TERMINAL
Modulus:
00:c0:ed:e2:11:01:66:60:d1:c6:50:cd:e0:7a:a3:
...
Exponent: 65537 (0x10001)
X509v3 extensions:
Digital Signature, Key Encipherment
X509v3 Extended Key Usage:
TLS Web Server Authentication
CA:FALSE

Basic Constraints
This certiﬁcate cannot act as a CA, so it cannot make child certiﬁcates.
TERMINAL
Modulus:
00:c0:ed:e2:11:01:66:60:d1:c6:50:cd:e0:7a:a3:
...
Exponent: 65537 (0x10001)
X509v3 extensions:
Digital Signature, Key Encipherment
X509v3 Extended Key Usage:
TLS Web Server Authentication
CA:FALSE

Authority Key Identifier
Matches the Subject Key Identifier of the Root certificate.
TERMINAL
X509v3 Authority Key Identifier:
keyid:93:EF:36:51:3D:94:46:01:8F:01:F7:B9:22:09:75:F9:E7:63:93:F9
X509v3 Subject Alternative Name:
DNS:localhost, IP Address:127.0.0.1, IP Address:0:0:0:0:0:0:0:1
4b:f5:d0:fa:27:43:c2:d8:ef:4c:be:5e:66:81:21:c1:c1:5f:
...

Subject Alternative Name
A list of DNS and IP addresses this certiﬁcate is allowed to represent.
TERMINAL
X509v3 Authority Key Identifier:
keyid:93:EF:36:51:3D:94:46:01:8F:01:F7:B9:22:09:75:F9:E7:63:93:F9
X509v3 Subject Alternative Name:
DNS:localhost, IP Address:127.0.0.1, IP Address:0:0:0:0:0:0:0:1
4b:f5:d0:fa:27:43:c2:d8:ef:4c:be:5e:66:81:21:c1:c1:5f:
...

Nginx
default.conf
CODE EDITOR
server {
listen [::]:443 ssl http2 ipv6only=on;
listen 443 ssl http2;
server_name localhost;
ssl_certificate /etc/nginx/conf.d/localhost+2.pem;
ssl_certificate_key /etc/nginx/conf.d/localhost+2-key.pem;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}

Run Nginx
TERMINAL
> docker run -p 4448:443 -v "`pwd`:/etc/nginx/conf.d" nginx
Unable to find image 'nginx:latest' locally
latest: Pulling from library/nginx
bc51dd8edc1b: Pull complete
66ba67045f57: Pull complete
bf317aa10aa5: Pull complete
Digest:
sha256:ad5552c786f128e389a0263104ae39f3d3c7895579d45ae716f528185b36
bc6f
Status: Downloaded newer image for nginx:latest

Visit https://localhost:4448/
The certiﬁcate is trusted, and the Serial Number matches the certiﬁcate we just created.

cloudﬂare/cfssl
Cloudflare’s PKI and TLS toolkit

bit.ly/consul-mtls
Learn mTLS with Consul!

Consul + TLS
1. Consul CLI has a tls command for setting up a CA and certiﬁcates.
2. Consul has an auto_encrypt feature for auto-managing certiﬁcates
3. Consul’s Connect API contains the CA endpoints, and supports automated
CA rotation through cross-signing.

bit.ly/vault-learn
Learn how to setup Vault!
Be sure to look up tls_require_and_verify_client_cert

bit.ly/nomad-mtls
Learn mTLS with Nomad!

Consul - Encryption
If verify_server_hostname is set, then outgoing
connections perform hostname veriﬁcation.
All servers must have a certiﬁcate valid for
server.<datacenter>.<domain> or the client will reject the
handshake.

package tlsutil - config.go L706-L726
CODE EDITOR
// Wrap a net.Conn into a client tls connection, performing any
// additional verification as needed.
//
// As of go 1.3, crypto/tls only supports either doing no certificate
// verification, or doing full verification including of the peer's
// DNS name. For consul, we want to validate that the certificate is
// signed by a known CA, but because consul doesn't use DNS names for
// node names, we don't verify the certificate DNS names. Since go 1.3
// no longer supports this mode of operation, we have to do it
// manually.
func (c *Configurator ) wrapTLSClient (dc string, conn net.Conn) (net.Conn, error) {
config := c.OutgoingRPCConfig ()
verifyServerHostname := c.VerifyServerHostname ()
verifyOutgoing := c.verifyOutgoing ()
domain := c.domain()
if verifyServerHostname {
// Strip the trailing '.' from the domain if any
domain = strings.TrimSuffix(domain, ".")
config.ServerName = "server." + dc + "." + domain
}
tlsConn := tls.Client(conn, config)

Nomad - Securing With TLS
To fulﬁll the desired security properties Nomad
certiﬁcates are signed with their region and role such as:
- client.global.nomad for a client node in the global
region
- server.us-west.nomad for a server node in the
us-west region

package tls - common.go L510-L521
CODE EDITOR
// VerifyPeerCertificate, if not nil, is called after normal
// certificate verification by either a TLS clientor server. It
// receives the raw ASN.1 certificates provided by the peer and also
// any verified chains that normal processing found. If it returns a
// non-nil error, the handshake is aborted and that error results.
//
// If normal verification fails then the handshake will abort before
// considering this callback. If normal verification is disabled by
// setting InsecureSkipVerify, or (for a server) when ClientAuth is
// RequestClientCert or RequireAnyClientCert, then this callback will
// be considered but the verifiedChains argument will always be nil.
VerifyPeerCertificate func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error

package nomad - server.go L451-L477
CODE EDITOR
// getTLSConf gets the server's TLS configuration based on the config supplied
// by the operator
func getTLSConf(enableRPC bool, tlsConf *tlsutil.Config, region string)
(*tls.Config, tlsutil.RegionWrapper , error) {
// omitted for slide
if tlsConf.VerifyServerHostname {
incomingTLS = itls.Clone()
incomingTLS .VerifyPeerCertificate =
rpcNameAndRegionValidator (region)
} else {
incomingTLS = itls
}
return incomingTLS , tlsWrap, nil
}

CODE EDITOR
// implements signature of tls.Config.VerifyPeerCertificate which is called
// after the certs have been verified. We'll ignore the raw certs and only
// check the verified certs.
func rpcNameAndRegionValidator(region string) func([][]byte, [][]*x509.Certificate)
error {
return func(_ [][]byte, certificates [][]*x509.Certificate) error {
if len(certificates) > 0 && len(certificates[0]) > 0 {
cert := certificates[0][0]
for _, dnsName := range cert.DNSNames {
if validateRPCRegionPeer(dnsName, region) {return nil}
}
if validateRPCRegionPeer(cert.Subject.CommonName, region) {
return nil
}
}
return errors.New("invalid role or region for certificate")
}

CODE EDITOR
func validateRPCRegionPeer (name, region string) bool {
parts := strings.Split(name, ".")
if len(parts) < 3 {
// Invalid SAN
return false
}
if parts[len(parts)-1] != "nomad" {
// Incorrect service
return false
}
if parts[0] == "client" {
// Clients may only connect to servers in their region
return name == "client."+region+".nomad"
}
// Servers may connect to any Nomad RPC service for federation.
return parts[0] == "server"
}

Thank You!
Alan Scherger
@ﬂyinprogrammer
bit.ly/learn-mtls-the-hard-way
73

HashiTLS Demystifying Security Certs

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a HashiTLS Demystifying Security Certs

Similar a HashiTLS Demystifying Security Certs (20)

Más de Mitchell Pronschinske

Más de Mitchell Pronschinske (20)

Último

Último (20)

HashiTLS Demystifying Security Certs