The document discusses how content delivery networks (CDNs) have evolved to better handle dynamic and unpredictable content. Traditionally, CDNs focused on static content and treated dynamic content as uncacheable. However, some dynamic content can actually be cached for periods of time. Modern CDNs now provide more granular caching controls and real-time APIs to allow custom caching configurations specified by the application. This improves cache hit ratios, offloads origin servers, and enhances performance even for long-tail content that changes infrequently.

1. Extending
Applica/ons
To
The
Edge
With
CDNs
Hooman
Behesh/,
VP
Technology
Webperf
meetup
NYC
–
May
12
2015

2. What
is
a
CDN
• Global
network
of
HTTP
caches
– Get
content
closer
to
users
• Performance
benefits
• offload

6. In
the
past…

7. Classic
CDNs
• Black
box
service
• Making
changes
sucked!
• LiRle
to
no
real-­‐/me
interfaces
– APIs
or
configura/on
management
• Lots
of
professional
services
• Decent
caching,
terrible
uncaching

8. Cacheable
Uncacheable
Sta3c
Dynamic
How
classic
CDNs
look
at
your
content

9. Cache-Control: max-age=31536000

10. Classic
CDNs:
dynamic
content
• Focused
on
delivery:
transport
op/miza/on

11. But
some3mes…

12. I
know
you’re
calling
it
dynamic,
but
it’s
kind
of
sta3c
some3mes

13. I
know
you’re
calling
it
dynamic,
but
it’s
kind
of
sta3c
some3mes
That’s
cool,
how
long
is
it
sta3c
for?

14. That’s
cool,
how
long
is
it
sta3c
for?
Some3mes
a
liFle
bit,
some3mes
a
lot

15. Some3mes
a
liFle
bit,
some3mes
a
lot
Neat!

16. Some3mes
a
liFle
bit,
some3mes
a
lot
You
look
great
today!

17. You
look
great
today!
Don’t
change
the
subject!
How
do
I
cache
it?

18. Don’t
change
the
subject!
How
do
I
cache
it?
Cache
it
for
half
an
hour?

19. Cache
it
for
half
an
hour?
Some3mes
it
changes
much
faster

20. Some3mes
it
changes
much
faster
Cache
it
for
10
minutes?

21. Cache
it
for
10
minutes?
Are
you
just
guessing
right
now?

22. Are
you
just
guessing
right
now?
Just
trying
to
figure
out
your
comfort
level
with
mistakes!

23. Expiry
guessing!

24. Terrible
Caching

25. Terrible
Caching
Terrible
Anxiety
or
(ex:
news
story)

26. Things
are
different
today

27. Today…
• Influenced
by
clouds
• Devops
requires
programmability
and
control
• CDNs
should
be
part
of
the
applica/on
• Real-­‐/me
integra/on
– Changes
and
APIs
• Unpredictable
content
expiry
• Your
content
à
you
need
control

28. What
does
this
all
mean?

29. Control
Visibility
Cacheability

30. Control
Visibility
Cacheability

31. Cacheability
• Granularity
in
caching
rules
(i.e.
cache
the
way
you
want)
– Customizable
cache
keys
– Ex:
geo-­‐ip
caching
– Ex:
normalizing
user
agents
• Cache
everything
– Expect
high
cache-­‐hit
ra/os
– No
cache-­‐control
anxiety!

32. Cacheability
• Granularity
in
caching
rules
(i.e.
cache
the
way
you
want)
– Customizable
cache
keys
– Ex:
geo-­‐ip
caching
– Ex:
normalizing
user
agents
• Cache
everything
– Expect
high
cache-­‐hit
ra/os
– No
cache-­‐control
anxiety!

33. Cache
hit
ra3os

34. Cache
Hit
Ra3o:
Tradi3onal
calcula3on
1
-­‐
Requests
to
Origin
Total
Requests

35. Origin

36. Origin
Cache

37. TCP
Origin
Cache

38. HTTP
Origin
Cache

39. Origin
Cache
HTTP

40. Origin
Cache
HTTP

41. Origin
Cache
HTTP

42. Origin
Cache
HOT
COLD

43. Origin
Cache
cache
“hit”

44. Cache
Hit
Ra3o:
Tradi3onal
calcula3on
1
-­‐
Requests
to
Origin
Total
Requests

45. Isn’t
this
beFer?
Hits
Total
Requests
@edge

46. Isn’t
this
beFer?
Hits
Hits
+
Misses
@edge

47. Cache
Hit
Ra3o
vs.
1
-­‐
Requests
to
Origin
Total
Requests
Hits
Hits
+
Misses
@edge

48. Cache
Hit
Ra3o
vs.
1
-­‐
Requests
to
Origin
Total
Requests
Hits
Hits
+
Misses
@edge
Offload

49. Cache
Hit
Ra3o
vs.
1
-­‐
Requests
to
Origin
Total
Requests
Hits
Hits
+
Misses
@edge
Offload
Performance

50. Effect
on
long
tail
content

51. Long
Tail:
Cacheable
but
seldom
fetched

52. Popular
Medium
Tail
(1hr)
Long
tail
(6hr)

53. Popular
Medium
Tail
(1hr)
Long
tail
(6hr)
Connect
(median)
Popular
14msec
1hr
Tail
15msec
6hr
Tail
16msec

54. Popular
Medium
Tail
(1hr)
Long
tail
(6hr)
Connect
(median)
Popular
14msec
1hr
Tail
15msec
6hr
Tail
16msec

55. Popular
Medium
Tail
(1hr)
Long
tail
(6hr)
Connect
(median)
Wait
(median)
Popular
14msec
19msec
1hr
Tail
15msec
26msec
6hr
Tail
16msec
32msec

56. Popular
Medium
Tail
(1hr)
Long
tail
(6hr)
Connect
(median)
Wait
(median)
Popular
14msec
19msec
1hr
Tail
15msec
26msec
6hr
Tail
16msec
32msec
6,400+
measurements
77,000+
measurements
38,000+
measurements

57. Popular
Medium
Tail
(1hr)
Long
tail
(6hr)
Isn’t
this
beFer?

58. “Dynamic”
Content

59. We’ve
been
told:
Sta3c
=
Cacheable
Dynamic
=
Uncacheable

60. Cacheable
Uncacheable
Sta3c
Dynamic
How
our
sites
used
to
look

61. Cacheable
Uncacheable
Sta3c
Dynamic
How
we’re
told
our
sites
look

62. Cacheable
Uncacheable
Sta3c
Dynamic
How
our
sites
actually
look
Event
Driven

63. Event-­‐Driven
content:
Sta3c
for
unpredictable
periods
of
3me

64. Three
types
of
content
• Sta/c
– Infrequent
changes,
can
cache
for
a
long
/me
– Cache-­‐control
headers
are
good
enough
– Examples:
Images,
javascript
,
css,
etc
• Dynamic
– Can’t
cache
at
all
-­‐
must
go
to
origin
– DSA
and
transport
is
key
for
performance
– Examples:
Logins,
credit
card
transac/ons,
Ajax,
etc
• Event-­‐driven
– Sta/c,
but
unpredictably
– Cache-­‐control
headers
are
not
good
enough
– Examples:
That
news
story,
wiki
pages,
user
comments,
sports
scores,
etc

65. Three
types
of
content
• Sta/c
– Infrequent
changes,
can
cache
for
a
long
/me
– Cache-­‐control
headers
are
good
enough
– Examples:
Images,
javascript
,
css,
etc
• Dynamic
– Can’t
cache
at
all
-­‐
must
go
to
origin
– DSA
and
transport
is
key
for
performance
– Examples:
Logins,
credit
card
transac/ons,
Ajax,
etc
• Event-­‐driven
– Sta/c,
but
unpredictably
– Cache-­‐control
headers
are
not
good
enough
– Examples:
That
news
story,
wiki
pages,
user
comments,
sports
scores,
etc

66. Three
types
of
content
• Sta/c
– Infrequent
changes,
can
cache
for
a
long
/me
– Cache-­‐control
headers
are
good
enough
– Examples:
Images,
javascript
,
css,
etc
• Dynamic
– Can’t
cache
at
all
-­‐
must
go
to
origin
– DSA
and
transport
is
key
for
performance
– Examples:
Logins,
credit
card
transac/ons,
Ajax,
etc
• Event-­‐driven
– Sta/c,
but
unpredictably
– Cache-­‐control
headers
are
not
good
enough
– Examples:
That
news
story,
wiki
pages,
user
comments,
sports
scores,
etc

67. Caching
event-­‐driven
content
• Not
easy
– Classic
CDNs
con/nue
to
consider
it
the
same
as
dynamic
(Bad
caching
or
panic!)
– Need
the
right
tools
to
cache
• You
may
not
know
expiry
ahead
of
/me,
but
your
app
knows
when
things
change

68. Cache
normally
and
then
“uncache”
when
it
changes

69. Uncaching!
• Purging
– Accessible
through
API
– Granular
• Dependencies
through
labels,
tags,
or
surrogate
keys
– Must
be
instantaneous
• Ex:
news
story,
part
2

70. Instant
Purge
• Enables
caching
of
event-­‐driven
content
– Can
deal
with
unpredictable
invalida/on
– As
pages
update/publish,
purge
command
is
sent
to
CDN
• Instant?
– Instant
≠
12
minutes!
– Instant
≠
quick
acknowledgement!
– <1sec
– Determinis/c
and
predictable

71. Example:
CMS
+
Purge

72. WordPress:
Before
CDN
Node

73. WordPress:
Before
CDN
Node

74. WordPress:
Before
CDN
Node

75. WordPress:
Before
CDN
Node

76. WordPress:
Before
CDN
Node
Cache

78. WordPress:
Ajer
CDN
Node

79. WordPress:
Ajer
CDN
Node
HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 55,666
Cache-Control: Totally Long Time!

80. WordPress:
Ajer
CDN
Node
HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 55,666
Cache-Control: Totally Long Time!

81. WordPress:
Ajer
CDN
Node

82. WordPress:
Ajer
CDN
Node

83. WordPress:
Ajer
CDN
Node
PURGE

84. WordPress:
Ajer
CDN
Node
PURGE

85. WordPress:
Ajer
CDN
Node
(Has
to
be
instantaneous!)
PURGE

86. WordPress:
Ajer
CDN
Node
HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 55,666
Cache-Control: Totally Long Time!

87. WordPress:
Ajer
CDN
Node

88. Example:
customer1.js

89. Before
CDN
Node
Origin
http://www.3rdparty.com/customer1.js
(Referer: www.customer1.com)

90. Before
CDN
Node
Origin
http://www.3rdparty.com/customer1.js

91. Before
CDN
Node
Origin
HTTP/1.1 200 OK
Cache-Control: max-age=60
Last-Modified: Tue, 12 May 2015 19:51:30 GMT
Content-Type: application/javascript
Date: Thu, 25 Sep 2014 12:22:20 GMT
Server: Apache
Content-Length: 7835
http://www.3rdparty.com/customer1.js

92. Before
CDN
Node
Origin
HTTP/1.1 200 OK
Cache-Control: max-age=60
Last-Modified: Tue, 12 May 2015 19:51:30 GMT
Content-Type: application/javascript
Date: Thu, 25 Sep 2014 12:22:20 GMT
Server: Apache
Content-Length: 7835
http://www.3rdparty.com/customer1.js

93. Before
CDN
Node
Origin
HTTP/1.1 200 OK
Cache-Control: max-age=60
Last-Modified: Tue, 12 May 2015 19:51:30 GMT
Content-Type: application/javascript
Date: Thu, 25 Sep 2014 12:22:20 GMT
Server: Apache
Content-Length: 7835
http://www.3rdparty.com/customer1.js

94. Before
CDN
Node
Origin
http://www.3rdparty.com/customer1.js
(Ajer
1
min)

95. Before
CDN
Node
Origin
http://www.3rdparty.com/customer1.js VALIDATION
If-Modified-Since: Tue, 12 May 2015
19:51:30 GMT
(Ajer
1
min)

96. Before
CDN
Node
Origin
http://www.3rdparty.com/customer1.js
304 Not Modified
(Ajer
1
min)

97. Before
CDN
Node
Origin
http://www.3rdparty.com/customer1.js
304 Not Modified
(Ajer
1
min)

98. Ajer
CDN
Node
Origin
http://www.3rdparty.com/customer1.js

99. Ajer
CDN
Node
Origin
http://www.3rdparty.com/customer1.js

100. Ajer
CDN
Node
Origin
HTTP/1.1 200 OK
Cache-Control: max-age=60, s-maxage=2592000
Last-Modified: Tue, 12 May 2015 19:51:30 GMT
Content-Type: application/javascript
Date: Thu, 25 Sep 2014 12:22:20 GMT
Server: Apache
Content-Length: 7835
http://www.3rdparty.com/customer1.js

101. Ajer
CDN
Node
Origin
HTTP/1.1 200 OK
Cache-Control: max-age=60, s-maxage=2592000
Last-Modified: Tue, 12 May 2015 19:51:30 GMT
Content-Type: application/javascript
Date: Thu, 25 Sep 2014 12:22:20 GMT
Server: Apache
Content-Length: 7835
http://www.3rdparty.com/customer1.js

102. Ajer
CDN
Node
Origin
HTTP/1.1 200 OK
Cache-Control: max-age=60, s-maxage=2592000
Last-Modified: Tue, 12 May 2015 19:51:30 GMT
Content-Type: application/javascript
Date: Thu, 25 Sep 2014 12:22:20 GMT
Server: Apache
Content-Length: 7835
http://www.3rdparty.com/customer1.js

103. Ajer
CDN
Node
Origin
HTTP/1.1 200 OK
Cache-Control: max-age=60, s-maxage=2592000
Last-Modified: Tue, 12 May 2015 19:51:30 GMT
Content-Type: application/javascript
Date: Thu, 25 Sep 2014 12:22:20 GMT
Server: Apache
Content-Length: 7835
http://www.3rdparty.com/customer1.js

104. Ajer
CDN
Node
Origin
http://www.3rdparty.com/customer1.js
(Ajer
1
min)

105. Ajer
CDN
Node
Origin
http://www.3rdparty.com/customer1.js
(Ajer
1
min)

106. This
happens
many
many
3mes!
(many
many
happy
visitors!)

107. Ajer
CDN
Node
Origin
Customer1
changes
config
(Ajer
1
min)

108. Ajer
CDN
Node
Origin
PURGE
customer1.js
Customer1
changes
config
(Ajer
1
min)

109. Ajer
CDN
Node
Origin
PURGE
customer1.js
Customer1
changes
config
(Ajer
1
min)

110. Ajer
CDN
Node
Origin
http://www.3rdparty.com/customer1.js
(Ajer
1
min)

111. Ajer
CDN
Node
Origin
http://www.3rdparty.com/customer1.js
HTTP/1.1 200 OK
Cache-Control: max-age=60, s-maxage=2592000
Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT
Content-Type: application/javascript
Date: Thu, 25 Sep 2014 12:22:20 GMT
Server: Apache
Content-Length: 7835
(Ajer
1
min)

112. Ajer
CDN
Node
Origin
HTTP/1.1 200 OK
Cache-Control: max-age=60, s-maxage=2592000
Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT
Content-Type: application/javascript
Date: Thu, 25 Sep 2014 12:22:20 GMT
Server: Apache
Content-Length: 7835
http://www.3rdparty.com/customer1.js
(Ajer
1
min)

113. Control
Visibility
Cacheability

114. Programmability
• Granular
Invalida/on
API
• Configura/on
API
• Logic
at
the
Edge
• All
must
be
instantaneous
and
real
/me

116. Control
at
the
Edge
• Moving
app
logic
to
the
edge
• Ex:
VCL
(Varnish
Configura/on
Language)
– Script-­‐like
configura/on
for
func/onality
at
the
edge
• Any
mechanism
offered
by
CDN
to
let
you
do
stuff
at
the
edge
is
good

117. Logic
at
the
Edge:
Examples
• Generate
content
at
the
edge
• HTTP
header
manipula/on
• Origin
selec/on
• Caching
rules
• Geo-­‐IP
rules
• Forcing
SSL
• Serving
stale
content
– stale-­‐while-­‐revalidate,
stale-­‐while-­‐error
• Etc…

118. Example:
Edge-­‐generated
Content

119. JSON
Data
Center
ID
CDN
Node
Origin
http://www.site.com/which_datacenter.js

120. JSON
Data
Center
ID
CDN
Node
Origin
{ ‘datacenter’ : ‘LAX’ }
http://www.site.com/which_datacenter.js

121. VCL
Snippet

122. JSONP
Geo
IP
CDN
Node
Origin
myFunction(
{
“city” : “New York”,
“state”: “New York”,
“country”: “United States”,
“ip”: “173.18.14.237”
});
http://www.site.com/geo_ip.js?jsonp=myFunction

123. JSONP
Geo
IP
CDN
Node
Origin
myFunction(
{
“city” : “New York”,
“state”: “New York”,
“country”: “United States”,
“ip”: “173.18.14.237”
});
http://www.site.com/geo_ip.js?jsonp=myFunction

124. JSONP
Geo
IP
CDN
Node
Origin
somethingElse(
{
“city” : “New York”,
“state”: “New York”,
“country”: “United States”,
“ip”: “173.18.14.237”
});
http://www.site.com/geo_ip.js?jsonp=somethingElse

125. Control
Visibility
Cacheability

126. Real-­‐3me
Analy3cs
• Stats
API
• Network
stats
• HTTP
stats
(status
codes,
etc)
• Caching
stats
(hits,
misses,
errors,
etc)
• All
have
to
be
real-­‐/me
– Historic
data
is
also
a
must
for
long
term
stats
analysis

127. Logging
• Real
/me!
• Log
streaming
– To
any
logging
des/na/on
endpoint
– Ex:
S3,
FTP,
syslog,
etc

128. Example:
Beacon
Termina3on
at
the
Edge

129. Before
CDN
Node
Origin
Log
Analysis
http://collector.site.com/beacon.img?a=1&b=2&c=3

130. Before
CDN
Node
Origin
Log
Analysis
http://collector.site.com/beacon.img?a=1&b=2&c=3

131. Before
CDN
Node
Origin
Log
Analysis
HTTP/1.1 200 OK
Pragma: no-cache
Expires: Wed, 19 Apr 2000 11:43:00 GMT
Cache-Control: no-cache, no-store
Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT
Content-Type: image/gif
Date: Fri, 20 Jun 2014 12:22:20 GMT
Server: Apache
Content-Length: 35
http://collector.site.com/beacon.img?a=1&b=2&c=3

132. Before
CDN
Node
Origin
Log
Analysis
HTTP/1.1 200 OK
Pragma: no-cache
Expires: Wed, 19 Apr 2000 11:43:00 GMT
Cache-Control: no-cache, no-store
Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT
Content-Type: image/gif
Date: Fri, 20 Jun 2014 12:22:20 GMT
Server: Apache
Content-Length: 35
http://collector.site.com/beacon.img?a=1&b=2&c=3

133. Ajer
CDN
Node
Origin
http://collector.site.com/beacon.img?a=1&b=2&c=3

134. Ajer
CDN
Node
Origin
HTTP/1.1 200 OK
Pragma: no-cache
Expires: Wed, 19 Apr 2000 11:43:00 GMT
Cache-Control: no-cache, no-store
Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT
Content-Type: image/gif
Date: Fri, 20 Jun 2014 12:22:20 GMT
Server: Apache
Content-Length: 35
http://collector.site.com/beacon.img?a=1&b=2&c=3

135. Ajer
CDN
Node
Origin
HTTP/1.1 204 No Content
Date: Sat, 21 Jun 2014 23:21:12 GMT
Server: Awesome Server
Content-Length: 0
http://collector.site.com/beacon.img?a=1&b=2&c=3

136. Ajer
CDN
Node
Origin
Syslog
/
S3
/
FTP/etc
http://collector.site.com/beacon.img?a=1&b=2&c=3

137. Summary
• CDNs
should
be
an
extension
of
your
applica/ons:
– Flexible
caching,
predictable
edge
caching,
and
instant
purging
to
un-­‐cache!
– Control
at
the
edge
– Real-­‐/me
visibility
and
analy/cs
• Full
real-­‐/me
programma/c
interface
is
a
must

138. Thank
you!
hooman@fastly.com