In this hands-on workshop you will attack a vulnerable web application while defending your own web service behind a Fastly WAF. Attendees will depart understanding how common web application attacks can be exploited as well defended against. They will experience WAF logging and analytics via sumologic to detect attacks realtime. For mitigation you will use a preview version of our newly built WAF rule management UI. We will close off the workshop by deep diving on how our security team analyzed and mitigated some of this summer major vulnerabilities.

1. presents
Welcome to the
WAF Workshop
Jose Nazario | Security Research Director
Enrique Hernandez | SOC Manager
Rex Belli | SOC Engineer

2. A brief intro to the Fastly
WAF
Introduction Chaotic
Capture the
Flag
In some CTF’s we attack,
in Others we defend. In
chaotic ones you do both!
Recent
Vulnerabilities
Review
A walkthrough of recent
vulnerabilities released in
blackhat this year
WAF Workshop

3. WAF Workshop
Incident patterns leading to breaches
Src: Verizon 2017 Data Breach Investigations Report (DBIR)

4. Introduction
What is a Fastly WAF
WAF Workshop

5. WAF Workshop
Fastly App Security In Layers
Fastly Edge cloud
Origin Server
Real-time programmable
security policy refresh
Policy changes are live
worldwide within ms
Client-side Web
Traffic
Secured Traffic
& Origin Protection
Client Devices
& Apps
Fastly VCLFastly Platform and Presence
TLS
Acceleration
Bot
Detection
Content
Validation
DDoS
Protection
Web Application
Firewall
Edge
Authentication
Edge
Access Control

6. ● Low Latency
● Real-Time visibility
● A complete rule set
● Instant Push changes
WAF Workshop
Fastly WAF

7. WAF Workshop
1.5 to 20 ms overhead

8. WAF Workshop
Real-time logging Integrations

9. WAF Workshop
Generic, Application and Critical

10. WAF Workshop
Semi-Instant Changes

11. ● SQL Injection (SQLi)
● Cross Site Scripting
● Local File Include/Directory Traversal
● Remote File Include
● Object Injection
● Command injection
WAF Workshop
Types of Attacks it Mitigates

12. # syslog soc_weblogs log
{"syslog 7YCnicdpjTvxxxxxxx soc-weblogs :: "}
{"{"type":"req","service_id":""} req.service_id
{"","request_id":""} req.http.fastly-soc-x-
request-id {"","datacenter":""} server.datacenter
{"","client_ip":""} req.http.Fastly-Client-IP
{"","req_method":""} req.request
{"","req_uri":""} cstr_escape(req.url)
{"","req_h_user_agent":""} {"","waf_logged":""}
waf.logged {"","waf_blocked":""} waf.blocked
{"","waf_failures":""} waf.failures
{"","anomaly_score":""} waf.anomaly_score
{"","resp_status":""} resp.status
{"","resp_bytes":""} resp.bytes_written
{"","resp_header_bytes":""}
resp.header_bytes_written
{"","resp_body_bytes":""} resp.body_bytes_written
{"""} "}";
Crucial Component
Logging of Attack and Request events
WAF Workshop

13. # syslog soc_weblogs log
{"syslog 7YCnicdpjTvxxxxxxx soc-weblogs :: "}
{"{"type":"req","service_id":""} req.service_id
{"","request_id":""} req.http.fastly-soc-x-
request-id {"","datacenter":""} server.datacenter
{"","client_ip":""} req.http.Fastly-Client-IP
{"","req_method":""} req.request
{"","req_uri":""} cstr_escape(req.url)
{"","req_h_user_agent":""} {"","waf_logged":""}
waf.logged {"","waf_blocked":""} waf.blocked
{"","waf_failures":""} waf.failures
{"","anomaly_score":""} waf.anomaly_score
{"","resp_status":""} resp.status
{"","resp_bytes":""} resp.bytes_written
{"","resp_header_bytes":""}
resp.header_bytes_written
{"","resp_body_bytes":""} resp.body_bytes_written
{"""} "}";
Crucial Component
API
WAF Workshop

14. SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|REQUEST_HEADERS:User-
Agent|REQUEST_HEADERS:Referer|ARGS_NAMES|ARGS|XML:/* "@detectSQLi"
"id:942100,
phase:2,
block,
capture,
t:none,t:utf8toUnicode,t:urlDecodeUni,t:removeNulls,
msg:'SQL Injection Attack Detected via libinjection',
logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',
tag:'application-multi',
tag:'language-multi',
tag:'platform-multi',
tag:'attack-sqli',
tag:'OWASP_CRS/WEB_ATTACK/SQL_INJECTION',
tag:'WASCTC/WASC-19',
tag:'OWASP_TOP_10/A1',
tag:'OWASP_AppSensor/CIE1',
tag:'PCI/6.5.2',
ver:'OWASP_CRS/3.1.0',
severity:'CRITICAL',
multiMatch,
setvar:'tx.anomaly_score_pl1=+%{tx.critical_anomaly_score}',
setvar:'tx.sql_injection_score=+%{tx.critical_anomaly_score}',
setvar:'tx.msg=%{rule.msg}',
setvar:'tx.%{rule.id}-OWASP_CRS/WEB_ATTACK/SQL_INJECTION-%{matched_var_name}=%{matched_var}'"
Crucial Component
Rules
WAF Workshop

15. if (!req.http.fastly-soc-x-request-id)
{
set req.http.fastly-soc-x-request-id = digest.hash_sha256(now
randomstr(64) req.http.host req.url req.http.Fastly-Client-IP
server.identity);
}
Crucial Component
VCL Snippets
WAF Workshop

16. WAF UI
Sumologic WAF App
WAF Workshop
Interfaces

17. Preview Rule MGMT UI

18. Preview of WAF Rules UI

19. waflyctl .. WAF Control CLI
FTW .. Framework For Testing WAFs
WAF Workshop
Tooling

20. Capture the Flag
Contest Introduction
WAF Workshop

21. WAF Workshop
Layout
Configure
Attack/Targets
Crawl
Velites
Warden
Legatus
Team 1
Fastly Service
WAF
Juice shop 1
Team 2
Fastly Service
WAF
Juice shop 2Sumo 1 Sumo 2

22. Rules
1. Juice shop host needs to be always available
2. Only juice shops are under scope (not Fastly API or
other teams service)
3. No DDoS/DoS
4. No IP Blacklisting
5. Cannot switch all rules to blocking mode blindly

23. Scoring
1. Lowest score wins
2. Attackers increase score by grabbing flags
3. Defenders decrease score by patching vulnerabilities
4. Points will be removed for mitigating Velites attacks
5. Velites triggers random flag with in each tier at a given time:
a. Tier 1 - 15 mins
b. Tier 2- 30 mins
c. Tier 3 - 45 mins
d. Tier 4 - 1 hour
e. Tier 5 - 1 hour and 15 mins

24. WAF Workshop
Scoreboard

25. WAF Workshop

26. End to End Example

27. Instructions https://goo.gl/3DavuY
Recommended Tools:
● ZAP
● Burp
● SQLMap
WAF Workshop
Configuration 10m

28. CTF START
Start of contest
WAF Workshop

29. WINNERS
Contest winners and
prizes
WAF Workshop

30. BH/Defcon
Disclosures
What we learned,
what was most difficult to
mitigate and
what you can take away
WAF Workshop

31. WAF Workshop
We received cooperation from the researchers in these cases, which
enables us to study and mitigate these risks ahead of public disclosure.
We at Fastly greatly appreciate the back and forth with these
independent researchers.
Many Thanks to the Researchers

32. Practical Web Cache Poisoning, BHUSA 2018, James Kettle
What we learned
Zend (PHP) supports X-Rewrite-URL and X-Forwarded-URL headers
Controls backend response similar to X-Forwarded-Host does
Tons of stuff - like Drupal - inherit Zend
Confusion between caching layer and application layer
Most difficult aspect to mitigate
Can’t apply blanket VCL unset - unintended consequences possible
Exploit author at BHUSA developed a Burp Suite addition to probe for new headers, it’s endless
What you can take away
Understand your dependencies
Unset unexpected headers
Cache on expected headers
WAF Workshop
Cache Poisoning via XFU/XRU

33. Edge Side Include Injection, BHUSA 2018, Louis Dion Marcil
What we learned
ESI comments get stripped away, revealing underlying script code
This enables the attacker to bypass XSS, SQLi etc checks
Most difficult aspect to mitigate
ESI is complicated: <esi:comment /> for example, or <!-- esi --> , as well as arbitrary ESI field
names.
What you can take away
Mostly affects Apache Traffic Server, Oracle Weblogic
New WAF rules to block ESI
Minimal exposure at Fastly - ESI not enabled by default
We did find (and fix) an ESI bug, however
WAF Workshop
ESI Injection

34. CVE-5390
What we learned
Long standing bug, exasperated in Linux 4.9 with larger receive queue
Slow send rate by attacker, long (O(N)) reassembly time by server - denial of service
Instrumented kernels (via kprobes) to look for high rate of receive queue pruning
Most difficult aspect to mitigate
Rolling kernels and preserving cache hit rate
Tuning kernel receive buffers depends on a lot of factors
Dropping fragments not always possible
What you can take away
Hard to clean up on the wire, requires kernel change - config or new kernel
WAF Workshop
SegmentSmack

35. Lost and Found Certificates, Defcon 26, Ian Foster & Dylan Ayrey
What we learned
.Difference in how TLS certificates are renewed and domain names renewed
Transfer of owner of domain name to a new party without expiry of TLS certificate can lead to
security property loss
Most difficult aspect to mitigate
Shared certificates in CDN, cloud - lots of risk, get yourself on a shared cert with a high value
domain name and get it revoked, destroy cert for everyone sharing with you
What you can take away
We rate the risk as minimal
No easy answer, operational safety relies status quo for not revoking immediately on domain
transfer
Invest in a certificate you wholly own for maximum safety
WAF Workshop
BygoneSSL - Certificate
Revocation