Security professionals and full-stack engineers will learn how to defend against distributed denial of service (DDoS) attacks and web application exploits by using automation to monitor activity, configure rate limiting, and deploy network filtering rules.
11. AWS Shield
Standard Protection Advanced Protection
Available to ALL AWS customers at
No Additional Cost
Paid service that provides additional
protections, features and benefits.
12. Benefits of AWS Shield
AWS Integration
DDoS protection without
infrastructure changes
Affordable
Don’t force unnecessary
trade-offs between cost and
availability
Flexible
Customize protections
for your applications
Always-On Detection
and Mitigation
Minimize impact on application
latency
13. AWS Shield Standard
Layer 3/4 protection
Automatic detection & mitigation
Protection from most common
attacks (SYN/UDP Floods, Reflection
Attacks, etc.)
Built into AWS services
Layer 7 protection
AWS WAF for Layer 7 DDoS attack
mitigation
Self-service & pay-as-you-go
Automatic Protection against
96% of Layer 3/4 attacks
Available globally on all internet-facing AWS services
14. AWS Shield Advanced
Additional Detection & Monitoring
Protection Against Large DDoS Attacks
Visibility Into Attack Detection & Mitigation
AWS WAF at No Additional Cost
24x7 DDoS Response Team
Cost Protection (Absorb DDoS Scaling Cost)
21. AWS Shield Advanced
Application Load Balancer Classic Load Balancer Amazon CloudFront Amazon Route 53
Available on ...
Northern Virginia (us-east-1)
Northern California (us-west-1)
Oregon (us-west-2)
Ireland (eu-west-1)
Tokyo (ap-northeast-1)
In the following regions ...
23. Types of Threats
Bad BotsDDoS Application Attacks
Content scrapers
Scanners & probes
Crawlers
SQL injection
Application exploits
Social
engineering
Sensitive data
exposureApplication
Layer
Network /
Transport
Layer
AWS WAF
24. Challenges of Web Application Firewalls
Setup is complex
and slow
Too many false
positives
Limited APIs for
automation
Expensive to
implement and
maintain
25. What is AWS WAF
Web traffic filtering
with custom rules
Malicious request
blocking
Active monitoring
and tuning
29. How Does AWS WAF Protect You?
Security
Automations
Preconfigured Protections
Highly Flexible Rule Language
30. Highly Flexible Rule Language
Quick Incident Response
Mitigations in < ~1 Min
Inspect Any Part of the Request
Security
Automations
Preconfigured
Protections
Highly Flexible Rule Language
31. Preconfigured Protections
You can get started quickly with built-in rules based on
common use-cases.
CloudFormation
template
AWS WAF Configuration
Security
Automations
Preconfigured
Protections
Highly Flexible Rules Engine
34. Private IP space in AWS
Familiar networking model
Customer-defined networking logic
Strong security controls
Private connectivity to their data centres
What customers asked for…
35. Key Features of VPC
Choosing an
address range
Setting up subnets
in Availability Zones
Creating a route to
the Internet
Authorizing traffic
to/from the VPC
Good afternoon and welcome to this session on Advanced Techniques for DDoS Mitigation and WAF defence. My name is Andrew Kane and I’m a Solutions Architect with AWS
PRE-CLOUD: Perimeter network challenging. Network engineers for security. How to architect firewalls. How to build WAF. How to protect app. How – if? – to mitigate DDoS
#1 Overprovision capacity, #2 Buy dedicated DDoS H/W, #3 Re-route traffic
Extra cost, complexity, difficult to manage, not cost-effective
AWS: do the heavy lifting, provide DDoS that is (a) automatic, (b) inline, and (c) just works under attack. You don’t worry about complexities of L3 (UDP Flood) or L4 (SYN Flood) attacks
AWS: tools to mitigate L7 (HTTP Flood), or other intrusions
So in this session, we’re going to discuss all of this in terms of the 3 AWS Services that our customers would use to mitigate DDoS attacks and to protect the perimeter of their network from intrusion and to build security into their applications.
Okay, before we start looking into how you can protect your assets from external threats, either in terms of DDoS mitigations at L3/L4 or WAF at L7 , let’s have a look at the most common threats – I think it’s important to start with a quick review of the kinds of threats that we are looking at
There are 3 different categories
DDoS at L7 (Application) or L3 (Network) or L4 (Transport, TCP/UDP protocols). These try to exhaust your resources – CPU, bandwidth or state
App-layer attacks at L7 – take advantage of vulnerabilites in the application itself, or perhaps your platform (such as PHP, Ruby). Maybe even CMS like Drupal, WordPress, etc. Or they could just flood you with proper HTTP requests
Bad Bots – crawlers, scanners, probes and content scrapers. Cause unnecessary load, might steal content, and can be expensive to manage
Philosophy of Mitigation
L3/L4 – use AWS services like CloudFront, R53 and ELB you get automatic mitigation for about 96% of DDoS attacks
L7 – different; this is shared responsibility. We’d need to know an awful lot about your app in order to automatically mitigate, so to do this properly we need to work with you. But the tools are all there for you in the WAF
This is a classic diagram – D standard for distributed, so here are countless bots or compromised systems acting in concert to attack your endpoint. Don’t forget that compromised systems now include web-cams and fridges; the scope for that first D is getting bigger every day
Goal – overrun your resources with v.high volume requests; bandwidth and back-end resources; hard for legitimate requests to get through
Result – users denied access to your site, which may be an eCommerce platform on Black Friday, because you have insufficient resources left to handle it
- 65% of DDoS attacks at L3/L4 are just volumetric – they bombard you with high RPS to try and take you offline
- 17% of those are state exhaustion attacks; e.g. SYN Flood (describe handshake).
Sometimes bad actors want to bring down a web site, using application-layer requests – a simple HTTP flood attack
Very high RPS, overwhelming your available port-80/443 connections,
Customers use WAFs to block these requests before they reach web server infrastructure using just a few simple rules
Commonly known as the OWASP Top 10
First and foremost, WAF is used to prevent Web application bugs from turning into security breaches.
WAFs are an effective tool for blocking bad actors and known back attack signatures when they occur
Natively support both cross-site scripting and SQL injection in the WAF engine
Write simple rules using the WAF’s flexible rule language to mitigate many of the other top-10 attacks
Use-Case – scraping. Bots scape your site for content or price information or to steal premium content; images, map-tiles, etc
Customers use AWS WAFs to find and block content abuse cases - I'll show an example later of how to do this
I mentioned when I first came on stage that we automatically protect you against the most common DDoS attacks, so let’s talk about this in the context of AWS Shield. We’ll talk about how we accomplish this, and what the extra services and capabilities are that we can provide.
During the discussions we are going to – hopefully, demo-gods allowing – going to do some demonstrations, and some we’ll go through quite quickly due to time constraints. But if you have any questions on how to setup these services yourself then please just seek us out after the session and we’ll be happy to walk you though it.
I’ve covered the kinds of threats that our customers are seeing.
AWS Shield covers these attacks at the network transport layers and at the application layers, including ones like
SSL Abuse – encryption key negotiation process
Slowloris - keep many conns to the target open and hold them open as long as poss by sending partial reqs
These last two are covered when you use AWS Shield specifically with Amazon Cloudfront, which is our application delivery service.
Available in two tiers: Standard and Advanced
Standard - you don’t have to do anything – nothing to configure, nothing to monitor, it’s just automatically there for you. And nothing to pay
[CLICK]
Advanced – this is an ADDITIONAL service, providing visibility, additional mitigation features, and other benefits.
Regardless of which tier you choose, we built AWS Shield with four key pillars in mind.
INTEG: No infra changes. Nothing extra: no DNS changes, no special net routing, no 3rd parties. You are protected at all times.
ALWAYS-ON: You cannot turn detection off. Mitigations just happen automatically. No phone to pick up. No one to call. We detect. We deploy. L3/L4 attacks we see are v. common with v. clear signatures
$$$: It’s always on. There’s no additional cost. There are no trade-offs – we cannot offer a better ratio of feature v. cost!
FLEX: For more advanced usage you can customise the visibility and the alerting that suits your company
Built-in detection and mitigation of Layer 3/4 attacks – all provided as standard, effective against 96% of common attacks.
”But what about the other 4%, the less common ones?”, I hear you say, such as complex L7 attacks. Customers who also want Layer 7 protections against application attacks can subscribe separately to AWS WAF.
The WAF is PAYG, and you use it where it makes sense for your application to have advanced L7 protection. You only pay for the rules you setup, against the traffic that flows towards your application. There are no huge up-front fees, and no licence fees, it’s a true PAYG model.
$5 / WebACL, $1 / Rule, $0.60 / million requests
With Shield Advanced you get a number of addition protections.
DETECT – monitor all v. baseline, look for deviations. High entropy/sig.deviation => we can detect DDoS in progress and deploy mitigations. With Sh.Adv. we do this for you – for your application. We maintain a baseline for your app, so if a deviation is from your use-case then DDoS is declared and we mitigate
PROT – large attacks (ones in the news). Provide global traffic engineering to defend yourself and ourselves (next slide)
VISIBILILTY – don’t take our work for it, look at CloudWatch metrics. Alerts will tell you attack in progress. Or not. Alarms => you can take action, notify SOC, do some app performance tests, raise tickets in ZenDesk, whatever
WAF - no additional cost if subscribed to Shield Advanced, all PAYG charges disappear
24x7 – Support. Design/build WAF, or custom mitigations for your app, call them. Also do things during incidents (next slide)
Cost – CloudFront req, R53 DNS queries, ELB/ALB scaling – all refunded if due to DDoS. DDoS no longer financial attack vector
So how do we actually do this on our network.
[Click] Internet Layer Mitigations – lg.scale attacks = traffic engineering. Some v.specific mitigation for CloudFront. Takes advantage of changes to BGP route announc., and also we control the CNAME DNS inside CloudFront. We can change BGP & DNS in concert to attempt to mitigate before it even hits the Region
Border Network – SYN floods, UDP floods, traffic sourced from susp.networks or geos not typical for your app. Currently uses DDoS mitigation technique that we internally call BlackWatch. Built on commodity H/W at a very large AWS scale. As we build out our networks, & Regions, & scale to meet customer demand, BW scales, we can monitor all of that traffic and mitigate attacks against our customers without hitting bottlenecks at the Border Network
AWS Service Mitigations – some specific to CloudFront. CloudFront can inspect traffic for these at the application layer, block those types of attack, and ensure they don’t follow through to your application
Web-Layer Mitigations – these are handled in the WAF if you create your own rules, or if you engage the DDoS response team. I will discuss a few techniques later on how you can mitigate against thing like Bad Bots and suspicious IPs
DRT – for the most sophisticated attacks you can directly engage the DRT and gain expert assistanced for mitigating those L7 attacks
Shield Advanced is available on 4 AWS services, but I concentrated on CloudFront as that provides some of the best resiliency against DDoS attacks. But you can also take advantage of it on ELB, ALB and Route53. It’s available in 5 Regions, but as CloudFront is a global service you can use CloudFront to protect your application in ANY region.
Let’s move now to AWS WAF, our web application firewall…
If you remember, during the introduction we mentioned there is a range of application level attacks
[CLIClK]
WAF can help you there. By looking into headers and payloads, it will help you mitigating the most common attacks.
Looking at traditional WAF appliances this is what we tended to see
[CLICK] Slow to setup, complex, many rules, required expertise to use them, tune them. Also messed with network to get them in in terms of VPC setup, routing rules, network policies, etc. HA scaling was also generally a nightmare. SOLVED
[CLICK] Most came with many rules you had to tune. This was hard, so often turned off! Great. ”You buy the elephant but turn on the mouse”
[CLICK] Customers want APIs – want security automation (DevSecOps), wanted WAF security rules as part of core code-base
[CLICK] Expensive – to buy, maintain and run. Hardware, licence fees, people costs – it all adds up and quickly become price prohibitive for smaller customers. We don’t see why every customer cannot have a WAF.
It was really because of these that we decided to create our own WAF for our customers, trying to remove the pain-points for you in mitigating L7 attacks
WAF development kept these customer requests in mind – 92%-95% roadmap is customer-led.
[CLICK] Allows customer to define good traffic and bad traffic profiles. Bad out. Good in. Whitelisting. Blacklisting. Not just IP
[CLICK] Good/Bad defined by the customer – malicious is differnet. WAF lets you write rules to block malicious traffic – e.g. SQL Injection, who wouldn’t block it? Also XSS, other tools for IP blacklists
[CLICK] We saw this as a key feature. Bit like the PagerDuty and Slack demo – all that was possible with metrics available in WAF (so CloudWatch, SNS, Lambda, Step Functions, etc)
Let’s go through some of the key benefits that customers have told us about AWS WAF:
[CLICK] It’s not just for detection, it’s also for mitigation. May have WAF upstream already, but need local mitigations quickly. If something gets through, you want to write a new rule to mitigate and deploy to global assets within minutes. Not hours or days. Hit submit, and within 55 seconds it will have propagated to all of your ALBs, and to all CloudFront POPs globally. 55 seconds is worst case – it’s normally more like 15-20 seconds. Vital, as DDoS attacks come and go in minutes
[CLICK] As L7 is a shared-responsibility model, the language and tools must be flexible. You can write rules against any part of the request – header, URI, content body (first 8Kb) and more
[CLICK] This was a frequent ask – for security automations is it key. You don’t want manual steps if you can avoid it – lets you deploy updated security rules globally without a security engineer being involved
[CLICK] Customers asked for some pre-configured rules – checkboxes – so we have something via CloudFormation to help against the common vulnerabilities.
Welcome to the Pyramid of Security!
ANIM: Flexible rules - Preconf protection - sec automations with Lamdba, APIG, etc
[GB]
[GB]
Traditional IT: security engineer inspects everything and updates the rules
[CLICK] Next-Gen: your app updates the rules. Given we can combine WAF with other services we can manage, maintain and update the WAF without any security engineers being informed
You can have dynamic rules based on anomalies from any data source, and use Lambda to take remedial action, which can include dynamically re-programming your global WAF fleet in under one minute.
Only because 100% API - this is real-time protection!
[AK] Let’s talk about the Amazon VPC and how customers are using it to protect the origin of their application
Customers had many demands in order to be able to move to AWS:
Wanted to move applications to the cloud but needed control over their network space
Wanted the flexibility of the cloud but needed to keep their data isolated to meet compliance needs
Wanted private connectivity from the cloud into their private network space
What VPCs gave them
[Click] VPC built to give customers logically separated private network with private IP space. PCI Layer 2 segregated network
[Click] Familiar network model essential – still need firewalls, routing tables, etc, still had to connect to on-prem
[Click] Customers get to define network routing logic – wanted us to do the heavy-lifting, but wanted control
[Click] Controls are strong - ACLs, Security Groups, egress-only IGWs for IPv6
[Click] Private connectivity in the form of IPsec VPN when VPC was released in mid-2011, followed by Direct Connect
[Click through each]
This is all easy via the AWS management console or via APIs provided by AWS
Standard 2-tier app, all in private IP space, so cannot reach and are not reachable by the internet. Inbound access is through the ALB.
Utilised Cloudfront to be the App surface, which helps mitigate attacks into the app. Users can't see web or app tier, nor can they see the ALB - they can only see CloudFront, so any attack that comes in is via CloudFront and be automatically mitigated.
[Click] Security Groups would often look like this, but this is not secure - it allows CloudFront to be bypassed, so we need to change our SGs
The more secure approach is to only allow CloudFront IP ranges to ingress. This stops anyone who knows the IP of your ALB from being able to hit it without going through CloudFront.
This is the important part - unfortunately, the service IP ranges can change.
[Click] We provide you with an IP ranges json file. You can take these IPs and generate SGs that only allow ingress from CloudFront and deny everything else.
<<You may have heard of Origin Protection-type services, common with CDN provider, but for CloudFront you don't need that kind of service, and instead just parse the JSON file you decide what services can get in to or can be reached from your VPC.>>
[Click] We provide an AWS-managed SNS topic that notifies you if the ip-ranges.json file changes
[Click] A Lambda function can be initiated by SNS when a change is detected in the ip-ranges.json file
[Click] This then takes the data in the file...
[Click] ...and automatically updates the SG of the ALB.
[Click] There is a blog post that describes how to set this up
- search for “Update Security Groups CloudFront Lambda"