Brandon Sherman, Twilio
You can’t physically touch your computing environment anymore, so how do you capture a forensic image? In this talk, learn how to take a selfie of an EC2 instance. Selfie is a tool that can jump in with an incident responder type role, trigger snapshots of a suspect instance, and copy those snapshots to a safe place. Of course, this can be automated. Did you even have to ask?
Taking a Selfie - Just Try to Resist! Doing Forensics the DevSecOps Way
1.
2.
3. B R A N D O N S H E R M A N
• Master of Information Security, Policy, and Management
• Carnegie Mellon University, December 2014
• Thesis topic: Cloud Forensics
• Presented at MirCon, October 2015 on Cloud Forensics
• Has broken many things in AWS 😈
4. TA K E A S E L F I E
J U S T T RY T O R E S I S T
5. A G E N D A
• What makes Cloud Forensics hard?
• Why do we care about forensics?
• Introduce Selfie
• Workflow
6. W H Y D O W E N E E D C L O U D F O R E N S I C S ?
• You can’t touch your hardware 🙉
• You don’t even know where your hardware is 🙈
• Storage is software-defined 📦
• Evidence of an incident needs to be preserved! %
8. W H Y D O W E N E E D F O R E N S I C S ?
• Answer questions
• What happened?
• How did it happen?
• Preserve answers 🗄
9. W H AT I S T H E O U T C O M E O F F O R E N S I C S ?
• A chain of events that can be logged and audited 📝
• Protect evidence from modification 🔏
• As automatic as possible ⚙
10. T L ; D R : T H E C L O U D
• Amazon Web Services
• EC2: Elastic Cloud Compute
• EBS: Elastic Block Storage
• S3: Simple Storage Service
• IAM: Identity & Access Management
11. E B S
• EBS annual failure rate is .1%~.4%
• Consumer drive AFR is ~4%
• The blocks must be duplicated in a redundant fashion to achieve a 10%
reduction in failure rates
• But… what happens to unallocated/overwritten blocks?
12. E B S
• EBS Snapshots create block-for-block copies in S3
• Spoiler alert: This includes deleted files 🚮🗃
• Spoiler alert: Just your files
13. R E A L TA L K : S E C U R I T Y I N C I D E N T S
• Let’s face it; 💩 happens
• Hopefully it was your Red Team 🚩
• 🔇
14. W H O Y O U G U N N A C A L L PA G E ? 👻
• We need a means to forensically preserve data on the host
• Inspection 👀
• Use in a legal environment 🕴
16. S E L F I E 📸
• Let computers do what computers are best at
• Perform a series of tasks the same way every time
• If you groan when your boss assigns you a task, it’s a prime candidate for
automation
26. S E L F I E W O R K F L O W
S T E P 0 : I D E N T I F Y I N S TA N C E S
27. S E L F I E W O R K F L O W
S T E P 0 : I D E N T I F Y I N S TA N C E S
⌖
28. I D E N T I F Y E C 2 I N S TA N C E ( S )
• ${INFORMATION_SOURCE} | ${ALERT_MECHANISM} | ${SELFIE}
• ⌖ ⇢🔔⇢📸
29. S E L F I E W O R K F L O W
S T E P 1 : R U N S E L F I E
⌖
30. S E L F I E W O R K F L O W
S T E P 2 : T R I G G E R S N A P S H O T S
⌖ /dev/sda
/dev/sdb
/dev/sdc
31. T R I G G E R S N A P S H O T S
• All EBS volumes need to be copied
• If you aren’t using EBS-backed root volumes… you should
• Ephemeral storage will be lost
• You can capture it with on-host tools
• If you still have control over the host
32. S E L F I E W O R K F L O W
S T E P 2 : T R I G G E R S N A P S H O T S
⌖ /dev/sda
/dev/sdb
/dev/sdc
33. S E L F I E W O R K F L O W
S T E P 3 : S H A R E S N A P S H O T S
⌖ /dev/sda
/dev/sdb
/dev/sdc
/dev/sda
/dev/sdb
/dev/sdc
34. S H A R E S N A P S H O T S
• AWS Accounts form a blast radius 💥
• Keeping your forensic snapshots in the same account that held a
compromised instance is not a good idea 🤔
• Did that instance hold API keys? Did it have an instance profile? 😓
35. S E L F I E W O R K F L O W
S T E P 3 : S H A R E S N A P S H O T S
⌖ /dev/sda
/dev/sdb
/dev/sdc
/dev/sda
/dev/sdb
/dev/sdc
36. S E L F I E W O R K F L O W
S T E P 4 : C O P Y S N A P S H O T S
⌖ /dev/sda
/dev/sdb
/dev/sdc
/dev/sda
/dev/sdb
/dev/sdc
/dev/sda
/dev/sdb
/dev/sdc
37. S E L F I E W O R K F L O W
S T E P 5 : C L E A N U P S N A P S H O T S
⌖ /dev/sda
/dev/sdb
/dev/sdc
/dev/sda
/dev/sdb
/dev/sdc
/dev/sda
/dev/sdb
/dev/sdc
38. S E L F I E W O R K F L O W
S T E P 5 : C L E A N U P S N A P S H O T S
⌖ /dev/sda
/dev/sdb
/dev/sdc
/dev/sda
/dev/sdb
/dev/sdc
39. S E L F I E W O R K F L O W
S T E P 5 : C L E A N U P S N A P S H O T S
⌖ /dev/sda
/dev/sdb
/dev/sdc
40. C O P Y S N A P S H O T S
• A shared snapshot doesn’t help if the original is still at risk
• Trigger a copy of the shared snapshot into your forensics account
C L E A N U P
• EBS volumes and snapshots can be deleted out of the original account 🚮
• Or keep them around if you like burning money 🔥💵
41. S E L F I E W O R K F L O W
S T E P 6 : I N V E S T I G AT E
☠ /dev/sda
/dev/sdb
/dev/sdc
42. S E L F I E W O R K F L O W
1. Identify EC2 instance(s)
2. Trigger snapshots of all EBS volumes attached to those instances
3. Share snapshots with a forensics-only AWS account
4. Trigger copies within the forensics account
5. Clean up snapshots
6. Inspect!
43. O P E N S O U R C E !
• Contributions welcome— Help
everyone take better selfies!
• https://github.com/devsecops/
selfie