AWS Community Day CPH - Three problems of Terraform
An intro to Amazon Web Services (AWS)
1. An intro to AWS
15 March 2012
Fotis Stamatelopoulos
Leonidas Tsementzis
Andreas Chatzakis
AWSUG Greece
Not affiliated with, sponsored or endorsed by AWS
2. DynamoDB
Storage Gateway
SES SWF Availability Zones
EC2 SimpleDB
Reserved instance
EMR SNS AWS CloudFront
CloudFormation
SQS Elastic IP
EBS ELB VPC
RDS CloudWatch Eventually consistent read
S3
Ephemeral storage
ElastiCache Route53
3. Elastic Compute Cloud: Virtual Servers
EC2 provides resizable compute capacity in the cloud
Elastic: Obtain or release server instances in minutes
Full Control: Root access, choice of OS and specs
Secure: Firewall settings, dedicated instances, VPC
Automation: Control servers via rich API, Autoscaling
Cost Effective:
OnDemand, Reserved & Spot pricing
Pay per hour + Data Transfer
Globally Available: Regions, AZs, Elastic IPs
Virtualized: Create Amazon Machine Images (AMI)
3
7. EC2: Elastic Block Storage
Non ephemeral disk volumes for your EC2 instances.
Virtual hard drive attached to single EC2 instance
Up to 1TB each, multiple drives per instance
Persists independently from instance life
AZ specific, Replication within that AZ
Snapshot volumes, persist to Amazon S3, clone
Variable performance
Better for larger instance types
Software RAID considered good practice
Pay-as-you-go ($ * M I/O + $ * GB-month)
7
9. EC2: Elastic Load Balancing
ELB automatically distributes incoming application traffic
across multiple Amazon EC2 instances
HTTP, HTTPS, TCP, SSL, Custom
Health Checks
CNAME to alias your domain name
Sticky sessions
Nothing to manage - Automagically scales
Pay-as-you-go ($ * hours + $ * GB)
Operational metrics (latency, count) @ Cloudwatch
Usually much cheaper than running your own on EC2
9
10. Simple Storage Service: Scalable Storage
S3 allows applications to store and retrieve any amount of
data objects, at any time, from anywhere on the web.
Scalable: Unlimited Objects, up to 5TB each
Region specific
Manage via REST & SOAP – POSIX (but s3fs works)
Not a true distributed filesystem
Download via HTTP,BitTorrent – Public vs Protected
High (99.999999999%) or reduced (99.99%) durability (cheaper)
Versioning and MFA deletion protection
Good for static content, logs, backups
$ * GBs stored+$ * GBs transfered+$ * 1K requests
10
11. Cloudfront: Content Delivery Network
Cloudfront allows content distribution with low latency &
high data transfer speeds
Caches copies of content close to end users
Worldwide network of Edge locations
Download or Stream
Plays nicely with S3 (origin store)
No support for query string invalidation (yet)
11
12. Amazon Relational Database Service
RDS makes it easy to set up, operate, and scale a
relational database in the cloud.
Managed service (no server admin work!)
MySQL or Oracle
Failover on different AZ for High Availability(extra cost)
Replica(s) on same AZ for Scalability (extra cost)
Configurable settings
Scale horizontally or vertically with no downtime
Automated Backups & point-in-time recovery
Automagic replacement of faulty instance
12
13. SimpleDB: Schema-less DB Service
SimpleDB provides a highly available and flexible non-
relational datastore
Managed service (no server admin work!)
High Availability and Data Durability
Data automatically indexed, flexible querying
No schema (Domains, Items, Attributes and Values)
Everything is a string (...and padding is your friend)
Consistent or eventually consistent read requests
Powerful when combined with S3 (simpleDB for meta-
data)
10GB limit per domain (aka table)
13
14. DynamoDB: Predictable Scalable NoSQL
High performance NoSQL DB with seamless scalability
Managed service (no server admin work!)
Any amount of data, Any level of request traffic
Durable data (replicated in multiple Azs)
Guaranteed provisioned throughput
Primary Key: Hash index or Hash + ranged index
Schema less but limited query flexibility vs simpleDB
Strong Consistency, Atomic Counters
On the background: Shared nothing architecture + SSD
Plays nice with Elastic Map Reduce
14
15. ElastiCache: In-Memory Cache
Memcache as a service
Managed service (no servers to manage)
Add or remove nodes to scale cache size up or down
Automagic replacement of faulty nodes
Cloudwatch monitors memcache specific metrics
Set maintenance window for upgrades
Configurable
15
16. SES: Email Sending Service
Highly scalable and cost-effective bulk and transactional
email-sending service
Send emails via API or via SMTP
Reliable & Scalable
Monitor sending activity & deliverability
Feedback on bounce backs & spam complaints
No replacement for your mailbox
Beware of sending quota
16
17. CloudWatch: Resource Monitoring
Cloudwatch is a monitoring system for AWS cloud
resources and custom metrics
View Graphs & Statistics for AWS services
Set Alerts and automated Response Actions
Autoscaling based on cloudwatch metrics
Submit custom metrics via API
17
18. SQS: Message Queue Service
Simple Queue Service is a hosted queue service for
storing messages in a distributed environment
Reliable and scalable
Accessible via an API
Supports object locking
Authentication mechanisms
18
19. There is more!
Elastic Map – Reduce: Hadoop framework as a service
SNS: Push Notification Service
Route53: Reliable DNS hosting
IAM: Control Access to AWS resources
SWF: Workflow Service for Coordinating Application
Components
Storage Gateway: Integrates on-premises IT environments
with Cloud storage
CloudFormation: Templated AWS Resource Creation
Elastic Beanstalk: AWS Application Container
VPC: Isolated Cloud Resources
19
21. Unstructured Data Storage Comparison
Amazon S3 Amazon EBS Amazon EC2 Instance Storage
Type Object Block Block
Performance Moderate (single thread) to Very High High
High (multiple threads)
Durability of the data High Moderate Low
Elasticity/scalability Automatic Manual (by adding more No
volumes)
Availability High Moderate to High (using EBS Low
snapshots)
Persistence across instantiations Yes Yes No
Cross-instance access Yes No No
Degree of redundancy Highly redundant across multiple Redundant within an Availability Not redundant
data centers Zone
Interfaces HTTP, REST, or SOAP Access through EC2 OS/file Block device map, access
system through EC2 OS/file system
Size limits Effectivley unlimited (5TiB per 1GiB to 1TiB per volume (can 160GiB to 3.3TiB (larger
object, unlimited objects per use multiple volumes or striping instances have both larger
bucket) for larger capacities) volumes and more volumes)
Shareability across EC2 nodes Yes No No
Management Manual Manual Automatic
21
22. DBs in AWS
You need.. ….consider
A relational database service with minimal Amazon RDS, a fully managed service that offers
administration? a choice of MySQL or Oracle engines, scale
compute & storage, Multi-AZ availability and
more.
A relational database you can manage and fully control Your choice of relational AMIs on Amazon EC2
on your own? and EBS that provide scale compute & storage,
complete control over instances, and more.
A NoSQL database service for smaller datasets Amazon SimpleDB, a fully managed service that
(<10GB) with flexible query capabilities? provides a schemaless database, reliability and
more.
A fast, highly scalable NoSQL database service? Amazon DynamoDB, a fully managed service that
offers extremely fast performance, seamless
scalability and reliability, low cost and more.
22