In this talk I gave at IIT ITMO TechTalks 2013, I discussed how YSFlight Headquarters is using Amazon Web Services to give more reliable service to its users, while keeping costs down and maintenance time minimal. After discussing what I learned when using AWS for YSFHQ, I delved into how to get started with Amazon Web Services and some of the neat things you can do.
--
Eric Tendian (eric@tendian.io)
Web Solutions Consultant, Tendian.io
Chief Technology Officer, YSFlight Headquarters
Bachelor of Info. Tech. and Mgmt., Illinois Institute of Technology (c/o 2017)
Website: tendian.io
29. Compute
Vertical Scaling
From $0.02/hr
Elastic Compute Cloud (EC2)
Basic unit of compute capacity
Range of CPU, memory & local disk options
18 Instance types available, from micro to cluster compute
Feature
Details
Flexible
Run Windows or Linux distributions
Scalable
Wide range of instance types from micro to cluster compute
Machine Images
Full control
Secure
Configurations can be saved as machine images (AMIs) from which new instances can be created
Full root or administrator rights
Full firewall control via Security Groups
Monitoring
Publishes metrics to Cloud Watch
Inexpensive
On-demand, Reserved and Spot instance types
VM Import/Export
Import and export VM images to transfer configurations in and out of EC2
30. EC2 instance types
High I/O 4XL 60.5 GB
35 EC2 Compute Units
16 virtual cores
2*1024 GB SSD-based local instance
storage
256
Memory (GB)
32
Cluster Compute 4XL 23 GB
33.5 EC2 Compute Units
Extra Large 15 GB
8 EC2 Compute
Units
4 virtual cores
16
2
Cluster Compute 8XL 60.5
GB
88 EC2 Compute Units
Hi-Mem 2XL 34.2 GB
13 EC2 Compute
Units
4 virtual cores
Hi-Mem XL 17.1 GB
6.5 EC2 Compute
Units
2 virtual cores
64
4
Hi-Mem Cluster Compute 8XL
244 GB
88 EC2 Compute Units
16 virtual cores
240 GB SSD
Hi-Mem 4XL 68.4 GB
26 EC2 Compute
Units
8 virtual cores
128
8
High Storage 8XL 117 GB
35 EC2 Compute Units,
24 * 2 TB ephemeral
drives
10 GB Ethernet
Medium 3.7 GB,
2 EC2 Compute
Units
1 virtual core
M3 XL 15 GB
13 EC2 Compute
Units 4 virtual
cores
EBS storage only
Large 7.5 GB
4 EC2 Compute
Units
2 virtual cores
Small 1.7 GB,
1 EC2 Compute
Unit
1 virtual core
Micro 613 MB
Up to 2 ECUs (for
short bursts)
1
1
2
4
8
High-CPU Med 1.7
GB
5 EC2 Compute
Units
2 virtual cores
16
32
M3 2XL 30 GB
26 EC2 Compute
Units 8 virtual
cores
EBS storage only
Cluster GPU 4XL 22 GB
33.5 EC2 Compute Units,
2 x NVIDIA Tesla “Fermi”
M2050 GPUs
High-CPU XL 7 GB
20 EC2 Compute
Units
8 virtual cores
64
EC2 Compute Units
128
256
37. Sign up
1
2
3
4
5
You will need
Credit card information – you won’t pay unless you use resources
A telephone – on which to receive an automated security call
38. Sign up
1
2
3
4
5
You will need
Credit card information – you won’t pay unless you use resources
A telephone – on which to receive an automated security call
Best practice
Setup billing alerts so you can be notified when levels of spend are
reached
If you have existing accounts, consider using consolidated billing to bring
them together under one payment
39. Sign up
1
Free tier
2
3
4
http://aws.amazon.com/free/
750 hours of Amazon EC2 Linux/RedHat/Suse Micro Instance usage
750 hours of Amazon EC2 Microsoft Windows Server Micro Instance usage
750 hours of an Elastic Load Balancer
30 GB of Amazon Elastic Block Storage
5 GB of Amazon S3 standard storage
100 MB of storage, 5 units of write capacity, and 10 units of read capacity for Amazon DynamoDB*
25 Amazon SimpleDB Machine Hours and 1 GB of Storage
1,000 Amazon SWF workflow executions*
1,000,000 Requests of Amazon Simple Queue Service*
1,000,000 Requests, 100,000 HTTP and 1,000 email notifications for Amazon Simple Notification Service*
10 Amazon CloudWatch metrics, 10 alarms, and 1,000,000 API requests*
15 GB of bandwidth out aggregated across all AWS services
750 hours of Amazon RDS for SQL Server Micro DB Instance usage
20 GB of RDS database storage
10 million RDS I/Os
20 GB of backup storage for your automated RDS database backups and any user-initiated DB Snapshots
20 minutes of SD transcoding or 10 minutes of HD transcoding in Amazon Elastic Transcoder*
5
42. Sign up
Key pairs
1
2
3
Instance key pairs
Standard SSH RSA Key pair
Public/Private Keys
4
5
Public Key
Inserted by Amazon into
each EC2 instance that
you launch
Public key provided by AWS to EC2
instance for secure, personalized, initial,
non-generic access
Supports NIST and other security standards
for providing non-default user access
EC2
Instance
Comms secured
with private key
Private Key
Downloaded and stored
by you
43. Sign up
Key pairs
1
2
3
Instance key pairs
Standard SSH RSA Key pair
Public/Private Keys
4
5
Public Key
Inserted by Amazon into
each EC2 instance that
you launch
Public key provided by AWS to EC2
instance for secure, personalized, initial,
non-generic access
Supports NIST and other security standards
for providing non-default user access
Private keys are not
stored by AWS
EC2
Instance
Comms secured
with private key
Private Key
Downloaded and stored
by you
44. Sign up
Key pairs
1
2
3
4
5
AWS generated keys
Select your region
Create keys
Give them a name
Private key is generated and downloaded by your browser immediately
Create 1 key pair for all resources or as many as you like (e.g 1 per server type)
Import your own keys
You supply only the public key to AWS
45. Sign up
Key pairs
1
2
3
4
5
1. Linux Launch (First Boot)
1. Instance initialization scripts insert public
key into ~/.ssh/authorized_keys
2. User connects with SSH using their Private
Key
ssh –I eu-west.pem
ec2-user@publicdns.amazonaws.com
46. Sign up
Key pairs
1
2
3
4
5
1. Linux Launch (First Boot)
1. Instance initialization scripts insert public
key into ~/.ssh/authorized_keys
ssh –I eu-west.pem
ec2-user@publicdns.amazonaws.com
2. User connects with SSH using their Private
Key
You can’t log into a Linux
instance without key
47. Sign up
Key pairs
1
2
3
4
5
1. Linux Launch (First Boot)
1. Instance initialization scripts insert public
key into ~/.ssh/authorized_keys
ssh –I eu-west.pem
ec2-user@publicdns.amazonaws.com
2. User connects with SSH using their Private
Key
Don’t lose it
48. Sign up
Key pairs
1
2
3
4
5
1. Windows Launch (First Boot Sequence)
2. Instance initialization scripts:
a) Creates a random Administrator password
b) Encrypts random password with Public Key
c) Reports encrypted password to Windows System Log
3. User retrieves the encrypted password and decrypts it with their Private Key (using
AWS Console or API Call)
53. Sign up
Key pairs
Access
1
2
3
4
5
Let’s install something
sudo yum -y install httpd
Install apache web server
sudo chkconfig httpd on
Set it to run as a service
sudo /etc/init.d/httpd start
Start the web server
54. Sign up
Key pairs
Access
1
2
3
4
5
Security groups
Port 22
(SSH)
Port 80
(HTTP)
Security
Group
Name
Description
Protocol
Port range
IP Address, range, or another security group
EC2 Classic
Inbound only
instance
EC2 VPC (virtual private cloud)
Inbound and outbound
TCP UDP ICMP only
,
,
Assigned at launch
Modify anytime
Any protocol
Assigned at launch or when running
Modify anytime
66. Sign up
Key pairs
Access
Image
IAM users
1
2
3
4
5
Identity and Access Management:
Securely control access to AWS
services and resources for your
users
67. Sign up
Key pairs
Access
Image
IAM users
1
2
3
4
5
Account owner
Access to all subscribed services
Access to billing reports
Access to console, REST and SOAP APIs
IAM users/groups
Access to specific services
Access to console and/or REST APIs and/or SOAP APIs
68. Sign up
Key pairs
Access
Image
IAM users
1
2
3
4
5
Account owner
Access to all subscribed services
Access to billing reports
Access to console, REST and SOAP APIs
Master user
account – owns
payment method
Regular users
IAM users/groups
Access to specific services
Access to console and/or REST APIs and/or SOAP APIs
69. Sign up
Key pairs
Access
Image
IAM users
1
2
3
4
5
Account
Administrators
Developers
Applications
Jim
Brad
Reporting
Bob
Mark
Console
Susan
Tomcat
Kevin
70. Sign up
Key pairs
Access
Image
IAM users
1
2
3
4
5
Groups
Account
Administrators
Developers
Applications
Jim
Brad
Reporting
Bob
Mark
Console
Susan
Tomcat
Kevin
Multi-factor authentication
71. Sign up
Key pairs
Access
Image
IAM users
1
2
3
4
5
Roles
Account
Administrators
Developers
Applications
Jim
Brad
Reporting
Bob
Mark
Console
Susan
Tomcat
Kevin
AWS system entitlements
72. Sign up
Key pairs
Access
Image
IAM users
1
2
3
4
5
{
"Statement": [
{
"Effect": "Allow",
"Action": [
"elasticbeanstalk:*",
"ec2:*",
"elasticloadbalancing:*",
"autoscaling:*",
"cloudwatch:*",
"s3:*",
"sns:*"
],
"Resource": "*"
}
]
Policy driven
Declarative definition of
rights for groups
Policies control access to
AWS APIs
}
74. Next Steps
Auto Scaling
Automatic re-sizing of compute clusters
based upon demand
Elastic Load Balancing
Create highly scalable applications
Distribute load across EC2 instances in multiple
availability zones
Relational Database Service
Database-as-a-Service
No need to install or manage database instances
Scalable and fault tolerant configurations