Database performance is a key factor for keeping your app running fast. As the amount of data and request throughput grow, it becomes harder to keep it nimble and performant. In this webinar, we will discuss using Redis, a popular NoSQL key-value store, to run your data fast at scale. Redis can be used as a cache in front of a database or as a fast in-memory data store. We will also cover how you can use ElastiCache for Redis to easily set up a large multi-terabyte Redis environment and operate it with zero management. Learning Objectives: • Understand fast data • Understand the capabilities of Amazon Elasticache for Redis to cache Relational and NoSQL databases • Use Amazon Elasticache for Redis as a primary data store Who Should Attend: • Developers, Data Architects

1. Fast Data
Acting on Information
µs are the new ms

2. What is Fast Data?
• Generated fast
• Actionable fast

3. Source: Financial Trading
Bids, asks, trades, equities, bonds, derivatives …

4. Source: Sensors
Any Thing in the Internet of Things
• Temperature
• Location
• Velocity
• Acceleration
• Radiation
• Electricity

5. Source: Clickstream
You click,
someone
tracks…

6. Source: Log Aggregation
• Servers
• Storage
• Applications
• Network
• Security

7. Capturing the Data
AWS
IoT
IoT Device Amazon
ElasticsearchAWS
Lambda
Hot Data
Amazon
ElastiCache
Amazon
DynamoDB
Longer
Retention
Data Lake
Amazon
S3
Amazon Glacier
Cold Data
Amazon
KinesisFirehose
7
Data Sources
1
Amazon Kinesis
Streams

8. 8
In-Memory Key-Value Store
High-performance
Redis and Memcached
Fully managed; Zero admin
Highly Available and Reliable
Hardened by Amazon

9. Amazon
RDS
Amazon S3
Request Rate
High Low
Latency
Low High
Data Volume
Low High
Amazon
Glacier
Amazon
CloudSearch & Elasticsearch
Structure
Low
High
Amazon
DynamoDB
Amazon
ElastiCache
HDFS

10. Redis – the fast in-memory database
Powerful
~200 commands + Lua scripting
In-memory database
Utility data structures
strings, lists, hashes, sets, sorted sets,
bitmaps & HyperLogLogs
Simple
Atomic operations
supports transactions
has ACID properties
Ridiculously fast!
<500microsecond latency for
most commands
Highly Available
replication
Persistent
snapshots or append-only log
Open Source
10

11. One Simple Example ….
+
on-demand c3.8xlarge
3,000,000 objects
+
100 bytes each
memtier benchmark
50 read : 50 write
1,228,432 TPS
Source: http://highscalability.com/blog/2014/8/27/the-12m-opssec-
redis-cloud-cluster-single-server-unbenchmar.html

12. 12
Fully managed service = Automated Operations
Redis datastore hosted on Amazon EC2 Amazon ElastiCache for Redis

13. ElastiCache for Redis Key Attributes
Extreme
Performance
Sub-millisecond
access latencies
4.5 million
writes / sec
20 million
reads / sec
Open Source
Compatible
Compatible with
Open Source
Redis
Works with any
Redis Client
Fully
Managed
Automates
node
replacement,
software
patching,
upgrades and
backups
CloudWatch
automates
monitoring of
cache
performance
metrics
Secure and
Hardened
Supports
Amazon VPC
and IAM for
secure and fine
grained access.
Monitors nodes
and applies
security patches
when needed
Highly
Available and
Scalable
Clusters of up to
15 shards, each
with a primary
node and up to
5 replica nodes.
Multi-AZ with
rapid automatic
failover, with no
human
intervention or
code changes
needed.
Cost Effective
Pay as low as
US$0.017 per
hour. Get
started with
AWS Free Tier.
Zero data
transfer costs
for cross-AZ
replication

14. Amazon
ElastiCache
Enhanced Redis Engine on ElastiCache
Hardened by Amazon
RDB Without Fork
•Mitigate the risk of increased swap
usage during syncs and snapshots.
Dynamic write throttling
•To improve output buffer
management when the node’s
memory is close to being exhausted
Smoother failovers
•Clusters recover faster as replicas will
avoid flushing their data to do a full
re-sync with the primary.

15. GEO Commands Store and retrieve longitude, latitude and
radius as a Sorted Set
• GEOADD
• GEODIST
• GEOHASH
• GEOPOS
• GEORADIUS
• GEORADIUSBYMEMBER
15
Amazon ElastiCache for Redis

16. BITFIELD
• Memory-efficient approach to
storing many small integers
as a large bitmap
• Treats a Redis string as an array of bits
• GET, SET on INCRBY any group of bits within a string
• Control increment / decrement behavior with OVERFLOW
16
Amazon ElastiCache for Redis

17. Clusters! • Enables Horizontal Scaling
• In ElastiCache for Redis 3.2:
• a Cluster is one or more Shards
• a Shard is a primary node plus up to 5
replica nodes
• Up to 15 Shards per Cluster
• 15 r3.8xlarge = 3.5TiB!
• Cluster-level backup and restore
17
S5
S1
S2
S4 S3
Client
Amazon ElastiCache for Redis
Clusters!

18. Autosharding • 16384 hash slots per Cluster
• Slot for a key is CRC16 modulo {key}
• Slots are distributed across the Cluster
into Shards
• Developers must use a Redis cluster client!
• Clients are redirected to the correct shard
• Smart clients store a map
18
S5
S1
S3
S4 S3
Client
Shard S1 = slots 0 – 3276
Shard S2 = slots 3277 – 6553
Shard S3 = slots 6554 – 9829
Shard S4 = slots 9830 – 13106
Shard S5 = slots 13107 - 16383
Amazon ElastiCache for Redis

19. Faster
Recovery
• A Cluster consists of 1 – 15 Shards
• Each Shard has a primary node and up to 5
replica nodes
• In the event of a primary node outage, a replica node
in that Shard will be promoted
Primary
S1a
S1b S1c
S2a
S2b S2c
S3a
S3b S3c
Replica
Shard Shard Shard
X
S2c
S2a
Amazon ElastiCache for Redis

20. 20
Open Source Distributed Index
Managed Service using Elasticsearch and Kibana
Fully managed; Zero admin
Highly Available and Reliable
RESTful API for easy integrationAmazon
Elasticsearch
Service

21. Logstash
REST
CWL Agent
EC2 Instances
Amazon
Kinesis
Amazon
RDS
Amazon
DynamoDB
Amazon
SQS
Queue
Logstash
Cluster
Amazon Elasticsearch
Service
Amazon
CloudWatch
AWS
Lambda
AWS
CloudTrail
Access Logs
Amazon
VPC Flow
Logs
Amazon S3
bucket
AWS IoT
Amazon Kinesis
Firehose
Integration with the AWS
ecosystem
Amazon ECS

22. Using Fast Data: Finance
Fraud Detection
Regulatory Compliance

23. Using Fast Data: Retail
Recommendations

24. Using Fast Data: Utilities
Incident Response

25. Using Fast Data: Health Care
Drug Interactions

26. What’s Next?
How will Fast Data change your industry?
Featured Tools:
Amazon ElastiCache for Redis
Amazon Elasticsearch Service
Darin Briskman ~ briskman@amazon.com