Learn more about the tools, techniques and technologies for working productively with data at any scale. This session will introduce the family of data analytics tools on AWS which you can use to collect, compute and collaborate around data, from gigabytes to petabytes. We'll discuss Amazon Elastic MapReduce, Hadoop, structured and unstructured data, and the EC2 instance types which enable high performance analytics.

1. Big Data Analytics
Peter Sirota
General Manager, Amazon Elastic MapReduce

2. Overview
1. Introducing Big Data
2. From data to actionable information
3. Analytics and Cloud Computing
4. The Big Data ecosystem

3. 1
Introducing Big Data

4. Generation
Collection & storage
Analytics & computation
Collaboration & sharing

5. The cost of data generation
is falling

6. Lower cost,
higher throughput Generation
Collection & storage
Analytics & computation
Collaboration & sharing

7. Lower cost,
higher throughput Generation
Highly
Collection & storage constrained
Analytics & computation
Collaboration & sharing

8. Data volume
Generated data
Available for analysis
Gartner: User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011
IDC: Worldwide Business Analytics Software 2012–2016 Forecast and 2011 Vendor Shares

9. Elastic and highly scalable
+
No upfront capital expense
Remove
+ =
Only pay for what you use constraints
+
Available on-demand

10. Lower cost,
higher throughput Generation
Highly
Collection & storage constrained
Analytics & computation
Collaboration & sharing

11. Generation
Collection & storage
Accelerated
Analytics & computation
Collaboration & sharing

12. Close the gap.

13. Big Data
Technologies and techniques for
working productively with data,
at any scale.

14. 2
From data to
actionable information

15. “Who buys video games?”

16. Per day:
3.5 billion records
13 TB of click stream logs
71 million unique cookies

19. Results:
500% return on ad spend
17,000% reduction in procurement time

20. “Who is using our
service?”

21. Finding signal in the noise of logs
Identified early mobile usage
Invested heavily in mobile development

22. In January 2013
9,432,061 unique mobile devices
used the Yelp mobile app.
4 million+ calls. 5 million+ directions.

23. Open web index.
3.4 billion records.
Available to all.

24. Full parse for impact of
social networks
300 lines of Ruby code.
14 hours.
$100.

25. Tweeting about Flu
You Are What You Tweet: Analyzing Twitter for Public Health. M. J. Paul and M. Dredze, 2011

26. Tweeting about Food
Tweets about
the price of rice
Official food
price inflation

27. 3
Analytics and
Cloud Computing

28. Generation
Collection & storage
Analytics & computation
Collaboration & sharing

29. Generation
S3, Glacier,
Collection & storage Storage Gateway,
DynamoDB,
Redshift, RDS,
HBase
Analytics & computation
Collaboration & sharing

30. Generation
Collection & storage
EC2 &
Analytics & computation Elastic MapReduce
Collaboration & sharing

31. Generation
Collection & storage
Analytics & computation
EC2 & S3,
Collaboration & sharing CloudFormation,
Elastic MapReduce,
RDS, DynamoDB, Redshift

32. Generation
S3, Glacier,
Storage Gateway,
DynamoDB,
Collection & storage Redshift, RDS,
HBase
AWS Data Pipeline
EC2 &
Analytics & computation Elastic MapReduce
EC2 & S3,
Collaboration & sharing CloudFormation,
Elastic MapReduce,
RDS, DynamoDB, Redshift

33. Generation
S3, Glacier,
Storage Gateway,
DynamoDB,
Collection & storage Redshift, RDS,
HBase
AWS Data Pipeline
EC2 &
Analytics & computation Elastic MapReduce
EC2 & S3,
Collaboration & sharing CloudFormation,
Elastic MapReduce,
RDS, DynamoDB, Redshift

34. Elastic MapReduce

35. Managed Hadoop analytics

36. S3, DynamoDB, Redshift
Input data

37. S3, DynamoDB, Redshift
Input data
Code Elastic
MapReduce

38. S3, DynamoDB, Redshift
Input data
Code Elastic Name
MapReduce node

39. S3, DynamoDB, Redshift
Input data
Code Elastic Name
MapReduce node
S3/HDFS
Elastic
cluster

40. S3, DynamoDB, Redshift
Input data
Code Elastic Name
MapReduce node
Queries
S3/HDFS
+ BI
Via JDBC, Pig, Hive
Elastic
cluster

41. S3, DynamoDB, Redshift
Input data
Code Elastic Name Output
MapReduce node
Queries
S3/HDFS
+ BI
Via JDBC, Pig, Hive
Elastic
cluster

42. S3, DynamoDB, Redshift
Input data
Output

53. 1. Elastic clusters

54. 10 hours

55. 6 hours

56. Peak capacity

57. 2. Rapid, tuned provisioning

58. Tedious.

59. Remove undifferentiated
heavy lifting.

60. 3. Hadoop all the way down

61. Robust ecosystem.
Databases, machine learning, segmentation,
clustering, analytics, metadata stores,
exchange formats, and so on...

62. 4. Agility for experimentation

63. Instance choice.
Stay flexible on instance type & number.

64. 5. Cost optimizations

65. Built for Spot.
Name-your-price supercomputing.

66. 1. Elastic clusters
2. Rapid, tuned provisioning
3. Hadoop all the way down
4. Agility for experimentation.
5. Cost optimizations

67. Vin Sharma vin.sharma@intel.com
Director, Product Strategy & Marketing
Big Data Software, Intel Corporation

68. Analysis of Data Can Transform Society
Enhance scientific Create new business Increase public safety
understanding, drive models and improve and improve
innovation, and organizational energy efficiency with
accelerate medical cures. processes. smart grids.

69. Intel’s Vision to Democratize Big Data
Unlock Value in Support Open Deliver Software Value
Silicon Platforms

70. Intel at the Intersection of Big Data
HPC Cloud Open Source
Enabling exascale Helping enterprises Contributing code
computing on massive build open and fostering
data sets interoperable clouds ecosystem

71. Intel® Technology at the Heart of the Cloud
Server
Storage
Network

72. Scale-Out Big Data
Compute Platform Optimization
Cost-effective performance
•Intel® Advanced Vector Extension Technology
•Intel® Turbo Boost Technology 2.0
•Intel® Advanced Encryption Standard New
Instructions Technology

73. Intel® Advanced Vector Extensions Technology
• Newest in a long line of
processor instruction
innovations
• Increases floating point
operations per clock up to
2X1 performance
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are
measured using specific computer See backup for configuration details. software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other
1 : Performance comparison using Linpack benchmark. systems, components,
information information on performance forecasts go to http://www.intel.com/performance
For more legal
and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.
73

74. Intel® Turbo Boost Technology 2.0
More Performance
Higher turbo speeds maximize
performance for single and
multi-threaded applications

75. Intel® Advanced Encryption
Standard New Instructions
• Processor assistance for
performing AES encryption
7 new instructions
• Makes enabled encryption
software faster and stronger

76. The Power of Intel® Platform Solutions:
TeraSort for 50% Richer
1 TB sort Reduction user
experiences
4 HRS 80%
Reduction 50%
Reduction 40%
Reduction
Previous
Intel®
Xeon®
Intel®
Xeon® Solid-State
10 MIN
Processor Drive 10G
Processor
E5 2600 Ethernet Intel® Apache
Hadoop

77. The Virtuous Cycle of User Experience
Clients
Cloud
Intelligent Systems

78. 4
The Big Data
Ecosystem

79. Data, data, everywhere...
Data is stored in silos.

80. S3 HBase on EMR RDS
DynamoDB EMR Redshift
On-premises

81. “How do I get my data to the cloud?”

82. Data mobility
Generated and stored in AWS
Inbound data transfer is free
Multipart upload to S3
Physical media
AWS Direct Connect
Regional replication of AMIs and snapshots

83. “How do I integrate my data for
maximum impact?”

84. S3 HBase on EMR RDS
DynamoDB EMR Redshift
On-premises

85. S3 HBase on EMR RDS
DynamoDB EMR Redshift
On-premises

86. S3 HBase on EMR RDS
DynamoDB EMR Redshift
On premises

87. S3 HBase on EMR RDS
DynamoDB EMR Redshift
On premises

88. S3 HBase on EMR RDS
DynamoDB EMR Redshift
On premises

89. AWS Data Pipeline
Orchestration for data-intensive workloads.
Announced in November, available now.

90. AWS Data Pipeline
Data-intensive orchestration and automation
Reliable and scheduled
Easy to use, drag and drop
Execution and retry logic
Map data dependencies
Create and manage temporary compute
resources

91. Anatomy of a pipeline

92. Additional checks and notifications

93. Arbitrarily complex pipelines

94. aws.amazon.com/datapipeline

95. aws.amazon.com/big-data

96. Summary
1. Introducing Big Data
2. From data to actionable information
3. Analytics and Cloud Computing
4. The Big Data ecosystem

97. Get 600 Hours of free supercomputing
time!
www.powerof60.com

98. Thank you!
sirota@amazon.com