Want to get ramped up on how to use Amazon's big data web services and launch your first big data application on AWS? Join us on our journey as we build a big data application in real-time using Amazon EMR, Amazon Redshift, Amazon Kinesis, Amazon DynamoDB, and Amazon S3. We review architecture design patterns for big data solutions on AWS, and give you access to a take-home lab so that you can rebuild and customize the application yourself.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
David Elliott, Solutions Architecture Manager, AWS
Anvar Karimson, Senior Application Developer, ICAP
7th July 2016
Building your First Big Data App on
AWS

2. Your First Big Data Application on AWS
PROCESS
STORE
ANALYZE & VISUALIZE
COLLECT

3. Did NASA’s STS-69 Mission Land …
… on the right homepage?

4. AccessLog - Common Log Format (CLF)
75.35.230.210 - - [20/Jul/2009:22:22:42 -0700]
"GET /images/pigtrihawk.jpg HTTP/1.1" 200 29236

5. Your First Big Data Application on AWS
PROCESS:
Amazon EMR with Spark & Hive
STORE
ANALYZE & VISUALIZE:
Amazon Redshift and Amazon QuickSight
COLLECT:
Amazon Kinesis Firehose

6. Demo
http://aws.amazon.com/big-data/use-cases/

7. London, 7 July 2016
Front Office Analytics at Scale
ICAP Front Office Technology
Anvar Karimson

8. ICAP Global Broking – Front Office Technology
• ICAP, leading markets operator and provider of post trade risk
mitigation and information services
• FTSE 250 company founded in 1986 – 30 years
• Operates in more than 60 locations in 32 countries
• More than 4000 employees
• Revenues over £1.2B
• Active investor in Fintech companies

9. The Challenge - Analytics prior to AWS
• Bespoke implementation per application
• Difficult to run analytics across applications
• Scalability-issues restricting number of
consumers
• Batch-oriented – No streaming
• On-premise solution restrictive in terms of events
that could be captured

10. The Solution – The Event Stream
• Amazon Kinesis combined with Amazon Redshift
• Standardised way of collecting, transforming, and analysing data across our estate
• Encrypted by default
• Amazon Kinesis allows for tapping into data as it is being streamed
• Empirical evidence suggests ~30 seconds between data being ingested and it being available for
queries in Amazon Redshift
• Fine-grained access/resource controls allow us to grant access to a large number of
consumers in a controlled and secure manner
• Much higher fidelity of events
• Authentication/Authorization Events
• Connection Quality Events
• Business Events
• Metrics
• Quality-of-Service
• End-to-end view of every action performed by a user across the whole estate over the complete user
lifecycle

11. Using AWS enabled
ICAP to deliver an
industrial-grade
analytics platform in
weeks, dramatically
decreasing costs
compared to an on-
premise solution
Greatest benefit of AWS

12. What’s next with AWS
• Integrate with Amazon Elastic MapReduce and
Spark Streaming for real-time analytics and
anomaly detection
• Expand the use of the Event Stream throughout
the organization
• Evaluate Amazon QuickSight as a complement
to existing business intelligence tools

13. Your First Big Data Application on AWS
PROCESS:
Amazon EMR with Spark & Hive
STORE
ANALYZE & VISUALIZE:
Amazon Redshift and Amazon QuickSight
COLLECT:
Amazon Kinesis Firehose

14. Demo
http://aws.amazon.com/big-data/use-cases/

15. Estimated pricing to run this demo is as follows:
Service Est. Cost*
Amazon EC2 $0.00
Amazon Kinesis Firehose $0.04
Amazon S3 $0.51
Amazon EMR $2.28
Amazon Redshift $0.00
Amazon QuickSight $12.00
Est. Total $14.83
Pricing estimate
*Estimated costs assumes: use of free tier where available, lower cost instances, dataset no bigger than 100MB and instances running
for 3 hours. Costs may vary depending on options selected, size of dataset, and usage.

16. DIY
Download steps: http://bit.ly/29fhcwu
http://aws.amazon.com/big-data/use-cases/

17. Please remember to rate this
session under My Agenda on
awssummit.london