If you've ever developed code for processing data, you know what a mess it can be—especially on Hadoop. You lack debugging tools, instant feedback, automated tests, and a sane deploy. Mortar has developed a modern framework for data processing on Hadoop and Amazon Elastic MapReduce. It is a free, open framework providing instant, step-by-step execution visibility, automated testing, reusable components, and one-button deployment. See how Mortar demonstrates this framework on Amazon EMR on a sample data set to solve a big data problem.

1. A Modern Framework for EMR
Mortar Data: K Young (CEO), Jeremy Karn (Lead Engineer)
November 15, 2013
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Friday, November 15, 13

2. K Young
Friday, November 15, 13
Jeremy Karn

3. This talk is technical
Friday, November 15, 13

4. Great: Hadoop
Greater: Amazon EMR
Friday, November 15, 13

5. A Modern Framework
Friday, November 15, 13

6. A Modern Framework
Friday, November 15, 13

7. Goal: 10x productivity
• Collaboration
• Efficient, Free Development
• Testing / Debug
• Reproducibility
• And More...
Friday, November 15, 13

8. For data science
(not a database)
Friday, November 15, 13

9. Iterate Locally
Deploy
Friday, November 15, 13
Cloud Execution
EMR + Mortar

10. Demo
Find the most popular
projects on GitHub
Friday, November 15, 13

11. Goal: Collaboration
• Easily run code from others
• Contribute changes back
Friday, November 15, 13

12. Goal: Efficient, Free Development
• Rapid iteration
• No cost
• Resilient to errors
Friday, November 15, 13

13. Goal: Testing & Debug
• Automated testing
• See what is happening at runtime
Friday, November 15, 13

14. Goal: Reproducibility
• Know what was run
• 1-button deploy
• Rollback
Friday, November 15, 13

15. Goal: Miscellaneous goals
• Easy scheduling
• Easily use other technologies: Python, Amazon
DynamoDB, MongoDB
• Results easy to locate
• Managed cluster lifetime
• More granular API
Friday, November 15, 13

16. A Modern Framework for Amazon EMR
What you saw
• Collaboration
• Efficient, Free Development
• Testing / Debug
• Reproducibility
• And More...
Friday, November 15, 13

17. Next steps
• bit.ly/mortar-reinvent
• Documentation: help.mortardata.com
• Follow: @mortardata
Friday, November 15, 13

18. Please give us your feedback on this
presentation
BDT309
As a thank you, we will select prize
winners daily for completed surveys!
Friday, November 15, 13
Thank You