(DEV303) Practical DynamoDB Programming in Java

1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Jason Fulghum, Development Manager – AWS SDK Karthi Thyagarajan, Enterprise Solutions Architect October 2015 DEV303 Practical DynamoDB with Java

2. What to Expect from the Session REAL-TIME PROCESSING • Lambda + DynamoDB Streams • Traditional approaches DEMO – Real-time processing w/ Lambda and DynamoDB Streams BATCH PROCESSING • Lambda-based • Traditional approaches DEMO – Batch processing w/ Lambda 1 2 3 4

3. Scenario - Game analytics service - Simple scenario, but concepts carry over for other applications - IoT - E-commerce - Connect mobile games NOTE: This session will be heavily focused on code

4. Lambda No infrastructure headaches nodeJS and Java Trigger on events 1

5. DynamoDB Streams Similar to DB triggers Change records available up to 24 hours Can process change records using λ 1

6. Near real-time processing REAL-TIME PROCESSING • Lambda + DynamoDB Streams • Traditional approaches DEMO – Real-time processing w/ Lambda and DynamoDB Streams BATCH PROCESSING • Lambda-based • Traditional approaches DEMO – Batch processing w/ Lambda 1 2 3 4

7. Near real-time processing: Key challenges  Multiple views for multiple constituents  Minimize impact on performance  Near real-time availability of data in views 1

8. Proposed architecture Main table – raw scores Player stats “BI” table (for internal use) 1 Satellite table – player info

9.  Multiple views for multiple constituents • Materialized views  Minimize impact on performance • Separate tables so no contention  Near real-time availability of data in views • Minimal delay Proposed architecture (cont’d) 1

10. Comparison: SQL-based architecture • Multiple views for multiple constituents • Joins, aggregations, group- bys, etc. • Minimize impact on performance • Potential for deadlock • Near real-time availability of data in views • Read replicas 1

11. Demo: Real-time processing REAL-TIME PROCESSING • Lambda + DynamoDB Streams • Traditional approaches DEMO – Real-time processing w/ Lambda and DynamoDB Streams BATCH PROCESSING • Lambda-based • Traditional approaches DEMO – Batch processing w/ Lambda 1 2 3 4

12. Batch processing REAL-TIME PROCESSING • Lambda + DynamoDB Streams • Traditional approaches DEMO – Real-time processing w/ Lambda and DynamoDB Streams BATCH PROCESSING • Lambda-based • Traditional approaches DEMO – Batch processing w/ Lambda 1 2 3 4

13. Batch processing • Why batch processing? • Backfill - may not have accounted for certain scenarios • Not all views need to be real-time 3

14. Batch processing: Key challenges  Timely completion – even as data set grows  Must not interfere with real-time processing 3

15. Batch processing: Architecture Raw scores “BI” table Player stats Player stats by date 3

16. Batch processing (cont’d)  Timely completion – even as data set grows  Must not interfere with real-time processing 3

17. Batch processing: Traditional approaches DB: SQL ETL May not scale w/ data set growth DB: NoSQL Impala, Spark, Presto Learning curve 3

18. Demo: Batch processing REAL-TIME PROCESSING • Lambda + DynamoDB Streams • Traditional approaches DEMO – Real-time processing w/ Lambda and DynamoDB Streams BATCH PROCESSING • Lambda-based • Traditional approaches DEMO – Batch processing w/ Lambda 1 2 3 4

19. Summary No infrastructure Low barriers to entry Force multiplier DynamoDB Streams λ +

20. Next steps - Links - AWS SDK for Java: http://aws.amazon.com/sdk-for-java/ - AWS Toolkit for Eclipse: http://aws.amazon.com/eclipse/ - AWS Java Development blog: https://java.awsblog.com/

21. Related sessions - ARC308 - The Serverless Company Using AWS Lambda: Streamlining Architecture with AWS - DAT401 - Amazon DynamoDB Deep Dive: Schema Design, Indexing, JSON, Search, and More

22. Thank you!

23. Remember to complete your evaluations!