Presented 14-Mar-2014 at Sendai, JAWS-UG.

1. Amazon DynamoDB
Lessen's Learned by Beginner
Correct me if anything wrong
JAWS-UG Aizu Tokuno

2. About me
● TOKUNO, Hirokazu
● IT department in factory
● 4th years moved to IT at age of 39
– Studying hard to catch up
● Little know about
– VBA, SQL, JavaScript, Java, Perl, Shell...
● Like to not write code
– Pentaho ETL, Oracle APEX, Mojolicious

3. Learning/Challenge in 2013
● Unicage Development Method
● Perl, Mojolicious
● Jenkins
● Automate Test jUnit, Mockito, shUnit2
● Twitter Bootstrap, Google Maps API etc.
● Fun to attend workshop/seminar!
– efsta!
– Open Data Cafe Aizu
– JAWS-US Aizu
● Join to CODE for AIZU

4. Learning/Challenge in 2014
● Enroll to Linked Open Data Challenge 2013
● Attend Hack for Town in Aizu
● Meteor(Node.js, MongoDB) on Nitrous.IO
● GitHub Debut
● Amazon DynamoDB Right now!

5. No experience in Could

6. Created AWS account
just 7 days ago

7. What's DynamoDB?
● Fully Managed NoSQL Database Service

8. What's DynamoDB?
● Fully Managed NoSQL Database Service
What's the hell are you talking about?

9. What's DynamoDB?
● Fully Managed NoSQL Database Service
– NoSQL
– Fully Managed Database Service
Split into peaces

10. Understanding of NoSQL
● Not only SQL
● Fast to read from huge amount of data
– Accept inconsistent
● Flexible Data Structure (DynamoDB etc.)
– Attributes can be differ for each record
● Not suitable for summary and query

11. Fully Managed DB Service
● Rent database service as server
– Pay for read/write speed, storage, transaction
● Managed by Amazon
– Install, Upgrade and Patching
– Keep storage space, no limit!
– Distribute 3 data centers
– Improve performance

12. Sum up!
● Ask Amazon for boring tasks!
● Extreme performance with SSD!
● Money solve the issue!
● Not to think about storage limitation!
● High reliability by distributing at 3 data center!
● Be careful! It's not traditional database.

13. Let's try!
● Service - DynamoDB

14. Create Table
● Create Table

15. Configure Table
● Primary Key
– Only Hash or Hash/Range combination

16. Primary Key
● Hash Key : Column for partitioning
– Pick not converged column like UserID
– Do not use picky column like Game ID
● Range Key :
– Column for search
● Push into 2 columns to be unique

17. Example of Primary Keys
● Present forum in 3 tables
Table Primary Key Attributes
Hash Range
Group JAWS-UG {
LastPostBy="Kurihara"
LastPostDateTime= "2012-01-
03T00:40:57.165Z"
}
Thread JAWS-UG Session {
Message = "How about unconference?"
Tags = [ "JAWS-UG", "Aizu" ]
LastPostDateTime = "2012-01-
03T00:40:57.165Z"
}
Reply JAWS-
UG#Session
2011-12-
11T00:40:57.
165Z
{
Message = "Extreeeeeeme!"
PostedBy = "Kurihara"
}

18. Attributes
● Contents other than Hash, Range
● Each entry may have own attributes
● Data Type
– String
– Number
– Binary
– String Set (Array)
– Number Set (Array)
– Binary Set (Array)
Convert Date to string

19. Create Index
● May skip since optional

20. Index
● Upto 5 indexes can be attached for each table
– Hash-Range table is created under the table
– Cost for read/write and storage
● Able to create at table creation. No edit, add.
● Local secondary index
– Create Hash-Range table with shared Hash
Key as table, Range Key is not
● Global secondary index
– Create Hash-Range table with non shared
Hash and Range

21. Index
● Sounds working like this...
Hash Range Attribute1 Attribute2
A 1 abc def
B 2 ghi jkl
Hash Attribute2 Attribute1 Range
A def abc 1
B jkl ghi 2
Primary Key
Primary Key
Index

22. Throughput
● Read Capacity Unit
– Able to read one item per second for 4KB
● Write Capacity Unit
– Able to write one item per second for 1KB

23. Alarm
● Send notification if exceed specified threathold,
75~95%

24. Final confirmation
● As usual
¥60/month!

25. Complete!
● 3 tables for forum

26. Disadvantages
● Unable to modify Index
– Need to re-build if need to modify
– Requires cost to migrate data
● (Possibly) No system to sync with doc like, ERD
● Suitable application and not suitable
– Need to scan all records to query by othet
than Primary Key, Index
– Requires RDB for consistent transaction or
use Java Transaction Library
– Complex with RDB as recommended

27. Access from programs!
● Official SDKs
● Unofficial SDK for Perl etc.
Android Browser iOS
Java .NET Node.js
Python PHP Ruby

28. SDK API
● Table
– CreateTable
– DeleteTable
– DescribeTable
– ListTables
– UpdateTable
● Batch processing
– BatchGetItem
– BatchWriteItem
● Item
– PutItem
– GetItem
– UpdateItem
– DeleteItem
● Search
– Query : Primary
or Index
– Scan : all record

29. Let's play with DynamoDB!

30. NoSQL is for Big Data

31. Big Data for
Twitter

32. Make clear
what's interesting!

33. What is to be investigated...

34. Voluntary Restraints!!

35. Okay, okay, let's try...

36. Forum with WebSock

37. Too many samples available...
Not challenging enough

38. Create RESTfull API!

39. Like this
DynamoDBNitrous.IOWeb Client
● All free of charge

40. RESTfull API
● Implemented very simple function
– No BatchGetItem/BatchWriteItem
Action Function Method Example of URL
Create putItem POST /Group/Hash?p=1&q=A
Read scan
getItem
GET /Groups
/Groups/Hash
Update updateItem PUT /Group/Hash?p=A&q=1
Delete deleteItem DELETE /Group/Hash

41. Create by POST
● curl -i -X POST -H "Content-Type:
application/json" -d '{"datetime":"2013-03-
13T23:27:32.256Z" , "by":"Tokuno"}'
http://your_host:8080/Group/JAWS-UG

42. Read all by GET
● curl -H "Content-Type: application/json"
http://your_host:8080/Groups

43. Read by GET
● curl -H "Content-Type: application/json"
http://your_host:8080/Group/JAWS-UG
May need URI encode for Japanese

44. Update by PUT
● curl -i -X PUT -H "Content-Type:
application/json" -d '{"datetime":"2021-03-
14T00:04:01.951Z" , "by":"No one tell me..."}'
http://your_host:8080/Group/JAWS-UG

45. Delete by DELETE
● curl -i -X DELETE
http://your_host:8080/Group/JAWS-UG

46. One more thing

48. Like this
DynamoDBNitrous.IOPentaho ETL
● All free!

49. Pentaho for data processing
● Limited by DynamoDB throughput
● Pay more to boost
Group LastPostDateTime LastPostBy
あ 2013-03-13T23:27:32.256Z とくのう
い 2013-03-13T23:27:32.256Z とくのう
う 2013-03-13T23:27:32.256Z とくのう
え 2013-03-13T23:27:32.256Z とくのう
お 2013-03-13T23:27:32.256Z とくのう
か 2013-03-13T23:27:32.256Z とくのう
き 2013-03-13T23:27:32.256Z とくのう
く 2013-03-13T23:27:32.256Z とくのう
け 2013-03-13T23:27:32.256Z とくのう
こ 2013-03-13T23:27:32.256Z とくのう
あ1 2013-03-13T23:27:32.256Z とくのう
い1 2013-03-13T23:27:32.256Z とくのう
う1 2013-03-13T23:27:32.256Z とくのう
え1 2013-03-13T23:27:32.256Z とくのう
お1 2013-03-13T23:27:32.256Z とくのう
か1 2013-03-13T23:27:32.256Z とくのう
き1 2013-03-13T23:27:32.256Z とくのう

50. Summary
● Easy to maintaing since managed by Amazon!
● NoSQL is not 'Relational' but application links
– Not easy to develop
● Unable to add/modify Index
● Not flexible to query. Need to query every 1MB
● Want method to actual and document
● Only suitable for disposable application and
enterprise whom has big data?

51. Thank you!

52. Timeline
● 2014.03.08 Create account
● 2014.03.11 Attend Hack for Town
2014.03.11 Cross over in last 3 years
● 2014.03.12 Presentation for Game Lab
● 2014.03.13 Develop RESTful
2014.03.13 Play with Pentaho
● 2014.03.14 Presentation, today

53. Appendix

54. Region
● Find region used
http://docs.aws.amazon.com/general/latest/gr/rande.html

55. Procedure to find
Credentialss

56. IAM

57. Create a New Group of Users

58. Group

59. Permissions
● Select Power User Access for full access

60. Verify Policy
● Required to review although nonsense

61. Create User

62. User

63. Credentials
● Finally