13. Is it Big?
Community
1227 active contributors in IceHouse
more than 166 organizations
Communication
44 active official IRC channels
2887 e-mails in March
15. Is it about data?
CERN
- 3 PB per custer
- 50000 + 35000 core
PayPal 80000 VMs (according to Forbes)
16. OpenStack Big Data
OpenStack Sahara - Elastic Hadoop clusters
provisioning and management on OpenStack and elastic data
processing (on-demand Hadoop job workflow)
OpenStack MagentoDB - key-value storage service
17. What is MagnetoDB?
MagnetoDB - OpenSource Amazon DynamoDB API
implementation for OpenStack, the key-value database service
for storing any amount of data with seamless scalability and
predictable performance.
18. MagnetoDB key features
● Easy-to-integrate REST-like API (AWS SDK, boto
clients)
● OpenStack standards following REST API
● Schemaless, non-relational table-based model
● Put/get/query/scan item operations
● Eventual and strong consistency reads
● Local Secondary indexes
● Batch read/write operations
● Designed to handle any amount of data and any level
of request traffic
● Seamless throughput and storage scaling
● Fault tolerance
19. MagnetoDB use cases
● High request traffic level applications
● Time series data
● Queries are distributed across the data
● Any other applications
● Migration of workloads between AWS and
OpenStack
23. Current status
○ 2.0.2 version released
○ Open source, Apache 2 license
○ Launchpad homepage: https://launchpad.net/magnetodb
○ Code published on github: https://github.
com/stackforge/magnetodb
○ Deployable to Vanilla OpenStack
○ Compatible with existing DynamoDB clients
24. MagnetoDB Pilot Features
● Table CRUD API
○ ListTables
○ DescribeTable
○ CreateTable
○ DeleteTable
● Item CRUD API
○ PutItem
○ UpdateItem
○ DeleteItem
○ GetItem
● Data querying API
○ Query
○ Scan
● Error Handling API
26. Next steps
● General
○ Move the development to OpenStack Infra - done
○ Integration with OpenStack Keystone - done
● MagnetoDB API, following OpenStack tenets - in progress
● DynamoDB API
○ Atomic counters
○ Asynchronous table creation
○ Atomic conditional write operations
○ Batch operations - in progress
● Other features
○ Management API
○ Quota management
27. Summary
● The 2.0.2 implementation is available on GitHub in public
repository
● MagnetoDB is compatible with AWS SDK and boto library
● It is now possible to run applications which use DynamoDB as
a storage on top of OpenStack using MagnetoDB