May our flexibility be with you! This past quarter has seen a lot of improvements. Join us to discover how to automate lifecycle enforcement on data, how to e iciently take snapshots of containers, how to compute storage usage whatever the connector, and how to shard a single container to reach millions of contents.

1. OpenIO Summit ‘17
OIO-SDS: Much Done, More Yet to Come

2. NOW
H1’17 Q3’17 Q4’17

3. Why?
(The starting point)

4. The original pain
Applicative
Silos
Colocated
Data
Organic
Growth
Exponential
TCO
JIT provisioning
Continuous small
increments
Misc. HW
New vendor deals,
Refurbishing
Versatile
Platforms
Powering
Real Humans
Buzz effects,
Daylight rythms
Known constraints

5. Design elements
State of the art (2006): CHORD-based solutions
● Static placement of the data
● Locations *known*
Wrong choice!
● Platform change → Rebalancing!
● Poor at managing heterogeneity

6. SDS → K.I.S.S.
No ring allowed...
● Choose locations:
○ Discovery & Qualification
○ Load-Balancing
● Remember:
○ Data structure for 10^14 contents
○ → Layered: containers
○ → Distributed
2 2 2 2...
1

7. protocol://service/resource
The key to our flexibility
● Proto: Plug backends/tiers
● Service/Resource: Policies
No regrets :)
Pointers everywhere!

8. Planned, Done
H1’17

9. A question of mapping names to objects
● Swift: the ideal mapping
● S3: tmpauth, ACL, Lifecycle, ...
● FS: with a side service to map inodes
● Even for “bucket-less” flat namespaces
○ Hashed containers
○ Regex containers
○ Divided containers
Planned, Done: Standard connectors

10. Archive containers
● Backup with Range’d GET
● Restore with Chunk’d POST
Snapshot containers
● Hardlink chunks (same volumes)
● Register new pointers
● No data copied!
Planned, Done: container services

11. Encompass the customer’s use cases
A service able to list container and match prefixes
Planned, Done: oio-cb
ACCT-pics / USER
ACCT-mail / USER
ACCT-snapshot / USER-20170928
ACCT / USER-segments
+ Their shards ...
GET /v1/bill/fetch?acct=ACCT&ref=USER

12. Cope with the customer’s cardinality
● Configure a limit the number of contents in each container
○ Per NS, Account or Container
● New shard are allocated when a full occurs
○ Atomically
○ Automatically
● ~ Seamless
○ Trade-Off: Slower LIST
○ Trade-Off: Slower random GET
Planned, Done: Sharded containers

13. ● Easy tasks
○ Necessary notions already present
● Lifecycle “à-la-S3”
○ Notify the containers
○ List items
○ Match the rules (prefixes, metadata)
○ Adapt the storage policy
● Self-healing … idem
○ Notify the erroneous items
○ List the chunks
○ Check the compliance to the storage policy
Planned, Done: Lifecycle rules, Self-healing

14. Now what?
H2’17

15. Improve:
● Behavior at the limits
● Robustness, Reliability
● QoS enforcement
A cycle of QA tests & CI improvements
starts for OIO-SDS
In a short term
Q1’17
Q3’17
Q4’17
Plan Do
CheckAct
Q2’17
17.10

16. SDS is just an enabler, not the star
● Stable set of features
● Only the data matter
G4A is the new star
● SDS as a source / sink
In a longer term
2015
2017
2018
Plan Do
CheckAct
2016
SDS

17. SDS Grid For Apps

18. OpenIO Summit ‘17
What’s next
Teezly, Boring and Working Use Case:
How to save 400K per year
Solvik Blum