Presentation by Emmanuel Bernard presented at the JBUG London Event on the 22nd of February 2012

1. jeudi 23 février 2012

2. Hibernate OGM
JPA for Infinispan and NoSQL
Emmanuel Bernard
Data Platform Architect
but actually doing things
JBoss By Red Hat
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

3. Before you leave
• JPA for NoSQL
• first key / value (Infinispan)
• Denormalization engine
• Reuse mature projects
• Influenced by the relational model
• Does queries too (gradual ramp up)
• Early phases
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

4. Emmanuel Bernard
• Hibernate
• JCP
• Ceylon
• Podcasts
• asylum.jboss.org
• lescastcodeurs.com
• The rest is at http://emmanuelbernard.com
• @emmanuelbernard
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

5. (No)SQL tour
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

6. Relational databases
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

7. Relational databases
• Data structure abstraction
• Transaction, referential integrity
• Common query language
• (Simple) type
• Proven usefulness
• tuning, backup, resilience
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

8. Relational databases
• (Some) limitations:
• planning for scale is hard
• data model changes are painful
• New needs
• limitless data for later analysis
• instant fame syndrome
• less query demanding data
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

9. NoSQL is not new
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

10. NoSQL is not new
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

11. NoSQL is not new
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

12. NoSQL alternatives
• Web “giants” needs
• Very different Goals
• data size / availability
• low latency / higher throughput
• Optimize some data access patterns
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

13. NoSQL families
• Graph oriented databases
• Key / value stores
• Document based stores key value
• BigTable-style 123 Address@23
126 “Booya”
A foo B bar C baz
{ "user" : {
1 Things
"id": "124",
2 Things C bam E coh People A Emmanuel "name": "Emmanuel",
"addresses" : [
3 Languages A C B Java C Ceylon { "city": "Paris", "country": "France" },
{ "city": "Atlanta", "country": "USA" }
]
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc. }
jeudi 23 février 2012

14. Flexibility at a cost
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

15. Flexibility at a cost
• Programming model
• no common API :(
• query (Map Reduce, specific DSL, ...)
• no schema => app driven schema
• Physical data structure transpires
• Transaction / durability / consistency
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

16. JPA for NoSQL
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

17. Demo
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

18. Goals
• Encourage new data usage patterns
• volume, types etc
• Familiar environment
• Full JPA support
• easy to jump in (and out!)
• Stop talking, act
• “PaaS on Java EE” initiative
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

19. What it does
• Today
• JPA front end for Infinispan and more
• CRUD support for @Entities
• Full-text queries
• Tomorrow
• JP-QL queries (simple ones)
• Other NoSQL
• The day after
• Complex JP-QL queries
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

20. Not a silver bullet!
• But JPA matches quite nicely
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

21. Domain model
POJO
persists indexes / searches
JPA: programmatic API
Hibernate
Hibernate Core
Search
searches / indexes
stored in
distributed
Innispan
key/value store
Innispan
Innispan
Innispan
Innispan
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

22. Domain model
POJO
persists indexes / searches
JPA: programmatic API
enables
Hibernate
Hibernate Core complex joins Teiid
Search and aggregation
searches / indexes
stored in
distributed
Innispan
key/value store
Innispan
Innispan
Innispan
Innispan
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

23. Domain model
POJO
persists indexes / searches
Search engine
JPA: programmatic API
enables
Hibernate
Hibernate Core complex joins Teiid
Search and aggregation
searches / indexes
stored in
distributed
Innispan
key/value store
Innispan
Innispan
Innispan
Innispan
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

24. Domain model
POJO
persists indexes / searches
Search engine
JPA: programmatic API
enables
Hibernate
Hibernate Core complex joins Teiid
Search and aggregation
delegates delegates
object logic to search to
Hibernate OGM JP-QL searches / indexes
Core converter stored in
persists into
distributed
Innispan
key/value store
Innispan
Innispan
Innispan
Innispan
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

25. Domain model
POJO
persists indexes / searches
Search engine
JPA: programmatic API
enables
Hibernate
Hibernate Core complex joins Teiid
Search and aggregation
delegates delegates
object logic to search to
Hibernate OGM JP-QL searches / indexes
Core converter stored in
Object/Grid Mapper
persists into
distributed
Innispan
key/value store
Innispan
Innispan
Innispan
Innispan
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

26. Domain model
POJO
NoSQL
persists indexes / searches
Search engine
JPA: programmatic API
enables
Hibernate
Hibernate Core complex joins Teiid
Search and aggregation
delegates delegates
object logic to search to
Hibernate OGM JP-QL searches / indexes
Core converter stored in
Object/Grid Mapper
persists into
distributed
Innispan
key/value store
Innispan
Innispan
Innispan
Innispan
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

27. Concepts
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

28. Schema or no schema?
• Schema-less
• developer friendly
• data structure migration?
• need strict development guidelines
• Schema
• strong documentation
• share with other apps / tooling
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

29. Entities as serialized blobs?
• Store the whole graph?
• Consistency with duplicated objects
• Concurrency / latency
• Structure change and (de)serialization
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

30. OGM’s approach
• Keep what’s best from relational model
• as much as possible
• Decorrelate object and data structure
• object model evolution
• Data stored as (self-described) tuples
• Limited set of core types
• CRUD operations are key lookups
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

31. Infinispan
• In-memory key / value store + cache store
• Data grid
• memory >> network >> local disk access
• Transactional
• JTA / XAResource
• Distributed
• virtual memory =
(sum of servers)/redundancy
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

32. Hibernate OGM’s
data structure
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

33. Storage - Entities
• Each entity in a unique key
• table name
• id column names and values
• Value is Map<String,Object>
• String: column name
• Object: simple type (serializable)
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

34. Storage - Associations
• Cannot store exactly like relational DBs
• Simulate navigation to associations
• one key per navigation
• Value is the list of tuples
• Focus on speedy reads
• writes involve several key lookups
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

35. User
Address
userId n n addressId
name
city
addresses
tbl_user tbl_user_address tbl_address
1 * 1
userId_pk * userId_fk addressId_pk
name addressId_fk city
key value
tbl_user,userId_pk,1 {userId_pk=1,name=””Emmanuel””}
tbl_user,userId_pk,2 {userId_pk=2,name=””Caroline””}
tbl_address,addressId_pk,3 {addressId_pk=3,city=””Paris””}
tbl_address,addressId_pk,5 {addressId_pk=5,city=””Atlanta””}
{ {userId_fk=1, addressId_fk=3},
tbl_user_address,userId_fk,1
{userId_fk=1, addressId_fk=5} }
tbl_user_address,userId_fk,2 { {userId_fk=2, addressId_fk=3} }
tbl_user_address,addressId_fk,5 { {userId_fk=1, addressId_fk=5} }
{ {userId_fk=1, addressId_fk=3},
tbl_user_address,addressId_fk,3
{userId_fk=2, addressId_fk=3} }
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

36. User
Address
userId n n addressId
name
city
addresses
tbl_user tbl_user_address tbl_address
1 * 1
userId_pk * userId_fk addressId_pk
name addressId_fk city
key value
tbl_user,userId_pk,1 {userId_pk=1,name=””Emmanuel””}
tbl_user,userId_pk,2 {userId_pk=2,name=””Caroline””}
tbl_address,addressId_pk,3 {addressId_pk=3,city=””Paris””}
tbl_address,addressId_pk,5 {addressId_pk=5,city=””Atlanta””}
{ {userId_fk=1, addressId_fk=3},
tbl_user_address,userId_fk,1
{userId_fk=1, addressId_fk=5} }
tbl_user_address,userId_fk,2 { {userId_fk=2, addressId_fk=3} }
tbl_user_address,addressId_fk,5 { {userId_fk=1, addressId_fk=5} }
{ {userId_fk=1, addressId_fk=3},
tbl_user_address,addressId_fk,3
{userId_fk=2, addressId_fk=3} }
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

37. User
Address
userId n n addressId
name
city
addresses
tbl_user tbl_user_address tbl_address
1 * 1
userId_pk * userId_fk addressId_pk
name addressId_fk city
key value
tbl_user,userId_pk,1 {userId_pk=1,name=””Emmanuel””}
tbl_user,userId_pk,2 {userId_pk=2,name=””Caroline””}
tbl_address,addressId_pk,3 {addressId_pk=3,city=””Paris””}
tbl_address,addressId_pk,5 {addressId_pk=5,city=””Atlanta””}
{ {userId_fk=1, addressId_fk=3},
tbl_user_address,userId_fk,1
{userId_fk=1, addressId_fk=5} }
tbl_user_address,userId_fk,2 { {userId_fk=2, addressId_fk=3} }
tbl_user_address,addressId_fk,5 { {userId_fk=1, addressId_fk=5} }
{ {userId_fk=1, addressId_fk=3},
tbl_user_address,addressId_fk,3
{userId_fk=2, addressId_fk=3} }
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

38. Queries
• Hibernate Search indexes entities
• Store Lucene indexes in Infinispan
• JP-QL to Lucene query
• Works for simple-ish queries
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

39. select a from Animal a where a.size > 20
> animalQueryBuilder
.range().onField(“size”).above(20).excludeLimit()
.createQuery();
select u from Order o join o.user u
where o.price > 100 and u.city = “Paris”
> orderQB.bool()
.must(
orderQB.range().onField(“price”)
.above(100).excludeLimit().createQuery() )
.must(
orderQB.keyword().onField(“user.city”)
.matching(“Paris”).createQuery() )
.createQuery();
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

40. Future
• More NoSQL families
• More JP-QL support
• JP-QL to “primitives”
• API for operations in bulk
• More denormalization options
• Hybrid deployment options
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

41. Hibernate OGM
• JPA for NoSQL (Infinispan initially)
• Reuse mature projects
• Relational structure
• Does queries too
• Status
• work in progress
• refining the core data structure
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

42. More info
• Documentation
• http://ogm.hibernate.org
• including reference doc
• Any good JPA book ;)
• Code
• come and contribute! It’s fun stuff
• https://github.com/hibernate/hibernate-ogm
• Q&A
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012

43. References
• Pictures under creative commons
• http://www.flickr.com/photos/tomsaint/3415333390/
• http://www.flickr.com/photos/anniewong/26473161/
• http://www.flickr.com/photos/jdhancock/5002736203/
• http://www.flickr.com/photos/liutao/280498401
• http://www.flickr.com/photos/ehw/243631365/
Copyright 2010-2012 Emmanuel Bernard and Red Hat Inc.
jeudi 23 février 2012