3. Agenda
• Meaning and reasons for NoSQL
NoSQL also means RDF stores: an Android case study 2
4. Agenda
• Meaning and reasons for NoSQL
• Introducing the blueBill (Mobile) project
NoSQL also means RDF stores: an Android case study 2
5. Agenda
• Meaning and reasons for NoSQL
• Introducing the blueBill (Mobile) project
• What’s a RDF store? Why a RDF store?
NoSQL also means RDF stores: an Android case study 2
6. Agenda
• Meaning and reasons for NoSQL
• Introducing the blueBill (Mobile) project
• What’s a RDF store? Why a RDF store?
• RDF open solutions in Java: OpenSesame, Elmo
NoSQL also means RDF stores: an Android case study 2
7. Agenda
• Meaning and reasons for NoSQL
• Introducing the blueBill (Mobile) project
• What’s a RDF store? Why a RDF store?
• RDF open solutions in Java: OpenSesame, Elmo
• RDF design issues
NoSQL also means RDF stores: an Android case study 2
8. Agenda
• Meaning and reasons for NoSQL
• Introducing the blueBill (Mobile) project
• What’s a RDF store? Why a RDF store?
• RDF open solutions in Java: OpenSesame, Elmo
• RDF design issues
• Running an RDF store on Android
NoSQL also means RDF stores: an Android case study 2
9. Agenda
• Meaning and reasons for NoSQL
• Introducing the blueBill (Mobile) project
• What’s a RDF store? Why a RDF store?
• RDF open solutions in Java: OpenSesame, Elmo
• RDF design issues
• Running an RDF store on Android
• Conclusions
NoSQL also means RDF stores: an Android case study 2
11. About the speaker
• Senior Software Architect, Mentor, Technical Writer
• Fourteen years of Java Experience (JSE, JEE, JME, etc...)
• Sun Consultant since 1998, Oracle consultant since 2010
• Author of a number of open source projects
• Speaker at JavaOne, Devoxx, Jazoon, JAX Italia and other events
• Member of the NetBeans Dream Team
• Co-leader of JUG Milano
• Java.Net blogger at http://www.java.net/blogs/fabriziogiudici
NoSQL also means RDF stores: an Android case study 3
13. NoSQL? What?
• SQL: query language, also refers to “relational model” (RDBM)
NoSQL also means RDF stores: an Android case study 4
14. NoSQL? What?
• SQL: query language, also refers to “relational model” (RDBM)
• SQL/RDBM are the de-facto standards for ACID persistence
NoSQL also means RDF stores: an Android case study 4
15. NoSQL? What?
• SQL: query language, also refers to “relational model” (RDBM)
• SQL/RDBM are the de-facto standards for ACID persistence
• Tired of that?
NoSQL also means RDF stores: an Android case study 4
16. NoSQL? What?
• SQL: query language, also refers to “relational model” (RDBM)
• SQL/RDBM are the de-facto standards for ACID persistence
• Tired of that?
• OO impedance
NoSQL also means RDF stores: an Android case study 4
17. NoSQL? What?
• SQL: query language, also refers to “relational model” (RDBM)
• SQL/RDBM are the de-facto standards for ACID persistence
• Tired of that?
• OO impedance
• Scalability, elastic scalability (performance)
NoSQL also means RDF stores: an Android case study 4
18. NoSQL? What?
• SQL: query language, also refers to “relational model” (RDBM)
• SQL/RDBM are the de-facto standards for ACID persistence
• Tired of that?
• OO impedance
• Scalability, elastic scalability (performance)
• Flexibility (data schema)
NoSQL also means RDF stores: an Android case study 4
19. NoSQL? What?
• SQL: query language, also refers to “relational model” (RDBM)
• SQL/RDBM are the de-facto standards for ACID persistence
• Tired of that?
• OO impedance
• Scalability, elastic scalability (performance)
• Flexibility (data schema)
• Distributed data model, distributed computation
NoSQL also means RDF stores: an Android case study 4
20. NoSQL? What?
• SQL: query language, also refers to “relational model” (RDBM)
• SQL/RDBM are the de-facto standards for ACID persistence
• Tired of that?
• OO impedance
• Scalability, elastic scalability (performance)
• Flexibility (data schema)
• Distributed data model, distributed computation
• etc
NoSQL also means RDF stores: an Android case study 4
22. NoSQL
• Interpretations
NoSQL also means RDF stores: an Android case study 5
23. NoSQL
• Interpretations
• #1: No SQL (at all)
NoSQL also means RDF stores: an Android case study 5
24. NoSQL
• Interpretations
• #1: No SQL (at all)
• #2: Not Only SQL (maybe on top of it)
NoSQL also means RDF stores: an Android case study 5
25. NoSQL
• Interpretations
• #1: No SQL (at all)
• #2: Not Only SQL (maybe on top of it)
• ACID or not?
NoSQL also means RDF stores: an Android case study 5
26. NoSQL
Five advantages of NoSQL
• Interpretations scaling
1. Elastic
2. Big data
• #1: No SQL Goodbye DBAs (see you later?)
3.
(at all)
4. Economics
• #2: Not Only SQL data models top of
5. Flexible (maybe on it)
• ACID orFive challenges of NoSQL
not?
1. Maturity
2. Support
3. Analytics and business intelligence
4. Administration
5. Expertise
Guy Harrison
NoSQL also means RDF stores: an Android case study 5
27. NoSQL
• Interpretations
• #1: No SQL (at all)
• #2: Not Only SQL (maybe on top of it)
• ACID or not?
NoSQL also means RDF stores: an Android case study 5
28. NoSQL
• Interpretations
• #1: No SQL (at all)
• #2: Not Only SQL (maybe on top of it)
• ACID or not?
• Examples
NoSQL also means RDF stores: an Android case study 5
29. NoSQL
• Interpretations
• #1: No SQL (at all)
• #2: Not Only SQL (maybe on top of it)
• ACID or not?
• Examples
• BigTable (Google), Cassandra (Facebook), Dynamo (Amazon), ...
NoSQL also means RDF stores: an Android case study 5
30. NoSQL
• Interpretations
• #1: No SQL (at all)
• #2: Not Only SQL (maybe on top of it)
• ACID or not?
• Examples
• BigTable (Google), Cassandra (Facebook), Dynamo (Amazon), ...
• RDF stores
NoSQL also means RDF stores: an Android case study 5
31. NoSQL
• Interpretations
• #1: No SQL (at all)
• #2: Not Only SQL (maybe on top of it)
• ACID or not?
• Examples
• BigTable (Google), Cassandra (Facebook), Dynamo (Amazon), ...
• RDF stores
• This talk is only about the latter topic
NoSQL also means RDF stores: an Android case study 5
33. Introducing blueBill (Mobile)
• Take field notes of bird observations
NoSQL also means RDF stores: an Android case study 6
34. Introducing blueBill (Mobile)
• Take field notes of bird observations
• Learn about birds
NoSQL also means RDF stores: an Android case study 6
35. Introducing blueBill (Mobile)
• Take field notes of bird observations
• Learn about birds
• Aggregating many different knowledge sources
NoSQL also means RDF stores: an Android case study 6
36. Introducing blueBill (Mobile)
• Take field notes of bird observations
• Learn about birds
• Aggregating many different knowledge sources
• Collaborative, open, community-oriented
NoSQL also means RDF stores: an Android case study 6
37. Introducing blueBill (Mobile)
• Take field notes of bird observations
• Learn about birds
• Aggregating many different knowledge sources
• Collaborative, open, community-oriented
• Subtle requirements
NoSQL also means RDF stores: an Android case study 6
38. Introducing blueBill (Mobile)
• Take field notes of bird observations
• Learn about birds
• Aggregating many different knowledge sources
• Collaborative, open, community-oriented
• Subtle requirements
• Complex data structure
NoSQL also means RDF stores: an Android case study 6
39. Introducing blueBill (Mobile)
• Take field notes of bird observations
• Learn about birds
• Aggregating many different knowledge sources
• Collaborative, open, community-oriented
• Subtle requirements
• Complex data structure
• Most advanced mobile version currently running on Android
NoSQL also means RDF stores: an Android case study 6
40. Introducing blueBill (Mobile)
• Take field notes of bird observations
• Learn about birds
• Aggregating many different knowledge sources
• Collaborative, open, community-oriented
• Subtle requirements
• Complex data structure
• Most advanced mobile version currently running on Android
NoSQL also means RDF stores: an Android case study 6
42. Complex data structure
• Checklists / Taxonomies
NoSQL also means RDF stores: an Android case study 7
43. Complex data structure
• Checklists / Taxonomies
• List of species in a given geographic area
NoSQL also means RDF stores: an Android case study 7
44. Complex data structure
• Checklists / Taxonomies
• List of species in a given geographic area
• Compiled and vetted by ornithologist and birdwatcher associations
NoSQL also means RDF stores: an Android case study 7
45. Complex data structure
• Checklists / Taxonomies
• List of species in a given geographic area
• Compiled and vetted by ornithologist and birdwatcher associations
• Define official common names (with variants)
NoSQL also means RDF stores: an Android case study 7
46. Complex data structure
• Checklists / Taxonomies
• List of species in a given geographic area
• Compiled and vetted by ornithologist and birdwatcher associations
• Define official common names (with variants)
• Continuously evolving
NoSQL also means RDF stores: an Android case study 7
47. Complex data structure
• Checklists / Taxonomies
• List of species in a given geographic area
• Compiled and vetted by ornithologist and birdwatcher associations
• Define official common names (with variants)
• Continuously evolving
• Taxonomies
NoSQL also means RDF stores: an Android case study 7
48. Complex data structure
• Checklists / Taxonomies
• List of species in a given geographic area
• Compiled and vetted by ornithologist and birdwatcher associations
• Define official common names (with variants)
• Continuously evolving
• Taxonomies
• Scientific names, hierarchical
NoSQL also means RDF stores: an Android case study 7
49. Complex data structure
• Checklists / Taxonomies
• List of species in a given geographic area
• Compiled and vetted by ornithologist and birdwatcher associations
• Define official common names (with variants)
• Continuously evolving
• Taxonomies
• Scientific names, hierarchical
• Not universal consensus
NoSQL also means RDF stores: an Android case study 7
50. Complex data structure
• Checklists / Taxonomies
• List of species in a given geographic area
• Compiled and vetted by ornithologist and birdwatcher associations
• Define official common names (with variants)
• Continuously evolving
• Taxonomies
• Scientific names, hierarchical
• Not universal consensus
• Split & lump
NoSQL also means RDF stores: an Android case study 7
52. What’s RDF?
• RDF is the core of the Semantic Web family of technologies
NoSQL also means RDF stores: an Android case study 8
53. What’s RDF?
• RDF is the core of the Semantic Web family of technologies
• Semantic Web: give semantic features to the web
NoSQL also means RDF stores: an Android case study 8
54. What’s RDF?
• RDF is the core of the Semantic Web family of technologies
• Semantic Web: give semantic features to the web
• Semantic technologies != Semantic Web
NoSQL also means RDF stores: an Android case study 8
55. What’s RDF?
• RDF is the core of the Semantic Web family of technologies
• Semantic Web: give semantic features to the web
• Semantic technologies != Semantic Web
• They can be used even when the web doesn’t matter
NoSQL also means RDF stores: an Android case study 8
56. What’s RDF?
• RDF is the core of the Semantic Web family of technologies
• Semantic Web: give semantic features to the web
• Semantic technologies != Semantic Web
• They can be used even when the web doesn’t matter
• Just a better way to model data
NoSQL also means RDF stores: an Android case study 8
57. What’s RDF?
• RDF is the core of the Semantic Web family of technologies
• Semantic Web: give semantic features to the web
• Semantic technologies != Semantic Web
• They can be used even when the web doesn’t matter
• Just a better way to model data
• RDF can be useful to you even though you don’t endorse the
Semantic Web model
NoSQL also means RDF stores: an Android case study 8
59. RDF principles
• Data are modeled with a single “schema”
NoSQL also means RDF stores: an Android case study 9
60. RDF principles
• Data are modeled with a single “schema”
• subject - predicate - object (statement, triple)
NoSQL also means RDF stores: an Android case study 9
61. RDF principles
• Data are modeled with a single “schema”
• subject - predicate - object (statement, triple)
• Formal ways to represent them
NoSQL also means RDF stores: an Android case study 9
62. RDF principles
• Data are modeled with a single “schema”
• subject - predicate - object (statement, triple)
• Formal ways to represent them
• URNs, URIs, URLs
NoSQL also means RDF stores: an Android case study 9
63. RDF principles
• Data are modeled with a single “schema”
• subject - predicate - object (statement, triple)
• Formal ways to represent them
• URNs, URIs, URLs
• namespaces
NoSQL also means RDF stores: an Android case study 9
64. RDF principles
• Data are modeled with a single “schema”
• subject - predicate - object (statement, triple)
• Formal ways to represent them
• URNs, URIs, URLs
• namespaces
• RDF != XML
NoSQL also means RDF stores: an Android case study 9
65. RDF principles
• Data are modeled with a single “schema”
• subject - predicate - object (statement, triple)
• Formal ways to represent them
• URNs, URIs, URLs
• namespaces
• RDF != XML
• RDF is a model
NoSQL also means RDF stores: an Android case study 9
66. RDF principles
• Data are modeled with a single “schema”
• subject - predicate - object (statement, triple)
• Formal ways to represent them
• URNs, URIs, URLs
• namespaces
• RDF != XML
• RDF is a model
• RDF-XML is just one of the many ways to serialize RDF data
NoSQL also means RDF stores: an Android case study 9
67. Ontologies
• Standard “vocabularies” of terms with agreed-on semantics
• Identified by their own namespaces
• Can be mixed together
• Semantic equivalence
• Declaring that two concepts from two different ontologies are the same thing
• Can be enforced locally
NoSQL also means RDF stores: an Android case study 10
68. RDF examples
• Show some examples
NoSQL also means RDF stores: an Android case study 11
69. What’s a RDF store?
NoSQL also means RDF stores: an Android case study 12
70. What’s a RDF store?
• A software facility for managing data modeled in RDF
NoSQL also means RDF stores: an Android case study 12
71. What’s a RDF store?
• A software facility for managing data modeled in RDF
• Persistent storage
NoSQL also means RDF stores: an Android case study 12
72. What’s a RDF store?
• A software facility for managing data modeled in RDF
• Persistent storage
• CRUD operations
NoSQL also means RDF stores: an Android case study 12
73. What’s a RDF store?
• A software facility for managing data modeled in RDF
• Persistent storage
• CRUD operations
• SPARQL and other query languages
NoSQL also means RDF stores: an Android case study 12
74. What’s a RDF store?
• A software facility for managing data modeled in RDF
• Persistent storage
• CRUD operations
• SPARQL and other query languages
• Indipendent implementation, or sitting upon a SQL database
NoSQL also means RDF stores: an Android case study 12
75. What’s a RDF store?
• A software facility for managing data modeled in RDF
• Persistent storage
• CRUD operations
• SPARQL and other query languages
• Indipendent implementation, or sitting upon a SQL database
• Transactions, in the standard way
NoSQL also means RDF stores: an Android case study 12
76. What’s a RDF store?
• A software facility for managing data modeled in RDF
• Persistent storage
• CRUD operations
• SPARQL and other query languages
• Indipendent implementation, or sitting upon a SQL database
• Transactions, in the standard way
• Typically suitable for distributed, web-oriented queries
NoSQL also means RDF stores: an Android case study 12
77. What’s a RDF store?
• A software facility for managing data modeled in RDF
• Persistent storage
• CRUD operations
• SPARQL and other query languages
• Indipendent implementation, or sitting upon a SQL database
• Transactions, in the standard way
• Typically suitable for distributed, web-oriented queries
• Not necessarily relevant for one’s purposes
NoSQL also means RDF stores: an Android case study 12
78. Why a RDF store?
NoSQL also means RDF stores: an Android case study 13
79. Why a RDF store?
• Most important point: no schema constraints
NoSQL also means RDF stores: an Android case study 13
80. Why a RDF store?
• Most important point: no schema constraints
• “It’s easier to add a row in a table than changing a schema”
NoSQL also means RDF stores: an Android case study 13
81. Why a RDF store?
• Most important point: no schema constraints
• “It’s easier to add a row in a table than changing a schema”
• Perfectly suitable for data models that:
NoSQL also means RDF stores: an Android case study 13
82. Why a RDF store?
• Most important point: no schema constraints
• “It’s easier to add a row in a table than changing a schema”
• Perfectly suitable for data models that:
• have got a “fuzzy” schema, initially or forever
NoSQL also means RDF stores: an Android case study 13
83. Why a RDF store?
• Most important point: no schema constraints
• “It’s easier to add a row in a table than changing a schema”
• Perfectly suitable for data models that:
• have got a “fuzzy” schema, initially or forever
• are subject to a (strong) iterative development
NoSQL also means RDF stores: an Android case study 13
84. Why a RDF store?
• Most important point: no schema constraints
• “It’s easier to add a row in a table than changing a schema”
• Perfectly suitable for data models that:
• have got a “fuzzy” schema, initially or forever
• are subject to a (strong) iterative development
• need to be integrated with other data models
NoSQL also means RDF stores: an Android case study 13
85. Why a RDF store?
• Most important point: no schema constraints
• “It’s easier to add a row in a table than changing a schema”
• Perfectly suitable for data models that:
• have got a “fuzzy” schema, initially or forever
• are subject to a (strong) iterative development
• need to be integrated with other data models
• The storage model is the same thing as the serialization model
NoSQL also means RDF stores: an Android case study 13
87. Designing RDF
• RDF has got its own design rules
NoSQL also means RDF stores: an Android case study 14
88. Designing RDF
• RDF has got its own design rules
• Validation rules, best practices, etc...
NoSQL also means RDF stores: an Android case study 14
89. Designing RDF
• RDF has got its own design rules
• Validation rules, best practices, etc...
• Not necessarily easy, but may be incremental
NoSQL also means RDF stores: an Android case study 14
90. Designing RDF
• RDF has got its own design rules
• Validation rules, best practices, etc...
• Not necessarily easy, but may be incremental
• Not relevant if used internally, apart your own maintenance burden
NoSQL also means RDF stores: an Android case study 14
91. Designing RDF
• RDF has got its own design rules
• Validation rules, best practices, etc...
• Not necessarily easy, but may be incremental
• Not relevant if used internally, apart your own maintenance burden
• Very relevant if used in a distributed data scenario
NoSQL also means RDF stores: an Android case study 14
92. Designing RDF
• RDF has got its own design rules
• Validation rules, best practices, etc...
• Not necessarily easy, but may be incremental
• Not relevant if used internally, apart your own maintenance burden
• Very relevant if used in a distributed data scenario
• The big challenge is to properly reuse others’ ontologies
NoSQL also means RDF stores: an Android case study 14
93. Designing RDF
• RDF has got its own design rules
• Validation rules, best practices, etc...
• Not necessarily easy, but may be incremental
• Not relevant if used internally, apart your own maintenance burden
• Very relevant if used in a distributed data scenario
• The big challenge is to properly reuse others’ ontologies
• RDFs, OWL, Dublin Core, Darwin Core
NoSQL also means RDF stores: an Android case study 14
94. Designing RDF
• RDF has got its own design rules
• Validation rules, best practices, etc...
• Not necessarily easy, but may be incremental
• Not relevant if used internally, apart your own maintenance burden
• Very relevant if used in a distributed data scenario
• The big challenge is to properly reuse others’ ontologies
• RDFs, OWL, Dublin Core, Darwin Core
• Which ones are established and which ones not?
NoSQL also means RDF stores: an Android case study 14
96. blueBill Specs
• Integrate data from different sources
NoSQL also means RDF stores: an Android case study 15
97. blueBill Specs
• Integrate data from different sources
• Wikipedia, media data bases, other information sources
NoSQL also means RDF stores: an Android case study 15
98. blueBill Specs
• Integrate data from different sources
• Wikipedia, media data bases, other information sources
• A common model core can be defined
NoSQL also means RDF stores: an Android case study 15
99. blueBill Specs
• Integrate data from different sources
• Wikipedia, media data bases, other information sources
• A common model core can be defined
• But many things are “discovered” by working on them
NoSQL also means RDF stores: an Android case study 15
100. blueBill Specs
• Integrate data from different sources
• Wikipedia, media data bases, other information sources
• A common model core can be defined
• But many things are “discovered” by working on them
• Iterative, incremental approach
NoSQL also means RDF stores: an Android case study 15
101. blueBill Specs
• Integrate data from different sources
• Wikipedia, media data bases, other information sources
• A common model core can be defined
• But many things are “discovered” by working on them
• Iterative, incremental approach
• Manipulating a SQL schema is cumbersome
NoSQL also means RDF stores: an Android case study 15
103. OpenSesame, Elmo
• Two major players in the Java ecosystem
NoSQL also means RDF stores: an Android case study 16
104. OpenSesame, Elmo
• Two major players in the Java ecosystem
• Jena (donated by HP)
NoSQL also means RDF stores: an Android case study 16
105. OpenSesame, Elmo
• Two major players in the Java ecosystem
• Jena (donated by HP)
• OpenSesame (by Aduna)
NoSQL also means RDF stores: an Android case study 16
106. OpenSesame, Elmo
• Two major players in the Java ecosystem
• Jena (donated by HP)
• OpenSesame (by Aduna)
• Both FLOSS
NoSQL also means RDF stores: an Android case study 16
107. OpenSesame, Elmo
• Two major players in the Java ecosystem
• Jena (donated by HP)
• OpenSesame (by Aduna)
• Both FLOSS
• My preference for OpenSesame, seems more modular
NoSQL also means RDF stores: an Android case study 16
108. OpenSesame, Elmo
• Two major players in the Java ecosystem
• Jena (donated by HP)
• OpenSesame (by Aduna)
• Both FLOSS
• My preference for OpenSesame, seems more modular
• Fundamental feature for Android development...
NoSQL also means RDF stores: an Android case study 16
109. OpenSesame
• RDF store implementation in 100% Java
• Provides store persistence, query support
• Small store can be manipulated in memory (Graph)
• Regular stores can be memory-based or persistent (Repository)
• Can be backed by flat file or SQL database
NoSQL also means RDF stores: an Android case study 17
111. Elmo
• Annotation-based persistence for OpenSesame
NoSQL also means RDF stores: an Android case study 18
112. Elmo
• Annotation-based persistence for OpenSesame
• Similar (and partially compatible) with JPA
NoSQL also means RDF stores: an Android case study 18
113. Repository repository = new SailRepository(new MemoryStore());
ElmoModule elmoModule = new ElmoModule();
Elmo
SesameManagerFactory smf = new SesameManagerFactory(elmoModule, repository);
ElmoManager em = smf.createElmoManager(); // similar to EntityManager, thread-bound
////////////////////////////////
RepositoryConnection conn = repository.getConnection();
ValueFactory valueFactory = conn.getValueFactory();
RepositoryResult<Statement> statements = conn.getStatements(null,
valueFactory.createURI(...), null, false);
while (statements.hasNext())
{
Statement statement = statements.next();
Object subject = statement.getSubject();
Object object = statement.getObject();
•
// ...
} Annotation-based persistence for OpenSesame
conn.close();
• Similar (and partially compatible) with JPA
//////////////////////////////// Elmo annotations
@rdf("http://http://rs.tdwg.org/dwc/2009-12-07/terms/Taxon)
public class Taxon
{
@rdf("http://http://rs.tdwg.org/dwc/2009-12-07/terms/scientificNameID")
private String scientificNameID;
@rdf("http://http://rs.tdwg.org/dwc/2009-12-07/terms/taxonRank")
private String rank;
// getters and setters
}
NoSQL also means RDF stores: an Android case study 18
114. Running OpenSesame on Android
NoSQL also means RDF stores: an Android case study 19
115. Running OpenSesame on Android
• Q: Is Android Java? Java™? JSE? A JDK? Something else?
NoSQL also means RDF stores: an Android case study 19
116. Running OpenSesame on Android
• Q: Is Android Java? Java™? JSE? A JDK? Something else?
• Not interested (here) in legal aspects: developer’s point of view
NoSQL also means RDF stores: an Android case study 19
117. Running OpenSesame on Android
• Q: Is Android Java? Java™? JSE? A JDK? Something else?
• Not interested (here) in legal aspects: developer’s point of view
• A: It can run most of Java bytecode, with differences
NoSQL also means RDF stores: an Android case study 19
118. Running OpenSesame on Android
• Q: Is Android Java? Java™? JSE? A JDK? Something else?
• Not interested (here) in legal aspects: developer’s point of view
• A: It can run most of Java bytecode, with differences
• Obvious missing parts: UI (not a problem for RDF)
NoSQL also means RDF stores: an Android case study 19
119. Running OpenSesame on Android
• Q: Is Android Java? Java™? JSE? A JDK? Something else?
• Not interested (here) in legal aspects: developer’s point of view
• A: It can run most of Java bytecode, with differences
• Obvious missing parts: UI (not a problem for RDF)
• Not-so-obvious differences: XML, JDBC (depending also on version)
NoSQL also means RDF stores: an Android case study 19
120. Running OpenSesame on Android
• Q: Is Android Java? Java™? JSE? A JDK? Something else?
• Not interested (here) in legal aspects: developer’s point of view
• A: It can run most of Java bytecode, with differences
• Obvious missing parts: UI (not a problem for RDF)
• Not-so-obvious differences: XML, JDBC (depending also on version)
• Much, much better than JME, but not plain JSE
NoSQL also means RDF stores: an Android case study 19
122. blueBill Android issues
• Targeting Android 1.5 (still 10% of appliances)
NoSQL also means RDF stores: an Android case study 20
123. blueBill Android issues
• Targeting Android 1.5 (still 10% of appliances)
• Some missing XML APIs
NoSQL also means RDF stores: an Android case study 20
124. blueBill Android issues
• Targeting Android 1.5 (still 10% of appliances)
• Some missing XML APIs
• e.g. Duration, XMLGregorianCalendar, QName
NoSQL also means RDF stores: an Android case study 20
125. blueBill Android issues
• Targeting Android 1.5 (still 10% of appliances)
• Some missing XML APIs
• e.g. Duration, XMLGregorianCalendar, QName
• Reflection works, but Google warns it’s slow
NoSQL also means RDF stores: an Android case study 20
126. blueBill Android issues
• Targeting Android 1.5 (still 10% of appliances)
• Some missing XML APIs
• e.g. Duration, XMLGregorianCalendar, QName
• Reflection works, but Google warns it’s slow
• Things might have changed with 2.2
NoSQL also means RDF stores: an Android case study 20
127. blueBill Android issues
• Targeting Android 1.5 (still 10% of appliances)
• Some missing XML APIs
• e.g. Duration, XMLGregorianCalendar, QName
• Reflection works, but Google warns it’s slow
• Things might have changed with 2.2
• OpenSesame doesn’t support sqlLite out-of-the-box
NoSQL also means RDF stores: an Android case study 20
128. blueBill Android issues
• Targeting Android 1.5 (still 10% of appliances)
• Some missing XML APIs
• e.g. Duration, XMLGregorianCalendar, QName
• Reflection works, but Google warns it’s slow
• Things might have changed with 2.2
• OpenSesame doesn’t support sqlLite out-of-the-box
• Performance (?)
NoSQL also means RDF stores: an Android case study 20
130. Fixing XML
• Android apps can’t bundle java.*, javax.* packages
NoSQL also means RDF stores: an Android case study 21
131. Fixing XML
• Android apps can’t bundle java.*, javax.* packages
• Obvious security reasons
NoSQL also means RDF stores: an Android case study 21
132. Fixing XML
• Android apps can’t bundle java.*, javax.* packages
• Obvious security reasons
• Solution: Maven Shade plugin
NoSQL also means RDF stores: an Android case study 21
133. Fixing XML
• Android apps can’t bundle java.*, javax.* packages
• Obvious security reasons
• Solution: Maven Shade plugin
• Missing classes from Harmony into it.tidalwave.android.xml.*
NoSQL also means RDF stores: an Android case study 21
134. Fixing XML
• Android apps can’t bundle java.*, javax.* packages
• Obvious security reasons
• Solution: Maven Shade plugin
• Missing classes from Harmony into it.tidalwave.android.xml.*
• Shade plugin renames references in existing bytecode
NoSQL also means RDF stores: an Android case study 21
135. Fixing XML
• Android apps can’t bundle java.*, javax.* packages
• Obvious security reasons
• Solution: Maven Shade plugin
• Missing classes from Harmony into it.tidalwave.android.xml.*
• Shade plugin renames references in existing bytecode
• Similar solution would work with Ant
NoSQL also means RDF stores: an Android case study 21
136. Fixing XML
<groupId>it.tidalwave.bluebill</groupId>
<artifactId>it-tidalwave-android-org-openrdf-sesame-model</artifactId>
<version>2.2.3.8-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Android Patch: OpenSesame Model</name>
<dependencies>
<dependency>
<groupId>org.openrdf.sesame</groupId>
<artifactId>sesame-model</artifactId>
</dependency>
<dependency>
<groupId>it.tidalwave.bluebill</groupId>
<artifactId>it-tidalwave-android-javax-xml-datatype</artifactId>
</dependency>
• Android apps can’t bundle java.*, javax.* packages
</dependencies>
<build>
<plugins>
• Obvious security reasons
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
• Solution: Maven Shade plugin
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
•
</goals>
Missing classes from Harmony into it.tidalwave.android.xml.*
<configuration>
....
<relocations>
•
<relocation>
Shade plugin renames references in existing bytecode
<pattern>javax.xml.namespace.QName</pattern>
<shadedPattern>it.tidalwave.android.javax.xml.namespace.QName</shadedPattern>
</relocation>
•
<relocation>
Similar solution would work with Ant
<pattern>javax.xml.datatype</pattern>
<shadedPattern>it.tidalwave.android.javax.xml.datatype</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
NoSQL also means RDF stores: an Android case study 21
138. Hand-written marshalling
• Replaces Elmo reflected-based support
NoSQL also means RDF stores: an Android case study 22
139. Hand-written marshalling
• Replaces Elmo reflected-based support
• Write a custom *Marshaller and *Unmarshaller for
each entity
NoSQL also means RDF stores: an Android case study 22
140. Hand-written marshalling
• Replaces Elmo reflected-based support
• Write a custom *Marshaller and *Unmarshaller for
each entity
• Might go away
NoSQL also means RDF stores: an Android case study 22
141. Hand-written marshalling
• Replaces Elmo reflected-based support
• Write a custom *Marshaller and *Unmarshaller for
each entity
• Might go away
• If Android 2.2 proves to support fast reflection
NoSQL also means RDF stores: an Android case study 22
142. Hand-written marshalling
• Replaces Elmo reflected-based support
• Write a custom *Marshaller and *Unmarshaller for
each entity
• Might go away
• If Android 2.2 proves to support fast reflection
• When 2.2 becomes the target Android version
NoSQL also means RDF stores: an Android case study 22
143. Hand-written marshalling
public class SimpleObservationMarshaller extends AsStatementMarshaller
{
@Override @Nonnull
public void marshal (final @Nonnull As entity, final @Nonnull Context context)
{
super.marshal(entity, context);
final Observation observation = (Observation)entity;
context.addStatement(observation, Vocabulary.RDF_TYPE, Vocabulary.OBSERVATION);
context.addStatement(observation, Vocabulary.DC_DATE, observation.getDate());
context.addStatement(observation, Vocabulary.MADE_AT, observation.getLocation());
• Replaces Elmo reflected-based support
context.addStatement(observation, Vocabulary.DC_PUBLISHER, observation.getSource());
for (final Observer observer : observation.findObservers().results())
•
{
Write a custom *Marshaller and *Unmarshaller for
}
context.addStatement(observation, Vocabulary.DC_CREATOR, observer);
each entityObservationItem observationItem : observation.findObservationItems().results())
for (final
{
•
context.addStatement(observation, Vocabulary.SKOS_NARROWER, observationItem);
Might go away
}
try
• If Android 2.2 proves to support fast reflection
{
context.addStatement(observation.as(Media.class), Vocabulary.FOAF_TOPIC, observation);
}
• When 2.2 becomes the target Android version
catch (AsException e)
{
// ok, no attachment
}
}
}
NoSQL also means RDF stores: an Android case study 22
144. Hand-written marshalling
public class SimpleObservationUnmarshaller extends StatementUnmarshallerSupport
public class SimpleObservationMarshaller extends AsStatementMarshaller
{
{@Override @Nonnull
public As unmarshal (final @Nonnull List<Statement> statements, final @Nonnull Context context)
@Override @Nonnull
{
public SimpleObservationSet observationSet = (SimpleObservationSet)context.peek();
final
void marshal (final @Nonnull As entity, final @Nonnull Context context)
{
final Date date = (Date)getLiteral(statements, Vocabulary.DC_DATE);
super.marshal(entity, context);
final Location location = (Location)findEntity(context, statements, Vocabulary.MADE_AT);
final Source source = (Source)findEntity(context, statements, Vocabulary.DC_PUBLISHER);
final Observation observation = (Observation)entity;
final Set<Observer> observers = new HashSet<Observer>();
context.addStatement(observation, Vocabulary.RDF_TYPE, Vocabulary.OBSERVATION);
final Set<ObservationItem> observationItems = new HashSet<ObservationItem>();
context.addStatement(observation, Vocabulary.DC_DATE, observation.getDate());
final List<Object> extras = new ArrayList<Object>();
extras.add(new DefaultIdentifiable(Converter.valueToId(statements.get(0).getSubject())));
context.addStatement(observation, Vocabulary.MADE_AT, observation.getLocation());
• Replaces Elmo reflected-based support observation.getSource());
context.addStatement(observation, Vocabulary.DC_PUBLISHER,
final String displayName = (String)getLiteral(statements, Vocabulary.RDFS_LABEL);
if for (final!= null)
(displayName Observer observer : observation.findObservers().results())
•
{ {
Write a custom *Marshaller and *Unmarshaller for
}
}
extras.add(new DefaultDisplayable(displayName, "SimpleObservation"));
context.addStatement(observation, Vocabulary.DC_CREATOR, observer);
each entityObservationItem observationItem : observation.findObservationItems().results())
for (final Statement statement : context.findStatements(Vocabulary.FOAF_TOPIC,
for (final
{
Converter.valueToId(statements.get(0).getSubject())))
{
extras.add(context.find(statement.getSubject()));
•
context.addStatement(observation, Vocabulary.SKOS_NARROWER, observationItem);
Might go away
}
}
final SimpleObservation observation =
new SimpleObservation(observationSet, date, location, observers, observationItems, source, extras);
try
context.push(observation);
• If Android 2.2 proves to support fast reflection
{
for (final Statement statement : findStatementsWithPredicate(statements, Vocabulary.SKOS_NARROWER))
{
context.addStatement(observation.as(Media.class), Vocabulary.FOAF_TOPIC, observation);
}
observationItems.add((SimpleObservationItem)context.find(statement.getObject()));
• When 2.2 becomes the target Android version
{
}catch (AsException e)
for (final Statement statement : findStatementsWithPredicate(statements, Vocabulary.DC_CREATOR))
// ok, no attachment
{
}
observers.add((SimpleObserver)context.find(statement.getObject()));
}}
} context.pop();
return observation;
}
}
NoSQL also means RDF stores: an Android case study 22
146. Iterations
• Iteration #1: no RDF, just ad hoc code (and a few data)
NoSQL also means RDF stores: an Android case study 23
147. Iterations
• Iteration #1: no RDF, just ad hoc code (and a few data)
• Current iteration #2: RDF, OpenSesame, custom marshalling
NoSQL also means RDF stores: an Android case study 23
148. Iterations
• Iteration #1: no RDF, just ad hoc code (and a few data)
• Current iteration #2: RDF, OpenSesame, custom marshalling
• 4400+ birds, about 500.000 statements
NoSQL also means RDF stores: an Android case study 23
149. Iterations
• Iteration #1: no RDF, just ad hoc code (and a few data)
• Current iteration #2: RDF, OpenSesame, custom marshalling
• 4400+ birds, about 500.000 statements
• Statements grouped per-bird in N3 syntax, zipped
NoSQL also means RDF stores: an Android case study 23
150. Iterations
• Iteration #1: no RDF, just ad hoc code (and a few data)
• Current iteration #2: RDF, OpenSesame, custom marshalling
• 4400+ birds, about 500.000 statements
• Statements grouped per-bird in N3 syntax, zipped
• Zip provides random access
NoSQL also means RDF stores: an Android case study 23
151. Iterations
• Iteration #1: no RDF, just ad hoc code (and a few data)
• Current iteration #2: RDF, OpenSesame, custom marshalling
• 4400+ birds, about 500.000 statements
• Statements grouped per-bird in N3 syntax, zipped
• Zip provides random access
• Future iteration #3: Using OpenSesame on a single flat file
NoSQL also means RDF stores: an Android case study 23
152. Iterations
• Iteration #1: no RDF, just ad hoc code (and a few data)
• Current iteration #2: RDF, OpenSesame, custom marshalling
• 4400+ birds, about 500.000 statements
• Statements grouped per-bird in N3 syntax, zipped
• Zip provides random access
• Future iteration #3: Using OpenSesame on a single flat file
• Future iteration #4: Using OpenSesame on a single sqlLite
NoSQL also means RDF stores: an Android case study 23
153. Iterations
• Iteration #1: no RDF, just ad hoc code (and a few data)
• Current iteration #2: RDF, OpenSesame, custom marshalling
• 4400+ birds, about 500.000 statements
• Statements grouped per-bird in N3 syntax, zipped
• Zip provides random access
• Future iteration #3: Using OpenSesame on a single flat file
• Future iteration #4: Using OpenSesame on a single sqlLite
• Future iteration #5: Eventually use Elmo
NoSQL also means RDF stores: an Android case study 23
155. Challenges
• How to import (once) 500.000 statements
NoSQL also means RDF stores: an Android case study 24
156. Challenges
• How to import (once) 500.000 statements
• Given that 50” are an acceptable only-once initialization time...
NoSQL also means RDF stores: an Android case study 24
157. Challenges
• How to import (once) 500.000 statements
• Given that 50” are an acceptable only-once initialization time...
• ... it would require 0.1ms per statement (probably impossible!)
NoSQL also means RDF stores: an Android case study 24
158. Challenges
• How to import (once) 500.000 statements
• Given that 50” are an acceptable only-once initialization time...
• ... it would require 0.1ms per statement (probably impossible!)
• Writing sqlLite integration code for OpenSesame
NoSQL also means RDF stores: an Android case study 24
159. Challenges
• How to import (once) 500.000 statements
• Given that 50” are an acceptable only-once initialization time...
• ... it would require 0.1ms per statement (probably impossible!)
• Writing sqlLite integration code for OpenSesame
• Query performance of a single database is unknown
NoSQL also means RDF stores: an Android case study 24
161. Conclusion
• RDF stores are a NoSQL approach for flexible data schemata
NoSQL also means RDF stores: an Android case study 25
162. Conclusion
• RDF stores are a NoSQL approach for flexible data schemata
• Not necessarily related to the full Semantic Web
NoSQL also means RDF stores: an Android case study 25
163. Conclusion
• RDF stores are a NoSQL approach for flexible data schemata
• Not necessarily related to the full Semantic Web
• 100% Java, FLOSS solutions exist
NoSQL also means RDF stores: an Android case study 25
164. Conclusion
• RDF stores are a NoSQL approach for flexible data schemata
• Not necessarily related to the full Semantic Web
• 100% Java, FLOSS solutions exist
• Can be at least partially deployed with Android smartphones
NoSQL also means RDF stores: an Android case study 25
165. Conclusion
• RDF stores are a NoSQL approach for flexible data schemata
• Not necessarily related to the full Semantic Web
• 100% Java, FLOSS solutions exist
• Can be at least partially deployed with Android smartphones
• Full deployment being researched
NoSQL also means RDF stores: an Android case study 25
166. Conclusion
• RDF stores are a NoSQL approach for flexible data schemata
• Not necessarily related to the full Semantic Web
• 100% Java, FLOSS solutions exist
• Can be at least partially deployed with Android smartphones
• Full deployment being researched
• Proved to be effective in production
NoSQL also means RDF stores: an Android case study 25
167. Conclusion
• RDF stores are a NoSQL approach for flexible data schemata
• Not necessarily related to the full Semantic Web
• 100% Java, FLOSS solutions exist
• Can be at least partially deployed with Android smartphones
• Full deployment being researched
• Proved to be effective in production
• blueBill Mobile works
NoSQL also means RDF stores: an Android case study 25
168. Conclusion
• RDF stores are a NoSQL approach for flexible data schemata
• Not necessarily related to the full Semantic Web
• 100% Java, FLOSS solutions exist
• Can be at least partially deployed with Android smartphones
• Full deployment being researched
• Proved to be effective in production
• blueBill Mobile works
• The Semantic Web has actually got good points
NoSQL also means RDF stores: an Android case study 25
169. Conclusion
• RDF stores are a NoSQL approach for flexible data schemata
• Not necessarily related to the full Semantic Web
• 100% Java, FLOSS solutions exist
• Can be at least partially deployed with Android smartphones
• Full deployment being researched
• Proved to be effective in production
• blueBill Mobile works
• The Semantic Web has actually got good points
• blueBill effort cut in half if people provided data in (good) RDF
NoSQL also means RDF stores: an Android case study 25
171. Question time
• Follow me
• at my Java.Net blog
• at DZone
• Useful links
• http://bluebill.tidalwave.it/mobile
• http://www.openrdf.org
NoSQL also means RDF stores: an Android case study 26