This document describes ExSchema, a tool that discovers schemas from the source code of polyglot persistence applications. It analyzes project structure, update operations, repositories, and annotations to understand implicit schemas across different databases. The tool represents schemas as a uniform metamodel and outputs results as PDF files. It is demonstrated on applications using Neo4j, Spring Data Neo4j, MongoDB, HBase, CouchDB, relational databases, and combinations of these. Limitations and future work are discussed. The implementation leverages Eclipse JDT and AST to analyze Java code and output results through Spring Data and as PDF files.

1. ExSchema
Discovering Schemas from Polyglot
Persistence Applications
Juan Castrejón - Université de Grenoble

2. Objective
Discover schemas from the source code
of polyglot persistence applications
Source
code
Relational Graph Document Column- Key-Value
DB DB DB Family DB DB
2

3. Why?
Polyglot persistence applications
are becoming widespread
Schema-less datastores
But for their development and maintenance,
software engineers have to deal with… Non-standard APIs
Implicit schemas described
in the source code
3

4. How? ExSchema
MetaLayer
Representation
Declarations Updates Repositories Annotations
Analyzer Analyzer Analyzer Analyzer
Analyze project structure
and update operations
Application source code
Neo4j API MongoDB API HBase API
JPA API Spring Data CouchDB API
4

5. MetaLayer
*
*
Set Attribute
*
* *
*
* Struct Relationship
* *
Based on: P. Atzeni, F. Bugiotti, and L. Rossi. Uniform access to non-relational database systems:
the SOS platform. In CAiSE’12, volume 7328 of LNCS, pages 160–174. Springer, 2012.
5

6. Results
PDF file
Spring Roo scripts
(JPA, MongoDB, Neo4j)
6

7. Demonstration
import org.neo4j.graphdb.Node;
Neo4j
import org.neo4j.graphdb.Relationship;
class ActorImpl implements Actor {
private static final String NAME_PROPERTY = "name”;
private final Node underlyingNode; Declaration
public void setName( final String name ) {
}
underlyingNode.setProperty( NAME_PROPERTY, name ); Update
public Role createRole( final Actor actor, final Movie movie, final String roleName )
final Node actorNode = ((ActorImpl) actor).getUnderlyingNode();
final Node movieNode = ((MovieImpl) movie).getUnderlyingNode();
final Relationship rel = actorNode.createRelationshipTo( movieNode, RelTypes.ACTS_IN );
…
} https://github.com/neo4j-examples/imdb
7

8. Demonstration Neo4j
PDF file 8

9. Demonstration Neo4j
Spring Roo script
9

10. Demonstration Spring Data Neo4j
import org.springframework.data.annotation.Indexed;
import org.springframework.data.graph.annotation.NodeEntity;
import org.springframework.data.graph.annotation.RelatedToVia;
import org.springframework.data.graph.core.Direction;
@NodeEntity
public class Actor { Annotation
@Indexed(indexName="actor_id")
private String id;
private String name;
@RelatedToVia(type=Participation.RELATIONSHIP_TYPE,
Declaration
direction=Direction.OUTGOING , elementClass=Participation.class)
Iterable<Participation> participations;
…
}
https://github.com/neo4j-examples/cineasts
10

11. Demonstration Spring Data Neo4j
PDF file 11

12. Demonstration Spring Data Neo4j
Spring Roo script 12

13. Demonstration Spring Data MongoDB
import javax.persistence.ManyToOne;
import org.springframework.roo.addon.layers.repository.mongo.RooMongoEntity;
@RooMongoEntity Annotation
public class Post {
private String idContact;
private String postGeoStamp;
private Date postTimeStamp;
private String idPost; Declaration
@ManyToOne
private Content content;
…
}
http://vargas-solar.imag.fr/academika/cloud-data-management/
13

14. Demonstration Spring Data MongoDB
import fr.imag.mynet.domain.Post;
import java.util.List;
import org.springframework.roo.addon.layers.repository.mongo.RooMongoRepository;
@RooMongoRepository(domainType = Post.class)
public interface PostRepository { Repository
List<fr.imag.mynet.domain.Post> findAll();
}
http://vargas-solar.imag.fr/academika/cloud-data-management/
14

15. Demonstration Spring Data MongoDB
PDF file
15

16. Demonstration Spring Data MongoDB
Spring Roo script

17. Demonstration HBase
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.util.Bytes;
public class PutExample {
public static void main(String[] args) throws IOException {
HTable table = new HTable(conf, "testtable");
Put put = new Put(Bytes.toBytes("row1")); Declaration
put.add(Bytes.toBytes("colfam1"), Bytes.toBytes("qual1"), Bytes.toBytes("val1"));
put.add(Bytes.toBytes("colfam1"), Bytes.toBytes("qual2"), Bytes.toBytes("val2")); Update
table.put(put);
…
}
https://github.com/larsgeorge/hbase-book/tree/master/ch03
17

18. Demonstration HBase
PDF file
18

19. Demonstration CouchDB
import com.fourspaces.couchdb.Database;
import com.fourspaces.couchdb.Document;
import com.fourspaces.couchdb.Update;
public class UpdateTest {
Database foo; Declaration
public void createTestDB() {
Document testDoc = new Document();
testDoc.put("_id", "test_data");
testDoc.put("Field1", "Default");
testDoc.put("Field2", "Default"); Update
foo.saveDocument(testDoc);
…
}
https://github.com/mbreese/couchdb4j 19

20. Demonstration CouchDB
PDF file
20

21. Demonstration MongoDB+JPA
import javax.persistence.Entity;
import org.springframework.data.mongodb.crossstore.RelatedDocument;
@Entity
public class Customer { Annotation
@Id private Long id;
private String firstName;
private String lastName;
Declaration
@RelatedDocument
private SurveyInfo surveyInfo;
…
}
https://github.com/SpringSource/cloudfoundry-samples/tree/master/cross-store
21

22. Demonstration MongoDB+JPA
PDF file
22

23. Demonstration MongoDB+JPA
Spring Roo script
23

24. Test applications
Neo4j:
- https://github.com/neo4j-examples/cineasts.git
- https://github.com/neo4j-examples/imdb.git
- https://github.com/neo4j-examples/java-astar-routing.git
- https://github.com/neo4j-examples/java-dijkstra.git
- https://github.com/neo4j-examples/java-tree-traverse.git
- MyNetContacts (http://vargas-solar.imag.fr/academika/cloud-data-management/)
MongoDB:
- https://github.com/mongolab/mongodb-driver-examples.git
HBase:
- https://github.com/larsgeorge/hbase-book.git (ch03)
- https://github.com/SpringSource/spring-hadoop-samples.git (original-samples/hbase-crud)
CouchDB:
- https://github.com/mbreese/couchdb4j.git
Relational:
- Indvalid-core (http://www.indvalid.com/)
Relational + MongoDB:
- https://github.com/SpringSource/cloudfoundry-samples.git (cross-store)
- MyNet (http://vargas-solar.imag.fr/academika/cloud-data-management/) Industrial application
- Indvalid-dao (http://www.indvalid.com/)
Neo4j + MongoDB + Relational:
- twitter-spring
- twitter-polyglot
(Based on: P. Atzeni, F. Bugiotti, and L. Rossi. Uniform access to non-relational database systems: 24
the SOS platform. In CAiSE’12, volume 7328 of LNCS, pages 160–174. Springer, 2012)

25. Limitations
Based on project structure and update operations
(Queries and get operations not considered)
Based on programming styles of test applications
(Heavily relies on local variables)
Limited associations between entities
(Besides Neo4j’s relationships and MongoDB cross-store)
25

26. Future work
Analysis of queries and get operations
Support additional languages besides Java
Increase support for different programming styles
26

27. Implementation
https://code.google.com/p/exschema/
Eclipse JDT
Eclipse AST
Spring Data
27

28. Contact
Juan.Castrejon@imag.fr
28