4. What is Datastore?
Google App Engine Datastore is a schema-less persistence
system, whose fundamental persistence unit is called Entity, c
omposed by an immutable Key and a collection of mutable pr
operties.
Entities can be created, updated, deleted, loaded by key and
queried for properties values.
DataStore is consistent and transactional, with support to
current transaction.
5. The DataStore
The Datastore is not a relational database nor a
façade.
Relational database technology doesn’t scale
horizontally
– Connection pools, shared caching are a problem
The Datastore is one of many public APIs used for
accessing Google’s
8. The DataStore : Operations
Transactions and Index are based on MegaTable.
File persistence it's done with Google File System
(GFS).
It's distributed by Chubby, a lock service for loosely-
coupled distributed systems.
9. BigTable
BigTable is a compressed, high performance, and
proprietary database system built on Google File
System (GFS), Chubby Lock Service, and a few other
Google programs
Currently not distributed or used outside of Google.
BigTable development began in 2004. and is now used
by a number of Google application Google Earth,
Google Map, Gmail, Youtube, etc..
10. BigTable : Design
BigTable is a fast and extremely large-scale DBMS.
It is a sparse, distributed multi-dimensional sorted map,
sharing characteristics of both row-oriented and column-
oriented databases.
sparse because only "not null" values are persisted
distributed in Google cloud
persistent on Google File System
multidimensional in columns values
ordered lexicographically by key
11. BigTable : Design
Tables are optimized for GFS by being split into
multiple tablets - segments of the table.
BigTable is designed to scale into the petabyte.
Each table has multiple dimensions (one of which is a
feld for time, allowing for versioning and garbage
collection).
It allows an infnite number of rows and columns.
12. Google File System
GFS is a proprietary distributed fle system developed
by Google.
It is designed to provide effcient, reliable access to
data using large clusters of commodity hardware.
GFS grew out of an earlier Google effort, BigFiles,
developed by Larry Page and Sergey Brin in the early
days of Google, while it was still located in Stanford.
14. DataStore Operations
Datastore operations are defned around entities (data
models) which are objects with one or more properties
Types: string, user, Boolean, and so on
Entities may be recursive or self-referential
Entity relationships are one-to-many or many-to-many.
Entities may be fxed or grow as needed.
15. DataStore Storage Model
Every entity is of a particular kind
Entities in a kind need not have the same properties
One entity may have different “columns” from another in
the same kind!
Unique IDs are automatically assigned unless the user
defnes a key_name
17. DataStore Storage Model
Basic unit of storage is an Entity consisting of
Kind (table)
Key (primary key)
Entity Group (partition)
0..N typed Properties (columns)
18. Datastore Quotas
Each call to Datastore counts towards the quota
The amount of data cannot exceed the billable
Includes properties and keys but not the indices
CPU and Datastore CPU time quotas apply
19. Using the Datastore
Applications may access the Datastore using the JDO
or the JPA classes.
The JDO and JPA classes are abstracted using the
DataNucleus API
Open source
Not very popular
Support for Java standards
Poor documentation
21. Setting Up JPA
The JPA and datastore JARs must be in the app's
war/WEB-INF/lib/ directory.
A confguration fle named persistence.xml must be in
the app's war/WEB-INF/classes/META-INF/ directory,
A confguration fle tells JPA to use the App Engine
datastore.
The appengine-api.jar must also be in the war/WEB-
INF/lib/ directory.
23. Getting an EntityManager Instance
An app interacts with JPA using an instance of the EntityManager.
import javax.persistence.EntityManagerFactory;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
import javax.persistence.Persistence;
public class EMF {{
public class EMF
private static final EntityManagerFactory emfInstance ==
private static final EntityManagerFactory emfInstance
Persistence.createEntityManagerFactory("transactions-optional");
Persistence.createEntityManagerFactory("transactions-optional");
public static EntityManagerFactory get() {{
public static EntityManagerFactory get()
return emfInstance;
return emfInstance;
}}
}}
24. Entity Class : Example
@Entity
@Entity
public class GuestList implements Serializable {{
public class GuestList implements Serializable
……
@Id
@Id
private String id;
private String id;
@Basic
@Basic
private User author;
private User author;
private String content;
private String content;
@Temporal(javax.persistence.TemporalType.DATE)
@Temporal(javax.persistence.TemporalType.DATE)
private Date visitDate;
private Date visitDate;
……
// Getter and Setter methods
// Getter and Setter methods
}}
25. Queries and Indices
A query operates on every entity of a given kind.
Specify zero or more sort orders
Specify zero or more flters on property values
Indices are defned in the App Engine confguration fles
Results are fetched directly from these indices; no indices are
created on the fly
WEB-INF/datastore-indexes.xml - non-standard fles
Normalization is not recommended
Optimization techniques for RDBMSs may result in poor
Datastore performance!
26. Query : Example
EntityManager em == EMF.get().createEntityManager();
EntityManager em EMF.get().createEntityManager();
try {{
try
Query query == em.createQuery("SELECT oo FROM GuestList AS o");
Query query em.createQuery("SELECT FROM GuestList AS o");
@SuppressWarnings("unchecked")
@SuppressWarnings("unchecked")
List<GuestList> results == (List<GuestList>) query.getResultList();
List<GuestList> results (List<GuestList>) query.getResultList();
for (Object obj :: results) {{
for (Object obj results)
GuestList guest == (GuestList) obj;
GuestList guest (GuestList) obj;
String nickname == guest.getAuthor().getNickname();
String nickname guest.getAuthor().getNickname();
out.println(nickname ++ "" "" ++ guest.getId());
out.println(nickname guest.getId());
}}
}} catch(Exception ex) {{
catch(Exception ex)
out.println(ex);
out.println(ex);
}}
27. Entity Relationships
Models association between entities.
There are four types of relationship multiplicities:
@OneToOne
@OneToMany
@ManyToOne
Supports unidirectional as well as bidirectional relationships
Unidirectional relationship: Entity A references B, but B doesn't
reference A.
30. Transactions and Entity Groups
Transaction = Group of Datastore operations that either
succeed or fail
Entity groups are required because all grouped entities are
stored in the same Datastore node
An entity may be either created or modifed once per
transaction
Transactions may fail if a different user or process tries an
update in the same group at the same time
Users decide whether to retry or roll the transaction back
31. Transaction in JPA : Example
Book book == em.find(Book.class, "9780596156732");
Book book em.find(Book.class, "9780596156732");
BookReview bookReview == new BookReview();
BookReview bookReview new BookReview();
bookReview.rating == 5;
bookReview.rating 5;
book.getBookReviews().add(bookReview);
book.getBookReviews().add(bookReview);
Transaction txn == em.getTransaction();
Transaction txn em.getTransaction();
txn.begin();
txn.begin();
try {{
try
book == em.merge(book);
book em.merge(book);
txn.commit();
txn.commit();
}} finally {{
finally
if (txn.isActive()) {{
if (txn.isActive())
txn.rollback();
txn.rollback();
}}
}}
32. Unsupported Features of JPA
Owned many-to-many relationships, and unowned
relationships.
"Join" queries.
Aggregation queries (group by, having, sum, avg, max, min)
Polymorphic queries.
33. Resources
Google App Engine for Java HOWTO, Andrew Lombardi, Mar
2010
The Softer Side Of Schemas, Max Ross, May 2009
Official Google App Engine Tutorial,
http://code.google.com/appengine/docs/java/gettingstarted/
Programming Google App Engine, Don Sanderson, O'Reilly,
2010
34. Thank you
thananum@gmail.com
www.facebook.com/imcinstitute
www.imcinstitute.com