6. 6
HRD – High Replication DatastoreHRD – High Replication Datastore
Schema-less
Dynamic properties
Watch out missing
or type-varying columns!
Database Access:
ORM:
DB API older
NDB newer – advanced caching
GQL
Low level API
7. 7
Forget about what you know on SQL...Forget about what you know on SQL...
No planned downtimes
(I haven't noticed unplanned ones too)
Scalability
100 records query performs the same when
searching in 100 records
as in 1 000 000 000 records
Query limitations
>, <, != conditions:
one field only
(who cares...)
No LIKE, GROUP BY,
COUNT or SUM (again, who cares...)
No JOIN queries
What?
8. 8
Eventual ConsistencyEventual Consistency
GAE
High Replication
Datastore
Nope...
?
SELECT *
WHERE
name='Alex'
GAE
High Replication
Datastore
__key__=5
name='Alex'
GAE
High Replication
Datastore
id=5
name='Alex'
SELECT *
WHERE
name='Alex'
?
9. 9
How can I live with that?How can I live with that?
>>> ses = Session(cookie=cookie, user=user)
>>> ses.put()
# next request:
>>> ses = GqlQuery(“SELECT * FROM Session WHERE cookie=:1”, cookie).get()
None # or <Session object> if lucky...
>>> ses = Session.all().filter('cookie', cookie).get()
None # or <Session object> if lucky...
>>> ses = Session(key_name=COOKIE, user=userid, …)
>>> ses.put()
# next request:
>>> ses = Session.get_by_key_name(cookie)
<Session object> # for sure
Most likely won't hurt
If hurts:
Entity groups/ancestor queries
Primary key queries
Some code?
Google BigTable is used by:
Google Search, Google Maps,
Gmail, YouTube, Orkut,
Google Earth,...
11. 11
Good and badGood and bad
✔New apps
✔Big data – small result sets
✔Google integration
✗Legacy apps
✗Binary code
✗CPU-intensive / background apps
¿Big data - big result sets – runtime analytics
•
Integrate with Google BigQuery
•
MapReduce
14. 14
Backup: Development setupBackup: Development setup
I. Create a free app on your google account:
https://appengine.google.com/
II. Download SDK:
http://goo.gl/hoK5tk
III. Create WSGI python app
IV. Create app.yaml configuration file
V. Run app at http://localhost:8000:
dev_appserver.py port=8000 <appname>
VI. Upload app google credentials
appcfg.py update app-name
VII. http://app-name.appspot.com
15. 15
Backup: The SandboxBackup: The Sandbox
An application cannot:
Open a socket
Access another host - except URL Fetch Service
Respond slowly – 60 secs
Write to the filesystem - reading is OK
Run other programs
Run binary (C) extensions – pure Python only