Talk from Pamela Fox (me) at YOW 2010 in Brisbane. Covers App Engine and the datastore, with Python examples.

1. WRITING APPS THE GOOGLE-Y WAY Pamela Fox, YOW! Australia 2010 (Brisbane)

2. Who am I? twitter.com/pamelafox [email_address] pamelafox.org you get the idea...

3. Who am I? Google Maps API Google Wave API 2006 2010 2008 Google App Engine

4. Who am I? wave side projects 92 apps

5. Who am I? Java pYthon

7. What is a “web app”?

8. Static vs. Dynamic

9. Anonymous vs. Users

10. Intranet vs. Internet ~2 billion Hundreds - Thousands

11. What is a “web app”?

12. Some Google web apps

13. Some Google App Engine web apps www.gifttag.com www.buddypoke.com

14. Google apps on App Engine panoramio.com pubsubhubbub.appspot.com

16. Demo: Guestbook awesomest-app.appspot.com http://code.google.com/p/google-app-engine-samples/source/browse/trunk/guestbook appengine.google.com localhost build deploy monitor

17. App Engine architecture

18. App Engine architecture user task

19. App Engine architecture

20. App Engine architecture LIMIT CPU LIMIT Memory LIMIT Time

21. App Engine architecture hardware ports globals file system Groovy, JRuby, Mirah, Clojure, Scala

22. App Engine architecture 141,241,791 calls 1 GB data $0.15 GB/month 45,000,000 calls 657,000 - 46,000,000 calls *Always check docs for latest quotas. 192,672,000 calls 558 GB data $0.15 GB/month 7,000 - 1,700,000 calls $0.0001 per mail sent 46,000,000 calls 1,046 GB data sent 100,000 - 20,000,000 calls

23. The tricky bits

24. Datastore Entity Properties Key Entity Entity Entity Entity Path Kind Name/ID

25. Example: Speaker Entities Key Path Kind ID First Name Last Name Speaker1 - Speaker 1 Rod Johnson Key Path Kind ID First Name Last Name Middle Name Suffix Speaker2 - Speaker 2 Guy Steele L Jr.

29. Queries & Indexes Query Index Index Index Index Query Query Query Query Query

30. Queries & Indexes SELECT * from Speaker ORDER BY lastname LIMIT! (# of results) key lastname Speaker3 Fox Speaker4 Hohpe Speaker1 Johnson Speaker2 Steele

31. Queries & Indexes SELECT * from Speaker ORDER by middlename key middlename Speaker2 L

32. Queries & Indexes SELECT * from Speaker WHERE keynote = True key keynote Speaker1 True Speaker2 True Speaker3 False Speaker4 False

33. Queries & Indexes SELECT * from Speaker WHERE keynote = False key keynote Speaker1 True Speaker2 True Speaker3 False Speaker4 False

37. Impossible Indexes SELECT * from Speaker WHERE lastname < 'Steele' and firstname > 'Gregory' ...not in subsequent rows! key lastname firstname Speaker3 Fox Pamela Speaker4 Hohpe Gregory Speaker1 Johnson Rod Speaker2 Steele Guy

38. Impossible Indexes SELECT * from Speaker WHERE lastname > 'Fox' ORDER BY firstname ...not in the correct order! key lastname firstname Speaker3 Fox Pamela Speaker4 Hohpe Gregory Speaker1 Johnson Rod Speaker2 Steele Guy

41. More Properties class Talk(db.Model): title = db.StringProperty(required=True) abstract = db.TextProperty(required=True) speaker = db.ReferenceProperty(Speaker) tags = db.StringListProperty() pamela = Speaker.all().filter('firstname = ', 'Pamela').get() talk = Talk('Writing Apps the Googley Way', 'Bla bla bla', pamela, ['App Engine', 'Python']) talk.put() talk = Talk('Wonders of the Onesie', 'Bluh bluh bluh', pamela, ['Pajamas', 'Onesies']) talk.put()

42. Back-References pamela = Speaker.all().filter('firstname = ', 'Pamela').get() for talk in pamela.talk_set: print talk.title SELECT * from Talk WHERE speaker = Speaker3 key speaker Talk6 Speaker2 Talk1 Speaker3 Talk2 Speaker3 Talk5 Speaker4

43. Searching List Properties talks = Talk.all().filter('tags = ', 'python') .fetch(10) SELECT * from Talk WHERE tags = 'Python' LIMIT! (# of index rows) key lastname Talk1 App Engine Talk2 Pajamas Talk1 Python Talk2 Onesies

44. Update Transactions commit journal apply entities apply indexes A B

45. Entity Groups pamela = Speaker.all().filter('firstname = ', 'Pamela').get() talk1 = Talk('Writing Apps the Googley Way', 'Bla bla bla', pamela, ['App Engine', 'Python'], parent=pamela) talk2 = Talk('Wonders of the Onesie', 'Bluh bluh bluh', pamela, ['Pajamas', 'Onesies'], parent=pamela) db.put(talk1, talk2) def update_talks(): talk1.title = 'Writing Apps the Microsoft Way' talk2.title = 'Wonders of the Windows' db.put(talk1, talk2) db.run_in_transaction(update_talks)

46. Common Features

47. Counters 1 2 3 4 5 people have done something.

48. RageTube: Global Stats ragetube.net http://github.com/pamelafox/ragetube

49. RageTube: Global Stats SongStat yaycount viewcount title artist Key Path Kind Name (song) naycount mehcount

50. RageTube: Global Stats viewcount viewcount viewcount datastore memcache

51. RageTube: Global Stats class Song(db.Model): viewcount = db.IntegerProperty(default=0) title = db.StringProperty() artist = db.StringProperty() def get_viewcount(self): viewcount = self.viewcount cached_viewcount = memcache.get('viewcount-' + self.key().name(), self.key().kind()) if cached_viewcount: viewcount += cached_viewcount return viewcount @classmethod def flush_viewcount(cls, name): song = cls.get_by_key_name(name) value = memcache.get('viewcount-' + name, cls.kind()) memcache.decr('viewcount-' + name, value, cls.kind()) song.viewcount += value song.put() @classmethod def incr_viewcount(cls, name, interval=5, value=1): memcache.incr('viewcount-' + name, value, cls.kind()) interval_num = get_interval_number(datetime.now(), interval) task_name = '-'.join([cls.kind(), name.replace(' ', '-'), 'viewcount', str(interval), str(interval_num)]) deferred.defer(cls.flush_viewcount, name, _name=task_name) LIMIT! (# of tasks)

52. Ratings Rated by 500 users.

53. App Gallery: Ratings google.com/analytics/apps/

54. App Gallery: Ratings Comment Application total_ratings sum_ratings avg_rating rated_index comment_count rating

55. App Gallery: Ratings def UpdateAppCommentData(self, rating, operation): def UpdateCommentData(self, rating, operation): self.comment_count += 1 * operation self.sum_ratings += rating * operation self.total_ratings += 1 * operation self.avg_rating = int(round(self.sum_ratings / self.total_ratings)) self.rated_index = '%d:%d:%d' % (self.avg_rating, self.total_ratings, self.index) self.put() db.run_in_transaction(UpdateCommentData, self, rating, operation) app.UpdateAppCommentData(rating, db_models.Comment.ADD) comment = db_models.Comment() comment.application = app comment.rating = rating comment.put() query.order('-avg_rating').order('-rated_index')

56. Geospatial Queries

57. City-Go-Round: Agencies citygoround.org https://github.com/walkscore/City-Go-Round

58. City-Go-Round: Geo Queries Agency GeoModel location (GeoPt) location_geocells (StringListProperty)

59. City-Go-Round: Geo Queries def fetch_agencies_near(lat, long, bbox_side_in_miles): query = Agency.all() bbox = bbox_centered_at(lat, long, bbox_side_in_miles) return Agency.bounding_box_fetch(query, bbox, max_results = 50) def bounding_box_fetch(query, bbox, max_results=1000,): results = [] query_geocells = geocell.best_bbox_search_cells(bbox) for entity in query.filter('location_geocells IN', query_geocells): if len(results) == max_results: break if (entity.location.lat >= bbox.south and entity.location.lat <= bbox.north and entity.location.lon >= bbox.west and entity.location.lon <= bbox.east): results.append(entity) return results

61. Disclosed.ca: Search https://github.com/nurey/disclosed disclosed.ca

62. Disclosed.ca: Search Contract agency_name vendor_name description comments uri

63. Disclosed.ca: Search from search.core import SearchIndexProperty, porter_stemmer class Contract(db.Model): uri = db.StringProperty(required=True) agency_name = db.StringProperty(required=True) vendor_name = db.StringProperty(required=True) description = db.StringProperty() comments = db.TextProperty() search_index = SearchIndexProperty(('agency_name', 'vendor_name', 'description', 'comments'), indexer=porter_stemmer) results = Contract.search_index.search(sheep').fetch(20)

64. Disclosed.ca: Search Contract agency_name vendor_name description comments uri search_index (StringListProperty) SearchIndex

65. Disclosed.ca: Search SELECT FROM ContractSearch WHERE search_index = &quot;sheep&quot; key search_index ContractSearch1 charter ContractSearch1 june ContractSearch1 sheep ContractSearch2 sheep ContractSearch1 wood

66. More Learning http://ae-book.appspot.com http://code.google.com/appengine http://blog.notdot.net/

68. Thanks for coming!