A review and update presentation on Google App Engine's latest features up through version 1.5.3 and including new experimental features. This presentation was given to the San Diego GTUG on Aug 26, 2011.

1. Introducing App Engine's Latest
Features
Chris Schalk
Developer Advocate

2. Agenda
● What is App Engine?
● Part I: App Engine Production Updates
○ Growth trajectory
○ App Engine Success so far
○ A Big Production Update: App Engine Leaving
Preview
● Part II: App Engine Technical Updates
○ Go Runtime, Backends, Channel API, FTS,
Mapreduce

3. What is App Engine?
Build and run your web apps on Google’s infrastructure
● Easy to build
● Easy to maintain
● Easy to scale
Focus on building your app, let us wear the pagers!

4. Cloud landscape as Gartner Sees It
SaaS
PaaS
IaaS
Source: Gartner AADI Summit Dec 2009

5. What is App Engine?
PaaS Development Platform in a Box
● Downloadable SDK
● Application runtimes
○ Java, Python, Go
● Local development tools
○ Eclipse plugin
● Specialized api services
● Cloud based dashboard
● Automatically scales
● Built in fault tolerance, load
balancing
● Continuously evolving...

6. Language Runtime Options
GO Java
Experimental

7. App Engine Services
Memcache Datastore URL Fetch
Mail XMPP Task Queue
Images Blobstore User Service

8. Agenda
● What is App Engine?
● Part I: App Engine Production Updates
○ Growth trajectory
○ App Engine Success so far
○ A Big Production Update: App Engine Leaving
Preview
● Part II: App Engine Technical Updates
○ Go Runtime, Backends, Channel API, FTS,
Mapreduce

9. App Engine Evolution Through The Years
2008 2009 2010 2011
App Engine Batch write/read Java Task Queues Blobstore Multitenancy Hi-Replication
Launch Https DB Import XMPP Appstats Instance Console Datastore
Python Status- cron incoming email cursors Always On Channel API
Datastore Dashboard Mapper hi-perf imag Files API
Memcache 10 min tasks Remote API
logs export Prosp Search

10. App Engine - By the Numbers
100,000+
Active developers per Month
200,000+
Active apps per week
1.5B+
Pageviews per day

11. Notable App Engine Customers

12. Royal Wedding - Scalability Success
Official Royal Wedding blog
on App Engine
On Wedding day, served:
● Up to 32k requests per second
● 37.7 million pageviews
● 13.7 million visitors
http://goo.gl/F1SGc

13. Enterprise Success on App Engine

14. Best Buy Applications on App Engine

15. WebFilings Financial SaaS on App Engine
Enterprise customers:
● Procter & Gamble
● Delta Airlines
● Valero Energy
● Sprint/Nextel
● eBay
● United Airlines
● Dish Network
● ...
Recently signed:
● One of the world's largest
banks
● The world's largest retailer
● One of the largest US
auto manufacturers
http://www.webfilings.com/customers/valued_customers

16. App Engine: Certified Security
App Engine is now fully SAS70/SSAE16/ISAE
3402 Certified!

17. One Big Production Update...
.... App Engine is coming out of preview!
● http://googleappengine.blogspot.com/2011/05/year-ahead-for-
google-app-engine.html

18. Production App Engine Features
App Engine Leaving Preview in 2011!
● Full Google Product - 3 year deprecation support
● 99.95% SLA for all paid users
● Operational and Developer Support available
● Billing via invoice
● Ongoing feature delivery: custom domain
SSL, MapReduce, etc..

19. What Does Leaving Preview Mean?
● Long-term committment by Google to App Engine
● Long-term committment by App Engine to users!
● It's a real product now, Enterprises can trust in it!
● http://googleappengine.blogspot.com/2011/05/year-
ahead-for-google-app-engine.html

20. Production App Engine Pricing
● Usage types:
○ Free - similar to today
○ Paid - $9/app/mo (+ usage) including SLA
○ Premier - $500/account/mo (+ usage) including operational
support
● Add-on fee for special services
○ For example: SSL for custom domains will be available for
monthly fee

21. Production App Engine Further Details
● Instance based charging
○ CPU-hours replaced by Instance Hours (IH)
○ 1 instance running for 1 hour
○ Similar to instances in admin console today
● APIs
○ Currently charged per CPU-hour
○ Will be charged per operation
● FAQ
○ https://groups.google.com/group/google-
appengine/browse_thread/thread/a1bfa432e0c002a7
○ https://groups.google.com/group/google-
appengine/browse_thread/thread/1e5ba8835fdade7e

22. Agenda
● What is App Engine?
● Part I: App Engine Production Updates
○ Growth trajectory
○ App Engine Success so far
○ A Big Production Update: App Engine Leaving
Preview
● Part II: App Engine Technical Updates
○ Go Runtime, Backends, Pull Queues, Channel
API, FTS, MapReduce

23. App Engine 1.5 Updates
● Introduced at Google IO, May 2011

24. 'Go' added as third language runtime
● Released SDK for Mac & Linux
○ Windows port in the works
● Uses tweaked Python SDK tools
● Launched for production 1.5.2 (Jul 2011)
● http://blog.golang.org/2011/05/go-and-google-app-engine.html

25. High Replication Datastore now default
● Master/Slave (M/S) original App Engine datastore
● High Replication Datastore (HRD) uses Paxos algorithm
● Was originally 3x the cost of M/S*
● Slightly slower (distributed writes)
● Queries eventually consistent
● Well-received
○ http://googleappengine.blogspot.com/2011/03/high-
replication-datastore-solid-choice.html

26. HRD as default - cont.
● *Price reduction to discourage M/S (no longer 3x)
● Has Delivered ~99.999% uptime since launch in January
● M/S price increases when App Engine leaves Preview
● Working on better tools
○ To make migration easier
○ To estimate updated cost

27. Backends feature introduced
● Long-running servers
● No more 30s or 10m deadline
○ Fully configurable instances with options:
■ RAM: 128MB to 1GB
■ CPU: 600MHz to 4.8GHz no request deadlines
○ Can run indefinitely
○ Resident or Dynamic - Public or Private
○ Automatic restarts
○ Easy to configure (backends.yaml, backends.xml)
● (Demo)
● http://www.google.com/events/io/2011/sessions/app-engine-
backends.html

28. "Pull Queues" introduced
● A new alternative to previous Task Queues,
now referred to as "Push Queues"
● Lets users decide when to execute tasks
● Uses a REST API
● Allows external access

29. Recalling "Push Queues"

30. "Pull Queues" introduced
● A new alternative to previous Task Queues, now
referred to as "Push Queues"
● Task is merely data, no longer a web request
● Workers can pull (lease) tasks from queue
● Uses a REST API
○ https://www.googleapis.com/taskqueue/myappid/taskqueues/myqueue
● Can configure ACLs for external access
● http://www.google.com/events/io/2011/sessions/putting-
task-queues-to-work.html

31. Pull Queues vs. (Push)Task Queues
● Push Task Queues take & execute work
○ Tasks are "pushed" to application
○ Execute at a given rate
● Pull queues offer more flexibility
○ Workers can pull (lease) tasks when
appropriate

32. Other App Engine 1.5.(0) Features
● Mail API updates
○ Reduced free recipients from 2000 to 100 for new apps
○ ...
● Source code downloading
○ Now Owner and Uploader can download
● http://googleappengine.blogspot.
com/2011/05/app-engine-150-release.html

33. App Engine 1.5.1 Features
● Geolocation header
○ X-AppEngine-country - can customize web pages based on
location
● Presence support in Channel API
○ Can take action when user connects/disconnects (demo)
● WebP support in Images API
○ WebP 39% smaller than JPEG!
● Create RPC services with ProtoRPC
○ Now part of App Engine
● http://googleappengine.blogspot.
com/2011/06/app-engine-151-release.html

34. App Engine 1.5.2 Features
● Production: adjustable scheduler parameters
○ Able to set the minimum pending latency and maximum
number of idle instances
● Namespaced datastore statistics
● New option to query datastore stats per namespace
● Tasks
○ New Task Queue details page
○ Pull Queue task size bump to 1MB
○ Pull Queue task lease extensions
● http://googleappengine.blogspot.com/2011/07/app-
engine-152-sdk-released.html

35. App Engine 1.5.3 Features
● Blobstore file uploads no longer limited
○ Allows you to serve much larger files: images, videos etc.
● Datastore Admin now easily enable via 1-click
○ No longer have to update app.yaml, and java developers
don't have to upload a python version of their app
● New HRD migration tool trusted tester program
● Download App
○ Can now download any files that were uploaded into your war
directory
● http://googleappengine.blogspot.com/2011/08/app-engine-153-
sdk-released.html

36. Experimental feature updates
● Introduced at Google I/O 2011 talks
● Full-text Search
○ Full-text search service coming to App Engine
○ Other queries: numeric, geo, date search capabilities, etc.
○ http://www.google.com/events/io/2011/sessions/full-text-
search.html
● MapReduce
○ Part 1: Mapper has been available for awhile
○ Work is ongoing for shuffle and reduce
○ http://www.google.com/events/io/2011/sessions/app-engine-
mapreduce.html
○ http://www.youtube.com/watch?v=EIxelKcyCC0

37. Q&A