1. What is App Engine?
Ikai Lan
OSCON
July 21st, 2010
Twitter: @ikai

2. Agenda
Topics we'll cover
Why App Engine?
Traditional web stack scalability
App Engine to the rescue!
Services and APIs
Deployment steps
App Engine and Open Source
Questions and Next Steps

3. What is Google App Engine?

4. Google App Engine is...
... a way for you to run
your web applications
on Google’s scalable
infrastructure.
Google’s Data Centers

5. Start with... the basic LAMP stack
LAMP:
Linux
Apache
MySql
Programming Language
(PHP, Python, Perl, etc.)
NOT Scalable
Single Point of Failure (SPOF)
Google Confidential and Proprietary

6. Database on a separate server
Still not Scalable!
TWO Single Points of Failure
Google Confidential and Proprietary

7. Multiple Web Servers
Now you need Load Balancing
Database is still Single Point of Failure
Google Confidential and Proprietary

8. Round Robin Load Balancing
Register list of IPs with DNS
DNS record is cached with a Time to Live (TTL)
Google Confidential and Proprietary

9. Round Robin Load Balancing
But the TTL takes time to propagate and might not be respected
So if a machine goes down... :-(
And the database is still SPOF
Google Confidential and Proprietary

10. Master Slave Database
:-) Better read throughput :-( Master is SPOF for writes
:-( Master may die before replication
Google Confidential and Proprietary

11. Partitioned Database
:-) Better R/W throughput :-( More machines, more management
:-( Re-architect data model
:-( Rewrite queries
Google Confidential and Proprietary

12. Why build it all yourself?
Google Confidential and Proprietary

13. Why not use Google App Engine?
Simple application configuration
No systems administration
No performance tuning
AUTOMATIC SCALING!
Google Confidential and Proprietary

14. The Cloud Computing Landscape
SaaS
PaaS
IaaS
Source: Gartner AADI Summit Dec 2009

15. App Engine Developers/Apps
Google Confidential and Proprietary

16. Google Confidential and Proprietary

17. By the numbers
Over 100,000 applications
250,000 developers
Over 250 million daily pageviews

18. Underneath the hood

19. App Engine Components
Load balancing
Routing
Hosts static content
Separate from programming files
Google Confidential and Proprietary

20. App Engine Components
Hosts application code
Handles concurrent requests
Enforces isolation for app safety
Maintains statelessness
Multiple Runtimes
Copyright © Sun Microsystems Inc. All rights reserved.
Google Confidential and Proprietary

21. App Engine Services/APIs

22. Bigtable - The App Engine datastore
Distributed, partitioned datastore
Arbitrary horizontal scaling - scales to “Internet scale”
Replicated and fault tolerant
Parallel processing
Predictable query performance
No deadlocks
Google Confidential and Proprietary

23. Memcache
Distributed, very fast,
in-memory cache
Optimistic caching
Very stable, robust and specialized
Google Confidential and Proprietary

24. URL Fetch
Simple, HTTP communication
HTTP GET/POST to external service
Allows integration with third-party REST APIs
Google Confidential and Proprietary

25. Mail
Inbound and outbound mail
Outbound mail
Inbound mail handling
Attachment processing
Google Confidential and Proprietary

26. XMPP
Instant messaging for your application
Incoming and outgoing XMPP
No need to worry about setting up servers
Google Confidential and Proprietary

27. Task Queue
Background and scheduled
computation
Background processing infrastructure
Scheduled jobs
Automatic handling of queuing and job polling
Google Confidential and Proprietary

28. Images
Image manipulation
Resize
Crop
Image compositions
Google Confidential and Proprietary

29. Blobstore
Heavy lifting for large files
Upload and distribute large files
Programmatic access to file contents
Google Confidential and Proprietary

30. User Accounts
Federated login for your application
Google Accounts or OpenID
Administrator management
No need to create user management system
Google Confidential and Proprietary

31. Getting started

32. Getting started with App Engine
Download the SDK
http://code.google.com/appengine
Register for an Appspot account
https://appengine.google.com
Write code - deploy!

33. Starting a project
Linux, MacOS, etc. command-line:
$ dev_appserver.py helloworld # run dev svr
$ appcfg.py update helloworld # deploy live
Windows GUI (also avail for Mac):

34. Project contents
app.yaml – main configuration file
index.yaml – automatically
generated to index your data
main.py – your main application
"controller" code goes here

35. main.py

36. Local development server
$ dev_appserver.py helloworld
(Can also use the launcher for Windows and OS X)

37. Deploying the application
Set application identifier
Run deploy script
You're live!

38. main.py: Adding a handler
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
class MainHandler(webapp.RequestHandler):
def get(self):
self.response.out.write('<h1>Hello world!</h1>')
self.response.out.write('''
<form action="/sign" method=post>
<input type=text name=content>
<br><input type=submit value="Sign Guestbook">
</form>
''')
class GuestBook(webapp.RequestHandler):
def post(self):
self.response.out.write(
'<h2>You wrote:</h2> %s' % self.request.get('content')
)
application = webapp.WSGIApplication([
('/', MainHandler),
('/sign', GuestBook),
], debug=True)
# start_wsgi_app etc ...

39. main.py: Persisting to the datastore
class GuestBook(webapp.RequestHandler):
def post(self):
greeting = Greeting()
greeting.content = self.request.get('content')
greeting.put()
self.redirect('/')

40. main.py: Collecting values from the datastore
class MainHandler(webapp.RequestHandler):
def get(self):
self.response.out.write('Hello world!')
self.response.out.write('<h1>My GuestBook</h1><ol>')
greetings = Greeting.all()
for greeting in greetings:
self.response.out.write('<li> %s' % greeting.content)
self.response.out.write('''
</ol><hr>
<form action="/sign" method=post>
<textarea name=content rows=3 cols=60></textarea>
<br><input type=submit value="Sign Guestbook">
</form>
''')

41. Running the deploy script
$ appcfg.py update helloworld
Scanning files on local disk.
Initiating update.
Email: ...

42. You're live!

43. Google App Engine and Open Source

44. TyphoonAE: Open Source implementation

45. Open Source JVM language runtimes
Clojure
Practical Lisp on the JVM
JRuby/Mirah
Ruby on the JVM. Mirah = Ruby-like language that
compiles to Java (speed!)
Groovy
Dynamically typed Java-like language
Scala
Strongly typed, functional imperative
Jython
Java implementation of Python

46. Open Source Persistence Frameworks
Objectify
http://code.google.com/p/objectify-appengine/
Twig
http://code.google.com/p/twig-persist/
Slim3 (popular in Japan)
http://sites.google.com/site/slim3appengine/
SimpleDS
http://code.google.com/p/simpleds/
Django-nonrel
http://www.allbuttonspressed.com/projects/django-
nonrel

47. Official features
Datanucleus JDO/JPA
http://code.google.com/p/datanucleus-appengine/
App Engine Map/Reduce
http://code.google.com/p/appengine-mapreduce/
Python SDK
Java SDK - soon!

48. Questions and Next Steps

49. Code time!

50. main.py: Skeleton application
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
class MainHandler(webapp.RequestHandler):
def get(self):
self.response.out.write('Hello world!')
def main():
application = webapp.WSGIApplication([('/', MainHandler)],
debug=True)
util.run_wsgi_app(application)
if __name__ == '__main__':
main()

51. Next steps
Download the SDK
http://code.google.com/appengine
Do the codelab
http://code.google.com/appengine
Register for an Appspot account
https://appengine.google.com
Get this presentation
http://slideshare.net/ikailan