Feihong talks about PEP 3156 and basic usage of Tulip, the reference implementation. Video: http://pyvideo.org/video/2194/asynchronous-io-in-python-3 Source code: https://github.com/feihong/tulip-talk/

1. Asychronous I/O in
Python 3
Feihong Hsu
Chicago Python User Group
July 11, 2013

2. What is PEP 3156?
• "Asynchronous IO Support Rebooted"
• Describes a new event loop API
• Aims to be compatible with current
popular event loops (Twisted,Tornado, etc.)
• Should land in the standard library in 3.4
(early 2014)

3. The big take-away
We're going to be able to write
asynchronous code without threads or
callbacks!

4. Threads vs event loop
• I'm not going to talk about this
• Twisted being around for more than a
decade kinda validates the event loop
model
• For a detailed discussion see An Intro to
Asynchronous Programming and Twisted

5. Twisted vs Tulip
Ugh, how to explain this?
I could just give you a boring list of
differences...
Or I could use an analogy...

6. Programming with
Twisted is like...
Being a mystical ninja warrior!

7. Mystical Ninja Warrior
• Can defeat demons and demigods by
deploying an army of shadow clones
• Mental gymnastics required to coordinate
shadow clones requires years of brutal
training
• Ninja magic comes with a terrible price: the
more clones you make, the more insane
you become

8. Programming with Tulip
is like...
Playing PlantsVs Zombies

9. Playing PlantsVs
Zombies
• Can defeat a zombie horde by deploying
and managing a large number of
horticultural specimens
• Not particularly mentally taxing
• Somehow end up playing for hours without
eating or sleeping

10. Alrighty
With the analogies out of the way, let's look
at some examples.

11. Fetching a web page in
Twisted
https://github.com/feihong/tulip-talk/blob/
master/examples/1-twisted-download.py

12. Control flow diagram
I was going to draw one but then I got
sleepy.

13. Problems with callback-
based asynch code
• Can be hard to understand control flow
• Error messages are not friendly
• Debugging is harder

14. Fetching a web page in
Tulip
https://github.com/feihong/tulip-talk/blob/
master/examples/2-tulip-download.py

15. Fetching a web page in
Tulip w/ callbacks
https://github.com/feihong/tulip-talk/blob/
master/examples/3-tulip-download-
callbacks.py

16. Fetching a web page in
Twisted w/ inline
callbacks
https://github.com/feihong/tulip-talk/blob/
master/examples/4-twisted-download-inline-
callbacks.py

17. The Tulip API
• Basics are documented in PEP 3156
• Not available in Python Package Index
• Includes sockets, file I/O, etc.
• Includes concurrency data structures like
locks, queues, semaphores
• Includes SSL and HTTP (with support for
websockets!)

18. Coroutines vs tasks
When I first started, the most confusing
thing about Tulip.
Coroutines are executed by "yield from".
Tasks are not executed by "yield from".

19. Coroutine
• Basically a function that contains at least
one "yield from" statement.
• Not the same thing as a generator
function, which is a function that contains a
yield expression.
• Tulip will barf if you try to make it execute
a generator.

20. Coroutine
@tulip.coroutine
def download(url):
response = yield from tulip.http.request('GET', url)
return yield from response.read()
Calling "download()" returns a generator
object, but otherwise does nothing!You need
to do "yield from download()" to run the
body of a coroutine.

21. Task
• An object that manages a coroutine
• Roughly equivalent to the Deferred object
in Twisted

22. Task
@tulip.task
def download(url):
response = yield from tulip.http.request('GET', url)
return yield from response.read()
Calling "download()" actually does run the
body of the function.The "yield from" part is
done implicitly for you.

23. Task
In Tulip, there are two ways of creating tasks:
tulip.task is a decorator that produces
a task-wrapping function
tulip.Task is a constructor that accepts
a coroutine

24. Why bother to use
tasks?
• To interoperate with callback-based
frameworks like Twisted
• To cancel an already-running coroutine
• To start a new coroutine from within
another coroutine

25. Wait, so what's a
Future?
PEP 3156 makes frequent mention of
Futures.
But talking about Futures is a little confusing,
since there are two Future classes:
tulip.Future and concurrent.futures.Future.

26. Future is the superclass
of Task
• Future don't necessarily manage a
coroutine
• In practice you never create Future
objects, only Task objects
• Futures are acceptable to yield from
expressions

27. Methods on Future
• cancel(), cancelled()
• running(), done()
• result(), set_result()
• add_done_callback(),
remove_done_callback()
• ...

28. Web development with
Tulip
• All the classes you need are in the tulip.http
module
• Make subclass of
tulip.http.ServerHttpProtocol
• Override the handle_request() method

29. Hello World!
https://github.com/feihong/tulip-talk/blob/
master/examples/5-hello-world-http.py

30. Some observations
The HTTP API is fairly simple, but a bit low
level for everyday web programming.
Expect a thousand microframeworks to
bloom in the near future.
Speaking of microframeworks...

31. Introducing viol
• Tiny web framework based on Tulip
• After initial page load, messages between
client and server are exchanged via
websockets
• Makes code demos a bit more visual

32. Tulip API demos
Now for a bunch of demos...

33. Tulip API demos

34. Serial
http://www.youtube.com/v/jMgRUI7V_mk?
version=3&autoplay=1&&start=1546

35. Simultaneous
http://www.youtube.com/v/jMgRUI7V_mk?
version=3&autoplay=1&&start=1666

36. Wait for both
coroutines to finish
http://www.youtube.com/v/jMgRUI7V_mk?
version=3&autoplay=1&&start=1713

37. Loop through
coroutine results in
order of finishing
http://www.youtube.com/v/jMgRUI7V_mk?
version=3&autoplay=1&&start=1869

38. Execute synchronous
code in another thread
http://www.youtube.com/v/jMgRUI7V_mk?
version=3&autoplay=1&&start=1930

39. Talk to event loop from
another thread
http://www.youtube.com/v/jMgRUI7V_mk?
version=3&autoplay=1&&start=2060

40. A regex debugging tool
http://www.youtube.com/v/jMgRUI7V_mk?
version=3&autoplay=1&&start=2158

41. Chat application
http://www.youtube.com/v/jMgRUI7V_mk?
version=3&autoplay=1&&start=2236

42. Tulip Links
• PEP 3156
• Guido's PyCon 2013 keynote
• Tulip project page on GoogleCode

43. Other links
• An Introduction to Asynchronous
Programming and Twisted
• Code for this talk on Github
• Viol project page on Github

44. Questions?