Describing how you can use python generator functions to process a file without loading it all in memory before starting

1. Python Generators
PySthlm Meetup #15 – Python
Best Practices – Oct 31, 2013
Samuel Lampa
twitter.com/smllmp

2. The “normal” (bad) way

3. The “normal” (bad) way
New temporary data structures
created and kept in memory
== BAD!!!

4. Using Generator function

5. Using Generator function
A yield (“return”) for
every iteration, so it can
be used immediately
without temp data structure
== GOOD!

6. Using Generator function
A yield (“return”) for
every iteration, so it can
be used immediately
without temp data structure
== GOOD!
So, this is the
generator function

7. Using Generator function
A yield (“return”) for
every iteration, so it can
be used immediately
without temp data structure
== GOOD!
So, this is the
generator function
… which can be
iterated over, like this ...

9. Using list-comprehension syntax

10. Using list-comprehension syntax
Here, the generator function
Is defined in one row

11. Chaining generator functions

12. Chaining generator functions

13. Creds and more info:
●
David Beazleys Python Generator intro:
→dabeaz.com/generators
David Beazley on twitter: @dabeaz

14. Thank you!
Samuel Lampa
G+: gplus.to/saml
twitter: @smllmp
Blog: saml.rilspace.org
Developer @
www.uppmax.uu.se
www.farmbio.uu.se
www.rilpartner.se