Python's "batteries included" philosophy means that it comes with an astonishing amount of great stuff. On top of that, there's a vibrant world of third-party libraries that help make Python even more wonderful. We'll go on a breezy, example-filled tour through some of my favorites, from treasures in the standard library to great third-party packages that I don't think I could live without, and we'll touch on some of the fuzzier aspects of the Python culture that make it such a joy to be part of.
10. _ ( =
255,
lambda
V ,B,c
:c and Y(V*V+B,B, c
-1)if(abs(V)<6)else
( 2+c-4*abs(V)**-0.4)/i
) ;v, x=1500,1000;C=range(v*x
);import struct;P=struct.pack;M,
j ='<QIIHHHH',open('M.bmp','wb').write
for X in j('BM'+P(M,v*x*3+26,26,12,v,x,1,24))or C:
i ,Y=_;j(P('BBB',*(lambda T:(T*80+T**9
*i-950*T **99,T*70-880*T**18+701*
T **9 ,T*i**(1-T**45*2)))(sum(
[ Y(0,(A%3/3.+X%v+(X/v+
A/3/3.-x/2)/1j)*2.5
/x -2.7,i)**2 for
A in C
[:9]])
/9)
) )
http://preshing.com/20110926/high-resolution-mandelbrot-in-obfuscated-python
12. The Interactive Shell
$ python
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build
2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for
more information.
>>>
13. Docstrings...
class Frobulator(object):
"""It frobulates things."""
def __init__(self, doohickey):
"""A Frobulator needs a doohickey."""
self.doohickey = doohickey
def frobulate(self):
"""Frobulate ALL the things!"""
print "Frobulating..."
doodad = self.doohickey()
return FrobulatedThing(doodad)
class FrobulatedThing(object):
"""A thing which has been frobulated."""
def __init__(self, thing):
"""Make a thing into a frobulated thing."""
self.thing = thing
14. ...and Help
>>> help(Frobulator)
Help on class Frobulator in module frobulator:
class Frobulator(__builtin__.object)
| It frobulates things.
|
| Methods defined here:
|
| __init__(self, doohickey)
| A Frobulator needs a doohickey.
|
| frobulate(self)
| Frobulate ALL the things!
|
...
16. List Comprehensions
x = [item for item in series]
x = [do_something(item) for item in series if expression]
things = [Thingy.from_data(x) for x in database_results]
partiers = [x for x in codemashers if x.slides_done()]
17. List Comprehensions
booze = ['beer', 'wine', 'scotch', 'gin']
soft_drinks = ['water', 'soda', 'juice']
a = [(x, y) for x in booze for y in soft_drinks]
[('beer', 'water'), ('beer', 'soda'), ('beer', 'juice'),
('wine', 'water'), ('wine', 'soda'), ('wine', 'juice'),
('scotch', 'water'), ('scotch', 'soda'), ('scotch',
'juice'), ('gin', 'water'), ('gin', 'soda'), ('gin',
'juice')]
18. List Comprehensions
b = [x for x in zip(booze, soft_drinks)]
[('beer', 'water'), ('wine', 'soda'), ('scotch', 'juice')]
19. Set Comprehensions
s = {v for v in 'CODEMASH ROCKS' if v not in 'ABCD'}
set([' ', 'E', 'H', 'K', 'M', 'O', 'S', 'R'])
20. Dictionary Comprehensions
d = {key: value for key, value in list_of_tuples}
d = {
'Mike': 'Python',
'Jim': 'Ruby',
'Brad': 'UX',
...
}
d1 = {val: key for key, val in d.items()}
{'Python': 'Mike', 'Ruby': 'Jim', 'UX': 'Brad', ...}
21. Generators
def f(how_many):
for number in range(how_many):
if number**2 > 3:
yield number * 2
for number in f(5):
print number
4
6
8
26. Properties
>>> foo = Foo()
>>> foo.bar
42
>>> foo.bar = 'abc'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 9, in bar
ValueError: invalid literal for int() with base 10:
'abc'
27. Decorators
• That funny-looking @ thing
• Similar to Java’s annotations
• Replaces a function with a wrapper
• Augment functionality in a reusable way
28. Decorators
def be_excellent(wrapped):
def wrapper(*args, **kwargs):
print "Be excellent to each other..."
return wrapped(*args, **kwargs)
return wrapper
@be_excellent
def party_on(who):
print "...and party on, {0}!".format(who)
>>> party_on('dudes')
Be excellent to each other...
...and party on, dudes!
29. Context Managers
• That funny “with” thing
• Simplify calling code that needs setup and
teardown
• Alternative to try/finally structures
• Acquire locks, open and close files, do
database transactions, etc.
36. Itertools
• Tools for iterating on sequences
• Inspired by functional languages
• Fast and memory efficient
• Combine to express more complicated
algorithms
42. Itertools: cycle
from itertools import cycle
for x in cycle(['wake up', 'meet Ned', 'romance Rita']):
print x
wake up
meet Ned
romance Rita
wake up
meet Ned
romance Rita
wake up
meet Ned
...
43. Itertools: repeat
from itertools import repeat
for x in repeat("stop hitting yourself", 5):
print x
stop hitting yourself
stop hitting yourself
stop hitting yourself
stop hitting yourself
stop hitting yourself
44. Functools
• Tools for manipulating functions
• Partial
• Wraps a callable with default arguments
• Alternative to lambdas and closures
45. Functools: Partial
from functools import partial
def f(a, b=2):
print a, b
f1 = partial(f, 'fixed_a')
f2 = partial(f, b='fixed_b')
>>> f1(b=1138)
fixed_a 1138
>>> f2(1138)
1138 fixed_b
46. Functools: Partial
def category_is(category, item):
categories = [x.lower() for x in item.categories]
if category.lower() in categories:
return True
return False
is_python = partial(category_is, 'python')
is_cat_pictures = partial(category_is, 'catpix')
...
python_posts = [item for item in blog_posts if is_python(item)]
cat_posts = [item for item in blog_posts if is_cat_pictures(item)]
47. Collections
• Beyond the basic list, dict, tuple, and set...
• Counter
• Defaultdict
• OrderedDict
• Namedtuple
• ...and more
52. Namedtuple
import math
def distance(a, b):
return math.sqrt(
(a[0] - b[0])**2 +
(a[1] - b[1])**2 +
(a[2] - b[2])**2
)
a = (1, 2, 3)
b = (-1, -2, 42)
print distance(a, b)
39.25557285278104
53. Namedtuple
from collections import namedtuple
Point = namedtuple('Point', 'x y z')
def distance(a, b):
return math.sqrt(
(a.x - b.x)**2 +
(a.y - b.y)**2 +
(a.z - b.z)**2
)
a = Point(x=1, y=2, z=3)
b = Point(-1, -2, 42)
print distance(a, b)
39.25557285278104
54. Difflib
s1 = """
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Phasellus dui nunc, faucibus id ullamcorper eget, tempus
vitae nisl. Donec quis semper risus. Curabitur sit amet
tellus eget metus accumsan porta nec nec lorem. Ut vitae
sem nisl. Praesent pulvinar feugiat nibh fringilla
semper. Nullam cursus tempor lorem ut egestas. Nullam
suscipit gravida turpis ac porttitor. Curabitur eleifend
augue at risus commodo pretium. Aliquam eget magna
risus, ut lobortis metus. Cum sociis natoque penatibus
et magnis dis parturient montes, nascetur ridiculus mus.
Etiam non magna sit amet nulla porttitor molestie sit
amet vel sem. Vestibulum sit amet nisl a velit
adipiscing porta id non urna. Duis ullamcorper dictum
ipsum sit amet congue.
"""
55. Difflib
s2 = """
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Phasellus dui nunc, faucibus id ullamcorper eget, tempus
vitae nisl. Donec quis semper risus. Curabitur sit amet
tellus eget metus accumsan porta nec nec lorem. Ut vitae
sem nisl. Praesent pulvinar feugiat nibh fringilla
semper. Nullam cursus tempor lorem ut egestas. Nullam
suscipit gravida turpis ac porttitor. Curabitur eleifend
augue at risus commodo pretium. Aliquam eget magna
risus, ut lobortis montes. Cum sociis natoque penatibus
et magnis dis parturient metus, nascetur ridiculus mus.
Etiam non magna sit amet nulla porttitor molestie sit
amet vel sem. Vestibulum sit amet nisl a velit
adipiscing porta id non urna. Duis ullamcorper dictum
ipsum sit amet congue.
"""
57. Difflib
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Phasellus dui nunc, faucibus id ullamcorper eget, tempus
vitae nisl. Donec quis semper risus. Curabitur sit amet
tellus eget metus accumsan porta nec nec lorem. Ut vitae
sem nisl. Praesent pulvinar feugiat nibh fringilla
semper. Nullam cursus tempor lorem ut egestas. Nullam
suscipit gravida turpis ac porttitor. Curabitur eleifend
augue at risus commodo pretium. Aliquam eget magna
- risus, ut lobortis metus. Cum sociis natoque penatibus
? --
+ risus, ut lobortis montes. Cum sociis natoque penatibus
? +++
- et magnis dis parturient montes, nascetur ridiculus mus.
? ^^ ^
+ et magnis dis parturient metus, nascetur ridiculus mus.
? ^ ^
Etiam non magna sit amet nulla porttitor molestie sit
amet vel sem. Vestibulum sit amet nisl a velit
adipiscing porta id non urna. Duis ullamcorper dictum
ipsum sit amet congue.
59. Difflib
---
+++
@@ -7,8 +7,8 @@
semper. Nullam cursus tempor lorem ut egestas. Nullam
suscipit gravida turpis ac porttitor. Curabitur eleifend
augue at risus commodo pretium. Aliquam eget magna
-risus, ut lobortis metus. Cum sociis natoque penatibus
-et magnis dis parturient montes, nascetur ridiculus mus.
+risus, ut lobortis montes. Cum sociis natoque penatibus
+et magnis dis parturient metus, nascetur ridiculus mus.
Etiam non magna sit amet nulla porttitor molestie sit
amet vel sem. Vestibulum sit amet nisl a velit
adipiscing porta id non urna. Duis ullamcorper dictum
60. Ast
• Use Python to process abstract syntax trees
of Python grammar
• Helps introspect about what the current
grammar looks like
• Helps write secure code
62. Multiprocessing
• Like threading, but with subprocesses
• Local and remote concurrency
• Spawn processes or entire pools
• Communicate via queues or pipes
• Shared state via shared memory or
manager/server process
63. import os
from multiprocessing import Process
def info(title):
print title
print 'parent process:', os.getppid()
print 'process id:', os.getpid()
def f(name):
info('function f')
print 'hello', name
if __name__ == '__main__':
info('main')
p = Process(target=f, args=('world',))
p.start()
p.join()
64. $ python process.py
main
parent process: 18647
process id: 31317
----------
function f
parent process: 31317
process id: 31318
----------
hello world
65. import os
from multiprocessing import Pool
def g(x):
info('function g('+str(x)+')')
return x * x
if __name__ == '__main__':
info('main')
pool = Pool(2)
print pool.map(g, range(100))
66. $ python process.py parent process: 31369
main parent process: 31369
module name: __main__ process id: 31370
parent process: 18647 process id: 31371
process id: 31369 ----------
---------- ----------
function g(0) function g(4)
module name: __main__ module name: __main__
parent process: 31369 parent process: 31369
process id: 31370 function g(14)
---------- process id: 31370
function g(1) module name: __main__
module name: __main__ ----------
parent process: 31369 parent process: 31369
process id: 31370 function g(5)
---------- process id: 31371
function g(13) module name: __main__
function g(2)
module name: __main__ ...
parent process: 31369
process id: 31370 [0, 1, 4, 9, 16, 25, ..., 8836,
---------- 9025, 9216, 9409, 9604, 9801]
module name: __main__
function g(3)
module name: __main__
68. Third Party Packages
• Most available on PyPI--http://pypi.python.org
• pip install packagename
• easy_install packagename
69. Virtualenv
• Makes an isolated Python environment
• Don’t pollute your global site-packages
• Insulates Python projects from one another
• Don't have to be root
75. howoldismykid.com
def periods_between(freq, start_date, end_date):
# dateutil.rrule falls down for monthly recurrences where the start
# date's day is greater than the number of days in a subsequent month
# in the range; ie, when start date is 10/31/2010, and end date is
# 3/13/2011, rrule's between will only produce 2 instances, 12/31 and
# 1/31, rather than 4.
if freq == MONTHLY and start_date.day > 28:
start_date = datetime.datetime(start_date.year, start_date.month,
28, start_date.hour, start_date.minute, start_date.second)
# Same problem but for "Pirates of Penzance" leap day birthdays...
elif freq == YEARLY and is_leap_day(start_date):
start_date = datetime.datetime(start_date.year, start_date.month,
28, start_date.hour, start_date.minute, start_date.second)
rr = rrule(freq, dtstart=start_date)
periods = len(rr.between(start_date, end_date))
return periods
77. Nose
• Find, run, and report results of test suites
• Low-friction setup
• Extendable with plugins (more later)
• http://readthedocs.org/docs/nose/en/latest/
79. Nose
$ nosetests tests/test_foo.py
F.
======================================================================
FAIL: test_foo.test_a_thing
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/mpirnat/Code/frobulator/lib/python2.7/site-packages/
nose-1.1.2-py2.7.egg/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/Users/mpirnat/Code/frobulator/src/frobulator/tests/test_foo.py",
line 2, in test_a_thing
assert False
AssertionError
----------------------------------------------------------------------
Ran 2 tests in 0.001s
FAILED (failures=1)
80. Nose
class TestWhenFrobulating(object):
def setup(self):
self.frobulator = Frobulator()
self.frobulator.frobulate()
def test_that_it_frobulated(self):
assert self.frobulator.has_frobulated
def teardown(self):
self.frobulator.cleanup()
83. Nose Plugins
• Add custom behavior to your test run
• Get coverage statistics (coverage)
• Measure cleanliness with PEP-8 (tissue)
• Emit human-readable spec-style results
(pinocchio)
• Or write your own...
84. class TestGeneratesLotsOfFailures(object):
def test_generates_failures(self):
def _make_a_test(i):
# start with some wins
if i < 7:
assert True
# but then it hits the fan...
elif i < 30:
assert False
# then be a little random
elif i % 3 == 0:
assert False
else:
assert True
for i in range(50):
yield _make_a_test, i
89. $ nosetests --with-f7u12 test_epic_fail.py
.......FFFFFFFFUUUUUUUUUUUUUUUU..U..U..U..U..U..U.
[lots of test failure output; use your imagination...]
--------------------------------------------------------
Ran 50 tests in 0.010s
FAILED (errors=30)
90. Mock
• Mock object framework
• Avoid writing custom stubs
• Uses “Action/Assert” model (not “Record/
Replay”)
• Can also patch out module and class
attributes in the scope of a test
• http://www.voidspace.org.uk/python/mock/
94. Coverage
• Measure coverage of your codebase during
program execution
• Integrates with several test runners to
measure test coverage
• http://nedbatchelder.com/code/coverage/
95. #!/usr/bin/env python
class Frobulator(object):
"""It frobulates things."""
def __init__(self, doohickey):
"""A Frobulator needs a doohickey."""
self.doohickey = doohickey
def frobulate(self):
"""Frobulate ALL the things!"""
print "Frobulating..."
doodad = self.doohickey()
return FrobulatedThing(doodad)
class FrobulatedThing(object):
"""A thing which has been frobulated."""
def __init__(self, thing):
"""Make a thing into a frobulated thing."""
self.thing = thing
if __name__ == '__main__':
x = FrobulatedThing(42)
96. #!/usr/bin/env python
class Frobulator(object):
"""It frobulates things."""
def __init__(self, doohickey):
"""A Frobulator needs a doohickey."""
self.doohickey = doohickey
def frobulate(self):
"""Frobulate ALL the things!"""
print "Frobulating..."
doodad = self.doohickey()
return FrobulatedThing(doodad)
class FrobulatedThing(object):
"""A thing which has been frobulated."""
def __init__(self, thing):
"""Make a thing into a frobulated thing."""
self.thing = thing
if __name__ == '__main__':
x = FrobulatedThing(42)
97. $ coverage run frobulator.py
$ coverage report -m
Name Stmts Miss Cover Missing
------------------------------------------
frobulator 12 4 67% 8, 12-14
99. $ nosetests --with-coverage --cover-package frobulator
..
Name Stmts Miss Cover Missing
------------------------------------------
frobulator 12 1 92% 26
------------------------------------------
Ran 2 tests in 0.008s
OK
103. Feature: The website has a homepage
In order to demo Lettuce
As a presenter
I want to test the homepage of a website
Scenario: Verify the site is up
Given I access the url "http://localhost:8000/"
Then I get a status "200"
104. from lettuce import step, world
from nose.tools import assert_equals
import requests
@step(r'I access the url "(.*)"')
def access_url(step, url):
world.response = requests.get(url)
@step(r'I get a status "(.*)"')
def get_status(step, expected_status):
expected_status = int(expected_status)
assert_equals(world.response.status_code,
expected_status)
106. Snake-Guice
• Bind identifiers to things you want to inject
• Decorate code with injection hints
• Use injector to get instance with all
dependencies resolved and injected
• Construct with mocks in tests, minimize the
need for patching
132. Planet Python
• http://planet.python.org/
• Aggregate feed of Python blogs
• Great way to follow what's going on
• Minimize newsgroup/mailing list burdens
• Easy to be included
133. The Python Ecosystem
• http://mirnazim.org/writings/python-
ecosystem-introduction/
• Great introduction to the Python ecosystem
• Everything you need to get up and running
134. The Hitchhiker’s Guide
• http://docs.python-guide.org/
• Opinionated advice about using Python
• Basics
• Scenario-specific details and
recommendations
135. PyMotW
• Explored a different standard library module
every week
• Examples, examples, examples
• http://www.doughellmann.com/PyMOTW/
• The Python Standard Library by Example
http://www.doughellmann.com/books/byexample/
137. PyOhio
• Free as in $0.00
• Columbus, Ohio
• Last weekend in July
• http://pyohio.org
138. PyCon
• Great people
• Great vibe
• All volunteer–personal ownership
• Video! http://pycon.blip.tv/
• 2012 & 2013: Santa Clara
• 2014 & 2015: Montreal
139. TiP BoF
• Testing in Python Birds of a Feather
• Lightning Talks
• Heckling...
142. The Zen
• Beautiful is better than ugly. • Readability counts.
• Explicit is better than • Special cases aren't special
implicit. enough to break the rules.
• Simple is better than • Although practicality beats
complex. purity.
• Complex is better than • Errors should never pass
complicated. silently.
• Flat is better than nested. • Unless explicitly silenced.
• Sparse is better than dense.
143. The Zen
• In the face of ambiguity, • Although never is often
refuse the temptation to better than right now.
guess.
• If the implementation is
• There should be one– hard to explain, it's a bad
and preferably only one– idea.
obvious way to do it.
• If the implementation is
• Although that way may not easy to explain, it may be a
be obvious at first unless good idea.
you're Dutch.
• Namespaces are one
• Now is better than never. honking great idea -- let's
do more of those!