Slides from "Testing in Django", the Django Boston Meetup on January 17, 2013.

1. testing in django
(browser-based testing too)
17 january 2013
Kevin Harvey
@kevinharvey
kevin@storyandstructure.com

2. Who is this guy?
Application Architect at story+structure
Djangonaut since 2007 (version 0.96)
For those of you who didn’t read the bio, I’m Kevin Harvey. I’m the Application Architect at story
+structure. We’re based in Brookline and we do software for higher ed. I cut my teeth on Django 0.96
in 2007 with djangobook.com

3. These are my twin sons Lane & Hank. They are almost 5 months old and awesome.

4. Who are y’all?
A quick poll...
I’d like to get an idea of who’s in the crowd tonight. If you would please raise your hand if you’ve:
- ever started a Python interpreter in Terminal or Command Prompt
- ever started a Django project (even just to play around)
- gone through the “Polls” app in the Django docs (or feel that you could)
- written a single automated test (even just to play around)
- released or written a project with a good test suite

5. What is a test?
An evolutionary perspective
So, what is a test? I’d like to answer that question from an evolutionary perspective.

6. What is a test?
An evolutionary perspective
def multiply_these_two_numbers(a,b):
! """
! Add 'b' to itself 'a' times
! """
! i = 0
! product = 0
! while a > i:
! ! product += b
! ! i += 1
! return product
Let’s say you wrote the function “multiply_these_two_numbers()”, which takes two arguments and adds
the second argument to itself by a factor of the first argument. It’s not pretty....

7. What is a test?
An evolutionary perspective
>>> from myapp.my_funcs import multiply_these_two_numbers
>>> multiply_these_two_numbers(4,5)
20
>>> multiply_these_two_numbers(7,8)
56
>>> multiply_these_two_numbers(720,617)
444240
... but it works! You can import it, you can give it two arguments, and it returns what you would
expect.

8. What is a test?
An evolutionary perspective
>>> from myapp.my_funcs import multiply_these_two_numbers
>>> multiply_these_two_numbers(4,5)
20
This is a test. No, it’s not automated, and yes you wrote it by hand in the terminal. But you did confirm
that the function:
1) was able to be imported,
2) took two arguments, and
3) returned the expected result.

9. What is a test?
An evolutionary perspective
from myapp.my_funcs import multiply_these_two_numbers
import math
def area_of_rectangle(length, width):
! """
! Length times width
! """
! return multiply_these_two_numbers(length, width)
!
def surface_area_of_sphere(radius):
! """
! 4 times pi times the radius squared
! """
! radius_squared = multiply_these_two_numbers(radius, radius)
! four_times_pi = multiply_these_two_numbers(4, math.pi)
! return multiply_these_two_numbers(radius_squared, four_times_pi)
You tell your colleagues about your new function, and they’re impressed. They start using your
function in other parts of the application.
As the code base grows, you’re starting to realize how important your little function is.

10. What is a test?
An evolutionary perspective
from myapp.my_funcs import multiply_these_two_numbers
print multiply_these_two_numbers(4,5)
$ python my_tests.py
20
$
As you find yourself checking that the function works more and more, you decide to save your test in a
file called ‘my_tests.py’ with a print statement to verify the result. Now you can run this saved
statement whenever you want and know that your function is working.

11. What is a test?
An evolutionary perspective
>>> import time
>>> time.sleep(604800)
Weeks go by. You work on other parts of the project. You work on different projects. You code in
different languages.

12. You have kids. Your mind is erased.

13. What is a test?
An evolutionary perspective
$ python my_tests.py
20
$
You’ve totally forgotten the context for this test. What does ’20’ mean? You would have to look at the
file to see what the inputs were. In short, you don’t know whether this test passed or failed. Can’t we
just let Python handle all of this stuff, from setting up the test to remembering what the appropriate
output should be?

14. What is a test?
An evolutionary perspective
from django.test import TestCase
from myapp.my_funcs import multiply_these_two_numbers
class SillyTest(TestCase):
def test_multiply_these_two_numbers(self):
"""
Tests that the function knows how to do math
"""
self.assertEqual(multiply_these_two_numbers(4,5), 20)
Enter Django’s TestCase with it’s assertEqual() method (based on Python’s own ‘assert’). Here we have
a test that knows how to run itself, and knows whether it failed or not.

15. What is a test?
An evolutionary perspective
$ python manage.py test myapp
Creating test database for alias 'default'...
.
---------------------------------------------------------
Ran 1 test in 0.000s
OK
Destroying test database for alias 'default'...
Just as a primer, here’s what it looks like when we run that test and it passes. We’ll talk about running
tests and the output you get in a moment.

16. Why write tests?
Why are we doing this?
What’s the point of all these tests? At the outset, it looks like more work:
- more code to write
- more code to debug
... but in fact, we write tests ....

17. Why write tests?
1. Drink More Beer
To drink more beer, or because we’re lazy. However you want to describe it, we write tests because we
want to fix stuff once and only once. Stop worrying about that rickety linchpin of a function holding
your entire project together and write a test for it.

18. Why write tests?
2. Take the fear out of refactoring
Speaking of rickety functions, let’s look back at that awful function we wrote earlier.

19. Why write tests?
UGLY CODE
def multiply_these_two_numbers(a,b):
! """
! Add 'b' to itself 'a' times
! """
! i = 0
! product = 0
! while a > i:
! ! product += b
! ! i += 1
! return product
Bleh. I’d love to fix this, wouldn’t you?

20. Why write tests?
UGLY, VITAL CODE
from myapp.my_funcs import multiply_these_two_numbers
import math
def area_of_rectangle(length, width):
! """
! Length times width
! """
! return multiply_these_two_numbers(length, width)
!
def surface_area_of_sphere(radius):
! """
! 4 times pi times the radius squared
! """
! radius_squared = multiply_these_two_numbers(radius, radius)
! four_times_pi = multiply_these_two_numbers(4, math.pi)
! return multiply_these_two_numbers(radius_squared, four_times_pi)
But it’s used everywhere in our app! What if we screw it up during the refactor?

21. Why write tests?
UGLY, VITAL, TESTED CODE
from django.test import TestCase
from myapp.my_funcs import multiply_these_two_numbers
class SillyTest(TestCase):
def test_multiply_these_two_numbers(self):
"""
Tests that the function knows how to do math
"""
self.assertEqual(multiply_these_two_numbers(4,5), 20)
This test (plus a couple more) severely limit the possibility that we’d screw up this function during a
refactor. This test is a guarantee that the function, given two integers, will return the product of those
two integers.

22. Why write tests?
BETTER CODE
def multiply_these_two_numbers(a,b):
! """
! Multiply a times b
! """
#! i = 0
#! product = 0
#! while a > i:
#! ! product += b
#! ! i += 1
#! return product
! return a*b
So refactor...

23. Why write tests?
The test guarantees the refactor
$ python manage.py test myapp
Creating test database for alias 'default'...
.
---------------------------------------------------------
Ran 1 test in 0.000s
OK
Destroying test database for alias 'default'...
... and run your tests. If the tests pass, you’re refactor works. Now wouldn’t it be nice if there were
tests for all those other functions that your team wrote...

24. Why write tests?
3. Explain your code
Good tests will serve as technical documentation for your code. You can show your colleagues how
your code works by walking them through your tests. And get new developers up to speed quickly by
explaining how your tests work.

25. Why write tests?
4. Clarify your thinking
This is sort of related to the last one, but for me writing tests really help me think through my app. If
all my code gets tested, I know I’m only writing the code necessary to get the job done. Testing forces
you to write code that’s more modular, which is easier to debug.

26. Why write tests?
5. Allow more developers to contribute
Think back about the first version of our multiply function: that code was obviously written by a junior
developer. Kinda nasty, BUT IT WORKED. It got us from point a to point b. We moved forward because
that junior developer contributed a function. We can hack faster because of the test, and we can go
back and clean up the mess later.

27. Why write tests?
6. Be taken seriously
Are you creating something you intend for other developers to use? The first thing I do when
evaluating a package from PyPI is check out the tests. It helps me to understand the code, and let’s me
know that the package is going to do what it’s supposed to do. It also gives me a glimpse into the way
(or whether) the developer thought about the package during development.

28. Why write tests?
7. Try something out before you screw up
your dev environment
This is specific to Django, but I think it’s worth mentioning here.

29. Why write tests?
Keep your dev database clean
$ python manage.py test myapp
Creating test database for alias 'default'...
.
---------------------------------------------------------
Ran 1 test in 0.000s
OK
Destroying test database for alias 'default'...
Did you notice the “Creating test database” bit when we ran the test? Django tests create a database
from scratch for you every time you run them, and deletes the database when your done. That means
you can write tests for new model fields and not have to do the syncdb/migrate/upgradedb dance.

30. Types of Tests
1. Functional tests
2. Unit tests
3. Performance tests (which I won’t cover)
The differences between functional tests exist on a series of spectra.

31. What’s the Difference?
Unit Functional
Fewer things Many things
(ideally one) (possibly all)
Unit tests test a small number of things in your app. Functional tests test a lot.

32. What’s the Difference?
Unit Functional
Small, write many Big, write few
Unit tests are small and you’ll write a ton. Functional tests are big and you’ll write just a few.

33. What’s the Difference?
Unit Functional
Stuff developers Stuff users
care about care about
Unit tests in general cover things that developers are worried about. Functional tests test things users
care about.

34. What’s the Difference?
Unit Functional
Fast Slow
Unit tests run fast (thousandths of a second). Functional tests run slow (seconds or more).

35. A few examples
Django tests in action!

36. A few examples
Unit test for Model
def test_questions_increment_votes_up(self):
! """
! Test voting up a question
! """
! question_1 = Question(text="How can my team get started?",
! ! ! ! ! ! votes=7)
! ! ! ! ! !
! question_1.increment_votes(4)
!
! self.assertEquals(question_1.votes, 11)
Unit testing a model.

37. A few examples
Unit test for Model
class Question(models.Model):
! text = models.CharField("Question", max_length=500)
! votes = models.IntegerField()
!
! def increment_votes(self, num):
! ! self.votes += num
! ! self.save()
This code makes the previous test pass.

38. A few examples
Unit test for a Form
def test_question_form_excludes_all_but_text(self):
! """
! The user can only supply the text of a question
! """
! form = QuestionForm()
! self.assertEquals(form.fields.keys(), ['text'])
! self.assertNotEquals(form.fields.keys(),
['text', 'votes'])

39. A few examples
Unit test for a Form
class QuestionForm(forms.ModelForm):
! class Meta:
! ! ! model = Question
! ! ! fields = ('text',)
Make the previous form unit test pass.

40. A few examples
Unit test for a POST action
def test_ask_question(self):
! """
! Test that POSTing the right data will result in a new question
! """
! response = self.client.post('/ask/',
{'text': 'Is there any more pizza?'})
! !
! self.assertRedirects(response, '/')
! !
! self.assertEqual(Question.objects.filter(text='Is there any
more pizza?').count(), 1)

41. A few examples
Unit test for a POST action
urlpatterns = patterns('',
...
url(r'^ask/$', 'questions.views.ask', name='ask'),
...
def ask(request):
if request.method == "POST":
! question_form = QuestionForm(request.POST)
! question_form.save()
return HttpResponseRedirect('/')
You have to edit two files to get the previous test to pass.

42. A few examples
Functional test: Logging in
def test_admin_can_manage_questions(self):
self.browser.get(self.live_server_url + '/admin/')
! !
username = self.browser.find_element_by_css_selector("input#id_username")
username.clear()
username.send_keys("peter")
password = self.browser.find_element_by_css_selector("input#id_password")
password.clear()
password.send_keys("password")
self.browser.find_element_by_css_selector("input[type='submit']").click()
body = self.browser.find_element_by_tag_name('body')
self.assertIn('Site administration', body.text)
This test uses Selenium to test that an admin user is able to login in to the admin site.

43. A few examples
Functional test: Logging in
# set up the admin site
Just set up the admin site to get it to pass.

44. Test Driven Development
What is Test Driven Development?
Simply put, TDD means writing tests that fail and THEN writing the code to make them pass. Guards
against code explosion because you only write enough code to make the test pass.

45. TDD by Example
Our example project: Torquemada

46. TDD by Example
http://bit.ly/XHjcAi
or
http://infinite-meadow-8366.herokuapp.com/
Code
https://github.com/kcharvey/testing-in-django
or use the link at the demo
See the demo at either of the first URLs. The code is available on GitHub.

47. TDD by Example
Our example project: Torquemada
Torquemada allows attendees at “Testing in Django
(browser based testing too)” to inquire of the presenter
asynchronously.
http://bit.ly/XHjcAi
A description of the app we’ll be building.

48. TDD by Example
Our example project: Torquemada
Use Case: Isabella the Inquisitive
Isabel has learned a lot by attending the “Testing in
Django” MeetUp but still has a few questions for the
presenter. She visits Torquemada in her web browser,
where she can see if any of her inquiries have been
addressed and see the question that is being currently
discussed. She is able to ‘vote up’ questions she would
like the presenter to answer, and ‘vote down’ questions
she thinks are unimportant. She is also able to ask her
own question.
http://bit.ly/XHjcAi
A use case for a user.

49. TDD by Example
Our example project: Torquemada
Use Case: Peter the Presenter
Peter is presenting at the “Testing in Django” MeetUp
and would like to answer any questions the attendees
may have during set periods in the talk. He views
Torquemada in his web browser to see what questions
attendees have asked, including relative importance
based on the number of votes they’ve received. When
the group is discussing a question, he uses the Django
admin site to set the question’s status to “Current”. After
the question has been discussed, he sets the status to
“Archived” http://bit.ly/XHjcAi
A use case for an admin.

50. TDD by example
Let’s do this.

51. Screencast 1
http://vimeo.com/57692050
(watch the video, then come back for the rest of the slides)
1) Set up a workspace 2) create a virtualenv 3) install django 4) startproject 5) startapp 6) test the app.
You can start TDD on a Django project without even touching settings.py!

52. TDD by example
from django.test import TestCase
class SimpleTest(TestCase):
Text
def test_basic_addition(self):
"""
Tests that 1 + 1 always equals 2.
"""
self.assertEqual(1 + 1, 2)
The test that ‘manage.py startapp’ generated.

53. Screencast 2
Text
http://vimeo.com/57693303
Replace the auto generated test with a meaningful one.

54. TDD by example
from django.test import LiveServerTestCase
from selenium import webdriver
class QuestionsTest(LiveServerTestCase):
Text
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(3)
def tearDown(self):
self.browser.quit()
...
Start with a functional test that tells (at least part of) a use case.

55. TDD by example
...
class QuestionsTest(LiveServerTestCase):
...
def test_can_read_v..._a_question(self):
Text browser and
# Isabel opens her web
# visits Torquemada
self.browser.get(self.live_server_url + '/')
# TODO
self.fail('finish this test')

56. Screencast 3
Text
http://vimeo.com/57692852
Extend the functional test with some use case as comments, and add a test for an h1 element.

57. TDD by example
Text
# She knows it's Torquemada because she sees the name in the heading
heading = self.browser.find_element_by_css_selector("h1#trq-heading")
self.assertEqual(heading.text, "Torquemada")
Here’s the assertion, using Selenium.

58. Screencast 4
Text
https://vimeo.com/57693096
Watch the new version of the functional test fail.

59. TDD by example
We have a working (but failing) test.
Commit it.
Meaningful failure = important unit of work we’ve done. Let’s commit it.

60. TDD by example
Let’s refactor a bit...

61. Screencast 5
Text
http://vimeo.com/57693617
Refactor the test.py file into a tests package.

62. TDD by example
... and commit that.

63. TDD by example
So, where are we exactly?

64. Screencast 6
Text
http://vimeo.com/57694735
In the final screencast, we finish up this round of TDD: dive into a unit test for a view, write the code to
get the test to pass, extend the unit test to cover more functionality, get it to pass, then confirm that
we’ve satisfied our functional test (as it is).

65. TDD by example
Rinse and repeat.
Repeat the cycle of writing tests that fail (or extending tests to make them fail) and writing code to
make the tests pass.

66. How about a demo?

67. Getting Around with Selenium
self.browser.get()
This command (if you’ve set self.browser = webdriver.FireFox(), or something like it), does a GET to
open a web page with Selenium. It can open any URL.

68. Getting Around with Selenium
self.browser.find_element_by_<METHOD>()
Find elements on the page using one of Seleniums ‘find_element’ methods.

69. Getting Around with Selenium
find_element_by_id
find_element_by_name
find_element_by_xpath
find_element_by_link_text
find_element_by_partial_link_text
find_element_by_tag_name
find_element_by_class_name
find_element_by_css_selector # use this one
find_element_by_css_selector is my favorite. It feels like jQuery.

70. Getting Around with Selenium
find_elements_by_id
find_elements_by_name
find_elements_by_xpath
find_elements_by_link_text
find_elements_by_partial_link_text
find_elements_by_tag_name
find_elements_by_class_name
find_elements_by_css_selector
# return Python lists
You can get lists of elements by pluralizing ‘elements’ in the method name.

71. Getting Around with Selenium
element.click()
Once you find an element, you can .click() it. This will behave just like a real user click, and respects an
JavaScript event.preventDefault() handlers.

72. Getting Around with Selenium
Working with form elements
text_field = self.browser.find_element_by_css_selector("#id_text")
text_field.clear()
text_field.send_keys("Why aren't you using reverse()?")
self.browser.find_element_by_css_selector("input#trq-submit-
question").click()
Use .clear() and .send_keys() to type test into a form element.

73. How do I get started?
I want to start testing on my team. What’s
the best way?

74. How do I get started?
1. Write tests for bug reports
An excellent way to start is to write tests for your bug reports. Find out the exact steps it takes to
reproduce a problem and write either a functional or unit test THAT FAILS as an illustration of the bug.
Then, fix the bug. Check in your fix and your test and rest easy knowing you’ve GUARANTEED that the
bug is fixed.

75. How do I get started?
2. Use TDD for new features
There’s no time like the present to start writing tests. When your team decides on a new feature to
implement, start by writing some use cases and develop from a functional test.

76. How do I get started?
3. Write unit tests for existing code you use
while doing 1. and 2.
If you’re using a function from somewhere else in the system when you write code, you need to be
able to guarantee that function does what you expect it to. Get in the habit of writing tests for parts of
your project as you come in to contact with them, PARTICULARLY if they are involved in a bug fix.

77. What about continuous
integration?
Continuous integration tools automatically checkout our code, build it, and run the tests. It’s to protect
ourselves from developers on our team that forget to run the tests before they commit.

78. What about continuous
integration?
We’re trying out Jenkins.
s+s is just starting to play with Jenkins, a CI platform in written in Java. NORMALLY, I’M LIKE...

79. .WAR

80. WHAT

81. IS

82. IT

83. GOOD

84. FOR?

85. What about continuous
integration?
But in this case I’ll make
an exception.
Jenkins is super easy to get up and running on your local machine, and there are plugins that play nice
with tools we’re all using.

86. Configuring Jenkins
$ pip install django-jenkins
INSTALLED_APPS = (
...
'django_jenkins',
)
...
JENKINS_TASKS = (
'django_jenkins.tasks.run_pylint',
'django_jenkins.tasks.with_coverage',
'django_jenkins.tasks.django_tests',
# there are more of these
)
$ python manage.py jenkins # Jenkins will run this command
django-jenkins is a plugin that runs out tests and outputs the files Jenkins needs to show our build
stats. pip install and add just a few lines to your settings.py

87. Configuring Jenkins
$ wget http://mirrors.jenkins-ci.org/war/latest/jenkins.war
$ java -jar jenkins.war
http://localhost:8080
Just get the .war file, run it, and hit port 8080 on your machine.

88. Configuring Jenkins
You’ll need some plugins:
- Jenkins Violations Plugin
- Jenkins Git Plugin
- Jenkins Cobertura Plugin
Install a few plugins.

89. Configuring Jenkins
1. Configure a new test (name, description)
2. Give it your repo URL
3. Tell it how often to build
4. Tell it the commands to run
5. Configure where to save the reports
6. Click “Build Now”
Check out the tutorials in the “Resources” section of this slide deck for more on configuring your repo.
It’ll take about 15 minutes the first time.

90. Configuring Jenkins
DONE
That’s it.

91. Next Steps
508 compliance
http://wave.webaim.org/toolbar
JavaScript Testing
Doctest.js
http://doctestjs.org/
There are things we definitely didn’t test. We’re looking into automated 508 compliance testing. If
you’re working in higher ed or the non-profit world, a non-compliant template should definitley
“break the build”.

92. Resources
Tutorials
• http://www.tdd-django-tutorial.com/
• http://toastdriven.com/blog/2011/apr/
10/guide-to-testing-in-django/ (and part 2)
•http://code.google.com/p/selenium/wiki/
IPhoneDriver

93. Resources
Jenkins
• https://github.com/kmmbvnr/django-
jenkins
• https://sites.google.com/site/kmmbvnr/
home/django-jenkins-tutorial
• http://jenkins-ci.org/

94. Resources
Performance Testing
• http://funkload.nuxeo.org/
• http://jmeter.apache.org/
• http://httpd.apache.org/docs/2.2/
programs/ab.html

95. Using fixtures
Manage data for tests runs
Sometimes you want to create models explicitly. Other times you want to create a lot of reusable data.
Enter fixtures.

96. [
Using fixtures
{
"pk": 1,
"model": "questions.question",
"fields": {
"status": "new",
"text": "How can my team get started with
testing?",
"votes": 0,
"created": "2013-01-17T16:15:37.786Z"
}
},
{
"pk": 2,
"model": "questions.question",
"fields": {
"status": "new",
"text": "Does Selenium only work in
Firefox?",
"votes": 0,
"created": "2013-01-17T16:17:48.381Z"
}
}
]
A fixture is just structured data (defaulting to JSON) that Django knows how to import and export:
- loaddata
- dumpdata

97. Using fixtures
Do NOT try to write fixtures by hand
$ python manage.py runserver
...
# Use the Django /admin site to make some test data
...
$ mkdir questions/fixtures/
$ python manage.py dumpdata questions --indent=4 >
questions/fixtures/questions.json
Use the admin site and ‘manage.py dumpdata’ to make fixtures easily.