The 2012 campaign to re-elect U.S. President Barack Obama took data and testing to a new level, not just in the field but in its digital operations as well. The campaign raised over half a billion dollars online, much of which can be traced directly to the rigorous testing and data-driven decision-making that defined the campaign’s culture. This rapid-fire training session will give you an insider’s view of the campaign’s most effective techniques for raising money and mobilizing supporters online, particularly through email. The campaign’s Director of Digital Analytics, Amelia Showalter, will show you how to design experiments that go beyond basic split testing, how to incorporate demographic and behavioral data into your outreach, and how to foster a culture of testing and innovation. Plus, you’ll learn surprising lessons and anecdotes from inside the campaign, with the goal of helping you brainstorm new ways to optimize your own digital operations.

1. Obama 2012: Lessons from a data-driven campaign

2. You probably thought we had it easy…

3. But victory was never assured
 Big-spending groups called Super-PACs were
largely supporting Romney
 The progressive base was less enthusiastic
than in 2008
 Obama was usually ahead in the polls, but the
advantage was narrow and volatile

4. It wasn’t 2008 (which O won by 7.3%)

5. So how did we win?
 Massive numbers of field offices
 786 Obama field offices vs. 284 Romney offices
 In Ohio:
Obama Romney
 Field offices are where hundreds of paid
staffers organized tens of thousands of
volunteers to mobilize millions of voters
Source: TheMonkeyCage.Org

6. So how did we win?
 Sophisticated technology and targeting
 Giant data & micro-targeting operation
 Fully integrated databases
 Smarter new methods of targeting TV ads
 New technology for social sharing, polling place
lookup, phone banking, volunteer
mobilization, vote tracking, election day rapid
response
 Thousands of staffers in Chicago HQ and the
battleground states

7. All of that costs money
 In 2008, Obama campaign raised $750 million
 Not enough to beat Romney 2012 & Super-PACs
 But fundraising was more difficult in 2012
 President less available for fundraising events
 In early campaign, we saw average online
donation was half of what it had been in 2008
 People were giving less, and less often
 We had to be smarter, and more tenacious

8. Obama 2012 Digital Department
 Digital department was very focused on
data, using evidence to drive decisions
 We had a culture of testing
 Constantly looking for ways to improve
 Humility was required
 Creativity was encouraged
 Lots of staff to create content, run
experiments, analyze results, develop new
technology

9. What impact can testing have?
Test sends Full send (in millions)
version Subject line donors money
v1s1 Hey 263 $17,646 $4
v1s2 Two things: 268 $18,830
v1s3 Your turn 276 $22,380 $3
v2s1 Hey 300 $17,644
v2s2 My opponent 246 $13,795 $2
v2s3 You decide 222 $27,185
v3s1 Hey 370 $29,976 $1
v3s2 Last night 307 $16,945
v3s3 Stand with me today 381 $25,881 $0
v4s1 Hey 444 $25,643 ACTUAL IF IF
v4s2 This is my last campaign 369 $24,759 ($3.7m) SENDING SENDING
v4s3 [NAME] 514 $34,308 AVG WORST
v5s1 Hey 353 $22,190
There won't be many more
v5s2 of these deadlines 273 $22,405
 $2.2 million additional revenue
v5s3 What you saw this week 263 $21,014 from sending best draft vs.
v6s1 Hey 363 $25,689 worst, or $1.5 million additional
v6s2 Let's win. 237 $17,154
v6s3 Midnight deadline 352 $23,244
from sending best vs. average

10. Testing = constant improvement
 Little improvements add up
 Improving 1% here and 2% there isn’t a lot at
first, but over time it adds up

11. Testing = data-driven decisions
 We don’t have all the answers
 Conventional wisdom is often wrong
 Gut instinct is often wrong
 On the Obama campaign we had this thing
called the Email Derby…
 Even the experts can’t reliably predict the most
effective messages
 Listen to your audience!

12. Experiments: Email Volume
 More email = more donations
 People say they get too much email
 But our experiment showed that sending a higher
volume of fundraising emails led to more
donations, without dire consequences
 Implementing a “more email” policy probably
led to $20-30 million in additional revenue for
the campaign

13. Experiments: Ugly vs. pretty
 At first we tried making emails prettier, with slick-
looking buttons and nice formatting
 That failed, so we asked: what about ugly instead?
 Ugly yellow highlighting got us better results
 But at some point it lost its novelty and stopped
working – always important to re-test!

14. Experiments: Personalization
 Adding “drop-in sentences” that reference people’s
past behavior can increase conversion rates
 Example: asking recent donors for more money
…it's going to take a lot more of us to …it's going to take a lot more of us to
match them. match them.
Will you donate $25 or more today? You stepped up recently to help out --
thank you. We all need to dig a little
deeper if we're going to win, so I'm
asking you to pitch in again. Will you
donate $25 or more today?
 The added sentence doubled the donation rate
 Confirmed in several similar experiments

15. Technologies: Saved Payment Info
 Campaign introduced “Quick Donate”
system, allowing donors the option of saving
their credit card info for easier donating later
 Allowed one-click donations within email

16. Obama 2012: Digital Department
 Raised over half a billion dollars online
 Recruited tens of thousands of volunteers
 Publicized thousands of events and rallies
 Did I mention raising half a billion dollars?
 Conservatively, testing resulted in about $100-
200 million in additional revenue

17. Digital Department Basics
 Grew from a small team in spring 2011 to a
department of 200+ in 2012
 Outbound (email, social, mobile, blog)
 Ads
 Front-End Development
 Design
 Video
 Project management
 Digital Analytics

18. Digital Analytics
 Goal was to help all the other teams do better
 Design & implement experiments
 Analyze & report results
 Provide data sets
 Three overlapping skill groups within team:
 Database management (SQL, Python)
 Data analysis (Stata, R, SPSS)
 Web analytics (Google Analytics, Optimizely)

19. Testing requires content
 Content creators
 Need more of these than you think
 Obama team had 18 email writers, 4 social media
writers, many designers and ad producers
 More people = more ideas
 Front-end developers
 Create new pages, improve user experience, etc

20. Conclusions
 Data-driven campaigning helped Obama win
 Big groups of smart people can accomplish a lot!
 But you don’t have to have a staff of hundreds to
have a good testing program
 Train existing staffers, hire more when you can
 Foster a culture of testing: every piece of
communication is an opportunity to test something
 Even a small list can be split in two – do what you can