SlideShare una empresa de Scribd logo
1 de 19
Optimizing Your Marketing Mix
A case study
May 1, 2014
Robert Welborn
Associate Vice President, Decision Scientist
USAA
@robert_welborn
Cautionary Tale
• In 1949 Alexander Kotov was challenge by Tigran
Petrosian (The Iron Tigran)
•Petrosian is the acknowledged Defensive Master,
able to see 1,000’s of variations in play. He is the
“hardest to defeat player in the history of chess”
• Kotov on the other hand was the chess rockstar.
He hobnobbed with supermodels and nuclear
physicists.
•This was Superman fighting The Incredible Hulk in
the Soviet Chess world
•Petrosian surrendered in 13 moves
Source: Wikimedia Commons
Kotov Syndrome
• A Player, confronted by 1,000’s of possible scenarios
• The obvious is missed
• The player makes decisions based on emotion and
expediency, instead of facts and discipline
Industrial Analysis requires Decision Frameworks
not just Decision Making Tools
Analytics must produce Insights, not reports, and
the Insights must be Useful (Results are better than
elegance)
The Usefulness of the insights is a function of the
integration of your analysts (not just your analysis)
Three Keys
Building Decision Frameworks
(not just buy more tools)
Organize To Use Specialists
Start with the CFO
Publish for the C-suite
Source: Beleant, 2007 “Paleolithic Flints” used under Michigan
Publishing Creative Commons http://hdl.handle.net/2027/spo.7523862.0005.006
When to Pay Attention to Analytics
The General Practitioner versus the Specialist
• When the GP says you are sick, you trust the GP to run some
tests
• When the specialist says that you have Achilles tendonopathy,
you go have surgery
•You want your analysts to be specialists
Source: Wikimedia Commons
So Who are The Specialists
The Threefold Path of Analysis
Ask someone
Experiment
Build a model
The CFO wants marketing to work
She may not know why
Or how it works
•
Transform the CMO into a portfolio
manager
•
Analytics becomes the risk manager
Publish for the C-suite
Connect competitor action and macroeconomic analysis to your marketing
Allow real scenario analysis, "what if"
Get out of the PowerPoint trap and into white papers and long reads
Become the CEO's weekend reading
Source: Longreads.com
Analytics Must Produce Insights
(Focus on Results)
• The Results should drive where
you start, remain, and stop looking
• Converting Invested Money into
Products is the kernel of every
useful insight in Marketing
Insights must be useful
• The more useful the insights, the
more credible your analysts are
• The more
http://blog.uber.com/wp-content/uploads/2011/12/New-Logo-Vertical-Dark.jpghttp://blog.uber.com/wp-content/uploads/2011/12/New-Logo-Vertical-Dark.jpg
Conclusion
• Align your organization for using specialists, and set them up so
that there is no punishment for telling the truth
• You need to hire the following specialists: Survey-ers,
Experimenters, and Modelers
• Connect the stuff the CFO wants to what the CMO delivers and
manage the risk
• Publish for the C-Suite and use your credibility
• When you are certain of something, do something
Questions?
Appendix
Alignment (or no one grades their own paper)
Keep the “Measure” separate from “Recommend” but using the
same data
Separate Research from Analytics
Ask someone -
• Survey
• Focus Group
• Social or Listening
Experiment
•
Test vs Control
•
Multivariate
Separate the method of marketing from
the method of measurement
Certainty in a measurement is not the same as certainty in results
Paid Search vs. Lift over control
Models
•
Audiences
•
Marketing Mix
•
Funnel Analytics
“Art is a lie that helps you understand the truth. Models are
abstractions that help you understand reality.” – Pablo Picasso

Más contenido relacionado

Más de Cross Channel Marketing Optimization Group (9)

Engaging with the Omni-Channel Consumer - Trappings & Opportunities
Engaging with the Omni-Channel Consumer - Trappings & OpportunitiesEngaging with the Omni-Channel Consumer - Trappings & Opportunities
Engaging with the Omni-Channel Consumer - Trappings & Opportunities
 
Multi-Channel Data Activation: Closing the Divide Between Offline and Online
Multi-Channel Data Activation: Closing the Divide Between Offline and OnlineMulti-Channel Data Activation: Closing the Divide Between Offline and Online
Multi-Channel Data Activation: Closing the Divide Between Offline and Online
 
Successful Marketing Requires Even Better Measurement and Analytics
Successful Marketing Requires Even Better Measurement and AnalyticsSuccessful Marketing Requires Even Better Measurement and Analytics
Successful Marketing Requires Even Better Measurement and Analytics
 
XCMO 2014 Opening Remarks
XCMO 2014 Opening RemarksXCMO 2014 Opening Remarks
XCMO 2014 Opening Remarks
 
XCMO 2013: Online & Offline Analytics Convergence (Peanut Butter & Chocolate)
XCMO 2013: Online & Offline Analytics Convergence (Peanut Butter & Chocolate)XCMO 2013: Online & Offline Analytics Convergence (Peanut Butter & Chocolate)
XCMO 2013: Online & Offline Analytics Convergence (Peanut Butter & Chocolate)
 
XCMO 2013: Engagement from a NASCAR Perspective
XCMO 2013: Engagement from a NASCAR PerspectiveXCMO 2013: Engagement from a NASCAR Perspective
XCMO 2013: Engagement from a NASCAR Perspective
 
XCMO 2013: The Expanding Role of Mobile in the Marketing Mix
XCMO 2013: The Expanding Role of Mobile in the Marketing MixXCMO 2013: The Expanding Role of Mobile in the Marketing Mix
XCMO 2013: The Expanding Role of Mobile in the Marketing Mix
 
XCMO 2013: Cross-Channel Audience Management
XCMO 2013: Cross-Channel Audience ManagementXCMO 2013: Cross-Channel Audience Management
XCMO 2013: Cross-Channel Audience Management
 
XCMO 2013: Opening Remarks
XCMO 2013: Opening RemarksXCMO 2013: Opening Remarks
XCMO 2013: Opening Remarks
 

Último

The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 

Último (20)

The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 

Optimize Your Marketing Mix

  • 1. Optimizing Your Marketing Mix A case study May 1, 2014 Robert Welborn Associate Vice President, Decision Scientist USAA @robert_welborn
  • 2. Cautionary Tale • In 1949 Alexander Kotov was challenge by Tigran Petrosian (The Iron Tigran) •Petrosian is the acknowledged Defensive Master, able to see 1,000’s of variations in play. He is the “hardest to defeat player in the history of chess” • Kotov on the other hand was the chess rockstar. He hobnobbed with supermodels and nuclear physicists. •This was Superman fighting The Incredible Hulk in the Soviet Chess world •Petrosian surrendered in 13 moves Source: Wikimedia Commons
  • 3. Kotov Syndrome • A Player, confronted by 1,000’s of possible scenarios • The obvious is missed • The player makes decisions based on emotion and expediency, instead of facts and discipline
  • 4. Industrial Analysis requires Decision Frameworks not just Decision Making Tools Analytics must produce Insights, not reports, and the Insights must be Useful (Results are better than elegance) The Usefulness of the insights is a function of the integration of your analysts (not just your analysis) Three Keys
  • 5. Building Decision Frameworks (not just buy more tools) Organize To Use Specialists Start with the CFO Publish for the C-suite Source: Beleant, 2007 “Paleolithic Flints” used under Michigan Publishing Creative Commons http://hdl.handle.net/2027/spo.7523862.0005.006
  • 6. When to Pay Attention to Analytics The General Practitioner versus the Specialist • When the GP says you are sick, you trust the GP to run some tests • When the specialist says that you have Achilles tendonopathy, you go have surgery •You want your analysts to be specialists Source: Wikimedia Commons
  • 7. So Who are The Specialists The Threefold Path of Analysis Ask someone Experiment Build a model
  • 8. The CFO wants marketing to work She may not know why Or how it works • Transform the CMO into a portfolio manager • Analytics becomes the risk manager
  • 9. Publish for the C-suite Connect competitor action and macroeconomic analysis to your marketing Allow real scenario analysis, "what if" Get out of the PowerPoint trap and into white papers and long reads Become the CEO's weekend reading Source: Longreads.com
  • 10. Analytics Must Produce Insights (Focus on Results) • The Results should drive where you start, remain, and stop looking • Converting Invested Money into Products is the kernel of every useful insight in Marketing
  • 11. Insights must be useful • The more useful the insights, the more credible your analysts are • The more http://blog.uber.com/wp-content/uploads/2011/12/New-Logo-Vertical-Dark.jpghttp://blog.uber.com/wp-content/uploads/2011/12/New-Logo-Vertical-Dark.jpg
  • 12. Conclusion • Align your organization for using specialists, and set them up so that there is no punishment for telling the truth • You need to hire the following specialists: Survey-ers, Experimenters, and Modelers • Connect the stuff the CFO wants to what the CMO delivers and manage the risk • Publish for the C-Suite and use your credibility • When you are certain of something, do something
  • 15. Alignment (or no one grades their own paper) Keep the “Measure” separate from “Recommend” but using the same data Separate Research from Analytics
  • 16. Ask someone - • Survey • Focus Group • Social or Listening
  • 18. Separate the method of marketing from the method of measurement Certainty in a measurement is not the same as certainty in results Paid Search vs. Lift over control
  • 19. Models • Audiences • Marketing Mix • Funnel Analytics “Art is a lie that helps you understand the truth. Models are abstractions that help you understand reality.” – Pablo Picasso

Notas del editor

  1. I'm here today to talk to you about how to do something with the stuff that you know. This is a case study involving USAA and other companies, and I want you to understand what works and what does not.
  2. Kotov was the Russian Austin Powers. He wrote books, hung out with spies and bragged about the superiority of Soviet chess players
  3. You can have all the metrics in the world at your finger tips but you have to do something or stop something with what you know from your metrics The more data points, analysis, and insights. Analytics produces insights, some are useful, some are not. The more general the application of the insight, like knowing every opening and defense for chess. One large exception, though, You can string together detailed analysis from the bottom up to get very useful broad interpretation, Consider Nate Silver's approach versus the Washington Redskins approach.
  4. Imagine what happens to you when you are confronted with Market Scenarios, Competitor Scenarios, Macroeconomic scenarios, Media Buying deadlines, Emerging Media Options, Agency Meetings, 30 second spots versus two split 15
  5. Oh, but what are the right topics? That brings me to the point of my presentation. Seriously, I'm 6 slides in and just now telling you what I'm going to talk about. The thing that USAA did that worked so well was that our Marketing, Analytics, Research, CFO and IT organizations all organized themselves around the idea of using Specialists. Really, that's it. If you get nothing else from our discussion it is this, you can hire specialists, you can let them do research, you can look at their results, but if you are not organized that when a specialist produces a result, you use it or start to use it immediately, you will waste your time. So 6 slides later, now let's talk about how to organize your business to use specialists
  6. If your GP says that you stage 4 ulnar encepholopathy, you would rightly ask if he would like to run some tests or send you to a specialist. This is always true if you can't pronouce your disease or point to it on one of the creepy charts on the wall. You trust the GP to run tests and to recommend. You seek out a specialist for everything else. If you hire generalists, what you will get is the opportunity to do more research, and research begets research, in a never ending spiral, called analysis paralysis You want to hire, train, groom, promote, cajole, and whatnot your way to analysts who are specialists in the right topics
  7. So where do you need specialists? There are really only three ways to analyze anything
  8. In many organizations the CFO and CMO are at odds with each other. The CFO's goals showing tangible returns quickly. The CMO has to look at customers like a farmer, there is a time to plant and a time to sow. Sometimes the objective of a campaign is to increase awarness, sometimes the objective is increase sales. That may be interesting, but what the CFO really wants to know is where she can measure money going out compared to money coming in. So often the CMO and the CFO struggle to find a common measurement framework Enter Analytics, with structural equation models, multivariate experiments and other methods, now we can connect the dots between awareness and products. The first place that analytics has to produce results is in translating the work of the CMO into a common vernacular for the CFO. It allows for a new role for the CMO as the CFO's portfolio manager with Marketing Managed budgets. Analytics can tell the CMO and CFO that if you raise awareness 5% you should see a 3% increase in sales. More importantly, for me, it allows Analytics to be the "there are too many eggs in this basket role". Which sounds more sophisticated when you call it, Risk Manager.
  9. The real secret to getting your analytics used, is to make them useful. Building a good reputation in the C-Suite Inspires action Unsolicited, we started publishing a Weekly Marketing Estimate Update. It bascially takes all of our analysis, roles it together, and says this is how many product you are going to get next week, next month, until the end of the year, and what marketing efforts are driving each product. Second, we started publishing scenarioario analysis, working with our risk and economics folks and saying what would happen if we had a second government shutdown, and how long is "too long". This takes us from looking in the rearview mirror to looking through the windshield. Everybody knows Marketers steer better when looking through the windshield. Third, the Powerpoint trap is turning senior management into third graders. These are smart people. You didn't need to explain everyting in 3 bullet points when they were VP's and asked informed questions. Yes, they are busy, This is why I really like Longreads. It's a website that is focused on long form journalism with interactive visuals. You actually know something new after reading one of these stories. Think "The Atlantic" meets crowd sourceing. It's the anti-Twitter, and thats from someone who loves Twitter. You need to give the CEO something worth reading and learning, Something that they can skim and understand or take half hour to an hour and truly be up to speed.
  10. I was once on a team with two paid search analysts. During the course of their first meeting with me, I was their peer, they explained that they were terrified because they had no evidence their paid search campaigns were working. I looked at their method, we restructured their tests and determined, their paid search campaigns really did not work. The first executive that I showed this too took me aside and said, "Are you crazy, I'm going to have to fire these two analysts, and stop doing paid search. Come back with a more optimistic analysis of paid search." Optimistic Analytics do not exist. Optimistic results occur occasionally. There is a reason economics is called the dismal science. USAA created an independent and impartial Chief Data and Analytics office, where we keep Measure seperate from Recommend. Measure as in campaign results and recommend like proposed marketing mix, and tactics. If you recommended something that didn't work, someone else, holds you accountable. If you measure the wrong thing or the wrong way, Recommend holds you accountable The last point is a bit finer. Research, the traditional research with marketing folks in lab coats questioning customers on how they feel when they see 237 shades of orange or thematic implications of infintessimal pertubations of sparse matrices of on driver habits, really needs to be seperate from analytics. For the most part we are working with data generated around actual implemenation of plans, while traditional research is working with theoretical data.
  11. Specialists in survey need to understand More importantly they need to be able to produce testable hypotheses that the experimenters and models can prove out In return, the Survey teams need to be able to validate models and experiments
  12. Experiments were once the arcane practices of direct mail archmages coming out of their cubes between sessions of Dungeons & Dragons to hand the marketing team an audience list, A guy at MIT once told me Experiments take you from correlation to causation. The skill has advanced and now it's heralded by terms like interactive and geospatial. The focus is still on addressable markets, places where you know the name or address of the individual involved in the experiements, but tools like Google paid search with geotargeting allow you to hold out samples that are seperated by a few blocks, instead of entire markets It's not just testing one thing anymore, the multivariate approach allows for partial control groups, multiple use cases However, the experiments can be confirmed with surveys and models, so you can grade experiment effectiveness
  13. The point is that you can at any time, test your method of analysis, and you should. To the point that you trust your analytics to execute on them immediately. That's certainty in measurement. You need to be certain that your method of measurement is right, not that the results will be right. Sometimes the Government shuts down during your new product launch, sometimes, your ad goes viral and way more people than you every planed see it. Second, you do not need a paid search measurement analyst or an email analyst or a direct mail analyst. You need analysts who understand paid search and who understands design of experiments and web analytics. Their toolset has to reach beyond the marketing method. If your email analyst figures out email doesn't work, is it his fault that its not working? There is no reason to put the analysts in the position that they are rewarded for concealling or embellishing the truth. When your are certain in your method of measurement and aligned toward your method, Then you are ready to go to the CFO.
  14. This is the one that I do for a living, so of course this is the one that I include a quote on. Pablo Picasso was of course not speaking about living breathing people employed as models. In marketing you will build abstractions of reality to determine propensity to consume, the yield of a campaign, the rate at which people convert from prospects to consumer to lost customers The important part here is that if your model says Christina over here will buy the product you can a) call Christina and ask her if she is interested in the product or b) send Christina a message, through the mail, her Twitter feed, on a banner ad, and see if she or people like her buy at the rate you expect