We began our experiments journey to speed up execution and learn more about the customer acquisition process quicker. We learned a lot along the way - here are those lessons. Read the full story on my blog: http://valchanova.me/growth-experiments-process/

1. growthGrowth Experiments:
Lessons from a failed attempt

2. about meVassilena
Valchanova
• 10 years of marketing experience
• Check out valchanova.me
• Digital Strategist @ Enhancv

3. growthAbout
Enhancv
• B2C SaaS startup
• Smart resume building platform
• 3 years old
• 300,000 users

4. our storyOur Growth Team Story
WHY?
Speed up and learn more
HOW?
Company-wide growth
experiment process

5. our storyThe Death of Experiments
• 10 experiment meetings
• 6 experiments carried out
• Participation from
different team members

6. our storyTime to Reset
RESET

7. The Growth
ProcessPROCESS
GROWTH
Analyze
1.
Set
Objective
2.
Ideate
3.
Prioritize
4.
Test
5.

8. growthBuild
YOUR TEAM

9. setupBuild Your Team
GROWTH LEAD
• Sets the process
• Leads the meetings
• Clears obstacles
MARKETER
• Acquisition
• Copy
• Driving traffic
PRODUCT/DEV
• Technical
implementation
• Automation
DESIGNER
• Mockups
• Wireframes
• Marketing visuals
DATA ANALYST
• Initial data
• Experiment setup
• Analysis
CUSTOMER SUCCESS
• Customer feedback
• Critical questions
• Early results

10. our storyAlways Use The Same Team
• Ownership
• Helps them prepare
• Teams require time to
form

11. growthEstablish
CLEAR RULES

12. setupClear Rules
• Is the experiment clear and
specific?
• Is the experiment related to the
objective?
• Can it be executed and analyzed
in 2 weeks?

13. setupDictatorship is good in meetings

14. growthStart with a
SPECIFIC PROBLEM

15. analyzeSpecific
Problem
• Work with data
• Growth lead and management
decide
• Concrete objective

16. analyzeWhat’s Concrete?
Our current blog new visitors is 4,000 per month.
We need to increase it by 15% without losing the
quality of the audience.
Let’s focus on acquiring new users

17. growthRespect everyone’s
TIME

18. ideateICE scores are
cool
1-10 average of three scores:
IMPACT
If successful, how big is the difference?
CONFIDENCE
How sure are we it will work?
EASE
How easy is it to implement?
(time/people/money)

19. ideateRespect everyone’s time
• Team members write down
ideas before the meeting.
• Each scores their idea.
• Only doubtful scores are
discussed.
• Scores are not the last word.

20. growthEnsure there is
TEST CADENCE

21. testTest Cadence
• Two week experiment
cycles
• 5 experiments per cycle
• 30% success rate
• 32 successful experiments
per year

22. growthSetup experiments
THE RIGHT WAY

23. testExperiment Types
A/B Tests
Testing 2 versions
simultaneously,
only 1 difference per
version.
Multivariate Tests
Testing many
versions at the same
time, lots of differen-
ces per version.
Sequential Tests
Testing many
versions one after the
other.
Smoke Tests
Landing page
collecting leads to
gauge interest.
Trigger Tests
Measuring click rate
of ads to see interest
in a value
proposition.
Wizard of Oz
Deliver value by
hand, if a feature
requires lots of dev.
High Hurdle
People pay with time
(surveys, long forms)
to see how engaged
they are with a
proposition.
Design your Own
There are other
options…

24. testGet to Know
Statistics
• VWO’s Significance Calculator
• Optimizely’s Sample Size Calculator
• Google Optimize

25. growthSettle on a working
TOOLSET

26. testTools to Try

27. growthThank you! Any Questions?
The slides: valchanova.me/slides