Russian Call Girls In Rajiv Chowk Gurgaon ❤️8448577510 ⊹Best Escorts Service ...
Optimizely Experience Customer Story - Atlassian
1. A/B Testing
How to break things and fail fast…
without breaking things
MARK HALVORSON • HEAD OF INTERACTIVE • ATLASSIAN SOFTWARE • @halv0112
2. About Atlassian
We help great teams build better software, together.
• Products include: JIRA, Confluence, Bitbucket, HipChat…
• Founded in 2002
• Over 37,000 Customers
• Over 700 employees globally
• No sales people
!
!
!
3.
4. “ Think Big, Act Small, Fail Fast, Learn Rapidly.
”
"LEAN SOFTWARE DEVELOPMENT” BY MARY AND TOM POPPENDIECK (2003)
5. Atlassian A/B Testing is cross functional
Central “Growth Hacking” team to support:
• Email
• Product Management
• Customer Platform
• Customer Advocates
• Internal Systems
• Leads
• Product Marketing
• Support
• Bitbucket
• HipChat
6.
7.
8. Our Process
COLLECT GREAT IDEAS
TRIAGE BASED ON POTENTIAL IMPACT
BUILD OUT GOOD CANDIDATES
SCHEDULE TO AVOID CONFLICTS
RUN EXPERIMENTS
ANALYZE RESULTS
9. Our Process
COLLECT GREAT IDEAS
TRIAGE BASED ON POTENTIAL IMPACT
BUILD OUT GOOD CANDIDATES
SCHEDULE TO AVOID CONFLICTS
RUN EXPERIMENTS
ANALYZE RESULTS
15. Types of tests
• Pebbles
• Change in copy or CTA presentation
• Rocks
• New page (e.g. no Dev Tools family page)
• Boulders
• Simplified order form
20. Our Process
COLLECT GREAT IDEAS
TRIAGE BASED ON POTENTIAL IMPACT
BUILD OUT GOOD CANDIDATES
SCHEDULE TO AVOID CONFLICTS
RUN EXPERIMENTS
ANALYZE RESULTS
27. Our Process
COLLECT GREAT IDEAS
TRIAGE BASED ON POTENTIAL IMPACT
BUILD OUT GOOD CANDIDATES
SCHEDULE TO AVOID CONFLICTS
RUN EXPERIMENTS
ANALYZE RESULTS
43. Our Process
COLLECT GREAT IDEAS
TRIAGE BASED ON POTENTIAL IMPACT
BUILD OUT GOOD CANDIDATES
SCHEDULE TO AVOID CONFLICTS
RUN EXPERIMENTS
ANALYZE RESULTS
45. Our Process
COLLECT GREAT IDEAS
TRIAGE BASED ON POTENTIAL IMPACT
BUILD OUT GOOD CANDIDATES
SCHEDULE TO AVOID CONFLICTS
RUN EXPERIMENTS
ANALYZE RESULTS
51. Our Process
COLLECT GREAT IDEAS
TRIAGE BASED ON POTENTIAL IMPACT
BUILD OUT GOOD CANDIDATES
SCHEDULE TO AVOID CONFLICTS
RUN EXPERIMENTS
ANALYZE RESULTS
60. Our Test Philosophy
• Everything affecting conversion is tested
• Be aggressive
61. Our Test Philosophy
• Everything affecting conversion is tested
• Be aggressive
• Regular cadence, no traffic wasted
62. Our Test Philosophy
• Everything affecting conversion is tested
• Be aggressive
• Regular cadence, no traffic wasted
• Never run a test that you wouldn’t want to win
64. Key Take Aways
• Ideas are everywhere collect as many as you can, prioritize based on impact.
65. Key Take Aways
• Ideas are everywhere collect as many as you can, prioritize based on impact.
• Use ticket number everywhere so everyone is referring to the same test.
66. Key Take Aways
• Ideas are everywhere collect as many as you can, prioritize based on impact.
• Use ticket number everywhere so everyone is referring to the same test.
• Involve stakeholders and developers… and get approvals.
67. Key Take Aways
• Ideas are everywhere collect as many as you can, prioritize based on impact.
• Use ticket number everywhere so everyone is referring to the same test.
• Involve stakeholders and developers… and get approvals.
68.
69. Thank you!
MARK HALVORSON • HEAD OF INTERACTIVE • ATLASSIAN SOFTWARE • @halv0112