Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Yntlassian i vi‘

A/ B Testing

How to break things and fail fast. ..
without breaking things

_ I’

HANK MAI‘! -"JR. 'fiN ...
How to break things and fail fast. ..
without breaking things

«Ln-

UAIIK NAUlOH$0N READ OF INTERACTIVE ATLA-SEIAN SOVTWA...
YAthSSi3fl Pi-. <ii-d.1 i" urn [til 7 T3‘;  Elm, ‘

 

n. .n. u Inna:  on-.1

-. - ---— . ..—q -. 
o

Atlassian A/ B Testin...
Atlassian A/ B Testing is cross functional

"if‘

- Ema. ’

- Product Management
- Customer Platform

- Customer Advocates...
Our Process

(cull!  I (. .lIAv it7I. 

l| ll»(. l Infill) Oh V(ll| Nll~l | V'»(_l

llllll T) Ulll (_. ()(ll) (, .Vl)Ir)A1l...
Our Process

I, ()Il| (,lI, llA1|I)l-

Hllr(. l lI'IlU ON VIIIINII-‘KI lV? A( I
IIIIII II IIIII I, ()(‘Il) LANDIDAII 5
SCH...
IIIIACI.  IASLO ON POIINTIAL IMPACT

BUILD OUT GOOD CANDIOATIJ

SCHEDULE TO AVOID CONFLICTS

RUN EXPERIMENTS

AN [VII RI. ...
AVALVII IIlSlJLI$

Where do
experiment ideas
come from? 

Anywhere. 

What are you are
trying to improve?
What are you are
trying to improve? 

 

+ . 
Blimp Dashboard '
+
4 _ ; ... . . ... _
lg.  _. ---. _. . ._. .._ . 
s-- Vhn...
Blimp Dashboard .  ’

 

+
e
. _. it-c-own -uo—- 0 can urn Oun-
i T
:3 3". " xxxk.  _V'''''‘"'’ .  xx. xk.  :: :"'
: _ -" ...
JIRA - Ticket *

‘ ---'-u -ru

unnnynutuoarn

Ion, -——--on-u --¢———~. —-wu——r non. -

. - . u- _.  —. .——_
A. .. . —. ~—. ...
- Rocks

- New page (9.9. no Dev Tools tamiy page)
- Boulders

- Simplified 0169: form

Examples

Reduce Friction ”

'| 
0...
Reduce Friction

A B
i _ ‘N , | ? «.“. ..'. 
6
74-!  I _
Increase Visibility
A B
_ _ - - 2 _ _ _ . .
Simplify
A B
._ . .._.  3.. . . . . ... ...  9.. .

. -

Simplify

Our Process

LUlll(_l(, llA1|[)l»3

l| ll»(. .l IIRSIU UN P(HlNll~l ...
T

0
1-: 

Our Process

Lnlllkl hllfll | Dl»3

Hll: (. l nnsln 0N PHl| Nl| ~|  IV? -(_l

IIIJHD l')l_| l LOUD (_rN()| l)AH5...
What are you are
trying to improve? 

  

+ . 
Blimp Dashboard ‘
~ smrs xxxk _ ‘v/ slim:  lr, - XXAXK
_ x. xx°/ o .  
E‘-a...
on no--—o no-no-cu 9-— as-—u o-uvn

   

. -. .  ._.   . ._. .  . _.. 
- - -.  - . . . .-.  ..  ..  .. ._.  .. -_. 
.__. -...
JIRA - Ticket

3 : - r:  ::  :- 
Experiment Illuminati

*1
«.9
l)

H
JIRA - Ticket

Experiment Illuminati

Our Process

[OI lI(_I (_Al' AV II)!  M, ‘
Iillr(, l Infill‘;  ON VHIINII-‘I lIrA(, l...
Our Process

l()lll( I (, llA1 II)! -
I| IIA(. l IHISIU ()N VIIIINIIAI ll! ‘A(, I
SCHEDULE TO AVOID CONFLICTS
RUN EXPERIME...
AVAIVII III Slll '8

 

Optimizely — Build

(WAY-92) Iloalwo cu for confluence product page -
Q
Xllflfiiflfl —. ... . 1

Whnrri...
v fit’ —  v
r .  . . 
. T

- o 4

 

'iw»Tu2‘)l:  I IC'. li'_ L
[ ‘l; |.L_. _..  H _ V

Xtonfluence
.1‘! -E.  r I ‘ . '.‘IV...
Confluence - Corresponding Test Page

. ..- u‘; ‘u

 

 
 

 

-o. a IE

 

‘W Y Y_V I 7. mi’? 
. . .._. .. . _. . .. 1 .1 ...
Confluence - Corresponding Test Page

'“v? iT V V V V‘? 
. I .  . , 
- a-—-o-no .1 ran . _.. _.: 

 

 
 

V Ian I, - C  . ...
A-rl Tjlfiti cc; 
acumen-nc'bo—

-nee-n¢»ene-r-eruieee1-ee--e an--e-we-cg-I-~oo—n-—-«-9-eeu-nus run-en—
u-——e—-e-——

» n na...
..  _—. . ———- . .-_. -

—. 
= . m. -. . gut
XI- ‘Z 3-05. COG.  3--I
?  not on son

9' —e-3 cu. -.c.  -. ... — cu. -.. -
o...
ll‘l%'l ll
liilllill
ill 5'
II I
II

era as - e. 
u q. .. o—--.  9-0:; 
—-A -as ‘as
r.  em. .. up. -no -cu. ..
‘m.  g. ‘ ....
iii“ 'v'tv ;  STE’

Confluence - Corresponding Test Page

—-—— e

mitts:  C71:

Confluence - Corresponding Test Page

Conflue...
4.

Confluence - Corresponding Test Page

Confluence - Corresponding Test Page

Confluence - Corresponding Test Page

Confluen...
Confluence - Corresponding Test Page

 

V Il“| _',   ‘_T’* ir-¢; _p§¢‘

. » v. — en». .—. . 1
‘4‘| _1lIV(‘: §l -I-Irma I-l...
{

Confluence - Corresponding Test Page

an-. .. . —-4 . .v- . —

-e. .. E. ..

 

'  v v v v x, 

r . 

rv n—. —.—.  ‘no. ...
Our Process

[HI I I (_I (_. II AT II)!  A)

IIIIACI llfifill) ON VUIINII-I lV'A(, l

Illlll I) rllll (, ()()I) (, vNI)I[)AI...
Our Process

[()l| |(, l(, llA1IK)l. §

lfllr(; l l| '_IU ON PHIINII-‘ll | V'»(l

lllllll) nlll (, ()(‘ll) (, vNK>Il)All 5
...
VTKVFTI ll TVII‘ F‘

Optimizely — Start Experiment

[3 jj‘ l 

(WAT '32)l«; -I j-:5: 'L_' L-;  i‘ ,  w; ~ L"; ': .J. : v, ...
o. —— 0
. .._.  -. ._

 

’' Magnolia CMS — Include Optimizely
Checkbox

,  +
‘ boo humus» I
‘| ~U- out-uvoauuno-nun --q-—...
..  .. .. . _. . .._

n. —.. .. o. ..- J
. ...  .. .

a. -.. . >. .. I. .-
y. .. -. -< x. ..

5.. .. s. -..  Inns. 
-«. . ...
Confluence - Corresponding Test Page

 

A/ AV la;  nub. .. <1." I on:  Ina R-mu 1.. 

-1 Ipotwva

 

Our Process

L()| ||...
IIIIACI IIASID ON l‘DllNTl»l IV5‘A(. l

| |IJ| lD DUI L000 LANOIDAYI 5

SCHEDULE TO AVOID CONFLICTS

5’. UN EXPERIMENTS

‘...
I: S_«. ... ..n. -.. ..—- -— —
—. .-. ._. - . ... ... ... _.. ... .

What are you are
trying to improve? 

Blimp Dashboard...
_ T‘ xxxk '3 T" xx. xk. 
— x. xx%.  
Ev " .  M. ‘-
x. xx. -< x. xxk. 
xx. xx35. - xx. xx%. ... 
Optimizely — Results
. ......
Confluence - Corresponding Test Page

T Confluence - Results Archive

Our Test Philosophy

- Everyth"1g afiectnng COITVSTSI-...
+ .  +
Confluence - Results Archive
+-
. . . . nun I-av j : - : ~a o-an-u--o-co -s- —---. o.. ~~. »-. ... n
Q1?‘ :1 -— nu ...
- Regular cadence‘ ll-Ll trulfi-:  was’. ed

- Never run a test that you wouldn‘t want to win

Key takeaways: 

| «‘_iea'_-...
+

+

VAII K IIALIIOI 130M

+

. ’."Atlassian 

HEAD OI INTERACTIVE

 

AVLA:5lAN SOD IWAJIL

-In -ill:
+ ‘ ’ Ymtlassiansummlr *

Thank you! 
Q

6
HAIR HALVORBDII HEAD OI INTERACTIVE - ATLABSIAN SOITIARE ~ Oholvolli ‘ +
+

‘I’...
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson
Próxima SlideShare
Cargando en…5
×

A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson

2.994 visualizaciones

Publicado el

Everyone wants to optimize their web property, especially if that web property is their only sales channel. In this talk, we will explore how Atlassian has used their CMS along with JIRA, Confluence, and other tools to operationalize A/B testing to continually optimize Atlassian's website. Using this process we can balance failing fast with keeping things running smoothly.

Publicado en: Software
  • Inicia sesión para ver los comentarios

A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson

  1. 1. Yntlassian i vi‘ A/ B Testing How to break things and fail fast. .. without breaking things _ I’ HANK MAI‘! -"JR. 'fiN MFAB OF INYFRQCYIVF ATI AS5|AN }’ufiVT'fl'A5IF G79.‘ -C115‘ About Atlassian . Yvv>“"‘. ‘VlV'i‘ -1-~. .r -~. ‘_ . l‘A“f li . . l i. .. l‘ . ..l in - Products include: JIRA. Confluence. Bitbucket. Hipchat. .. - Founded in 2002 - Ovet 37.000 Customers - Ovet 700 employees globally - No sales people , _| _ + + Atlassian
  2. 2. How to break things and fail fast. .. without breaking things «Ln- UAIIK NAUlOH$0N READ OF INTERACTIVE ATLA-SEIAN SOVTWANE GPIJ4-0112 About Atlassian ‘ Products include: JIRA. Confluence. Bitbucket. Hipchat. .. ‘ Founded in 2002 ‘ Over 37.000 Customers ‘ Over 700 employees globally ‘ No sales people Xlttlassian 4 _ . | |:__. _ n| . _ _o , --I. - - 5 , __o -, __| I- -2.
  3. 3. YAthSSi3fl Pi-. <ii-d.1 i" urn [til 7 T3‘; Elm, ‘ n. .n. u Inna: on-.1 -. - ---— . ..—q -. o Atlassian A/ B Testing is cross functional - Email - Leads - Product Management - Product Marketing - Customer Platform - Support - Customer Advocates - Bitbucket - Internal Systems - HipChat ‘ + + +
  4. 4. Atlassian A/ B Testing is cross functional "if‘ - Ema. ’ - Product Management - Customer Platform - Customer Advocates - Internet‘ Systems - Lea-:15 - Prc-duct Marketing - Support - Bitbucket - H pChat
  5. 5. Our Process (cull! I (. .lIAv it7I. l| ll»(. l Infill) Oh V(ll| Nll~l | V'»(_l llllll T) Ulll (_. ()(ll) (, .Vl)Ir)A1l N SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS
  6. 6. Our Process I, ()Il| (,lI, llA1|I)l- Hllr(. l lI'IlU ON VIIIINII-‘KI lV? A( I IIIIII II IIIII I, ()(‘Il) LANDIDAII 5 SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS -AlVll III . III ' Our Process I| l|r(. | IUISIK) (IN PIIIINIIAI lV'r(, l IIIIII I) THII LOUD (_rN()| K)AI| 5 SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS ANAIYII III , III I5
  7. 7. IIIIACI. IASLO ON POIINTIAL IMPACT BUILD OUT GOOD CANDIOATIJ SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS AN [VII RI. SUI. lS Our Process IIIIACI. IASU) ON POTINTIAL IMPACT BUILD OUT 6000 CANOIDAIIJ SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS AN l. Yll IIIJULTS
  8. 8. AVALVII IIlSlJLI$ Where do experiment ideas come from? Anywhere. What are you are trying to improve?
  9. 9. What are you are trying to improve? + . Blimp Dashboard ' + 4 _ ; ... . . ... _ lg. _. ---. _. . ._. .._ . s-- Vhnn xxxk‘ Vhmdfry ' __"_f'_" em-an X “R Luau ‘Ch. I xx xx%. .. ‘I xx. xx%. ... + JIRA - Ticket ’ 4- nnunuovau -9- 4 rncuu. ,mor-cum on. i-—---—. .--—-. —-u--u-¢—. —u— accu-
  10. 10. Blimp Dashboard . ’ + e . _. it-c-own -uo—- 0 can urn Oun- i T :3 3". " xxxk. _V'''''‘"'’ . xx. xk. :: :"' : _ -" -5-an -no-«cu 32- I ' x. xx%. ... ___'_ -_ _ -In-mu lib -_-; _-_-_ »E“"' xxxk. “"' x. xxlt. . I -1512 C151} " i xx. xx%. .. xx. xx%. .. 1 215 9% —. Gil-III 5 ‘I’ ' * JIFIA-Ticket * + O I 13'? Zn l ‘| ’d&PUD‘| ’fl1I In Din 103- bk-tumult: --u I92- : one Inn E ‘@21- 1 01 u—— j u— a-ocu —u r---—- be Rs “' "' ——n—'-1 Qua? --. - huunceunnovu V9--eououe--n-naunuu ecu. uuqennnu su- Ina; -~¢encn—v~a---—qu-o. u—n--: -o-o-op-ecu son-a-—¢a@ onus-we-2 more-uno~—-a——-an--1 -9-n--uc—v-nu-an-—n -4-noungwnnnnvunp-u Q-—-ecjcnuceovuuc-Q won-canto-U ct--our -ulu uusut 1 j'§1UD&jI1Q§i. QiQ&QZjj wit Inc-that-nub-3 -Q10!-I-j unfit: -uvtuunonuei can-xvoo-nu-Q: -s 3-aww~—1 jecnalau H‘ {I ll’ l“l I‘ : u— ‘ inc I’-01¢: Types of tests ~ Pebbles -Changelncopyorcflpreeentatlon ~ Rocks ~ New page (e. g. no Dev Tools tamiy page) - Boulders - Sinplltled order form I l
  11. 11. JIRA - Ticket * ‘ ---'-u -ru unnnynutuoarn Ion, -——--on-u --¢———~. —-wu——r non. - . - . u- _. —. .——_ A. .. . —. ~—. — . ~—— -. -.—. . —. .. —. . - -. - 1 ~_. .o. ... ... . —. — ~—- fin-~— 4 ——: ._i. .-. %¢o~ . -—u— —- . ~ . -— M u—----. —q. ~—vo-—u——--o-. —-—. .—-4-. -n--g-‘co-—-su- iv. —--s-—-nu-no-zq-. -u-u-. -u--—¢-~—. ._ -one-:1--¢ : nn¢~auv«- -A-no-uc—u4-no-an. -— «*4--—--un-ca-an-u nu-. uuu. —u. -—--~. —uq¢~--. u.n—v. a—u. -nu v»---nu c-usounomuuu -. --¢ unsoun- : j~-. —u»-—-uuu-nuu---agu--oau-—a-———-—-. —«cu. ,__. _ -q--—. -.-»—-—n—. .¢ -4-nu--. --n -nay. -» in--—. nuuou-an. .. --u--4--—-n_—n~. — -up none» I now. - Types of tests - Pebbles « Change in copy or CTA presentation - Rocks « New page (eg. no Dev Tools Iamily page) - Boulders « Sinplified order form Examples * Reduce Friction *
  12. 12. - Rocks - New page (9.9. no Dev Tools tamiy page) - Boulders - Simplified 0169: form Examples Reduce Friction ” '| 0 I W 2;? ’ «- """‘_‘. _'. . . .._ "‘“ gg * 3 j '1'" F :0 "= »-' ; ;-. -. -‘ ' 3; I’ O C v— Own: v— Own. -. ._. - . .._ . . . . -. ._. _. ... . -. ..
  13. 13. Reduce Friction A B i _ ‘N , | ? «.“. ..'. 6 74-! I _ Increase Visibility A B _ _ - - 2 _ _ _ . . Simplify A B
  14. 14. ._ . .._. 3.. . . . . ... ... 9.. . . - Simplify Our Process LUlll(_l(, llA1|[)l»3 l| ll»(. .l IIRSIU UN P(HlNll~l ll'r(, l IIIIIIU UIII (, ()(‘ll) (_rND| [)A| l _ SCHKOULE TO AVOID CONFLICTS RUN EXPERIMENTS -VAlVIl Ill Slll '
  15. 15. T 0 1-: Our Process Lnlllkl hllfll | Dl»3 Hll: (. l nnsln 0N PHl| Nl| ~| IV? -(_l IIIJHD l')l_| l LOUD (_rN()| l)AH5 SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS »NAlVIl | ll_| ,ILlS Blimp Dashboard '. "»sn; rx V sand Ir, ‘ xxxk xx. xk ' x. xx°/ o. . ._ E'mw. .n: r3 Le-ac: . . _. ‘n. .. .. ..I.
  16. 16. What are you are trying to improve? + . Blimp Dashboard ‘ ~ smrs xxxk _ ‘v/ slim: lr, - XXAXK _ x. xx°/ o . E‘-aatrs ‘ xxk " Leacs X xxk xx xx°. /o - ‘ xx. xx°/3.. .. 'f' + + . . . JIRA — Prnontnzed Backlog ’ C I . I e I . O 0 , ,. . 7. ‘ V . I + non _ 'r; n|, n+ +
  17. 17. on no--—o no-no-cu 9-— as-—u o-uvn . -. . ._. . ._. . . _.. - - -. - . . . .-. .. .. .. ._. .. -_. .__. - . _:. _ . . . _. . . .._. .- . .. z_ . ’____Q C jji . _. _. .. .. 1.1‘ . ... . . ———-‘—-. .._ . —-. -n-up . --my-i . _-= ‘—-nun-: —u. .—u. :—- . —:a_-_-- "" "' "-‘. _.. °"' "- "' """'. ... ... ... . I-.4 . a-1 . -4. I—4 I I-an . _u am} Q1‘ 'EQj a1CIZ? j O W I I—1fl ow D . ... ... .__ :33.. - . .._. . If 01 1 Ijj . - - . _ . ‘ . . . -s: u-: .-u -—-q—- . no—m>--- . ~&-nag-n . .. .. ... ... - . - - . .— .4 . u . . I-4 . .. ... ... . . ... - . . . .._-. . . ... ... ... ... .. _.. ..- . ... - . . . =.. ._. ... . TNT é . - . ._ _ . .. . ._ _ . -._-. - . .. ._. .. . .._. .- . ... . _. ... .. ... . . . . _ . - . - . - . .._. -. . ... ... ... ... . . .._. ... ... ... . T1 jbhccccfi ‘wlhéfi £13-. + JIRA - Ticket * I . ... ._. .. -- Yn§yFrIo1’IUOaIri 4-i. <.——-uu-. —-~. _-—-.9.-o. —.--. ~ 99¢ B 1 ‘so am. an —. .. . ..i . .._ ‘W -—~ can u_. — ‘.1 -—. - - . ._. —— —. -. .—. — _, n, __. _ ‘1. é J jfléqhu u. —.. g. .. »t-«—- : -— ~—--—u-‘-v-tuu—n~¢c——a—. --. -u-n---uq~c—-b--— _ I--——-cu--—--cc--—n--—uu——-—-nu-¢-~—-nu OQO S ' Z - -. ._. .- “. '_‘: _'f' . -. - -—-—. .. . .._. .. , . . .._. _____ ~ao-. ~-—c—- an-raga: <-son. -—u~-on-nu— --—--—uou—-a--uoe-v--n-ounn nu--nun. -our-or —-no-no--r -unur announ- f c—-—-v~—¢--u--nu. ---u—o-cc. -pa-av-inn-can. -—--1-luau __ , __, -nou-~. -u-no-ca. --o c--u----. -n--aq-up--—--nus-unqn : —.—c-o—un-—-- I. -In u-an o -us-no I -—. .r-. .. + JIRA - Ticket * -- . .. . — _. _ . —__ _. .._
  18. 18. JIRA - Ticket 3 : - r: :: :- Experiment Illuminati *1 «.9 l) H
  19. 19. JIRA - Ticket Experiment Illuminati Our Process [OI lI(_I (_Al' AV II)! M, ‘ Iillr(, l Infill‘; ON VHIINII-‘I lIrA(, l SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS ANAITII I” , III T Optimizely - Build
  20. 20. Our Process l()lll( I (, llA1 II)! - I| IIA(. l IHISIU ()N VIIIINIIAI ll! ‘A(, I SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS AVAITII | II , || l I3 Optimizely - Build (WAT«92J float-no (H for confluence produn page - _j: _ '3 XCUIIQQ , —.. ... T when wcxk becomes inumvrarl: T Optimizely - Build (T4:AT‘9Z) float-no (H for confluence product page ' _. ___; _ 7 - O Xtcrifliimcc
  21. 21. AVAIVII III Slll '8 Optimizely — Build (WAY-92) Iloalwo cu for confluence product page - Q Xllflfiiflfl —. ... . 1 Whnrri wcxk be-coma in: -nvrark - Optimizely - Build (lllrAT~92) Iloalna cu for confluence product poo: - Xcflflflfl —. ... . 3 Wham wcxk becomes fumvrari
  22. 22. v fit’ — v r . . . . T - o 4 'iw»Tu2‘)l: I IC'. li'_ L [ ‘l; |.L_. _.. H _ V Xtonfluence .1‘! -E. r I ‘ . '.‘IV ‘I T ' 1 ' -‘ I, l -A I - . . L Confluence - Corresponding Test Page ? '—m—e _ V —_—. ._. ueen hi :2» n . _ o I T? Confluence — Corresponding Test Page vwerr
  23. 23. Confluence - Corresponding Test Page . ..- u‘; ‘u -o. a IE ‘W Y Y_V I 7. mi’? . . .._. .. . _. . .. 1 .1 _ . ..__. war‘ C , i r 'vvV'C . ._. . . . -.4 -: - . .._. .. ._ -. ... ... ... ... .. , .. %r<— _ . . -
  24. 24. Confluence - Corresponding Test Page '“v? iT V V V V‘? . I . . , - a-—-o-no .1 ran . _.. _.: V Ian I, - C . .. . ‘v . ,-r. .- h-. :4, Viv: U"-I ‘ Confluence - Corresponding Test Page "_“r""'* . .-. ... .I. ..~ L. 2.. ;_ _ . ~.. '.. . . — -- wnv w, - r . vv . ,r-. ..r. . :4, »i. <---us §. ~_-— .3. Confluence - Corresponding Test Page cow
  25. 25. A-rl Tjlfiti cc; acumen-nc'bo— -nee-n¢»ene-r-eruieee1-ee--e an--e-we-cg-I-~oo—n-—-«-9-eeu-nus run-en— u-——e—-e-—— » n nae-no-e-cu roe-—u--p-o-um. --u—o—ieq’v-yaw-' . .. -an--—u—-you-new--u--—: .u-qt-id - I was-you--can-nnecueu-—ocoe‘e: v—ir --as-enuuuoou . ga- —— 3 U , _ . »«-. —.—. ... ..». . . wo. .-. ..- . —-. .-——. ..--. .~—. .= ._——- ——- . .— . -.. - . ---. ~ . -._‘. T I‘ Q. 9' an‘ -; Qj 1C e— -a — . .~ to ii. C ¢<Q¢ 9- G. 0-C- '-—' . .s cc. «. .~ g. .. . . - an. - A. .. .. .. cn—o j “ " "‘ ‘ " -I‘ — $-QQQ C-‘fin I-RI ac-e hue sue . - W9 9 beam . .4.. ._ -. ... .. -nu. .. ll eve‘ III I I T ej Onvut 191$ ebjé T Q Q‘ 1»? an‘ O QT. -jré-omji - DC 51 up‘ 33 15. C O<G. o~ Q. ‘>13 ‘—"' ‘gs . ..~ -. - -u —. - . ... - -. .. .-. -:1 T ‘ " "‘ "" (O) I Z-0&3 34-. I4!) oest- sun 5: 5-: 'I% 0 bet. —. a-. — -. ... .. -nun. ‘ ‘I “III I i
  26. 26. .. _—. . ———- . .-_. - —. = . m. -. . gut XI- ‘Z 3-05. COG. 3--I ? not on son 9' —e-3 cu. -.c. -. ... — cu. -.. - oj ' 1 T T U§vOO§ Ij®%% Dbié -—— e. .. . .ri .9 . - “ Q O IC 1 2 1 -9- C CQQQ C-‘Q. D-CI '3 ‘on ‘o. .os 10 it Q can 3 -no In ‘Q. = g». e. .. . .- flj ‘— ICQQ QCQO Z-oh- -C "‘ “ "‘ 0' be: um. .. co. ..’ -. -.. a~ _ ' 1 . - . C C T1 Q f It i 7 I an. - --. o—-Q- .4. '-' ‘on go. —-A it It Q can 3-30 to 1-35 :1 ‘mu -. < has -n— - noun. -4-. .. .n- °‘: gun -— . - 0 nu: gun. .. -. ... a- ¢eae~ua§ O i confluence - Corresponding Test Page
  27. 27. ll‘l%'l ll liilllill ill 5' II I II era as - e. u q. .. o—--. 9-0:; —-A -as ‘as r. em. .. up. -no -cu. .. ‘m. g. ‘ . .- jI- ‘C 3-CEO $16. 1055 3 -- -- -- 0 $01 cannon up-~u¢ gun. .- oer: T Confluence - Corresponding Test Page Confluence - Conesponding Test Page . N _. ... ... ... .., ... .— I
  28. 28. iii“ 'v'tv ; STE’ Confluence - Corresponding Test Page —-—— e mitts: C71: Confluence - Corresponding Test Page Confluence - Corresponding Test Page
  29. 29. 4. Confluence - Corresponding Test Page Confluence - Corresponding Test Page Confluence - Corresponding Test Page Confluence - Corresponding Test Page
  30. 30. Confluence - Corresponding Test Page V Il“| _', ‘_T’* ir-¢; _p§¢‘ . » v. — en». .—. . 1 ‘4‘| _1lIV(‘: §l -I-Irma I-l)l) 1709.17 "! ‘57'-‘ill’ ‘:3-air m 7"i’1.-u
  31. 31. { Confluence - Corresponding Test Page an-. .. . —-4 . .v- . — -e. .. E. .. ' v v v v x, r . rv n—. —.—. ‘no. _ ¢- -_ . __ . , Al I -? o . n— I‘/ pl] l*"' ll-I3-I-fix‘ §*°'. :', F’5'* ‘1'C"' ‘W9 — . . . .~ ~ .4 . ~ . . mind rub -i uni-xi-ri i-um I V. V e. l . .. e :3 SI mix 4‘ PINK! !! SCHEDULE TO AVOID CONFLICTS l M . ?.£J'3J''', ..‘, :‘i'‘. ‘ 9!! ~. = n 'I""]4 NHII-is JIRA — Prioritized Backlog
  32. 32. Our Process [HI I I (_I (_. II AT II)! A) IIIIACI llfifill) ON VUIINII-I lV'A(, l Illlll I) rllll (, ()()I) (, vNI)I[)AIl SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS ANAIVII III Slll l JIRA - Prioritized Backlog I1 (1. .4 I1. '7 [I .4 Our Process l()lIl(l(illA1II)lr I| lI»(. | IIASIK) ON VHIINIIAI | V'»(_I Illlll I) UIII (_. ()(lI) (, .NI)IK)AIl 5 SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS AVAIVII III , III I5 Optimizely - Start Experiment IW‘T'91)"OIl-fifl (ll '01 CDHHIJOVOCI DIOGMEE DIOI ‘ :4:
  33. 33. Our Process [()l| |(, l(, llA1IK)l. § lfllr(; l l| '_IU ON PHIINII-‘ll | V'»(l lllllll) nlll (, ()(‘ll) (, vNK>Il)All 5 SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS -NAlVIl III , III '3 Optimizely - Start Experiment (WAY-92) float-no cu Iov confluence moducl page - G Xtnvntluaultn ». ... . T Wham woxk booomu tumvrark Optimizely - Project Code
  34. 34. VTKVFTI ll TVII‘ F‘ Optimizely — Start Experiment [3 jj‘ l (WAT '32)l«; -I j-:5: 'L_' L-; i‘ , w; ~ L"; ': .J. : v, ‘l; |4L, .,. '» H Xtonfluence : :"': ’I-: I W 7 7 7 x , f, -‘ , . u 1 . _ Optimizely — Project Code Magnolia CMS — Include Optimizely Checkbox
  35. 35. o. —— 0 . .._. -. ._ ’' Magnolia CMS — Include Optimizely Checkbox , + ‘ boo humus» I ‘| ~U- out-uvoauuno-nun --q-— AA. -M ~-v- ~ U-0 v--u am ----. -.. . mu" ~_Ej “WM -. r._. ... --. .-. .- . ... ... ... M. h--no. -my-— g . .-_. Anal-baby: - , ... .. -. ._--. ... -. -_. .. h—n~¢ ‘ _. - . .._. .. . ._. .- I——-u| -n—bu-- A-n—&Iann-N: .. ..——~“ . -r + Magnolia CMS - Include Optimizely . Checkbox + ' coo Iuunnnnnn I [T jtunfllooué -wr-~ o4«—~- --o-— - Adv u---- -~o- v--~-'-- ““§"' ~ hnulh2§— . -‘N--. “-‘-”‘. ‘_- _ ____-. ... -—-—~o—— 4 1-<46-fit ‘I; - W. -. ._. .. ---_. Q. -—-0-. 4 h-—oI-—ui —. “‘- . .._- . .._- . .._- “‘_"' . -r * Confluence - Corresponding Test Page + 1 ' «v. «o —v-w-q». - . I -4-. mvnounanuuyrmvunnnu ‘ V . I». o. .- 1-. .. Q
  36. 36. .. .. .. . _. . .._ n. —.. .. o. ..- J . ... .. . a. -.. . >. .. I. .- y. .. -. -< x. .. 5.. .. s. -.. Inns. -«. . . .._ . . hit 1.. -or. .. . ... ».. ... . -.. ... -.. . . ... --. .._. + Magnolia CMS - Include Optimizely Checkbox + ' .00 I-than I ‘ti ¢Q-owns-no-ac '~— .4.-—-- '-1" - '--~- --‘-~ mp , «.. -«-- 4--<‘- 1 —¢ -. .. I. .. . ..r u". g. . wAVw0eIAa-oolMn'h¢oYvUIi: Ou Onmva 'I $0 *——h. vm'-cha- QM %j loco-o c-hgtvn ‘OA- . .._. 1 ‘ E. X1 X1 -gm-u---. ~—. >.. —.-y. -cw-. ... —--u-. —-. ... _-. _.. _.-. ».—. u—~. ... <._. -~. _.. . van-. .._ -. ._-an-. .._ . . ua. a—a. _.—-. --o nc. -A. -.——_. ... -.. ... .-~. ~.—-—. * an- _-. —.. ... ... -.-—‘o. ».. .—. ———. v. -—‘w . . . ... ... ..—n. —.-.4--up. -———o. ~.-u-. -—. ngac --an‘. -—. -.q. .. —-nun--U Our Process (Dill! I LIIAY | l‘Il> Hll»(, l llflisll) Oh Vl)l| Nll~l lVPr(_l lllllll) nlll (_4()(IlJ (, :il)Il)A| l SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS
  37. 37. Confluence - Corresponding Test Page A/ AV la; nub. .. <1." I on: Ina R-mu 1.. -1 Ipotwva Our Process L()| ||(, I LIIAI I[)| .) IIIIACI IIQSIU UN VHIINIIAI IVVALI lllllll) rllll I, Ol‘ll) (, All>| l)All 5 SCHEDULE TO AVOID CONFLICTS RUN EXPERIMENTS ANAIYII Ill Slll I3 Optimizely - Results
  38. 38. IIIIACI IIASID ON l‘DllNTl»l IV5‘A(. l | |IJ| lD DUI L000 LANOIDAYI 5 SCHEDULE TO AVOID CONFLICTS 5’. UN EXPERIMENTS ‘EE. +PEE I ""‘§. "i1;- What are you are trying to improve?
  39. 39. I: S_«. ... ..n. -.. ..—- -— — —. .-. ._. - . ... ... ... _.. ... . What are you are trying to improve? Blimp Dashboard . ... - c. .. Q . . . ._ ': 'f"""". -__ ___ - II: .o-— yum “xx vhnnafry xx xk ______ :1“ -‘ --o M-. ‘ n--S--Non I. .- : - - ' x. xx% . .. '1'; v u -uA vwu :2 -j: _ E_"""' x. xxk. _; “"" P x. xxk. _. __ ' -. -A-nu. . -. -A-nos. "' - xx-xx% . . ' xx. xx% . . h —-to '1 t I I--d0-CAIN L». .. -. r »-. H. l. , . ._. —— -— Oufiunnuy . .-—. .~. ... . . ... .. ._ __. TM! -__ 39:- __- re __ ; Iv— """ _ . ... ... ... .._. —.-. . . 5-D 3-I-no-«tints-I """ _ _. ... ... ... .. ... ... _.. ... ..
  40. 40. _ T‘ xxxk '3 T" xx. xk. — x. xx%. Ev " . M. ‘- x. xx. -< x. xxk. xx. xx35. - xx. xx%. ... Optimizely — Results . ... . ’TT 1 :4; ” «W ! ".__. "_"Y 7. f; ."I -‘ 1:2: Q Confluence - Corresponding Test Page Confluence - Results Archive
  41. 41. Confluence - Corresponding Test Page T Confluence - Results Archive Our Test Philosophy - Everyth"1g afiectnng COITVSTSI-Sn is tested - Be aggessive - Regular cadence. no trafinc wasted - Never run a test that you wouldn‘t want to win [1]
  42. 42. + . + Confluence - Results Archive +- . . . . nun I-av j : - : ~a o-an-u--o-co -s- —---. o.. ~~. »-. ... n Q1?‘ :1 -— nu —-unn¢uc—‘-1’ :2 2'. '‘: .‘. ‘.‘. '' 122-: -‘* : : — :2 '_-. ::‘: ::: :.. ;: :-- "rm--~ -« ‘A --- on-n «-o¢~-an-run--In '*' . o v-no-—a—-n-p jyfllfico 1- - - I jg; _ Our Test Philosophy - Every1hing aflecting conversion is tested - Be aggessive - Regular cadence. no trafiic wasted - Never nn a test that you wouldn't want to Wit , Key takeaways: | «'_leu3 we e-very'w'Ie'e CUHF-"' as 'na'ry a5 3 L‘-'l<'. JflI| .IE t: ~;i5e; : on Irrj: -a~: I, l_. "_se- nuke’. rwnber ex~-er‘, - where so ever‘, u3r‘e | '_i re'e'rm-g to he same test. | n'. 'r_1|‘. 'e sIalv: .eI‘ulder-.5 and develope-': a.. a'1r_i get apprmals
  43. 43. - Regular cadence‘ ll-Ll trulfi-: was’. ed - Never run a test that you wouldn‘t want to win Key takeaways: | «‘_iea'_-‘ are F: 'yEFy"h'lF_-"l: ‘ r; ;;l| e~; :’. as 'na'r, ‘ as yL". .l car‘. ;_vlr_1r‘tIze t": l.35.: "LI an lr"pa< I, Use tn; -net r'u'nber e-x~er‘, 'where so e'«er*, :or'e | ':’ F£"f: "'l’l'l-‘:1 to he same test. | rwr_1|'. 'e '_$IilF2,‘Jl‘L. "lUF: ‘f'_5 and develope-'5.. and get approvals B T§’TAtlassian s u M MIT Thank you!
  44. 44. + + VAII K IIALIIOI 130M + . ’."Atlassian HEAD OI INTERACTIVE AVLA:5lAN SOD IWAJIL -In -ill:
  45. 45. + ‘ ’ Ymtlassiansummlr * Thank you! Q 6 HAIR HALVORBDII HEAD OI INTERACTIVE - ATLABSIAN SOITIARE ~ Oholvolli ‘ + + ‘I’ + + 9

×