www.BrandLessBox.com White Paper

1.
Aaron
Abbott—www.BrandLessBox.com
1
Running
Head:
SPLIT
TESTING
Split
Testing
Aaron
M.
Abbott
www.BrandLessBox.com

2.
Aaron
Abbott—www.BrandLessBox.com
2
The
opportunity
to
directly
test
the
effectiveness
of
your
ad
in
real
time
is
unquestionably
an
innovation
for
marketers.
The
value
is
not
only
in
understanding
how
to
read
the
results
of
your
testing,
but
more
importantly,
in
the
understanding
of
how
to
conduct
your
tests.
Sometimes,
the
logical
thought
is
not
the
foundation
for
an
appropriate
testing
methodology,
leading
to
misleading
results.
Dan
Thies
sets
us
straight
with
his
breakdown
of
how
to
“set
up
a
valid
test”
(Thies,
2008).
While
the
idea
of
A/B
split
testing
would
seem
a
natural
choice
to
determine
which
ad
is
most
effective,
you
really
need
to
split
up
your
variables
and
eliminate
factors
that
could
lead
to
inequality
between
your
variables.
As
Thies
points
out,
by
not
setting
a
true
control
parameter
that
will
allow
you
to
control
other
variables
that
affect
placement
and
impressions—such
as
quality
score—‘”the
‘usual
method’
of
ad
testing
[will
deliver]
invalid
results.”
His
ingenious
approach
allows
you
to
not
only
take
out
the
fact
that
the
successful
ad
has
higher
quality
score
from
existing
time
in
rotation,
and
the
existing
click
thru
rate;
but
to
also
maximize
the
ability
for
your
top
producing
ad
to
still
be
placed
most
often
to
maximize
your
spending.
We
need
not
use
our
entire
budget
for
testing,
but
maximizing
what
we
should
spend
for
testing
will
maximize
spending.
For
www.real-­‐CSI-­‐videos.com,
ad
testing
will
become
a
most
valuable
tool
for
determining
the
measures
of
best
spending
with
a
very
limited
budget.
The
keyword
“autopsy”
is
our
most
cost-­‐effective
term
to
launch
the
campaign
with.
After
launching
with
about
5
versions
of
ads
optimized
with
landing
pages
for
the
term,
we
can
get
a
sense
of
which
ad
is
proving
most
effective.
At
this
point
we
would
eliminate
the
other
ads
that
are
not
performing
and
would
want
to
test
newer

3.
Aaron
Abbott—www.BrandLessBox.com
3
concepts
against
the
one
that
works.
Figure
1
shows
how
this
second
stage
testing
will
work.
The
reason
for
this
requiring
a
second
stage
is
that
the
initial
launch
of
the
campaign
allows
all
ads
to
start
from
the
same
null
point.
Figure
1.
Allows
working
ad
to
receive
80%
of
exposure
while
simultaneously
creating
true
control
variables
to
test
the
effective
ad
against
the
new
concept.
Autopsy
photos.
Autopsy
Videos.
Real
Autopsies.
250+
Narrated
videos.
This
is
the
Existing
Ad.
Copy
1
of
Copy
2
of
Copy
3
of
Existing
Ad
Existing
Ad
Existing
Ad
You
want
Autopsy
Pictures?
Watch
Autopsy
Videos.
Here.
Now.
This
is
the
New
Ad
being
tested.

4.
Aaron
Abbott—www.BrandLessBox.com
4
References
Thies,
D.
(May
6,
2008).
Split
testing
AdWords:
you’re
doing
it
wrong.
SEO
Fast
Start.
Retrieved
on
February
14,
2010
from
http://www.seofaststart.com/
blog/split-­‐testing-­‐adwords-­‐youre-­‐doing-­‐it-­‐wrong