Learning about Amazon free cross platform service - AB Testing. And adding knobs to your apps, and fine tune the user experiences and get the max results out of UI, monetization and etc.

1. Adding Knobs to your App – Modifying &
Measuring Behavior in the Wild (without
Republishing)
Cheng
Luo,
Amazon
Appstore
11th
January,
2014
@chengluo
#mceconf
#mce-addingknobs
All
rights
reserved.
May
not
be
copied,
modified,
or
distributed
in
whole
or
in
part
without
the
express
consent
of
Amazon.com,
Inc.
©
2013
Amazon.com,
Inc.
and
its
affiliates.
11 Jan 2014

2. Agenda
•
•
•
•
Why Amazon Mobile SDK
How to use A|B testing
Add your A|B testing in 10 minutes
Demo
#mceconf
#mce-addingknobs
11 Jan 2014

3. Active Users
Monthly Unique
Visitors (US):
Active Global Customer
Accounts:
200M
1
Age Composition of Amazon
Unique Visitors2
97M
Avg. Daily Unique Visitors
(US):
2
Household Income of Amazon
Unique Visitors2
18-­‐34
% of Amazon Shoppers Who have
Made an Online Purchase2
38%
32%
45%
30%
18%
0-­‐18
10%
35-­‐44
6.6
2
52%
8%
42%
12.9M
2
Avg. Visits per
Visitor per
Month (US):
45+
<$25K
$25K-­‐$75K
$75K
+
<$50
25%
$50-­‐$200
Amount Spent
$200+
Note: Active Customer Accounts are unique email addresses that have placed an order in the last 12 months
Sources: 1. Amazon Investor Relations; 2. ComScore PlanMetrix January, 2013.
#mceconf
#mce-addingknobs
11 Jan 2014

4. Simplicity
• Amazon
Appstore
runs
Android
apps
• 75%
of
Android
apps
work
without
any
need
for
customisaTon
• Simple
APIs
to
publish
free-­‐to-­‐user,
freemium
and
premium
apps
Device
Messaging
API
In-­‐App
Purchasing
API
#mceconf
#mce-addingknobs
A/B
TesTng
Service
API
Maps
API
GameCircle
API
11 Jan 2014

5. The latest Kindle Fire HDX
Kindle
Fire
HDX
7"
Tablet
$229,
with
24x7
May
Day
support
#mceconf
#mce-addingknobs
11 Jan 2014

6. What A|B Testing can do for you
Precisely
control
who
is
affected
#mceconf
#mce-addingknobs
Safely
test
back-­‐end
load
Change
in
the
wild;
No
new
APK
needed
11 Jan 2014

7. Case Study - Air Patriots
Used A/B Testing to decide:
• Include in-game advertising
• Provide purchasing short-cuts
• Notification frequency
• Level difficulty
• Promotional imagery
#mceconf
#mce-addingknobs
11 Jan 2014

8. Case 1 – Image
#mceconf
#mce-addingknobs
11 Jan 2014

9. Case 2 – Ads placing
#mceconf
#mce-addingknobs
11 Jan 2014

10. Case 3 - Notification
Notification appears
• x1 day
• x 3 days → Sweet spot
• x 7 days
#mceconf
#mce-addingknobs
11 Jan 2014

11. Case 4 - User Retention
Level:
1
Difficulty:
Easy
#mceconf
#mce-addingknobs
Level:
5
Difficulty:
Medium
11 Jan 2014

12. Air Patriots Sample Results
#mceconf
#mce-addingknobs
11 Jan 2014

13. Add A/B Testing to Your App
#mceconf
#mce-addingknobs
11 Jan 2014

14. Add A/B Testing to Your App
#mceconf
#mce-addingknobs
11 Jan 2014

15. Add A/B Testing to Your App
#mceconf
#mce-addingknobs
11 Jan 2014

16. Initialize A/B Testing in Your App
private static final String APP_KEY = "876c809ba7084b9fbe2c08a22e5acda1";
private static final String PRIVATE_KEY = "9nxxk4//qL89ymXl/P5aL7w8uhs3DVDrLcyYViG6yn0=";
AmazonInsights insightsInstance;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
. . .
InsightsCredentials credentials = AmazonInsights.newCredentials(APP_KEY, PRIVATE_KEY);
insightsInstance = AmazonInsights.newInstance(credentials, getApplicationContext());
}
#mceconf
#mce-addingknobs
11 Jan 2014

17. Create an A/B Testing Project
#mceconf
#mce-addingknobs
11 Jan 2014

18. Create an A/B Testing Project
#mceconf
#mce-addingknobs
11 Jan 2014

19. Create an A/B Testing Project
#mceconf
#mce-addingknobs
11 Jan 2014

20. Add a New Launch
#mceconf
#mce-addingknobs
11 Jan 2014

21. Add a New Launch
#mceconf
#mce-addingknobs
11 Jan 2014

22. Add a New Launch
#mceconf
#mce-addingknobs
11 Jan 2014

23. Multiple Projects Are Allowed
private static final String AB_PRJ_ACC = "Acceleration";
private static final String AB_VAR_MAXACC = "maxAcceleration";
//Project name
//Variable name
private static final String AB_PRJ_VEL = "Velocity";
private static final String AB_VAR_MAXVEL = "maxVelocity";
//Project name
//variable name
#mceconf
#mce-addingknobs
11 Jan 2014

24. Load Variation Variables
private static final int DEF_MAXACC = 5;
private static final int DEF_MAXVEL = 50;
private int maxAcc;
private int maxVel;
private void loadTreatment() {
ABTestClient abClient = insightsInstance.getABTestClient();
abClient.getVariations(AB_PRJ_VEL, AB_PRJ_ACC).setCallback(new InsightsCallback<VariationSet>() {
@Override
public void onComplete(VariationSet variations) {
Variation variation = variations.getVariation(AB_PRJ_ACC);
maxAcc = variation.getVariableAsInt(AB_VAR_MAXACC, DEF_MAXACC);
variation = variations.getVariation(AB_PRJ_VEL);
maxVel = variation.getVariableAsInt(AB_VAR_MAXVEL, DEF_MAXVEL);
}
});
}
#mceconf
#mce-addingknobs
11 Jan 2014

25. Changing Values on the Fly
#mceconf
#mce-addingknobs
11 Jan 2014

26. Changing Values on the Fly
#mceconf
#mce-addingknobs
11 Jan 2014

27. Changing Values on the Fly
#mceconf
#mce-addingknobs
11 Jan 2014

28. Changing Values on the Fly
#mceconf
#mce-addingknobs
11 Jan 2014

29. Add a Segment
#mceconf
#mce-addingknobs
11 Jan 2014

30. Add a Segment
#mceconf
#mce-addingknobs
11 Jan 2014

31. Add a Segment
#mceconf
#mce-addingknobs
11 Jan 2014

32. Use the Segment
Use
App
Segngs
to
idenTfy
your
user
segments
//get user age from the app settings
SharedPreferences myPrefs;
myPrefs = getSharedPreferences("com.example.testingapp", Context.MODE_PRIVATE);
int userAge = myPrefs.getInt("com.example.testingapp.USER_AGE", "");
//Initial the segments here
UserProfile userProfile = insightsInstance.getUserProfile();
userProfile.addDimensionAsNumber("age", userAge);
#mceconf
#mce-addingknobs
11 Jan 2014

33. Demo
#mceconf
#mce-addingknobs
11 Jan 2014

34. Wrap up
• A/B Testing
– Free cross platform services
– Precise measuring behaviours using custom
segments
– Modifying app behaviors without
republishing
#mceconf
#mce-addingknobs
11 Jan 2014

35. Thank you
@AmazonAppDev
#mceconf
#mce-addingknobs
11 Jan 2014