Speech of Alexey Vasiliev, Software Engineer at Railsware, at Ruby Meditation #25 Kyiv 08.12.2018
Next conference - http://www.rubymeditation.com/
In this talk, Alexey will tell about the project in which was necessary to implement A/B testing and what came out of it in result
Announcements and conference materials https://www.fb.me/RubyMeditation
News https://twitter.com/RubyMeditation
Photos https://www.instagram.com/RubyMeditation
The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation
1. Road to A/B testing
Brought to you by Oleksii Vasyliev
1
2. Oleksii Vasyliev
Web and Mobile Developer (Ruby, Java, JavaScript, Objective-C, C/C++,
Golang, Elixir/Erlang), DevOps
Open-Source libs: PGTune, SQL Joins Visualizer, RWbox, Go-Kinesis, ElixirV8,
WebP-ffi, Zopfli-ffi, MongodbLogger, SMTRails, SHTRails, ...
Open-Source books: Setting up and scaling of PostgreSQL (Russian),
Cooking Infrastructure by Chef
Leading RWpod podcast about Ruby and JavaScript
•
•
•
•
2
10. Split Extensions
Split::Export - Easily export A/B test data out of Split.
Split::Analytics - Push test data to Google Analytics.
Split::Mongoid - Store experiment data in mongoid (still uses redis).
Split::Cacheable - Automatically create cache buckets per test.
Split::Counters - Add counters per experiment and alternative.
Split::Cli - A CLI to trigger Split A/B tests.
•
•
•
•
•
•
10
12. Small changes from client
Support SPA applications (React, Vue.js, Marionette.js)
Cover only some percentage of visitors to target an experiment
I need determines when the experiment is shown: geo data, browser/mobile
device, third-party cookies, query parameter, etc
Running simple experiments (text/image variations, etc) without
development and deployment
Support Multivariate and Multi-Armed Bandit testing
•
•
•
•
•
12
15. Ploblems
I need better admin system for experiments (editor, settings, etc)
I need collect info about customers (geo, device, etc) and do not forget
about GDPR/Data Processing Addendum/etc
It will create load on system
Your customers not need this feature
You as developer not need this code
It is only need for marketing/business analytic team
•
•
•
•
•
•
15
17. PaaS pricing
Plans start at a minimum of $36,000 a year, for 200,000 visitors or more.
https://www.quora.com/How-much-is-optimizely
AB Tasty’s package starts at $1,000 per month
https://reviews.financesonline.com/p/ab-tasty/#pricing
Enterprise Pricing starts from $1,899/month
https://reviews.financesonline.com/p/omniconvert/#pricing
”
”
” 17
36. Conclusion
If you need PaaS for A/B testing - try Google Optimize
Free version limits: 5 experiments, 10 personalizations, 16 MVT
combinations. You can update to Optimize 360 if needed
Ability to do experiments without development
Works on top of a Google Analytics
Integration with Google Tag Manager provide additional flexibility in settings
•
•
•
•
•
36