Talk at GDS'19

1. 29.11.2019

2. Miroslav Pikhart
Geewa.a.s.
Head of BI
miroslav.pikhart@geewa.com
Why your game needs a data scientist?
Plus some use cases from Smashing Four

3. About Smashing Four - core game
● Free-to-play PvP game
● Teams of four heroes fight in the arena
● Characters resemble pool balls
● Each hero has an ability

4. About Smashing Four - economy
● Players get orbs for winning or buy them in shop
● Orbs contain random hero cards and gold
● Players collect these resources to upgrade heroes
and reach higher arenas
● Higher arenas lead to higher rewards, completing
the loop

5. Why your game needs a data scientist
and what personalized content can do for your revenue

6. Why your game needs a data scientist
and what personalized content can do for your revenue

7. Why your game needs a data scientist
and what personalized content can do for your revenue

8. How data can help improve your game
● Learn when and why players are leaving
○ Improve new player experience
● Understand how players play your game
○ Keep possible strategies balanced
○ Forecast how many players will use new content
● Find out what players want the most
○ Tailor progression to your players’ needs
○ Increase revenue through content personalization

9. Identifying when players leave your game
And how easy it sometimes is to improve that

10. Identifying when players leave your game
And how easy it sometimes is to improve that

11. Identifying when players leave your game
And how easy it sometimes is to improve that

12. Identifying when players leave your game
There’s more...
● The problematic goes beyond a simple funnel
○ Not only you need to know where you lost your players
○ It also helps to know why - is this part too long? Too boring?
● Similar, more ‘advanced’ use case are churn models
○ Basically predicting whether a player is likely to leave your
game
○ Using features such as spend, winrate, matches per day...

13. Sometimes things go off balance
● Another area where data shine is keeping the game in balance
● Sometimes things interact in a way you did not anticipate
○ This happens often, especially if your game is complex
● If anything suddenly changes - you have to know why

14. Sometimes things go off balance

15. Using data to drive revenue increase
● With all the data you’re getting, you know all about your players
● That puts you into the perfect position to leverage all this data
● Why would you try to sell the same thing to everyone if you know
that they are not the same?

16. Intermezzo - Special offers in Smashing Four
● Essentially a chance to buy bundled orbs and
currency at a discount
● Only available at limited times
● Positive impact on conversion

17. How special offers affected conversion

18. How special offers affected conversion

19. Data usage case study - Hero orb
● Goal was to create a new product to improve monetization
● Instead of pushing richer special offers, we wanted to offer our
players something that they personally would like
● In order to reach wider audience, low price-point to start with

20. Hero orb - what it looks like

21. Hero orb - what’s behind it
● There’s a relatively simple algorithm behind the
scenes that scores heroes based on:
○ How often players use the hero
○ How often they request him from their clan
○ How much they upgrade him

22. Lesson learned #1 - Define your tests carefully
● When we first created it, we tried to A/B test the feature:
○ One group got the hero orb offer for $5 USD
○ The other got the very same offer for 500 gems (in-
game currency that’s worth exactly $5 USD)

23. Lesson learned #1 - Define your tests carefully
● Then we looked at the conversion to payer...

24. Lesson learned #1 - Define your tests carefully
● Then we looked at the conversion to payer...

25. Lesson learned #1 - Define your tests carefully
● Based on the figure just shown, the product
originally prefered to go with gems, however...

26. Lesson learned #1 - Define your tests carefully
● Based on the figure just shown, the product
originally prefered to go with gems, however...

27. Lesson learned #1 - Define your tests carefully
● The initial graph was significantly skewed by ‘old’ players
● These only wanted to find a way to spend their ‘free’ gems
● This shows how important it is to set your metrics precisely

28. Lesson learned #2 - Don’t test for the sake of testing
● This is the most common mistake I have encountered so far
● Sometimes people just love A/B tests too much
○ They are easy to set up and perform
○ Even if they sometimes offer almost no information
● Knowledge of statistics is a big asset in this area

29. Lesson learned #2 - Don’t test for the sake of testing

30. Lesson learned #2 - Don’t test for the sake of testing
● Specify the hypothesis you want to prove before running any
test
● Come up with some KPIs you can measure that are both
relevant and not corrupted
● Always have a control group to measure against

31. Hero orb - the results
● The product turned into our best performer and still runs strong

32. How special offers affected conversion

33. How personalized offers affected conversion

34. Summary
● With the rising trend of GaaS, data became significantly more important
● Data helps you improve both player experience and your revenue
● There’s (almost) never enough data
○ But it’s not that easy to differentiate the vital information from the rest
● Collecting data is only the first step
○ Then you need to interpret them and use the knowledge

35. So why would you want a data scientist?
● You are technically able to collect data down to a single click level
○ Having someone who can draw significant conclusions is a boon
● You can get answers to business-related questions like:
○ How much is this player likely to spend in my game?
○ How many players will actually reach endgame content?
○ How overpowered is this new item we designed?
○ If I create this new product, how many players will see it?

36. Why would you want to work in game dev?
● The whole field is only a few years old
○ There are problems yet to be completely solved
○ There aren’t agreed upon best practices for everything
○ You can still discover something innovative
● The amount of available data is huge
○ You’re working with large, complete data sets
○ The amount of possible features is (almost) endless

37. Thank you!
Questions?
geewa.com
Miroslav Pikhart
Geewa.a.s.
Head of BI
miroslav.pikhart@geewa.com