A fairy tale about falling into a trap of wrong interpretation of results. Shows the importance of building models and understanding them.

1. Fairy tales in the land of data
Or - do I know what I’m doing?
By @przemur from
http://about.me/przemek.maciolek

2. A story

3. http://yamao.deviantart.com/art/Cleric-comm-343786321 https://www.flickr.com/photos/jsjgeology/8359854092/

7. Suspense

8. <
?
“The hammers
from the new
provider are no
good, sayr.”

9. What would you do?

10. New hammers
since this month

11. install.packages('ggplot2')
require('ggplot2')
setwd("/Users/pmm/Desktop/hammer")
all <- read.csv(file="all.csv")
!
qplot(all$month_sequence, all$dwarfs) + geom_smooth()
qplot(all$month_sequence, all$production) + geom_smooth()
!
all$prod_per_dwarf <- all$production / all$dwarfs
qplot(all$month_sequence, all$prod_per_dwarf) + geom_smooth()

12. Number of dwarfs working in the mine
The hammers from the new
provider started being
distributed to the new miners.

13. Total production of gold

14. Per-dwarf average production

15. Who sees any problem?

16. Lets look at the production of each
dwarf, relative to the time one applied…
Dwarfs which are using the
OLD hammer design
Dwarfs which are using the
NEW hammer design

17. new <- read.csv(file="new_relative.csv")
old <- read.csv(file="old_relative.csv")
!
qplot(new$relative_month, new$production)
ggplot(new, aes(x=relative_month, y=production)) + geom_point(shape=19,
position=position_jitter(width=.5,height=0), alpha=.2)
# This will look much better!
old$type='old'
new$type='new'
old_and_new = rbind(old,new)
ggplot(old_and_new, aes(x=relative_month, y=production, color=type)) + geom_point(shape=19,
position=position_jitter(width=.5,height=0), alpha=.2)

18. Scatterplot showing relative production
done using old and new hammers

19. What now?

20. ggplot(old_and_new, aes(x=relative_month, y=production, color=type)) + geom_point(shape=19,
position=position_jitter(width=.5,height=0), alpha=.1) + geom_smooth(method=lm)
The new hammers
wear much faster!

21. How much did the dwarfs lost?

22. old_m = lm(production ~ relative_month, old)
new$possible_production <- predict(old_m, new)
sum(new$possible_production) - sum(new$production)
(sum(new$possible_production) - sum(new$production))/sum(new$production)
0.5%
Now, taking into account the price of
hammer, one can select the optimal
strategy… but that’s another story…

23. Lessons learned …?
• Don’t trust the data blindly, ask questions
• Try to understand underlying rules of the system
• Don’t be shy with trying various models
• If using R, go for ggplot2