My talk at 2015 FOSS Community Metrics. Overview of the dangers of certain metrics about open source communities, Liferay's methodology used to select, track, and act on, and what we're doing in the future,

1. liferay community metrics
james falkner
liferay community
@schtool
2015 FLOSS Community Metrics

2. Value of
Community

3. liferay community
13 years
4.3M LOC
140k registered site members
102 external contributors in 2014
150 SI partners
5M downloads
1200 marketplace developers
43 user groups
114 translators for 43 languages
300k forum posts (~100/day)
25k forum participants
2 Top 50 github devs ( , )
#5 #45

4. body count

5. downloads
Exploded Eclipse Plugins
added to SourceForge

7. Ø identified metrics goals
Ø searched our feelings
Ø decided what to
measure
Ø defined success/fail
actions
how we developed metrics

8. liferay community
metric goal #1:
increase the value of
participation &
contribution

10. liferay community
metric goal #2:
Measure the effect
of business decisions

13. liferay community
metric goal #3:
understand the
relationship between
community metrics and
business performance

15. decide what
to measure

16. basic 1st-order metrics
for the liferay community
# of apps/reviews
# of pull requests
# of commits/lines
# of authors
# of bug reports
# of forum posts
# of downloads
# open tickets / code reviews

17. 1st Order Metrics
WE DON’T
REPORT THEM

18. more interesting 2nd-order metrics
non-liferay vs.liferay
time between contribution->codebase
location of community members
evolution of contributor age over time
evolution of active/inactive users
time of bug report -> fixed
time from forum q to a
# ignored/unanswered messages
# of abandoned code reviews
distribution & evolution of commits across
functional areas

19. for each metric…
Ø How accurate is it?
Ø Does it help achieve goals?
Ø What to do when it turns
good/bad?
Ø What is its relationship to
others?

20. Example: Community Contributions

21. future:
career evolution

22. newbie: 0-5 posts,0 answers,1 download,1 fork,…
junior: 20 posts,10% engaged,>1 patch,5 tweets,1 app
senior: 50 posts,30% engaged,10 tweets,5 apps,moderator,
…
champion: 100 posts, 50% engaged, bugsquad,…

23. summary
identify values and goals
decide actions ahead of time
assume inaccuracies
experiment & evolve
avoid focusing on vanity metrics

24. thank you!
james falkner
liferay community
@schtool
2015 FLOSS Community Metrics