With more and more digitized companies and software being a crucial asset to gain competitive advantage, some firms choose to beat competition adopting Open Source to boost innovation. However, this journey requires a clear Software Data Strategy to succeed. This data-driven approach creates new ways to gather and visualize data to better find meaningful insights among software development activity, performance and community, for future reporting and decision making. GrimoireLab (https://chaoss.github.io/grimoirelab) produces integrated, open source software for analyzing software and visualizing metrics for measuring community, activity, contributions, performance and project health. Is one of the founding software projects of CHAOSS (Community Health Analytics Open Source Software) project, from The Linux Foundation (https://chaoss.community).

1. Measuring Software
development with
GrimoireLab
Valerio Cosentino
valcos@bitergia.com
@_valcos_
bitergia.com
OW2con’19, 12th-13th June

2. /context

3. /context
How can I measure my software
project?

4. /context
Activity
How many contributors do we have ?
Community
How many companies are contributing
to my project?
Processes
How good we are in handling issues?

5. /grimoirelab
chaoss.github.io/grimoirelab

6. /grimoirelab

7. /overview

8. /demo
What you need:
● Docker-compose [link]
● Micro mordred [link]
○ setup.cfg
○ projects.json
What you will see:
● Hands on Perceval, Graal, ELK ...
● A live dashboard

9. /takeaway
Helps you to measure your project
Easy to use
Easy to extend
100% Open Source
Active community
Join GrimoireLab

10. Valerio Cosentino
valcos@bitergia.com
@_valcos_
Let’s go for questions!
@grimoirelab
@bitergia
@CHAOSSproj
@crossminer

11. Backup slides

12. /mordred
chaoss/grimoirelab-sirmordred

13. /perceval
chaoss/grimoirelab-perceval

14. /perceval
Perceval: Software Project data at your Will

15. /perceval
{ "backend_name": "Git",
"backend_version": "0.10.2",
"category": "commit",
"data": {
"Author": "...,
"AuthorDate": "Mon Sep 3 11:37:40 ...",
"Commit": ...",
"CommitDate": "Mon Sep 3 11:37:40 …”,
"commit": "d3a8e97444dd9a63b0c ….",
"files": [...], "message": "Merge ...",
"parents": [...], "refs": [...]
},
"origin": "https://github.com/chaoss/grimoirelab-perceval",
"perceval_version": "0.11.11",
"tag": "https://github.com/chaoss/grimoirelab-perceval",
"timestamp": 1537685906.51857,
"updated_on": 1535967460.0,
"uuid": "9b7b4722fef72a8ce69781641a8220ed96868d0d" }
Data source
data
}
]Perceval
data

16. /perceval
As a program
As a Python3 library
$ pip3 install perceval
$ perceval git https://github.com/chaoss/grimoirelab-perceval
--from-date=2018-09-01
[2018-09-23 08:58:25,085] - Sir Perceval is on his quest.
[2018-09-23 08:58:25,088] - Fetching commits ...
… JSON Documents
[2018-09-23 08:58:26,540] - Sir Perceval completed his quest.
from perceval.backends.core.git import Git
from_date = datetime.datetime(2018, 9, 1)
backend = Git(".../grimoirelab-perceval", "/tmp/test")
commits = [commit for commit in backend.fetch(from_date=...)]

17. /arthur
chaoss/grimoirelab-kingarthur

18. /arthur
Arthur allows to schedule and run Perceval executions at
scale through distributed Redis queues.

19. /elk
chaoss/grimoirelab-elk

20. /elk
Fields
Documents
Indexes
ElasticSearch
author_name:
“Kenny Sampson”
created_at:
01/05/2017
question_tags:
[python, code]
author_name:
“John Smith”
created_at:
02/07/2019
question_tags:
[ATmega328P]
author_name:
“Mary Poppins”
created_at:
03/05/2019
question_tags:
[arduino]
author_name:
“John Smith”
created_at:
01/01/2019
question_tags:
[diy, arduino,
ATmega328P]
author_name:
“Jane Doe”
created_at:
01/02/2018
question_tags:
[oss, license]

21. /sortinghat
chaoss/grimoirelab-sortinghat

22. /sortinghat

23. /sortinghat
Manrique López <jsmanrique@bitergia.com>
José Manrique López de la Fuente <jsmanrique@gmail.com>
Manrique López <jsmanrique@gmail.com>
jsmanrique
jsmanrique@gmail.com
jsmanrique@bitergia.com
correo@jsmanrique.es
jsmanrique
jsmanrique@bitergia.com
02/2005 - 12/2010 CTIC
01/2010 - 12/2012 Andago
01/2013 - 06/2013 TapQuo
07/2013 - 12/2015 freelance (ASOLIF, CENATIC)
07/2013 - now Bitergia

24. /sortinghat
Who is who?

25. Hatstall
/sortinghat
SortingHat: Wizardry on Software Project Members

26. /hatstall

27. /manuscripts
chaoss/grimoirelab-manuscripts

28. /manuscripts

29. /sigils
chaoss/grimoirelab-sigils

30. /sigils

31. /kibiter
Index Patterns
Visualizations
Panels
Kibana
Fields
Documents
Indexes
ElasticSearch
Kibiter is a downstream of Kibana tailored
to GrimoireLab needs
chaoss/grimoirelab-kibiter

32. /index-pattern

33. /visualizations
* Extra visualizations (Kibana plugins)
developed by Bitergia

34. /visualizations

35. /visualizations

36. /panels

37. /context
How can I measure my software
project?

38. /grimoirelab
Activity
How many contributors we have ?
Community
How many companies are contributing
to my project?
Processes
How good we are in handling issues?

39. Answering
questions with
visualizations
Data from CHAOSS project
chaoss.biterg.io

40. /activity
How many contributors
do we have ?

41. /activity
Project attraction and retention

42. /activity
Active organizations

43. /community
How many companies are
contributing to my project?

44. /community
Organizations in a project

45. /community
Comparing organizations activity

46. /processes
How good we are in handling
issues?

47. /processes
Responsiveness

48. /processes
Performance

49. /processes
Sustainability

50. /processes
Efficiency panel