Meaningful thresholds are essential for promoting source code metrics as an effective instrument to control the internal quality of software systems. Despite the increasing number of source code measurement tools, no publicly available tools support extraction of metric thresholds. Moreover, earlier studies suggest that in larger systems significant number of classes exceed recommended metric thresholds. Therefore, in our previous study we have introduced the notion of a relative threshold, i.e., a pair including an upper limit and a percentage of classes whose metric values should not exceed this limit.
In this paper we propose RTTOOL, an open source tool for extracting relative thresholds from the measurement data of a benchmark of software systems. RTTOOL is publicly available at http://aserg.labsoft.dcc.ufmg.br/rttool.
FAIRSpectra - Enabling the FAIRification of Spectroscopy and Spectrometry
RTTOOL: A Tool for Extracting Relative Thresholds for Source Code Metrics
1. A Tool for Extracting Relative Thresholds
for Source Code Metrics
APPLIED SOFTWARE
ENGINEERING RESEARCH
GROUP
/Federal Institute
Minas Gerais
Paloma Oliveira @PalomaFormiga
Fernando P. Lima
Marco Túlio Valente @mtov
Alexander Serebrenik @aserebrenik
5. Relative Thresholds
Paloma Oliveira, Marco Tulio Valente,
Fernando Paim Lima: Extracting relative
thresholds for source code metrics.
CSMR-WCRE 2014: 254-263
For metric M,
p% of the entities should not exceed k
84%
94%
RTTool - A Tool for Extracting Relative Thresholds for Source Code Metrics
In this video, I will present RTTool, this tool aims to extract relative thresholds from the measurement
data of a benchmark of software systems.
Numerous metrics have been proposed to measure quality –or mostly- maintainability of software.
However, application of metrics in practice requires an ability to distinguish “desired” metric values from the undesired ones.
One way to achieve this is by establishing thresholds.
Of course, there are two problems with thresholds. First of all, they seem to be arbitrary as they are often based on rather intangible experience of practitioners; it is difficult to claim that threshold designed for systems in a different domain, developed using a different technology etc are applicable to a given situation. Alternatively, one can try to derive the thresholds from a collection of benchmark systems
The second problem is related to presence of the “bad guys” – large or highly complex entities. Numerous studies show that *most* software metrics follow a skewed heavy-tailed distributions, suggesting that presence of the “bad guys” in large projects is unavoidable: if this was not the case, distributions had different shapes.
Basically, the goal is to derive relatives (ruelatives) thresholds with the following format:
P percent (pôrcent) of the classes should have M less than or equal to K (key).
where M is a source code metric and p is the minimal percentage (Pôrcentade) of entities in each system, that should respect (rIspect), the upper (apperr) limit k (key).
The question is, of course, how to choose the relative thresholds. Clearly, by opting for a very high k p can get up to 100%. This is, however, not the intention.
If the demo does not work use backup slides
First, we need to choose the input format: XML or CSV, and click start button
Next, we need select the corpus. In this case, i will select 106 systems of the qualitas corpus.
Then, we need select the directory that RTTools generates its results. And, click next.
In this interface RTTools show the list of available metrics.
In this case, there are 20 metrics avaiable.
We have some options to select the metrics.
We can to select all metrics or we can to select some metrics. For this example, we are select 4 metric: Number of Output, Number of Attributes
, Number of Lines of Code (LOC), and Number of Methods (NOM). Then, we should click in Next button.
In this moment, RTToll is analyzing this datas and it is calculating functions to generates relative thresholds and outlier systems.
Finally, RTTool generates the results. We can see in this table the values of P and k to each metric analysed. This table also shows the number and the name of the systems with outlier behavior.
In this interface we can save this result in csv file.
We can reset the tool.
We can back for the last interface
Or we can see the graphics.
This interface shows 3 types of graphics:
First: Final result compliance rate penalty function
Second: compliance rate function
And third: the graph of percentile function.
First, this graph shows the result of each metric. This is the result of compliance rate penalty function.
We can see that, the threshold is the black dot in the graph. and the colors represents the possible values of p.
As you can see, the values of p in the graph can be modified to allow different views of the results.
In this interface, We can also see the graph of compliance rate function. The ComplianceRate Function returns the percentage of systems in the Corpus that follows the relative threshold defined by the pair [p; k].
Finally, this interface also presents the graph of percentile function.
The ideia of this graph is to show that the metric follows a heavy tail distribution.
Numerous metrics have been proposed to measure quality –or mostly- maintainability of software.
However, application of metrics in practice requires an ability to distinguish “desired” metric values from the undesired ones.
One way to achieve this is by establishing thresholds.