call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
Iwsm2014 evaluating software product quality (ali idri)
1. +
Evaluating Software Product Quality:
A systematic mapping study
Sofia Ouhbi, Ali Idri, José Luis Fernández-Alemán, Ambrosio Toval
IWSM MENSURA Conference
October 6-8,2014, Rotterdam
Presented by: Ali Idri
2. + Presentation Outline
1. Motivations and Objectives
2. Software Product Quality: Introduction
3. Research Methodology
4. Results
5. Principal findings
6. Implications
7. Conclusion and Future work
2
Evaluating Software Product Quality: A systematic mapping study
3. +
Motivations and Objectives
10/29/2014 Evaluating Software Product Quality: A systematic mapping study 3
4. + Motivations and Objectives
4
Motivations
• Obtain the current approaches used to SPQE
• No systematic mapping study of SPQE approaches has been
published
Objectives
• Summarize the existing SPQE approaches and whether or not
these approaches are based on well-known SQ models
• Discover if SPQE is done through one or many SQ characteristics
• Identify the research types and empirical types that exist in SQPE
literature
• List the data sets used in the evaluation of the SQPE approaches
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
5. +
Software Product Quality
Introduction
10/29/2014 Evaluating Software Product Quality: A systematic mapping study 5
6. + Software Product Quality
Evaluating Software Product Quality: A systematic mapping study
6
“Although quality is pursued in order to satisfy users, it is
important to note that the quality desired by users is not
universal but rather subject to constant change”
SQuBOK, JUSE 2007
Software product quality could be defined as “the totality of
characteristics of an entity that bear on it’s ability to satisfy
stated and implied needs”
ISO/IEC 9126-1
7. + Software Product Quality
o McCall
o Boehm
o Dromey
o ISO/IEC 9126 standard, which has been
replaced with ISO/IEC 25010
Evaluating Software Product Quality: A systematic mapping study
7
Several SQ models and standards with which to improve Software
Product Quality have been proposed such as:
These models have some common SQ characteristics, such as:
o Efficiency
o Reliability
o Portability
o Maintainability
9. +
Research Methodology
Systematic mapping study
10/29/2014 Evaluating Software Product Quality: A systematic mapping study 9
10. + Systematic mapping study
10
The systematic mapping study principal goal is to provide an
overview of a research area, and identify the quantity and type of
research and results available within it.
This method focuses on classification, conducting a thematic
analysis and identifying publication fora.
A mapping process consists of three activities:
1) the search for relevant publications,
2) the definition of a classification scheme,
3) the mapping of publications.
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
11. + Systematic mapping process
11
Process Steps
Definition of
Research
Questions
Conduct
Search
Screening of
Papers
Key wording
using
Abstracts
Data Extraction
and Mapping
Process
Review Scope All papers
Relevant
Papers
Classification
Scheme
Systematic
Map
Outcomes
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
12. + Systematic mapping questions
• Which publication MQ1 channels are the main targets for SPQE research?
• How has the frequency of SPQE research dissemination changed over
time? MQ2
MQ3 • In which research types are SPQE papers classified?
MQ4 • Which approaches have been used for SPQE?
MQ5 • Are the SPQE selected studies empirically validated?
MQ6 • What are the datasets that were used in SPQE literature?
MQ7 • Which artifacts have been reported in SPQE selected studies?
• What are the well-known SQ models that have been mentioned in SPQE
literature? MQ8
MQ9 • Which characteristics were used to evaluate SPQ?
12
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
13. + Search string
“Software quality" AND (evaluat* OR measur* OR assess*) AND
(technique* OR approach* OR method* OR model* OR framework*
OR tool* OR data mining OR learning OR artificial intelligence OR
pattern recognition OR analogy OR case based reasoning OR
nearest neighbo* OR decision tree* OR classification tree* OR
neural net* OR genetic programming OR genetic algorithm* OR
bayesian belief network* OR bayesian net* OR association rule* OR
support vector machine* OR regression OR fuzzy logic)
13
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
14. + Study selection
IC1
• The studies that address evaluation, measurement or assessment of the
quality of software product in overall or through SQ characteristics
EC1
• Papers that focus on system quality not on SPQ
EC2
• Papers whose subject was one or many SQ characteristics (e.g.
maintainability, reliability) which were not used in the evaluation of SPQ
EC3
• Papers that were published before the nineties and after 2013
14
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
15. + Study selection
15
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
16. + Data extraction
16
Empirical type
Research type
Artifact
SQPE Approach
-
- Data mining technique
- Process: A serie of
actions, or functions leading
to an SPQE result and
performing operations on
data.
- Method: A regular and
systematic means of
accomplishing SPQE.
- Tool-based technique: A
technique based on a
software tool to accomplish
SPQE tasks.
- Model: A system
representation that allows
SPQE to be investigated
through a hierarchical
structure.
- Framework: A real or
conceptual structure
intended to serve as a
support or guide for SPQE.
- Other, e.g. guidelines.
- Evaluation research:
Existing SPQE
approaches are
implemented in practice
and an evaluation of
them is conducted.
- Solution proposal:
An SPQE solution is
proposed. This solution
may be a new SPQE
approach or a
significant extension of
an existing approach.
- Other, e.g. experience
paper, review.
- Case study: An empirical
inquiry that investigates an
SPQE approach within its real-life
context.
- Survey: A method for
collecting quantitative
information concerning an
SPQE approach.
- Experiment: An empirical
method applied under controlled
conditions, in order to evaluate
a SPQE approach.
- History-based evaluation:
Studies evaluating SPQE
approaches in previously
completed software projects.
- Other
- Documentation
- Design module
- Source code
- Other
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
17. +
Results
10/29/2014 Evaluating Software Product Quality: A systematic mapping study 17
18. + MQ1. Publication channels
Publication Channel Total
Journal 33
Conference 12
Workshop 6
Symposium 3
Other 3
18
The Journal of Systems and Software is a recurrent publication source for
SPQE techniques, followed by the Information and Software
Technology Journal.
• There is no obvious conference target that attracts publication concerning
SPQE approaches
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
20. + MQ3,4,5. Research types, approaches and empirical types
20
20 4 4 4 1
1
7 1 4 3
3 16 2 10
2 9 3
2
2 4
1
1
8
2
Research
type
Solution
proposal
68%
Evaluation
research
28%
SPQE
Approaches
Empirical
type
Case
study
18%
Experiment
9%
History-based
evaluation
44%
Survey
9%
Theory
21%
Data mining
techniques
47%
Framework
7%
Method
11%
Model
14%
Other
7%
Process
5%
Tool
9%
Review
4%
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
21. + MQ4. SPQE Data Mining Approaches
21
DT
3%
GP
3%
Learning
4%
ANN
7%
BN
7%
Classification
7%
Clustering
11%
Pattern
matching
11%
Fuzzy
29%
Regression
18%
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
22. + MQ6. Data Sets
22
0 5 10 15 20 25 30
History-based evaluation
Experiment
Case study
DSP NASA OSS Other
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
23. + MQ7. Artifacts
23
0 5 10 15 20 25 30 35 40 45
Documentations
Design module
Source code
Papers reporting only one artifact Papers reporting more than one artifact
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
24. + MQ8. SQ Models
24
around 50% of SPQE papers do not cite any well-known SQ model
ISO/ IEC
9126
59%
Dromey
10%
Other
12%
ISO/ IEC
25010
2%
Boehm
10%
McCall
7%
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
25. + MQ9. SQ Characteristics
25
25% of SPQE papers did not mention any SQ characteristics
The main characteristic used in the evaluation of SPQ in literature is
reliability including fault-proneness, followed by maintainability and
functionality
23% of the selected studies had used all the ISO/IEC 9126
characteristics to evaluate SPQ
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
26. +
Principal findings
10/29/2014 Evaluating Software Product Quality: A systematic mapping study 26
27. + Principal findings
27
SPQE is taken very seriously by researchers
The interest in evaluating SPQ has increased dramatically last year
Researchers are still searching for good approaches with which to enhance
SPQE
Few researchers chose to evaluate existing SPQE techniques
The majority of SPQE approaches that were found are data mining-based
techniques, principally fuzzy logic, regression and clustering techniques
Data used was extracted from open source projects, domain specific projects
or from existing data sets such as NASA to evaluate SPQE solutions
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
28. + Principal findings
28
Case studies were used to evaluate SPQE approaches and a few experiments
were conducted in the SPQE papers selected
Source code was the main artifact concerned by SPQE approaches
Researchers are more interested to evaluate internal SQ
Half of the selected studies based their solutions on SQ models, particularly
the ISO/IEC 9126 standard
50% of the selected studies have not mentioned any well-known SQ models,
however they have evaluated SQ by using an SQ characteristic
Reliability was the most frequently reported SQ characteristic in SPQE
literature, and was in most cases achieved using fault-proneness
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
30. + Implications
30
This study will allow researchers and practitioners to discover the existing
SPQE approaches and techniques in literature.
The empirical studies presented may provide an overview of the efficiency of
each approach.
More studies involving recent SQ models are needed to develop approaches
that will meet SQ standards, particularly the ISO/IEC 25010 standard.
The evaluation of SQ characteristics which have an impact on the emerging
market of mobile software applications, should receive more attention from
researchers.
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
31. +
Conclusions & Future work
10/29/2014 Evaluating Software Product Quality: A systematic mapping study 31
32. + Conclusions and Future work
32
Our study has summarized the existing knowledge as regards SPQE
approaches
Papers dealing with SPQE approaches between 1990 and 2013 were
identified. 57 papers were selected
Ongoing research is based on proposing an empirical method with
which to evaluate SPQ in some specific environments
Evaluating Software Product Quality: A systematic mapping study 10/29/2014
33. +
Thank you for your attention!
Any questions?
10/29/2014 Evaluating Software Product Quality: A systematic mapping study 33