A controlled experiment in assessing and estimating software maintenance tasks

1. A CONTROLLED EXPERIMENT IN
ASSESSING AND ESTIMATING
SOFTWARE MAINTENANCE TASKS
Pondicherry University
By:
SADIQUE
NAYEEM

2. Outline
Motivation and Background
Experiment Design
Results and Explanatory Models
Conclusions
2

3. MOTIVATION AND
BACKGROUND
3

4. Software Maintenance
 Software Maintenance
 “Modification of a software product after
delivery to correct faults, to improve
performance or other attributes, or to adapt the
product to a modified environment” [IEEE ‘98]
 “Changes are made in response to change
requirement but the fundamental software
structure is stable.
4

5. Maintenance is crucial in software
engineering
 Systems are tightly coupled with their environment
 Environment changes require changing its software systems
 Technologies and requirements are continuously changing
 Software systems are outdated quickly
 Software systems must be updated and upgraded to maintain their
values
 Maintenance is important in market competition
 New software has more advantages than the existing one
 Software system must be upgraded to keep its market share
5

6.  Maintenance cost is usually 2x to 100x as much as new
development cost (Sommerville, 2006)
K$100 200 300
System 1
System 2
(Summerville, 2006)New Development
Maintenance
400
Development and Maintenance
cost6

7. For software maintenance, the modeling process
is even
more challenging
 The maintenance effort is affected by a large number of
factors such as:
 Size and types of maintenance work
 Team stability
 Personnel capabilities
 The level of programmer’s familiarity with the system
being maintained
 Processes and standards in use
 Complexity
 Technologies
 The quality of existing source code and its supporting
documentation
7

8. Software estimation community has paid
little attention to software maintenance
 Most estimation models regard maintenance estimation
as secondary
 Models were built using mainly data of new development projects
 Use of SLOC metrics for models is inconsistent
 Some models use SLOC added, modified, deleted
 Others use only SLOC added and modified
8

9. Types of Software Maintenance
 Types of Maintenance
Different author proposed different classification of Software
Maintenance
 Swanson ’76 : Adaptive, Corrective, Perfective
 IEEE ’98: all Swanson’s plus Preventive
 Chapin et al, 2001: 12 types, 4 clusters: support interface,
documentation, software properties, and business rules. The last
cluster, includes all activities that alter the business rules of the
software :
 Enhancive : ‘Did the activities replace, add to, or extend the
customer-experienced functionality?’
 Corrective : ‘Did the activities fix the customer-experienced
functionality or make it more correct?’
 Reductive : ‘Did the activities restrict or reduce the customer-
9

10. Impact of the types of software
evolution and maintenance.
Ned Chapin, et al., “Types of software evolution and software maintenance,” Jan. 2001
10
Figure 1. Impact of the types of software evolution and maintenance.

11. The purpose of the study
 To assess the size and effort implications
 Effort distribution of three different maintenance types
 To describe estimation models to predict the
programmer’s effort on maintenance tasks.
 The focus of study is on enhancive, corrective, and
reductive maintenance types according to a
maintenance topology proposed by Chapin et al.
because they are the ones that change the business
rules of the system by adding, modifying, and deleting
the source code.
11

12. Research Questions and Hypotheses
 RQ1: Are there any differences in the
productivity of enhancive, corrective, and
deductive maintenance?
 Hypothesis 1 (H1): no difference
 RQ2: Are there any differences in the effort
distribution among the maintenance types?
 Hypothesis 2 (H2): no difference
12

13. EXPERIMENTAL
DESIGN
13

14. Experiment Description
 23 students and 1 senior, computer science major
 Participants worked on tasks individually in the lab
 Time limit of 7 hour.
 Participants were given the original source code, a list of maintenance
activities, and a timesheet form.
 Participants were required to record time on paper for every activity
performed to complete maintenance tasks. Time information includes start
clock time, stop clock time, and interruption time measured in minute.
 Participants used Visual Studio 2005 for maintenance on Windows XP
 UCC as a target program
 5188 source statements (logical SLOC) in 20 C++ classes
 The program was well-structured and well-commented
14

15. Matrices
Independent variable -- type of maintenance (enhancive,
corrective, and reductive)
Dependent variables -- programmer’s effort and size of
change.
The programmer’s effort = total time the programmer spent
to work on the maintenance task excluding interruption time.
The size of change is measured by three LOC metrics, LOC
added, modified, and deleted.

16. Calculating Maintenance SLOC
Equivalent SLOC = TRCF x AAM




>+
≤−−+
=
=
1,
100
1,])1(1[1( 2
AAFfor
UNFMxSU
AAF
AAFforUNFMxSUxAAFAAF
AAM
TRCF
S
AAF
TRCF = the total SLOC of task-relevant code fragments
S = the size in SLOC (added, modified, or deleted)
SU = the software understandability(10% to 50%)
UNFM = the level of programmer unfamiliarity with the program(0.0 to 1.0)
AAF= adaptation adjustment factor
AAM= adaptation adjustment multiplier
Task-relevant
code fragment
(TRCF)
16
In this experiment, COCOMO II reuse model is used to determine the equivalent
LOC of the maintenance tasks.

17. RESULTS AND MODELS
17

18. Resulted Data
 Each task requires four maintenance activities
 Task comprehension (reading, understanding task requirements,
and asking for further clarification)
 Code isolation(locating and understanding code segments to be
adapted)
 Editing code(programming and debugging the affected code)
 Unit test(performing tests on the affected code)
 Timesheet has 490 activity records, totaling 4621 min.
 Total of 909 SLOC added, modified, and deleted
402 added
216 modified
291 deleted
18

19. Effort distribution is different among the
groups (For H2)
 Corrective group spent much
time for code isolation
 twice as much as that of the
enhancive group
 Enhancive group spent
majority of time for editing
code
 Effort distribution is statistically
different among three groups
 H2 is rejected
19

20. Productivity
20
Enhancive group produced
1.5 times as many Reductive group deleted and
4 times as many Corrective group modified.
Work done by each group as
per the pattern dictated by
the task design.

21. Productivity is significantly different
among the groups (For H1)
Box Plot
 Corrective group has lowest
productivity
 Reductive group has
highest productivity
 Productivity between groups
are statistically different
 H1 is rejected
Enhancive Reductive Corrective
Productivity(SLOC/Hour)
40
30
20
10
21

22. Participant Time Explanatory Models
M1 Effort = 78.1 + 2.2 * S * EAF
M2 Effort = 43.9 + (2.8*Add + 5.3*Mod + 1.3*Del) * EAF
Effort = Time spent by the participant on all maintenance tasks
EAF = Effort adjustment factor, a product of Programmer capability (PCAP),
Language experience (LTEX), and Platform Experience (PLEX)
Add, Mod, Del = Equivalent SLOC added, modified, and deleted by the participant, respectively
S = Add + Mod + Del
R2
= The coefficient of determination values
R2
= 0.5 R2
= 0.75
22
R2
= 0.55 R2
= 0.64
M3 Effort = 110.0 + 2.2 * S * EAF
M4 Effort = 79.1 + (2.3*Add + 4.6*Mod) * EAF
In this experiment, EAF is defined as the multiplicative of programmer capability (PCAP), language and
tools experience (LTEX), and platform experience (PLEX). The same rating values for these cost drivers
that are defined in the COCOMO II Post-Architecture model is used.

23. Model M2 outperforms M1 , M3, M4
i
ii
i
Actual
EstimateActual
MRE
−
=
23
Fig The MRE values obtained by the models in estimating the effort spent by 24
participants
Magnitude of relative error (MRE)

24. Threats to Validity
 Internal design
 Groups have imbalanced skills and capability
 Imbalanced complexity of tasks on three groups
 Incorrect time recorded
 Generalizability
 Professional programmers are more experienced
 Professional programmers are more familiar with
the software maintained
 Maintenance process is different in industry
24

25. Conclusions
 Productivity and effort distribution are significantly different
among the maintenance types
 SLOC metrics are relevant factors for estimating effort
 Three SLOC metrics (added, modified, and deleted) have
different impact on effort
 SLOC deleted is an important factor for estimating effort
 It is more expensive to modify than to add or delete a statement
 Assigning experienced programmers to fixing defect can
save up to 40% of effort
25