SSBSE 2012 Keynote

SBSE Meets
Software Maintenance:
Achievements and Open Problems
Massimiliano Di Penta
University of Sannio, Italy
dipenta@unisannio.it

Friday, September 28, 12

Aim


Aim
Why SBSE is important for software
maintenance


Aim
maintenance

Tasks to which SBSE can be applied


Aim
maintenance


Challenges in supporting maintenance
through SBSE


Aim
maintenance


Challenges in supporting maintenance
through SBSE

Some possible solutions....


Deﬁnition

The process of modifying a software system
or component after delivery to correct faults,
improve performance or other attributes, or
adapt to a changed environment.
IEEE Glossary of Software Engineering Terminology


SBSE topics breakout

Source: SEBASE Repository
http://crestweb.cs.ucl.ac.uk/resources/sbse_repository/

SBSE is not only
Testing!


SBSE @ ICSM 2012
Ameni Ben Fadhel, Marouane Kessentini, Philip Langer
and Manuel Wimmer. Search-based Detection of High-
level Model Changes
Ali Ouni, Marouane Kessentini, Houari Sahraoui and
Mohamed Salah Hamdi. Search-based Refactoring:
Towards Semantics Preservation
Mathew Hall, Neil Walkinshaw and Phil McMinn.
Supervised Software Modularisation

However, if you look at many other papers,
some of the problems could have been nicely solved using
search based methods too!

Maint. @ SSBSE 2012
Roberto Erick Lopez-Herrejon, Jose Galindo, David Benavides, Sergio
Segura and Alexander Egyed. Reverse Engineering Feature Models With
Evolutionary Algorithms: An Exploratory Study
Gergõ Balogh, Ádám Zoltán Végh and Árpád Beszédes. Prediction of
Software Development Modiﬁcation Effort Enhanced by Genetic Algorithm
Alex Tomasi, Alessandro Marchetto and Chiara Di Francescomarino.
Domain-driven Optimization of Recovered Business Processes
Shadi Ghaith and Mel Ó Cinnéide. Improving Software Security using
Search-Based Refactoring
Mathew Hall and Phil McMinn. An Analysis of the Performance of the
Bunch Modularisation Algorithm s Hierarchy Generation Approach
Gabriele Bavota, Filomena Carnevale, Andrea De Lucia, Massimiliano Di
Penta and Rocco Oliveto. Putting the Developer in-the-loop: an Interactive
GA for Software Re-Modularization


Why
Software Maintenance
needs SSBSE?


Large space to explore...


Presence of conflicting
objectives
Maintenance tasks have to fulfill some design
goal / non-functional requirements
Design goals are likely conflictual
Maintainability

Performance

Examples
• maintainability vs performance
• time to market vs reliability
• functionality vs power consumption
• functionality vs cost
• customers satisfaction vs resource
requirements
• portability vs performance


Reducing power consumption
Customer’s satisfaction
Battery
consumption

Code size

Nasir Ali, Wei Wu, Giuliano Antoniol, Massimiliano Di
Penta,Yann-Gaël Guéhéneuc, Jane Huffman Hayes:
MoMS: Multi-objective miniaturization of software.
ICSM 2011: 153-162


Three objectives
cust.
satisf.

consumption

size

Size vs. consumption vs. customer satisfaction


Maintenance Activities
(std. IEEE 1219)
Plan Plan changes, bug triaging, staff activities

Analysis
Balance across conﬂicting goals
Design
Support to program comprehension
Implementation Support to change, automatic bug
ﬁxing self-change
Regression
testing (Regression) test generation,
prioritization, test suite repairing
System testing

Delivery Dynamic binding, self-healing

Two dimensions
Regression Self- Self-
testing repair evolving
Level of automation

Support to
Support to
program
change
comprehension

Project
management

Presence of human-centric rationale

Automation
vs. human-centric
• SBSE has been very successful in activities
that can be (almost) completely automated
• Think about test data generation, or non-
functional test
• Many maintenance activities are highly
human-centric
• Cannot fully and properly be captured by a
ﬁtness function


Where and how SBSE can help in software
maintenance in its different phases
Challenges, difﬁculties, open problems


Challenge I
Managing Maintenance
Activities

Some research
problems
• Schedule maintenance activities over time
• Staff software tasks
• Release planning
• Bug triaging


Some work in the area
J. Francisco Chicano, Francisco Luna, Antonio J. Nebro, Enrique Alba: Using multi-objective metaheuristics to
solve the software project scheduling problem. GECCO 2011: 1915-1922
Ahilton Barreto, Márcio de Oliveira Barros, Cláudia Maria Lima Werner: Stafﬁng a software project: A
constraint satisfaction and optimization-based approach. Computers & OR 35(10): 3073-3089 (2008)
Jian Ren, Mark Harman, Massimiliano Di Penta: Cooperative Co-evolutionary Optimization of Software
Project Staff Assignments and Job Scheduling. SSBSE 2011: 127-141
Massimiliano Di Penta, Mark Harman, Giuliano Antoniol: The use of search-based optimization techniques to
schedule and staff software projects: an approach and an empirical study. Softw., Pract. Exper. 41(5): 495-519
(2011)
Stefan Gueorguiev, Mark Harman, Giuliano Antoniol: Software project planning for robustness and
completion time in the presence of uncertainty using multi objective search based software engineering.
GECCO 2009: 1673-1680
Massimiliano Di Penta, Mark Harman, Giuliano Antoniol, Fahim Qureshi: The Effect of Communication
Overhead on Software Maintenance Project Stafﬁng: a Search-Based Approach. ICSM 2007: 315-324
Chao C, Komada J, Liu Q, Muteja M, Alsalqan Y, Chang C. An application of genetic algorithms to software
project management. Proceedings of the Ninth International Advanced Science and Technology, Chicago, IL,
U.S.A., 1993; 247–252.
Chang CK, Christensen MJ, Zhang T. Genetic algorithms for project management. Annals of Software
Engineering 2001; 11(1):107–139.
Alba E, Chicano F. Software project management with GAs. Information Sciences 2007; 177(11):2380–2401.


Problem statement
Project WBS Maintenance phases
Project

Abandonment
WP1 WP2 WP3

WP11 WP12 WP31 WP32

WP1 WP2 … WPn

WP
Ordering Rework

People distribution

Multi-objective
project scheduling


Handling specialized teams
Database GUI Networking Middleware
Core

10

Compl. Time
70 (person months)
58 56
8 53
50 49 48
Staffing

40 39
40 37
Staffing

5

20
10
3
0

0
1 2 3 4 5 6 7 8

Configuration
Configuration


Issues and open
problems
• Project management is a human-centric
activity
• A mathematical model seldom captures
personal capabilities
• Skill, experience, synergy, capability to
cooperate with others
• Even if we capture such variables in a model,
how do we measure them?


So where can we get
this data from?


Measure ﬁtness through
software repositories
Capture expertise
Versioning
Mails
Capture
communication and
mentoring skills
Identify emerging
Mylyn

Runtime
Bug
tracking
traces groups
Logs


Capture skills

• Given an incoming change request, who will
be the most suitable developer for ﬁxing it?
• See who, previously, ﬁxed very similar
change requests
• Textual similarity between bug reports
John Anvik, Gail C. Murphy: Reducing the effort of bug report triage: Recommenders for
development-oriented decisions. ACM Trans. Softw. Eng. Methodol. 20(3): 10 (2011)


Identify mentors ...
Messages
exchanged between
newcomers and
senior developers
used to identify
mentors
Similarly to what
ArnetMiner does
for academics

Gerardo Canfora, Massimiliano Di Penta, Rocco Oliveto, Sebastiano Panichella, "Who is going to
Mentor Newcomers in Open Source Projects?", To appear in Proceedings of FSE 2012, November
10-17 2012, North Carolina, USA

Who induced fixes?
f 1.5 f 1.6 f 1.7 f 1.8

bug fixing

SZZ algorithm based on CVS annotate or SVN/Git blame
Jacek Sliwerski, Thomas Zimmermann, Andreas Zeller: When do changes induce fixes? MSR 2005
Sunghun Kim, Thomas Zimmermann, Kai Pan, E. James Whitehead Jr.: Automatic Identification of Bug-
Introducing Changes. ASE 2006: 81-90

Who induced fixes?
f 1.5 f 1.6 f 1.7 f 1.8

bug intro bug intro bug fixing

SZZ algorithm based on CVS annotate or SVN/Git blame
Jacek Sliwerski, Thomas Zimmermann, Andreas Zeller: When do changes induce fixes? MSR 2005
Sunghun Kim, Thomas Zimmermann, Kai Pan, E. James Whitehead Jr.: Automatic Identification of Bug-
Introducing Changes. ASE 2006: 81-90

Mine your ﬁtness!

Software repositories provide you quite a lot
of data to build a ﬁtness function
e.g., capturing people attitude to perform a
maintenance task, or to work together


Beyond project staffing
• Make project scheduling and staffing that
self-adapt over time
• Developers availability, different
workloads, request for unplanned skills
• Evolutionary models should learn from
project managers’ experience
• Embedded in the fitness functions

Challenge II -
Supporting Program
Comprehension

A keynote from ICPC 2007...
Search Based Software Engineering for Program Comprehension
Mark Harman
CREST,
King’s College London
Strand, London, WC2R 2LS
United Kingdom

Abstract an initial survey of early work on Search Based Test-
ing, Modularisation and Cost Estimation by Clark et
Search Based Software Engineering (SBSE) is an ap-
al. [19]. However, work on search based approaches to
proach to software engineering in which search based
software engineering problems dates back much further,
optimization algorithms are used to identify optimal or
with work on optimization for testing starting as early
near optimal solutions and to yield insight. SBSE tech-
as 1976, when Miller and Spooner [60] used classical op-
niques can cater for multiple, possibly competing objec-
timization techniques for test data generation. In 1992,
tives and/or constraints and applications where the po-
Xanthakis et al. were the first1 to apply meta heuristic
tential solution space is large and complex. Such situa-
search to software engineering, when they used search
tions are common in software engineering, leading to an
based optimization for test data generation [81].
increasing interest in SBSE. This paper provides a brief
Since the 2001 manifesto paper, there has been a
overview of SBSE, explaining some of the ways in which
great deal of activity in this area, with SBSE being
it has already been applied to program–comprehension
increasingly applied to a wide range of diverse areas
related activities. The paper also outlines some possi-
within software engineering. There have also been sev-
ble future applications of and challenges for the further
eral workshops, conferences and special issues on SBSE.
application of SBSE to Program Comprehension.
Search based optimization techniques have proved to
be highly applicable in software engineering. For exam-
1. Introduction ple, in the past five years, Search Based Software En-
Search Based Software Engineering (SBSE) aims to ap- gineering has been applied to requirements engineering
ply search based optimization algorithms to problems [5, 82], project planning and cost estimation [2, 3, 44],
drawn from software engineering. This approach to op- testing [6, 7, 13, 16, 33, 48, 58, 78], automated main-
timization is a natural one, because so many engineer- tenance [14, 26, 32, 61, 62, 66, 71, 72], service-oriented
ing applications are characterised by many complex and software engineering [18], compiler optimization [20] and
competing objectives in large search spaces. In these sit- quality assessment [15, 43]. The application of op-
uations, automated optimization techniques are natural timization techniques to software testing has recently
witnessed intense activity. In 2004 there was sufficient
candidates.
Search based optimization techniques are widely used material to warrant a survey paper on this sub-area of
in other engineering domains, for example mechanical activity [57]. However, as the list of application areas
engineering [46], chemical engineering [12], biomedical above indicates, optimization can be applied right across
engineering [67, 69, 79], civil engineering [4, 10, 27, 42] the spectrum of software engineering activity.
and electronic engineering [9, 22, 63]. Software engineer- This paper accompanies the author’s keynote at the
ing is coming of age as a mature engineering discipline. 15th International Conference on Program Comprehen-
It is, therefore, natural to ask: sion. It explores ways in which SBSE ideas and tech-
niques can be applied to problems in Program Com-
“Why not apply search based optimization to prehension. Following the explosion of work in search
software engineering?” based optimization for software engineering in general,
this paper asks the more specific question:
This question was posed by Harman and Jones in
2001 [34], in a ‘manifesto paper’ that drew attention “Why not apply search based optimization to
to the huge potential of automated search based opti- Program Comprehension?”
mization for software engineering problems and also in 1 So far as the author is aware.

15th IEEE International Conference on Program Comprehension (ICPC'07)
0-7695-2860-0/07 $20.00 © 2007


What did Mark discuss?
• Optimized pretty-printing
• Search-based amorphous program slicing
• Optimizing code and design for program
comprehension
• Self-evolving visualizations
• Co-evolving program comprehension
• Linguistic evolution
• ...


Comprehension-
oriented optimization
• Many problems are similar to those
encountered in modularization and
refactoring
• Different objective: comprehension
• Different persistence of produced artifacts:
may be used for a comprehension task only
• High interaction with humans

What’s the primary
source of information for
program comprehension?


Ideally, high-level
artifacts, models...


At least, you have
source code...


At least, you have
source code...
and source code is text!

Leveraging Natural Language Analysis
of Software: Achievements,
Challenges, and Opportunities

Keynote by Lori Pollock @ICSM 2012


Applications of
textual analysis
• Traceability link recovery
• Feature/concern location
• Code search
• Clone detection
• Building quality models / defect predictors
• Refactoring
• Redocumentation
• ....

How can search-based techniques be useful in this context?

IR process
Stop
Lexical Stemming/ Term Algebraic
Documents words
analysis lemmatization Indexing model
removal

• Various choices for each step of the process
• Each technique requires a careful calibration
• E.g. number of concepts for LSI, number of
topics, α, β for LDA
• Choices are highly dependent on the
corpus and on the problem
• That’s a search problem too!

Identiﬁer splitting
expansion


expansion
OpenFile


expansion
OpenFile Open File
Easy!


expansion
OpenFile Open File
Easy!

prmcntr


expansion
OpenFile Open File
Easy!

pr mcntr
prmcntr


expansion
OpenFile Open File
Easy!

pr mcntr
prmcntr prm cntr


expansion
OpenFile Open File
Easy!

pr mcntr
prmcntr prm cntr
prmc ntr


expansion
OpenFile Open File
Easy!

pr mcntr
prmcntr prm cntr
prmc ntr parameter control

expansion
OpenFile Open File
Easy!

pr mcntr
prmcntr prm cntr
program control

expansion
OpenFile Open File
Easy!

pr mcntr
parameter counter
prmcntr prm cntr
program control

expansion
OpenFile Open File
Easy!

pr mcntr program counter
parameter counter
prmcntr prm cntr
program control

Identifier splitting/expansion
as a search based problem
• Search among possible words in dictionaries
• Context dictionary, English dictionary,
domain terms, known abbreviations/
acronyms
• Techniques: Dynamic Time Warping, Hill
Climbing

L. Guerrouj, P. Galinier,Y.G. Guéhéneuc, G. Antoniol, M. Di Penta - TRIS: a Fast and Accurate Identifiers
Splitting and Expansion Algorithm - To appear in proceedings of WCRE 2012, Oct. 2012
L. Guerrouj, M. Di Penta, G. Antoniol, and Y. G. Guéhéneuc. TIDIER: An identifier splitting approach using
speech recognition techniques, Journal of Software Maintenance - Research and Practice, p. 31, 2011.


Challenge III
Recommending
Changes

Recommendation
overload!

Maintenance-related recommenders provide useful
suggestions to developers
The number of possible improvement for a system
snapshot can be huge
Recommending all of them constitutes just overload
of information for the developer [Murphy 2007]

How SBSE can help?
• Searching for a (near) optimal set of
recommendations
• With respect to what?
• Cost
• Performance
• Code quality...
• Security
• ... conﬂicting objectives, hence it’s a multi-objective
optimization problem


Search-based
modularization/refactoring
Maximizing inter-package/
dependencies
Minimizing intra-package
dependencies
Minimizing CK metrics
...use other metric
proﬁles, e.g. security-
related


Examples of work in
the area
Brian S. Mitchell, Spiros Mancoridis: On the Automatic Modularization of Software
Systems Using the Bunch Tool. IEEE Trans. Software Eng. 32(3): 193-208 (2006)
Kata Praditwong, Mark Harman, Xin Yao: Software Module Clustering as a Multi-
Objective Search Problem. IEEE Trans. Software Eng. 37(2): 264-282 (2011)
Mark Harman, Robert M. Hierons, Mark Proctor: A New Representation And
Crossover Operator For Search-based Optimization Of Software
Modularization. GECCO 2002: 1351-1358
Mark O'Keeffe, Mel Ó Cinnéide: Search-based refactoring: an empirical study.
Journal of Software Maintenance 20(5): 345-364 (2008)
Mark Kent O'Keeffe, Mel Ó Cinnéide: Search-based refactoring for software
maintenance. Journal of Systems and Software 81(4): 502-516 (2008)
Shadi Ghaith, Mel Ó Cinnéide: Improving Software Security Using Search-Based
Refactoring. SSBSE 2012: 121-135


Traditional modularization...
Create initial population

Evaluate individuals

Select individuals to reproduce

Recombination

Mutation

no
stop?

yes

Traditional modularization...



Modularization
Recombination Quality
Metrics

Mutation

no
stop?

yes

Do the obtained
refactoring make sense
for developers?
...also discussed yesterday during
the refactoring session @ICSM

Hybrid interactive GA



Recombination

Mutation

no
stop?

yes





Recombination

Mutation Modularization
Quality
Metrics
no
stop?

yes





Recombination

Mutation Modularization
Human
Quality
Feedback
Metrics
no
stop?

yes


Want to learn more?
Mathew Hall, Neil Walkinshaw and Phil McMinn.
Supervised Software Modularisation. In Proceedings of
ICSM 2012

...and... wait for the next session of SSBSE 2012!

Gabriele Bavota, Filomena Carnevale, Andrea De Lucia,
Massimiliano Di Penta, Rocco Oliveto: Putting the
Developer in-the-Loop: An Interactive GA for Software
Re-modularization. SSBSE 2012: 75-89


Issues in interactive GA
Too many interactions required high effort required

• Learn the ﬁtness function
• Use a mixed interactive and non-interactive, with
periodic feedback
• Achieve a tradeoff between number of feedbacks
(cost) and quality of results (effectiveness)
How to experiment/evaluate it?
• Simulate the human feedback
• Run a user study


Challenge IV
Automatic, Run-time,
Self Changing

Achievements: automatic
program repair
• Use of GP to modify code so that test suite passes
• Not really automatic enacted, but it produces a
patch for a bug automatically
Claire Le Goues, Michael Dewey-Vogt, Stephanie Forrest, Westley Weimer: A systematic
study of automated program repair: Fixing 55 out of 105 bugs for $8 each. ICSE 2012: 3-13

Claire Le Goues, ThanhVu Nguyen, Stephanie Forrest, Westley Weimer: GenProg: A
Generic Method for Automatic Software Repair. IEEE Trans. Software Eng. 38(1): 54-72
(2012)

Andrea Arcuri: Evolutionary repair of faulty software. Appl. Soft Comput. 11(4): 3494-3514
(2011)

Westley Weimer, ThanhVu Nguyen, Claire Le Goues, Stephanie Forrest: Automatically
ﬁnding patches using genetic programming. ICSE 2009: 364-374


Achievements:
QoS-aware binding
Liangzhao Zeng, Boualem Benatallah, Anne H. H. Ngu, Marlon Dumas,
Jayant Kalagnanam, Henry Chang: QoS-Aware Middleware for Web
Services Composition. IEEE Trans. Software Eng. 30(5): 311-327
(2004)
Gerardo Canfora, Massimiliano Di Penta, Raffaele Esposito, Maria
Luisa Villani: A framework for QoS-aware binding and re-binding of
composite web services. Journal of Systems and Software 81(10):
1754-1769 (2008)
Gerardo Canfora, Massimiliano Di Penta, Raffaele Esposito, Maria
Luisa Villani: An approach for QoS-aware service composition based
on genetic algorithms. GECCO 2005: 1069-1075
Gerardo Canfora, Massimiliano Di Penta, Raffaele Esposito, Francesco
Perfetto, Maria Luisa Villani: Service Composition (re)Binding Driven
by Application-Speciﬁc QoS. ICSOC 2006: 141-152


Service automatic binding
Receive
cost=10 cost=15
resp . time=12 resp . time=10 cost=15 cost=15
resp . time=10 resp . time=12
CS1B CS1A getAddress checkFlight
AS1 AS2 CS2A CS2B
cost=30
resp . time=10 AS3

CS3A hotelSearch cost=20 cost=15
resp . time=13 resp . time=30
getHotelInfo CS4A CS4B
AS4

cost=5 cost=15
resp . time=18 getDistance
resp . time=13
CS5A AS7
[enoughTime ] [not enoughTime ] CS7A

AS5 AS6
[not bigDistance ] cost=10
getShuttleTicketPrice getCabPrice [bigDistance ] resp . time=16
AS9 CS9A
AS8
getCarPrice getMetroCardPrice
cost=15 cost=16
resp . time=30 resp . time=20 cost=15
resp . time=4
CS6A CS6B
CS8A
Reply


Estimating the aggregate QoS
QoS Attribute Sequence Switch Flow Loop
P
m P
n
Time (T) T (ti ) pai · T (ti ) M ax{T (ti )i2{1...p} } k · T (t)
i=1 i=1
Pm Pn p
P
Price (P) P (ti ) pai · P (ti ) P (ti ) k · P (t)
i=1 i=1 i=1
Q
m Pn p
Q
Availability (A) A(ti ) pai · A(ti ) A(ti ) A(t)k
i=1 i=1 i=1
Q
m Pn p
Q
Reliability (R) R(ti ) pai · R(ti ) R(ti ) R(t)k
i=1 i=1 i=1
Custom Attr. (F) fS (F (ti )i2{1...m} ) fB ((pai , F (ti ))i2{1...n} ) fF (F (ti )i2{1...p} ) fL (k, F (t))

Estimated cost=2 cost=3
30% 70%
Iterations=5 avail 0.9 avail=0.8
cost=2
r.time=3 r.time=2
avail=0.9 cost=2 cost=3
avail=0.9 avail=0.8

cost=3 cost=2
avail=0.8 avail 0.9

cost=2+3=5 cost=2 · 5=10 cost=0.3 · 2+0.7 · 3=2.7
avail=0.95=0.59 cost=2+3=5
avail=0.9 · 0.8=0.72 avail=0.3 · 0.9+0.7 · 0.8=0.83 avail=0.9 · 0.8=0.72
r.time=max(3,2)=3

Sequence Loop Switch Fork (flow)


Run-Time
Re-binding
40.0000

30.0000
Response Time [s]

20.0000

10.0000

0
initial estimate replan. value new estimate final value

Re-binding introduces
overhead

Beyond dynamic binding...
Beyond automatic bug repair...


“From off-line to continuous
on-line maintenance”

Keynote by Mauro Pezzè @ ICSM 2012


Various contexts...


Run-time replacement
of failing behavior
• Identiﬁcation of sequences of methods that
can replace failing sequences with
equivalent behavior
• This can be done at design time but...
• It can be a search-based problem too!


Open problems...
• Humans are not in the loop, hence the provided
solution must be reliable enough
• Applicable possibly without a human check
• Techniques like GA do not guarantee a
convergence
• For most of the SBSE applications, we did not
care (a lot) about time needed to ﬁnd a solution
• In run-time adaptation we do


Parallelize it!
Metaheuristics such as GA are intrinsically parallel,
however (nearly) all implementation do not exploit such a
parallelism
Multi core and GPU provide great opportunities to
improve the performances of your search-based technique
See tutorial by Simon Poulding tomorrow!


Carefully select the
technique to be used
Recall the trace segmentation problem
Often the most obvious technique is not the best one
Also.. sometimes you don’t even need a search-based
optimization technique!


Example: GA vs.
Dynamic Programming
Task: splitting execution traces in conceptually cohesive segments

GA has a median
convergence time
of 1650 s
DP of 0.93 S

DP decomposes the problem in sub-problems
memorizing cohesion/coupling for different parts of the trace

Conclusions


Taking out


Taking out
Maintenance is human-centric


Taking out

Need to balance conﬂicting goals


Taking out


Conﬁguring analysis techniques crucial to
achieve good performance


Taking out


Conﬁguring analysis techniques crucial to
achieve good performance

Performance and quality of solutions crucial
for run-time applications

SSBSE 2012 Keynote

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (8)

Similar a SSBSE 2012 Keynote

Similar a SSBSE 2012 Keynote (20)

Último

Último (20)

SSBSE 2012 Keynote