1. AURA
Wu, Gu´h´neuc,
e e
Antoniol, and Kim
AURA: A Hybrid Approach to Identify
Framework
Evolution Framework Evolution
Introduction
Outline
Wei Wu1 , Yann-Ga¨l Gu´h´neuc1 ,
Examples
Requirements e e e
Previous Work
Giuliano Antoniol2 , and Miryung Kim3
Approach
Background
Algorithm
1 ´
Ptidej Team, DGIGL, Ecole Polytechnique de Montr´al, Canada
e
Evaluation
2 ´
SOCCER Lab, DGIGL, Ecole Polytechnique de Montr´al, Canada
e
Background 3
Indicators
ECED, The University of Texas at Austin, USA
Comparisons
Conclusion wuwei@iro.umontreal.ca,
Bibliography yann-gael.gueheneuc@polymtl.ca,
antoniol@ieee.org,
miryung@ece.utexas.edu

2. AURA
Framework Evolution
Wu, Gu´h´neuc,
e e
Antoniol, and Kim
Framework Compilation Error
Evolution
final Graphics2D g2 = ...;
Introduction
final Rectangle2D dataArea = ...;
Outline
Examples final CategoryPlot plot = ...;
Requirements final ChartRenderingInfo info = ...;
Previous Work
...
Approach final AbstractCategoryItemRenderer renderer =
Background
Algorithm
new BarRenderer().initialise(g2, dataArea, plot, info);
...
Evaluation
Background
Indicators
Comparisons Typical Questions
Conclusion
The developer would wonder:
Bibliography
▶ did the method signature change?
▶ if not, how should she now initialize a renderer:
▶ one equivalent method?
▶ many new methods?
▶ just not the way anymore?
2 / 30

3. AURA
Introduction
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Outline
Framework
Evolution Introduction
Introduction Examples
Outline
Examples
Requirements
Requirements
Previous Work
Previous Work
Approach
Background Approach
Algorithm
Evaluation
Background
Background Algorithm
Indicators
Comparisons
Conclusion
Evaluation
Bibliography Background
Indicators
Comparisons
Conclusion
Bibliography
3 / 30

4. AURA
Introduction
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Examples (1/4)
Framework
Evolution
Introduction
Outline
JHotDraw
Examples
Requirements
package CH.ifa.draw.application;
Previous Work public class DrawApplication ... {
Approach protected JMenu createEditMenu() {
Background ...
Algorithm Version 5.2: menu.add(new CutCommand("Cut", view()),
Evaluation Version 5.2: new MenuShortcut(’x’));
Background Version 5.3: menu.add(new UndoableCommand(
Indicators
Comparisons Version 5.3: new CutCommand("Cut", this)),
Version 5.3: new MenuShortcut(’x’));
Conclusion
...
Bibliography
(1) CutCommand.CutCommand(DrawingView...)
↺ CutCommand.CutCommand(Alignment, DrawingEditor)
and UndoableCommand.UndoableCommand(Command)
4 / 30

5. AURA
Introduction
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Examples (2/4)
Framework
Evolution
Introduction
Outline
JEdit
Examples package org.gjt.sp.jedit;
Requirements
Previous Work public class GUIUtilities {
public static JMenu loadMenu(...) {
Approach
Background
Version 4.1: DirectoryMenu.DirectoryMenu(...);
Algorithm Version 4.1: MarkersMenu.MarkersMenu();
Evaluation Version 4.1: RecentDirectoriesMenu.RecentDirectoriesMenu(
Background Version 4.2: EnhancedMenu.EnhancedMenu(...);
Indicators
Comparisons
...
Conclusion
Bibliography (1) DirectoryMenu.DirectoryMenu(...)
and MarkersMenu.MarkersMenu()
and RecentDirectoriesMenu.RecentDirectoriesMenu(
↺ EnhancedMenu.EnhancedMenu(...)
5 / 30

6. AURA
Introduction
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Examples (3/4)
Framework
Evolution
Introduction
Outline
Examples
Requirements
JFreeChart
Previous Work package org.jfree.data;
Approach public class DefaultBoxAndWhiskerDataset {
Background Version 0.9.11: public static ... createNumberArray(...) {
Algorithm
Version 0.9.11: ...
Evaluation Version 0.9.11: }
Background
Indicators
Version 0.9.11: ...
Comparisons Version 0.9.12: ...
Conclusion
Bibliography (1) DefaultBoxAndWhiskerDataset.createNumberArray(
↺ ∅
6 / 30

7. AURA
Introduction
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Examples (4/4)
Framework
Evolution Eclipse
Introduction public ReplaceEdit[] getModifications(String source) {
Outline
Examples
...
Requirements Version 3.1: return Indents.getChangeIndentEdits(...);
Previous Work
Version 3.3: return IndentManipulation.getChangeIndentEdits(
Approach }
Background Version 3.1: class Indents {
Algorithm
Version 3.3: class IndentManipulation {
Evaluation
void getChangeIndentEdits(...) {
Background
Indicators ...
Comparisons Version 3.1: int length= Indents.computeIndentLength(...);
Conclusion Version 3.3: int length= this.indexOfIndent(...);
Bibliography
...
(1) Indents.getChangeIndentEdits(...)
↺ IndentManipulation.getChangeIndentEdits(...)
(2) Indents.computeIndentLength(...)
↺ IndentManipulation.indexOfIndent(...)
7 / 30

8. AURA
Introduction
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Requirements
Framework
Evolution
Main Requirements
Introduction
Outline
Examples
We want an approach that offers:
Requirements
Previous Work ▶ one-to-many replacements (JHotDraw);
Approach
Background
▶ many-to-one replacements (JEdit);
Algorithm
Evaluation
▶ simply-deleted methods (JFreeChart);
Background
Indicators
▶ cascade replacements (Eclipse).
Comparisons
Conclusion
Bibliography
Additional Requirements
We also want an approach that is:
▶ automatic and does not require framework developers’
involvement (FDI);
▶ system/contex independent ⇒ no thresholds.
8 / 30

9. AURA
Introduction
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Previous Work
Framework
Evolution
Introduction
Outline
Examples
Requirements Features
Previous Work One-to- Many- Simply- No Fully No
Approaches
Approach many to-one deleted FDI Auto- Thres-
Background Rules Rules Rules matic holds
Algorithm
Chow et al. [1] × × ✓ × × ✓
Evaluation
SemDiff [2] ✓ ✓ ✓ ✓ × ×
Background
Indicators Godfrey et al. [3] ✓ ✓ ✓ ✓ × ×
Comparisons CatchUp! [4] × × × × ✓ ✓
Conclusion M. Kim et al. [5] × ✓ ✓ ✓ ✓ ×
Bibliography S. Kim et al. [6] × × × ✓ ✓ ×
Sch¨fer et al. [8]
a × × × ✓ ✓ ×
Diff-CatchUp [9] ✓ ✓ ✓ ✓ × ×
AURA ✓ ✓ ✓ ✓ ✓ ✓
9 / 30

10. AURA
Approach
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Background (1/4)
Framework
Evolution
Introduction
Outline
Examples Inspiration
Requirements
Previous Work AURA uses:
Approach
Background ▶ Call dependency, as in previous work by:
Algorithm
Evaluation
▶ Dagenais and Robillard [2];
Background
Indicators
▶ Godfrey et al. [3];
Comparisons ▶ Sch¨fer et al. [8];
a
Conclusion
▶ Text similarity, as in previous work by:
Bibliography
▶ M. Kim et al. [5];
▶ S. Kim et al. [6];
▶ Xing and Stroulia [9].
10 / 30

11. AURA
Approach
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Background (2/4)
Framework
Evolution Assumptions
Introduction
Outline We only consider method(s) in the same framework,
Examples
Requirements excluding other vendors’ frameworks.
Previous Work
Approach
Background We only use the old and new releases of a framework, not
Algorithm
Evaluation
any of its client programs.
Background
Indicators
Comparisons A target method t can be replaced by:
Conclusion
Bibliography
▶ no method if it has been simply-deleted;
▶ one or more candidate methods, set {c};
A rule is a conjunction of method replacements, while M.
Kim et al.’s change-rules [5] are groups of similar changes.
11 / 30

12. AURA
Approach
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Background (3/4)
Framework
Evolution
Introduction
Underlying Techniques
Outline
Examples
We use transitive call dependency analyses:
Requirements
Previous Work ▶ let A be the number of anchors, i.e., methods calling
Approach
Background
target methods, then the confidence value is:
Algorithm
CV(t, c) = A(t, c)/A(t);
Evaluation
Background ▶ let KR(t) be the key-replacement method of t, i.e., the
Indicators
Comparisons method that is most similar to t from the candidate
Conclusion
replacement methods whose names are equal to t’s or
Bibliography
with CV(t, c) = 100%, then the support value is:
S(t, c) = ∣∣{ m ∣
m ∈ {all the methods in the new release}
∧ m → KR(t) ∧ m → c }∣∣.
12 / 30

13. AURA
Approach
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Background (4/4)
Framework
Evolution
Introduction
Outline
Examples
Requirements Underlying Techniques
Previous Work
Approach We use Lawrie et al.’s technique [7] to tokenize and compare
Background
Algorithm method signatures:
Evaluation
Background
▶ most of return types, declaring classes, method names,
Indicators
Comparisons and parameter lists;
Conclusion ▶ Levenshtein distance;
Bibliography
▶ longest common subsequence (LCS)
13 / 30

14. AURA
Approach
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Algorithm (1/2)
Framework
Evolution
Introduction Steps
Outline
Examples Our algorithm divide in 6 steps (see pages 329–330):
Requirements
Previous Work 1. Global Data Set Generation: generates target method set, anchor
Approach set and candidate replacement method set.
Background
Algorithm 2. Target Methods Classification: divides methods into those called
Evaluation by an anchor from the others.
Background
Indicators 3. Candidate Replacement Method Set Generation: uses call
Comparisons
dependency to build sets.
Conclusion
4. Confidence Value Computation: build one-to-one, many-to-one,
Bibliography
and one-to-many rules.
5. Text Similarity Only Rule Generation: uses text similarity for
methods not called by anchors.
6. Simply-deleted Method Rule Identification: builds list of deleted
methods.
14 / 30

15. AURA
Approach
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Algorithm (2/2)
Framework
Evolution
Introduction
Outline
Examples
Requirements
Previous Work
Approach
Background
Implementation
Algorithm
AURA is implemented in Java, as an Eclipse plug-in,
Evaluation
Background available at http://www.ptidej.net/downloads/
Indicators
Comparisons experiments/icse10b.
Conclusion
Bibliography
15 / 30

16. AURA
Evaluation
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Background
Framework
Evolution
Introduction
Outline
Examples
Requirements
Previous Work
Approach
Background
Hypothesis
Algorithm
AURA will find more relevant change rules than the previous
Evaluation
Background approaches with comparable precision, i.e., it will have a
Indicators
Comparisons better recall than and similar precision to those of the
Conclusion previous approaches.
Bibliography
16 / 30

17. AURA
Evaluation
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Indicators (1/2)
Framework
Evolution
Introduction
Outline
Examples Precision and Recall
Requirements
Previous Work
∩
Approach ∣∣{relevant rules} {retrieved rules}∣∣
Background Precision =
Algorithm ∣∣{retrieved rules}∣∣
Evaluation ∩
Background
∣∣{relevant rules} {retrieved rules}∣∣
Indicators Recall =
Comparisons ∣∣{relevant rules}∣∣
Conclusion
Bibliography
Problem
We cannot know {relevant rules} without an expensive and
error-prone manual analysis.
17 / 30

18. AURA
Evaluation
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Indicators (2/2)
Framework
Evolution
Introduction Δs of Precision (P) and Recall (R)
Outline
Examples We introduce the manually-build set:
Requirements
Previous Work {correct rules}X = {relevant rules}X
∩
Approach {retrieved rules}X
Background
Algorithm
Then:
Evaluation PrecisionA − PrecisionB
Background ΔP(A, B) =
Indicators PrecisionB
Comparisons
∣∣{correct rules}A ∣∣ × ∣∣{retrieved rules}B ∣∣
Conclusion = −1
∣∣{retrieved rules}A ∣∣ × ∣∣{correct rules}B ∣∣
Bibliography
RecallA − RecallB
ΔR(A, B) =
RecallB
∣∣{correct rules}A ∣∣ − ∣∣{correct rules}B ∣∣
=
∣∣{correct rules}B ∣∣
18 / 30

19. AURA
Evaluation
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Comparisons (1/6)
Framework
Evolution
Introduction
Outline
Examples
Requirements
Previous Work
Approach Considered Rules
Background
Algorithm We noticed that large numbers of target methods are
Evaluation deleted in new releases: on average 31.93% of AURA rules
Background
Indicators are simple-deleted rules ⇒ we include simply-deleted rules.
Comparisons
Conclusion
Bibliography
We convert many-to-one rules into as many one-to-one rules.
19 / 30

20. AURA
Evaluation
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Comparisons (2/6)
Framework
Evolution
Introduction
Outline Subject Systems
Examples
Requirements
Previous Work
We reuse systems analysed in previous work to benefit from
Approach the sets {correct rules}X built by their authors.
Background
Algorithm
Systems Releases # Methods
Evaluation
0.9.11 4,751
Background JFreeChart
0.9.12 5,197
Indicators
5.2 1,486
Comparisons JHotDraw
5.3 2,265
Conclusion 4.1 2,773
JEdit
4.2 3,547
Bibliography 1.1 5,973
Struts
1.2.4 6,111
org.eclipse.jdt.core 3.1 35,439
org.eclipse.jdt.ui 3.3 47,237
20 / 30

21. AURA
Evaluation
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Comparisons (3/6)
Framework
Evolution
Introduction
Small/Medium Systems
Outline
Examples We benefit from existing sets {correct rules}X and compare
Requirements
Previous Work precision, recall, and deltas.
Approach
Background Systems Indicators AURA M. Kim et al. [5] ΔR ΔP
Algorithm
JHotDraw # Correct rule 97 81
19.49% -6.69%
Evaluation 5.2-5.3 Precision 92.38% 99.00%
Background
JEdit # Correct rule 356 217
64.29% -13.78%
Indicators 4.1-4.2 Precision 80.18% 93.00%
Comparisons JFreeChart # Correct rule 155 88
75.86% 3.50%
0.9.11-0.9.12 Precision 80.73% 78.00%
Conclusion
Systems Indicators AURA Sch¨fer et al. [8]
a ΔR ΔP
Bibliography JHotDraw # Correct rule 97 88
10.23% 4.98%
5.2-5.3 Precision 92.38% 88.00%
Struts # Correct rule 129 66
95.49% 11.50%
1.1-1.2.4 Precision 96.56% 85.70%
Total Precision of AURA 88.25%
Average ΔR 53.07%
ΔP -0.10%
21 / 30

22. AURA
Evaluation
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Comparisons (4/6)
Framework
Evolution Large System
Introduction ∣∣{retrieved rules}AURA ∣∣ > 4, 500 ⇒ we use the number of
Outline
Examples errors in scope, i.e., the number of compilation errors related
Requirements
Previous Work to method and type replacements:
Approach
Background ▶ to build {correct rules}AURA ;
Algorithm
Evaluation ▶ to reuse {correct rules}SemDiff .
Background
Indicators
Comparisons
Systems AURA SemDiff [2]
Conclusion org.eclipse. # Errors in Scope 4
jdt.debug.ui # Found Rules 4 4
Bibliography 3.1 - 3.3 # Correct Rules 4 4
Mylyn # Errors in Scope 2
0.5-2.0 # Found Rules 2 2
# Correct Rules 1 2
JBossIDE # Errors in Scope 8
1.5-2.0 # Found Rules 8 8
# Correct Rules 8 8
Precision 92.86% ≤ 100.00%
22 / 30

23. AURA
Evaluation
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Comparisons (5/6)
Framework
Evolution
Introduction
Outline
Examples Threats to Validity
Requirements
Previous Work The following threats affect our comparisons:
Approach
Background ▶ construct validity: errors in the algorithm and bias in
Algorithm
Evaluation
the manual validation of AURA rules
Background ⇒ implementation and data available on-line;
Indicators
Comparisons ▶ internal validity: confounding factors that could affect
Conclusion
the results of the study
Bibliography
⇒ exploratory study and systematic comparison of
AURA with the previous approaches.
23 / 30

24. AURA
Evaluation
Wu, Gu´h´neuc,
e e
Antoniol, and Kim Comparisons (6/6)
Framework
Evolution
Threats to Validity
Introduction
Outline The following threats also affect our comparisons:
Examples
Requirements
Previous Work
▶ conclusion validity: relation between the treatment and
Approach the outcome of the study
Background
Algorithm
⇒ unbiased measures and data provided by the authors
Evaluation of previous approaches;
Background
Indicators ▶ reliability validity: possibility of replicating the study
Comparisons
Conclusion
⇒ details in the paper and studied systems and data
Bibliography available on-line;
▶ external validity: possibility to generalize our the results
of the study
⇒ five systems of different size, domains, and evaluated
by previous approaches.
24 / 30

25. AURA
Conclusion (1/4)
Wu, Gu´h´neuc,
e e
Antoniol, and Kim
Framework
Evolution
Introduction
Outline
Examples Strengths
Requirements
Previous Work AURA has the following advantages over previous work:
Approach
Background ▶ combination of call-dependency and text-similarity
Algorithm
Evaluation
analyses;
Background
Indicators
▶ multi-iteration algorithm;
Comparisons
Conclusion
▶ three text similarity measures;
Bibliography ▶ many-to-one, one-to-many, simply-deleted rules;
▶ no threshold.
25 / 30

26. AURA
Conclusion (2/4)
Wu, Gu´h´neuc,
e e
Antoniol, and Kim
Framework
Evolution
Introduction
Outline
Examples
Requirements
Limitations
Previous Work
Still, AURA has the following limitations:
Approach
Background ▶ it cannot detect one-to-many and many-to-one change
Algorithm
Evaluation
rules for target methods called by no other method;
Background
Indicators
▶ it only generates change rules for methods.
Comparisons
Conclusion
▶ it assumes no major changes to the internal
Bibliography implementation of anchors.
26 / 30

27. AURA
Conclusion (3/4)
Wu, Gu´h´neuc,
e e
Antoniol, and Kim
Framework
Evolution
Introduction Future Work
Outline
Examples Consequently, future work includes to:
Requirements
Previous Work ▶ analyze systems in other programming languages;
Approach
Background ▶ develop heuristics that generate change rules for types
Algorithm
Evaluation
and fields using inheritance relations and polymorphism;
Background
Indicators
▶ combine AURA with approaches that use other
Comparisons
matching techniques;
Conclusion
Bibliography
▶ present AURA results in first-order relational logic rules,
as introduced by M. Kim et al. [5];
▶ perform usability studies of the efficacy of AURA.
27 / 30

28. AURA
Conclusion (4/4)
Wu, Gu´h´neuc,
e e
Antoniol, and Kim
Framework
Evolution
Introduction
Outline
Examples
Requirements
Previous Work
Approach
Background
Algorithm
Evaluation
Any questions, comments?
Background
Indicators
Comparisons
Conclusion
Bibliography
28 / 30

29. AURA
Bibliography (1/2)
Wu, Gu´h´neuc,
e e
Antoniol, and Kim
Framework K. Chow and D. Notkin.
Evolution Semi-automatic update of applications in response to library changes.
In ICSM ’96: Proceedings of the 1996 International Conference on Software Maintenance, page
Introduction 359, Washington, DC, USA, 1996. IEEE Computer Society.
Outline
Examples B. Dagenais and M. P. Robillard.
Requirements Recommending adaptive changes for framework evolution.
Previous Work In ICSE ’08: Proceedings of the 30th international conference on Software engineering, pages
Approach 481–490, New York, NY, USA, 2008. ACM.
Background M. W. Godfrey and L. Zou.
Algorithm
Using origin analysis to detect merging and splitting of source code entities.
Evaluation IEEE Trans. Softw. Eng., 31(2):166–181, 2005.
Background
Indicators J. Henkel and A. Diwan.
Comparisons Catchup!: capturing and replaying refactorings to support api evolution.
In ICSE ’05: Proceedings of the 27th international conference on Software engineering, pages
Conclusion 274–283, New York, NY, USA, 2005. ACM.
Bibliography M. Kim, D. Notkin, and D. Grossman.
Automatic inference of structural changes for matching across program versions.
In ICSE ’07: Proceedings of the 29th international conference on Software Engineering, pages
333–343, Washington, DC, USA, Not Available 2007. IEEE Computer Society.
S. Kim, K. Pan, and E. J. Whitehead, Jr.
When functions change their names: Automatic detection of origin relationships.
In WCRE ’05: Proceedings of the 12th Working Conference on Reverse Engineering, pages
143–152, Washington, DC, USA, 2005. IEEE Computer Society.
29 / 30

30. AURA
Bibliography (2/2)
Wu, Gu´h´neuc,
e e
Antoniol, and Kim
Framework
Evolution
Introduction
Outline
Examples
Requirements D. Lawrie, H. Feild, and D. Binkley.
Previous Work Syntactic identifier conciseness and consistency.
In Sixth IEEE International Workshop on Source Code Analysis and Manipulation., pages
Approach
139–148, Sept. 2006.
Background
Algorithm T. Sch¨fer, J. Jonas, and M. Mezini.
a
Evaluation Mining framework usage changes from instantiation code.
Background In ICSE ’08: Proceedings of the 30th international conference on Software engineering, pages
Indicators 471–480, New York, NY, USA, May 2008. ACM.
Comparisons
Z. Xing and E. Stroulia.
Conclusion API-evolution support with diff-CatchUp.
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 33(12):818 – 836, December 2007.
Bibliography
30 / 30