Más contenido relacionado Similar a Safe-Commit Analysis to Facilitate Team Software Development (20) Safe-Commit Analysis to Facilitate Team Software Development1. SAFE-COMMIT Analysis to Facilitate
Team Software Development
Jan Wloka
PROLANGS
Rutgers University
Barbara G. Ryder Frank Tip Xiaoxia Ren
Virginia Tech IBM Research Rutgers University
© Jan Wloka 2009 All Rights reserved.
2. Parallel Work on Shared Code
Problems: Merge conflicts, duplicative work
Revision control systems (RCS) detect
quot;directquot; merge conflicts
Commit policy governs commit process
“Commit early and commit often.”
“Don’t commit compilation or test failures.”
© Jan Wloka 2009 All Rights reserved. 2
3. Parallel Work on Shared Code
Problems: Merge conflicts, duplicative work
Revision control systems (RCS) detect
quot;directquot; merge conflicts
Commit policy governs commit process
“Commit early and commit often.”
“Don’t commit compilation or test failures.”
Problems are still not solved!
RCS offer limited conflict detection/resolution
Policies are not enforced
Policies may be too restrictive
© Jan Wloka 2009 All Rights reserved. 2
4. Enforceable SAFE-COMMIT Policies
Idea: Observe effects on test results to detect unsafe
changes, e.g.:
Changes affect a failing test
Changes affect a worsening test
Changes not tested
© Jan Wloka 2009 All Rights reserved. 3
5. Enforceable SAFE-COMMIT Policies
Idea: Observe effects on test results to detect unsafe
changes, e.g.:
Changes affect a failing test
Changes affect a worsening test
Changes not tested
Policies of varying degrees of strictness (intuition)
Commit changes that affect only passing tests
Commit changes that affect only non-worsening tests
Commit changes that don’t degrade test outcomes
© Jan Wloka 2009 All Rights reserved. 3
6. Change Models
[PASTE’01, OOPSLA’04, ICSE’05, TSE’06, ICSE’09]
Decompose an edit into 19 distinct atomic changes
Reflecting language semantics
Are atomic
Potential change in dispatch behavior
Lookup change: explicit effect on a dispatch entry
Inter-dependences between changes
Syntactic, structural requirements
Cause – effect mapping
© Jan Wloka 2009 All Rights reserved. 4
7. Explicit Representation of Change
class A { public class Tests extends TestCase { Test
int y; public void test1() {
int zip(int x) { A a = new A();
this.y = x; a.bar(3); test
x = zap(x); assertEquals(5, a.zip(3));
test
return x + 2; }
public void test2() { test
}
A a = new A();
int zap(int x) { return 2 * x; } test
int temp = a.bar(2);
int foo(int x) { return 2 * x; }
assertEquals(2, a.foo(temp)); test
int bar(int x) { return x / 2 ; } }
int wiff(int x) { return x; } public void test3() {
int baz(int x) { return x + 1; } B b = new B();
int val() { return 2; } assertEquals(5, b.waff(5));
} }
class B extends A { public void test4() {
int waff(int x) { this.y = x; return x; } B b = new B();
int bla(int x) { return 7 + this.val(); } assertEquals(8, b.bla(5));
int val() { return 1; } }
public void test5() {
}
A a = new B();
assertEquals(a.baz(1), a.val());
}
© Jan Wloka 2009 All Rights reserved. } 5
8. Explicit Representation of Change
Class
class A { A public class Tests extends TestCase { Test
Method void test1() Method
public Method { Method
int y;
foo(int) = new A();
A a bar(int) wiff(int) baz(int)
int zip(int x) {
this.y = x; a.bar(3); test
x = zap(x); assertEquals(5, a.zip(3));
test
return x + 2; }
public void test2() { test
}
A a = new A();
int zap(int x) { return 2 * x; } test
Field tempMethod
int = a.bar(2);
Method Method
int foo(int x) { return 2 * x; }
assertEquals(2, a.foo(temp)); val()
y zip(int) zap() test
int bar(int x) { return x / 2 ; } }
int wiff(int x) { return x; } public void test3() {
int baz(int x) { return x + 1; } Class B b = new B();
int val() { return 2; } B
assertEquals(5, b.waff(5));
} }
class B extends A { Method void test4() Method
public Method {
int waff(int x) { this.y = x; return x; } waff(int) = new B();
B b bla(int) val()
int bla(int x) { return 7 + this.val(); } assertEquals(8, b.bla(5));
int val() { return 1; } }
public void test5() {
}
A a = new B();
assertEquals(a.baz(1), a.val());
}
© Jan Wloka 2009 All Rights reserved. } 5
9. Explicit Representation of Change
Class
class A { A public class Tests extends TestCase { Test
Method void test1() Method
public Method { Method
int y;
foo(int) = new A();
A a bar(int) wiff(int) baz(int)
int zip(int x) {
this.y = x; a.bar(3); AM test
x = zap(x); assertEquals(5, a.zip(3));
test
return x + 2; }
public void test2() { test
}
A a = new A();
int zap(int x) { return 2 * x; } test
Field tempMethod
int = a.bar(2);
Method Method
int foo(int x) { return 2 * x; }
assertEquals(2, a.foo(temp)); val()
y zip(int) zap() test
int bar(int x) { return x / 2 ; } }
AF AM
int wiff(int x) { return x; } public void test3() {
int baz(int x) { return x + 1; } Class B b = new B();
int val() { return 2; } B
assertEquals(5, b.waff(5));
} }
class B extends A { Method void test4() Method
public Method {
int waff(int x) { this.y = x; return x; } waff(int) = new B();
B b bla(int) val()
int bla(int x) { return 7 + this.val(); } assertEquals(8, b.bla(5));
DM
int val() { return 1; } }
public void test5() {
}
A a = new B();
assertEquals(a.baz(1), a.val());
}
© Jan Wloka 2009 All Rights reserved. } 5
10. Explicit Representation of Change
Class
class A { A public class Tests extends TestCase { Test
Method void test1() Method
public Method { Method
int y;
foo(int) = new A();
A a bar(int) wiff(int) baz(int)
int zip(int x) {
CM a.bar(3); CM AM CM test
this.y = x;
CM
assertEquals(5, a.zip(3));
x = zap(x); test
return x + 2; }
public void test2() { test
}
A a = new A();
int zap(int x) { return 2 * x; } test
Field tempMethod
int = a.bar(2);
Method Method
int foo(int x) { return 2 * x; }
assertEquals(2, a.foo(temp)); val()
y zip(int) zap() test
int bar(int x) { return x / 2 ; } }
AF CM AM
int wiff(int x) { return x; } public void test3() CM {
int baz(int x) { return x + 1; } Class B b = new B();
int val() { return 2; } B
assertEquals(5, b.waff(5));
} }
class B extends A { Method void test4() Method
public Method {
int waff(int x) { this.y = x; return x; } waff(int) = new B();
B b bla(int) val()
int bla(int x) { return 7 + this.val(); } CM assertEquals(8, b.bla(5));
CM DM
int val() { return 1; } }
CM
public void test5() {
}
A a = new B();
assertEquals(a.baz(1), a.val());
}
© Jan Wloka 2009 All Rights reserved. } 5
11. Explicit Representation of Change
Class
class A { A public class Tests extends TestCase { Test
Method void test1() Method
public Method { Method
int y;
foo(int) = new A();
A a bar(int) wiff(int) baz(int)
int zip(int x) {
CM a.bar(3); CM AM CM test
this.y = x;
CM
assertEquals(5, a.zip(3));
x = zap(x); test
return x + 2; }
public void test2() { test
}
A a = new A();
int zap(int x) { return 2 * x; } test
Field tempMethod
int = a.bar(2);
Method Method
int foo(int x) { return 2 * x; }
assertEquals(2, a.foo(temp)); val()
y zip(int) zap() test
int bar(int x) { return x / 2 ; } }
AF CM AM
int wiff(int x) { return x; } public void test3() CM {
int baz(int x) { return x + 1; } Class B b = new B();
int val() { return 2; } B
assertEquals(5, b.waff(5));
} }
class B extends A { Method void test4() Method
public Method {
int waff(int x) { this.y = x; return x; } waff(int) = new B();
B b bla(int) val()
int bla(int x) { return 7 + this.val(); } CM assertEquals(8, b.bla(5));
CM DM
int val() { return 1; } }
CM
public void test5() {
}
A a = new B();
assertEquals(a.baz(1), a.val());
}
© Jan Wloka 2009 All Rights reserved. } 5
12. Explicit Representation of Change
Class
class A { A public class Tests extends TestCase { Test
Method void test1() Method
public Method { Method
int y;
foo(int) = new A();
A a bar(int) wiff(int) baz(int)
int zip(int x) {
CM a.bar(3); CM AM CM test
this.y = x;
CM
assertEquals(5, a.zip(3));
x = zap(x); test
} LC
return x + 2;
public void test2() { LC test
}
A a = new A();
int zap(int x) { return 2 * x; } test
Field tempMethod
int = a.bar(2);
Method Method
int foo(int x) { return 2 * x; }
assertEquals(2, a.foo(temp)); val()
y zip(int) zap() test
int bar(int x) { return x / 2 ; } }
AF CM AM LC
int wiff(int x) { return x; } public void test3() CM {
int baz(int x) { return x + 1; } Class B b = new B();
B LC
int val() { return 2; } assertEquals(5, b.waff(5));
LC
} }
class B extends A { Method void test4() Method
public Method {
int waff(int x) { this.y = x; return x; } waff(int) = new B();
B b bla(int) val()
int bla(int x) { return 7 + this.val(); } CM assertEquals(8, b.bla(5));
CM DM
int val() { return 1; } }
CM
public void test5() {
} LC
A a = new B();
assertEquals(a.baz(1), a.val());
}
© Jan Wloka 2009 All Rights reserved. } 5
13. Determining Effects on Program
Behavior
Program Source Code Atomic Change Model Program Behavior
AST :: original
P
I C
M M
AST :: edited
P
I C C
M M M M
© Jan Wloka 2009 All Rights reserved. 6
14. Determining Effects on Program
Behavior
Program Source Code Atomic Change Model Program Behavior
AST :: original
P
I C
C
M M
M M
AST :: edited
P
I C C
M M M M
© Jan Wloka 2009 All Rights reserved. 6
15. Determining Effects on Program
Behavior
Program Source Code Atomic Change Model Program Behavior
AST :: original
P
I C
AC
C
M M
AM AM
M M
AST :: edited
P CM CM
I C C
M M M M
© Jan Wloka 2009 All Rights reserved. 6
16. Determining Effects on Program
Behavior
Program Source Code Atomic Change Model Program Behavior
AST :: original LC
P
LC
I C
AC
C
M M
LC
AM AM
M M
AST :: edited
P CM CM
I C C
M M M M
© Jan Wloka 2009 All Rights reserved. 6
17. Determining Effects on Program
Behavior
Program Source Code Atomic Change Model Program Behavior
AST :: original LC
call graph :: test1
P
LC
I C
AC
C
M M
LC
AM AM call graph :: test2
M M
AST :: edited
P CM CM
I C C
M M M M
© Jan Wloka 2009 All Rights reserved. 6
18. Determining Effects on Program
Behavior
Program Source Code Atomic Change Model Program Behavior
AST :: original LC
call graph :: test1
P
LC
I C
AC
C
M M
LC
AM AM call graph :: test2
M M
AST :: edited
P CM CM
I C C
M M M M
© Jan Wloka 2009 All Rights reserved. 6
19. Determining Effects on Program
Behavior
Program Source Code Atomic Change Model Program Behavior
AST :: original LC
call graph :: test1
P
LC
I C
AC
C
M M
LC
AM AM call graph :: test2
M M
AST :: edited
P CM CM
I C C
M M M M
© Jan Wloka 2009 All Rights reserved. 6
20. Determining Effects on Program
Behavior
Program Source Code Atomic Change Model Program Behavior
AST :: original LC
call graph :: test1
P
LC
I C
AC
C
M M
LC
AM AM call graph :: test2
M M
AST :: edited
P CM CM
I C C
M M M M
© Jan Wloka 2009 All Rights reserved. 6
21. Determining Effects on Program
Behavior
Program Source Code Atomic Change Model Program Behavior
AST :: original LC
call graph :: test1
P
LC
I C
AC
C
M M
LC
AM AM call graph :: test2
M M
AST :: edited
P CM CM
I C C
M M M M
© Jan Wloka 2009 All Rights reserved. 6
22. Determining Effects on Program
Behavior
Program Source Code Atomic Change Model Program Behavior
AST :: original LC
call graph :: test1
P
LC
I C
AC
C
M M
LC
AM AM call graph :: test2
M M
AST :: edited
P CM CM
I C C
M M M M
© Jan Wloka 2009 All Rights reserved. 6
23. Classifications of Unsafe Changes
Definition Change Coverage: A change c is covered by a
test t if
(i) c is exercised by t
(ii) c depends on c′, c′ is covered by t, or
(iii) c′ depends on c , c′ is covered by t.
© Jan Wloka 2009 All Rights reserved. 7
24. Classifications of Unsafe Changes
Definition Change Coverage: A change c is covered by a
test t if
(i) c is exercised by t
(ii) c depends on c′, c′ is covered by t, or Test Outcomes
pass → pass
(iii) c′ depends on c , c′ is covered by t. fail → pass
Classification of test outcomes pass → fail
fail → fail
Passing: {p→p, f→p, ∅→p} ∅ → pass
Not worsening: Passing + {f→f, ∅→f} ∅ → fail
pass → ∅
Worsening: {p→f}
fail → ∅
© Jan Wloka 2009 All Rights reserved. 7
25. Classifications of Unsafe Changes
Definition Change Coverage: A change c is covered by a
test t if
(i) c is exercised by t
(ii) c depends on c′, c′ is covered by t, or Test Outcomes
pass → pass
(iii) c′ depends on c , c′ is covered by t. fail → pass
Classification of test outcomes pass → fail
fail → fail
Passing: {p→p, f→p, ∅→p} ∅ → pass
Not worsening: Passing + {f→f, ∅→f} ∅ → fail
pass → ∅
Worsening: {p→f}
fail → ∅
➾ E.g, c is unsafe, because it’s covered by a worsening test.
© Jan Wloka 2009 All Rights reserved. 7
26. Preserving Test Behavior
public class Tests extends TestCase {
public void test1() {
A a = new A();
a.bar(3);
assertEquals(5, a.zip(3));
}
public void test2() { /* ... */ }
public void test3() { /* ... */ }
public void test4() { /* ... */ }
public void test5() { /* ... */ }
}
{ Test Results
original edited
test1 ! quot;
test2 ! !
test3 ! !
test4 ! quot;
test5 ! !
© Jan Wloka 2009 All Rights reserved. 8
27. Preserving Test Behavior
public class Tests extends TestCase {
public void test1() {
A a = new A();
a.bar(3);
assertEquals(5, a.zip(3)); Tests.test1()
}
<A,A.bar()> <A,A.zip()>
public void test2() { /* ... */ }
public void test3() { /* ... */ }
A.A() A.bar() A.zip() Assert.assertEquals()
public void test4() { /* ... */ }
public void test5() { /* ... */ }
} <A,A.zap()>
{ Test Results A.zap()
original edited
test1 ! quot;
test2 ! !
test3 ! !
test4 ! quot;
test5 ! !
© Jan Wloka 2009 All Rights reserved. 8
28. Preserving Test Behavior
public class Tests extends TestCase {
public void test1() {
A a = new A();
a.bar(3);
assertEquals(5, a.zip(3)); Tests.test1()
}
<A,A.bar()> <A,A.zip()>
public void test2() { /* ... */ }
public void test3() { /* ... */ }
A.A() A.bar() A.zip() Assert.assertEquals()
public void test4() { /* ... */ }
public void test5() { /* ... */ } CM1 CM2
} <A,A.zap()>
{ Test Results A.zap()
CM:B.waff() AF:A.y
original edited CM3
LC1
test1 ! quot;
LC2:<B,A.zap()> AM1:A.zap()
test2 ! !
test3 ! !
test4 ! quot;
test5 ! !
© Jan Wloka 2009 All Rights reserved. 8
29. Preserving Test Behavior
public class Tests extends TestCase {
public void test1() {
A a = new A();
a.bar(3);
assertEquals(5, a.zip(3)); Tests.test1()
}
<A,A.bar()> <A,A.zip()>
public void test2() { /* ... */ }
public void test3() { /* ... */ }
A.A() A.bar() A.zip() Assert.assertEquals()
public void test4() { /* ... */ }
public void test5() { /* ... */ } CM1 CM2
} <A,A.zap()>
{ Test Results A.zap()
CM:B.waff() AF:A.y
original edited CM3
LC1
test1 ! quot;
LC2:<B,A.zap()> AM1:A.zap()
test2 ! !
test3 ! !
test4 ! quot;
test5 ! !
© Jan Wloka 2009 All Rights reserved. 8
30. Preserving Test Behavior
public class Tests extends TestCase {
public void test1() {
A a = new A();
a.bar(3);
assertEquals(5, a.zip(3)); Tests.test1()
}
<A,A.bar()> <A,A.zip()>
public void test2() { /* ... */ }
public void test3() { /* ... */ }
A.A() A.bar() A.zip() Assert.assertEquals()
public void test4() { /* ... */ }
public void test5() { /* ... */ } CM1 CM2
} <A,A.zap()>
{ Test Results A.zap()
CM:B.waff() AF:A.y
original edited CM3
LC1
test1 ! quot;
LC2:<B,A.zap()> AM1:A.zap()
test2 ! !
test3 ! !
test4 ! quot;
test5 ! !
© Jan Wloka 2009 All Rights reserved. 8
31. Preserving Test Behavior
public class Tests extends TestCase {
public void test1() {
A a = new A();
a.bar(3);
assertEquals(5, a.zip(3)); Tests.test1()
}
<A,A.bar()> <A,A.zip()>
public void test2() { /* ... */ }
public void test3() { /* ... */ }
A.A() A.bar() A.zip() Assert.assertEquals()
public void test4() { /* ... */ }
public void test5() { /* ... */ } CM1 CM2
} <A,A.zap()>
{ Test Results A.zap()
CM:B.waff() AF:A.y
original edited CM3
LC1
test1 ! quot;
LC2:<B,A.zap()> AM1:A.zap()
test2 ! !
test3 ! !
test4 ! quot;
{CM1, CM2, CM3, LC1, LC2, AM1}
test5 ! ! may affect a worsening test.
© Jan Wloka 2009 All Rights reserved. 8
32. SAFE-COMMIT Algorithm Sk etch
Input: CHANGES, EXERCISED(t), COVERED(t), INPUTTESTS
Add INPUTTESTS to set PRESERVEDTESTS
Step1: Selection of NONCOMMITTABLE
Find changes in EXERCISED(t)
Find changes in DEPENDENCES(c)
Step2: Selection of new PRESERVEDTESTS
Find tests t : COVERED(t) contains c in NONCOMMITTABLE
COMMITTABLE= CHANGES (NONCOMMITTABLE ∪ UNCOVERED)
Output: COMMITTABLE, PRESERVEDTESTS
© Jan Wloka 2009 All Rights reserved. 9
33. SAFE-COMMIT Example Run
Example: Find all successfully tested changes.
AM:A.zap() LC:<A, A.zap()> CM:A.bar() CM:A.foo()
CM:A.zap test1() test2() test3()
LC:<B, A.zap()> CM:A.zip() AF:A.y CM:B.waff()
DM:B.val() LC:<B, A.val()>
CM:A.wiff()
AM:B.wiff() LC:<A, A.wiff()> CM:B.bla() LC:<[B, B.val()> CM:B.val()
CM:A.baz()
LC:<B, A.wiff()>
test4() test5()
© Jan Wloka 2009 All Rights reserved. 10
34. SAFE-COMMIT Example Run
Example: Find all successfully tested changes.
AM:A.zap() LC:<A, A.zap()> CM:A.bar() CM:A.foo()
CM:A.zap test1() test2() test3()
LC:<B, A.zap()> CM:A.zip() AF:A.y CM:B.waff()
DM:B.val() LC:<B, A.val()>
CM:A.wiff()
AM:B.wiff() LC:<A, A.wiff()> CM:B.bla() LC:<[B, B.val()> CM:B.val()
CM:A.baz()
LC:<B, A.wiff()>
test4() test5()
© Jan Wloka 2009 All Rights reserved. 10
35. SAFE-COMMIT Example Run
Example: Find all successfully tested changes.
AM:A.zap() LC:<A, A.zap()> CM:A.bar() CM:A.foo()
CM:A.zap test1() test2() test3()
LC:<B, A.zap()> CM:A.zip() AF:A.y CM:B.waff()
DM:B.val() LC:<B, A.val()>
CM:A.wiff()
AM:B.wiff() LC:<A, A.wiff()> CM:B.bla() LC:<[B, B.val()> CM:B.val()
CM:A.baz()
LC:<B, A.wiff()>
test4() test5()
© Jan Wloka 2009 All Rights reserved. 10
36. SAFE-COMMIT Example Run
Example: Find all successfully tested changes.
AM:A.zap() LC:<A, A.zap()> CM:A.bar() CM:A.foo()
CM:A.zap test1() test2() test3()
LC:<B, A.zap()> CM:A.zip() AF:A.y CM:B.waff()
DM:B.val() LC:<B, A.val()>
CM:A.wiff()
AM:B.wiff() LC:<A, A.wiff()> CM:B.bla() LC:<[B, B.val()> CM:B.val()
CM:A.baz()
LC:<B, A.wiff()>
test4() test5()
© Jan Wloka 2009 All Rights reserved. 10
37. SAFE-COMMIT Example Run
Example: Find all successfully tested changes.
AM:A.zap() LC:<A, A.zap()> CM:A.bar() CM:A.foo()
CM:A.zap test1() test2() test3()
LC:<B, A.zap()> CM:A.zip() AF:A.y CM:B.waff()
DM:B.val() LC:<B, A.val()>
CM:A.wiff()
AM:B.wiff() LC:<A, A.wiff()> CM:B.bla() LC:<[B, B.val()> CM:B.val()
CM:A.baz()
LC:<B, A.wiff()>
test4() test5()
© Jan Wloka 2009 All Rights reserved. 10
38. SAFE-COMMIT Example Run
Example: Find all successfully tested changes.
AM:A.zap() LC:<A, A.zap()> CM:A.bar() CM:A.foo()
CM:A.zap test1() test2() test3()
LC:<B, A.zap()> CM:A.zip() AF:A.y CM:B.waff()
DM:B.val() LC:<B, A.val()>
CM:A.wiff()
AM:B.wiff() LC:<A, A.wiff()> CM:B.bla() LC:<[B, B.val()> CM:B.val()
CM:A.baz()
LC:<B, A.wiff()>
test4() test5()
© Jan Wloka 2009 All Rights reserved. 10
39. SAFE-COMMIT Example Run
Example: Find all successfully tested changes.
AM:A.zap() LC:<A, A.zap()> CM:A.bar() CM:A.foo()
CM:A.zap test1() test2() test3()
LC:<B, A.zap()> CM:A.zip() AF:A.y CM:B.waff()
DM:B.val() LC:<B, A.val()>
CM:A.wiff()
AM:B.wiff() LC:<A, A.wiff()> CM:B.bla() LC:<[B, B.val()> CM:B.val()
CM:A.baz()
LC:<B, A.wiff()>
test4() test5()
© Jan Wloka 2009 All Rights reserved. 10
40. SAFE-COMMIT Example Run
Example: Find all successfully tested changes.
AM:A.zap() LC:<A, A.zap()> CM:A.bar() CM:A.foo()
CM:A.zap test1() test2() test3()
LC:<B, A.zap()> CM:A.zip() AF:A.y CM:B.waff()
DM:B.val() LC:<B, A.val()>
CM:A.wiff()
AM:B.wiff() LC:<A, A.wiff()> CM:B.bla() LC:<[B, B.val()> CM:B.val()
CM:A.baz()
LC:<B, A.wiff()>
test4() test5()
© Jan Wloka 2009 All Rights reserved. 10
41. SAFE-COMMIT Example Run
Example: Find all successfully tested changes.
AM:A.zap() LC:<A, A.zap()> CM:A.bar() CM:A.foo()
CM:A.zap test1() test2() test3()
LC:<B, A.zap()> CM:A.zip() AF:A.y CM:B.waff()
DM:B.val() LC:<B, A.val()>
CM:A.wiff()
AM:B.wiff() LC:<A, A.wiff()> CM:B.bla() LC:<[B, B.val()> CM:B.val()
CM:A.baz()
LC:<B, A.wiff()>
test4() test5()
© Jan Wloka 2009 All Rights reserved. 10
42. SAFE-COMMIT Example Run
Example: Find all successfully tested changes.
AM:A.zap() LC:<A, A.zap()> CM:A.bar() CM:A.foo()
CM:A.zap test1() test2() test3()
LC:<B, A.zap()> CM:A.zip() AF:A.y CM:B.waff()
DM:B.val() LC:<B, A.val()>
CM:A.wiff()
AM:B.wiff() LC:<A, A.wiff()> CM:B.bla() LC:<[B, B.val()> CM:B.val()
CM:A.baz()
LC:<B, A.wiff()>
test4() test5()
© Jan Wloka 2009 All Rights reserved. 10
43. SAFE-COMMIT Example Run
Example: Find all successfully tested changes.
AM:A.zap() LC:<A, A.zap()> CM:A.bar() CM:A.foo()
CM:A.zap test1() test2() test3()
LC:<B, A.zap()> CM:A.zip() AF:A.y CM:B.waff()
DM:B.val() LC:<B, A.val()>
CM:A.wiff()
AM:B.wiff() LC:<A, A.wiff()> CM:B.bla() LC:<[B, B.val()> CM:B.val()
CM:A.baz()
LC:<B, A.wiff()>
test4() test5()
© Jan Wloka 2009 All Rights reserved. 10
44. SAFE-COMMIT Example Run
Example: Find all successfully tested changes.
AM:A.zap() LC:<A, A.zap()> CM:A.bar() CM:A.foo()
CM:A.zap test1() test2() test3()
LC:<B, A.zap()> CM:A.zip() AF:A.y CM:B.waff()
DM:B.val() LC:<B, A.val()>
CM:A.wiff()
AM:B.wiff() LC:<A, A.wiff()> CM:B.bla() LC:<[B, B.val()> CM:B.val()
CM:A.baz()
LC:<B, A.wiff()>
test4() test5()
© Jan Wloka 2009 All Rights reserved. 10
45. Modeling Commit Policies
RESTRICTIVE: “Covered by passing tests only”
Tight control of commits
E.g., major release is imminent
MODERATE: “Covered by non-worsening tests only”
Tested changes, that don't degrade test outcomes
E.g., for test-first methodologies
PERMISSIVE: “Not covered by a worsening test”
Moderate Policy + untested changes
E.g., for technology prototypes
© Jan Wloka 2009 All Rights reserved. 11
46. Experiments
Goal: Set of committable changes is significant!
Bi-weekly repository snapshots of Daikon
Six pairs with failling tests
Daikon
Version Avg p1 p2 p3 p4 p5 p6
Changes 1751 274 1485 614 302 6050
Covered 1013 5 1225 20 130 185
Restrictive Total % 4.6 6.3 1.5 1.8 1.0 16.6 0.5
Moderate Total % 31.4 57.9 1.5 82.5 3.3 43.0 0.5
Permissive Total % 99.5 100 99.6 100 100 100 97.4
© Jan Wloka 2009 All Rights reserved. 12
47. Experiments
Goal: Set of committable changes is significant!
Bi-weekly repository snapshots of Daikon
Six pairs with failling tests
Daikon
Version Avg p1 p2 p3 p4 p5 p6
Changes 1751 274 1485 614 302 6050
Covered 1013 5 1225 20 130 185
Total % 4.6 6.3 1.5 1.8 1.0 16.6 0.5
Restrictive
Covered % 29.4 11.0 80.0 2.1 30.0 38.5 15.1
Total % 31.4 57.9 1.5 82.5 3.3 43.0 0.5
Moderate
Covered % 82.5 100 80.0 100 100 100 15.1
Permissive Total % 99.5 100 99.6 100 100 100 97.4
© Jan Wloka 2009 All Rights reserved. 12
48. Related Work
Revision Control, Software Merging
No indirect merge conflicts, manual resolution
SAFE-COMMIT semantic detection w/ behavior in tests
Workspace Awareness Tools
Palantír, Hipicat, Jazz detect conflicts or similarities
SAFE-COMMIT prevents premature change releases
Continuous Integration
Automatic builds w/ testing to detect conflicts early
SAFE-COMMIT enables partial but safe commits
© Jan Wloka 2009 All Rights reserved. 13
49. Conclusions and Future Work
SAFE-COMMIT can reduce risk of duplicative work and
merge conflicts:
Compute safely committable changes
Policies of varying strictness using test outcomes
Commit policies can be enforced automatically
Experiments committed significant #changes
Future directions
Change-centric test development/generation
Automated resolution of merge conflicts
© Jan Wloka 2009 All Rights reserved. 14