Changing source code in large software systems is complex and requires a good understanding of dependencies between software components. Modification to components with little regard to dependencies may have an adverse impact on the quality of the latter, i.e., increase their risk to fail. We conduct an empirical study to understand the relationship between the quality of components and the characteristics of their dependencies such as their frequency of change, their complexity, number of past failures and the like. Our study has been conducted on two large software systems: Microsoft VISTA and ECLIPSE. Our results show that components that have outgoing dependencies to components with higher object-oriented complexity tend to have fewer field failures for VISTA, but the opposite relation holds for ECLIPSE. Likewise, other notable observations have been made through our study that (a) confirm that certain characteristics of components increase the risk of their dependencies to fail and (b) some of the characteristics are project specific while some were also found to be common. We expect that such results can be leveraged for use to provide new directions for research in defect prediction, test prioritization and related research fields that utilize code dependencies in their empirical analysis. Additionally, these results provide insights to engineers on the potential reliability impacts of new component dependencies based upon the characteristics of the component.

1. presented @
Factors Relating Field Failures
and Dependencies
Thomas Zimmermann ● Nachiappan Nagappan ● Kim Herzig
Rahul Premraj ● Laurie Williams

2. The Value Of A House

3. The Value Of A House
Condition matters!

4. The Value Of A House
And so does the neighborhood!

5. What has this to do with
code quality?

6. What has this to do with
code quality?
Consider software projects as a city
with classes as houses

7. What has this to do with
code quality?
Consider software projects as a city
with classes as houses
The value of a class determined
by the number of bugs found!

8. What has this to do with
code quality?
Consider software projects as a city
with classes as houses
The value of a class determined
by the number of bugs found!
How to determine
the condition of a class?

9. The Quality Of Source Code
/**
* Used by {@link #getThreadInformation()} for a traversal search of
* {@link Thread}s/{@link ThreadGroup}s
*
* @param group
* @param level
* @return /**
*/ * Used by {@link #getThreadInformation()} for a traversal search of
private String visit(final ThreadGroup group, * {@link Thread}s/{@link ThreadGroup}s
final int level) { *
// Get threads in `group' /** * @param group
* @param level
StringBuilder builder = new StringBuilder(); #getThreadInformation()} for a traversal search of
* Used by {@link
* @return
int numThreads = group.activeCount(); Thread}s/{@link ThreadGroup}s
* {@link
Thread[] threads = new Thread[numThreads * 2];
* */
numThreads = group.enumerate(threads, group
* @param false); private String visit(final ThreadGroup group,
* @param level final int level) {
StringBuilder indent = new StringBuilder();
* @return // Get threads in `group'
for (int i = 0; i < level; ++i) {
*/ StringBuilder builder = new StringBuilder();
indent.append(" "); private String visit(final ThreadGroup group, group.activeCount();
int numThreads =
} final int level) { Thread[] threads = new Thread[numThreads * 2];
// Get threads in `group' numThreads = group.enumerate(threads, false);
// Enumerate each thread in `group'
StringBuilder builder = new StringBuilder();
for (int i = 0; i < numThreads; i++) { StringBuilder indent = new StringBuilder();
int numThreads = group.activeCount();
// Get thread Thread[] threads = new Thread[numThreads * i2]; level; ++i) {
for (int i = 0; <
Thread thread = threads[i]; numThreads = group.enumerate(threads, false); ");
indent.append("
builder.append(indent); }
builder.append("|-"); StringBuilder indent = new StringBuilder();
builder.append(thread.getName()).append(" ["); level; // Enumerate each thread in `group'
for (int i = 0; i < ++i) {
builder.append(thread.getClass().getSimpleName()).append("], "); = 0; i < numThreads; i++) {
indent.append(" "); for (int i
builder.append(thread.getPriority()).append(", ");
} // Get thread
builder.append(thread.getState().name()); Thread thread = threads[i];
builder.append(indent);
builder.append(FileUtils.lineSeparator); each thread in `group'
// Enumerate
builder.append("|-");
for (StackTraceElement element : thread.getStackTrace()) { i++) {
for (int i = 0; i < numThreads;
builder.append(indent); // Get thread builder.append(thread.getName()).append(" [");
builder.append("| "); Thread thread = threads[i]; builder.append(thread.getClass().getSimpleName()).append("], ");
builder.append(element.toString());
builder.append(indent); builder.append(thread.getPriority()).append(", ");
builder.append(FileUtils.lineSeparator);
builder.append("|-"); builder.append(thread.getState().name());
} builder.append(FileUtils.lineSeparator);
builder.append(thread.getName()).append(" [");
// builder.append(FileUtils.lineSeparator); for (StackTraceElement element : thread.getStackTrace()) {
builder.append(thread.getClass().getSimpleName()).append("], ");
} builder.append(indent);
builder.append(thread.getPriority()).append(", ");
builder.append("| ");
builder.append(thread.getState().name());
// Get thread subgroups of `group' builder.append(element.toString());
builder.append(FileUtils.lineSeparator);
int numGroups = group.activeGroupCount(); builder.append(FileUtils.lineSeparator);
for (StackTraceElement element : thread.getStackTrace()) {
builder.append(indent); }
ThreadGroup[] groups = new ThreadGroup[numGroups * 2];
numGroups = group.enumerate(groups, false);
builder.append("| "); // builder.append(FileUtils.lineSeparator);
}
builder.append(element.toString());
// Recursively visit each subgroup builder.append(FileUtils.lineSeparator);
for (int i = 0; i < numGroups; i++) {
} // Get thread subgroups of `group'
builder.append(indent); int numGroups = group.activeGroupCount();
// builder.append(FileUtils.lineSeparator);
builder.append(visit(groups[i], level + 1));
} ThreadGroup[] groups = new ThreadGroup[numGroups * 2];
} numGroups = group.enumerate(groups, false);
// Get thread subgroups of `group'
return builder.toString(); int numGroups = group.activeGroupCount(); visit each subgroup
// Recursively
} ThreadGroup[] groups = new for (int i = 0; i < numGroups; i++) {
ThreadGroup[numGroups * 2];
builder.append(indent);
numGroups = group.enumerate(groups, false);
builder.append(visit(groups[i], level + 1));
}
// Recursively visit each subgroup
for (int i = 0; i < numGroups; i++) {
builder.append(indent); return builder.toString();
}
builder.append(visit(groups[i], level + 1));
}
return builder.toString();
}
Code Metrics

10. The Quality Of Source Code
/**
* Used by {@link #getThreadInformation()} for a traversal search of
* {@link Thread}s/{@link ThreadGroup}s
*
* @param group
* @param level
* @return /**
*/ * Used by {@link #getThreadInformation()} for a traversal search of
private String visit(final ThreadGroup group, * {@link Thread}s/{@link ThreadGroup}s
final int level) { *
// Get threads in `group' /** * @param group
* @param level
StringBuilder builder = new StringBuilder(); #getThreadInformation()} for a traversal search of
* Used by {@link
* @return
int numThreads = group.activeCount(); Thread}s/{@link ThreadGroup}s
* {@link
Thread[] threads = new Thread[numThreads * 2];
* */
numThreads = group.enumerate(threads, group
* @param false); private String visit(final ThreadGroup group,
* @param level final int level) {
StringBuilder indent = new StringBuilder();
* @return // Get threads in `group'
for (int i = 0; i < level; ++i) {
*/ StringBuilder builder = new StringBuilder();
indent.append(" "); private String visit(final ThreadGroup group, group.activeCount();
int numThreads =
} final int level) { Thread[] threads = new Thread[numThreads * 2];
// Get threads in `group' numThreads = group.enumerate(threads, false);
// Enumerate each thread in `group'
StringBuilder builder = new StringBuilder();
for (int i = 0; i < numThreads; i++) { StringBuilder indent = new StringBuilder();
int numThreads = group.activeCount();
// Get thread Thread[] threads = new Thread[numThreads * i2]; level; ++i) {
for (int i = 0; <
Thread thread = threads[i]; numThreads = group.enumerate(threads, false); ");
indent.append("
builder.append(indent); }
builder.append("|-"); StringBuilder indent = new StringBuilder();
builder.append(thread.getName()).append(" ["); level; // Enumerate each thread in `group'
for (int i = 0; i < ++i) {
builder.append(thread.getClass().getSimpleName()).append("], "); = 0; i < numThreads; i++) {
indent.append(" "); for (int i
builder.append(thread.getPriority()).append(", ");
} // Get thread
builder.append(thread.getState().name()); Thread thread = threads[i];
builder.append(indent);
builder.append(FileUtils.lineSeparator); each thread in `group'
// Enumerate
builder.append("|-");
for (StackTraceElement element : thread.getStackTrace()) { i++) {
for (int i = 0; i < numThreads;
builder.append(indent); // Get thread builder.append(thread.getName()).append(" [");
builder.append("| "); Thread thread = threads[i]; builder.append(thread.getClass().getSimpleName()).append("], ");
builder.append(element.toString());
builder.append(indent); builder.append(thread.getPriority()).append(", ");
builder.append(FileUtils.lineSeparator);
builder.append("|-"); builder.append(thread.getState().name());
} builder.append(FileUtils.lineSeparator);
builder.append(thread.getName()).append(" [");
// builder.append(FileUtils.lineSeparator); for (StackTraceElement element : thread.getStackTrace()) {
builder.append(thread.getClass().getSimpleName()).append("], ");
} builder.append(indent);
builder.append(thread.getPriority()).append(", ");
builder.append("| ");
builder.append(thread.getState().name());
// Get thread subgroups of `group' builder.append(element.toString());
builder.append(FileUtils.lineSeparator);
int numGroups = group.activeGroupCount(); builder.append(FileUtils.lineSeparator);
for (StackTraceElement element : thread.getStackTrace()) {
builder.append(indent); }
ThreadGroup[] groups = new ThreadGroup[numGroups * 2];
numGroups = group.enumerate(groups, false);
builder.append("| "); // builder.append(FileUtils.lineSeparator);
}
builder.append(element.toString());
// Recursively visit each subgroup builder.append(FileUtils.lineSeparator);
for (int i = 0; i < numGroups; i++) {
} // Get thread subgroups of `group'
builder.append(indent); int numGroups = group.activeGroupCount();
// builder.append(FileUtils.lineSeparator);
builder.append(visit(groups[i], level + 1));
} ThreadGroup[] groups = new ThreadGroup[numGroups * 2];
} numGroups = group.enumerate(groups, false);
// Get thread subgroups of `group'
return builder.toString(); int numGroups = group.activeGroupCount(); visit each subgroup
// Recursively
} ThreadGroup[] groups = new for (int i = 0; i < numGroups; i++) {
ThreadGroup[numGroups * 2];
builder.append(indent);
numGroups = group.enumerate(groups, false);
builder.append(visit(groups[i], level + 1));
}
// Recursively visit each subgroup
for (int i = 0; i < numGroups; i++) {
builder.append(indent); return builder.toString();
}
builder.append(visit(groups[i], level + 1));
}
return builder.toString();
}
Code Metrics Neighborhood

11. Does code neighborhood
influence code quality?

12. What We Need
for each software artifact
• Determine the value (quality)
• Determine the neighborhood
• Determine the condition (metrics)

13. Quality As Value
We consider failure-proness as a measure of quality.

14. Quality As Value
We consider failure-proness as a measure of quality.
12 bugs
• determine all post-
release bug 1 bugs
0 bugs
2 bugs

15. Quality As Value
We consider failure-proness as a measure of quality.
12 bugs
• determine all post- FP
release bug 1 bugs
FP
• failure-prone <=> at
0 bugs
least one post- NFP
release bug
2 bugs
FP

16. The Neighborhood
Artifacts communicate with each other
(functions/methods, variables)
B

17. The Neighborhood
Artifacts communicate with each other
(functions/methods, variables)
B
incoming neighborhood

18. The Neighborhood
Artifacts communicate with each other
(functions/methods, variables)
B
incoming neighborhood
outgoing neighborhood

19. The Neighborhood
Artifacts communicate with each other
(functions/methods, variables)
B
incoming neighborhood
outgoing neighborhood
} neighborhood

20. Code-Metrics As Condition
Code metrics are useful to predict bugs.
Churn & Size Metrics
LOC, Churn, Churn Frequency
Code Metrics
FanIn, FanOut
Cyclomatic Complexity
#methods, inheritance, coupling
Code Coverage
Block Coverage, Arc Coverage
People Metrics
Org. level, #authors
and more ...

21. Code-Metrics As Condition
Code metrics are useful to predict bugs.
Churn & Size Metrics
LOC, Churn, Churn Frequency
Code Metrics
FanIn, FanOut B
Cyclomatic Complexity
#methods, inheritance, coupling
Code Coverage
Block Coverage, Arc Coverage
People Metrics
Org. level, #authors
and more ...

22. Code-Metrics As Condition
Code metrics are useful to predict bugs.
Churn & Size Metrics
LOC, Churn, Churn Frequency Compute the
Code Metrics metrics for
FanIn, FanOut B neighbors
Cyclomatic Complexity
#methods, inheritance, coupling
Code Coverage
Block Coverage, Arc Coverage
People Metrics
Org. level, #authors
and more ...

23. Code-Metrics As Condition
Code metrics are useful to predict bugs.
Churn & Size Metrics
LOC, Churn, Churn Frequency Compute the
Code Metrics metrics for
FanIn, FanOut B neighbors
Cyclomatic Complexity
#methods, inheritance, coupling
Code Coverage Aggregate
Block Coverage, Arc Coverage metrics
People Metrics (med,max,total)
Org. level, #authors
and more ...

24. Code-Metrics As Condition
Code metrics are useful to predict bugs.
Churn & Size Metrics
LOC, Churn, Churn Frequency Compute the
Code Metrics metrics for
FanIn, FanOut B neighbors
Cyclomatic Complexity
#methods, inheritance, coupling
Code Coverage Aggregate
Block Coverage, Arc Coverage Assign to B metrics
People Metrics (med,max,total)
Org. level, #authors
and more ...

25. Data Collection
Quality: FP or NFP
for each
component
Neighborhood metrics (aggregated)

26. Data Collection
Quality: FP or NFP
Is there a relation between
quality and neighborhood condition?
for each
component
Neighborhood metrics (aggregated)

27. Experimental Setup
FP
NFP
FP
NFP
FP

28. Experimental Setup
m = churn_median
FP
NFP
FP
NFP
FP

29. Experimental Setup
m = churn_median
FP NFP
FP
NFP
FP
NFP
FP

30. Experimental Setup
m = churn_median
FP NFP
FP 400
NFP 330
FP 450
NFP 405
FP 500

31. Experimental Setup
m = churn_median
FP NFP
FP 400
NFP 330
FP 450
NFP 405
FP 500
Are the values in FP significantly
greater/lower that values in NFP?

32. Projects

33. Projects
functional & object oriented object oriented
metrics on binary level metrics on class level
slightly different set of metrics

34. Projects
functional & object oriented object oriented
metrics on binary level metrics on class level
slightly different set of metrics

35. Projects
functional & object oriented object oriented
metrics on binary level metrics on class level
slightly different set of metrics

36. Survey
We asked 700 Microsoft developers
to share their opinion on:
How do dependency characteristics
influence the risk of failures?

37. Survey
110 responded (15.7%)
We asked 700 Microsoft developers
to share their opinion on:
How do dependency characteristics
influence the risk of failures?

38. Survey
Binary has a dependency with another that ...
11%
6%
83%
churned a lot
No Effect Increases Decreases No Opinion
risk of failure of binary

39. Survey
Binary has a dependency with another that ...
1%
11%
18%
6%
4%
83% 77%
churned a lot has high test
coverage
No Effect Increases Decreases No Opinion
risk of failure of binary

40. Survey
Binary has a dependency with another that ...
1%
11% 2%
18%
6% 6%
1%
4%
77% 91%
83%
churned a lot has high test failed many times in
coverage the past
No Effect Increases Decreases No Opinion
risk of failure of binary

41. Results Vista
indicates medians for
FP are significantly
lower than for NFP
indicates medians for
FP are significantly
higher than for NFP

42. Results Vista
Survey

43. Results Vista
Survey
Developers were right! High
degree of churn increases failure-
proness.

44. Results Vista
Survey
Code coverage does not matter!

45. Results Vista
Survey
Many users requires better
testing!

46. Results Vista
Survey
Depending on bad code does not
harm!

47. Results Eclipse
Survey

48. Results Eclipse
Survey
as expected!

49. Results Eclipse
Survey
depending on
components
with fewer
interfaces
more bugs

50. Results Eclipse
Results between Vista and EclipseSurvey a lot!
differ
depending on
components
with fewer
interfaces
more bugs

51. Contradictions!
Vista Eclipse Survey
churn
complexity
coverage n/a
post-release
failures
Contradiction raises questions regarding the relationship
between dependencies and code quality!