8. !8
Issue XWIKI-13031not always available
• Not always easy to find the reproduction
scenario
• Manual
• Labor intensive
• Requires Accurate information from the user
10. !10
Exception:
at x(…)
at y(…)
at e(…)
Stack trace
Search-based
algorithm
a() e()d()
Exception:
at x(…)
at y(…)
at e(…)
Crash
reproducing
test case
Our Solution
11. Java Stack Trace (Issue XWIKI-13031)
!11
java.lang.ClassCastException: […]
at org…..SolrEntityReferenceResolver.getWikiReference(....java:93)
at org…..SolrEntityReferenceResolver.getEntityReference(….java:70)
at org…..SolrEntityReferenceResolver.resolve(….java:63)
at org…..SolrDocumentReferenceResolver.resolve(….java:48)
at …
Exception
Frames
{
12. Java Stack Trace (Issue XWIKI-13031)
!12
java.lang.ClassCastException: […]
at org…..SolrEntityReferenceResolver.getWikiReference(....java:93)
at org…..SolrEntityReferenceResolver.getEntityReference(….java:70)
at org…..SolrEntityReferenceResolver.resolve(….java:63)
at org…..SolrDocumentReferenceResolver.resolve(….java:48)
at …
Exception
Frames
{Target→
13. Java Stack Trace (Issue XWIKI-13031)
!13
java.lang.ClassCastException: […]
at org…..SolrEntityReferenceResolver.getWikiReference(....java:93)
at org…..SolrEntityReferenceResolver.getEntityReference(….java:70)
at org…..SolrEntityReferenceResolver.resolve(….java:63)
at org…..SolrDocumentReferenceResolver.resolve(….java:48)
at …
Exception
Frames
{Target→
14. Java Stack Trace (Issue XWIKI-13031)
!14
java.lang.ClassCastException: […]
at org…..SolrEntityReferenceResolver.getWikiReference(....java:93)
at org…..SolrEntityReferenceResolver.getEntityReference(….java:70)
at org…..SolrEntityReferenceResolver.resolve(….java:63)
at org…..SolrDocumentReferenceResolver.resolve(….java:48)
at …
Exception
Frames
{Target→
15. !15
a() b()
c() e()d()
e()
c() e()
a() b()
c() e()d()
e()
c() e()
a() b()
c() e()d()
e()
c() e()
a() b()
c() e()d()
e()
c() e()
Random initial
test suite
a() b()
c() e()d()
e()
c() e()
a() b()
c() e()d()
e()
c() e()
a()
c() e()d()
e()
c() b() e()
a()
c() e()d()
e()
c() b() e()
Evolutionary
search
a() e()d()
Exception:
at x(…)
at y(…)
at e(…)
Exception:
at x(…)
at y(…)
at e(…)
Crash
reproducing
test case
Stack trace
Soltani, M., Panichella, A. and van Deursen, A. 2018. Search-Based Crash Reproduction and Its Impact on
Debugging. Software Engineering, IEEE Transactions on. (2018).
23. Crash-reproducing Test Case
!23
public void test0() throws Throwable {
…
SolrEntityReferenceResolver solrEntityReferenceResolver0 = new …();
EntityReferenceResolver entityReferenceResolver0 = … mock(…);
solrDocument0.put("wiki", (Object) entityType0);
Injector.inject(solrEntityReferenceResolver0, …);
Injector.validateBean(solrEntityReferenceResolver0, …);
…
// Undeclared exception!
solrEntityReferenceResolver0.resolve(solrDocument0, entityType0, objectArray0);
}
24. Crash-reproducing Test Case
!24
public void test0() throws Throwable {
…
SolrEntityReferenceResolver solrEntityReferenceResolver0 = new …();
EntityReferenceResolver entityReferenceResolver0 = … mock(…);
solrDocument0.put("wiki", (Object) entityType0);
Injector.inject(solrEntityReferenceResolver0, …);
Injector.validateBean(solrEntityReferenceResolver0, …);
…
// Undeclared exception!
solrEntityReferenceResolver0.resolve(solrDocument0, entityType0, objectArray0);
}
java.lang.ClassCastException: […]
at org…..SolrEntityReferenceResolver.getWikiReference(....java:93)
at org…..SolrEntityReferenceResolver.getEntityReference(….java:70)
at org…..SolrEntityReferenceResolver.resolve(….java:63)
25. JCrashPack
• 200 crashes from various open source projects
• XWiki (STAMP partner)
• From XWiki issue tracking system: 51 crashes
• Defects4J applications
• State of the art fault localization benchmark
• 73 crashes (with fixes)
• Elasticsearch
• Based on popularity
• From Elasticsearch issue tracking system: 76 crashes
• Filtered, verified, cleaned up, right jar versions, …
!25
26. 12 Key Challenges
• Input data generation
• For complex inputs, generic types, etc.
• Environmental dependencies
• Environment state hard to manage at unit level
• Complex code
• Long methods, with lot of nested predicates
• Abstract classes and methods
• Cannot be instantiated and one concrete implementation is
picked randomly
• […]
!26