4. How it works (Ranking-Based) I have calculated most likely location of bug! Give me a failing program. Calculating… Here is your ranked list of statements.
5. How it works (Ranking-Based) I have calculated most likely location of bug! Give me input. But how does a programmer usea ranked list of statements? Calculating… Here is your rankedlist of statements.
6. Conceptual Model 1) 2) 3) 4) … Here is a list of places to check out Ok, I will check out your suggestions one by one.
13. Assumption #1: Perfect bug understanding must also exist when using automated tool. Do you see a bug?
14. Assumption #2 Programmer inspects statements linearly and exhaustively until finding bug. Is this realistic?
15. How can we evaluate the conceptual model? How could we measure the benefit of an automated tool?
16. Conceptual model: What if gave a developer a list of statements to inspect? How would they use the list? Would they be able to see the bug after visiting it? Is ranking important?
17. Benefit: What if we evaluate programmers with and without automated debuggers? > ? We also could observe what works and what doesn’t.
24. Bugs When running the NanoXML program (main is in class Parser1_vw_v1), the following exception is thrown: Exception in thread "main" net.n3.nanoxml.XMLParseException: XML Not Well-Formed at Line 19: Closing tag does not match opening tag: `ns:Bar' != `:Bar' at net.n3.nanoxml.XMLUtil.errorWrongClosingTag(XMLUtil.java:497) at net.n3.nanoxml.StdXMLParser.processElement(StdXMLParser.java:438) at net.n3.nanoxml.StdXMLParser.scanSomeTag(StdXMLParser.java:202) at net.n3.nanoxml.StdXMLParser.processElement(StdXMLParser.java:453) at net.n3.nanoxml.StdXMLParser.scanSomeTag(StdXMLParser.java:202) at net.n3.nanoxml.StdXMLParser.scanData(StdXMLParser.java:159) at net.n3.nanoxml.StdXMLParser.parse(StdXMLParser.java:133) at net.n3.nanoxml.Parser1_vw_v1.main(Parser1_vw_v1.java:50) The input, testvm_22.xml, contains the following input xml document: <Foo a=”test”> <ns:Bar> <Blah x=”1” ns:x=”2”/> </ns:Bar> </Foo> Bug #2: Exception on input xml document.
29. How do developers use a ranked list? Lowperformers did follow list. Survey says searched through statements. 37% of visits jumped avg. 10. Navigation pattern zig-zagged (avg. 10 zigzags)
30. Is perfect bug understanding realistic? Only 1 out of 10 programmers who clicked on bug stopped investigation. The others spent on average ten minutes continuing investigation.
36. Are automated toolsspeeding up debugging? No ✘ = Traditional Automated group Yes! ✔ Experts Experts > Traditional Automated group
37. Observations Developers searched through statements. Developers without tool fixed symptoms (not problem). Developers wanted explanations rather than recommendations.
39. Moving beyond fault space reduction 39 We can keep building better tools. But we can’t keep abstracting away the human.
40. Performing further studies Does different granularity work better for inspection? Documents? Methods? How does different interfaces or visualizations impact technique? Do other automated debugging techniques fare any better? 40
41. How do developers use a ranked list? Is perfect bug understanding realistic? Are Automated Debugging Tools Helpful? Human studies, human studies, human studies!
43. 352 LOC (median 8 programs) 63.5 LOC (median 4 programs) 30 years 30 years of Scientific Progress 1981 2011 43
Notas del editor
Just ask this guy.
Maybe you have some questions…
Maybe you have some questions…
Cons
58,300 papers on automated debugging… 1,170 this year! (google scholar)
Both linear and exhaustive?
Do two things
Intutively, mental model etc
37% of the visits jumped more than one position and, on average, each jump skipped 10 positionsEach participant zigzagged, with an average of 10.3 zigzags,with an overall range between 1 and 36 zigzags.would scan the ranked list to find a statement that might confirm theirhypothesis about the cause of the failure, whereas othertimes they skipped statements that did not appear relevant.
Or are we blind to the bugs in front of us.
Info graphics, N=X
Can we leveragehypothesises
Does perfect bug understanding exist?Are Automated Debugging Tools Faster?
