Enviar búsqueda
Cargar
Characterizing and Predicting Which Bugs Get Reopened
•
1 recomendación
•
2,002 vistas
Thomas Zimmermann
Seguir
Presented at ICSE 2012 in Zurich.
Leer menos
Leer más
Tecnología
Empresariales
Denunciar
Compartir
Denunciar
Compartir
1 de 32
Recomendados
Predicting Method Crashes with Bytecode Operations
Predicting Method Crashes with Bytecode Operations
Thomas Zimmermann
Analytics for smarter software development
Analytics for smarter software development
Thomas Zimmermann
Design for Testability: A Tutorial for Devs and Testers
Design for Testability: A Tutorial for Devs and Testers
TechWell
Design for Testability: A Tutorial for Devs and Testers
Design for Testability: A Tutorial for Devs and Testers
TechWell
Web Application Remediation - OWASP San Antonio March 2007
Web Application Remediation - OWASP San Antonio March 2007
Denim Group
Peter Zimmerer - Evolve Design For Testability To The Next Level - EuroSTAR 2012
Peter Zimmerer - Evolve Design For Testability To The Next Level - EuroSTAR 2012
TEST Huddle
Software Design for Testability
Software Design for Testability
amr0mt
Defect effort prediction models in software
Defect effort prediction models in software
IAEME Publication
Recomendados
Predicting Method Crashes with Bytecode Operations
Predicting Method Crashes with Bytecode Operations
Thomas Zimmermann
Analytics for smarter software development
Analytics for smarter software development
Thomas Zimmermann
Design for Testability: A Tutorial for Devs and Testers
Design for Testability: A Tutorial for Devs and Testers
TechWell
Design for Testability: A Tutorial for Devs and Testers
Design for Testability: A Tutorial for Devs and Testers
TechWell
Web Application Remediation - OWASP San Antonio March 2007
Web Application Remediation - OWASP San Antonio March 2007
Denim Group
Peter Zimmerer - Evolve Design For Testability To The Next Level - EuroSTAR 2012
Peter Zimmerer - Evolve Design For Testability To The Next Level - EuroSTAR 2012
TEST Huddle
Software Design for Testability
Software Design for Testability
amr0mt
Defect effort prediction models in software
Defect effort prediction models in software
IAEME Publication
Software testability slide share
Software testability slide share
BeBo Technology
AV-Comparatives Performance Test
AV-Comparatives Performance Test
Herbert Rodriguez
SW Engineering Management
SW Engineering Management
Robert Sayegh
Dtl 2012 kl-app_ctl1.2
Dtl 2012 kl-app_ctl1.2
Комсс Файквэе
Protecting Enterprise - An examination of bugs, major vulnerabilities and exp...
Protecting Enterprise - An examination of bugs, major vulnerabilities and exp...
ESET Middle East
Bill of-rights-white-paper-final-012312
Bill of-rights-white-paper-final-012312
Cristiano Caetano
Alm 4 Azure with screenshots
Alm 4 Azure with screenshots
Clemens Reijnen
Online exa-syste
Online exa-syste
Anand Bavarava
Soffront Defect tracking
Soffront Defect tracking
Soffront Software
Automatic Proactive Troubleshooting with IBM Rational Build Forge
Automatic Proactive Troubleshooting with IBM Rational Build Forge
Bill Duncan
Cross-project defect prediction
Cross-project defect prediction
Thomas Zimmermann
Magenic-White-Paper-Continuous-Inegration-for-QA-Teams
Magenic-White-Paper-Continuous-Inegration-for-QA-Teams
Aaron Humerickhouse
RSA 2015 Blending the Automated and the Manual: Making Application Vulnerabil...
RSA 2015 Blending the Automated and the Manual: Making Application Vulnerabil...
Denim Group
Software Testing Basics
Software Testing Basics
sachinmistry786
Open Source tools in Continuous Integration environment (case study for agil...
Open Source tools in Continuous Integration environment (case study for agil...
suwalki24.pl
San Jose Selenium Meetup 22 Mar 2012: The Restless Are Getting Native
San Jose Selenium Meetup 22 Mar 2012: The Restless Are Getting Native
Dante Briones
Real Cost of Software Remediation
Real Cost of Software Remediation
Denim Group
Attacking antivirus
Attacking antivirus
UltraUploader
Quality Assurance 1: Why Quality Matters
Quality Assurance 1: Why Quality Matters
Marc Miquel
Not my bug! Reasons for software bug report reassignments
Not my bug! Reasons for software bug report reassignments
Thomas Zimmermann
Characterizing and predicting which bugs get fixed
Characterizing and predicting which bugs get fixed
Thomas Zimmermann
Bug Advocacy
Bug Advocacy
nazeer pasha
Más contenido relacionado
La actualidad más candente
Software testability slide share
Software testability slide share
BeBo Technology
AV-Comparatives Performance Test
AV-Comparatives Performance Test
Herbert Rodriguez
SW Engineering Management
SW Engineering Management
Robert Sayegh
Dtl 2012 kl-app_ctl1.2
Dtl 2012 kl-app_ctl1.2
Комсс Файквэе
Protecting Enterprise - An examination of bugs, major vulnerabilities and exp...
Protecting Enterprise - An examination of bugs, major vulnerabilities and exp...
ESET Middle East
Bill of-rights-white-paper-final-012312
Bill of-rights-white-paper-final-012312
Cristiano Caetano
Alm 4 Azure with screenshots
Alm 4 Azure with screenshots
Clemens Reijnen
Online exa-syste
Online exa-syste
Anand Bavarava
Soffront Defect tracking
Soffront Defect tracking
Soffront Software
Automatic Proactive Troubleshooting with IBM Rational Build Forge
Automatic Proactive Troubleshooting with IBM Rational Build Forge
Bill Duncan
Cross-project defect prediction
Cross-project defect prediction
Thomas Zimmermann
Magenic-White-Paper-Continuous-Inegration-for-QA-Teams
Magenic-White-Paper-Continuous-Inegration-for-QA-Teams
Aaron Humerickhouse
RSA 2015 Blending the Automated and the Manual: Making Application Vulnerabil...
RSA 2015 Blending the Automated and the Manual: Making Application Vulnerabil...
Denim Group
Software Testing Basics
Software Testing Basics
sachinmistry786
Open Source tools in Continuous Integration environment (case study for agil...
Open Source tools in Continuous Integration environment (case study for agil...
suwalki24.pl
San Jose Selenium Meetup 22 Mar 2012: The Restless Are Getting Native
San Jose Selenium Meetup 22 Mar 2012: The Restless Are Getting Native
Dante Briones
Real Cost of Software Remediation
Real Cost of Software Remediation
Denim Group
Attacking antivirus
Attacking antivirus
UltraUploader
Quality Assurance 1: Why Quality Matters
Quality Assurance 1: Why Quality Matters
Marc Miquel
La actualidad más candente
(19)
Software testability slide share
Software testability slide share
AV-Comparatives Performance Test
AV-Comparatives Performance Test
SW Engineering Management
SW Engineering Management
Dtl 2012 kl-app_ctl1.2
Dtl 2012 kl-app_ctl1.2
Protecting Enterprise - An examination of bugs, major vulnerabilities and exp...
Protecting Enterprise - An examination of bugs, major vulnerabilities and exp...
Bill of-rights-white-paper-final-012312
Bill of-rights-white-paper-final-012312
Alm 4 Azure with screenshots
Alm 4 Azure with screenshots
Online exa-syste
Online exa-syste
Soffront Defect tracking
Soffront Defect tracking
Automatic Proactive Troubleshooting with IBM Rational Build Forge
Automatic Proactive Troubleshooting with IBM Rational Build Forge
Cross-project defect prediction
Cross-project defect prediction
Magenic-White-Paper-Continuous-Inegration-for-QA-Teams
Magenic-White-Paper-Continuous-Inegration-for-QA-Teams
RSA 2015 Blending the Automated and the Manual: Making Application Vulnerabil...
RSA 2015 Blending the Automated and the Manual: Making Application Vulnerabil...
Software Testing Basics
Software Testing Basics
Open Source tools in Continuous Integration environment (case study for agil...
Open Source tools in Continuous Integration environment (case study for agil...
San Jose Selenium Meetup 22 Mar 2012: The Restless Are Getting Native
San Jose Selenium Meetup 22 Mar 2012: The Restless Are Getting Native
Real Cost of Software Remediation
Real Cost of Software Remediation
Attacking antivirus
Attacking antivirus
Quality Assurance 1: Why Quality Matters
Quality Assurance 1: Why Quality Matters
Similar a Characterizing and Predicting Which Bugs Get Reopened
Not my bug! Reasons for software bug report reassignments
Not my bug! Reasons for software bug report reassignments
Thomas Zimmermann
Characterizing and predicting which bugs get fixed
Characterizing and predicting which bugs get fixed
Thomas Zimmermann
Bug Advocacy
Bug Advocacy
nazeer pasha
Bug best practice
Bug best practice
gaoliang641
Classic Testing Mistakes 0226
Classic Testing Mistakes 0226
MBA_Community
Improving developer tester collaboration with microsoft visual studio 2010
Improving developer tester collaboration with microsoft visual studio 2010
Mohamed Samy
10 Reasons Why You Fix Bugs As Soon As You Find Them
10 Reasons Why You Fix Bugs As Soon As You Find Them
Rosie Sherry
DevOps - Continuous Integration, Continuous Delivery - let's talk
DevOps - Continuous Integration, Continuous Delivery - let's talk
D Z
bug-advocacy
bug-advocacy
KALYAN Chakravarthy
5-Ways-to-Revolutionize-Your-Software-Testing
5-Ways-to-Revolutionize-Your-Software-Testing
Mary Clemons
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
michael.labriola
Debugging
Debugging
Jonathan Holloway
Continuous integration
Continuous integration
Boris Dominic
How to Reproduce a Non Reproducible Bug
How to Reproduce a Non Reproducible Bug
BugRaptors
midterm_fa08.pdf
midterm_fa08.pdf
RavinderKSingla
Quality of Bug Reports in Open Source
Quality of Bug Reports in Open Source
Thomas Zimmermann
Software engineering 22 error detection and debugging
Software engineering 22 error detection and debugging
Vaibhav Khanna
179 black-box-software-testing-copyright-2003-cem-kaner1652
179 black-box-software-testing-copyright-2003-cem-kaner1652
ngothanhtungth
Agile testing overview
Agile testing overview
raianup
Agile testingoverview
Agile testingoverview
Rangabashyam S
Similar a Characterizing and Predicting Which Bugs Get Reopened
(20)
Not my bug! Reasons for software bug report reassignments
Not my bug! Reasons for software bug report reassignments
Characterizing and predicting which bugs get fixed
Characterizing and predicting which bugs get fixed
Bug Advocacy
Bug Advocacy
Bug best practice
Bug best practice
Classic Testing Mistakes 0226
Classic Testing Mistakes 0226
Improving developer tester collaboration with microsoft visual studio 2010
Improving developer tester collaboration with microsoft visual studio 2010
10 Reasons Why You Fix Bugs As Soon As You Find Them
10 Reasons Why You Fix Bugs As Soon As You Find Them
DevOps - Continuous Integration, Continuous Delivery - let's talk
DevOps - Continuous Integration, Continuous Delivery - let's talk
bug-advocacy
bug-advocacy
5-Ways-to-Revolutionize-Your-Software-Testing
5-Ways-to-Revolutionize-Your-Software-Testing
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
Debugging
Debugging
Continuous integration
Continuous integration
How to Reproduce a Non Reproducible Bug
How to Reproduce a Non Reproducible Bug
midterm_fa08.pdf
midterm_fa08.pdf
Quality of Bug Reports in Open Source
Quality of Bug Reports in Open Source
Software engineering 22 error detection and debugging
Software engineering 22 error detection and debugging
179 black-box-software-testing-copyright-2003-cem-kaner1652
179 black-box-software-testing-copyright-2003-cem-kaner1652
Agile testing overview
Agile testing overview
Agile testingoverview
Agile testingoverview
Más de Thomas Zimmermann
Software Analytics = Sharing Information
Software Analytics = Sharing Information
Thomas Zimmermann
MSR 2013 Preview
MSR 2013 Preview
Thomas Zimmermann
Klingon Countdown Timer
Klingon Countdown Timer
Thomas Zimmermann
Data driven games user research
Data driven games user research
Thomas Zimmermann
Empirical Software Engineering at Microsoft Research
Empirical Software Engineering at Microsoft Research
Thomas Zimmermann
Security trend analysis with CVE topic models
Security trend analysis with CVE topic models
Thomas Zimmermann
Analytics for software development
Analytics for software development
Thomas Zimmermann
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Thomas Zimmermann
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Thomas Zimmermann
Predicting Defects using Network Analysis on Dependency Graphs
Predicting Defects using Network Analysis on Dependency Graphs
Thomas Zimmermann
Meet Tom and his Fish
Meet Tom and his Fish
Thomas Zimmermann
Predicting Subsystem Defects using Dependency Graph Complexities
Predicting Subsystem Defects using Dependency Graph Complexities
Thomas Zimmermann
Got Myth? Myths in Software Engineering
Got Myth? Myths in Software Engineering
Thomas Zimmermann
Mining Workspace Updates in CVS
Mining Workspace Updates in CVS
Thomas Zimmermann
Mining Software Archives to Support Software Development
Mining Software Archives to Support Software Development
Thomas Zimmermann
Unit testing with JUnit
Unit testing with JUnit
Thomas Zimmermann
esolang: Esoterische Programmiersprachen
esolang: Esoterische Programmiersprachen
Thomas Zimmermann
TA-RE: An Exchange Language for Mining Software Repositories
TA-RE: An Exchange Language for Mining Software Repositories
Thomas Zimmermann
Fine-grained Processing of CVS Archives with APFEL
Fine-grained Processing of CVS Archives with APFEL
Thomas Zimmermann
DynaMine: Finding Common Error Patterns by Mining Software Revision Histories
DynaMine: Finding Common Error Patterns by Mining Software Revision Histories
Thomas Zimmermann
Más de Thomas Zimmermann
(20)
Software Analytics = Sharing Information
Software Analytics = Sharing Information
MSR 2013 Preview
MSR 2013 Preview
Klingon Countdown Timer
Klingon Countdown Timer
Data driven games user research
Data driven games user research
Empirical Software Engineering at Microsoft Research
Empirical Software Engineering at Microsoft Research
Security trend analysis with CVE topic models
Security trend analysis with CVE topic models
Analytics for software development
Analytics for software development
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Predicting Defects using Network Analysis on Dependency Graphs
Predicting Defects using Network Analysis on Dependency Graphs
Meet Tom and his Fish
Meet Tom and his Fish
Predicting Subsystem Defects using Dependency Graph Complexities
Predicting Subsystem Defects using Dependency Graph Complexities
Got Myth? Myths in Software Engineering
Got Myth? Myths in Software Engineering
Mining Workspace Updates in CVS
Mining Workspace Updates in CVS
Mining Software Archives to Support Software Development
Mining Software Archives to Support Software Development
Unit testing with JUnit
Unit testing with JUnit
esolang: Esoterische Programmiersprachen
esolang: Esoterische Programmiersprachen
TA-RE: An Exchange Language for Mining Software Repositories
TA-RE: An Exchange Language for Mining Software Repositories
Fine-grained Processing of CVS Archives with APFEL
Fine-grained Processing of CVS Archives with APFEL
DynaMine: Finding Common Error Patterns by Mining Software Revision Histories
DynaMine: Finding Common Error Patterns by Mining Software Revision Histories
Último
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
Antenna Manufacturer Coco
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Miguel Araújo
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘
RTylerCroy
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
Pixlogix Infotech
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
The Digital Insurer
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
wesley chun
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
sudhanshuwaghmare1
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
The Digital Insurer
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
apidays
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Rafal Los
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
debabhi2
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
UK Journal
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
wesley chun
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
The Digital Insurer
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
Delhi Call girls
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
naman860154
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
Gabriella Davis
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
Último
(20)
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Characterizing and Predicting Which Bugs Get Reopened
1.
Characterizing and Predicting
Which Bugs Get Reopened Thomas Zimmermann Nachiappan Nagappan Microsoft Research Philip J. Guo Stanford University Brendan Murphy Microsoft Research © Microsoft Corporation
2.
A bug’s life Picture
on the right via http://www.bugzilla.org/docs/2.18/html/lifecycle.html © Microsoft Corporation
3.
Final part of
a trilogy… Which bugs are fixed? Bug reassignments Bug reopens (this paper) ICSE 2010 CSCW 2011 ICSE 2012 SEIP © Microsoft Corporation
4.
…and partly a
remake Emad Shihab, Akinori Ihara, Yasutaka Kamei, Walid M. Ibrahim, Masao Ohira, Bram Adams, Ahmed E. Hassan, Ken-ichi Matsumoto: Predicting Re-opened Bugs: A Case Study on the Eclipse Project. WCRE 2010: 249-258 © Microsoft Corporation
5.
Shihab et. al:
Predicting Reopened Bugs This paper: Characterizing Reopened Bugs (WCRE 2010) Four dimensions: work habits, bug report, bug fix and team Predicted reopened bugs with a precision of 62.9% and a recall of 84.5% (d-trees) Top node analysis found that bug report dimension was most influential | © Microsoft Corporation
6.
Shihab et. al:
Predicting Reopened Bugs This paper: Characterizing Reopened Bugs (WCRE 2010) Four dimensions: work habits, bug report, Partial replication of Shihab et al. bug fix and team New measurements: organizational and Predicted reopened bugs with a precision geographic distance, reputation, how found of 62.9% and a recall of 84.5% (d-trees) Qualitative component on the causes of Top node analysis found that bug report bug reopens (identified with a survey) dimension was most influential Descriptive models (logistic regression) | | © Microsoft Corporation
7.
Methodology
Qualitative • “In your experience, what are reasons why a bug would be reopened multiple times” survey • 358 out of 1,773 responded. Card sort. Quantitative • All bug reports for Windows Vista and Windows 7 analysis • Logistic regression model for reopened bugs Manual • Random sample of reopened bugs inspection • 20 bug reports © Microsoft Corporation
8.
What are reasons
for bug report reopens? © Microsoft Corporation
9.
Causes of bug
reopens Not FIXED Related to Root Cause Bugs difficult to reproduce Developers misunderstood root cause Bug had insufficient information Related to Priority Priority of the bug increased FIXED Regression bugs Process-related Process © Microsoft Corporation
10.
#1: Difficult to
reproduce “The bug is hard to reproduce and so the fix was made without being able to fully verify it. A good example is a customer who reports something. We think we see the issue in house and fix that. It turns out we saw something different…” “Bugs which are difficult to reproduce generally get re-activated multiple times. At first, developers will give a simple repro attempt before resolving bugs 'Not repro'. But if the bug opener is able to reproduce the issue again, or perhaps comes up with better repro instructions, then the developer will pay more attention the second time the bug is activated.” “Heisenbugs” © Microsoft Corporation
11.
#2: Misunderstood root
cause “The bug is tracking an unidentified symptom and it takes a while to fully root cause. This comes up a lot with memory leaks: there will be an unknown memory leak in a component and the owning team plays whack-a-mole with the code defects to remove memory issues one-by-one.” “Not fixing the root cause and only addressing symptoms. Without root cause understood for the bug a patch/hack can often be done that will then be reactivated.” © Microsoft Corporation
12.
#3: Insufficient information
“Poor bug quality. If the bug wasn't described well enough, or not enough diagnostic info was there, the dev will guess and fix *something* in order to make the bug go away. What they fix isn't always what the person who filed the bug ran into.” “If a bug report does not accurately convey enough information about what is actually wrong (i.e. it describes incorrect behavior but neglects to mention data loss) or if the bug does not convey a dependency (such as another team relying on a fix), a bug may be de-prioritized and resolved without fixing.” © Microsoft Corporation
13.
#4: Increased priority
“Bugs are closed because one person or triage team believes the bug is not worthy of fixing (i.e. too risky, don't care, etc.), but then a few days later a VP or external customer reports the same issue, then the bug has a higher priority.” “Other reason is lack of business justification or too late in product cycle; reopened when sufficient justification exists or new cycle begins.” “One team may feel an issue is critical while the other does not see it as important enough, and instead of carrying a discussion, the bug is bounced around.” © Microsoft Corporation
14.
#5: Regression bugs
“First attempt at fix was flawed in some way, and wasn't caught because of lack of testing or unknown related scenario regression.” “I've seen cases in the past where it was thought that a bug was fixed only to find that a corner case had been missed.” “I've also seen cases where the bug was only being hit due to a timing issue and something changed that affected the timing and the bug disappeared again.” © Microsoft Corporation
15.
#6: Process-related bugs
“Sometimes bugs are reopened due to a misunderstanding of process. e.g. dev resolves bug when fix is submitted, but tester reactivates because bug still repros (because fix has not yet reached tester).” “Bug is verified fixed in a feature of developer’s branch and the fix takes too long to hit the main branch.” © Microsoft Corporation
16.
#6: Process-related bugs
“First of all, I don’t like the model where we reactivate bugs that were Fixed but the issue was not resolved. Logically it makes sense, but tracking the thread of the issue through multiple checkins & reactivates can be hell if it happens more than once or twice. I would prefer a model where once a checkin has been made for a bug, that bug is done! New issues, or issues that linger despite a previous fix, should/ would be tracked in a new bug.” © Microsoft Corporation
17.
What factors
correlate with bug reopens? © Microsoft Corporation
18.
Does the source
of a bug (how it was found) influence the likelihood of bug reopens? Bug Sources Vista Win7 Reopen rate for all bugs P Q Code analysis tools 0.52P 0.73Q less likely to be reopened Human review 0.85P 0.66Q Ad-hoc testing 0.87P 0.99Q Internal user 1.12P 0.97Q Component testing 1.13P 0.81Q System testing 1.21P 1.46Q more likely to be reopened Customer 1.33P 1.12Q © Microsoft Corporation
19.
Does opener reputation
influence the likelihood of bug reopens? For each bug, calculate opener’s reputation by aggregating over all bugs in the past. Hooimeijer and Weimer: Modeling bug report quality. ASE 2007. © Microsoft Corporation
20.
Does opener reputation
influence the likelihood of bug reopens? more likely to be reopened less likely to be reopened © Microsoft Corporation
21.
Does organizational and
geographic distance influence the likelihood of bug reopens? Vista Win7 Organizational Opened by and initially assigned to … distance … the same person X R … someone with the same manager 1.13X 0.96R … someone with a different manager 1.37X 1.07R Geographic Opened by and initially assigned to … distance … the same person X R … someone in the same building 1.27X 0.93R … someone in a different building but in the same country 1.45X 1.00R … someone in a different country 1.52X 1.14R © Microsoft Corporation
22.
Does organizational and
geographic distance influence the likelihood of bug reopens? Vista Win7 Assigned to opener at some point in time Y S Never assigned to opener, but assigned to 0.54Y 0.39S someone with the same manager as opener Never assigned to anyone with same manager 0.27Y 0.34S Never assigned to opener, but assigned to 0.41Y 0.37S someone in the same building Never assigned to anyone in same building, but 0.31Y 0.43S assigned to someone in the same country Never assigned to anyone in the same country 0.20Y 0.20S © Microsoft Corporation
23.
Descriptive statistical analysis
• All pre- and post-release bug reports for Windows Vista and Windows 7 until July 2009 • Logistic regression model to characterize – Probability that a bug will be reopened • Logistic regression model to characterize – Probability that a bug will be fixed after the bug has been reopened – Probability that a bug will be fixed (Guo et al., ICSE 2010) • Same factors as in Guo et al., ICSE 2010 © Microsoft Corporation
24.
Factor
Reopen (Vista) Human review not significant Code analysis tool -0.503 Bug source: Component testing 0.238 (categorical) Ad-hoc testing (baseline) System testing 0.204 Customer 0.239 Internal user not significant Reputation of bug opener -0.266 Reputation of 1st assignee not significant Opened by temporary employee 0.178 Initial severity level 0.127 Severity upgraded? 0.331 Opener / any assignee same manager? 0.721 Opener / any assignee same building? 0.468 Num. editors 0.236 Num. assignee building 0.090 Num. component path changes -0.160 © Microsoft Corporation
25.
Factor
Reopen (Vista) Human review not significant Code analysis tool decrease (-0.503) Bug source: Component testing increase (0.238) (categorical) Ad-hoc testing (baseline) System testing increase (0.204) Customer increase (0.239) Internal user not significant Reputation of bug opener -0.266 Reputation of 1st assignee not significant Opened by temporary employee 0.178 Initial severity level 0.127 Severity upgraded? 0.331 Opener / any assignee same manager? 0.721 Opener / any assignee same building? 0.468 Num. editors 0.236 Num. assignee building 0.090 Num. component path changes -0.160 © Microsoft Corporation
26.
Factor
Reopen (Vista) Human review not significant Code analysis tool decrease (-0.503) Bug source: Component testing increase (0.238) (categorical) Ad-hoc testing (baseline) System testing increase (0.204) Customer increase (0.239) Internal user not significant Reputation of bug opener decrease Reputation of 1st assignee not significant Opened by temporary employee increase Initial severity level increase Severity upgraded? increase Opener / any assignee same manager? increase Opener / any assignee same building? increase Num. editors increase Num. assignee building increase Num. component path changes decrease © Microsoft Corporation
27.
Which *reopened*
bugs get fixed? vs. Which bugs get fixed? © Microsoft Corporation
28.
Fixed When
Fixed (Vista) Factor Reopened (Vista) [Guo, ICSE 2010] Human review 0.377 0.511 Code analysis tool not significant 0.357 Bug source: Component testing -0.160 0.065 (categorical) Ad-hoc testing System testing not significant -0.129 Customer -0.498 -0.347 Internal user -0.465 -0.454 Reputation of bug opener 1.632 2.193 Reputation of 1st assignee 1.651 2.463 Opened by temporary employee -0.144 -0.125 Initial severity level not significant 0.033 Severity upgraded? not significant 0.256 Opener / any assignee same manager? not significant 0.676 Opener / any assignee same building? not significant 0.270 Num. editors 0.127 0.240 Num. assignee building -0.213 -0.257 Num. component path changes -0.162 -0.232 Num. re-opens n/a -0.135 © Microsoft Corporation
29.
Fixed When
Fixed (Vista) Factor Reopened (Vista) [Guo, ICSE 2010] Human review 0.377 0.511 Code analysis tool not significant 0.357 Bug source: Component testing -0.160 0.065 (categorical) Ad-hoc testing System testing not significant -0.129 Customer -0.498 -0.347 Internal user -0.465 -0.454 Reputation of bug opener 1.632 2.193 Reputation of 1st assignee 1.651 2.463 Opened by temporary employee -0.144 -0.125 Initial severity level not significant 0.033 Severity upgraded? not significant 0.256 Opener / any assignee same manager? not significant 0.676 Opener / any assignee same building? not significant 0.270 Num. editors 0.127 0.240 Num. assignee building -0.213 -0.257 Num. component path changes -0.162 -0.232 Num. re-opens n/a -0.135 © Microsoft Corporation
30.
Fixed When
Fixed (Vista) Factor Reopened (Vista) [Guo, ICSE 2010] Human review 0.377 0.511 Code analysis tool not significant 0.357 Bug source: Component testing -0.160 0.065 (categorical) Ad-hoc testing System testing not significant -0.129 Customer -0.498 -0.347 Internal user -0.465 -0.454 Reputation of bug opener 1.632 2.193 Reputation of 1st assignee 1.651 2.463 Opened by temporary employee -0.144 -0.125 Initial severity level not significant 0.033 Severity upgraded? not significant 0.256 Opener / any assignee same manager? not significant 0.676 Opener / any assignee same building? not significant 0.270 Num. editors 0.127 0.240 Num. assignee building -0.213 -0.257 Num. component path changes -0.162 -0.232 Num. re-opens n/a -0.135 © Microsoft Corporation
31.
Lessons learned
• Improve reproducibility of bug reports • Provide better tools to identify root cause • Better estimate initial priorities • Reduce the complexity of branching (bugs were “verified” in the wrong branch) © Microsoft Corporation
32.
Thank you!
Partial replication of Shihab et al. New measurements: organizational and geographic distance, reputation, how found Qualitative component on the causes of bug reopens (survey): root cause, priority, process Descriptive models based on logistic regression http://research.microsoft.com/ese © Microsoft Corporation