SlideShare una empresa de Scribd logo
1 de 32
Descargar para leer sin conexión
.
...... Defect Localization
Leo Arias
Canonical
September 24, 2013
Leo Arias (Canonical) Defect Localization September 24, 2013 1 / 26
Contents
...1 Goals
...2 Isolation
...3 Defect Localization problems
...4 Proposal
...5 Open discussion
Leo Arias (Canonical) Defect Localization September 24, 2013 2 / 26
Goals
Contents
...1 Goals
...2 Isolation
...3 Defect Localization problems
...4 Proposal
...5 Open discussion
Leo Arias (Canonical) Defect Localization September 24, 2013 3 / 26
Goals
Goals of Test Automation
Improve quality.
Understand the system under test.
Reduce and not introduce risk.
Leo Arias (Canonical) Defect Localization September 24, 2013 4 / 26
Goals
Improve Quality
Tests enable us to specify the behavior in various scenarios
captured in a form that we can then execute.
Prevent bugs from being introduced.
Defect localization.
Leo Arias (Canonical) Defect Localization September 24, 2013 5 / 26
Goals
Defect Localization
Customer tests tell us that some behavior isn’t working as expected,
the Unit tests tell us why.
We should be able to pinpoint the bug quickly based on which test fails.
If a customer test fails but no unit tests fail, it indicates a missing unit
test.
Leo Arias (Canonical) Defect Localization September 24, 2013 6 / 26
Goals
Defect Localization
Customer tests tell us that some behavior isn’t working as expected,
the Unit tests tell us why.
We should be able to pinpoint the bug quickly based on which test fails.
If a customer test fails but no unit tests fail, it indicates a missing unit
test.
Leo Arias (Canonical) Defect Localization September 24, 2013 6 / 26
Isolation
Contents
...1 Goals
...2 Isolation
...3 Defect Localization problems
...4 Proposal
...5 Open discussion
Leo Arias (Canonical) Defect Localization September 24, 2013 7 / 26
Isolation
Isolation
We should strive to isolate the system under test as much as possible
the parts we choose not to test.
Failure to isolate lead to context sensitivity because we end up testing
too much software all at once.
Leo Arias (Canonical) Defect Localization September 24, 2013 8 / 26
Isolation
Isolation
We should strive to isolate the system under test as much as possible
the parts we choose not to test.
Failure to isolate lead to context sensitivity because we end up testing
too much software all at once.
Leo Arias (Canonical) Defect Localization September 24, 2013 8 / 26
Isolation
Isolation, cont.
Leo Arias (Canonical) Defect Localization September 24, 2013 9 / 26
Isolation
Isolation, cont.
Leo Arias (Canonical) Defect Localization September 24, 2013 10 / 26
Isolation
Isolation, cont.
Leo Arias (Canonical) Defect Localization September 24, 2013 11 / 26
Defect Localization problems
Contents
...1 Goals
...2 Isolation
...3 Defect Localization problems
...4 Proposal
...5 Open discussion
Leo Arias (Canonical) Defect Localization September 24, 2013 12 / 26
Defect Localization problems
Dependencies
Leo Arias (Canonical) Defect Localization September 24, 2013 13 / 26
Defect Localization problems
More Dependencies
Leo Arias (Canonical) Defect Localization September 24, 2013 14 / 26
Defect Localization problems
Canonical Dependencies
Leo Arias (Canonical) Defect Localization September 24, 2013 15 / 26
Defect Localization problems
Canonical Dependencies
Leo Arias (Canonical) Defect Localization September 24, 2013 16 / 26
Defect Localization problems
Canonical Dependencies
Leo Arias (Canonical) Defect Localization September 24, 2013 17 / 26
Defect Localization problems
Canonical Dependencies
Leo Arias (Canonical) Defect Localization September 24, 2013 18 / 26
Defect Localization problems
Canonical Dependencies
Leo Arias (Canonical) Defect Localization September 24, 2013 19 / 26
Defect Localization problems
Defect Localization
We should be able to pinpoint the bug quickly based on which test fails.
If a customer test fails on a project B because of a change in project A,
it indicates a missing customer test on project A.
(and a missing dependency test on project B).
Leo Arias (Canonical) Defect Localization September 24, 2013 20 / 26
Defect Localization problems
Defect Localization
We should be able to pinpoint the bug quickly based on which test fails.
If a customer test fails on a project B because of a change in project A,
it indicates a missing customer test on project A.
(and a missing dependency test on project B).
Leo Arias (Canonical) Defect Localization September 24, 2013 20 / 26
Defect Localization problems
Defect Localization
We should be able to pinpoint the bug quickly based on which test fails.
If a customer test fails on a project B because of a change in project A,
it indicates a missing customer test on project A.
(and a missing dependency test on project B).
Leo Arias (Canonical) Defect Localization September 24, 2013 20 / 26
Proposal
Contents
...1 Goals
...2 Isolation
...3 Defect Localization problems
...4 Proposal
...5 Open discussion
Leo Arias (Canonical) Defect Localization September 24, 2013 21 / 26
Proposal
The problems
Projects publish an API or behavior without testing it from the point of
view of the consumers.
Projects start relying on an API or behavior even when it’s not properly
tested.
Projects change behaviors without notifying the other projects they
will affect.
Leo Arias (Canonical) Defect Localization September 24, 2013 22 / 26
Proposal
The problems
Projects publish an API or behavior without testing it from the point of
view of the consumers.
Projects start relying on an API or behavior even when it’s not properly
tested.
Projects change behaviors without notifying the other projects they
will affect.
Leo Arias (Canonical) Defect Localization September 24, 2013 22 / 26
Proposal
The problems
Projects publish an API or behavior without testing it from the point of
view of the consumers.
Projects start relying on an API or behavior even when it’s not properly
tested.
Projects change behaviors without notifying the other projects they
will affect.
Leo Arias (Canonical) Defect Localization September 24, 2013 22 / 26
Proposal
Proposal
We should focus on defect localization and isolation while adding test
coverage.
Leo Arias (Canonical) Defect Localization September 24, 2013 23 / 26
Proposal
Short-term actions
When an automated test fails because of an error or a behavior change
on another project:
File a bug describing the problem and the projects affected.
<QA> Subscribe to the bug and follow the process to its
resolution.
Make sure all the people affected are notified.
<developers> Add automated customer tests with every fix to
prevent regressions.
<QA> Make sure no fix comes without a customer test. Check that
they are added on the right project.
Leo Arias (Canonical) Defect Localization September 24, 2013 24 / 26
Proposal
After the release...
<QA> Analyze the current API and GUI coverage.
<developers> Add isolated tests to fill the holes.
<developers> Make sure we only rely on APIs and behaviors with
automated customer tests.
<QA, developers> Write a test suite specific to test the
integration of all the projects on the different Ubuntu images and
devices.
<design, QA, developers> Start adding user tests since the
expected user experience is defined.
Establish an effective way for us to communicate between teams.
Leo Arias (Canonical) Defect Localization September 24, 2013 25 / 26
Open discussion
Open discussion
Thank you!
Creative Commons - atribución 3.0
Leo Arias (Canonical) Defect Localization September 24, 2013 26 / 26

Más contenido relacionado

Similar a Defect localization

Neotys PAC - Adding Performance Verifications in Continuous Delivery
Neotys PAC - Adding Performance Verifications in Continuous DeliveryNeotys PAC - Adding Performance Verifications in Continuous Delivery
Neotys PAC - Adding Performance Verifications in Continuous DeliveryFederico Toledo
 
Rethinking Accessibility: Role-based Accessibility of WCAG 2.1
Rethinking Accessibility: Role-based Accessibility of WCAG 2.1Rethinking Accessibility: Role-based Accessibility of WCAG 2.1
Rethinking Accessibility: Role-based Accessibility of WCAG 2.1Bill Tyler
 
PAC 2019 virtual Federico Toledo
PAC 2019 virtual Federico Toledo   PAC 2019 virtual Federico Toledo
PAC 2019 virtual Federico Toledo Neotys
 
October 2014 | A211y - An Exploration into Accessibility, Agile & WCAG 2.0
October 2014 | A211y - An Exploration into Accessibility, Agile & WCAG 2.0October 2014 | A211y - An Exploration into Accessibility, Agile & WCAG 2.0
October 2014 | A211y - An Exploration into Accessibility, Agile & WCAG 2.0UXPA MN
 
Defect free development - QS Tag2019
Defect free development - QS Tag2019Defect free development - QS Tag2019
Defect free development - QS Tag2019Arnon Axelrod
 
Agile Testing - presentation for Agile User Group
Agile Testing - presentation for Agile User GroupAgile Testing - presentation for Agile User Group
Agile Testing - presentation for Agile User Groupsuwalki24.pl
 
Caring about Code Quality
Caring about Code QualityCaring about Code Quality
Caring about Code QualitySaltmarch Media
 
A Call to Arms for Private Cloud Builders
A Call to Arms for Private Cloud BuildersA Call to Arms for Private Cloud Builders
A Call to Arms for Private Cloud BuildersAngie Hirata
 
How Agile changed Software Development
How Agile changed Software DevelopmentHow Agile changed Software Development
How Agile changed Software DevelopmentSteve Maraspin
 
The Business Value of Agile Engineering Practices
The Business Value of Agile Engineering PracticesThe Business Value of Agile Engineering Practices
The Business Value of Agile Engineering PracticesRob Myers
 
The Tester's Role in Agile Planning
The Tester's Role in Agile PlanningThe Tester's Role in Agile Planning
The Tester's Role in Agile PlanningTechWell
 
SeleniumCamp 2020 - Shift Right and Observability
SeleniumCamp 2020 - Shift Right and ObservabilitySeleniumCamp 2020 - Shift Right and Observability
SeleniumCamp 2020 - Shift Right and ObservabilityMarcus Merrell
 
The Testing Strategy
The Testing StrategyThe Testing Strategy
The Testing Strategymimmozzo_
 
AgileTour Toulouse 2012 : testing strategy
AgileTour Toulouse 2012 : testing strategyAgileTour Toulouse 2012 : testing strategy
AgileTour Toulouse 2012 : testing strategyAgile Toulouse
 
DevOps is for Everyone - DevOps East
DevOps is for Everyone - DevOps EastDevOps is for Everyone - DevOps East
DevOps is for Everyone - DevOps EastChris Riley ☁
 
Joseph Yoder : Being Agile about Architecture
Joseph Yoder : Being Agile about ArchitectureJoseph Yoder : Being Agile about Architecture
Joseph Yoder : Being Agile about ArchitectureHironori Washizaki
 
Introduction to the CII Badge Programe, OW2con'16, Paris.
Introduction to the CII Badge Programe, OW2con'16, Paris. Introduction to the CII Badge Programe, OW2con'16, Paris.
Introduction to the CII Badge Programe, OW2con'16, Paris. OW2
 
apidays London 2023 - Why and how to apply DDD to APIs, Radhouane Jrad, QBE E...
apidays London 2023 - Why and how to apply DDD to APIs, Radhouane Jrad, QBE E...apidays London 2023 - Why and how to apply DDD to APIs, Radhouane Jrad, QBE E...
apidays London 2023 - Why and how to apply DDD to APIs, Radhouane Jrad, QBE E...apidays
 

Similar a Defect localization (20)

Neotys PAC - Adding Performance Verifications in Continuous Delivery
Neotys PAC - Adding Performance Verifications in Continuous DeliveryNeotys PAC - Adding Performance Verifications in Continuous Delivery
Neotys PAC - Adding Performance Verifications in Continuous Delivery
 
Rethinking Accessibility: Role-based Accessibility of WCAG 2.1
Rethinking Accessibility: Role-based Accessibility of WCAG 2.1Rethinking Accessibility: Role-based Accessibility of WCAG 2.1
Rethinking Accessibility: Role-based Accessibility of WCAG 2.1
 
PAC 2019 virtual Federico Toledo
PAC 2019 virtual Federico Toledo   PAC 2019 virtual Federico Toledo
PAC 2019 virtual Federico Toledo
 
October 2014 | A211y - An Exploration into Accessibility, Agile & WCAG 2.0
October 2014 | A211y - An Exploration into Accessibility, Agile & WCAG 2.0October 2014 | A211y - An Exploration into Accessibility, Agile & WCAG 2.0
October 2014 | A211y - An Exploration into Accessibility, Agile & WCAG 2.0
 
Defect free development - QS Tag2019
Defect free development - QS Tag2019Defect free development - QS Tag2019
Defect free development - QS Tag2019
 
Agile Testing - presentation for Agile User Group
Agile Testing - presentation for Agile User GroupAgile Testing - presentation for Agile User Group
Agile Testing - presentation for Agile User Group
 
Agile testing
Agile testingAgile testing
Agile testing
 
Caring about Code Quality
Caring about Code QualityCaring about Code Quality
Caring about Code Quality
 
A Call to Arms for Private Cloud Builders
A Call to Arms for Private Cloud BuildersA Call to Arms for Private Cloud Builders
A Call to Arms for Private Cloud Builders
 
How Agile changed Software Development
How Agile changed Software DevelopmentHow Agile changed Software Development
How Agile changed Software Development
 
The Business Value of Agile Engineering Practices
The Business Value of Agile Engineering PracticesThe Business Value of Agile Engineering Practices
The Business Value of Agile Engineering Practices
 
Software engineering
Software engineeringSoftware engineering
Software engineering
 
The Tester's Role in Agile Planning
The Tester's Role in Agile PlanningThe Tester's Role in Agile Planning
The Tester's Role in Agile Planning
 
SeleniumCamp 2020 - Shift Right and Observability
SeleniumCamp 2020 - Shift Right and ObservabilitySeleniumCamp 2020 - Shift Right and Observability
SeleniumCamp 2020 - Shift Right and Observability
 
The Testing Strategy
The Testing StrategyThe Testing Strategy
The Testing Strategy
 
AgileTour Toulouse 2012 : testing strategy
AgileTour Toulouse 2012 : testing strategyAgileTour Toulouse 2012 : testing strategy
AgileTour Toulouse 2012 : testing strategy
 
DevOps is for Everyone - DevOps East
DevOps is for Everyone - DevOps EastDevOps is for Everyone - DevOps East
DevOps is for Everyone - DevOps East
 
Joseph Yoder : Being Agile about Architecture
Joseph Yoder : Being Agile about ArchitectureJoseph Yoder : Being Agile about Architecture
Joseph Yoder : Being Agile about Architecture
 
Introduction to the CII Badge Programe, OW2con'16, Paris.
Introduction to the CII Badge Programe, OW2con'16, Paris. Introduction to the CII Badge Programe, OW2con'16, Paris.
Introduction to the CII Badge Programe, OW2con'16, Paris.
 
apidays London 2023 - Why and how to apply DDD to APIs, Radhouane Jrad, QBE E...
apidays London 2023 - Why and how to apply DDD to APIs, Radhouane Jrad, QBE E...apidays London 2023 - Why and how to apply DDD to APIs, Radhouane Jrad, QBE E...
apidays London 2023 - Why and how to apply DDD to APIs, Radhouane Jrad, QBE E...
 

Último

Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.francesco barbera
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 

Último (20)

Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 

Defect localization

  • 1. . ...... Defect Localization Leo Arias Canonical September 24, 2013 Leo Arias (Canonical) Defect Localization September 24, 2013 1 / 26
  • 2. Contents ...1 Goals ...2 Isolation ...3 Defect Localization problems ...4 Proposal ...5 Open discussion Leo Arias (Canonical) Defect Localization September 24, 2013 2 / 26
  • 3. Goals Contents ...1 Goals ...2 Isolation ...3 Defect Localization problems ...4 Proposal ...5 Open discussion Leo Arias (Canonical) Defect Localization September 24, 2013 3 / 26
  • 4. Goals Goals of Test Automation Improve quality. Understand the system under test. Reduce and not introduce risk. Leo Arias (Canonical) Defect Localization September 24, 2013 4 / 26
  • 5. Goals Improve Quality Tests enable us to specify the behavior in various scenarios captured in a form that we can then execute. Prevent bugs from being introduced. Defect localization. Leo Arias (Canonical) Defect Localization September 24, 2013 5 / 26
  • 6. Goals Defect Localization Customer tests tell us that some behavior isn’t working as expected, the Unit tests tell us why. We should be able to pinpoint the bug quickly based on which test fails. If a customer test fails but no unit tests fail, it indicates a missing unit test. Leo Arias (Canonical) Defect Localization September 24, 2013 6 / 26
  • 7. Goals Defect Localization Customer tests tell us that some behavior isn’t working as expected, the Unit tests tell us why. We should be able to pinpoint the bug quickly based on which test fails. If a customer test fails but no unit tests fail, it indicates a missing unit test. Leo Arias (Canonical) Defect Localization September 24, 2013 6 / 26
  • 8. Isolation Contents ...1 Goals ...2 Isolation ...3 Defect Localization problems ...4 Proposal ...5 Open discussion Leo Arias (Canonical) Defect Localization September 24, 2013 7 / 26
  • 9. Isolation Isolation We should strive to isolate the system under test as much as possible the parts we choose not to test. Failure to isolate lead to context sensitivity because we end up testing too much software all at once. Leo Arias (Canonical) Defect Localization September 24, 2013 8 / 26
  • 10. Isolation Isolation We should strive to isolate the system under test as much as possible the parts we choose not to test. Failure to isolate lead to context sensitivity because we end up testing too much software all at once. Leo Arias (Canonical) Defect Localization September 24, 2013 8 / 26
  • 11. Isolation Isolation, cont. Leo Arias (Canonical) Defect Localization September 24, 2013 9 / 26
  • 12. Isolation Isolation, cont. Leo Arias (Canonical) Defect Localization September 24, 2013 10 / 26
  • 13. Isolation Isolation, cont. Leo Arias (Canonical) Defect Localization September 24, 2013 11 / 26
  • 14. Defect Localization problems Contents ...1 Goals ...2 Isolation ...3 Defect Localization problems ...4 Proposal ...5 Open discussion Leo Arias (Canonical) Defect Localization September 24, 2013 12 / 26
  • 15. Defect Localization problems Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 13 / 26
  • 16. Defect Localization problems More Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 14 / 26
  • 17. Defect Localization problems Canonical Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 15 / 26
  • 18. Defect Localization problems Canonical Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 16 / 26
  • 19. Defect Localization problems Canonical Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 17 / 26
  • 20. Defect Localization problems Canonical Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 18 / 26
  • 21. Defect Localization problems Canonical Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 19 / 26
  • 22. Defect Localization problems Defect Localization We should be able to pinpoint the bug quickly based on which test fails. If a customer test fails on a project B because of a change in project A, it indicates a missing customer test on project A. (and a missing dependency test on project B). Leo Arias (Canonical) Defect Localization September 24, 2013 20 / 26
  • 23. Defect Localization problems Defect Localization We should be able to pinpoint the bug quickly based on which test fails. If a customer test fails on a project B because of a change in project A, it indicates a missing customer test on project A. (and a missing dependency test on project B). Leo Arias (Canonical) Defect Localization September 24, 2013 20 / 26
  • 24. Defect Localization problems Defect Localization We should be able to pinpoint the bug quickly based on which test fails. If a customer test fails on a project B because of a change in project A, it indicates a missing customer test on project A. (and a missing dependency test on project B). Leo Arias (Canonical) Defect Localization September 24, 2013 20 / 26
  • 25. Proposal Contents ...1 Goals ...2 Isolation ...3 Defect Localization problems ...4 Proposal ...5 Open discussion Leo Arias (Canonical) Defect Localization September 24, 2013 21 / 26
  • 26. Proposal The problems Projects publish an API or behavior without testing it from the point of view of the consumers. Projects start relying on an API or behavior even when it’s not properly tested. Projects change behaviors without notifying the other projects they will affect. Leo Arias (Canonical) Defect Localization September 24, 2013 22 / 26
  • 27. Proposal The problems Projects publish an API or behavior without testing it from the point of view of the consumers. Projects start relying on an API or behavior even when it’s not properly tested. Projects change behaviors without notifying the other projects they will affect. Leo Arias (Canonical) Defect Localization September 24, 2013 22 / 26
  • 28. Proposal The problems Projects publish an API or behavior without testing it from the point of view of the consumers. Projects start relying on an API or behavior even when it’s not properly tested. Projects change behaviors without notifying the other projects they will affect. Leo Arias (Canonical) Defect Localization September 24, 2013 22 / 26
  • 29. Proposal Proposal We should focus on defect localization and isolation while adding test coverage. Leo Arias (Canonical) Defect Localization September 24, 2013 23 / 26
  • 30. Proposal Short-term actions When an automated test fails because of an error or a behavior change on another project: File a bug describing the problem and the projects affected. <QA> Subscribe to the bug and follow the process to its resolution. Make sure all the people affected are notified. <developers> Add automated customer tests with every fix to prevent regressions. <QA> Make sure no fix comes without a customer test. Check that they are added on the right project. Leo Arias (Canonical) Defect Localization September 24, 2013 24 / 26
  • 31. Proposal After the release... <QA> Analyze the current API and GUI coverage. <developers> Add isolated tests to fill the holes. <developers> Make sure we only rely on APIs and behaviors with automated customer tests. <QA, developers> Write a test suite specific to test the integration of all the projects on the different Ubuntu images and devices. <design, QA, developers> Start adding user tests since the expected user experience is defined. Establish an effective way for us to communicate between teams. Leo Arias (Canonical) Defect Localization September 24, 2013 25 / 26
  • 32. Open discussion Open discussion Thank you! Creative Commons - atribución 3.0 Leo Arias (Canonical) Defect Localization September 24, 2013 26 / 26