SlideShare una empresa de Scribd logo
1 de 25
Descargar para leer sin conexión
Balancing Your Test Effort


Planning test with Google’s approach



                                           Nikita Knysh
                                       nik@scanjour.dk
                                Ciklum, August 17, 2011
The Speaker

• Nikita Knysh, 30 y.o., ~11 years in IT

• Background: IT education, webdev,
  support+lead, PM, TW, BA+lead,
  ISTQB FL

• Now: 4 years with Scanjour, Test TL
Agenda

•   The Challenge
•   The Directions
•   The Method
•   The Tools
•   Probs & Cures
The Challenge
•   What to test if not enough time?
•   What to test if have buffer?
•   Much test, few bugs
•   No overview of test needs
•   Not enough sync between roles
The Directions

• Discuss
• Summarize
• Prioritize
… team-wide & easily


So what’s the right approach then??
The Method: Origin

ACC (Attribute-Component-Capability)

                       James A. Whittaker,
                       Test Director at Google,
                       2010
Microsoft and Google use this.


          Can we?
The Method: Concept

• List product’s selling points (Attributes)
• Break down the product into tech
  Components.
• Break down the product based on
   WHAT it does (Capabilities).

We get a model that reflects all
the vital views on the product!
The Method: ACC Modeling
• ACC list




• Time to give it some numbers!
The Method: Giving it Numbers

• Complexity

     increases risk of human mistakes during code
     development and maintenance and therefore
     risk of introducing bugs



• Use full scale (1 to 5)
• Track averages
The Method: Giving it Numbers

• Complexity factors




                       unit test coverage
The Method: Giving it Numbers

• Frequency of Use

     how often the capability is called by user or
     automatically and therefore how often failures
     caused by defects in code will most likely
     occur
The Method: Giving it Numbers

• User Impact

     damage dealt to user and / or system intent
     should the capability fail completely or
     severely
The Method: Outcome

• Testing Needs

     = Complexity * Frequency * Impact
Now we know where the risk is.
Now we know where we need more test.



               and…
Our knowledge is based on cumulated
      vision of the whole team!



…and it is extremely easy to overview!
The Method: Outcome

• ACC list, now with numbers
The Method: Outcome

• Matrix view of capability count
The Method: Outcome

• Matrix view of testing needs
The Tools: How We Do It

•   ACC items are TFS work items
•   ACC linked to TCs and bugs for metrics
•   Excel book for each model
•   Two-way sync between TFS and Excel
•   Instant update: DWH cube is avoided
•   Pivot tables and charts
    make the beauty
The Tools: How It Looks

• Model overview in Excel
That easy?

 Really?
Probs & Cures

• “No time for modeling!”. Include into
  DoD.
• “It shows what we knew!”
   Be strong! Don’t manipulate!
   +True for any document.
More Probs & Cures

• Can’t test individual capabilities.
  Reconsider product breakdown.
• Hard to bind test cases and bugs to
  capabilities. Reconsider model’s detail
  level. Think starting from ACC, not test
  cases.
Thank you!

• Questions & Answers

Más contenido relacionado

La actualidad más candente

The Challenge of Accepting Software
The Challenge of Accepting SoftwareThe Challenge of Accepting Software
The Challenge of Accepting SoftwareSQALab
 
Vipul Kocher - Software Testing, A Framework Based Approach
Vipul Kocher - Software Testing, A Framework Based ApproachVipul Kocher - Software Testing, A Framework Based Approach
Vipul Kocher - Software Testing, A Framework Based ApproachTEST Huddle
 
Reporting On The Testing Process
Reporting On The Testing ProcessReporting On The Testing Process
Reporting On The Testing Processgavhays
 
Agile Testing Process
Agile Testing ProcessAgile Testing Process
Agile Testing ProcessIntetics
 
Test strategy &-testplanning
Test strategy &-testplanningTest strategy &-testplanning
Test strategy &-testplanningsrivinayak
 
Predictive Analytics in Software Testing
Predictive Analytics in Software TestingPredictive Analytics in Software Testing
Predictive Analytics in Software TestingPavan Kumar Kodedela
 
New Model Testing: A New Test Process and Tool
New Model Testing:  A New Test Process and ToolNew Model Testing:  A New Test Process and Tool
New Model Testing: A New Test Process and ToolTEST Huddle
 
Niels Malotaux - Help We Have a QA Problem!
Niels Malotaux -  Help We Have a QA Problem!Niels Malotaux -  Help We Have a QA Problem!
Niels Malotaux - Help We Have a QA Problem!TEST Huddle
 
[Vu Van Nguyen] Value-based Software Testing an Approach to Prioritizing Tests
[Vu Van Nguyen]  Value-based Software Testing an Approach to Prioritizing Tests[Vu Van Nguyen]  Value-based Software Testing an Approach to Prioritizing Tests
[Vu Van Nguyen] Value-based Software Testing an Approach to Prioritizing TestsHo Chi Minh City Software Testing Club
 
4. The Software Development Process - Testing
4. The Software Development Process - Testing4. The Software Development Process - Testing
4. The Software Development Process - TestingForrester High School
 
System Testingin Agile Environment
System Testingin Agile EnvironmentSystem Testingin Agile Environment
System Testingin Agile EnvironmentMary Jiang
 
ЄРМЕК КАДИРБАЄВ & АЛЕКС РИБКІН «How we train QAEs to join automation» Online ...
ЄРМЕК КАДИРБАЄВ & АЛЕКС РИБКІН «How we train QAEs to join automation» Online ...ЄРМЕК КАДИРБАЄВ & АЛЕКС РИБКІН «How we train QAEs to join automation» Online ...
ЄРМЕК КАДИРБАЄВ & АЛЕКС РИБКІН «How we train QAEs to join automation» Online ...GoQA
 
Managing agile testing
Managing agile testing Managing agile testing
Managing agile testing PractiTest
 
Decreasing false positives in automated testing
Decreasing false positives in automated testingDecreasing false positives in automated testing
Decreasing false positives in automated testingSauce Labs
 

La actualidad más candente (20)

Exploratory testing
Exploratory testingExploratory testing
Exploratory testing
 
The Challenge of Accepting Software
The Challenge of Accepting SoftwareThe Challenge of Accepting Software
The Challenge of Accepting Software
 
Vipul Kocher - Software Testing, A Framework Based Approach
Vipul Kocher - Software Testing, A Framework Based ApproachVipul Kocher - Software Testing, A Framework Based Approach
Vipul Kocher - Software Testing, A Framework Based Approach
 
Reporting On The Testing Process
Reporting On The Testing ProcessReporting On The Testing Process
Reporting On The Testing Process
 
Agile Testing Process
Agile Testing ProcessAgile Testing Process
Agile Testing Process
 
Test strategy &-testplanning
Test strategy &-testplanningTest strategy &-testplanning
Test strategy &-testplanning
 
Predictive Analytics in Software Testing
Predictive Analytics in Software TestingPredictive Analytics in Software Testing
Predictive Analytics in Software Testing
 
Tabara testare adrian matei
Tabara testare adrian mateiTabara testare adrian matei
Tabara testare adrian matei
 
Agile Testing - Not Just Tester’s Story _ Dang Thanh Long
Agile Testing - Not Just Tester’s Story _ Dang Thanh LongAgile Testing - Not Just Tester’s Story _ Dang Thanh Long
Agile Testing - Not Just Tester’s Story _ Dang Thanh Long
 
Agile QA Process
Agile QA ProcessAgile QA Process
Agile QA Process
 
New Model Testing: A New Test Process and Tool
New Model Testing:  A New Test Process and ToolNew Model Testing:  A New Test Process and Tool
New Model Testing: A New Test Process and Tool
 
Niels Malotaux - Help We Have a QA Problem!
Niels Malotaux -  Help We Have a QA Problem!Niels Malotaux -  Help We Have a QA Problem!
Niels Malotaux - Help We Have a QA Problem!
 
[Paul Holland] Bad Metrics and What You Can Do About It
[Paul Holland] Bad Metrics and What You Can Do About It[Paul Holland] Bad Metrics and What You Can Do About It
[Paul Holland] Bad Metrics and What You Can Do About It
 
[Vu Van Nguyen] Value-based Software Testing an Approach to Prioritizing Tests
[Vu Van Nguyen]  Value-based Software Testing an Approach to Prioritizing Tests[Vu Van Nguyen]  Value-based Software Testing an Approach to Prioritizing Tests
[Vu Van Nguyen] Value-based Software Testing an Approach to Prioritizing Tests
 
Agile Testing
Agile TestingAgile Testing
Agile Testing
 
4. The Software Development Process - Testing
4. The Software Development Process - Testing4. The Software Development Process - Testing
4. The Software Development Process - Testing
 
System Testingin Agile Environment
System Testingin Agile EnvironmentSystem Testingin Agile Environment
System Testingin Agile Environment
 
ЄРМЕК КАДИРБАЄВ & АЛЕКС РИБКІН «How we train QAEs to join automation» Online ...
ЄРМЕК КАДИРБАЄВ & АЛЕКС РИБКІН «How we train QAEs to join automation» Online ...ЄРМЕК КАДИРБАЄВ & АЛЕКС РИБКІН «How we train QAEs to join automation» Online ...
ЄРМЕК КАДИРБАЄВ & АЛЕКС РИБКІН «How we train QAEs to join automation» Online ...
 
Managing agile testing
Managing agile testing Managing agile testing
Managing agile testing
 
Decreasing false positives in automated testing
Decreasing false positives in automated testingDecreasing false positives in automated testing
Decreasing false positives in automated testing
 

Destacado

Future Of Software Testing
Future Of Software TestingFuture Of Software Testing
Future Of Software Testing99tests
 
House of quality matrix
House of quality matrixHouse of quality matrix
House of quality matrixRohit K.
 
Non Functional Testing
Non Functional TestingNon Functional Testing
Non Functional TestingNishant Worah
 
Agile Testing Framework - The Art of Automated Testing
Agile Testing Framework - The Art of Automated TestingAgile Testing Framework - The Art of Automated Testing
Agile Testing Framework - The Art of Automated TestingDimitri Ponomareff
 

Destacado (7)

Future Of Software Testing
Future Of Software TestingFuture Of Software Testing
Future Of Software Testing
 
House of quality matrix
House of quality matrixHouse of quality matrix
House of quality matrix
 
House of quality
House  of  qualityHouse  of  quality
House of quality
 
House of Quality
House of QualityHouse of Quality
House of Quality
 
Ch 7 qm
Ch 7 qmCh 7 qm
Ch 7 qm
 
Non Functional Testing
Non Functional TestingNon Functional Testing
Non Functional Testing
 
Agile Testing Framework - The Art of Automated Testing
Agile Testing Framework - The Art of Automated TestingAgile Testing Framework - The Art of Automated Testing
Agile Testing Framework - The Art of Automated Testing
 

Similar a ACC presentation for QA Club Kiev

Dare to Explore: Discover ET!
Dare to Explore: Discover ET!Dare to Explore: Discover ET!
Dare to Explore: Discover ET!Raj Indugula
 
'A critique of testing' UK TMF forum January 2015
'A critique of testing' UK TMF forum January 2015 'A critique of testing' UK TMF forum January 2015
'A critique of testing' UK TMF forum January 2015 Georgina Tilby
 
Effective engineer
Effective engineerEffective engineer
Effective engineerTony Vu
 
Artur Suchwalko “What are common mistakes in Data Science projects and how to...
Artur Suchwalko “What are common mistakes in Data Science projects and how to...Artur Suchwalko “What are common mistakes in Data Science projects and how to...
Artur Suchwalko “What are common mistakes in Data Science projects and how to...Lviv Startup Club
 
Aect2018 workshop-v6ij-compressed
Aect2018 workshop-v6ij-compressedAect2018 workshop-v6ij-compressed
Aect2018 workshop-v6ij-compressedIsa Jahnke
 
software testing metrics do's - don'ts-XBOSoft-QAI Webinar
software testing metrics do's - don'ts-XBOSoft-QAI Webinarsoftware testing metrics do's - don'ts-XBOSoft-QAI Webinar
software testing metrics do's - don'ts-XBOSoft-QAI WebinarXBOSoft
 
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI Webinar
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI WebinarSoftware Quality Metrics Do's and Don'ts - XBOSoft-QAI Webinar
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI WebinarXBOSoft
 
Beyond "Quality Assurance"
Beyond "Quality Assurance"Beyond "Quality Assurance"
Beyond "Quality Assurance"Jason Benton
 
Why do you think your DevOps works.
Why do you think your DevOps works.Why do you think your DevOps works.
Why do you think your DevOps works.Borys Drozhak
 
Effective Business Analysis in a Changing World
Effective Business Analysis in a Changing WorldEffective Business Analysis in a Changing World
Effective Business Analysis in a Changing WorldDevFactoTechnologies
 
Session-Based Test Management
Session-Based Test ManagementSession-Based Test Management
Session-Based Test Managementcaltonhill
 
Adapt your testing for Agile
Adapt your testing for AgileAdapt your testing for Agile
Adapt your testing for AgileEmma Armstrong
 
A New Model for Testing
A New Model for TestingA New Model for Testing
A New Model for TestingSQALab
 
Challenges to Effective Performance Testing in CI
Challenges to Effective Performance Testing in CIChallenges to Effective Performance Testing in CI
Challenges to Effective Performance Testing in CIFederico Toledo
 
Becoming a better programmer - unit testing
Becoming a better programmer - unit testingBecoming a better programmer - unit testing
Becoming a better programmer - unit testingDuy Tan Geek
 
Test analysis & design good practices@TDT Iasi 17Oct2013
Test analysis & design   good practices@TDT Iasi 17Oct2013Test analysis & design   good practices@TDT Iasi 17Oct2013
Test analysis & design good practices@TDT Iasi 17Oct2013Tabăra de Testare
 
Agile and semat v0.91
Agile and semat v0.91Agile and semat v0.91
Agile and semat v0.91Yury Katkov
 

Similar a ACC presentation for QA Club Kiev (20)

Dare to Explore: Discover ET!
Dare to Explore: Discover ET!Dare to Explore: Discover ET!
Dare to Explore: Discover ET!
 
'A critique of testing' UK TMF forum January 2015
'A critique of testing' UK TMF forum January 2015 'A critique of testing' UK TMF forum January 2015
'A critique of testing' UK TMF forum January 2015
 
Methodology: IT test
Methodology: IT testMethodology: IT test
Methodology: IT test
 
Effective engineer
Effective engineerEffective engineer
Effective engineer
 
Artur Suchwalko “What are common mistakes in Data Science projects and how to...
Artur Suchwalko “What are common mistakes in Data Science projects and how to...Artur Suchwalko “What are common mistakes in Data Science projects and how to...
Artur Suchwalko “What are common mistakes in Data Science projects and how to...
 
Aect 2018 workshop
Aect 2018 workshopAect 2018 workshop
Aect 2018 workshop
 
Aect2018 workshop-v6ij-compressed
Aect2018 workshop-v6ij-compressedAect2018 workshop-v6ij-compressed
Aect2018 workshop-v6ij-compressed
 
software testing metrics do's - don'ts-XBOSoft-QAI Webinar
software testing metrics do's - don'ts-XBOSoft-QAI Webinarsoftware testing metrics do's - don'ts-XBOSoft-QAI Webinar
software testing metrics do's - don'ts-XBOSoft-QAI Webinar
 
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI Webinar
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI WebinarSoftware Quality Metrics Do's and Don'ts - XBOSoft-QAI Webinar
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI Webinar
 
Beyond "Quality Assurance"
Beyond "Quality Assurance"Beyond "Quality Assurance"
Beyond "Quality Assurance"
 
Why do you think your DevOps works.
Why do you think your DevOps works.Why do you think your DevOps works.
Why do you think your DevOps works.
 
Effective Business Analysis in a Changing World
Effective Business Analysis in a Changing WorldEffective Business Analysis in a Changing World
Effective Business Analysis in a Changing World
 
Session-Based Test Management
Session-Based Test ManagementSession-Based Test Management
Session-Based Test Management
 
Adapt your testing for Agile
Adapt your testing for AgileAdapt your testing for Agile
Adapt your testing for Agile
 
DevOps 101
DevOps 101DevOps 101
DevOps 101
 
A New Model for Testing
A New Model for TestingA New Model for Testing
A New Model for Testing
 
Challenges to Effective Performance Testing in CI
Challenges to Effective Performance Testing in CIChallenges to Effective Performance Testing in CI
Challenges to Effective Performance Testing in CI
 
Becoming a better programmer - unit testing
Becoming a better programmer - unit testingBecoming a better programmer - unit testing
Becoming a better programmer - unit testing
 
Test analysis & design good practices@TDT Iasi 17Oct2013
Test analysis & design   good practices@TDT Iasi 17Oct2013Test analysis & design   good practices@TDT Iasi 17Oct2013
Test analysis & design good practices@TDT Iasi 17Oct2013
 
Agile and semat v0.91
Agile and semat v0.91Agile and semat v0.91
Agile and semat v0.91
 

Más de Nikita Knysh

Overview of test process improvement frameworks
Overview of test process improvement frameworksOverview of test process improvement frameworks
Overview of test process improvement frameworksNikita Knysh
 
Fundamental Test Process New
Fundamental Test Process NewFundamental Test Process New
Fundamental Test Process NewNikita Knysh
 
Software Testing Foundations Part 7 - Basics of Test Management
Software Testing Foundations Part 7 - Basics of Test ManagementSoftware Testing Foundations Part 7 - Basics of Test Management
Software Testing Foundations Part 7 - Basics of Test ManagementNikita Knysh
 
Software Testing Foundations Part 8 - Test Tools
Software Testing Foundations Part 8 - Test ToolsSoftware Testing Foundations Part 8 - Test Tools
Software Testing Foundations Part 8 - Test ToolsNikita Knysh
 
Software Testing Foundations Part 5 - White Box Testing
Software Testing Foundations Part 5 - White Box TestingSoftware Testing Foundations Part 5 - White Box Testing
Software Testing Foundations Part 5 - White Box TestingNikita Knysh
 
Software Testing Foundations Part 4 - Black Box Testing
Software Testing Foundations Part 4 - Black Box TestingSoftware Testing Foundations Part 4 - Black Box Testing
Software Testing Foundations Part 4 - Black Box TestingNikita Knysh
 
Software Testing Foundations Part 6 - Intuitive and Experience-based testing
Software Testing Foundations Part 6 - Intuitive and Experience-based testingSoftware Testing Foundations Part 6 - Intuitive and Experience-based testing
Software Testing Foundations Part 6 - Intuitive and Experience-based testingNikita Knysh
 
Software Testing Foundations Part 2 - Testing in Software Lifecycle
Software Testing Foundations Part 2 - Testing in Software LifecycleSoftware Testing Foundations Part 2 - Testing in Software Lifecycle
Software Testing Foundations Part 2 - Testing in Software LifecycleNikita Knysh
 

Más de Nikita Knysh (8)

Overview of test process improvement frameworks
Overview of test process improvement frameworksOverview of test process improvement frameworks
Overview of test process improvement frameworks
 
Fundamental Test Process New
Fundamental Test Process NewFundamental Test Process New
Fundamental Test Process New
 
Software Testing Foundations Part 7 - Basics of Test Management
Software Testing Foundations Part 7 - Basics of Test ManagementSoftware Testing Foundations Part 7 - Basics of Test Management
Software Testing Foundations Part 7 - Basics of Test Management
 
Software Testing Foundations Part 8 - Test Tools
Software Testing Foundations Part 8 - Test ToolsSoftware Testing Foundations Part 8 - Test Tools
Software Testing Foundations Part 8 - Test Tools
 
Software Testing Foundations Part 5 - White Box Testing
Software Testing Foundations Part 5 - White Box TestingSoftware Testing Foundations Part 5 - White Box Testing
Software Testing Foundations Part 5 - White Box Testing
 
Software Testing Foundations Part 4 - Black Box Testing
Software Testing Foundations Part 4 - Black Box TestingSoftware Testing Foundations Part 4 - Black Box Testing
Software Testing Foundations Part 4 - Black Box Testing
 
Software Testing Foundations Part 6 - Intuitive and Experience-based testing
Software Testing Foundations Part 6 - Intuitive and Experience-based testingSoftware Testing Foundations Part 6 - Intuitive and Experience-based testing
Software Testing Foundations Part 6 - Intuitive and Experience-based testing
 
Software Testing Foundations Part 2 - Testing in Software Lifecycle
Software Testing Foundations Part 2 - Testing in Software LifecycleSoftware Testing Foundations Part 2 - Testing in Software Lifecycle
Software Testing Foundations Part 2 - Testing in Software Lifecycle
 

Último

Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...IMARC Group
 
Borderless Access - Global Panel book-unlock 2024
Borderless Access - Global Panel book-unlock 2024Borderless Access - Global Panel book-unlock 2024
Borderless Access - Global Panel book-unlock 2024Borderless Access
 
Talent Management research intelligence_13 paradigm shifts_20 March 2024.pdf
Talent Management research intelligence_13 paradigm shifts_20 March 2024.pdfTalent Management research intelligence_13 paradigm shifts_20 March 2024.pdf
Talent Management research intelligence_13 paradigm shifts_20 March 2024.pdfCharles Cotter, PhD
 
Introduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptxIntroduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptxJemalSeid25
 
Lecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb toLecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb toumarfarooquejamali32
 
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)tazeenaila12
 
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John MeulemansBCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John MeulemansBBPMedia1
 
Developing Coaching Skills: Mine, Yours, Ours
Developing Coaching Skills: Mine, Yours, OursDeveloping Coaching Skills: Mine, Yours, Ours
Developing Coaching Skills: Mine, Yours, OursKaiNexus
 
Intellectual Property Licensing Examples
Intellectual Property Licensing ExamplesIntellectual Property Licensing Examples
Intellectual Property Licensing Examplesamberjiles31
 
Upgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsUpgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsIntellect Design Arena Ltd
 
Team B Mind Map for Organizational Chg..
Team B Mind Map for Organizational Chg..Team B Mind Map for Organizational Chg..
Team B Mind Map for Organizational Chg..dlewis191
 
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for BusinessQ2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for BusinessAPCO
 
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView
 
Chicago Medical Malpractice Lawyer Chicago Medical Malpractice Lawyer.pdf
Chicago Medical Malpractice Lawyer Chicago Medical Malpractice Lawyer.pdfChicago Medical Malpractice Lawyer Chicago Medical Malpractice Lawyer.pdf
Chicago Medical Malpractice Lawyer Chicago Medical Malpractice Lawyer.pdfSourav Sikder
 
The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003
The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003
The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003believeminhh
 
PDT 88 - 4 million seed - Seed - Protecto.pdf
PDT 88 - 4 million seed - Seed - Protecto.pdfPDT 88 - 4 million seed - Seed - Protecto.pdf
PDT 88 - 4 million seed - Seed - Protecto.pdfHajeJanKamps
 
Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024Winbusinessin
 
Cracking the ‘Business Process Outsourcing’ Code Main.pptx
Cracking the ‘Business Process Outsourcing’ Code Main.pptxCracking the ‘Business Process Outsourcing’ Code Main.pptx
Cracking the ‘Business Process Outsourcing’ Code Main.pptxWorkforce Group
 
Building Your Personal Brand on LinkedIn - Expert Planet- 2024
 Building Your Personal Brand on LinkedIn - Expert Planet-  2024 Building Your Personal Brand on LinkedIn - Expert Planet-  2024
Building Your Personal Brand on LinkedIn - Expert Planet- 2024Stephan Koning
 

Último (20)

Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
 
WAM Corporate Presentation Mar 25 2024.pdf
WAM Corporate Presentation Mar 25 2024.pdfWAM Corporate Presentation Mar 25 2024.pdf
WAM Corporate Presentation Mar 25 2024.pdf
 
Borderless Access - Global Panel book-unlock 2024
Borderless Access - Global Panel book-unlock 2024Borderless Access - Global Panel book-unlock 2024
Borderless Access - Global Panel book-unlock 2024
 
Talent Management research intelligence_13 paradigm shifts_20 March 2024.pdf
Talent Management research intelligence_13 paradigm shifts_20 March 2024.pdfTalent Management research intelligence_13 paradigm shifts_20 March 2024.pdf
Talent Management research intelligence_13 paradigm shifts_20 March 2024.pdf
 
Introduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptxIntroduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptx
 
Lecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb toLecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb to
 
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
 
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John MeulemansBCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
 
Developing Coaching Skills: Mine, Yours, Ours
Developing Coaching Skills: Mine, Yours, OursDeveloping Coaching Skills: Mine, Yours, Ours
Developing Coaching Skills: Mine, Yours, Ours
 
Intellectual Property Licensing Examples
Intellectual Property Licensing ExamplesIntellectual Property Licensing Examples
Intellectual Property Licensing Examples
 
Upgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsUpgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking Applications
 
Team B Mind Map for Organizational Chg..
Team B Mind Map for Organizational Chg..Team B Mind Map for Organizational Chg..
Team B Mind Map for Organizational Chg..
 
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for BusinessQ2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
 
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
 
Chicago Medical Malpractice Lawyer Chicago Medical Malpractice Lawyer.pdf
Chicago Medical Malpractice Lawyer Chicago Medical Malpractice Lawyer.pdfChicago Medical Malpractice Lawyer Chicago Medical Malpractice Lawyer.pdf
Chicago Medical Malpractice Lawyer Chicago Medical Malpractice Lawyer.pdf
 
The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003
The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003
The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003
 
PDT 88 - 4 million seed - Seed - Protecto.pdf
PDT 88 - 4 million seed - Seed - Protecto.pdfPDT 88 - 4 million seed - Seed - Protecto.pdf
PDT 88 - 4 million seed - Seed - Protecto.pdf
 
Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024
 
Cracking the ‘Business Process Outsourcing’ Code Main.pptx
Cracking the ‘Business Process Outsourcing’ Code Main.pptxCracking the ‘Business Process Outsourcing’ Code Main.pptx
Cracking the ‘Business Process Outsourcing’ Code Main.pptx
 
Building Your Personal Brand on LinkedIn - Expert Planet- 2024
 Building Your Personal Brand on LinkedIn - Expert Planet-  2024 Building Your Personal Brand on LinkedIn - Expert Planet-  2024
Building Your Personal Brand on LinkedIn - Expert Planet- 2024
 

ACC presentation for QA Club Kiev

  • 1. Balancing Your Test Effort Planning test with Google’s approach Nikita Knysh nik@scanjour.dk Ciklum, August 17, 2011
  • 2. The Speaker • Nikita Knysh, 30 y.o., ~11 years in IT • Background: IT education, webdev, support+lead, PM, TW, BA+lead, ISTQB FL • Now: 4 years with Scanjour, Test TL
  • 3. Agenda • The Challenge • The Directions • The Method • The Tools • Probs & Cures
  • 4. The Challenge • What to test if not enough time? • What to test if have buffer? • Much test, few bugs • No overview of test needs • Not enough sync between roles
  • 5. The Directions • Discuss • Summarize • Prioritize … team-wide & easily So what’s the right approach then??
  • 6. The Method: Origin ACC (Attribute-Component-Capability) James A. Whittaker, Test Director at Google, 2010
  • 7. Microsoft and Google use this. Can we?
  • 8. The Method: Concept • List product’s selling points (Attributes) • Break down the product into tech Components. • Break down the product based on WHAT it does (Capabilities). We get a model that reflects all the vital views on the product!
  • 9. The Method: ACC Modeling • ACC list • Time to give it some numbers!
  • 10. The Method: Giving it Numbers • Complexity increases risk of human mistakes during code development and maintenance and therefore risk of introducing bugs • Use full scale (1 to 5) • Track averages
  • 11. The Method: Giving it Numbers • Complexity factors unit test coverage
  • 12. The Method: Giving it Numbers • Frequency of Use how often the capability is called by user or automatically and therefore how often failures caused by defects in code will most likely occur
  • 13. The Method: Giving it Numbers • User Impact damage dealt to user and / or system intent should the capability fail completely or severely
  • 14. The Method: Outcome • Testing Needs = Complexity * Frequency * Impact
  • 15. Now we know where the risk is. Now we know where we need more test. and…
  • 16. Our knowledge is based on cumulated vision of the whole team! …and it is extremely easy to overview!
  • 17. The Method: Outcome • ACC list, now with numbers
  • 18. The Method: Outcome • Matrix view of capability count
  • 19. The Method: Outcome • Matrix view of testing needs
  • 20. The Tools: How We Do It • ACC items are TFS work items • ACC linked to TCs and bugs for metrics • Excel book for each model • Two-way sync between TFS and Excel • Instant update: DWH cube is avoided • Pivot tables and charts make the beauty
  • 21. The Tools: How It Looks • Model overview in Excel
  • 23. Probs & Cures • “No time for modeling!”. Include into DoD. • “It shows what we knew!” Be strong! Don’t manipulate! +True for any document.
  • 24. More Probs & Cures • Can’t test individual capabilities. Reconsider product breakdown. • Hard to bind test cases and bugs to capabilities. Reconsider model’s detail level. Think starting from ACC, not test cases.

Notas del editor

  1. This is my first appearance on QA gathering. I regret I did not start earlier – it’s a pleasure for me to be among many people of the same occupation as me, who want to know more, who want more who want to be more.
  2. Scanjour R&D in Kiev: Team of 21; 9 in test!CPH part of the R&D dept has about the same dev-test ratio.We have good quality-oriented culture (unit tests and sometimes  TDD, help to test from dev), automation testing and most modern tools.Things going well, but we have where to improve.Not enough sync: sometimes people are syncronizing only after a bug is found – want to have that earlier in process to reduce defect fix cost…So what are the improvement areas… List/explainSo this is our areas to improve. How do we see the direction for our progress?
  3. We want to discuss test object team-wide, in STRUCTURED and easy mannerWe want to have a short but still descriptive summary of discussion result so we can ->Prioritize test effort,Which allows to knowwhat to test,what not to test in given time/resource constraints,and where to put additional effort if have free buffer
  4. “ACC” name of method indicates three dimensions we look at or product at.ACC has no ‘date of start’.ACC was probably first mentioned ‘globally’ around 2010 by WhittakerWhittaker started practicing in at Microsoft (for MTM 2010 we are currently using ) and brought it to Google.Google has its self-developed tools to support the method “Testify” aka “Google Test Analytics”. Tools are not public (yet).Almost no info in Internet or offline -> We (Scanjour) has taken the idea and developed the method (theory (basic definitions and explanations) and implementation based on tools we use in our teams)
  5. I heard many times: “But this is Google! This is Microsoft!”. Don’t even try to compare!!… Let see what the method is. And is there any magic.
  6. Attribute is qualities of the productsthat users / customers pay for, what solves their problems and satisfies their needs. Examples: Functional, Secure, Reliable, User-friendly, Fast… This is what POs, sales and executives brag about.Components are technical pieces of the product you can find in architectural or design documents. Libraries, classes, parts of UI, files of code or configurations, or groups of those serving some architectural or design need. This is what developers will tell you they work on. Examples: dialog in UI, form to enter/modify data, installer, API, configuration mechanism…Capabilities are small actions the product can do, both for servicing user needs and infrastructure needs. Examples:User needs: search, update item data in DB, validate form input, repair installed product, accept keyboard shortcutsInfrastructure needs: cleanup, prioritize jobs, balance load.
  7. A Capability is technically implemented in a component, and contributes to one of the product’s AttributesA pair of Attribute and Component can have any number of CapabilitiesNow it is time to quantify each Capability by several parametersto be able to compare them to each other and understand where we need more analysis and test.
  8. How often user calls the capabilityHow often the capability is called automatically (by other capabilities or external systems) – for infrastructure capabilities
  9. - The equation is a variation of risk equation (Risk = Probability * Impact). Distribution of Testing Needs in ACC corresponds in distribution of classical Risk.All team members are free and very welcome to share their opinion and influence numbers. However, practice shows that some number are more influenced by some roles.- Complexity is mostly a ‘tech’ number, less a ‘business’ numberFrequency is 50-50 tech-bizImpact is mostly business number
  10. Next slide is probably the most important in the presentation.It SHOWS MAIN VALUE OF THE METHOD.
  11. Once again – every team member, every role has contributed. PO, devs, testers. So this is the most objective model we can have.…And the result is seen on 1-2-3 screens, depending on how deep you want to dig.Bunus: discussing numbers, people share their business and technical knowledge and opinions with each other, which is great for a scrum team. Team is much more in sync about the product, which is extremely important when team does not rely on documentation too much.
  12. - Now we have the complete list.- This is a model, and we can see things when we look at it (and the object it represents) from different perspectives (next two slides).
  13. We see how much we implemented to enhance product’s attributes (column totals)We see what are most ‘heavy’ components (row totals)In the example, we can see that we did more for the product to be compatible than to be functional. This is natural, as the product heavily uses one product and vitally relies on another product.
  14. We see where we need more analysis and test.Easy to overview by single capabilities (cells), but also by attributes (column totals) and Components (row totals).In the example, we can see that Crawler component needs more test, even though there are twice less components. The reason is that Crawler is used more frequently and its crash has extremely high impact (system does not fulfill its intent).
  15. “No time” – Plan and perform ACC work instead of doing ACC when you don’t have other tasks.“It shows what we knew!” Why we need it?? – build model step-by-step, without trying to achieve certain level of testing need. Otherwise you see what you want to see! Also, remember, that putting our knowledge into written is what we always do when create/update documents – this does not mean that we don’t need documents.
  16. Capabilities are too small to test them separately – try another level of detail in the ACC model.Hard to bind test cases to capabilities (one test case cover many, the overlap a lot!). Hard to bind bugs to capabilities (hard to define what capability bug belongs to, especially when there are a lot of capabilities, >100, >200). Solved in two ways:Reconsider (decrease?) model detail level – you’ll get less overlapping of test cases covering capabilities, and each capability will represent more code so it will be easier to define to which capability bug belongs to… It’s all about balance.You are using the ACC model.Think of testing in terms of ACC. You don’t just execute test cases. You TEST CAPABILITIES. At any moment you know what capability you test, and you find most of bugs in what you are currently testing. So it is much easier to link bugs to capabilities.Conclusion: There are many good methods. Each of them can succeed or fail. This method belongs to good ones. Let us remember why:Structured. There is structure for discussion and algorithm for actions. Method is reproducible and usable.Whole team participates. Everyone contributes to the model and shares knowledge with everyone.Easy-to-use result. Short, good looking result that can easily be analysied and presented in the team and outside the team.