Agile Qa Framework Jacky Wu
•
14 likes•5,942 views
Brief how to run QA in Agile development process
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Agile Qa Framework Jacky Wu
Similar to Agile Qa Framework Jacky Wu (20)
Recently uploaded
Recently uploaded (20)
Agile Qa Framework Jacky Wu
- 1. Agile QA Framework Jacky Wu jackywxd@gmail.com
- 2. Agile Manifesto We Value the items on the left more !
- 3. Agenda Goal of QA Framework Roles and Role Interchangeability in Scrum Key success factor: Whole Team Approach QA Framework and Execution Guideline Infrastructure Process Automation in Agile Whole team and Incremental Approach Automation Defects Derivative Model BPT Automation Framework What’s Next Recap and Q&A
- 4. Goal of QA Framework
- 5. Roles in Scrum DEV Model system design Knowledge of product internals Focus on how it can work QA Model user behavior Domain knowledge Focus on how it can go wrong AQA Mixed both DEV/QA
- 6. Role Interchangeability in Scrum Sprint planning Sprint Review Design, Coding, Debug Unit testing, code review Test cases execution Translate User Stories into Test Cases Automation feasible analysis Automation testing
- 7. Whole Team Approach Whole team is responsible for quality Testers are not quality Police
- 8. Agile Development Framework QA Framework QA Framework
- 9. Execution Guideline: Balance and Adaptive Process Guidance above Process Absolute Prevention above detection Automation above manual Reusable lists above detailed test plans Exploratory testing above detailed test scripts The Key: Balance and Adaptive
- 10. QA Framework: Infrastructure Quality Center Built-in Agile support Defect Defect model in QC Testing bed Real World Performance/stress testing bed
- 11. QA Framework: Process Test plan management (Quality Center) Iterations and Requirements Mapping user story to test plan Test data management Efficient Test Data Management process benefits manual and automated testing and it directly impacts testing effectiveness Obtaining/creating test data could be time consuming Defect management Continuous Integration Data collection and reporting Defect trending/Test progress/Automation coverage
- 12. QA Process in Sprint (Sample)
- 13. QA Framework: Automation in Agile Automation is MUST in Agile Purpose of Automation Improve testing effectiveness and efficiency Ultimate goal: Improve Quality Automation v.s. Manual Maximize ROI Balance: what can automated, what cannot or should not Whole team approach Incremental approach
- 14. Whole Team Approach Automation in Scrum End User Code
- 15. Incremental Approach Automation in Sprint C C C C G G G B B C: Component automation G: GUI automation B: BPT automation I: Integration I Development Stable
- 16. Automation Measurement Define Automation Coverage Code coverage = UT Test case coverage Unique case (with or without iteration)= Automated test cases/Total test cases Run coverage= Automated test runs/Total test runs Requirement coverage Automation can find more bugs?
- 17. Defects Derivative Model Cost Requirements Wrong requirements $ Plan Correct Design Correct Design Wrong Design $$ Design Correct implementation Correct implementation Correct implementation Wrong implementation Correct implementation Wrong implementation $$$ Coding Correct behaviors Unexpected behavior Limitations Known bugs Unknown bug Testing $$$$
- 18. Business Process Testing Process Separate automation script from test data and business logic Less test cases, many iterations (testing data) Central management: testing plan, testing data, testing result QC Application Being Tested QTP Login Data-Driver BPT Test Cases BPT Backup BPT Testing data Job Status BPT
- 19. BPT Automation Framework Automation Define components Create Function Libraries Create Object Repositories Create Business Components Create Business Components Design user scenarios SME Drag Components to create test plan Configure Input/Output parameters Add cases to test set in Test Lab and Execute
- 20. Recap and Q&A Q&A
- 21. Thank you!
- 22. References http://en.wikipedia.org/wiki/Test_automation http://testingeducation.org/BBST/ http://www.ncpmi.org/userfiles/File/NCPMI_AE2010_Lawson.pdf http://c-spin.net/2010/cspin201001eMids_QA_in_Agile.pdf http://www.cigital.com/presentations/Agile%20Automation%20Testing.pdf http://www.benchmarkqa.com/pdf/papers_automation_myths.pdf http://www.methodsandtools.com/archive/archive.php?id=94 Case in point: Microsoft Vista http://www.joelonsoftware.com/items/2007/12/03.html
Editor's Notes
- So how to execute the QA framework in Agile context? There are few guidelines, and the key is balance and adaptive. As mentioned previously, every scrum team is unique. Scrum team has to find the best suitable approach based on the actual status of each scrum team.
- So far QA is the major user of QC. This is an example process which involves all roles in Scrums.
- Automation in Sprint also will incremental
- There is no doubt that Automation is important. But how do we know how are we doing? How to measure automation testing? One of the common way is automation coverage.Code coverage: cover how many line of codes and possible code pathTest case coverage: percentage test cases that are automatedRequirement coverage: cover how much percentage of user scenariosOne execution of test case is one test run.How is test coverage defined? Are we measuring test cases against requirements (generally during system testing), or are we measuring test cases against all possible paths taken through the units and components (generally used for unit testing)? In other words, are we looking at unit testing coverage, code coverage, or requirements coverage?Another measurement could be how many bugs are found by automation? So can automation testing find more bugs? In next slide of defect derivative model, we can look into it.
- Which type of defects can be found by Automation?From the cost effective perspective, the best place to use automation is to increase the chance of correct implementation during coding. Example, plain password in the log file. So UT automation, component level automation is very important. These type of automation can help us find problem before it becomes defect.Regression testing also is another type of testing can leverage automation. Actually I believe regression has to use automation.But no matter what, we don’t think today we are over-automated, given the same amount of investment how can we increase automation coverage? The answer is BPT and Data driven.
- So what’s BPT? BPT stands for Business Process Testing. In the end of the this slide, there are more details information about BPT. It also fits into whole team approach and incremental approach.Less test cases; ASBU used to have more than 10 thounds of test case, one of the reason is we mixed test data with our function test case. Sankar and I actually discussed it previously, we all agree we should reduce the number of test cases. And using BPT is one of the effective way to reduce test cases.
- BPT is the automation framework which can leverage both strength of DQA and AQA. It also can free AQA’s hand so AQA can focus on the most important thing: coding, instead of maintain. Once component is turned over to DQA, DQA will take the ownership of automation, create, execute and monitor automation test cases will be handled by DQA. Moving forward, we also expect DQA can pick up some of the tasks from AQA, moving up the bar.Benefits of BPT:Fully leverage strength of AQA and DQAIt fits into Agile development and QA processMaximize Automation ROIKeep track automation usages and provide comprehensive reportsNo additional cost incurred