2. Different Roles in a Modern Product Development
• Project/Program manager.
• Product Marketing people
• Development/QA manager. (people manager)
• Technical lead
• Team member (engineers)
3. QA Manager’s Role
• Managing team (people management)
• Interact with other function team in the product development
• Guardian of the product quality
• Make sure the team’s work is on time with Quality
QA Manager QA Team
Development
Manager
Documentation
Team
Product
Manager
SE/Support
4. Pre-Project Planning
• Understand the working scope and requirement
• Logistics
• Resource and Schedule.
• Define clear deliverables and acceptance criteria
• Communication with peers and team members
5. Understand the working
scope• Talk to Program manager
• What is the over all release schedule.
• How long is the coding, how long is the testing.
• Talk to product manager and marketing folks.
• Why we need this product/features.
• Impact on the revenu
• Talk to Development manager
• Understand the implementation in high level
• Define the code hand over criteria
6. Logistics
• Any product related trainings your team need and when?
• Any technology related trainings your team need and when?
• Equipments needed, budget and where to get them
• Device under test
• PCs
• Routers/switches
• Software
7. Resource and Schedule.
• Need to justify the resource and schedule that you need
• Need to have a productivity estimation of the team’s capability
on
• Number of test cases develop per day per engineer
• Number of test cases manual execution per day per engineer
• Number of test scripts develop per day per engineer
• A Week by Week detail schedule would be good
Sample Manual
Execution Report
Sample Regression
Report
8. Resource and Schedule.
• Leave buffer for the emergency and last minute change.
• Have your priority list of the projects you are doing right now.
• Communicate your priority list with your peers and upper
management
10. Test Case & Test Plan
Development
• Deliverables:
• Test case high level design (test purpose)
• Detail test case & test plan documentation.
• Acceptance Criteria
• On Schedule
• High level design reviewed with development team and
marketing people, passed development team and marketing
people’s internal review process.
• Test case & test plan reviewed with development team and
marketing people, passed development team and marketing
people's internal review process.
11. Test Case Manual Execution
• Deliverables
• Test case execution log
• Defects
• Acceptance Criteria
• On Schedule
• Case execution log reviewed and approved by development team
and marketing people.
• All test cases are either in a state of “PASS” or in a state of “FAIL”
with a open defect ID.
• No invalid defects filed by Sigma
• All defects are valid & reproducible.
• Defects response time from Sigma no later than 12 hours.
12. Test Case Automated
Regression
• Deliverable
• Automated Regression Report.
• Regression Defects
• Acceptance Criteria
• On Schedule.
• All scripts results are either in a state of “PASS” or in a state of
“FAIL” with an open defect ID.
• Results are in consistent with baseline result.
• All bugs reported are valid regression bugs after a clean baseline
establishment.
• Defects response time from Sigma no later than 12 hours.
13. Script Development –
Deliverables.
• Deliverables
• Script template
• Scripts
• Execution logs (results are either in a state of “PASS” or in a state
of “FAIL” with an open defect ID)
• Script code structure documentation
• Script integration guideline (to integrate in team’s testing
environment)
14. Script Development –
Acceptance Criteria
• Acceptance Criteria
• On schedule
• Each script passed on team predefined
• Hardware product list
• Software (platform) version list.
Log will be provided. (can fail but Sigma will provide an open bug
id)
• Scripts are reviewed and approved by team internal review
process
• Scripts can be executed with same results in team’s testing
environment as in Sigma testing environment
• 30 days post-project script support.
15. Communication and Sale
• Propose your test project plans to the upper management and
your peer.
• Justify your claim on the cost/budget
• Equipments
• Resources.
• Justify your schedules
• Communicate with the team on the project background, scope
and requirement
16. Standard Template and Standard Test Case
• Standard template of test plan/test cases.
• Standard template of scripts
• Standard test case library that every one can share
• CLI test cases
• GUI test cases.
• Negative values library
Test Plan Template Sample Scripts
17. Effective Review Meetings
• At lest 2 sets of reviews on the deliverables
• Internal review
• External review
• Invite
• Coder, development manager
• Marketing, product managers
• Sales Engineers
• Assign roles of “Recorder”, “Reviewer”, “Moderator”, “Reader”
in the review meeting
• Make sure all feedbacks are recorded, have a follow up plan
and have been updated on the deliverables
18. Action Item List
• Weekly reviews on the action item and its state
• New
• Pending
• In progress
• Completed (can be removed from the list)
• Make sure once a AI is on the list, it will be there for ever if not
“completed”, and track the status every week
• Always list last week's objectives and this
week'saccomplishments against them
20. Review with Team Member’s
Work• In the tester's cube, not yours
• Show me your best work from last week
• Show me your most interesting bugs
• Show me your most interesting test cases
• What have you been testing? Why did you do it that way? Have
you thought about this?
21. Daily report for the bug trend and bug state
• Have a dedicated team member to send out
• Number of bugs filed today and by who
• Total number of bugs filed by each of the team members till
today
• Bugs that need team members to response today
22. Daily report on the team member’s productivity
• Number of the test cases manually or auto executed today by
each of the team member
• Number of the scripts developed today by each of the team
member
• Number of the test cases developed today by each of the team
member
23. Report and Post Mortem
Analysis• Generate your testing report to the upper management
• Defects: very important, no defects, no talk.
• Participate the release meeting to sign off or not
• Post Mortem analysis with the team
24. Generate your testing report
• Make sure your report is accurate and clean
• Failed test case must have an open bug ID
• High level summary for your boss:
• Total number of test cases developed and executed
• Total number of defects
• Total number of test cases.
• Bug trend – Bug open rate, close rate during the project period
25. Participate the release
meeting• Make a decision if you will sign off or not on the release
• Prepare your bug report to justify your claim
• Remember to release or not is a business decision, not a
testing decision
• Make sure you provide the accurate risk analysis to the upper
management
26. Post Mortem Analysis
• A round table survey and talk with in your team
• What you think we do good in this project
• What you think we do bad on this project
• If there is one thing you wish we could do better ,what would
that be?
• Read the post mortem report, it is a very good way to
understand your team, and do better in the next project.