1. Key Principles for Efficient Test Reporting
Providing the right information to the right stakeholders

2. Purpose
• This document outlines the key principles of efficient test reporting
• This is a high level presentation about test reporting that must be
implemented differently for every context and stakeholder
• Following these key principles will give a good base for setting up efficient
test reporting

3. Key Principles for Efficient Test Reporting

4. Key Principle: Stakeholder Expectations
• What info is suitable for the intended stakeholders?
– What info do they want?
– What info do they actually need?
– What decisions are being made based on the information?
• How do the stakeholders need the information reported to easily be able to
use it?
• What granularity do they need? High level overview or technical details?
• What data to they need to pass along in their consolidated reports for others
to make decisions on?
• Answers the question: What, how and why do we report?

5. Key Principle: Risk / Mitigation
• Why did we test at all?
• What were the goals of the test activities?
• What product or project risks initiated the test activity/activities that the report
covers?
• Did the testing mitigate the problems/risks?
• How can remaining risks be mitigated?
• Answers the question: Why did we test?

6. Key Principle: Confidence in Measurements
• It must be clear what the scope for the test activities were on different abstraction
levels so that stakeholders can get a good overview but also gradually go into more
detail
• The stakeholder must be able to understand how confident we are in our test
results
• If status is green, how good are the chances that it will stay green, and how much
risk is there that the status will change over time
• Are the test results reliable as decision material
• Answers the question: What did we test, and what didn’t we test?

7. Key Principle: Qualitative / Quantitative
• We need both data and an analysis of that data from a professional in the report
• The analysis leads to a statement about the product, and this statement needs to
be backed by empirical data
• The empirical data needs context that only a professional can put it in – the
product statement is this context
• One cannot be reported without the other
• If focus should be on the data or the analysis depends on the stakeholders
• Answers the question: How do we report?

8. Key Principle: Traceability in Results
• The results reported in the Test Report must be reproducible by stakeholders
• There must be traceability between the results and how it was executed
• Some or all of the following variables must be covered to create traceability
– Which tools were used
– How was the test environment configured
– Work process
– Links to test cases
– Links to defects found
– Links to logs and other detailed test artifacts
• Answers the question: How did we test?

9. Summary
• Using these key principles will not automatically give good test reporting, but not
following these key principles will most likely give insufficient test reporting
• Know your stakeholders, and understand the key principles of test reporting