Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Reporting Bugs:
Errors Made and Lessons Learned
Peter Sabev,
QA Manager,
Questers / MIRACL
On average,
every single
bug report
is read by
at least
5 people!
Examine and
approve the bug
Set priority and
assign to d...
But
sometimes
the bug
report is
read dozen
times!
QA Failed
Works
fine on
my
machine.
Fixed. Try
now.
QA Failed
Fixed. Try...
Investing just
3 more minutes
in your bug report
may save hours
to the entire team.
Problem 1: FAIL Big Time
Correct
BUILD,
VERSION
and CONFIG
(Automated)
Sanity Tests
< 5 min
1. Check AC
2. Compiles
3. Installs
4. Runs
5. Unit Tes...
Problem 2:
Already found
Problem 3:
Are you able to reproduce the bug?
The story
about the six
men and the
dark room
Problem 4:
“It doesn’t work!”
Which one is better?
“I feel unwell”
OR
“I have headache, runny nose
and 38.5° C temperature
since yesterday”
Anamnesis Bug Description
1. Name
2. Age, height, weight
3. Family and childhood history
4. Allergies, work environment
5....
Which one is better?
“I feel unwell”
OR
“I have headache, runny nose
and 38.5° C temperature
since yesterday”
Short But Enough Description
One line should say it all:
Canceling a Copy dialog crashes File Manager
Software crashes
Steps to Reproduce
The most important part!
1. Press Cmd+N to open a new browser window
2. Type https://mail.google.com in...
Expected vs. Actual Result
Instead of showing my inbox, it shows the
message “Your browser does not support
cookies (error...
Problem 5:
Setting priority and severity
PriorityVery High
Fix immediately and ASAP, absolutely must fix in 24h.
Notify!!!
High
Must be fixed until the next releas...
SeverityBlocker (Showstopper)
Serious crash, testing completely blocked, security, data loss
Critical (Severe)
Incorrect c...
QA team sets severity and
proposes priority
Product team can amend
priority later.
Priority / Severity Mapping Script
•If both are set, good.
•If one is set, map the other to it.
•If none is set, disallow ...
Problem 6:
Incorrect Environment
Environment / Versions
Include information about the
environment and all software related
to the project you are testing.
Semantic versioning (semver.org)
MAJOR.MINOR.PATCH.EXTENSION
incrementing
•MAJOR when you make incompatible API changes,
•...
Problem 7:
Not showing them how to reproduce it
Providing screenshots and videos
•Good bug report has at least 1 screenshot (or video), but only
if appropriate.
•Logs and...
Problem 8:
Investigate!
Computer is working slowly.
I am using 8-core i7-5960X, and I saw CPU
0 utilization is 99.9% while the others...
Problem 9:
Being too scientific
(Especially valid for developers)
API request to CDN forces refetch from
origin Azure Storage that invalidas CC
header set...
Problem 10:
Stop before it’s too late
Know when to stop
DBData was accidentally deleted.
DBData was accidentally deleted.
I tried reinstalling the software and
...
Problem 11:
Be able to restore!
Always have a backup plan
Use virtual machines or containers, create
snapshots and backups, and revert when
needed.
Test [...
Problem 12:
Finding the best balance
Distinguish:
1. Facts
What is ACTUALLY happening.
2. Observations
What YOU THINK is happening.
Doctors don’t want patients...
Problem 13:
Formatting your reports
Windows
Calculator:
2+2=5
Description
Add headings and sections
Bullet and numbers in front of items
If the order of items is important, use a
numbered list.
Otherwise, use a bulleted li...
Minify and be consistent
Use bold, italic and monospaced
Use traffic lights logic
GREEN – for good things
RED – for bad things
Just compare now…
Even Stamps!!!
Even
app?
If you are good in HTML…
Tested on the following environment:
192.168.12.23
CPU: Intel Xeon E5-2670
RAM: 8 GB
OS: Windows ...
Problem 14:
Make it look fast and easy
Use present tense
1. Click “Send” button.
2. Note that asterisks determine
mandatory
1. Be assured you clicked “Send”
butt...
To maintain a file, click File | Open.
To edit a file, click File | Open.
Use precise words and actions.
System restart can be requested via
Restart.
To restart the system, choose Restart.
Address the user directly. This holds ...
Win10 Home should have blue icon,
however, the icon for Windows 10
Professional is purple.
Win10 Home should have blue ico...
Choose Delete to delete the text.
To delete the text, choose Delete.
If required by the system, order information
chronolo...
Delete the file if the disk is full.
If the disk is full, delete the file.
If a sentence has one condition, place the
cond...
Test creating text, editing text and
formatting text.
Test the following features:
● Create text
● Edit text
● Format text...
When you delete an item from the table,
all of its assigned values are deleted.
Whose values? Table’s or system’s ones?
Wh...
Check the device adapter card port
signals.
Check the port signals from the device
adapter card.
If possible, avoid noun s...
Introduction Initial introduction
Outcome Final outcome
Because The reason is because
To In order to
Avoid redundancies.
usually, often, virtually, apparently, in
some ways, to a certain extent, for all
intends and purposes, in general,
genera...
Problem 15:
Practice Together
Peter Sabev
QA Manager
MIRACL / Questers
psabev@gmail.com @BORIME4KA
Reporting bugs: Errors Made and Lessons Learned
Reporting bugs: Errors Made and Lessons Learned
Reporting bugs: Errors Made and Lessons Learned
Reporting bugs: Errors Made and Lessons Learned
Reporting bugs: Errors Made and Lessons Learned
Reporting bugs: Errors Made and Lessons Learned
Reporting bugs: Errors Made and Lessons Learned
Próxima SlideShare
Cargando en…5
×

Reporting bugs: Errors Made and Lessons Learned

624 visualizaciones

Publicado el

How to report bugs effectively, what mistakes to avoid, how to make your defect reports easily readable by developers, software quality assurance engineers and everyone involved in software testing

Publicado en: Software
  • Sé el primero en comentar

Reporting bugs: Errors Made and Lessons Learned

  1. 1. Reporting Bugs: Errors Made and Lessons Learned Peter Sabev, QA Manager, Questers / MIRACL
  2. 2. On average, every single bug report is read by at least 5 people! Examine and approve the bug Set priority and assign to developer Fix the bug Update test casesTest the fix
  3. 3. But sometimes the bug report is read dozen times! QA Failed Works fine on my machine. Fixed. Try now. QA Failed Fixed. Try now.QA Failed (again) Need more information You must have the wrong version.
  4. 4. Investing just 3 more minutes in your bug report may save hours to the entire team.
  5. 5. Problem 1: FAIL Big Time
  6. 6. Correct BUILD, VERSION and CONFIG (Automated) Sanity Tests < 5 min 1. Check AC 2. Compiles 3. Installs 4. Runs 5. Unit Tests Entry Criteria QADev
  7. 7. Problem 2: Already found
  8. 8. Problem 3: Are you able to reproduce the bug?
  9. 9. The story about the six men and the dark room
  10. 10. Problem 4: “It doesn’t work!”
  11. 11. Which one is better? “I feel unwell” OR “I have headache, runny nose and 38.5° C temperature since yesterday”
  12. 12. Anamnesis Bug Description 1. Name 2. Age, height, weight 3. Family and childhood history 4. Allergies, work environment 5. Details about the complaints 6. Habits 7. Medical encounters 8. Other useful info 1. Short description / Title 2. Priority / Severity 3. Build / Version 4. Environment 5. Steps to reproduce 6. Expected vs. Actual Results 7. Screenshots / Logs 8. Other useful Info
  13. 13. Which one is better? “I feel unwell” OR “I have headache, runny nose and 38.5° C temperature since yesterday”
  14. 14. Short But Enough Description One line should say it all: Canceling a Copy dialog crashes File Manager Software crashes
  15. 15. Steps to Reproduce The most important part! 1. Press Cmd+N to open a new browser window 2. Type https://mail.google.com in the address bar 3. Press Enter Open Gmail in another window
  16. 16. Expected vs. Actual Result Instead of showing my inbox, it shows the message “Your browser does not support cookies (error -91)”. It doesn't work.
  17. 17. Problem 5: Setting priority and severity
  18. 18. PriorityVery High Fix immediately and ASAP, absolutely must fix in 24h. Notify!!! High Must be fixed until the next release cycle Medium Fix as part of standart SDLC. Must be fixed before GA. Low Could live with it, schedule when time is available. Very Low Nice to have. May never fix.
  19. 19. SeverityBlocker (Showstopper) Serious crash, testing completely blocked, security, data loss Critical (Severe) Incorrect calculations, missing features, problems with no workaround, far from expected, unable to test partially Major (Serious) Functionality not working but able to test system Normal (Moderate) Not meeting criteria, bad UI/UX, missing images/msgs Very Low Very minor UI/UX or cosmetic issues, typos, proposals
  20. 20. QA team sets severity and proposes priority Product team can amend priority later.
  21. 21. Priority / Severity Mapping Script •If both are set, good. •If one is set, map the other to it. •If none is set, disallow publishing the bug report.
  22. 22. Problem 6: Incorrect Environment
  23. 23. Environment / Versions Include information about the environment and all software related to the project you are testing.
  24. 24. Semantic versioning (semver.org) MAJOR.MINOR.PATCH.EXTENSION incrementing •MAJOR when you make incompatible API changes, •MINOR when you add functionality in a backwards- compatible manner, and •PATCH when you make backwards-compatible bug fixes.
  25. 25. Problem 7: Not showing them how to reproduce it
  26. 26. Providing screenshots and videos •Good bug report has at least 1 screenshot (or video), but only if appropriate. •Logs and/or stack traces should be provided only if they are related to the defect and can be used as a source of information. •If you think there are other sources of valuable information about the defect, include these sources as well. •Give more information rather than less.
  27. 27. Problem 8:
  28. 28. Investigate! Computer is working slowly. I am using 8-core i7-5960X, and I saw CPU 0 utilization is 99.9% while the others are idle. The process that uses most CPU time is SearchIndexer.exe
  29. 29. Problem 9: Being too scientific
  30. 30. (Especially valid for developers) API request to CDN forces refetch from origin Azure Storage that invalidas CC header set for the blob on ARM when BSP is enabled together with ZDF. Old website logo is loaded instead of the rebranded one
  31. 31. Problem 10: Stop before it’s too late
  32. 32. Know when to stop DBData was accidentally deleted. DBData was accidentally deleted. I tried reinstalling the software and defragmented the HDD but nothing helped.
  33. 33. Problem 11: Be able to restore!
  34. 34. Always have a backup plan Use virtual machines or containers, create snapshots and backups, and revert when needed. Test [on production] without having way to go back if something goes wrong.
  35. 35. Problem 12: Finding the best balance
  36. 36. Distinguish: 1. Facts What is ACTUALLY happening. 2. Observations What YOU THINK is happening. Doctors don’t want patients to tell them their diagnosis
  37. 37. Problem 13: Formatting your reports
  38. 38. Windows Calculator: 2+2=5
  39. 39. Description
  40. 40. Add headings and sections
  41. 41. Bullet and numbers in front of items If the order of items is important, use a numbered list. Otherwise, use a bulleted list.
  42. 42. Minify and be consistent
  43. 43. Use bold, italic and monospaced
  44. 44. Use traffic lights logic GREEN – for good things RED – for bad things
  45. 45. Just compare now…
  46. 46. Even Stamps!!!
  47. 47. Even app?
  48. 48. If you are good in HTML… Tested on the following environment: 192.168.12.23 CPU: Intel Xeon E5-2670 RAM: 8 GB OS: Windows 7 64-bit Google Chrome Ver 52.0.2743.116 192.168.12.240 Version: 1.0.4-rc3 SDK: 1.0.0 Apache: 2.0.63 PHP: 5.2.9 OS: Ubuntu 14.04.1 LTS 192.168.12.223 DB: MySQL 5.0.92 Records: 283 Users: 3 Submissions: 4322 Last submission: A12.xml User: TestAdmin23
  49. 49. Problem 14: Make it look fast and easy
  50. 50. Use present tense 1. Click “Send” button. 2. Note that asterisks determine mandatory 1. Be assured you clicked “Send” button. 2. Asterisks will determine mandatory fields.
  51. 51. To maintain a file, click File | Open. To edit a file, click File | Open. Use precise words and actions.
  52. 52. System restart can be requested via Restart. To restart the system, choose Restart. Address the user directly. This holds the user's attention.
  53. 53. Win10 Home should have blue icon, however, the icon for Windows 10 Professional is purple. Win10 Home should have blue icon Win10 Pro should have purple icon. Use consistent abbreviations and parallel sentence structure.
  54. 54. Choose Delete to delete the text. To delete the text, choose Delete. If required by the system, order information chronologically. Order information according to the logic what before how. Enter the address, then page forward. Page forward after entering the address.
  55. 55. Delete the file if the disk is full. If the disk is full, delete the file. If a sentence has one condition, place the condition before the action.
  56. 56. Test creating text, editing text and formatting text. Test the following features: ● Create text ● Edit text ● Format text Avoid including a long series of items in a sentence. Use a list or table instead.
  57. 57. When you delete an item from the table, all of its assigned values are deleted. Whose values? Table’s or system’s ones? When you delete an item from the table, the system deletes all values that are assigned to this item. Avoid vague or ambiguous pronominal references.
  58. 58. Check the device adapter card port signals. Check the port signals from the device adapter card. If possible, avoid noun strings, that is, a series of nouns that modify one another.
  59. 59. Introduction Initial introduction Outcome Final outcome Because The reason is because To In order to Avoid redundancies.
  60. 60. usually, often, virtually, apparently, in some ways, to a certain extent, for all intends and purposes, in general, generally, in some respects, for the most part, occasionally, most often, as a rule, probably Avoid imprecise statements. If you cannot make a concrete statement because of exceptions, then list these exceptions.
  61. 61. Problem 15: Practice Together
  62. 62. Peter Sabev QA Manager MIRACL / Questers psabev@gmail.com @BORIME4KA

×