The document summarizes notes from a testing workshop on the Redmine project management application. Participants tested various aspects of Redmine and identified 15 bugs, including issues with PDF exports not containing all data, character encoding problems in CSVs, ability to delete the only admin account, and other data validation and synchronization bugs. Workshop leaders provided hints and interrupts to prompt deeper investigation and testing of areas like HTTP traffic, DOM manipulation, and breaking file exports.
3. Introduction
● 3 * 5 minute lightning talks
● We expect you to test stuff
● We will coach & ask questions as you test
● We may periodically debrief
4. What you are about to test
● Redmine
○ www.redmine.org
● Project Planning App
○ GUI, Rest API, Feeds, DB, Web Server
5. Alan’s Bit at the start
● Model, Observe, Interrogate, Reflect,
Manipulate
● Tools help me observe and manipulate
● Note Taking
6. A model of how Alan tests
● Model
○ What I think I understand. Different viewpoints.
● Observe
○ at different points to corroborate/invalidate model
● Reflect
○ find gaps, lack of depth, derive intent
● Interrogate
○ Focussed, deep dive observation with intent
● Manipulate
○ Hypothesis exploration and “how we do stuff”
12. Note Taking
● Why:
○ Questions, Ideas, Risks, etc.
● What:
○ ToDos, Issues, Observations,
Notes, etc.
● When:
○ Timestamps, sequential order
● Where:
○ urls, environment, users, etc.
● How:
○ commands, methods,
tools used, etc.
Evidence:
● logs, screenshots, output, files,
etc.
13. Tony’s bit
● Test ideas
● Tools
● Information & Intelligence
14. Explore for test ideas
Prep
- Notes
- Summary
- Important bits
- Ideas
- Comments
- Questions
- Thoughts
- Six Honest Men
"I Keep Six Honest
Serving Men ..."
I KEEP six honest serving-men
(They taught me all I knew);
Their names are What and Why and
When And How and Where and
Who.
…….. Rudyard Kipling
18. Steve’s bit
What are we going to test?
● What’s new?
● What’s changed?
● What’s important?
● What are known buggy areas?
● What has not been tested previously?
23. Collaboration Rules
● Don’t load test the app, we are all using it
○ If you accidentally bring it down through a clever then that’s fine,
● Don’t change data you didn’t create
24. Where is the app?
● You can install it locally if you want
○ http://redmine.org
● You can get an install or VM from bitnami
○ https://bitnami.com/stack/redmine
● You can use the redmine demo
○ http://demo.redmine.org/
● You can use our server
○ ….
25. Where is the app?
● Links removed as
only valid at the
time we conducted
the workshop
26. Testing Phase 1
● Consider what we said
● Test the app in new ways, take notes, try
new tools
● Black Ops Team will mingle - do ask for help
● We will debrief prior to the break
27. “...no plan of operations extends
with any certainty beyond the first
contact with the main hostile force.”
Field Marshall Helmuth Carl Bernard Graf von Moltke, 1871
41. Bugs we found 1
● The PDF does not contain the Start Date,
Estimated Time, % Done or File Description
for the attachments that are included in the
New Issue form.
● The PDF does not contain the Target
Version and Spent Time values that are
included in the View Issue form.
42. Bugs we found 2
The Atom feed from the View Issue page has
no content when it is viewed immediately after
creating an issue. It did have content after
adding a quote to the issue.
44. Bugs we found 3
The File Description for an image is not saved if
too many characters are entered in the New
Issue form. We did not investigate where the
boundary is.
45. Bugs we found 4
The PDF that is generated from the Gantt page
always shows the default zoom level regardless
of the zoom level that has been selected.
The URL of the PDF link contains a ‘zoom’
parameter (which does nothing). Changing the
‘months’ parameter has the desired effect.
46. Bugs we found 5
Some non-Roman characters are displayed
correctly on all HTML pages but they are not
displayed in PDFs.
47. Bugs we found 6
Some non-Roman characters are not displayed
correctly in CSVs.
48. Bugs we found 7
You’re able to delete all user accounts,
including admin.
There is only 1 admin (as far as I could see)
Tell us how….
49. Bugs we found 7 cont.
Record browser traffic while deleting a account.
Find the delete POST
50. Bugs we found 8
"Your account has been activated. You can
now log in. "
System says I am already "logged in as eris"
and I am on my account page
Minor issue about wording
51. Bugs we found 9
Error message about emails already in use
when registering - privacy concern
“Email has already been taken”
52. Bugs we found 10
Maximum length of email is 60 chars but needs
to accept 254
53. Bugs we found 11
Can use an invalid language when registering a
user.
54. Bugs we found 12
Truncation on project identifier with no error or
warning message
i.e. create project with 255 char identifier -
truncated to 100
55. Bugs we found 13
When creating a project, the ID and name are
populated via javascript but if I change the
name then the identifier is not kept in sync.
56. Bugs we found 14
Can create an invalid enabled_modules entry
by submitting a module name which does not
exist when creating a project
57. Bugs we found 15
Change url to have csv or pdf views
System should respond differently to csv and
pdf on projects when GUI request rather than
an API request 406 is better for API, 404 with
html or 406 with html payload might be better
58. Rathole 1 - Password
Alan thought there was a bug with password
lengths, and storing in a varchar 40, since
password can be very long.
But, a ‘hash’ is stored, not the password, this
took time to discover.