Oplægget blev holdt ved et seminar i InfinIT-interessegruppen Processer & IT Nord den 11. juni 2014. Læs mere om interessegruppen her: http://infinit.dk/dk/interessegrupper/processer_og_it/processer_og_it.htm

1. Copyright © 2007 – 14 Callis
Making Your
’Peer Reviews Happen!
Callis Reviewer
2014

2. Copyright © 2007 – 14 Callis
Rightshifting Organizational Performance
Callis Performer Callis Reviewer
Organize, author, tailor, follow, and
improve business processes
Implement, sustain and monitor
technical reviews of “documents”
Tie into
planning &
tracking!
Take the
pain away!
Make the
invisible
visible!
Author
consistent
processes
Clear &
accessible
descriptions
Tailor and
profile
usage

3. Copyright © 2007 – 14 Callis
Why Technical Reviews?

4. Copyright © 2007 – 14 Callis
Why are (Technical) Reviews interesting?
Peer reviews are the only technique available to
detect defects in life-cycle work products in the early
phases of development
Peer reviews are a reliable way of measuring the quality of the
work that has been performed
Peer reviews reduces cost and time in later phases of
development, e.g. test and integration
Peer reviews reduces total system cost dramatically

5. Copyright © 2007 – 14 Callis
From Capers Jones:Software ”Engineering
Best Practices”
s125: "Most forms of testing are less than 35
percent efficient in finding errors or bugs. The
measured defect removal efficiency of both
formal design inspections and formal code
inspections averages more than 65 percent
efficient, or twice as efficient as most forms of
testing."
s601: "Formal inspections are so effective in
terms of defect prevention that they reduce
defect potential by more than 25 percent per
year. In fact one issue with inspections is that
after about three years of continuous usage,
so few defects occur that inspections become
boring"

6. Copyright © 2007 – 14 Callis
Pre-project
Early
Planning
Preliminary
Design
Final Design Construction Acceptance
Define Implement Accept
Top-level Development Life Cycle
Scrum Based Planning and Tracking Sprint
Week
Your Business Depends on Reviews!
Business Case
Approval
Project
Defined
Preliminary
Design Review
Critical
Design Review
Integration
Complete
System
Approved
LIfe- Cycle Related Reviews
Operational
Concept Review
Project Planning
Review
Requirements
Review
Architecture
Review
Detailed Design
Review
Test Case
Review
Acceptance Test
Review
Plan
Sprint N+1Sprint N
Schedule
Design, build, test feature
Review feature
descriptions
Review sprint
plan
Review feature
design Review test
cases
Review UI
design
Review sprint
results
Review user
manual update

7. Copyright © 2007 – 14 Callis
Key Design Criteria

8. Copyright © 2007 – 14 Callis
Often seen issues
When and how to apply reviews are not defined/operational
Data registration is cumbersome and provides only indirect
value
Issues are not systematically tracked to closure
Limited visibility into review participation, scheduling and
results
Limited tool support means low efficiency and high barrier
Often, many of these issues can
be traced back to poor tool
support / implementation

9. Copyright © 2007 – 14 Callis
Key Design Criteria
Take the pain away!
• Automate, automate,automate…
Tie into planning and tracking!
• Ensure the “when” and “how” to apply review methods
Make the invisible visible!
• Use visualizations to drive review execution
We took out the pain of performing reviews by
automating and optimizing
all the trivial parts of the process -
unleashing energy to find and remove defects!

10. Copyright © 2007 – 14 Callis
Review ReportIndicators
Acrossfindings,
Reviews,projects
Findings into Word / Excel / PDF for correctíon
Add-Ins to support complete review workflow
Review Templates support Q planning
Pre-
planning
Take the Pain Away 1#3
1: Plan – Roles & docs. 2: Prepare – Register findings
3: Conclude – Consolidate findings4: Correct – Remove defects, verify
Callis
Reviewer
Server
Created in-application
using Callis Add-ins
Created by parsing
- Word comments
- PDF comments
Created manually from
-Callis Reviewer Web
”Bulk-processing of findings”
In-application
display of findings
Findings overview
Plan Prepare
ConcludeCorrect

11. Copyright © 2007 – 14 Callis
Take the Pain Away 2#3!
Callis
Reviewer
Server ExcelImages
PDFWord
Callis AddIn:
Native Word comments:

12. Copyright © 2007 – 14 Callis
AddIns for Word & Excel
The complete review workflow supported
Take the Pain Away 3#3!
Mark, Create, Classify
Bulk processing – Export, update, and upload findings using Excel
Back into Word – Insert findings as native Word comments for easy correction
Click and get the
Review Report (PDF)

13. Copyright © 2007 – 14 Callis
Tie into Planning and Tracking 1#3
Each RT defines
A review method
A number of artifacts types
A number of roles
A number of checklist
templates
Review
Template
Review
Method
Artifacts types Roles Checklist
templates
Critical
design review
Inspection • Design doc (Man.)
• Reqs. doc (Man.)
• Contract (Rec.)
• Architect (Acc.)
• Designer (Res.)
• Engineer (Sup.)
• Design
complete
checklist
Test script
review
Desk
Check
• Test script (Man.)
• Test mgmt. Plan (Rec.)
• Tester (Resp.)
• Test Mgr. (Acc.)
• Test script
check
Release
review
Walk-
through
• Product (Man.)
• User Manual (Man.)
• Project
Manager (Res.)
• Release ready
checklist
Benefits
More specific guidance to
review planners and review
participants
Better ways to monitor review
status and progress
Example:

14. Copyright © 2007 – 14 Callis
Real Life Example: Review Matrix
Template
Name
Review
Method
M: Mandatory
R: Recommended
Roles
Artifact
TypesReview
Template

15. Copyright © 2007 – 14 Callis
Project X’s
Quality
Assurance
Plan
Review
Template
Review
Method
Artifacts types Roles Checklist
templates
Critical
design
review
Inspection • Design doc (Man.)
• Reqs. doc (Man.)
• Contract (Rec.)
• Architect (Acc.)
• Designer (Res.)
• Engineer (Sup.)
• Design
complete
checklist
Test script
review
Desk
review
• Test script (Man.)
• Test. Plan (Rec.)
• Tester (Resp.)
• Test Mgr. (Acc.)
• Test script
check
Release
review
Walk-
through
• Product (Man.)
• User Manual (Man.)
• Project
Manager (Res.)
• Release ready
checklist
Tie into Planning and Tracking 2#3
The Review Matrix
Generate
Close the
loop
Callis
Reviewer
Server
Configure
Correct
PreparePlan
Conclude
Execute Reviews

16. Copyright © 2007 – 14 Callis
Schedule
Design, build, test feature
Backlog
Features
to
implement
Plan activities
Sprint N+1
Tie into Planning and Tracking 3#3
Review
Plan & StatusPlanned
review
activites
Persons &
schedule
Sprint N
Status &
findings
Review
Matrix
Which
review
activities?

17. Copyright © 2007 – 14 Callis
Make the invisible visible!
Timeline of project review activities
Time out this axis
Review Status Report
(Covering one or more projects)
Review
instance
Detailed
review info
CLICK

18. Copyright © 2007 – 14 Callis
Three Important Learnings
Implement a mechanism where:
20% / 50% / 80% … reviews are encouraged
• This leaves effort for “major corrections of course”
Checklists are an integrated part of the reviews
• This increases defects detected and builds-in a “learning loop”
Review activities and participation are visible
• This makes it easy to ask the “right questions” and increases
“accountability/responsibility”

19. Copyright © 2007 – 14 Callis
Product Road Map
In progress
• Uploading files as alternative to referencing files
Next step
• Better filtering before importing comments into review object
• More indicators/metrics / project level overview page
Longer term ideas (not committed)
• Integrate closer to SharePoint, e.g. to automatically store Review
Reports
• AddIns for other tools, e.g. PowerPoint, Visio

20. Copyright © 2007 – 14 Callis
Thank you for your interest!
For further information and demos
• pvp@callis.dk