Find Requirements Defects to Build Better Software

BW6
Session
6/5/2013 2:15 PM

"Identifying Requirements Defects"

Presented by:
John Terzakis
Intel Corporation

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073
888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

John Terzakis
Intel

John Terzakis has more than twenty-five years of experience developing, writing, and testing
software. With Intel for fourteen years, John is currently a staff engineer working with teams on
enhancing product requirements to reduce planning and development times, minimize defects,
and improve overall product quality. He is a certified Intel instructor for Requirements
Engineering courses. John’s prior experience includes director and manager roles with Shiva,
Racal InterLan, and Dataproducts. He was also a member of the technical staff at Bell Labs.
John is a fellow with the International Academy, Research and Industry Association (IARIA).

Find Requirements Defects to
Build Better Software
John Terzakis
Intel Corporation
john.terzakis@intel.com
June 5, 2013
Better Software Conference
Las Vegas, NV

Version 1.0

Legal Disclaimers
Intel Trademark Notice:
Intel and the Intel Logo are trademarks of Intel Corporation in the U.S.
and other countries.
countries

Non-Intel Trademark Notice:
*Other names and brands may be claimed as the property of others

2

Copyright © 2013 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written
permission of Intel Corporation.

Agenda

• Defects and Their Impact on Software Projects
• Common Requirements Problems
• Techniques to Find Requirements Defects
• Examples
• Wrap up

3


Defects and Their Impact on Software
Projects

4


What is a Defect?
A defect is the result of some mistake during work product
creation
These mistakes come primarily from two sources:

• Lack of knowledge
• Lack of attention (usually due
to schedule pressure)
This session will focus on how to identify requirements
defects so they don’t propagate into the software
5


The Requirements Defect Problem
Architecture
Specifications
Design
Documents
Requirements

Code
Test Plans
Documentation
& Help Files

Requirements Defects Propagate Downstream
6


Requirements Defects
Requirements defects account for the vast majority of the
total cost of all defects – often 70% or more (Leffingwell &
Widrig, 2003)
• Requirements defects are often the most expensive
defects because requirements form the basis of so
many other work products
• Requirements defects account for up to 40% of many
projects’ total budget (Leffingwell & Widrig, 2003)

SW teams commonly spend too much time and money
developing the wrong thing!
7


Impact of Requirements Defects on Projects
Poor requirements accounted for 41-56% of errors
discovered, and 5 of the top 8 reasons for project failure
(The CHAOS Report, 1995)

IBM and Bell Labs studies show that 80% of all product
defects are inserted at the requirements definition stage
(Hooks and Farry, 2001)

Requirements errors consume from 28% to more than 40%
of a typical project's budget (Hooks and Farry, 2001)
122% average schedule overrun, 45% of delivered
functions never used (Standish Group Report, 1995)
8


Relative Cost to Correct a Defect

Phase
Inspection
Design & Coding
Testing
Production

Relative Cost (avg.)
1
10x
25x
100x or greater

Correcting defects earlier in the SW lifecycle pays huge dividends

9


Common Requirements Problems

10


Common Types of Defects in Requirements
Requirements contain defects because of the following
common problems:
• Missing information
• Missing triggers
• Implicit collections
• Lack of a consistent syntax
• Weak words
• Unbounded lists
• Ambiguity
Anything that causes the reader to guess is likely to produce a defect

11

Missing Information
Requirements that are missing information are incomplete.
They cause the reader to make their own interpretation of
intent of the requirement and thus introduce the potential
for f
f defects.
Examples:
• The software shall display a list of TBD users.
• The software shall comply with the latest Windows®
SDK.
S
Missing information may be due to schedule pressure or author
“laziness”
12


Missing Triggers
Requirements that state a fundamental property of the
software or occur all the time are called ubiquitous.
Most requirements are not ubiquitous. Requirements
written ubiquitously may actually be missing triggers
(events, states or optional features needed for them to
execute).
Examples:
• The software shall sound the alarm
alarm.
• The software shall blink the low battery light.
Under what states or triggers do these occur?
13


Lack of a Consistent Syntax
Requirements that lack a consistent syntax cause the
reader to potentially miss key information or misinterpret
the intent of the requirement.

Examples:
• An invoice shall be created when the customer places
an order.
g
• The software shall turn on the light when there is an AC
power failure condition provided the “visible error
indicator” option has been selected.

14


Implicit Collections
Implicit collections are sets of objects within requirements
are not explicitly defined anywhere
Without a definition readers may assume an incorrect
definition,
meaning
Example:
• The software must support 802.11 and other network
protocols supported by competing applications under
Linux.


15

Weak Words
Weak words are subjective or lack a common or precise
definition. Examples include:
• Quick, Quickly

• Normal

#1 overused weak word:

• Easy, Easily

• Reliable

• S
Support
t

• Timely

• State-of-the-art

• Fast

• Effortless

• Frequently

• Friendly, User-friendly

• Intuitive

• Secure

• Feel, Feeling

• Immediate

Example:
• The software shall install quickly and effortlessly under normal
conditions using a user-friendly and intuitive UI.
16


Unbounded Lists
An unbounded list is one that lacks a starting point, an end
point, or both. Examples include:
• At least
• Including, but not limited to
• Or later
• Such as

Unbounded lists are impossible to design for or test against
Examples:
p
• The SW shall must maintain a list of at least 250 users
• The SW shall install on Windows® 7 or later in under 5
seconds

17

Ambiguity
Ambiguity occurs when a word or statement has multiple
meanings or there is doubt about the meaning.
These problems create ambiguity:
•
•
•
•
•
•

Vagueness
Subjectivity
Incompleteness
Optionality
Under-specification
Under-reference

•
•
•
•
•
•

Over-generalization
Non-intelligibility
Coordination ambiguity
Passive voice
Time-logic confusion
Incomplete logic
See Backup slides for examples

18


Ambiguity Exercise

What does “The bank is green” mean to you?

The bank is
green (paint
color)
www.shutterstock.com

The bank is
green (ecofriendly)
www.fumare.com

The pool table
bank is green
(the felt color)

. www.on.aol.com

19

The river bank is
green (lush).

www.debtshepherd.com


Techniques to Find Requirements
Defects

20


Techniques for Finding Requirements Defects

The following techniques can be used to find
requirements defects:
• Ensure requirements use a common Requirements
Syntax
• Test requirements against a checklist of Attributes of
Well Written Requirements
• Test requirements using an Ambiguity Checklist
• Ensure that Non-Functional Requirements (NFRs) are
Testable
• Test for Missing Triggers

21

Common Requirements Syntax

[Trigger] [Precondition] Actor Action [Object]
Example:

When an Order is shipped and Order Terms are not
“Prepaid”, the system shall create an Invoice.
•
•
•
•
•

Trigger: When an Order is shipped
Precondition: Order Terms are not “Prepaid”
Actor: the system
Action: create
Object: an Invoice

Identify requirements that do not adhere to the common syntax
22


Checklist for Well-Written Requirements
Use a checklist that defines the attributes or qualities that a
Well-Written Requirement (WWR) must posses:
Complete
Correct
Concise
Feasible
Necessary

Prioritized
Unambiguous
Verifiable
Consistent
Traceable

Most of these attributes apply equally to a single
requirement and the entire set of requirements

23


Complete

A requirement i “
i
t is “complete” when it contains sufficient
l t ” h
t i
ffi i t
detail for those that use it to guide their work
Every gap forces designers and developers to guess –who do you
want specifying your product?

24


Correct

A requirement is correct when it is error-free
error free
Requirements can be checked for errors by stakeholders &
Subject Matter Experts (SMEs)
Requirements can be checked against source materials for errors
Correctness is related to other attributes – ambiguity,
consistency,
consistency and verifiability

25


Concise

A requirement is concise when it contains just the needed
information, expressed in as few words as possible
Requirements often lack conciseness because of:
•Compound statements (multiple requirements in one)
•Embedded rationale, examples, or design
•Overly-complex grammar
Overly complex

26


Feasible

A requirement i f
i
t is feasible if there is at least one design
ibl
th
i tl
t
d i
and implementation for it
Requirements may have been proven feasible in previous products
Evolutionary or breakthrough requirements can be shown feasible at
acceptable risk levels through analysis and prototyping

27


Necessary

A requirement is necessary when at least one of the
following apply:
• It is included to be market competitive
• It can be traced to a need expressed by a customer, end user, or
other stakeholder
• It establishes a new product differentiator or usage model
strategy, roadmaps,
• It is dictated by business strategy roadmaps or sustainability
needs

28


Prioritized
A requirement is prioritized when it is ranked or ordered
according to its importance.

All requirements are in competition for limited resources. There are
many possible ways to prioritize:
• Customer Value, Development Risk, Value to the Company,
Competitive Analysis, Cost, Effort, TTM
Several scales can be used for prioritization:
• Essential, Desirable, Nice to Have
• High, Medium, Low
• Other ordinal scales based on cost, value, etc.
29


Unambiguous

A requirement is unambiguous when it possesses a
single interpretation
Ambiguity is often dependent on the background of the reader
Reduce ambiguity by defining terms, writing concisely, and testing
understanding among the target audience
Augment natural language with diagrams, tables and algorithms
to remove ambiguity and enhance understanding

30


Verifiable

A requirement is verifiable if it can be proved that the
requirement was correctly implemented
Verification may come via demonstration, analysis, inspection, or
testing.
Requirements are often unverifiable because they are ambiguous,
can’t be decided, or are not worth the cost to verify.

31


Consistent

A requirement is consistent when it does not conflict
with any other requirements at any level

Consistency is improved by referring to the original statement
where needed instead of repeating statements.

32


Traceable

A requirement is traceable if it is uniquely and
persistently identified with a Tag
Requirements can be traced to and from designs, tests, usage
models, and other project artifacts.
Traceability enables improved
• Change impact assessment
• Schedule and effort estimation
• Coverage analysis (requirements to tests, for example)
• Scope management, prioritization, and decision making

33


Ambiguity Checklist
Utilize a checklist that defines and helps identify ambiguity
in requirements:
Vagueness
Subjectivity
Incompleteness
Optionality
Underspecification
Under-reference

34

Over-generalization
Non-intelligibility
Coordination
ambiguity
Passive voice
Time-logic
Time logic confusion
Incomplete logic


Ambiguity Checklist Definitions
• Vagueness: is caused by weak words without a precise meaning
• Subjectivity: is caused by weak words that rely on personal experience
or opinion

• Incompleteness: comes from insufficient detail, use of TBD, and
unbounded lists

• Optionality: is caused by use of should, may, if possible, when
appropriate, etc. (Note: use shall and must instead)

• Under-specification: results from use of verbs such as support,
analyze, or respond, or implicit collections of objects
• Under-reference: consists of incomplete or ambiguous references to
other documents, standards, requirements, etc.
35


Ambiguity Checklist Definitions
• Over-generalization: is caused by use of universal qualifiers such as
all or every, and even unmodified nouns like users.
• Non-intelligibility: results from poor grammar, complex logic, “and/or”
ambiguity, ambiguous negation or enumeration, and missing definitions
• Coordination Ambiguity: results from use of a conjunction between a
modified noun and a pure noun (among other cases)
• Passive Voice: occurs when a requirement does not explicitly name an
actor
• Ti
Time-Logic Confusion: exists when a l i l condition i used i
L i C f i
i t h
logical
diti is
d in
place of time-related language
• Incomplete Logic: refers to missing logical conditions, such as missing
the else of an if-then
36


Ensure NFRs are Testable
Non-functional requirements are testable if they contain a
Scale, Meter and Goal. We can ensure that the NFR has
been implemented correctly using these properties.
Scale: The scale of measure used to quantify the
statement
Meter: The process or device used to establish location
on a Scale
Goal: minimum level required to achieve success
Identify NFRs missing any of these properties
37


Test for Missing Triggers
Most legitimate ubiquitous requirements state a fundamental
property of the software:
• The software shall be distributed on CD-ROM and DVD media.
• The software shall prevent Unauthorized Access to patient data
data.

Question requirements that appear to be ubiquitous by
looking for unstated triggers or preconditions:
• The software shall wake the PC from standby
• The software shall log the date, time and username of failed logins.

The last two requirements are missing triggers
38


Finding Defects in Requirements:
Examples

39


Example 1

The LED should be on while data is being read from the DVD.

Defect Finding Technique

Pass or Fail? Issue(s)

Common Syntax

Fail

Location of “while data…”

10 Attributes

Fails 9 of 10

Not complete, not correct, etc.

Ambiguity Checklist

Fails 3

Vagueness, passive voice,
optionality
ti
lit

Testable NFR

N/A

Missing Trigger

Pass

40


Example 2

When the Restaurant Locator app is selected, the app must
quickly display a list of the TBD nearest restaurants.


Common Syntax

Pass

10 Attributes

Fails 10 of 10

Not correct, not verifiable, etc.

Ambiguity Checklist

Fails 3

Incompleteness, subjectivity,
vagueness

Testable NFR

Fail

What does “quickly” mean?

Missing Trigger

Pass

41


Example 3

#_of_Alternatives: The thesaurus software shall display at
least five alternatives for the selected word using all sources.


Common Syntax

Pass

10 Attributes

Fails 8 of 10

Not feasible, not verifiable, etc.

Ambiguity Checklist

Fails 3

Incompleteness , underreference, over-generalization
f
li ti

Testable NFR

N/A

Missing Trigger

Fail

42


Example 4

The software shall support measured or manual dispensing of
water if user mode is selected. Priority: High


Common Syntax

Fail

Location of “if user mode…”

10 Attributes

Fails 8 of 10

Not complete, not verifiable, etc.

Ambiguity Checklist

Fails 3

Under specification, nonintelligibility,
intelligibilit incomplete logic

Testable NFR

N/A

Missing Trigger

Pass

43


Example 5

Rx_Access: Only authorized nurses and doctors shall have
access to the patient’s prescription drug history.
patient s


Common Syntax

Pass

10 Attributes

Fails 8 of 10

Ambiguous, not verifiable, not
prioritized, etc.

Ambiguity Checklist

Fails 2

Coordination ambiguity passive
ambiguity,
voice

Testable NFR

N/A

Missing Trigger

Pass

44


Example 6

If three unsuccessful logins are detected, the software shall
log/report the event to the administrator.


Common Syntax

Pass

10 Attributes

Fails 10 of 10

Not complete, not correct, etc.

Ambiguity Checklist

Fails 3

Incompleteness, non-intelligibility,
time-logic confusion
ti
l i
f i

Testable NFR

N/A

Missing Trigger

Pass

45


Wrap up

46


Session Summary
In this session we have:

• Defined requirements defects and shown their impact
on software projects

• Provided examples of common requirements problems
f
• Introduced techniques to help find requirements
defects

• Analyzed examples using the techniques discussed

47


Final Thoughts
• Requirements defects lead to cost and schedule overruns
Requirement defects can proliferate into designs, tests and code.

• Focus on requirements defect prevention
The earlier a defect is detected, the less it costs to fix.

• Use checklists and rules to find requirements defects
They can provide good feedback to authors on the exact nature of the
requirements defects.

“An ounce of prevention is worth a pound of cure”

48


Contact Information
Thank You!

For more information, please contact:

John Terzakis
john.terzakis@intel.com

49


Backup

50


Examples of Ambiguity (1 of 3)
• Vagueness
The software must support all current standards for video encoding
before launch.

• Subjectivity
The software must be able to easily and seamlessly transfer media
between connected devices.

• Incompleteness
The software must support at least 50 concurrent users.

• Optionality
The software should include as many end-user help mechanisms as
possible.
possible

• Under-specification
The software must support 802.11a, b, g, and other network protocols
supported by competing applications.

51


• Under-reference
Users must be able to complete all previously-defined operations in
under 5 minutes 80% of the time.
• Over-generalization
Over generalization
All users must be able to delete all data they have entered.
• Non-intelligibility
The software shall report/log improper access attempts and notify
administrators if a user does not respond to warning messages or lock
out the account.
• Coordination Ambig it
Ambiguity
The software shall allow automated updates and deletions.
The software must display categorized instructions and help
documentation.
52


• Passive Voice
When shipping information has been verified, shipping labels must be
printed for each container in the order.
• Time/Logic Confusion
Conf sion
If two orders are received from the same customer for the same part,
the software shall follow the process described below.
• Incomplete Logic
When automatic calibration fails, the software shall switch to manual
calibration.

53


10 Attributes of a WWR: Examples (1 of 5)
Not Complete:
The software must allow a TBD number of incorrect login
attempts.
Complete:
When more than 3 incorrect login attempts occur for a
single user ID within a 30 minute period, the software
shall lock the account associated with that user ID.
Not Correct:
The 802.3 Ethernet frame shall be 2048 bytes or less.
Correct:
The 802.3 Ethernet frame length shall be between 64 and 1518
Th 802 3 Eth
tf
l
th h ll b b t
d
bytes inclusive.

54


Not Concise:
The outstanding software written by the talented development
team shall display the current local time when selected by the
intelligent and educated user from the well designed menu.
Concise:
when selected by the user from the menu, the software shall
display the current local time.
Not Feasible:
The software shall allow an unlimited number of concurrent
users.
Feasible:
F
ibl
The software shall allow a maximum of twenty concurrent users

55


Not Necessary:
The software shall be backwards compatible with all prior versions
of Windows®
Necessary:
The software shall be backwards compatible with Windows® Vista
SP2 and SP1, and Windows XP SP3.
Not Prioritized:
All requirements are critical and must be implemented.
Prioritized:
80% of requirements High, 15% Medium and 5% Low.

56


Ambiguous:
The software must install quickly.
Unambiguous:
When using unattended installation with standard options, the
software shall install in under 3 minutes 80% of the time and under
4 minutes 100% of the time.100% of the time.
Not Verifiable:
The manual shall be easy to find on the CD-ROM.
Verifiable:
The manual shall be located in a folder named User Manual in the
root directory of the CD-ROM.
t di t
f th CD ROM

57


Inconsistent:
#1: The user shall only be allowed to enter whole numbers.
#2: The user shall be allowed to enter the time interval in seconds
and tenths of a second.
Consistent:
#1: The user shall only be allowed to enter whole numbers except
if the time interval is selected.
#2: The user shall be allowed to enter the time interval in seconds
and tenths of a second.
Not Traceable:
The ft
Th software shall prompt th user f th PIN
h ll
t the
for the PIN.
Traceable:
Prompt_PIN: The software shall prompt the user for the PIN.

58


Find Requirements Defects to Build Better Software

Find Requirements Defects to Build Better Software

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (18)

Similar a Find Requirements Defects to Build Better Software

Similar a Find Requirements Defects to Build Better Software (20)

Más de TechWell

Más de TechWell (20)

Último

Último (20)

Find Requirements Defects to Build Better Software