Más contenido relacionado La actualidad más candente (19) Similar a Control model testing (20) Control model testing1. Matthew Sullivan
MANAGING RISK FOR Scott Barber
S o f t wa r e Te s t
SOFTWARE PRODUCTS Professionals
Conference
Fa l l 2 01 1
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
2. “STATE OF THE S/W TESTING PRACTICE”
• Find bugs (identify risks) OR
“Role” of • Check for compliance (V&V)
QA/Testing
• Appears undervalued, BUT
“Value” of • Doesn’t provide nearly the value it
QA/Testing could
• Business goals & value propositions
QA/Testing is • Business risks & risk controls
“out of sync” with • Executive information needs
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
3. “THE UNDER-INFORMED DIRECTING THE
UNDER-TRAINED TO DO THE UNIMPORTANT”
Executives Testers Artifacts
(the Uninformed): (the Untrained) (the Unimportant)
•Don’t know how to ask •Don’t know what the •Bugs no one wants to fix
for what they need, SO executives need, SO •Metrics no one
•They ask for what they •They do what they are understands
know asked to •Documents no one
reads
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
4. IMPROVING THE SITUATION (PART 1)
Focus on:
•Delivering business value
•Reducing business risk
At every business layer, identify & balance:
•Responsibility
•Accountability
Get your superiors to read Ch 16:Rightsizing the Cost of Testing:
Tips for Executives of How to Reduce the Cost of Software Testing;
CRC Press 2011
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
6. FEELING UNDER SIEGE?
Businesses reduce
allocation of
resources to
testing because of
a perception of
diminished value.
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
7. WHAT DIMINISHES VALUE FOR TESTING?
1. Lack of insight into
future
2. Redundancy
3. Specification
blocks
4. Lack of
independence
5. Scope
constraint
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
8. LACK OF INSIGHT INTO THE FUTURE
Why didn’t this
come up in
testing!
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
9. REDUNDANCY
Sign here, and then sign
the next box attesting to
the authenticity of the
previous signature.
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
10. SPECIFICATION BLOCK
Honestly I’d love to start testing today, but
first I need detailed requirements. VERY
detailed requirements
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
11. LACK OF INDEPENDENCE
Its not fun being
the captain’s “no-
man”.
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
12. SCOPE CONSTRAINT
Someone else was
supposed to be watching
for icebergs.
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
14. THE MEANING OF LIFE (FOR TESTERS)
The purpose of
testing is to reduce
uncertainty about
the future impact
of technology.
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
16. RISK AS A COMMON LANGUAGE
Security
Whether explicitly
or implicitly, all
Compliance Functional forms of testing
Risk revolve around the
reduction and
management of
Usability Performance
risk.
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
17. THE SECRET TO MANAGING RISK
To effectively
manage risk, you
must effectively
manage
knowledge.
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
18. WHAT IS CONTROL MODEL TESTING?
Control Model Testing
is a business-aligned
approach to software
testing that derives
“test cases” from
knowledge models
of the system based on
a risk-based
taxonomy .
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
19. WHAT IS OUR TAXONOMY BASED UPON?
COSO
Enterprise Risk
Management Integrated
Framework
The Open Group
Technical Standard on
Risk Taxonomy
PerfTest Plus
Taxonomy Extensions for
Control Model Testing
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
20. WHAT ARE THE BASIC ENTITIES?
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
21. THE OPEN GROUP’S RISK ASSESSMENT
FRAMEWORK
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
22. RISK LAYERS
Business
• Financial
• Legal
• Brand or Reputation
Product
• Security
• Performance
• Usability
• Other Qualities
Project
• Budget
• Schedule
• Communication
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
24. HOW CAN TESTS ADDRESS THREATS AND
LEVEL OF RISK?
Controls prevent or
mitigate risk which
may impact business
objectives.
Control Model Testing
helps identify and
assess these controls.
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
25. T YPES OF CONTROLS
Systems
• Firewalls
• Encryption
• Load Balancing
Preferences
• Settings
• Security and Access Model
Policies
• Code Standards
• Monitor and Response
• HR
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
26. CONTROLS CONTEXT
Development
• Development and Test Tools
• Code standards
• Software components
Implementation
• Checklists
• Installation scripts
Maintenance
• Alerts and Triggers
• SOPs
• Configuration Management
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
27. “SAMSARIC” TEST LIFECYCLE
Analyze
Report Assess
Effort
Evaluate
Knowledge
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
28. ANALYSIS
Examine
• System
• Users
• Environment
Identify
• Objectives
• Processes
• Threats
• Controls
Output
• Initial Control Model
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
32. EVALUATION
Activities
•Execute planned and
derivative tests
•Identify discrepancies
•Determine capability
Outcomes
•Tested Control Model
•Test results
•Issues /
recommendations
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
35. THE FOUR ROLES IN CONTROL MODEL
TESTING
Leader
Manager
Coordinator
Tester
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
36. LEADER
Responsibilities:
• Representation
• Roadmaps
Interests
• Information
• Certainty
Talents
• Communication
• Vision
Typical Business Titles
• Director of Testing or Quality Assurance
• Chief Audit Officer (or Assistant to..)
• Principle Consultant
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
38. COORDINATOR
Responsibilities
• Planning
• Oversight
Interests
• Successful outcome
• Thoroughness
Talents
• Teamwork
• Attention
Typical Business Titles
• Test or QA Lead or Senior
• Analyst or Engineer Level 2 or 3
• Manager 1
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
40. RISK LAYERS AND ROLES
Business
Product
Test
Leader Project
Test
Manager
Test Coordinator Tester
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
41. SUMMARY
Testing should be an indispensible advisor for leadership
Testing should not be a convenience or scapegoat for
development
All types of testing revolve around risk management
The key to managing risk is managing knowledge
Testing needs to be a learning discipline in the context of risk
taxonomy
The test process should be a continuous cycle reducing ef fort
through increased knowledge
Testing roles should correlate to management or risk, not
resources
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
43. RECOURCES
The Open Group (http://www3.opengroup.org/):
Risk Taxonomy Technical Standard -
https://www2.opengroup.org/ogsys/jsp/publications/PublicationDetail
s.jsp?publicationid=12156
The Committee of Sponsoring Organizations of the Treadway
Commission, or COSO (http://www.coso.org/)
Enterprise Risk Management-Integrated Framework -
http://www.coso.org/ERM-IntegratedFramework.htm
PerfTestPlus, Inc. (http://www.perftestplus.com/)
Control-Model Testing – (http://www.perftestplus.com/control-model-
testing)
Rightsizing the Cost of Testing: Tips for Executives of How to Reduce
the Cost of Software Testing; CRC Press 2011
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.
44. ABOUT US
Matthew Sullivan Scott Barber
Quality Control Engineer CTO, PerfTestPlus, Inc
CCH TeamMate
Wolter s Kluwer Widely regarded exper tise in
per formance.
Test and Suppor t Engineer for
PricewaterhouseCooper s for 10 Contributor to:
year s Performance Testing Guidance for Web
Applications– Microsoft Press
Extensive experience in audit Beautiful Testing- O’Reilly Press
and risk management industr y
How to Reduce the Cost of Testing-
Specialist in testing Microsof t Taylor and Francis
.NET, MS SQL Ser ver, and Lotus Executive Director of the
Notes applications
Association for Sof tware Testing
MS in Sof tware Engineering
from Regis University Co-Founder of the Workshop of
Per formance and Reliability
Copyright © 2011 PerfTestPlus, Inc. All rights
reserved.