This presentation from IVT's 4th Annual reviews what to do when you have an exception, critical vs. non-critical exceptions, and learning how to prevent exceptions.

1. Validation Exceptions – Managing
life’s little (and big) unexpected events
( g) p
Brian K. Nunnally, Ph.D.

2. What is an exception?
p
An error or a failure which occurs during
Verification or Validation
William Shakespeare: "It is neither good nor
bad, but thinking makes it so."

3. Types of Exceptions
yp p
Three levels of exceptions:
Critical
Non-critical
N iti l
Simple

4. Simple Exceptions
p p
Simple exceptions are documentation or
protocol errors
These errors are found prior to execution or are
Th f d i t ti
“obvious” errors or typos
These e ceptions ha e no impact on the
exceptions have
validation (e.g. no risk)

5. Simple Exceptions examples
p p p
Wrong test selected for product
Incorrect specification for product
Loss of power to the equipment being
tested.
Other examples from the participants?

6. Non-critical Exceptions
p
Non-critical
Non critical exceptions are errors in the
protocol or execution of the validation which
have no impact on the validation
These errors are found during or after execution
Assessment is needed t d t
A ti d d to determine no i
i impactt
Need to document the risk (or lack thereof)

7. Non-critical Exceptions
examples
l
Operator not trained to perform the operation
An interruption in the computer system or
the
th equipment’s performance
i t’ f
Other examples from the participants?

8. Critical Exceptions
p
Critical exceptions are errors which have an
impact on the validation
These errors are found during or after execution
Th f dd i ft ti
Assessment is used to determine impact
Need to document th risk
N dt d t the i k

9. Critical Exceptions examples
p p
Acceptance criterion failure
An interruption in the computer system or
the
th equipment’s performance
i t’ f
Other examples from the participants?

10. Basic Exception Process Flow
p
SME involvement throughout is
critical
You cannot document enough,
even if all of it does not make it
into the final event write-up
CAPA may not be needed for
simple events, but consider if
there are many of them or the
same ones repeatedly

11. General Process Considerations
Commence exception resolution process as soon
as event is observed
Involve SMEs/QA throughout process as
appropriate
Get agreement on next steps and/or results
g p
Prior to execution, or after the completion
Including all elements of the process
Immediately notify the relevant SME/QA if there is
any potential Product/Material impact
Conduct a risk assessment to determine impact
and actions

12. What is the risk and impact of
the exception?
Event severity x Testing Stage = Impact
Event Severity:
What kind of event is it? How was the event
caused?
Testing Stage:
Where are we in the validation/verification
process? Installation? Performance
Qualification?

13. Event Types
yp
Typographical errors
These errors are found prior to execution or are “obvious” errors or
typos
External Issues
Failures caused by factors external to the process or system under
test
Execution Errors
Failures to follow instructions within the Test Document or
referenced procedures.
Document Generation Errors
Incorrect detail in procedures, acceptance criteria or referenced
documentation
Acceptance Criteria Failures
Pre-defined acceptance criteria have not been achieved.

14. Testing Stage
g g
New system installation and the exception
does not impact the validation intent
New system i t ll ti and th exception
N t installation d the ti
does impact the validation intent
New or Existing system qualification
Process Validation
Re-qualification

15. Should testing stop?
g p
Depends on the impact score
Depends on consultation with SMEs and QA
Are you willing to live with a failure?
Will it open more q
p questions than it solves?
Note: It does not depend on the timeline!

16. Exception example #1
p p
An acceptance criterion was failed
failed.
However, the acceptance criterion was poorly set
(
(limited knowledge was available to set the acceptance
g p
criterion).
What level of exception is this?
What do you do?

17. Exception example #2
p p
The acceptance criterion was passed but the
passed,
result was right at the limit.
Assay variability is high for the test.
Is this an exception? Why or why not?
What do you do?

18. Exception example #3
p p
During validation of the cleaning process the
process,
Operator executing the protocol failed to sample
the correct locations.
Is this an exception? Why or why not?
What do you do?

19. Preventing Exceptions

20. The key to preventing
exceptions
ti
Understanding and reducing variation
variation.
Systems thinking (managing the flow through the
entire process as an entity)
entity).

21. Types of Variation (Including
Sampling,
Sampling Analytical and Production)
Variation in Quality Characteristic (specific activity
activity,
potency, pH, temperature…etc.
Process variation Variation in the way our
variation.
processes behave and are managed or controlled.
Variation in volume flowing through a process.

22. Why focus on variability?
y y
Reduced variation in Quality Characteristics
reduces rejects, rework and deviations. It
provides high First Pass Yields.
Reduced variation in Volume (smoother flow)
increases output capacity and reduces capital
requirements.
requirements
Reduced variation in Volume (smoother flow)
decreases cycle time and Work-In-Progress.
y g

23. Which one will be more likely to
fail an acceptance criterion?
Day shift Following night shift
2600
2400
2200
1-Point Average
2000
1800
1600
1400
1200
1000
50
70
90
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
49

24. More examples…
p
Potency
y
pH

25. Correlation?
2
Before Project
27 R = 0.2594
25
23
21
19
17
15
3000 4000 5000 6000 7000 8000
29 2
R = 0.9107
After Project
27
25
23
21
19
17
15
3000 4000 5000 6000 7000 8000

26. What should you do?
y
CRITICAL ERRORS PER WORKER PER WEEK
WEEK
1 2 3 4 5 6 7 8 9 10 11 12 TOTAL
Mary 0 0 0 0 0 0 0 0 0 0 0 0 0
Joe 0 0 0 0 0 0 0 0 0 0 0 0 0
Eva 1 0 0 2 0 0 3 0 0 1 0 0 7
Fred 0 0 0 1 0 0 2 0 0 0 0 0 3
Jim 0 0 0 0 0 0 0 0 0 0 0 0 0
Ed 0 0 0 0 0 2 0 0 0 0 0 0 2
Kate 0 0 0 0 0 0 0 0 0 0 0 0 0
Carl 0 0 0 1 0 0 0 0 0 0 0 0 1
TOTAL 13

27. The lesson from Tribus
Bring a team together to focus on ensuring the aim
is clear to all participants
Focus on systems to reduce and remove possibility
for errors
Improve systems as a way to prevent human error.
Be very careful when assigning blame to human
error or to a specific p
p person

28. Expectations on Exceptions

29. Three keys to exceptions
y p
Document
Document
Document!

30. Keep it simple!
p p
Document the standard questions:
Who
What
Wh t
When
Why
Where
How

31. Other requirements
q
Root Cause Determination
If root cause is unknown, a study should be
co duc ed
conducted
Impact Determination
How severe is the exception?
What is the impact to intent?
Corrective Action Determination
How will this be prevented in routine use?
Re Testing Re execution
Re-Testing / Re-execution
Is it necessary? Why or why not?

32. More exceptions examples…

33. Exception example #4
p p
During the annual re qualification / re-verification of
re-qualification re verification
the performance of an autoclave which has been
used for manufacturing of p
g product, the
,
thermocouples did not achieve the required
temperature.
Is this an exception? Why or why not?
What do you do?

34. Exception example #5
p p
During the verification of the installation of Vessel
which has not been used to manufacture product,
p
power was lost during testing of the agitator.
g g g
Is this an exception? Why or why not?
What do you do?

35. Exception example #6
p p
During the re validation / re verification of Cleaning
re-validation re-verification
validation of a process which has NOT been used
to manufacture product, some samples were
p , p
accidentally not taken.
Is this an exception? Why or why not?
p y y
What do you do?

36. Exception example #7
p p
During the execution of an alarm check
(operational verification) for a system not used to
manufacture product, the test document had the
p ,
incorrect sequence of events.
Is this an exception? Why or why not?
p y y
What do you do?