1. 1
Proficiency Testing (PT)
– a tool to improve laboratory performance
Science
for a safer world
3rd GCC Laboratory Proficiency Conference
Dubai, 24-25 March 2015

2. Overview
PT as a quality tool
Purpose, types and benefits of PT
PT participation strategy
Selecting fit for purpose PT schemes
Interpretation of PT Results

3. 3
Proficiency Testing
- as a quality tool

4. Measurements
Measurement
Health
Security
Trade
Compliance Decisions
Reliable
measurements
Reliable decisions

5. Decision on
measurement
Report on
measurement
The measurement cycle
Client issue
Client
Define issue
Client
Sampling
Analysis
Analyst
Interface
Decision on result
Evaluation
Data evaluation
AnalystValidated
procedures

6. Method
Validation
Calibration
Training
IQC
Quality Assurance System
?
Measurement
System
Input Output
RMs PT

7. Fit-for-purpose analysis requires:
Careful development and validation
of methods
Accurate calibration of instruments
and appropriate equipment
Qualified and competent staff
Comparability with measurements
made in other laboratories
Well defined QC and QA procedures
Independent evidence of performance
CRMs
Traceable Standards
RMs & PT
RMs
RMs & QC
PT

8. 8
Proficiency Testing
- purpose, types and benefits

9. Purpose of PT
The primary aim of proficiency testing is:
“To provide the infrastructure for a laboratory to monitor and
improve the quality of its routine analytical measurements”
• PT provides an objective evidence of the competence of
the participant which can be used to
– Improve the performance of the participant
– Give confidence in the participant’s ability to perform a
specific measurement
• PT provides information on other aspects of the
management system
– Reception/treatment of the sample, treatment of the data,
result reporting etc

10. Types of PT
• Various types of PT available – based on one or more
elements of four different categories:
Qualitative
Quantitative
Interpretive
1
Descriptive – nominal or ordinal scale
Numeric – interval or ratio scale
No measurement involved

11. Types of PT
• Various types of PT available – based on one or more
elements of four different categories:
Qualitative
Quantitative
Interpretive
Pre-measurement
Measurement
Post-measurement
1 2

12. Types of PT
• Various types of PT available – based on one or more
elements of four different categories:
Qualitative
Quantitative
Interpretive
Sequential
Simultaneous
Pre-measurement
Measurement
Post-measurement
1 32

13. Monitor
Performance
Report
Sequential participation
scheme structure
PT
Provider
Test
Item
Test
Result
Test
Result
Test
Result
Monitor
Performance
Monitor
Performance
Lab 1
Lab 2
Lab 3
PT
P
r
o
v
i
d
e
r
Evaluation
of
Results
Lab 1
Lab 2
Lab 3
NMi

14. Simultaneous scheme structure
PT
Provider
Test
Items
Participating
Laboratories
Test
Results
PT
Provider
Evaluation
of
Results
Report
Participating
Laboratories
Monitoring
Performance

15. Types of PT
• Various types of PT available – based on one or more
elements of four different categories:
Qualitative
Quantitative
Interpretive
Single
Continuous
Sequential
Simultaneous
Pre-measurement
Measurement
Post-measurement
1 32 4

16. Use of PT by laboratories
• Basic use of PT for a laboratory is:
– to assess its performance for the conduct of specific
measurements or calibrations
• Results and information received from participation will
provide laboratories with:
– Confirmation that the laboratory’s performance is
satisfactory
– An indication that there are potential problems and that
corrections should be made
• However, the use of PT should be much wider
– Many additional benefits from PT participation

17. Benefits of PT to laboratories
Benefits
of PT
Third Party
Demonstration
Method/ Procedure
Evaluation
Performance
Comparison
Monitoring
Trends
Independent
Check
Training and
Education

18. Maximising the benefits of PT
• To obtain the optimum benefit from PT participation,
laboratories should:
– treat PT scheme test items as routine samples (i.e. do not
give them any special treatment)
– vary the analyst testing the test items
– vary the instrumentation used for the test items
• To learn from PT participation, laboratories should:
– use the scheme results to give feedback to laboratory
procedures and staff; both positive and negative
– use the scheme results constructively
– be honest; the scheme is there to help
– remember that PT is education not regulation

19. 19
Proficiency Testing
- Participation Strategy

20. Strategy of PT participation
• All laboratories need to develop an adequate PT
participation strategy
• A laboratory should decide on an appropriate level and
frequency of participation
– Level: number of specific proficiency tests in which to participate
– Frequency: How often the laboratory will participate in each of
the specific test identified
• This will be specific to the laboratory’s circumstances
• Guidance provided by the European co-operation for
Accreditation (EA) EA-4/18: 2010
– Referenced in ILAC-P9: 2010

21. Level of PT Participation
• Consider areas of technical competence based on:
• Measurement technique e.g. ICP-MS, GC-MS
• Property to be measured e.g. PAHs, Fat, hardness
• Products to be tested e.g. soil, milk, human serum
• An area of technical competence may encompass
several products, properties and/or measurement
techniques
• The laboratory must be able to demonstrate equivalence
within each area of technical competence
21

22. Level and frequency of PT participation
• Important to consider a number of different aspects
• Other QA measures implemented:
- Use of CRMS
- Comparisons by independent techniques
- Participation in validation or
characterisation studies
- Use of internal QC measures
- Other laboratory comparisons
• Level of risk:
- No measurements undertaken
- Turnover of technical staff
- Staff experience/knowledge
- Known stability/instability of
measurement technique
- Significance and final use of data
•Different types of PTs can be used
• Difficulty of PT participation e.g. technical characteristics of the measurement,
low number of laboratories etc
• Legislative requirements for frequency of type of participation

23. 23
Proficiency Testing
- selecting fit for purpose schemes

24. Fit for purpose PT schemes
• Need to decide which scheme is fit for purpose
– Often a scheme will not have a perfect fit, so need to find the
best fitness for purpose
• A number of factors to consider
– Test Items
– Distribution
– Participants
– Results
– Reports
– PT Provider

25. Selection factors
• Test Item
– Matrix (real/simulated)
– Parameters
– Concentrations
• Distribution
– Dates available
– Frequency
– Flexibility
• Participants
– National/International
– Language
– Methods
• Results
– Deadlines
– Reporting mechanism
– Statistical approach
• Reports
– Speed
– Information/Language
– Format
• PT Provider
– Experience
– Scope
– Competence

26. 26
Proficiency Testing
- Interpretation of results

27. Performance evaluation
• Limited value unless performance is evaluated
• Evaluate after each round and also over time
• Interpretation of PT performance concerns all
management levels
• PT is not about “passing” or “failing” – it is about learning
from the results

28. Review of single PT round
performance
• Results of each PT round should be evaluated
regardless of performance
– A satisfactory result may not mean good performance
– Unsatisfactory performance in one round does not indicate
that the laboratory is not competent
• Important to consider a number of factors:
– The overall results in the round
– Test method performance
– Test item factors
– Bi-modal result distribution
– PT Scheme factors

29. Trends in PT performance
• Trends are valuable in preventive and corrective actions,
and root cause analysis
– Performance over time
– Common groups of determinands
– Performance of methods
– Performance of analysts

30. Need for an investigation
• All laboratories will occasionally have unsatisfactory or
questionable PT results
• A laboratory should establish its own criteria for when an
investigation is necessary
• The depth of investigation will depend on a number of
factors:
– Criticality of the method
– Frequency of unsatisfactory results
– Evidence of a bias

31. Causes of unsatisfactory performance
• Can be analytical and non-analytical
• Both are equally serious
– the result reported is the important factor for a laboratory
• Analytical Errors
– Calibration
– Instrument problems
– Extraction/clean-up
– Interferences
– Method performance
– Analyst
• Non-Analytical Errors
– Calculation
– Transcription
– Units
– Reporting format

32. 32
Proficiency Testing
- in conclusion

33. PT – powerful quality tool
PT is an essential tool to all laboratories making
measurements
PT participation brings many benefits
PT enables measurements to be monitored and
improved
PT is a very powerful quality assurance tool
underpinning laboratory accreditation

34. PT - drives laboratory quality
PT Reliable
Measurements
Reliable
Decisions