Annals of Clinical Biochemistry: An international journal of biochemistry and laboratory medicine http://acb.sagepub.com/
Practical, transparent prospective risk analysis for the clinical laboratory Pim MW Janssens Ann Clin Biochem 2014 51: 695 originally published online 19 February 2014 DOI: 10.1177/0004563214521160 The online version of this article can be found at: http://acb.sagepub.com/content/51/6/695
Published by: http://www.sagepublications.com
On behalf of: Association for Clinical Biochemistry and Laboratory Medicine
Netherlands Society for Clinical Chemistry and Laboratory Japan Society of Clinical Chemistry
Additional services and information for Annals of Clinical Biochemistry: An international journal of biochemistry and laboratory medicine can be found at: Email Alerts: http://acb.sagepub.com/cgi/alerts Subscriptions: http://acb.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav
>> Version of Record - Oct 20, 2014 OnlineFirst Version of Record - Feb 19, 2014 What is This?
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
Original Article Annals of Clinical Biochemistry 2014, Vol. 51(6) 695–704 ! The Author(s) 2014 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/0004563214521160 acb.sagepub.com
Practical, transparent prospective risk analysis for the clinical laboratory Pim MW Janssens
Abstract Background: Prospective risk analysis (PRA) is an essential element in quality assurance for clinical laboratories. Practical approaches to conducting PRA in laboratories, however, are scarce. Methods: On the basis of the classical Failure Mode and Effect Analysis method, an approach to PRA was developed for application to key laboratory processes. First, the separate, major steps of the process under investigation are identified. Scores are then given for the Probability (P) and Consequence (C) of predefined types of failures and the chances of Detecting (D) these failures. Based on the P and C scores (on a 10-point scale), an overall Risk score (R) is calculated. The scores for each process were recorded in a matrix table. Based on predetermined criteria for R and D, it was determined whether a more detailed analysis was required for potential failures and, ultimately, where risk-reducing measures were necessary, if any. Results: As an illustration, this paper presents the results of the application of PRA to our pre-analytical and analytical activities. The highest R scores were obtained in the stat processes, the most common failure type in the collective process steps was ‘delayed processing or analysis’, the failure type with the highest mean R score was ‘inappropriate analysis’ and the failure type most frequently rated as suboptimal was ‘identification error’. Conclusions: The PRA designed is a useful semi-objective tool to identify process steps with potential failures rated as risky. Its systematic design and convenient output in matrix tables makes it easy to perform, practical and transparent.
Keywords Laboratory testing process, prospective risk analysis, risk management, FMEA, clinical laboratory Accepted: 2nd January 2014
Introduction Product quality for clinical laboratories means on-time delivery of the correct analysis results and products, for the right patient, to the right doctor. This requires sound validation, employee training and quality assurance with respect to the analytical activities. It also requires that attention be paid to the processes involved in both the pre-analytical and analytical phases, for example by means of risk analysis.1–6 As a tool to identify and prevent errors in processes, risk analysis is more suitable as a form of quality management than
the well-known and widely used LEAN approach, which primarily revolves around logistic-economic factors in processes. The well-known Six Sigma approach, on the other hand, is primarily intended to count and control known non-conformities, not so much to newly Laboratory of Clinical Chemistry and Haematology, Rijnstate Hospital, The Netherlands Corresponding author: Pim MW Janssens, Laboratory of Clinical Chemistry and Haematology, Rijnstate Hospital, PO Box 9555, 6800 TA Arnhem, the Netherlands. Email: [email protected]
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
696
Annals of Clinical Biochemistry 51(6)
identify them. Retrospective risk analysis is well known in clinical laboratories, and is used to identify the causes of failures. In contrast, prospective risk analysis (PRA)4–6 is driven not by incidents that have actually occurred, but by the examination of laboratory processes and their individual steps from an independent, more distant perspective to identify potential incidents in advance. Ideally, all main processes within a clinical laboratory should be subjected to PRA. Depending on the type of laboratory, these include the pre-analytical process, the analytical process (on site or elsewhere, for tests that the laboratory does not carry out itself), the process for obtaining urgent (‘stat’) test results, the provision of blood products for transfusion or the processing and issuing of gametes for fertility treatment. To date, the insight into and application of PRA in clinical laboratories is still rather limited, despite the fact that PRA is mentioned in various guidelines regulating the organisation of clinical laboratories.6–11 For instance, clauses 4.11 and 4.14 of ISO directive 15189 stipulate that clinical laboratories should conduct risk assessments to examine potential failures in their processes,9 alluding to the investigation of activities and processes in a preventive manner by way of PRA. Sound and convenient methods for executing PRAs are therefore desirable. The few methods described for executing PRAs in clinical laboratories, however, are either somewhat vague or rather complex and laborious.4–6,12,13 This article describes a practical and transparent approach to PRA that is suitable for laboratory processes, and presents the results of four of our PRAs to serve as illustration.
is made as to whether measures need to be taken. Elsewhere, we described an application of FMEA for the fertility laboratory which is situated in our department.13 In comparison, the PRA method described in the present paper purposefully used a less detailed approach,13 with processes divided into broader steps. In addition, a simple matrix was introduced to capture the risk scores. These improvements allow for a better overview of the processes and the results obtained. To conduct the PRAs, the steps in each process were first inventoried, resulting in an outline in the form of a flow diagram (Figure 1). The flow diagrams provide an overview of each process and come in handy when documenting potential failures step by step. PRAs were conducted for the pre-analytical and analytical processes for both the regular and stat processing of laboratory tests. These processes have multiple ‘entries’ and ‘exits’ in our laboratory, giving rise to flow diagrams with multiple arms, each of which were described (provided they were of some importance). Each process step in the flow diagram was given a brief description and a code that was subsequently used in the score matrix (Figure 2). Prior to conducting the PRAs, we identified in a general sense the basic potential failures (also referred to in the literature as non-conformities, failure modes, incidents and errors) that can occur in the processes. We decided on a limited number of failures, so as to maximise clarity when performing the PRAs. The potential failures across laboratories are largely consistent,5 but some variation can occur depending on the process considered, the personal choices of the individuals performing the PRA and the responsible staff. Our PRAs focused on the failure types:
Methods Study site
1. 2. 3. 4. 5. 6.
The PRAs described were performed in the Clinical Chemical and Haematological Laboratory at Rijnstate Hospital, Arnhem, the Netherlands, a 900bed general care and teaching hospital. The laboratory department provides blood sampling services and laboratory testing around the clock for approximately 300 analytes for inpatients and outpatients, patients in primary care and a number of affiliated institutes.
Design of the risk analysis approach Our PRA is an elaboration of the established Failure Mode and Effect Analysis (FMEA).6,14,15 In FMEA, a process is divided into separate steps. The potential failures identified per step each receive a Risk score (R). On the basis of this score and the chance of identifying failures after they have occurred, an assessment
identification error; mixing up or loss of material; incorrect sample labelling, processing or handling; incorrect execution of analysis; incorrect reporting or product delivery; and delayed execution.
Failure types 1 to 5 mainly relate to content. Failure type 6 primarily concerns the factor time, although time may also be involved in failure types 3 to 5. Space and environmental factors as reflected in issues such as temperature, noise, commotion, equipment, distances, power facilities, etc., can affect any time- and contentrelated aspect of the process, so play a role in all six failure types. For the sake of convenience, no additional failure types specific to the factors time or space/environment were introduced, as they were considered to be sufficiently addressed in the other failure types. To keep the scoring matrix uniform and universally applicable to all laboratory processes, specific
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
Janssens
697 (a)
CP010
CP011
CP012
CP013
order entry in HIS by clinic
order entry in HIS by outpatient dept.
order entry by general practitioners
order entry from hard-copy forms in LIMS by lab dept.
CP030 producing overview + labels for sampling by lab
CP020
CP021
informing patient on requirements
sample collection by non-lab pers. Ordering on paper forms
CP031 producing sample labels by external or clinical dept.
CP032 producing sample labels by lab dept.
CP040
CP041
CP042
sample collection by lab in clinic, by stat employee
sample collection by clinical dept.
sample collected at lab desk (for outpatients)
CP050 sample transport by pneumatic tube system
CP060 sample registration in LIMS by lab dept.
CP070 transfer of samples to site in lab for analysis
CP999 to CA001 for stat analysis
Figure 1. Process description in the form of flow diagrams of (a) the stat pre-analytical process and (b) the stat analytical process in our laboratory or elsewhere.
headings were not deleted from the scoring table in cases where a specific failure type did not exist, but rather left blank (e.g. ‘incorrect execution of analysis’, which does not apply to pre-analysis, or ‘product delivery’, which applies not to the analytical but rather to the blood transfusion and fertility laboratory processes in our department).
Execution of the risk analysis The PRAs were conducted, amid open consultation and discussion, by groups of four to seven members. In our experience, the group should neither be so large as to
prevent efficient discussion and progress, nor too small, to guarantee the presence of sufficient knowledge and experience. One group member had experience performing PRAs, some members had a good overview of the process concerned and clinical laboratories in general, and several were regularly involved in executing the process. For each potential failure in a process step, an inventory was made of the following: . the Probability (P) of a certain failure occurring over time (also referred to as likelihood), scored on a scale of 0–10 (0 ¼ almost never or impossible, 1 ¼ less than once a year, 2 ¼ once a year, 3 ¼ several times a year,
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
698
Annals of Clinical Biochemistry 51(6)
(b)
CA010 sample storage, waiting for analysis
CA020 manual analysis
CA001 step following CP999
CA021
CA022
automated analysis (lab robots)
analysis elsewhere (external)
CA030
CA031
CA032
judging + confirmation of result
judging + confirmation of result
judging + confirmation of result
CA040
CA041
CA042
manual transfer of results to LIMS
automatic transfer of results to LIMS
manual transfer of results to LIMS
CA050 sample storage
CA060 CA070
data confirmation in LIMS
reporting results from LIMS to general practitioners
CA071
CA072
phoning results to practitioners, on request
phoning extreme results (initiative of lab)
CA080 progress control
CA090 manual autorisation of abnormal results by lab doctor
CA100 definite rapporting of results
CA110 billing
Figure 1. Continued.
4 ¼ once a month, 5 ¼ several times a month, 6 ¼ once a week, 7 ¼ several times a week, 8 ¼ each day, 9 ¼ several times a day, 10 ¼ many times a day). . the Consequence (C) (also referred to as the impact) of a failure, scored on a scale of 0–10. The degree of the consequence is assessed in light of the potential damage for patients (health risk, inadequate or
ineffective treatment, discomfort), although serious consequences for employees are also taken into account in the higher ratings (i.e. 8 and higher) (0 ¼ no problem, 1 ¼ practically no problem, 3 ¼ minor problem that can be solved by staff, 4 ¼ somewhat inconvenient, 5 ¼ inconvenient, 6 ¼ discomfort, 7 ¼ serious discomfort, 8 ¼ injury to patient or
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
Janssens (a) Pre-analysis stat
699 Identification error
Code and step in process P C P*C D CP010 2 4 8 3 Order entry in HIS by clinic CP011 1 4 4 3 Order entry in HIS by outpatient dept. CP012 1 4 4 3 Order entry in ordering syst. by general practitioners CP013 1 4 4 3 Order entry from hard-copy order forms in LIMS by lab dept. CP020 na na na Informing patient on requirements for investigations and ordering CP021 Sample collection by non-lab pers. with lab ordering on hard-copy 1 4 4 3 forms (GPs, external relations, etc.) CP030 Production of overview of patients plus sample labels for sampling by na na na lab CP031 na na na Production of sample labels by external or clinical department CP032 na na na Production of sample labels by laboratory CP040 1 8 8 2 Sample collection by laboratory in clinic (by stat employee) CP041 1 8 8 2 Sample collection by clinical department CP042 1 8 8 2 Sample collected at lab desk (for outpatients) CP050 na na na Transportation of samples by pneumatic tube system CP060 na na na Registration of samples in LIMS by laboratory CP070 na na na Transfer of samples to analysis site in laboratory
Number of steps rated Overall Risk score, mean Overall Risk score, s.d. Probability of detection, mean Probability of detection, s.d. Number of steps scored suboptimal (marked orange or red)
(b)
Number of steps rated: Overall Risk score (mean): Overall Risk score (s.d.): Probability of detection (mean): Probability of detection (s.d.): Number of steps scored suboptimal (marked orange or red):
C na
P*C
na
na
3
8
24
1
8
3
8
C
P*C D
P
C
P*C D
P
C
P*C
D
Inappropriate reporting or product provided P C P*C D
na
na
na
na
na
na
na
1
7
7
3
na
na
na
na
na
na
na
na
na
na
na
na
1
7
7
3
na
na
na
na
na
na
na
na
na
na
na
na
1
7
7
3
na
na
na
na
na
na
na
na
na
na
na
na
1
7
7
3
na
na
na
0
1
0
3
1
8
1
na
na
na
2
7
14
3
1
3
1
3
3
3
na
na
na
na
na
na
1
7
7
3
na
na
8
3
3
na
1
3
3
3
na
na
na
na
na
na
2
7
14
3
na
na
na
1
3
3
3
na
na
na
na
na
na
1
7
7
3
na
na
na
1
3
3
3
na
na
na
na
na
na
1
7
7
3
na
na
na
2
5
10
3
na
na
na
na
na
na
4
7
28
3
na
na
na
2
5
10
3
na
na
na
na
na
na
2
7
14
3
na
na
na
2
5
10
3
na
na
na
na
na
na
1
7
7
3
na
na
na
2
5
10
3
na
na
na
na
na
na
1
7
7
3
na
na
na
1
3
3
3
na
na
na
na
na
na
2
7
14
3
na
na
na
2
5
10
3
2
5
3
na
na
na
1
7
7
3
na
na
na
10
Per failure type 2 9,0 1,4 2,0 1,4 0
11 5,9 4,0 3,0 0,0 0 Loss of sample, incl. collection errors
P
P
Delayed processing or analysis
Inappropriate analysis
na
Identification error
na
Inappropriate sample processing
na
8 6,0 2,1 2,6 0,5 0
Analysis stat (analysis in own laboratory or elswhere) Code and step in process CA001 See CP999, Pre-analysis stat CA010 Sample storage, waiting for analysis CA020 Manual analysis CA021 Automated analysis (lab robots) CA022 Analysis elsewhere (external) CA030 Judging (potential rerun) and confirmation of result CA031 Judging (potential rerun) and confirmation of result CA032 Judging and confirmation of result CA040 Manual transfer of results to LIMS CA041 Automatic transfer of results to LIMS CA042 Manual transfer of results to LIMS CA050 Post-run transfer of samples to sample storage CA 060 Data confirmation in LIMS CA070 Reporting of results from LIMS to GPs CA071 Phoning of results to practitioners, on request CA072 Phoning extreme results to practitioners (on initiative of lab) CA080 Progress control CA090 Manual authorisation of abnormal results by lab physician CA100 Definitive reporting of results CA110 Billing
Loss of sample, incl. collection errors P C P*C D
Inappropriate sample processing
D
P
C
P*C
D
P
C
na
2
8
16
3
na
na
na
2
8
16
3
2
8
2
2
8
16
3
3
8
8
3
2
8
16
3
1
24
2
3
8
24
3
3
P*C
0
15 10,3 5,8 3,0 0,0 1
0
D
P
C
na
na
na
P*C
16
2
na
na
24
2
2
8
16
8
8
2
2
8
8
24
1
2
8
D
P
C
P*C
D
P
C
2
7
14
2
na
na
na
2
7
14
2
na
na
2
2
7
14
2
3
7
21
16
2
2
7
14
2
2
7
14
3
16
1
3
7
21
2
3
7
21
2
21
7
na
na
na
na
na
na
na
na
na
na
na
na
3
7
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
2
8
16
2
na
na
na
na
na
na
na
na
na
2
7
14
0
8
0
3
na
na
na
na
na
na
na
na
na
1
7
2
8
16
2
na
na
na
na
na
na
na
na
na
2
3
8
24
1
3
4
2
3
5
2
na
na
na
2
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
1
8
8
1
na
na
na
na
na
na
na
na
na
1
8
8
1
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
15
0 Inappropriate reporting or product provided
na
na
12
3,0
Delayed processing or analysis
Inappropriate analysis
Total process 37 7,8 4,9 2,9 0,4 1
1 3,0
P*C
D na na 2
3
1
7
na
na
na
2
na
na
na
2
2
7
14
2
7
3
0
7
0
3
7
14
2
3
7
21
3
6
2
na
na
3
7
21
2
1
7
1
7
7
2
na
na
3
7
21
2
1
7
7
5
7
35
2
1
7
7
1
na
2
7
14
2
1
7
7
2
12
na na
2 na
7
2 na 1
na
na
na
na
na
na
na
na
na
na
na
na
3
3
9
2
2
6
na
na
na
na
na
na
na
na
na
na
na
na
1
4
4
3
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
1
3
3
3
na
na
na
9 14.2 8.7 1.9 0.8 7
6 16.7 3.9 2.8 0.4 0
Per failure type 5 17.4 6.8 1.8 0.4 3
3 16.0 0.0 1.7 0.6 3
18 14.1 7.8 2.2 0.4 4
12 11.5 6.9 1.9 0.7 6
1
Total process 53 14.2 7.1 2.1 0.6 0
na: not applicable s.d.: standard deviation
Figure 2. PRA of (a) stat pre-analytical process and (b) stat analytical process in our laboratory or elsewhere. Abbreviations: P: Probability; C: Consequence; D: chance of Detection; na: not applicable. The codes CP and CA in column 2 for each process step refer to the relevant steps in the flow diagrams (Figure 1). Boxes marked in grey are those for which risk scoring is not applicable.
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
700
Annals of Clinical Biochemistry 51(6)
someone else, 9 ¼ severe injury to patient or someone else, 10 ¼ catastrophic, severe injury with permanent and/or life threatening consequences). . the chance of Detecting (D) a failure, scored on a scale of 1–3 (1 ¼ low, 2 ¼ medium, 3 ¼ high). The scores were obtained by consensus among the group members, after debate where necessary. We recommend that each laboratory establish its own definitions for each score, as these may depend on the local conditions and culture as well as the process considered. To estimate the overall Risk score (R), the Probability is multiplied by the Consequence of the failure; in other words, R ¼ P C, on a scale of 1–100. The P, C, R and D scores were recorded in a matrix table created in Excel (Figure 2).
Processing of the PRA results and evaluation The overall Risk scores (R) were automatically calculated in the Excel table from the estimated Probability and Consequence of the different failure types (Figure 2). The R scores were evaluated in combination with the chance of Detecting the failure (D) as follows: Acceptable: . R max. 5, low D . R max. 15, medium D . R max. 25, high D Unacceptable: . R > 30, low D . R > 40, medium D . R > 50, high D Suboptimal: . R 5–30, low D . R 15–40, medium D . R 25–50, high D Overall Risk–Detection chance combinations with acceptable outcomes were highlighted in green in the matrix table, those not acceptable in red and those suboptimal in orange. This colour scheme permitted a quick overview of the risk analysis for each process. In our opinion, potential failures with unacceptable outcomes (red) require an immediate, more detailed risk analysis of the individual components of the process step. In the meantime, the process or sub-process concerned must be put on hold until the Probabilities or Consequences of failure have lessened or additional Detection chances have been introduced. More detailed analysis is also advisable for potential failures with suboptimal outcomes with relatively high R–low D
combinations (orange). We did this for one of the potential failures in the stat analytical process.
Results Figure 2 shows the completed scoring matrices for our stat pre-analytical and analytical processes. Table 1 presents the results of the PRAs of all four processes (stat pre-analytical and stat analytical, regular preanalytical and regular analytical), compiled per process and failure type. In total, 180 potential failures were identified in the various steps of all four PRAs. Of these, 33 potential failures were rated as suboptimal (18.3%). More potential failures were identified in the analytical processes (53) than in the pre-analytical processes (37), a difference mainly explained by the higher frequency of the failure type ‘inappropriate reporting’ (12 in the analytic process, 1 in the pre-analytic process). The absence of the failure type ‘inappropriate analysis’ in the pre-analytic process also helps to account for this difference, although to a lesser extent (3 in the analytic process, 0 in the pre-analytic process). All processes taken together, the overall mean Risk score (R) was 9.6 6.7 (means s.d.) and the overall mean chance of Detection (D) was 2.4 0.7. Considering the scores per process, the highest mean R was observed for the stat analytical process (14.2 7.1) and the lowest for regular pre-analysis (5.3 3.2). For both stat processes, the mean R was higher than for the non-stat processes. The mean D was comparable for all processes (range 2.1–2.9). In the stat analytical process, potential failures with suboptimal scores were identified more frequently (23) than in the stat preanalytical process, whereas the fewest potential failures with suboptimal scores were identified in regular preanalysis (0). Considering the scores per failure type, ‘delayed processing or analysis’ was observed most often (66), ‘inappropriate analysis’ the least (6). The highest mean R was obtained for ‘inappropriate analysis’ (15.0) and the lowest for ‘loss of sample/collection error’ (8.2). The mean D for the different failure types varied from 1.7 to 2.9. ‘Identification error’ was the most frequent suboptimal failure type (12) and ‘loss of sample/collection error’ the least frequent (0). No potential failures were evaluated as unacceptable, although ‘Phoning extreme results to practitioners’ in step CA072, evaluated as suboptimal, had a relatively high R score (35) with a medium D (2) (Figures 1(b) and 2(b)). For this potential failure, a more detailed analysis was conducted using the same PRA approach described here. The step was divided into eight separate sub-steps, each activity was briefly described, and the Probability, Consequence and
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
33 9 0 1 s.d.: standard deviation.
9.6 (6.7)
Potential failures in steps rated as suboptimal, number
23
2.1 (0.6)
9.9 (6.5) 5.3 (3.2)
2.8 (0.5) 2.1 (0.6)
7.8 (4.9)
2.9 (0.4) Chance of Detection D, mean (s.d.)
14.2 (7.1)
2.4 (0.7)
Discussion
Risk score R, mean (s.d.)
9
chance of Detection of potential failures were rated. The resulting R–D combinations were evaluated as described. One of the eight sub-steps proved to be overtly insufficient, that is, laboratory technicians were taking too long to phone extreme results to practitioners. Improvement measures were devised based on this insight: a pop-up was made that appears on the computer screen as soon as the laboratory technician confirms the test result, indicating that the result must be phoned through to the practitioner immediately.
33
1.9 (0.7)
2.4 (0.7)
26
9.6 (6.7) 53
12 1
37 53
12 1
37 Total number of potential failures per step
Inappropriate reporting or product provided
180
10.2 (6.5)
3
9.2 (7.3)
6
1.7 (0.5)
2.6 (0.5)
15.0 (1.1)
66
6 3
18 15
0 3
Delayed processing or analysis
18
0
15
Inappropriate analysis
8.2 (5.3) 13.0 (5.6)
0 3
2.2 (0.7)
2.9 (0.2) 1.9 (0.6)
10.0 (7.3) 34
34 14 6 5
9 8
11 2 6 5
9 8
11 2 Loss of sample, incl. collection error Inappropriate sample processing
Number of potential failures per step Identification error
Chance of Detection D mean (s.d.) Risk score R mean (s.d.) Total number Analysis regular Pre-analysis regular Analysis stat Pre-analysis stat
Process
Table 1. Overview of results of PRAs of four laboratory processes involved in production of test results.
12
701
Potential failures in different steps rated as suboptimal, number
Janssens
The working methods, guidelines and procedures for quality assurance for analyses in clinical laboratories have been detailed elsewhere.9,16,17 However, laboratory analyses are merely part of the wider processes performed in laboratories. Matters such as sample collection and handling, timely processing and reporting are also essential and valued by users. It is the total product that counts. PRA can help laboratory processes to operate in an optimal, error-free manner. In diagnostic laboratories, however, PRA is still fairly novel. This paper therefore describes a practical, transparent form of PRA for clinical laboratories. All aspects relevant to the key laboratory processes, from sampling to reporting test results, were evaluated step by step for all conceivable failures. Predefined potential failures were used, as this made performing the PRAs easier and more systematic, prevented arbitrariness in identifying failures and guarded against the omission of certain failure types in specific process steps. The risk scores obtained were recorded in convenient matrix tables, making each process immediately visible in its entirety. The overall Risk scores and the chances of Detection were compared against predetermined standards and the outcomes highlighted in the matrix tables, making it immediately visible in which step and for which failure type improvement actions were required. The overall Risk scores were evaluated in light of the chances of Detection, as the ultimate consequence of a certain risk depends on whether the failure is likely to be detected in time (and prevented) or not. Each completed PRA is accompanied by a brief report, which is convenient for further evaluation and revision both in itself and for auditing and review by others (cf. clause 4.13 of ISO 15189).9 The exact results of PRAs are, of course, dependent on the local situation. Processes and their organisation differ between laboratories, as do the risks for certain failures and the safeguards applied to prevent or detect them in a timely manner. In addition, the rating of the Probabilities, Consequences and chances of Detection of potential failures may differ between laboratories,
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
702
Annals of Clinical Biochemistry 51(6)
at least to some extent, as may the norms used in evaluating the scores. It should be clear, therefore, that the present results are merely illustrative. Still, the salient features of our four PRAs may provide some useful insights also applicable to other situations. One observation was that, as expected, the highest Risk scores were obtained in the stat processes. Stat processes must satisfy higher standards than regular processes, which makes them more vulnerable to deviations and interference. Potential failures, and potential failures rated as suboptimal, were most often found in the stat and regular analytical processes. The failure type with the highest mean R score was ‘inappropriate analysis’. These observations underscore the importance of the analytical process carried out by clinical laboratories, as well as its susceptibility to errors. The failure type most frequently rated as suboptimal was ‘identification error’. As this involves manual, human work which is prone to error, this is not altogether unexpected. While human failure can never be completely eliminated, measures can be taken to reduce the chance of error, minimise the consequences and maximise the chances of detection of failures, for instance by using the Human Reliability Analysis and the Skills, Rules, Knowledge approach.18 Finally, the failure type ‘delayed processing or analysis’ was most frequently represented in the collective process steps. This underlines the importance of timeliness in the various pre-analytical and analytical laboratory processes. The PRAs previously described for our fertility lab involved 49 separate process steps for our semen bank, and 45 for the processing of fresh semen for intrauterine insemination.13 Such high numbers make these PRAs laborious and unclear. The PRA approach described here is less detailed, identifying only 15 (pre-analysis) to 20 (analysis) process steps. This more general approach makes the present PRA easier to perform and less labour intensive, while providing a better overview of the processes. It is undoubtedly true that sub-steps can often be identified per process step. However, we purposefully viewed the steps from a broader perspective, based on the idea that employees familiar with a process normally have a good idea whether a certain step as a whole is risky with respect to a certain type of failure. A potential failure in a step was considered risky if subprocesses within it were assumed or known to be characterised by a greater Probability or Consequence of failure, resulting in a significant Risk score, together with a relatively low chance of Detection. In such cases, a further, more detailed risk analysis was conducted for the individual sub-steps relating to the potential failure. This can be done using the FMEA method described here. It is worth noting that, in addition to the more detailed inspection of potential
failures (if considered necessary), the results of the PRA should also be considered at higher aggregation levels; that is, across multiple steps and potential failures. This helps to shed light on whether risks can be traced to more general factors influencing quality, such as equipment, working conditions or the immediate environment. The transparent presentation of our PRA results facilitates such a holistic perspective. The separate evaluation of Probability and Consequences and their combination in an overall Risk score allows divergent potential failures to be assessed, irrespective of whether they are more or less likely or serious. Further, assigning scores for the Probabilities, Consequences and chances of Detection helps to make the results of the PRA comparable with other PRAs. The scoring method, which is characteristic of FMEA, is semi-quantitative and naturally somewhat subjective, as described.19 We scored the Probabilities of potential failures in terms of time, as this is how laboratory staff are most used to thinking (as opposed to in terms of deviations per number of samples processed or analyses performed). This approach entails taking into account the number of issues involved in the process considered. For a failure in a process involving thousands of issues each week (e.g. 8000 samples for routine chemistry analysis, as in our lab), one failure per week might be scored with a 6 (a rather low probability), while the Probability of one failure per week in a process involving few issues each week (e.g. the 10 samples for DNA testing in our lab) might be rated 8 or 9 (high). In other words, the expression of Probability in terms of time requires the scoring to be adjusted to the process under inspection. This allows for comparison of the overall Risk scores – and thereby the results of the PRAs – of different processes. In scoring Probabilities, one could of course consider using previously identified failures from retrospective risk analyses. However, it is by no means guaranteed that the scores of both analyses will agree, as has been shown.20 One factor in particular, reporting bias, is difficult to control for. In PRA, processes are viewed differently than in retrospective risk analysis. Retrospective risk analysis is based on the registration of failures from everyday practice, which may lead to a distorted, overly detailed, view of the process as a whole. This highlights the added value of conducting PRA in addition to retrospective risk analysis: the two approaches are complementary. Note also that the ratings of the chances of Detection and the acceptability criteria for the overall Risk scores are unavoidably subjective. The present PRAs made use of acceptability criteria deemed reasonable elsewhere,13 based on consideration by experienced staff well acquainted with the culture, rules and regulations in clinical laboratories in
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
Janssens
703
particular and healthcare in general. From the foregoing it should be clear, however, that laboratories must decide for themselves on the scoring and acceptability criteria to be used in their own PRAs, taking into account their specific environment, constraints and processes considered. PRA is just one approach for gaining insight into laboratory processes. It is primarily aimed at preventing failures. The FMEA approach, on which the present PRA is based, is well suited for this. To reduce risks as far as possible, PRA should be applied in conjunction with retrospective risk analysis.4,21 From a broader perspective, risk analysis merges with risk management.13 Risk management based on prospective and retrospective risk analyses should be an ongoing process, as indicated in ISO 15189.9 The complete elimination of failures may be impossible, but patients and society at large may expect medical professionals to have done their utmost to minimise them. Acknowledgements I am grateful to Alison Edwards, Cambridge, for her assistance with the text. A pre-version of the present manuscript appeared in Dutch (Janssens PMW, de Waard H, van Welt E, Lentink B, Staring W en de Geest A. Ned Tijdschr Klin Chem Labgeneesk 2012; 37: 247–251; no copy rights). In the present manuscript, new data have been incorporated and the approach to PRA, the view on its possibilities and limitations have been detailed and extended.
Declaration of conflicting interests None.
Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Ethical approval Not applicable.
Guarantor PMWJ.
Contributorship PMWJ sole author.
References 1. Chiozza ML and Plebani M. Clinical Governance: from clinical risk management to continuous quality improvement. Clin Chem Lab Med 2006; 44: 694–698.
2. Carraro P and Plebani M. Errors in a stat laboratory: types and frequencies 10 years later. Clin Chem 2007; 53: 1338–1342. 3. Plebani M. The detection and prevention of errors in laboratory medicine. Ann Clin Biochem 2010; 47: 101–110. 4. Signori C, Ceriotti F, Sanna A, et al. Process and risk analysis to reduce errors in clinical laboratories. Clin Chem Lab Med 2007; 45: 742–748. 5. Lippi G and Guidi GC. Risk management in the preanalytical phase of laboratory testing. Clin Chem Lab Med 2007; 45: 720–727. 6. Chiozza ML and Ponzetti C. FMEA: a model for reducing medical errors. Clin Chim Acta 2009; 404: 75–78. 7. ISO 31000:2009. Risk management – Principles and guidelines, http://webstore.ansi.org/RecordDetail.aspx? sku=ISO+31000%3a2009&gclid=CKGGrvies7cCFQj KtAodaD4AHA (2009, accessed 2 February 2014). 8. EP23-A Laboratory quality control based on risk management; Approved Guideline. Clinical and Laboratory Standards Institute, http://shopping.netsuite.com/s.nl/ c.1253739/it.A/id.282/.f (2011, accessed 2 February 2014). 9. ISO 15189:2012 Medical laboratories – Requirements for quality and competence http://www.iso.org/iso/catalogue_detail?csnumber=56115; see also ISO/FDIS 15189 Medical laboratory accreditations. International Laboratory Accreditation Coordination, ILAC, https:// www.ilac.org/documents/ILAC_Medical_Lab_% 20Accred.pdf (2011, accessed 2 February 2014). 10. Praktijkrichtlijn CCKL [CCKL practical guidelines], versie 4, www.cckl.nl/index.php?pagina=107 (2005, accessed 2 February 2014). 11. Landelijke richtlijn voor de semenbank. Specifieke kwaliteitseisen voor het laboratoriumdeel [National guidelines for semen banks: Specific quality requirements for the laboratory part]. Netherlands Society for Clinical Chemistry and Laboratory Medicine (NVKC) and Association of Clinical Embryologists (KLEM),www. embryologen.nl/images/stories/documenten/richtlijn_ standpunt/landelijke_richtlijn_voor_de_semenbank_ versie_2.0_status_definitief_1_feb_2010f.pdf (2010, accessed 2 February 2014). 12. Mortimer D and Mortimer ST. Quality and risk management in the IVF Laboratory. Cambridge, MA: Cambridge University Press, 2005. 13. Janssens PMW and Cheung KS. Approaching risk analysis and risk management in the fertility laboratory and semen bank. Int J Androl 2009; 32: 656–665. 14. ISO/TS 22367:2008. Medical Laboratories – reduction of error through risk management and continual improvement, www.iso.org/iso/catalogue_detail.htm?csnumber=40918 (2008, accessed 2 February 2014). 15. Krouwer JS, Halim A-B, Hatjimihail AR, et al. Risk management techniques to identify and control laboratory error sources; approved guideline. 2nd ed. Clinical and Laboratory Standards Institute, http://shopping. netsuite.com/c.1253739/site/Sample_pdf/EP18A2_sample. pdf (2009, accessed 2 February 2014).
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
704
Annals of Clinical Biochemistry 51(6)
16. Passey RB and Maluf KC. Foundations for validation of quantitative analytical methods in the clinical laboratory. Arch Pathol Lab Med 1992; 116: 732–738. 17. Westgard JO. Basic method validation. 3rd ed. FAQs, www.westgard.com/basic-method-validation-3rd-editionfaqs.htm (2009, accessed 2 February 2014). 18. Schindhelm RK and Janssens PMW. Risk management: choosing optimal tools on the basis of psychological analysis. In: Zhiyong Z (ed.) Risk assessment and management. AcademyPublish.org, pp.55–60, www. academypublish.org/book/show/title/risk-assessmentand-management (2012, accessed 2 February 2014).
19. Shebl N, Franklin B, Barber N, et al. Failure mode and effects analysis: views of hospital staff in the UK. J Health Serv Res Policy 2012; 17: 37–43. 20. Kessels-Habraken M, Van der Schaaf T, De Jonge J, et al. Integration of prospective and retrospective methods for risk analysis in hospitals. Int J Qual Health Care 2009; 21: 427–432. 21. Snijders C, van der Schaaf TW, Klip H, et al. NEOSAFE study group. Feasibility and reliability of PRISMA-medical for specialty-based incident analysis. Qual Saf Health Care 2009; 18: 486-491. [Erratum in: Qual Saf Health Care. 2011; 20: 396].
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
Practical, transparent prospective risk analysis for the clinical laboratory Pim MW Janssens Ann Clin Biochem 2014 51: 695 originally published online 19 February 2014 DOI: 10.1177/0004563214521160 The online version of this article can be found at: http://acb.sagepub.com/content/51/6/695
Published by: http://www.sagepublications.com
On behalf of: Association for Clinical Biochemistry and Laboratory Medicine
Netherlands Society for Clinical Chemistry and Laboratory Japan Society of Clinical Chemistry
Additional services and information for Annals of Clinical Biochemistry: An international journal of biochemistry and laboratory medicine can be found at: Email Alerts: http://acb.sagepub.com/cgi/alerts Subscriptions: http://acb.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav
>> Version of Record - Oct 20, 2014 OnlineFirst Version of Record - Feb 19, 2014 What is This?
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
Original Article Annals of Clinical Biochemistry 2014, Vol. 51(6) 695–704 ! The Author(s) 2014 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/0004563214521160 acb.sagepub.com
Practical, transparent prospective risk analysis for the clinical laboratory Pim MW Janssens
Abstract Background: Prospective risk analysis (PRA) is an essential element in quality assurance for clinical laboratories. Practical approaches to conducting PRA in laboratories, however, are scarce. Methods: On the basis of the classical Failure Mode and Effect Analysis method, an approach to PRA was developed for application to key laboratory processes. First, the separate, major steps of the process under investigation are identified. Scores are then given for the Probability (P) and Consequence (C) of predefined types of failures and the chances of Detecting (D) these failures. Based on the P and C scores (on a 10-point scale), an overall Risk score (R) is calculated. The scores for each process were recorded in a matrix table. Based on predetermined criteria for R and D, it was determined whether a more detailed analysis was required for potential failures and, ultimately, where risk-reducing measures were necessary, if any. Results: As an illustration, this paper presents the results of the application of PRA to our pre-analytical and analytical activities. The highest R scores were obtained in the stat processes, the most common failure type in the collective process steps was ‘delayed processing or analysis’, the failure type with the highest mean R score was ‘inappropriate analysis’ and the failure type most frequently rated as suboptimal was ‘identification error’. Conclusions: The PRA designed is a useful semi-objective tool to identify process steps with potential failures rated as risky. Its systematic design and convenient output in matrix tables makes it easy to perform, practical and transparent.
Keywords Laboratory testing process, prospective risk analysis, risk management, FMEA, clinical laboratory Accepted: 2nd January 2014
Introduction Product quality for clinical laboratories means on-time delivery of the correct analysis results and products, for the right patient, to the right doctor. This requires sound validation, employee training and quality assurance with respect to the analytical activities. It also requires that attention be paid to the processes involved in both the pre-analytical and analytical phases, for example by means of risk analysis.1–6 As a tool to identify and prevent errors in processes, risk analysis is more suitable as a form of quality management than
the well-known and widely used LEAN approach, which primarily revolves around logistic-economic factors in processes. The well-known Six Sigma approach, on the other hand, is primarily intended to count and control known non-conformities, not so much to newly Laboratory of Clinical Chemistry and Haematology, Rijnstate Hospital, The Netherlands Corresponding author: Pim MW Janssens, Laboratory of Clinical Chemistry and Haematology, Rijnstate Hospital, PO Box 9555, 6800 TA Arnhem, the Netherlands. Email: [email protected]
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
696
Annals of Clinical Biochemistry 51(6)
identify them. Retrospective risk analysis is well known in clinical laboratories, and is used to identify the causes of failures. In contrast, prospective risk analysis (PRA)4–6 is driven not by incidents that have actually occurred, but by the examination of laboratory processes and their individual steps from an independent, more distant perspective to identify potential incidents in advance. Ideally, all main processes within a clinical laboratory should be subjected to PRA. Depending on the type of laboratory, these include the pre-analytical process, the analytical process (on site or elsewhere, for tests that the laboratory does not carry out itself), the process for obtaining urgent (‘stat’) test results, the provision of blood products for transfusion or the processing and issuing of gametes for fertility treatment. To date, the insight into and application of PRA in clinical laboratories is still rather limited, despite the fact that PRA is mentioned in various guidelines regulating the organisation of clinical laboratories.6–11 For instance, clauses 4.11 and 4.14 of ISO directive 15189 stipulate that clinical laboratories should conduct risk assessments to examine potential failures in their processes,9 alluding to the investigation of activities and processes in a preventive manner by way of PRA. Sound and convenient methods for executing PRAs are therefore desirable. The few methods described for executing PRAs in clinical laboratories, however, are either somewhat vague or rather complex and laborious.4–6,12,13 This article describes a practical and transparent approach to PRA that is suitable for laboratory processes, and presents the results of four of our PRAs to serve as illustration.
is made as to whether measures need to be taken. Elsewhere, we described an application of FMEA for the fertility laboratory which is situated in our department.13 In comparison, the PRA method described in the present paper purposefully used a less detailed approach,13 with processes divided into broader steps. In addition, a simple matrix was introduced to capture the risk scores. These improvements allow for a better overview of the processes and the results obtained. To conduct the PRAs, the steps in each process were first inventoried, resulting in an outline in the form of a flow diagram (Figure 1). The flow diagrams provide an overview of each process and come in handy when documenting potential failures step by step. PRAs were conducted for the pre-analytical and analytical processes for both the regular and stat processing of laboratory tests. These processes have multiple ‘entries’ and ‘exits’ in our laboratory, giving rise to flow diagrams with multiple arms, each of which were described (provided they were of some importance). Each process step in the flow diagram was given a brief description and a code that was subsequently used in the score matrix (Figure 2). Prior to conducting the PRAs, we identified in a general sense the basic potential failures (also referred to in the literature as non-conformities, failure modes, incidents and errors) that can occur in the processes. We decided on a limited number of failures, so as to maximise clarity when performing the PRAs. The potential failures across laboratories are largely consistent,5 but some variation can occur depending on the process considered, the personal choices of the individuals performing the PRA and the responsible staff. Our PRAs focused on the failure types:
Methods Study site
1. 2. 3. 4. 5. 6.
The PRAs described were performed in the Clinical Chemical and Haematological Laboratory at Rijnstate Hospital, Arnhem, the Netherlands, a 900bed general care and teaching hospital. The laboratory department provides blood sampling services and laboratory testing around the clock for approximately 300 analytes for inpatients and outpatients, patients in primary care and a number of affiliated institutes.
Design of the risk analysis approach Our PRA is an elaboration of the established Failure Mode and Effect Analysis (FMEA).6,14,15 In FMEA, a process is divided into separate steps. The potential failures identified per step each receive a Risk score (R). On the basis of this score and the chance of identifying failures after they have occurred, an assessment
identification error; mixing up or loss of material; incorrect sample labelling, processing or handling; incorrect execution of analysis; incorrect reporting or product delivery; and delayed execution.
Failure types 1 to 5 mainly relate to content. Failure type 6 primarily concerns the factor time, although time may also be involved in failure types 3 to 5. Space and environmental factors as reflected in issues such as temperature, noise, commotion, equipment, distances, power facilities, etc., can affect any time- and contentrelated aspect of the process, so play a role in all six failure types. For the sake of convenience, no additional failure types specific to the factors time or space/environment were introduced, as they were considered to be sufficiently addressed in the other failure types. To keep the scoring matrix uniform and universally applicable to all laboratory processes, specific
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
Janssens
697 (a)
CP010
CP011
CP012
CP013
order entry in HIS by clinic
order entry in HIS by outpatient dept.
order entry by general practitioners
order entry from hard-copy forms in LIMS by lab dept.
CP030 producing overview + labels for sampling by lab
CP020
CP021
informing patient on requirements
sample collection by non-lab pers. Ordering on paper forms
CP031 producing sample labels by external or clinical dept.
CP032 producing sample labels by lab dept.
CP040
CP041
CP042
sample collection by lab in clinic, by stat employee
sample collection by clinical dept.
sample collected at lab desk (for outpatients)
CP050 sample transport by pneumatic tube system
CP060 sample registration in LIMS by lab dept.
CP070 transfer of samples to site in lab for analysis
CP999 to CA001 for stat analysis
Figure 1. Process description in the form of flow diagrams of (a) the stat pre-analytical process and (b) the stat analytical process in our laboratory or elsewhere.
headings were not deleted from the scoring table in cases where a specific failure type did not exist, but rather left blank (e.g. ‘incorrect execution of analysis’, which does not apply to pre-analysis, or ‘product delivery’, which applies not to the analytical but rather to the blood transfusion and fertility laboratory processes in our department).
Execution of the risk analysis The PRAs were conducted, amid open consultation and discussion, by groups of four to seven members. In our experience, the group should neither be so large as to
prevent efficient discussion and progress, nor too small, to guarantee the presence of sufficient knowledge and experience. One group member had experience performing PRAs, some members had a good overview of the process concerned and clinical laboratories in general, and several were regularly involved in executing the process. For each potential failure in a process step, an inventory was made of the following: . the Probability (P) of a certain failure occurring over time (also referred to as likelihood), scored on a scale of 0–10 (0 ¼ almost never or impossible, 1 ¼ less than once a year, 2 ¼ once a year, 3 ¼ several times a year,
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
698
Annals of Clinical Biochemistry 51(6)
(b)
CA010 sample storage, waiting for analysis
CA020 manual analysis
CA001 step following CP999
CA021
CA022
automated analysis (lab robots)
analysis elsewhere (external)
CA030
CA031
CA032
judging + confirmation of result
judging + confirmation of result
judging + confirmation of result
CA040
CA041
CA042
manual transfer of results to LIMS
automatic transfer of results to LIMS
manual transfer of results to LIMS
CA050 sample storage
CA060 CA070
data confirmation in LIMS
reporting results from LIMS to general practitioners
CA071
CA072
phoning results to practitioners, on request
phoning extreme results (initiative of lab)
CA080 progress control
CA090 manual autorisation of abnormal results by lab doctor
CA100 definite rapporting of results
CA110 billing
Figure 1. Continued.
4 ¼ once a month, 5 ¼ several times a month, 6 ¼ once a week, 7 ¼ several times a week, 8 ¼ each day, 9 ¼ several times a day, 10 ¼ many times a day). . the Consequence (C) (also referred to as the impact) of a failure, scored on a scale of 0–10. The degree of the consequence is assessed in light of the potential damage for patients (health risk, inadequate or
ineffective treatment, discomfort), although serious consequences for employees are also taken into account in the higher ratings (i.e. 8 and higher) (0 ¼ no problem, 1 ¼ practically no problem, 3 ¼ minor problem that can be solved by staff, 4 ¼ somewhat inconvenient, 5 ¼ inconvenient, 6 ¼ discomfort, 7 ¼ serious discomfort, 8 ¼ injury to patient or
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
Janssens (a) Pre-analysis stat
699 Identification error
Code and step in process P C P*C D CP010 2 4 8 3 Order entry in HIS by clinic CP011 1 4 4 3 Order entry in HIS by outpatient dept. CP012 1 4 4 3 Order entry in ordering syst. by general practitioners CP013 1 4 4 3 Order entry from hard-copy order forms in LIMS by lab dept. CP020 na na na Informing patient on requirements for investigations and ordering CP021 Sample collection by non-lab pers. with lab ordering on hard-copy 1 4 4 3 forms (GPs, external relations, etc.) CP030 Production of overview of patients plus sample labels for sampling by na na na lab CP031 na na na Production of sample labels by external or clinical department CP032 na na na Production of sample labels by laboratory CP040 1 8 8 2 Sample collection by laboratory in clinic (by stat employee) CP041 1 8 8 2 Sample collection by clinical department CP042 1 8 8 2 Sample collected at lab desk (for outpatients) CP050 na na na Transportation of samples by pneumatic tube system CP060 na na na Registration of samples in LIMS by laboratory CP070 na na na Transfer of samples to analysis site in laboratory
Number of steps rated Overall Risk score, mean Overall Risk score, s.d. Probability of detection, mean Probability of detection, s.d. Number of steps scored suboptimal (marked orange or red)
(b)
Number of steps rated: Overall Risk score (mean): Overall Risk score (s.d.): Probability of detection (mean): Probability of detection (s.d.): Number of steps scored suboptimal (marked orange or red):
C na
P*C
na
na
3
8
24
1
8
3
8
C
P*C D
P
C
P*C D
P
C
P*C
D
Inappropriate reporting or product provided P C P*C D
na
na
na
na
na
na
na
1
7
7
3
na
na
na
na
na
na
na
na
na
na
na
na
1
7
7
3
na
na
na
na
na
na
na
na
na
na
na
na
1
7
7
3
na
na
na
na
na
na
na
na
na
na
na
na
1
7
7
3
na
na
na
0
1
0
3
1
8
1
na
na
na
2
7
14
3
1
3
1
3
3
3
na
na
na
na
na
na
1
7
7
3
na
na
8
3
3
na
1
3
3
3
na
na
na
na
na
na
2
7
14
3
na
na
na
1
3
3
3
na
na
na
na
na
na
1
7
7
3
na
na
na
1
3
3
3
na
na
na
na
na
na
1
7
7
3
na
na
na
2
5
10
3
na
na
na
na
na
na
4
7
28
3
na
na
na
2
5
10
3
na
na
na
na
na
na
2
7
14
3
na
na
na
2
5
10
3
na
na
na
na
na
na
1
7
7
3
na
na
na
2
5
10
3
na
na
na
na
na
na
1
7
7
3
na
na
na
1
3
3
3
na
na
na
na
na
na
2
7
14
3
na
na
na
2
5
10
3
2
5
3
na
na
na
1
7
7
3
na
na
na
10
Per failure type 2 9,0 1,4 2,0 1,4 0
11 5,9 4,0 3,0 0,0 0 Loss of sample, incl. collection errors
P
P
Delayed processing or analysis
Inappropriate analysis
na
Identification error
na
Inappropriate sample processing
na
8 6,0 2,1 2,6 0,5 0
Analysis stat (analysis in own laboratory or elswhere) Code and step in process CA001 See CP999, Pre-analysis stat CA010 Sample storage, waiting for analysis CA020 Manual analysis CA021 Automated analysis (lab robots) CA022 Analysis elsewhere (external) CA030 Judging (potential rerun) and confirmation of result CA031 Judging (potential rerun) and confirmation of result CA032 Judging and confirmation of result CA040 Manual transfer of results to LIMS CA041 Automatic transfer of results to LIMS CA042 Manual transfer of results to LIMS CA050 Post-run transfer of samples to sample storage CA 060 Data confirmation in LIMS CA070 Reporting of results from LIMS to GPs CA071 Phoning of results to practitioners, on request CA072 Phoning extreme results to practitioners (on initiative of lab) CA080 Progress control CA090 Manual authorisation of abnormal results by lab physician CA100 Definitive reporting of results CA110 Billing
Loss of sample, incl. collection errors P C P*C D
Inappropriate sample processing
D
P
C
P*C
D
P
C
na
2
8
16
3
na
na
na
2
8
16
3
2
8
2
2
8
16
3
3
8
8
3
2
8
16
3
1
24
2
3
8
24
3
3
P*C
0
15 10,3 5,8 3,0 0,0 1
0
D
P
C
na
na
na
P*C
16
2
na
na
24
2
2
8
16
8
8
2
2
8
8
24
1
2
8
D
P
C
P*C
D
P
C
2
7
14
2
na
na
na
2
7
14
2
na
na
2
2
7
14
2
3
7
21
16
2
2
7
14
2
2
7
14
3
16
1
3
7
21
2
3
7
21
2
21
7
na
na
na
na
na
na
na
na
na
na
na
na
3
7
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
2
8
16
2
na
na
na
na
na
na
na
na
na
2
7
14
0
8
0
3
na
na
na
na
na
na
na
na
na
1
7
2
8
16
2
na
na
na
na
na
na
na
na
na
2
3
8
24
1
3
4
2
3
5
2
na
na
na
2
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
1
8
8
1
na
na
na
na
na
na
na
na
na
1
8
8
1
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
15
0 Inappropriate reporting or product provided
na
na
12
3,0
Delayed processing or analysis
Inappropriate analysis
Total process 37 7,8 4,9 2,9 0,4 1
1 3,0
P*C
D na na 2
3
1
7
na
na
na
2
na
na
na
2
2
7
14
2
7
3
0
7
0
3
7
14
2
3
7
21
3
6
2
na
na
3
7
21
2
1
7
1
7
7
2
na
na
3
7
21
2
1
7
7
5
7
35
2
1
7
7
1
na
2
7
14
2
1
7
7
2
12
na na
2 na
7
2 na 1
na
na
na
na
na
na
na
na
na
na
na
na
3
3
9
2
2
6
na
na
na
na
na
na
na
na
na
na
na
na
1
4
4
3
na
na
na
na
na
na
na
na
na
na
na
na
na
na
na
1
3
3
3
na
na
na
9 14.2 8.7 1.9 0.8 7
6 16.7 3.9 2.8 0.4 0
Per failure type 5 17.4 6.8 1.8 0.4 3
3 16.0 0.0 1.7 0.6 3
18 14.1 7.8 2.2 0.4 4
12 11.5 6.9 1.9 0.7 6
1
Total process 53 14.2 7.1 2.1 0.6 0
na: not applicable s.d.: standard deviation
Figure 2. PRA of (a) stat pre-analytical process and (b) stat analytical process in our laboratory or elsewhere. Abbreviations: P: Probability; C: Consequence; D: chance of Detection; na: not applicable. The codes CP and CA in column 2 for each process step refer to the relevant steps in the flow diagrams (Figure 1). Boxes marked in grey are those for which risk scoring is not applicable.
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
700
Annals of Clinical Biochemistry 51(6)
someone else, 9 ¼ severe injury to patient or someone else, 10 ¼ catastrophic, severe injury with permanent and/or life threatening consequences). . the chance of Detecting (D) a failure, scored on a scale of 1–3 (1 ¼ low, 2 ¼ medium, 3 ¼ high). The scores were obtained by consensus among the group members, after debate where necessary. We recommend that each laboratory establish its own definitions for each score, as these may depend on the local conditions and culture as well as the process considered. To estimate the overall Risk score (R), the Probability is multiplied by the Consequence of the failure; in other words, R ¼ P C, on a scale of 1–100. The P, C, R and D scores were recorded in a matrix table created in Excel (Figure 2).
Processing of the PRA results and evaluation The overall Risk scores (R) were automatically calculated in the Excel table from the estimated Probability and Consequence of the different failure types (Figure 2). The R scores were evaluated in combination with the chance of Detecting the failure (D) as follows: Acceptable: . R max. 5, low D . R max. 15, medium D . R max. 25, high D Unacceptable: . R > 30, low D . R > 40, medium D . R > 50, high D Suboptimal: . R 5–30, low D . R 15–40, medium D . R 25–50, high D Overall Risk–Detection chance combinations with acceptable outcomes were highlighted in green in the matrix table, those not acceptable in red and those suboptimal in orange. This colour scheme permitted a quick overview of the risk analysis for each process. In our opinion, potential failures with unacceptable outcomes (red) require an immediate, more detailed risk analysis of the individual components of the process step. In the meantime, the process or sub-process concerned must be put on hold until the Probabilities or Consequences of failure have lessened or additional Detection chances have been introduced. More detailed analysis is also advisable for potential failures with suboptimal outcomes with relatively high R–low D
combinations (orange). We did this for one of the potential failures in the stat analytical process.
Results Figure 2 shows the completed scoring matrices for our stat pre-analytical and analytical processes. Table 1 presents the results of the PRAs of all four processes (stat pre-analytical and stat analytical, regular preanalytical and regular analytical), compiled per process and failure type. In total, 180 potential failures were identified in the various steps of all four PRAs. Of these, 33 potential failures were rated as suboptimal (18.3%). More potential failures were identified in the analytical processes (53) than in the pre-analytical processes (37), a difference mainly explained by the higher frequency of the failure type ‘inappropriate reporting’ (12 in the analytic process, 1 in the pre-analytic process). The absence of the failure type ‘inappropriate analysis’ in the pre-analytic process also helps to account for this difference, although to a lesser extent (3 in the analytic process, 0 in the pre-analytic process). All processes taken together, the overall mean Risk score (R) was 9.6 6.7 (means s.d.) and the overall mean chance of Detection (D) was 2.4 0.7. Considering the scores per process, the highest mean R was observed for the stat analytical process (14.2 7.1) and the lowest for regular pre-analysis (5.3 3.2). For both stat processes, the mean R was higher than for the non-stat processes. The mean D was comparable for all processes (range 2.1–2.9). In the stat analytical process, potential failures with suboptimal scores were identified more frequently (23) than in the stat preanalytical process, whereas the fewest potential failures with suboptimal scores were identified in regular preanalysis (0). Considering the scores per failure type, ‘delayed processing or analysis’ was observed most often (66), ‘inappropriate analysis’ the least (6). The highest mean R was obtained for ‘inappropriate analysis’ (15.0) and the lowest for ‘loss of sample/collection error’ (8.2). The mean D for the different failure types varied from 1.7 to 2.9. ‘Identification error’ was the most frequent suboptimal failure type (12) and ‘loss of sample/collection error’ the least frequent (0). No potential failures were evaluated as unacceptable, although ‘Phoning extreme results to practitioners’ in step CA072, evaluated as suboptimal, had a relatively high R score (35) with a medium D (2) (Figures 1(b) and 2(b)). For this potential failure, a more detailed analysis was conducted using the same PRA approach described here. The step was divided into eight separate sub-steps, each activity was briefly described, and the Probability, Consequence and
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
33 9 0 1 s.d.: standard deviation.
9.6 (6.7)
Potential failures in steps rated as suboptimal, number
23
2.1 (0.6)
9.9 (6.5) 5.3 (3.2)
2.8 (0.5) 2.1 (0.6)
7.8 (4.9)
2.9 (0.4) Chance of Detection D, mean (s.d.)
14.2 (7.1)
2.4 (0.7)
Discussion
Risk score R, mean (s.d.)
9
chance of Detection of potential failures were rated. The resulting R–D combinations were evaluated as described. One of the eight sub-steps proved to be overtly insufficient, that is, laboratory technicians were taking too long to phone extreme results to practitioners. Improvement measures were devised based on this insight: a pop-up was made that appears on the computer screen as soon as the laboratory technician confirms the test result, indicating that the result must be phoned through to the practitioner immediately.
33
1.9 (0.7)
2.4 (0.7)
26
9.6 (6.7) 53
12 1
37 53
12 1
37 Total number of potential failures per step
Inappropriate reporting or product provided
180
10.2 (6.5)
3
9.2 (7.3)
6
1.7 (0.5)
2.6 (0.5)
15.0 (1.1)
66
6 3
18 15
0 3
Delayed processing or analysis
18
0
15
Inappropriate analysis
8.2 (5.3) 13.0 (5.6)
0 3
2.2 (0.7)
2.9 (0.2) 1.9 (0.6)
10.0 (7.3) 34
34 14 6 5
9 8
11 2 6 5
9 8
11 2 Loss of sample, incl. collection error Inappropriate sample processing
Number of potential failures per step Identification error
Chance of Detection D mean (s.d.) Risk score R mean (s.d.) Total number Analysis regular Pre-analysis regular Analysis stat Pre-analysis stat
Process
Table 1. Overview of results of PRAs of four laboratory processes involved in production of test results.
12
701
Potential failures in different steps rated as suboptimal, number
Janssens
The working methods, guidelines and procedures for quality assurance for analyses in clinical laboratories have been detailed elsewhere.9,16,17 However, laboratory analyses are merely part of the wider processes performed in laboratories. Matters such as sample collection and handling, timely processing and reporting are also essential and valued by users. It is the total product that counts. PRA can help laboratory processes to operate in an optimal, error-free manner. In diagnostic laboratories, however, PRA is still fairly novel. This paper therefore describes a practical, transparent form of PRA for clinical laboratories. All aspects relevant to the key laboratory processes, from sampling to reporting test results, were evaluated step by step for all conceivable failures. Predefined potential failures were used, as this made performing the PRAs easier and more systematic, prevented arbitrariness in identifying failures and guarded against the omission of certain failure types in specific process steps. The risk scores obtained were recorded in convenient matrix tables, making each process immediately visible in its entirety. The overall Risk scores and the chances of Detection were compared against predetermined standards and the outcomes highlighted in the matrix tables, making it immediately visible in which step and for which failure type improvement actions were required. The overall Risk scores were evaluated in light of the chances of Detection, as the ultimate consequence of a certain risk depends on whether the failure is likely to be detected in time (and prevented) or not. Each completed PRA is accompanied by a brief report, which is convenient for further evaluation and revision both in itself and for auditing and review by others (cf. clause 4.13 of ISO 15189).9 The exact results of PRAs are, of course, dependent on the local situation. Processes and their organisation differ between laboratories, as do the risks for certain failures and the safeguards applied to prevent or detect them in a timely manner. In addition, the rating of the Probabilities, Consequences and chances of Detection of potential failures may differ between laboratories,
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
702
Annals of Clinical Biochemistry 51(6)
at least to some extent, as may the norms used in evaluating the scores. It should be clear, therefore, that the present results are merely illustrative. Still, the salient features of our four PRAs may provide some useful insights also applicable to other situations. One observation was that, as expected, the highest Risk scores were obtained in the stat processes. Stat processes must satisfy higher standards than regular processes, which makes them more vulnerable to deviations and interference. Potential failures, and potential failures rated as suboptimal, were most often found in the stat and regular analytical processes. The failure type with the highest mean R score was ‘inappropriate analysis’. These observations underscore the importance of the analytical process carried out by clinical laboratories, as well as its susceptibility to errors. The failure type most frequently rated as suboptimal was ‘identification error’. As this involves manual, human work which is prone to error, this is not altogether unexpected. While human failure can never be completely eliminated, measures can be taken to reduce the chance of error, minimise the consequences and maximise the chances of detection of failures, for instance by using the Human Reliability Analysis and the Skills, Rules, Knowledge approach.18 Finally, the failure type ‘delayed processing or analysis’ was most frequently represented in the collective process steps. This underlines the importance of timeliness in the various pre-analytical and analytical laboratory processes. The PRAs previously described for our fertility lab involved 49 separate process steps for our semen bank, and 45 for the processing of fresh semen for intrauterine insemination.13 Such high numbers make these PRAs laborious and unclear. The PRA approach described here is less detailed, identifying only 15 (pre-analysis) to 20 (analysis) process steps. This more general approach makes the present PRA easier to perform and less labour intensive, while providing a better overview of the processes. It is undoubtedly true that sub-steps can often be identified per process step. However, we purposefully viewed the steps from a broader perspective, based on the idea that employees familiar with a process normally have a good idea whether a certain step as a whole is risky with respect to a certain type of failure. A potential failure in a step was considered risky if subprocesses within it were assumed or known to be characterised by a greater Probability or Consequence of failure, resulting in a significant Risk score, together with a relatively low chance of Detection. In such cases, a further, more detailed risk analysis was conducted for the individual sub-steps relating to the potential failure. This can be done using the FMEA method described here. It is worth noting that, in addition to the more detailed inspection of potential
failures (if considered necessary), the results of the PRA should also be considered at higher aggregation levels; that is, across multiple steps and potential failures. This helps to shed light on whether risks can be traced to more general factors influencing quality, such as equipment, working conditions or the immediate environment. The transparent presentation of our PRA results facilitates such a holistic perspective. The separate evaluation of Probability and Consequences and their combination in an overall Risk score allows divergent potential failures to be assessed, irrespective of whether they are more or less likely or serious. Further, assigning scores for the Probabilities, Consequences and chances of Detection helps to make the results of the PRA comparable with other PRAs. The scoring method, which is characteristic of FMEA, is semi-quantitative and naturally somewhat subjective, as described.19 We scored the Probabilities of potential failures in terms of time, as this is how laboratory staff are most used to thinking (as opposed to in terms of deviations per number of samples processed or analyses performed). This approach entails taking into account the number of issues involved in the process considered. For a failure in a process involving thousands of issues each week (e.g. 8000 samples for routine chemistry analysis, as in our lab), one failure per week might be scored with a 6 (a rather low probability), while the Probability of one failure per week in a process involving few issues each week (e.g. the 10 samples for DNA testing in our lab) might be rated 8 or 9 (high). In other words, the expression of Probability in terms of time requires the scoring to be adjusted to the process under inspection. This allows for comparison of the overall Risk scores – and thereby the results of the PRAs – of different processes. In scoring Probabilities, one could of course consider using previously identified failures from retrospective risk analyses. However, it is by no means guaranteed that the scores of both analyses will agree, as has been shown.20 One factor in particular, reporting bias, is difficult to control for. In PRA, processes are viewed differently than in retrospective risk analysis. Retrospective risk analysis is based on the registration of failures from everyday practice, which may lead to a distorted, overly detailed, view of the process as a whole. This highlights the added value of conducting PRA in addition to retrospective risk analysis: the two approaches are complementary. Note also that the ratings of the chances of Detection and the acceptability criteria for the overall Risk scores are unavoidably subjective. The present PRAs made use of acceptability criteria deemed reasonable elsewhere,13 based on consideration by experienced staff well acquainted with the culture, rules and regulations in clinical laboratories in
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
Janssens
703
particular and healthcare in general. From the foregoing it should be clear, however, that laboratories must decide for themselves on the scoring and acceptability criteria to be used in their own PRAs, taking into account their specific environment, constraints and processes considered. PRA is just one approach for gaining insight into laboratory processes. It is primarily aimed at preventing failures. The FMEA approach, on which the present PRA is based, is well suited for this. To reduce risks as far as possible, PRA should be applied in conjunction with retrospective risk analysis.4,21 From a broader perspective, risk analysis merges with risk management.13 Risk management based on prospective and retrospective risk analyses should be an ongoing process, as indicated in ISO 15189.9 The complete elimination of failures may be impossible, but patients and society at large may expect medical professionals to have done their utmost to minimise them. Acknowledgements I am grateful to Alison Edwards, Cambridge, for her assistance with the text. A pre-version of the present manuscript appeared in Dutch (Janssens PMW, de Waard H, van Welt E, Lentink B, Staring W en de Geest A. Ned Tijdschr Klin Chem Labgeneesk 2012; 37: 247–251; no copy rights). In the present manuscript, new data have been incorporated and the approach to PRA, the view on its possibilities and limitations have been detailed and extended.
Declaration of conflicting interests None.
Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Ethical approval Not applicable.
Guarantor PMWJ.
Contributorship PMWJ sole author.
References 1. Chiozza ML and Plebani M. Clinical Governance: from clinical risk management to continuous quality improvement. Clin Chem Lab Med 2006; 44: 694–698.
2. Carraro P and Plebani M. Errors in a stat laboratory: types and frequencies 10 years later. Clin Chem 2007; 53: 1338–1342. 3. Plebani M. The detection and prevention of errors in laboratory medicine. Ann Clin Biochem 2010; 47: 101–110. 4. Signori C, Ceriotti F, Sanna A, et al. Process and risk analysis to reduce errors in clinical laboratories. Clin Chem Lab Med 2007; 45: 742–748. 5. Lippi G and Guidi GC. Risk management in the preanalytical phase of laboratory testing. Clin Chem Lab Med 2007; 45: 720–727. 6. Chiozza ML and Ponzetti C. FMEA: a model for reducing medical errors. Clin Chim Acta 2009; 404: 75–78. 7. ISO 31000:2009. Risk management – Principles and guidelines, http://webstore.ansi.org/RecordDetail.aspx? sku=ISO+31000%3a2009&gclid=CKGGrvies7cCFQj KtAodaD4AHA (2009, accessed 2 February 2014). 8. EP23-A Laboratory quality control based on risk management; Approved Guideline. Clinical and Laboratory Standards Institute, http://shopping.netsuite.com/s.nl/ c.1253739/it.A/id.282/.f (2011, accessed 2 February 2014). 9. ISO 15189:2012 Medical laboratories – Requirements for quality and competence http://www.iso.org/iso/catalogue_detail?csnumber=56115; see also ISO/FDIS 15189 Medical laboratory accreditations. International Laboratory Accreditation Coordination, ILAC, https:// www.ilac.org/documents/ILAC_Medical_Lab_% 20Accred.pdf (2011, accessed 2 February 2014). 10. Praktijkrichtlijn CCKL [CCKL practical guidelines], versie 4, www.cckl.nl/index.php?pagina=107 (2005, accessed 2 February 2014). 11. Landelijke richtlijn voor de semenbank. Specifieke kwaliteitseisen voor het laboratoriumdeel [National guidelines for semen banks: Specific quality requirements for the laboratory part]. Netherlands Society for Clinical Chemistry and Laboratory Medicine (NVKC) and Association of Clinical Embryologists (KLEM),www. embryologen.nl/images/stories/documenten/richtlijn_ standpunt/landelijke_richtlijn_voor_de_semenbank_ versie_2.0_status_definitief_1_feb_2010f.pdf (2010, accessed 2 February 2014). 12. Mortimer D and Mortimer ST. Quality and risk management in the IVF Laboratory. Cambridge, MA: Cambridge University Press, 2005. 13. Janssens PMW and Cheung KS. Approaching risk analysis and risk management in the fertility laboratory and semen bank. Int J Androl 2009; 32: 656–665. 14. ISO/TS 22367:2008. Medical Laboratories – reduction of error through risk management and continual improvement, www.iso.org/iso/catalogue_detail.htm?csnumber=40918 (2008, accessed 2 February 2014). 15. Krouwer JS, Halim A-B, Hatjimihail AR, et al. Risk management techniques to identify and control laboratory error sources; approved guideline. 2nd ed. Clinical and Laboratory Standards Institute, http://shopping. netsuite.com/c.1253739/site/Sample_pdf/EP18A2_sample. pdf (2009, accessed 2 February 2014).
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
704
Annals of Clinical Biochemistry 51(6)
16. Passey RB and Maluf KC. Foundations for validation of quantitative analytical methods in the clinical laboratory. Arch Pathol Lab Med 1992; 116: 732–738. 17. Westgard JO. Basic method validation. 3rd ed. FAQs, www.westgard.com/basic-method-validation-3rd-editionfaqs.htm (2009, accessed 2 February 2014). 18. Schindhelm RK and Janssens PMW. Risk management: choosing optimal tools on the basis of psychological analysis. In: Zhiyong Z (ed.) Risk assessment and management. AcademyPublish.org, pp.55–60, www. academypublish.org/book/show/title/risk-assessmentand-management (2012, accessed 2 February 2014).
19. Shebl N, Franklin B, Barber N, et al. Failure mode and effects analysis: views of hospital staff in the UK. J Health Serv Res Policy 2012; 17: 37–43. 20. Kessels-Habraken M, Van der Schaaf T, De Jonge J, et al. Integration of prospective and retrospective methods for risk analysis in hospitals. Int J Qual Health Care 2009; 21: 427–432. 21. Snijders C, van der Schaaf TW, Klip H, et al. NEOSAFE study group. Feasibility and reliability of PRISMA-medical for specialty-based incident analysis. Qual Saf Health Care 2009; 18: 486-491. [Erratum in: Qual Saf Health Care. 2011; 20: 396].
Downloaded from acb.sagepub.com at OhioLink on November 13, 2014
