-
SPecial Article
A Model for Quality Assurance in Anesthesiology
Terry Stephen Vitez, MD Division Houston,
Keywords: technique.
Anesthesiology;
quality
of Cardiovascular TX.
assurance;
Introduction Until recently, medicine has paid little attention to the regular assurance of clinical performance and competence. Now hospital administrations and regulatory agencies require medical staffs to conduct quality-assurance programs. Proposals also have been made to base reimbursement and relicensing of hospitals on quality-assurance mechanisms designed to determine clinical competence. Physicians themselves agree that fair and effective quality-assurance systems benefit both patients and practitioners, but most existing efforts are largely pro forma record reviews capable of detecting only the most flagrant errors. While some programs meet the minimum requirements of accrediting agencies, most do not require impartial peer review and are too limited in scope to identify critical errors, clarify the genesis of those errors, and guide judgments about competence. This article describes the principles and processes of a quality-assurance system that monitors the clinical performance of anes-
Address reprint requests to Dr. Vitez at the Division of Cardiovascular Anesthesia, Texas Heart Institute, P.0. Box 20345, Houston, TX 77225, USA. Received for publication July 11, 1989; cepted for publication January 23, 1990.
revised manuscript
0 1990 Butterworth-Heinemann
280
J. Clin. Anesth., vol. 2, July/August 1990
ac-
Anesthesia,
Texas
Heart
Institute,
thesiologists. In contrast to other programs, this system collects information about all perioperative occurrences and uses an objective peer review to judge competence.
Background ‘This model was developed in Las Vegas, Nevada, and has been in use for 4 years. Each year, 60 anesthesiologists perform an estimated 30,000 to 40,000 anesthetics at 5 hospitals in Las Vegas. The hospitals range in size from as few as 100 to more than 700 beds. The anesthesiologists practice at all 5 hospitals, either as members of a fee-for-service group or as solo practitioners. Each hospital has its own departmental hierarchy to administer the quality-assurance model. The hospitals share statistics to form a large unified data base. Examples and results of this 4-year experience are presented in this article.
Principles of the Model The model is based on three principles. First, determining competence is a human decision. Competence cannot be decided by an algorithm or checklist. Competence must be decided by knowledgeable, unbiased peers who consider a wide range of evidence. Second, the best indication of competence is outcome. Physicians with acceptable outcomes are competent; those with unacceptable outcomes are incompetent. Third, humans are inherently,fallibk. ‘The occurrence of an er-
Quality assurance: Vitez
ror, even one of great importance, does not necessarily indicate incompetence. These principles led to a technique ofjudging competence by outcome analysis and error analysis. Outcome analysis determines what happened and why it happened. To be objective and fair, outcome analysis must meet four criteria: 1. Outcome analysis must be continual and collective. Important anesthesia errors are committed infrequently, even by unacceptable practitioners. In addition, standards and performance change with time: what was acceptable last year may not be acceptable today, and a practitioner’s performance can improve or decline. Therefore, it takes time to accumulate evidence about competence. 2. Outcome analysis must be conditional. A practitioner may perform acceptably under the circumstances of his or her usual practice but not under other conditions (e.g., an anesthesiologist performing competently in a practice consisting of adult orthopedic and gynecologic surgery may not be competent to anesthetize infants undergoing repair of complex congenital cardiac defects). 3. Outcome analysis must be comparative. For any incident, an individual’s performance must be compared to that of local peers performing similar anesthesia tasks and cases. 4. Minimal acceptable levels must be established. No matter what the average performance, certain outcomes must be defined as “unacceptable” under any circumstance.
ment and to submit formal reports to the Department. To detect events that may be related to anesthetic management, the Committee formulates criteria for identifying occurrences that should be reported and reviewed. Such criteria direct the system toward collecting nontrivial information. Following full departmental approval, the criteria are distributed to all anesthesia providers and other personnel working in anesthetizing locations. Standard reporting forms are made available at all anesthetizing locations, and the Department requires that any event that might be related to anesthesia management be reported to the Committee. All occurrence reports are submitted to the Quality Assurance Department of the hospital. Under the direction of the Department of Anesthesiology, the Quality Assurance Department collects information about each occurrence. Included in this information is a report from the anesthesia provider describing how the case was managed and why that course of management was chosen. To prevent bias, the Quality Assurance Department replicates the relevant documents and removes all names and identifying marks from these replicates. These anonymous copies are then sent to a member of the Anesthesia Quality Assurance Committee for review. Following a standard protocol, the reviewer prepares a report for the Committee. The reviewer must do the following:
1. Judgments about competence are based on errors made. 2. Incompetent practitioners have error patterns that differ markedly from those of competent practitioners.
the event is related to anesthesia management 2. Grade the seriousness of the event by assigning a negative outcome score (Table I) (This scoring system is important to the process because scoring transforms narrative and descriptive terms into numerical values that can be used for more objective comparisons.) 3. Identify and classify the errors committed [Errors are classified into three categories: management area, nature of error, and genesis of error. Management area defines the aspect of anesthesia prac-
The model uses these principles to create the processes for data collection and data analysis.
Table 1.
Error analysis is the mechanism by which performance and competence are judged. Error analysis is based on two principles:
Data Collection The Department of Anesthesiology forms a committee to perform quality-assurance activities for the Department: the Anesthesiology Quality Assurance Committee. The Committee is directed to research all events that might be related to anesthetic manage-
1. Decide whether
Negative Outcome Scores
0 = no sequelae (e.g., reintubation after esophageal intubation). l-3 = no harm to patient but escalation of care (e.g., postop ventilator due to improper use of relaxant). 4-6 = reversible damage to integrity of an organ (e.g., pulmonary edema from fluid overload). 7-9 = irreversible organ damage (e.g., myocardial infarction). 10 = death.
J. Clin. Anesth., vol. 2, July/August
1990
281
tice involved in the occurrence (7hDlr 2). Kuture qf responsibility (Table 3). If an error in judgment is present, the genusis of‘m-or is determined from the list in Table 4. This list is designed to provide insight into why anesthetic errors occur.] 4. Present a brief explanation and references supporting the conclusions as to how similar errors 5. Make recommendations can be prevented in the future. error designates
After the Committee approves a report, it is presented to the Department of Anesthesiology at a regular departmental meeting. The anonymity of the practitioner is maintained to prevent bias and promote open discussion. Only after the entire Department approves a report does it become a permanent part of a practitioner’s quality-assurance file.
Table 3.
2.
Management Areas
Airway Behavior Violation of’bylaws, rules, regulations, or procedures Unprofessional conduct Substance abuse Circulatory Central and peripheral nervous system Drug action Inhalational agents Intravenous agents Relaxants Opioids Sedative-hypnotics Local anesthetics Allergic reaction Cardiovascular drugs Drug interaction Electrical Endocrine Hematologic Anemia Transfusion Coagulation Hepatic Instrumentation Invasive monitors Noninvasive monitors Intravenous infusion Metabolic Fluid-electrolyte Malignant hyperthermia Pulmonary (lung f-unction) Oxygenation Parenchymal Ventilation Position injury Regional technique Renal Thermoregulation
282
J. Clin. Anesth., vol. 2, July/August 1990
_-._
.__
-.
Konr = xi unavoidable circumstance. Mechanical = failure of’a device. Human = related to formulation or execution 01 a decision. ,~udgrnental = errors occurring when “the action taken is the action intended.” ‘These errors constitute f’ault! decision processes (c’.,~.. mask anesthesia in “full stomach” situation). .Technical = errors occurring when “the action taken is not the action intended” (P.R., syringe swap). Such errors involve mistakes in the execution of a decision. Vigilance = errors associated with a lack of adequate general attention (e.g.. IV infiltration).
Table 4. Table
Kature of E:IX)I -___--
Genesis of Errol
l‘his category is designed to provide insight into whk human errors occur. The classifications used to describe the etiology of human errors are listed here: Inadequate knowledge Didactic Experience Inadequate data Failure to seek data (:ollection of irrelevant data Disrqgard for data Failure to recognize a pattern Failure to accept a conclusion Lack of ari alternative plan
Data Analysis The conclusions of the reports are entered into a commercially available data base designed for personal computers (REFLEX, Borland/Analytics, Inc., Scotts Valley, CA). As the data accumulate, patterns of errors committed by the individuals in the Department become clear. The data base is reviewed periodically (P.R., when an error is uncovered, at reappointment, or annually) or whenever an incident generates concern about a practitioner’s competence. ‘rhe cornerstone of judging competence is error analysis. The principle underlying error analysis is that competence can be judged from the number, type, and severity of errors made. Incompetent practitioners commit errors that are more numerous, less common, and/or more severe than those committed by competent members of the Department. Error analysis is based on information about a provider’s errors that is contained in the data base. When a review is necessary, the practitioner’s error
Quality assurance: Vitez
profile is compiled and compared to the Department’s profile. An error profile contains six comparative elements and three minimal performance elements (Table 5). The comparative elements are used to compare an individual’s performance to that of his or her peers; minimal levels are designed to ensure that all members of the Department are held to a reasonable standard of practice. Minimal levels are meant to be general and outcome oriented rather than specific and task related. For example, a minimum such as “ASA Class I patients should not suffer organ damage or death” is more appropriate than a minimum such as “All patients should receive oxygen in the recovery room.” The minimal levels in Table 5 are based on an analysis of data collected from the hospitals in Las Vegas, Nevada. The analysis demonstrates no reported incidents of anesthesia-related, irreversible organ damage or death in ASA I or II patients in 4 years (estimated 100,000 cases). Therefore, this condition is a minimal level of practice. In dealing with serious illnesses, not every life-threatening situation is resolvable. In the Las Vegas experience, however, the most unacceptable outcomes involve evidence that the practitioner failed to recognize the severity of a problem and to react appropriately. Accordingly, the secthat in life-threatening ond standard requires situations, there must be evidence that the practitioner instituted appropriate life-sustaining actions. This wording acknowledges that even a competent practitioner may make a fatal error but that he or she will recognize the gravity of the situation and take appropriate action to save the patient. Finally, the Las Vegas experience indicates that lack of insight is a common and persuasive factor in decisions to restrict clinical activities. Thus, the final minimal standard requires that when an appropriate peer review judges that an event is related to anesthesia, the practitioner involved should acknowledge that judgment. Table
5.
Judging Clinical Performance The following examples guides judgment about competence.
illustrate how the model clinical performance and
Case 1 An anesthesia provider is applying for reappointment. There is no indication of inadequate clinical performance. A routine error profile is constructed (Figure 1). Error analysis shows a greater than average frequency of anesthesia-related events and an elevated negative outcome score. No minimal levels were violated, and the practitioner committed a normal number of errors per event. The graphic display shows a clustering of errors involving airway management. Further review of negative outcome scores for airway management shows that the average negative outcome score for airway problems is half the practitioner’s score. The practitioner appears to have more than average difficulty in recognizing potential airway problems. The data support reappointment with counseling, education about recognition and management of airway problems, and periodic reassessment of clinical performance.
I
Practitioner’s Error Profile
Comparative elements 1. Frequency of anesthesia-related events 2. Average negative outcome score 3. Number of errors per event 4. Area of clinical management 5. Nature of errors 6. Genesis of errors Minimal performance levels 1. Anesthetizes ASA I and II patients without negative outcome score >6 2. Institutes appropriate life-sustaining actions in lifethreatening situations 3. Displays insight when involved in an important error
1. Error pattern acceptable for reappointment with counseling. (CIRC = circulation; RLXNTS = relaxants; TECH = technical errors; JUDGE = judgmental errors; FSD = failure to seek data; FRP = failure to recognize pattern; FAC = failure to accept conclusion; solid bars = departmental profile; open bars = practitioner’s profile.)
Figure
MD #I: AVEKAGE:
EW?llJ 5 2
N/X. 4 2.6
EvorsiEuent 1-2 l-2
Mill #I 0 0
J. Clin. Anesth., vol. 2, July/August
Min #2 0 0
1990
Mm #3 0 0
283
F+pr_~re 2 shows the profile of an anesthesia provider who has been accused of inadequate clinical performance. Error analysis shows that although the physician had more than the average number of anesthesia-related events, his error profile reflects the departmental average for other comparative elements: he committed only one error per event, and his negative outcome score is acceptable. In addition, he violated no minimal standards. Graphic representation of his errors demonstrates only one important deviation: his pattern for genesis of errors shows a clustering in “failure to accept conclusions.” The data do not support the allegations. In this instance, the system protects the practitioner from unfair criticism.
Case
3
Figure 3 is the profile of an individual who has been involved in a serious complication. The question of suspending his privileges arises. Error analysis demonstrates notable deviations from the normal pattern: there is an increased frequency of events, the individual commits multiple errors per event, the negative outcome score is elevated, and two of the three minimal standards have been violated. Graphic represen-
70 60 50 -
70 60 -
3. Lnacc-eptablr et-1-or p;ittern. ((:lKC = c-ircuLlion; KI.XN~I‘S = relaxants: TE(IH = technical c‘mor.5: ,] LJl)(;k: = jutlgmcntal errors; II)K = inadequate didactic knowledge; FSD = failure to seek data; solid bars = &partniental protile; open b;n-s = pr-actitioner’s protile.) Figure
I: ,'C,,/S Ml) #I:
8
.\Vlx,\(;t.
i'
\ 0, ; L'.fl
k ,,,,,\/I:I~~t!l lf,Ii#I
.\I/,, #-7
\Il,l#i
:i-1
0
0
0
II'
0
0
0
tation of the errors shows a pattern that is cause for concern. There is evidence of repeated errors involving inadequate didactic knowledge. This clustering represents a significant deviation from normal and is important not only because it is elevated but also because others rarely make this type of error. In addition, the data point to the practitioner’s inability to recognize the signs of a serious complication. Lack of knowledge and the inability to recognize danger constitute a serious deficiency. Such a deficiency makes it impossible for the Department to allow the provide] to continue practicing anesthesia. The data support suspension of privileges.
40 30 -
Discussion
20 10 O-
Figure
(CIRC
2. Error pattern acceptable for reappointment. = circulation; RLXNTS = relaxants; TECH = tech-
nical errors; JUDGE = judgmental errors; FSD = f’ailure to seek data; FRP = failure to recognize pattern; FAC = failure to accept conclusion; solid bars = departmental profile; open
284
bars
= practitioner’s
J. Clin. Anesth.,
profile.)
vol. 2, July/August
1990
In 4 years in Las Vegas, 191 problems were judged to be related to anesthesia management. In general, an anesthesia provider made one to two important errors per year. The average negative outcome score was between 2 and 3 (escalation of care). The average number of errors per event was one to two. ‘l’hese data indicate that the average provider is rarely involved in a serious anesthetic error. When an anesthesia-related event occurs, it is usually the result of a single error, and the provider discovers that error before serious harm comes to the patient. Like any other quality-assurance system, the model
Quality assurance:
numerical rating and also describe the outcome. The description must match the type of outcome listed for that numerical score. This process prevents reviewers from using negative outcome scores to reflect perceived culpability and from allocating erroneous escalations in care. Since the model was developed in a large physiciansome people have questioned only department, whether it is applicable to other practice formats. The model is currently being tested in two anesthesia care team practices, in a small department of a rural hospital, in a private practice group to monitor performance of the partners, and in a training program to track the progress of anesthesia residents.* Under these different circumstances, the model may be modified to solve problems particular to a practice setting. For example, in small departments where it is difficult to conceal the identity of the provider, the reviewer may have to be granted access to original records. However, only the reviewer will know the provider’s identity, and the reviewer’s conclusion must pass the scrutiny of the Anesthesiology Quality Assurance Committee. In very small departments, case reviews may be shared with other institutions, or department members may submit reviews of their own cases. In contrast to these difficulties, the model has several benefits. The system offers insight into the type and cause of anesthesia complications occurring in the anesthesiology departments. Such insights can be used to identify common or individual problems. For example, Table 6 shows the areas of management and the negative outcome scores most frequently involved in 19 1 anesthetic-related errors. Seventy-five percent of these errors involved errors of judgment. The genesis of the errors and their negative outcome scores are shown in Table 7.
has potential failings. The reporting system is semivoluntary and allows reporting from nonphysicians. Voluntary reporting systems may allow important events to go undetected. Although the model relies heavily on volunteer reporting from physicians and nurses, other sources for detecting errors include medical records review, surgical case review, periodic audits, and transfusion committee reports. While no system can guarantee 100% reporting, biases may be introduced if many events are not reported. To estimate the sensitivity of the model, a review of 4,500 anesthesia records was performed in Las Vegas. Each record was reviewed to discover indicators of possible anesthesia-related errors. Thirty-three events that should have been reported to the Department of Anesthesiology were identified. Twentytwo of these events were reported; 6 of the remaining 11 were judged not related to anesthesia; the 5 cases related to anesthesia were associated with minor negative outcome scores. Thus, the method does appear to detect most of the important events. The issue of allowing nonphysicians to report events anonymously to the Department of Anesthesiology should not be of great concern because the reports are submitted only to the Department, which then reviews and judges them in anonymity. This method is clearly preferable to having individuals report problems to other departments or to the hospital administration. In reviewing a case, difficulties may arise in judging whether an event is related to anesthesia and in assigning a negative outcome score. In deciding whether an event is related to anesthesia, the model requires that there be good evidence of what occurred. Inadequate data (e.g., no autopsy) or a concomitant disease often make it impossible to determine whether the event was related to anesthesia. Under these circumstances, the model stipulates that judgment favor the practitioner, and a classification of “not related to anesthesia” is made. In assigning a negative outcome score, the model requires that the reviewer record a
Table 6.
Vitez
*Aultman Anesthesia in Canton, Ohio, and Rochester General in Rochester, New York; St. Francis Hospital in Bend, Oregon; Anesthesia Associates in Reno, Nevada; Mary-Hitchcock Clinic in Hanover, New Hampshire.
Negative Outcome Scores for Areas of Management
Most Frequently
Involved in Anesthetic Errors
Negative Outcome Scores (% of all scores)
Airway Circulatory Relaxants All other drugs Regional % of all scores
0
l-3
4-6
12 3 3 5 3 36
8 5 9 6 5 41
5 3 0 0 0 12
7-9
10
% of All Errors
0 0 0 4
2 5 0 1 1 8
29 18 12 12 9 -
J. Clin.
Anesth.,
vol.
2, July/August
1990
285
Table 7.
Negativr
Outcome
Scores
for Genesis
of Judgmental
Errors
Negative Outcome Scores (% of all scores)
1K:DID IK:EXP ID:FSD DD:FRP DD:FAC LAP 70 of all scores
0
l-3
2 1 H 6 2 0 36
4 2 17 fi 2 0 41
4-6
7-9
10
% of All Errors
1
0
1
2 1
1 2 1
H H 34
3
20
4 3 2 I 12
1 1 0 4
I
8
0 x
1 -
IK:DID = inadequate didactic knowledge; IK:EXP = inadequate experience; ID:FSD = l’ailure to seek adequate data; to recognize a pattern; DD:FAC = failure to accept a conclusion; LAP = lack of’ an alternative plan.
The most frequent error patterns involved mismanagement of the airway. Most airway errors stemmed from a failure to perform an appropriate examination (“failure to seek data”) or failure to recognize that the history and physical examination indicated a potential airway problem (“failure to recognize pattern”). The most dangerous errors involved mismanagement of the circulation. Circulatory errors involved inadequate invasive monitors (“failure to seek data”) and failure to recognize that existing data (history, examination, or an intraoperative event) indicated a circulatory problem. Once a departmental problem is identified, strategies for prevention can be devised and implemented. For example, early quality-assurance data in Las Vegas showed a common problem to be inadequate reversal of muscle relaxants. Starting in 1986, the anesthesiology departments took steps to reduce this problem (promoted more frequent use of nerve stimulators, held educational sessions, and required measurements of ventilatory capabilities prior to removal of ventilatory support). These measures decreased the frequency of inadequate ventilation secondary to residual paralysis from as high as 18% of all complications to 2% to 3% (FZgure 4). The system provides the opportunity to institute changes that will prevent future errors or lessen negative outcome scores. As stated above, the most frequent errors involved mismanagement of the airway, due partly to a failure to examine the airway preoperatively. These findings prompted a change in the anesthetic preoperative evaluation form. The new form has an area specifically labeled for evaluation of the airway, drawing the physician’s attention to this important aspect of preoperative assessment. The most dangerous errors involved mismanagement of the circulation stemming from inadequate invasive monitors and failure to recognize a data pattern indicative of 286
J. Clin. Anesth.,
vol. 2, July/August
1990
86
Figure 4. Kelaxant
errors
Year
87
;LS a percent
DD:FRP
= t’ailure
88
ot‘ all el-rors
(IW.‘,
to 198X). a circulatory problem. In response to this disclosure, the departments encouraged more frequent use of invasive monitors. The Anesthesiology Quality Assurance Committee recognized, however, that many of these situations involved a reluctance to place an invasive monitor because of time, cost, or fear of complication. Some situations involved an unfamiliarity with invasive monitors. With these insights, the departments pursued the purchase of noninvasive monitors (electrical impedance devices-two-dimensional transesophageal echo monitors) and created a program for education about, and certification for the use of, these devices. The departments also modified the requirements for staff appointment to include the documented ability to insert and interpret invasive monitors (arterial, central venous, and pulmonary artery catheters). In addition to these actions, it is possible that periodic review of practitioners’ profiles could identify those individuals with error patterns predictive of high negative outcome scores. The practitioners could then be advised that they must change their practice patterns.
Quality assurance: Viter
In this model of quality assurance, when decisions about errors and competence are made, practitioners have many opportunities to present their observations and interpretations. Anonymity is maintained throughout all the discussions, and decisions are made by the entire department. Therefore, this model provides a method to deal with conflicts between competitors. All members are treated equally. Practitioners are judged by all their peers, not just a few. In 4 years under this system, the departments have made two recommendations to suspend privileges and one recommendation to monitor a practitioner. If the instances where suspension was recommended, a specific case raised the issue of competence. Common factors leading a department to recommend suspension were (1) violation of minimal standards 2 and 3, (2) a patient’s death, (3) multiple errors per case, (4) errors involving inadequate didactic knowledge, and (5) an error profile demonstrating other high negative outcome scores. The factors prompting the recommendation that a provider’s practice be evaluated by appointed members of the department (monitorship)
were (1) violation of minimal standard 2 and (2) a lack of other data about the provider’s practice. The model emphasizes that the goal of quality assurance is to improve care, not to harass innocent physicians. It supports those goals without compromising its responsibility of identifying incompetent practitioners. In this system, outcome analysis helps identify the problems, and error analysis helps determine whether the problems are remediable. When problems are judged remediable, the system provides information about what needs to be changed and a method for reevaluation. When problems are judged irremediable, the model provides data to substantiate difficult decisions about interrupting an individual’s medical career.
References Cooper JB, Newbower RS, Kitz RJ: An analysis of major errors and equipment failures in anesthesia management: considerations for prevention and detection. Anesthesiology 1984;60:34-42.
J. Clin. Anesth., vol. 2, July/August 1990
287
SPecial Article
A Model for Quality Assurance in Anesthesiology
Terry Stephen Vitez, MD Division Houston,
Keywords: technique.
Anesthesiology;
quality
of Cardiovascular TX.
assurance;
Introduction Until recently, medicine has paid little attention to the regular assurance of clinical performance and competence. Now hospital administrations and regulatory agencies require medical staffs to conduct quality-assurance programs. Proposals also have been made to base reimbursement and relicensing of hospitals on quality-assurance mechanisms designed to determine clinical competence. Physicians themselves agree that fair and effective quality-assurance systems benefit both patients and practitioners, but most existing efforts are largely pro forma record reviews capable of detecting only the most flagrant errors. While some programs meet the minimum requirements of accrediting agencies, most do not require impartial peer review and are too limited in scope to identify critical errors, clarify the genesis of those errors, and guide judgments about competence. This article describes the principles and processes of a quality-assurance system that monitors the clinical performance of anes-
Address reprint requests to Dr. Vitez at the Division of Cardiovascular Anesthesia, Texas Heart Institute, P.0. Box 20345, Houston, TX 77225, USA. Received for publication July 11, 1989; cepted for publication January 23, 1990.
revised manuscript
0 1990 Butterworth-Heinemann
280
J. Clin. Anesth., vol. 2, July/August 1990
ac-
Anesthesia,
Texas
Heart
Institute,
thesiologists. In contrast to other programs, this system collects information about all perioperative occurrences and uses an objective peer review to judge competence.
Background ‘This model was developed in Las Vegas, Nevada, and has been in use for 4 years. Each year, 60 anesthesiologists perform an estimated 30,000 to 40,000 anesthetics at 5 hospitals in Las Vegas. The hospitals range in size from as few as 100 to more than 700 beds. The anesthesiologists practice at all 5 hospitals, either as members of a fee-for-service group or as solo practitioners. Each hospital has its own departmental hierarchy to administer the quality-assurance model. The hospitals share statistics to form a large unified data base. Examples and results of this 4-year experience are presented in this article.
Principles of the Model The model is based on three principles. First, determining competence is a human decision. Competence cannot be decided by an algorithm or checklist. Competence must be decided by knowledgeable, unbiased peers who consider a wide range of evidence. Second, the best indication of competence is outcome. Physicians with acceptable outcomes are competent; those with unacceptable outcomes are incompetent. Third, humans are inherently,fallibk. ‘The occurrence of an er-
Quality assurance: Vitez
ror, even one of great importance, does not necessarily indicate incompetence. These principles led to a technique ofjudging competence by outcome analysis and error analysis. Outcome analysis determines what happened and why it happened. To be objective and fair, outcome analysis must meet four criteria: 1. Outcome analysis must be continual and collective. Important anesthesia errors are committed infrequently, even by unacceptable practitioners. In addition, standards and performance change with time: what was acceptable last year may not be acceptable today, and a practitioner’s performance can improve or decline. Therefore, it takes time to accumulate evidence about competence. 2. Outcome analysis must be conditional. A practitioner may perform acceptably under the circumstances of his or her usual practice but not under other conditions (e.g., an anesthesiologist performing competently in a practice consisting of adult orthopedic and gynecologic surgery may not be competent to anesthetize infants undergoing repair of complex congenital cardiac defects). 3. Outcome analysis must be comparative. For any incident, an individual’s performance must be compared to that of local peers performing similar anesthesia tasks and cases. 4. Minimal acceptable levels must be established. No matter what the average performance, certain outcomes must be defined as “unacceptable” under any circumstance.
ment and to submit formal reports to the Department. To detect events that may be related to anesthetic management, the Committee formulates criteria for identifying occurrences that should be reported and reviewed. Such criteria direct the system toward collecting nontrivial information. Following full departmental approval, the criteria are distributed to all anesthesia providers and other personnel working in anesthetizing locations. Standard reporting forms are made available at all anesthetizing locations, and the Department requires that any event that might be related to anesthesia management be reported to the Committee. All occurrence reports are submitted to the Quality Assurance Department of the hospital. Under the direction of the Department of Anesthesiology, the Quality Assurance Department collects information about each occurrence. Included in this information is a report from the anesthesia provider describing how the case was managed and why that course of management was chosen. To prevent bias, the Quality Assurance Department replicates the relevant documents and removes all names and identifying marks from these replicates. These anonymous copies are then sent to a member of the Anesthesia Quality Assurance Committee for review. Following a standard protocol, the reviewer prepares a report for the Committee. The reviewer must do the following:
1. Judgments about competence are based on errors made. 2. Incompetent practitioners have error patterns that differ markedly from those of competent practitioners.
the event is related to anesthesia management 2. Grade the seriousness of the event by assigning a negative outcome score (Table I) (This scoring system is important to the process because scoring transforms narrative and descriptive terms into numerical values that can be used for more objective comparisons.) 3. Identify and classify the errors committed [Errors are classified into three categories: management area, nature of error, and genesis of error. Management area defines the aspect of anesthesia prac-
The model uses these principles to create the processes for data collection and data analysis.
Table 1.
Error analysis is the mechanism by which performance and competence are judged. Error analysis is based on two principles:
Data Collection The Department of Anesthesiology forms a committee to perform quality-assurance activities for the Department: the Anesthesiology Quality Assurance Committee. The Committee is directed to research all events that might be related to anesthetic manage-
1. Decide whether
Negative Outcome Scores
0 = no sequelae (e.g., reintubation after esophageal intubation). l-3 = no harm to patient but escalation of care (e.g., postop ventilator due to improper use of relaxant). 4-6 = reversible damage to integrity of an organ (e.g., pulmonary edema from fluid overload). 7-9 = irreversible organ damage (e.g., myocardial infarction). 10 = death.
J. Clin. Anesth., vol. 2, July/August
1990
281
tice involved in the occurrence (7hDlr 2). Kuture qf responsibility (Table 3). If an error in judgment is present, the genusis of‘m-or is determined from the list in Table 4. This list is designed to provide insight into why anesthetic errors occur.] 4. Present a brief explanation and references supporting the conclusions as to how similar errors 5. Make recommendations can be prevented in the future. error designates
After the Committee approves a report, it is presented to the Department of Anesthesiology at a regular departmental meeting. The anonymity of the practitioner is maintained to prevent bias and promote open discussion. Only after the entire Department approves a report does it become a permanent part of a practitioner’s quality-assurance file.
Table 3.
2.
Management Areas
Airway Behavior Violation of’bylaws, rules, regulations, or procedures Unprofessional conduct Substance abuse Circulatory Central and peripheral nervous system Drug action Inhalational agents Intravenous agents Relaxants Opioids Sedative-hypnotics Local anesthetics Allergic reaction Cardiovascular drugs Drug interaction Electrical Endocrine Hematologic Anemia Transfusion Coagulation Hepatic Instrumentation Invasive monitors Noninvasive monitors Intravenous infusion Metabolic Fluid-electrolyte Malignant hyperthermia Pulmonary (lung f-unction) Oxygenation Parenchymal Ventilation Position injury Regional technique Renal Thermoregulation
282
J. Clin. Anesth., vol. 2, July/August 1990
_-._
.__
-.
Konr = xi unavoidable circumstance. Mechanical = failure of’a device. Human = related to formulation or execution 01 a decision. ,~udgrnental = errors occurring when “the action taken is the action intended.” ‘These errors constitute f’ault! decision processes (c’.,~.. mask anesthesia in “full stomach” situation). .Technical = errors occurring when “the action taken is not the action intended” (P.R., syringe swap). Such errors involve mistakes in the execution of a decision. Vigilance = errors associated with a lack of adequate general attention (e.g.. IV infiltration).
Table 4. Table
Kature of E:IX)I -___--
Genesis of Errol
l‘his category is designed to provide insight into whk human errors occur. The classifications used to describe the etiology of human errors are listed here: Inadequate knowledge Didactic Experience Inadequate data Failure to seek data (:ollection of irrelevant data Disrqgard for data Failure to recognize a pattern Failure to accept a conclusion Lack of ari alternative plan
Data Analysis The conclusions of the reports are entered into a commercially available data base designed for personal computers (REFLEX, Borland/Analytics, Inc., Scotts Valley, CA). As the data accumulate, patterns of errors committed by the individuals in the Department become clear. The data base is reviewed periodically (P.R., when an error is uncovered, at reappointment, or annually) or whenever an incident generates concern about a practitioner’s competence. ‘rhe cornerstone of judging competence is error analysis. The principle underlying error analysis is that competence can be judged from the number, type, and severity of errors made. Incompetent practitioners commit errors that are more numerous, less common, and/or more severe than those committed by competent members of the Department. Error analysis is based on information about a provider’s errors that is contained in the data base. When a review is necessary, the practitioner’s error
Quality assurance: Vitez
profile is compiled and compared to the Department’s profile. An error profile contains six comparative elements and three minimal performance elements (Table 5). The comparative elements are used to compare an individual’s performance to that of his or her peers; minimal levels are designed to ensure that all members of the Department are held to a reasonable standard of practice. Minimal levels are meant to be general and outcome oriented rather than specific and task related. For example, a minimum such as “ASA Class I patients should not suffer organ damage or death” is more appropriate than a minimum such as “All patients should receive oxygen in the recovery room.” The minimal levels in Table 5 are based on an analysis of data collected from the hospitals in Las Vegas, Nevada. The analysis demonstrates no reported incidents of anesthesia-related, irreversible organ damage or death in ASA I or II patients in 4 years (estimated 100,000 cases). Therefore, this condition is a minimal level of practice. In dealing with serious illnesses, not every life-threatening situation is resolvable. In the Las Vegas experience, however, the most unacceptable outcomes involve evidence that the practitioner failed to recognize the severity of a problem and to react appropriately. Accordingly, the secthat in life-threatening ond standard requires situations, there must be evidence that the practitioner instituted appropriate life-sustaining actions. This wording acknowledges that even a competent practitioner may make a fatal error but that he or she will recognize the gravity of the situation and take appropriate action to save the patient. Finally, the Las Vegas experience indicates that lack of insight is a common and persuasive factor in decisions to restrict clinical activities. Thus, the final minimal standard requires that when an appropriate peer review judges that an event is related to anesthesia, the practitioner involved should acknowledge that judgment. Table
5.
Judging Clinical Performance The following examples guides judgment about competence.
illustrate how the model clinical performance and
Case 1 An anesthesia provider is applying for reappointment. There is no indication of inadequate clinical performance. A routine error profile is constructed (Figure 1). Error analysis shows a greater than average frequency of anesthesia-related events and an elevated negative outcome score. No minimal levels were violated, and the practitioner committed a normal number of errors per event. The graphic display shows a clustering of errors involving airway management. Further review of negative outcome scores for airway management shows that the average negative outcome score for airway problems is half the practitioner’s score. The practitioner appears to have more than average difficulty in recognizing potential airway problems. The data support reappointment with counseling, education about recognition and management of airway problems, and periodic reassessment of clinical performance.
I
Practitioner’s Error Profile
Comparative elements 1. Frequency of anesthesia-related events 2. Average negative outcome score 3. Number of errors per event 4. Area of clinical management 5. Nature of errors 6. Genesis of errors Minimal performance levels 1. Anesthetizes ASA I and II patients without negative outcome score >6 2. Institutes appropriate life-sustaining actions in lifethreatening situations 3. Displays insight when involved in an important error
1. Error pattern acceptable for reappointment with counseling. (CIRC = circulation; RLXNTS = relaxants; TECH = technical errors; JUDGE = judgmental errors; FSD = failure to seek data; FRP = failure to recognize pattern; FAC = failure to accept conclusion; solid bars = departmental profile; open bars = practitioner’s profile.)
Figure
MD #I: AVEKAGE:
EW?llJ 5 2
N/X. 4 2.6
EvorsiEuent 1-2 l-2
Mill #I 0 0
J. Clin. Anesth., vol. 2, July/August
Min #2 0 0
1990
Mm #3 0 0
283
F+pr_~re 2 shows the profile of an anesthesia provider who has been accused of inadequate clinical performance. Error analysis shows that although the physician had more than the average number of anesthesia-related events, his error profile reflects the departmental average for other comparative elements: he committed only one error per event, and his negative outcome score is acceptable. In addition, he violated no minimal standards. Graphic representation of his errors demonstrates only one important deviation: his pattern for genesis of errors shows a clustering in “failure to accept conclusions.” The data do not support the allegations. In this instance, the system protects the practitioner from unfair criticism.
Case
3
Figure 3 is the profile of an individual who has been involved in a serious complication. The question of suspending his privileges arises. Error analysis demonstrates notable deviations from the normal pattern: there is an increased frequency of events, the individual commits multiple errors per event, the negative outcome score is elevated, and two of the three minimal standards have been violated. Graphic represen-
70 60 50 -
70 60 -
3. Lnacc-eptablr et-1-or p;ittern. ((:lKC = c-ircuLlion; KI.XN~I‘S = relaxants: TE(IH = technical c‘mor.5: ,] LJl)(;k: = jutlgmcntal errors; II)K = inadequate didactic knowledge; FSD = failure to seek data; solid bars = &partniental protile; open b;n-s = pr-actitioner’s protile.) Figure
I: ,'C,,/S Ml) #I:
8
.\Vlx,\(;t.
i'
\ 0, ; L'.fl
k ,,,,,\/I:I~~t!l lf,Ii#I
.\I/,, #-7
\Il,l#i
:i-1
0
0
0
II'
0
0
0
tation of the errors shows a pattern that is cause for concern. There is evidence of repeated errors involving inadequate didactic knowledge. This clustering represents a significant deviation from normal and is important not only because it is elevated but also because others rarely make this type of error. In addition, the data point to the practitioner’s inability to recognize the signs of a serious complication. Lack of knowledge and the inability to recognize danger constitute a serious deficiency. Such a deficiency makes it impossible for the Department to allow the provide] to continue practicing anesthesia. The data support suspension of privileges.
40 30 -
Discussion
20 10 O-
Figure
(CIRC
2. Error pattern acceptable for reappointment. = circulation; RLXNTS = relaxants; TECH = tech-
nical errors; JUDGE = judgmental errors; FSD = f’ailure to seek data; FRP = failure to recognize pattern; FAC = failure to accept conclusion; solid bars = departmental profile; open
284
bars
= practitioner’s
J. Clin. Anesth.,
profile.)
vol. 2, July/August
1990
In 4 years in Las Vegas, 191 problems were judged to be related to anesthesia management. In general, an anesthesia provider made one to two important errors per year. The average negative outcome score was between 2 and 3 (escalation of care). The average number of errors per event was one to two. ‘l’hese data indicate that the average provider is rarely involved in a serious anesthetic error. When an anesthesia-related event occurs, it is usually the result of a single error, and the provider discovers that error before serious harm comes to the patient. Like any other quality-assurance system, the model
Quality assurance:
numerical rating and also describe the outcome. The description must match the type of outcome listed for that numerical score. This process prevents reviewers from using negative outcome scores to reflect perceived culpability and from allocating erroneous escalations in care. Since the model was developed in a large physiciansome people have questioned only department, whether it is applicable to other practice formats. The model is currently being tested in two anesthesia care team practices, in a small department of a rural hospital, in a private practice group to monitor performance of the partners, and in a training program to track the progress of anesthesia residents.* Under these different circumstances, the model may be modified to solve problems particular to a practice setting. For example, in small departments where it is difficult to conceal the identity of the provider, the reviewer may have to be granted access to original records. However, only the reviewer will know the provider’s identity, and the reviewer’s conclusion must pass the scrutiny of the Anesthesiology Quality Assurance Committee. In very small departments, case reviews may be shared with other institutions, or department members may submit reviews of their own cases. In contrast to these difficulties, the model has several benefits. The system offers insight into the type and cause of anesthesia complications occurring in the anesthesiology departments. Such insights can be used to identify common or individual problems. For example, Table 6 shows the areas of management and the negative outcome scores most frequently involved in 19 1 anesthetic-related errors. Seventy-five percent of these errors involved errors of judgment. The genesis of the errors and their negative outcome scores are shown in Table 7.
has potential failings. The reporting system is semivoluntary and allows reporting from nonphysicians. Voluntary reporting systems may allow important events to go undetected. Although the model relies heavily on volunteer reporting from physicians and nurses, other sources for detecting errors include medical records review, surgical case review, periodic audits, and transfusion committee reports. While no system can guarantee 100% reporting, biases may be introduced if many events are not reported. To estimate the sensitivity of the model, a review of 4,500 anesthesia records was performed in Las Vegas. Each record was reviewed to discover indicators of possible anesthesia-related errors. Thirty-three events that should have been reported to the Department of Anesthesiology were identified. Twentytwo of these events were reported; 6 of the remaining 11 were judged not related to anesthesia; the 5 cases related to anesthesia were associated with minor negative outcome scores. Thus, the method does appear to detect most of the important events. The issue of allowing nonphysicians to report events anonymously to the Department of Anesthesiology should not be of great concern because the reports are submitted only to the Department, which then reviews and judges them in anonymity. This method is clearly preferable to having individuals report problems to other departments or to the hospital administration. In reviewing a case, difficulties may arise in judging whether an event is related to anesthesia and in assigning a negative outcome score. In deciding whether an event is related to anesthesia, the model requires that there be good evidence of what occurred. Inadequate data (e.g., no autopsy) or a concomitant disease often make it impossible to determine whether the event was related to anesthesia. Under these circumstances, the model stipulates that judgment favor the practitioner, and a classification of “not related to anesthesia” is made. In assigning a negative outcome score, the model requires that the reviewer record a
Table 6.
Vitez
*Aultman Anesthesia in Canton, Ohio, and Rochester General in Rochester, New York; St. Francis Hospital in Bend, Oregon; Anesthesia Associates in Reno, Nevada; Mary-Hitchcock Clinic in Hanover, New Hampshire.
Negative Outcome Scores for Areas of Management
Most Frequently
Involved in Anesthetic Errors
Negative Outcome Scores (% of all scores)
Airway Circulatory Relaxants All other drugs Regional % of all scores
0
l-3
4-6
12 3 3 5 3 36
8 5 9 6 5 41
5 3 0 0 0 12
7-9
10
% of All Errors
0 0 0 4
2 5 0 1 1 8
29 18 12 12 9 -
J. Clin.
Anesth.,
vol.
2, July/August
1990
285
Table 7.
Negativr
Outcome
Scores
for Genesis
of Judgmental
Errors
Negative Outcome Scores (% of all scores)
1K:DID IK:EXP ID:FSD DD:FRP DD:FAC LAP 70 of all scores
0
l-3
2 1 H 6 2 0 36
4 2 17 fi 2 0 41
4-6
7-9
10
% of All Errors
1
0
1
2 1
1 2 1
H H 34
3
20
4 3 2 I 12
1 1 0 4
I
8
0 x
1 -
IK:DID = inadequate didactic knowledge; IK:EXP = inadequate experience; ID:FSD = l’ailure to seek adequate data; to recognize a pattern; DD:FAC = failure to accept a conclusion; LAP = lack of’ an alternative plan.
The most frequent error patterns involved mismanagement of the airway. Most airway errors stemmed from a failure to perform an appropriate examination (“failure to seek data”) or failure to recognize that the history and physical examination indicated a potential airway problem (“failure to recognize pattern”). The most dangerous errors involved mismanagement of the circulation. Circulatory errors involved inadequate invasive monitors (“failure to seek data”) and failure to recognize that existing data (history, examination, or an intraoperative event) indicated a circulatory problem. Once a departmental problem is identified, strategies for prevention can be devised and implemented. For example, early quality-assurance data in Las Vegas showed a common problem to be inadequate reversal of muscle relaxants. Starting in 1986, the anesthesiology departments took steps to reduce this problem (promoted more frequent use of nerve stimulators, held educational sessions, and required measurements of ventilatory capabilities prior to removal of ventilatory support). These measures decreased the frequency of inadequate ventilation secondary to residual paralysis from as high as 18% of all complications to 2% to 3% (FZgure 4). The system provides the opportunity to institute changes that will prevent future errors or lessen negative outcome scores. As stated above, the most frequent errors involved mismanagement of the airway, due partly to a failure to examine the airway preoperatively. These findings prompted a change in the anesthetic preoperative evaluation form. The new form has an area specifically labeled for evaluation of the airway, drawing the physician’s attention to this important aspect of preoperative assessment. The most dangerous errors involved mismanagement of the circulation stemming from inadequate invasive monitors and failure to recognize a data pattern indicative of 286
J. Clin. Anesth.,
vol. 2, July/August
1990
86
Figure 4. Kelaxant
errors
Year
87
;LS a percent
DD:FRP
= t’ailure
88
ot‘ all el-rors
(IW.‘,
to 198X). a circulatory problem. In response to this disclosure, the departments encouraged more frequent use of invasive monitors. The Anesthesiology Quality Assurance Committee recognized, however, that many of these situations involved a reluctance to place an invasive monitor because of time, cost, or fear of complication. Some situations involved an unfamiliarity with invasive monitors. With these insights, the departments pursued the purchase of noninvasive monitors (electrical impedance devices-two-dimensional transesophageal echo monitors) and created a program for education about, and certification for the use of, these devices. The departments also modified the requirements for staff appointment to include the documented ability to insert and interpret invasive monitors (arterial, central venous, and pulmonary artery catheters). In addition to these actions, it is possible that periodic review of practitioners’ profiles could identify those individuals with error patterns predictive of high negative outcome scores. The practitioners could then be advised that they must change their practice patterns.
Quality assurance: Viter
In this model of quality assurance, when decisions about errors and competence are made, practitioners have many opportunities to present their observations and interpretations. Anonymity is maintained throughout all the discussions, and decisions are made by the entire department. Therefore, this model provides a method to deal with conflicts between competitors. All members are treated equally. Practitioners are judged by all their peers, not just a few. In 4 years under this system, the departments have made two recommendations to suspend privileges and one recommendation to monitor a practitioner. If the instances where suspension was recommended, a specific case raised the issue of competence. Common factors leading a department to recommend suspension were (1) violation of minimal standards 2 and 3, (2) a patient’s death, (3) multiple errors per case, (4) errors involving inadequate didactic knowledge, and (5) an error profile demonstrating other high negative outcome scores. The factors prompting the recommendation that a provider’s practice be evaluated by appointed members of the department (monitorship)
were (1) violation of minimal standard 2 and (2) a lack of other data about the provider’s practice. The model emphasizes that the goal of quality assurance is to improve care, not to harass innocent physicians. It supports those goals without compromising its responsibility of identifying incompetent practitioners. In this system, outcome analysis helps identify the problems, and error analysis helps determine whether the problems are remediable. When problems are judged remediable, the system provides information about what needs to be changed and a method for reevaluation. When problems are judged irremediable, the model provides data to substantiate difficult decisions about interrupting an individual’s medical career.
References Cooper JB, Newbower RS, Kitz RJ: An analysis of major errors and equipment failures in anesthesia management: considerations for prevention and detection. Anesthesiology 1984;60:34-42.
J. Clin. Anesth., vol. 2, July/August 1990
287
