ORIGINAL ARTICLE
Development of an Electronic Pediatric All-Cause Harm Measurement Tool Using a Modified Delphi Method David Christopher Stockwell, MD, MBA,*† Hema Bisarya, MHSA, RD,‡ David C. Classen, MD,§k Eric S. Kirkendall, MD,¶# Peter I. Lachman, MBBS,** Anne G. Matlow, MD,†† Eric Tham, MD,‡‡§§ Dan Hyman, MD,‡‡kk Samuel M. Lehman, MD,¶¶ Elizabeth Searles, RN,## Stephen E. Muething, MD,# and Paul J. Sharek, MD***††† Objectives: To have impact on reducing harm in pediatric inpatients, an efficient and reliable process for harm detection is needed. This work describes the first step toward the development of a pediatric all-cause harm measurement tool by recognized experts in the field. Methods: An international group of leaders in pediatric patient safety and informatics were charged with developing a comprehensive pediatric inpatient all-cause harm measurement tool using a modified Delphi technique. The process was conducted in 5 distinct steps: (1) literature review of triggers (elements from a medical record that assist in identifying patient harm) for inclusion; (2) translation of triggers to likely associated harm, improving the ability for expert prioritization; (3) 2 applications of a modified Delphi selection approach with consensus criteria using severity and frequency of harm as well as detectability of the associated trigger as criteria to rate each trigger and associated harm; (4) developing specific trigger logic and relevant values when applicable; and (5) final vetting of the entire trigger list for pilot testing. Results: Literature and expert panel review identified 108 triggers and associated harms suitable for consideration (steps 1 and 2). This list was pared to 64 triggers and their associated harms after the first of the 2 independent expert reviews. The second independent expert review led to further refinement of the trigger package, resulting in 46 items for inclusion (step 3). Adding in specific trigger logic expanded the list. Final review and voting resulted in a list of 51 triggers (steps 4 and 5). Conclusions: Application of a modified Delphi method on an expertconstructed list of 108 triggers, focusing on severity and frequency of harms as well as detectability of triggers in an electronic medical record, resulted in a final list of 51 pediatric triggers. Pilot testing this list of pediatric triggers to identify all-cause harm for pediatric inpatients is the next step to establish the appropriateness of each trigger for inclusion in a global pediatric safety measurement tool. From the *Division of Critical Care Medicine, Department of Pediatrics, School of Medicine, The George Washington University, Washington, DC; †Center for Quality and Improvement Science, Children's National Medical Center, Washington, DC; ‡Children's Hospital Association, Overland Park, KS; §Department of Infectious Disease, School of Medicine, University of Utah, Salt Lake City, UT; kPascal Metrics, Washington, DC; ¶Division of Biomedical Informatics, Division of Hospital Medicine, Department of Pediatrics, University of Cincinnati, Cincinnati, OH; #James M. Anderson Center for Health Systems Excellence, Cincinnati Children's Hospital Medical Center, Cincinnati, OH; **Medical Directors Office, Quality and Safety, Great Ormond Street Hospital NHS Foundation Trust, London, England; ††Departments of Paediatrics and Medicine and Centre for Patient Safety, University of Toronto, Toronto, ON; ‡‡Department of Pediatrics, School of Medicine, University of Colorado, Aurora, CO; §§Research Institute, and kkDepartment of Quality and Patient Safety, Children's Hospital Colorado, Aurora, CO; ¶¶Department of Anesthesia and Critical Care Medicine, and ##Department of Quality, Children's Hospital Central California, Madera, CA; ***Division of General Pediatrics, Department of Pediatrics, School of Medicine, Stanford University palo Alto, CA; and †††Center for Quality and Clinical Effectiveness, Lucile Packard Children's Hospital, Palo Alto, CA. Correspondence: David Christopher Stockwell, MD, MBA, Center for Quality and Improvement Science, Children's National Medical Center, 111 Michigan Ave, NW, Suite M-4800, Washington, DC 20010 (e‐mail: [email protected]). The authors disclose no conflict of interest. Supported by the Children's Hospital Association. Copyright © 2014 by Lippincott Williams & Wilkins
J Patient Saf • Volume 00, Number 00, Month 2014
Key Words: harms, patient safety, harm, trigger, Delphi method, adverse event (J Patient Saf 2014;00: 00–00)
A
ccording to a recent study of adult inpatients in North Carolina, rates of medically induced harm of hospitalized adults have not improved in the last 10 years since the release of the Institute of Medicine's To Err Is Human report in 1999.1 This study provided multiple significant conclusions including the following: (1) despite major efforts to improve patient safety, harm rates have not decreased over time, and (2) previously used harm detection techniques have not produced consistent approaches to measure harm. A robust and reliable method of harm detection is needed to have consistent measurement across hospitals, as well as to provide reliable evidence of change in harm rates over time. The mainstay of harm detection in most hospitals is voluntary incident reporting. Unfortunately, voluntary reports have been shown to capture only 2% to 8% of all harms that occur.2 Several other detection methods have been implemented with various successes and limitations.3 Recently, the U.S. Office of the Inspector General commissioned work to compare voluntary reporting and the Institute for Healthcare Improvement (IHI) Adult Global Trigger Tool.4,5 A trigger is defined as an “occurrence, prompt, or flag found on manual review of the medical chart that ‘triggers’ further investigation to determine the presence or absence of an adverse event.”6,7 The findings suggested that the trigger tool approach is significantly more sensitive than voluntary reporting for identifying harm.4 Another study of the IHI Global Trigger Tool revealed 10-fold higher harm rates compared with either the administrative coding-based Agency for Healthcare Research and Quality's (AHRQ) Patient Safety Indicators or hospital voluntary reporting.8 Evidence consistently reveals the trigger tool methodology to be the most sensitive “all-cause harm” detection method presently available. Numerous pediatric investigators have attempted to augment voluntary reporting by using triggers similar to those in the adult-focused IHI Global Trigger Tool used in the North Carolina Patient Safety Study, the U.S. Office of the Inspector General report, and others.9–14 Pediatric-focused trigger tools have been created for the neonatal intensive care unit (ICU), the pediatric ICU, and the general pediatric inpatient environments as harm detection tools. Similar to those focused on adult populations, pediatricfocused trigger tools have been found to be more efficient and effective at identifying harms compared with traditional methods.9–14 Trigger tool–derived harm rates in the pediatric population have ranged from 11 drug-related harms per 100 patients in the pediatric acute care environment to 203 all-cause harms per 100 patients in the pediatric ICU.12,14 Our ability to successfully pursue and demonstrate reduction of all-cause harm in pediatric inpatients will require having an efficient and reliable process for harm identification that can be implemented across disparate sites.15 Building on the work of the www.journalpatientsafety.com
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
1
J Patient Saf • Volume 00, Number 00, Month 2014
Stockwell et al
Canadian and European pediatric trigger tools, we sought to develop a pediatric version of the IHI's Global Trigger Tool to reliably measure all-cause harm across pediatric institutions.16,17 Because the Canadian and European pediatric global trigger tools did not focus on electronic detection of triggers, did not include a large number of experts in trigger use and development, and had limited validity testing, further development and validation of a pediatric trigger tool are warranted. Most significantly, to minimize the time and resource burden of manual chart review as well as leverage the rapid transition toward electronic health records, special attention was given to developing triggers that are easily extractable from an electronic medical record (EMR).12–15 Using the EMR in this manner is an efficient application of triggers and is likely to be recommended as a national patient safety goal in the future.18–21
METHODS A modified Delphi method22 was used to develop and identify the pediatric triggers in this study (Fig. 1). Specifically, our modified Delphi approach mirrored that described by Williams and Webb22 as follows: “the technique consists of questioning a panel of experts on specific questions or issues. Information concerning the issue is posted individually to each expert, who then responds to the researcher. This procedure is anonymous and confidential. The individual responses of the panel are scrutinized and collated by the researcher, who next compiles a comprehensive list for resubmission to the panel. At this stage, the experts are asked to reconsider the list and respond by post again, indicating their agreement or disagreement with items. The replies are collated once more and the process repeated until consensus is reached.” A panel of 11 international leaders (DS, DC, EK, PL, AM, ET, DH, SM, ES, SM, PS) in the fields of patient safety; informatics; as well as pediatric trigger tool creation, use, and research served as experts. The Children's Hospital Association (CHA), formerly known as the Child Health Corporation of America, provided facilitation of our modified Delphi method, consistent with previously published U.S. pediatric trigger tools.12–14 The model for this approach to trigger tool development was based on earlier CHA work in trigger development and testing12–14 combined with a recent description of adult outpatient trigger development.23
Step 1 (Development) The panel conducted an exhaustive literature review, synthesis, and reconciliation of all established pediatric trigger tools in active use and included the adult patient–based IHI Global Trigger Tool to create a trigger compendium.9–14,16,17 Each trigger was translated into associated harms (e.g., the trigger of “creatinine doubling” translated to the harm “renal injury,” and hyponatremia was translated to seizures related to electrolyte imbalance). This translation and linkage were performed so that prioritization of potential harm would be more cognitively accessible and are consistent with earlier pediatric trigger development efforts.12–14 All obstetric-related triggers/harms were removed.
Step 2 (Prioritization) Once each trigger was linked to a potential harm, the panelists independently reviewed and scored each trigger and associated harm according to 3 prioritization categories (frequency of the harm, severity of the harm, and electronic detectability of the associated trigger). This prioritization survey was developed as a framework to initiate an iterative Delphi review process of the triggers and corresponding harm as a package. Scoring placed triggers and their harm into 3 separate categories throughout the entire process: a “retain” group, an “ambiguous” group, or a “remove” group. All 3 categories scored used a 1 (low frequency, low severity, or low electronic detectability) to 5 (high frequency, high severity, or high electronic detectability) scale. Surveys were e-mailed to each expert panelist, including instructions on how to apply the criteria and rating scales, for completion during a 2-week review period. Results from this first round of scoring were analyzed and anonymously reported back to the work group by the CHA. Because the emphasis of this work was to generate a compendium of pediatric triggers amenable to an automated/EMR approach and the more traditional manual approach, high emphasis was placed on electronic detectability. Therefore, harms and their associated triggers needed to have a mean score of 3 or higher in each of the 3 categories to be retained after the first independent review as part of their consensus criteria for this round. If detectability of the trigger and associated harm was lower than a mean score of 3, the trigger was placed in the remove group; if equal to or higher than 3, it was then assessed for frequency and severity. If both frequency and severity had scores at or higher than 3, they were placed in the retain group; if both were lower than 3, they were placed in the remove group. If either the frequency or the severity category was at or higher than 3 or the other was not, they were placed in the ambiguous group.
Step 3 (Refinement) A second round of assessment of the triggers and associated harms in the remove, ambiguous, and retain categories was then undertaken. Each of the triggers within each category was assessed independently by each expert to determine appropriateness for permanent removal or inclusion in the next step. This was undertaken to enable the experts to confirm and revise former answers in light of the replies of their peer panelists, with consensus criteria being majority opinion. This repeated process allowed convergence toward a “short list” of triggers through consensus building.
Step 4 (Adding Specification) The list of prioritized triggers and associated harms was then reviewed using the working experience and knowledge of all panelists to provide greater detail on the required logic specific to each trigger. The goal was to enhance the anticipated positive predictive value of each trigger, building on the expert opinion of the convened panel. A series of guiding questions were provided to the panelists for consideration as part of the process of building
FIGURE 1. Pediatric patient all-cause harm measurement tool development multistep approach.
2
www.journalpatientsafety.com
© 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
© 2014 Lippincott Williams & Wilkins
Death
Hospital readmissions/ICU readmissions/ICU admissions
Complication related to CVC
Infiltration/phlebitis/phentolamine or hyaluronidase administration
Order to “pull back or push in ETT” or CXR with right/left mainstem intubation Dissatisfaction with care and/or evidence of complaint lodged
Elevated pain score (equivalent to or greater than a score of 6/10)
Extreme temperature of 41°C
Fall
Hypotension
Hypoxia, O2 saturation of 6 h
Diagnostic imaging for embolus or thrombus confirmation
Cares: iatrogenic thrombus
ED
D-dimer (positive by local laboratory normal values)
Cares: iatrogenic thrombus
Readmission to the ED within 48 h
Health care–associated infections: new positive blood culture, urine culture, or viral testing; SIRS criteria; and Clostridium difficile, VRE, MRSA, etc Anticoagulant use (LMWH, warfarin, or heparin drip)
Cares: HAI
ED
Antibiotic use
Cares: HAI
CXR with pulmonary edema or pleural effusion or pneumothorax Racemic epinephrine administration
Transfusion, use of blood products
Cares: general
Critical care
Seizures
Trigger
Cares: general
Group
TABLE 1. (Continued)
2.7 2.0 (2.0–3.3) 2.8 3.0 (2.0–3.0) 3.7 4.0 (3.0–4.3)
2.6 2.0 (2.0–4.0) 2.2 2.0 (1.8–3.0) 1.9 2.0 (1.8–2.0) 1.2 1.0 (1.0–1.3) 2.9 3.0 (2.0–3.0) 2.5 3.0 (2.0–3.0) 1.9 2.0 (1.8–2.0) 3.2 3.0 (3.0–4.0)
3.1 2.5 (2.0–4.3) 2.8 3.0 (2.0–4.0) 4.3 5.0 (3.5–5.0) 3.5 3.5 (2.8–4.3)
3.2 3.0 (2.0–4.0) 3.1 3.0 (2.0–4.0) 2.4 2.0 (1.8–3.0)
3.7 4.0 (3.0–4.0) 3.5 4.0 (3.0–4.0) 3.9 4.0 (3.8–4.0) 4.8 5.0 (4.8–5.0) 3.0 3.0 (2.8–4.0) 4.3 4.5 (3.8–5.0) 4.6 5.0 (4.0–5.0) 3.7 4.0 (3.0–4.0)
3.1 3.0 (3.0–4.0) 2.9 3.0 (2.0–3.3) 2.9 3.0 (2.0–4.0) 3.6 3.5 (3.0–4.3)
Median (IQR)
Mean
Mean Median (IQR)
Severity (Not at All Serious, 1, to Very Serious, 5)
Frequency (Rarely, 1, to Very Common, 5)
4.6 5.0 (4.0–5.0) 4.4 4.5 (4.0–5.0) 4.2 5.0 (3.5–5.0)
3.6 3.5 (2.8–5.0) 3.7 4.0 (2.0–5.0) 3.3 3.5 (2.0–4.3) 3.0 3.0 (2.0–4.3) 3.0 3.0 (2.0–4.0) 3.1 3.0 (2.0–4.3) 2.4 2.0 (2.0–3.3) 2.9 3.0 (2.0–3.5)
2.2 2.0 (1.8–3.0) 3.7 4.0 (2.8–5.0) 3.2 4.0 (1.5–5.0) 3.7 4.0 (3.0–4.0)
Median (IQR)
Mean
Detectability (Not at All Easy, 1, to Very Easy, 5)
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Ambiguous
Retain
Retain
Remove
Remove
Retain
Retain
Ambiguous
Ambiguous
Ambiguous
Retain
Retain
Retain
Retain
Remove
Categorization as a Result of Delphi Round
3
8
7
3
1
8
4
4
4
3
6
9
3
7
3
Expert Panel Votes on First Categorization: Retain Votes
Removed
Retained
Retained
Removed
Removed
Retained
Removed
Removed
Removed
Removed
Retained
Retained
Removed
Retained
Removed
Categorization Resulting From Expert Panel Votes on First Categorization
(Continued on next page)
6
2
3
7
9
2
6
6
6
6
4
1
7
3
7
Expert Panel Votes on First Categorization: Exclude Votes
Stockwell et al J Patient Saf • Volume 00, Number 00, Month 2014
© 2014 Lippincott Williams & Wilkins
© 2014 Lippincott Williams & Wilkins
2.7 2.5 (2.0–3.0) 1.1 1.0 (1.0–1.0) 1.9 1.5 (1.0–2.3) 2.0 2.0 (1.0–2.3) 2.8 2.0 (2.0–4.0) 2.9 2.5 (2.0–4.0)
Oversedation/lethargy/fall
Terbutaline use
Anti-Xa > 1.5
aPTT > 100
INR elevated, vitamin K (excluding newborns), vitamin K + warfarin Platelet count < 50,000
Acute kidney injury/dialysis/ creatinine elevation
ALT rising and taken isoniazid, phenytoin, cyclosporine, methotrexate, or warfarin during the past 7 d
Medications: general
3.5 3.5 (2.0–5.0) 2.7 2.0 (2.0–4.0) 2.0 2.0 (1.0–3.0)
2.8 3.0 (2.0–3.0)
Naloxone
Abrupt medication stop
1.7 2.0 (1.0–2.0)
N-acetylcysteine
1.5 1.5 (1.0–2.0)
3.3 3.0 (2.8–4.3)
2.0 2.0 (1.0–3.0)
Methadone administration
Protamine
3.9 4.0 (3.0–4.3)
1.8 2.0 (1.0–2.0)
Flumazenil (Romazicon)
2.9 3.0 (2.0–4.0) 3.5 4.0 (2.8–4.0) 3.3 3.0 (3.0–4.0)
3.3 3.5 (3.0–4.0 )
3.6 4.0 (2.75-4.0)
3.6 4.0 (3.0–4.0)
3.3 3.0 (3.0–4.0)
3.5 3.5 (3.0–4.0)
4.2 4.0 (3.8–5.0)
3.7 4.0 (2.8–4.3)
3.1 3.0 (2.0–4.0)
4.1 4.0 (3.0–5.0)
4.6 5.0 (4.0–5.0)
1.0 1.0 (1.0–1.0)
Digibind administration
3.2 3.0 (3.0–4.0)
1.5 1.0 (1.0–2.0)
Benztropine administration
Medications: general
High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: antithrombotics High-risk medications: antithrombotics High-risk medications: antithrombotics High-risk medications: antithrombotics High-risk medications: antithrombotics Medications: general 2.9 3.5 (1.0–4.25) 4.3 4.5 (4.0–5.0) 4.0 4.0 (3.8–5.0)
4.6 5.0 (4.0–5.0)
4.4 5.0 (3.8–5.0)
4.6 5.0 (4.0–5.0)
4.6 5.0 (4.0–5.0)
4.4 5.0 (3.8–5.0)
4.1 5.0 (3.5–5.0)
2.1 2.0 (1.8–2.3)
4.7 5.0 (4.0–5.0)
4.3 5.0 (3.8–5.0)
4.3 5.0 (3.8–5.0)
4.7 5.0 (4.0–5.0)
4.6 5.0 (4.0–5.0)
4.3 5.0 (3.0–5.0)
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Ambiguous
Retain
Remove
Ambiguous
Retain
Retain
Ambiguous
Ambiguous
Ambiguous
Remove
Retain
Ambiguous
Ambiguous
Ambiguous
Ambiguous
Ambiguous
3
10
4
8
7
9
7
6
1
1
10
3
2
8
6
3
www.journalpatientsafety.com
Removed
Retained
Removed
Retained
Retained
Retained
Retained
Retained
Removed
Removed
Retained
Removed
Removed
Retained
Retained
Removed
(Continued on next page)
6
0
5
1
3
1
2
3
8
9
0
5
7
1
3
6
J Patient Saf • Volume 00, Number 00, Month 2014 Pediatric All-Cause Harm Measurement Tool
5
6
www.journalpatientsafety.com
Drug level out of range
Gentamicin/tobramycin: (except CF patients) trough > 2 mg/l or peak > 10 mg/l Hypercalcemia
Medications: general
Medications: general
Hyperkalemia/sodium polystyrene administration
Hypernatremia
Hypoglycemia/glucagon administration/dextrose bolus + insulin Hypokalemia
Medications: general
Medications: general
Medications: general
Hyponatremia
Laxative or stool softeners
New allergy added after 24 h
Phenobarbital
Rash
STAT orders
Medications: general
Medications: general
Medications: general
Medications: general
Medications: general
Medications: general
Medications: general
Hyperglycemia
Medications: general
Medications: general
Antihistamines
Medications: general
Trigger
Antiemetic administration
Medications: general
Group
TABLE 1. (Continued)
3.0 3.0 (2.0–4.0) 2.9 3.0 (2.0–4.0) 3.9 3.5 (3.0–5.0) 3.0 2.5 (2.0–4.3) 1.8 2.0 (1.0–2.0) 4.0 4.0 (3.0–5.0) 3.6 3.5 (2.8–5.0)
2.6 3.0 (2.0–3.0) 3.6 4.0 (2.6–4.3) 2.5 2.0 (2.0–3.3) 2.3 2.0 (1.3-3.0) 3.0 3.0 (2.0–4.0)
4.1 4.5 (3.0–5.0) 4.0 4.0 (4.0–4.5) 3.7 3.0 (3.0–5.0) 2.7 2.0 (2.0–3.3)
2.6 2.5 (2.0–3.3) 3.4 3.0 (3.0–4.0) 2.0 2.0 (1.8–2.3) 2.7 3.0 (2.0–3.0) 3.1 3.0 (2.8–4.0) 1.8 2.0 (1.0–2.0) 2.5 3.0 (1.8–3.0)
3.0 3.0 (2.8–4.0) 2.8 3.0 (2.0–3.3) 3.5 4.0 (2.8–4.0) 3.1 3.0 (2.0–4.0) 3.8 4.0 (3.0–4.3)
2.1 2.0 (1.8–2.3) 2.1 2.0 (2.0–2.5) 2.8 3.0 (2.0–3.0) 3.0 3.0 (2.0–4.0)
Median (IQR)
Mean
Mean Median (IQR)
Severity (Not at All Serious, 1, to Very Serious, 5)
Frequency (Rarely, 1, to Very Common, 5)
4.5 5.0 (4.0–5.0) 4.5 5.0 (4.0–5.0) 4.3 5.0 (4.0–5.0) 4.0 4.0 (3.0–5.0) 4.0 5.0 (2.0–5.0) 2.8 3.0 (2.0–4.0) 3.6 4.0 (2.0–5.0)
4.3 5.0 (3.8–5.0) 4.2 4.5 (3.8–5.0) 4.5 5.0 (4.0–5.0) 4.4 5.0 (4.0–5.0) 4.5 5.0 (4.0–5.0)
4.0 4.5 (3.0–5.0) 4.3 5.0 (3.5–5.0) 4.0 4.0 (3.8–5.0) 4.7 5.0 (4.0–5.0)
Median (IQR)
Mean
Detectability (Not at All Easy, 1, to Very Easy, 5)
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Retain
Remove
Ambiguous
Retain
Ambiguous
Retain
Retain
Retain
Ambiguous
Retain
Retain
Retain
Retain
Retain
Ambiguous
Ambiguous
Categorization as a Result of Delphi Round
6
3
0
6
5
8
6
10
6
8
6
5
9
6
3
1
Expert Panel Votes on First Categorization: Retain Votes
Retained
Removed
Removed
Retained
Retained
Retained
Retained
Retained
Retained
Retained
Retained
Retained
Retained
Retained
Removed
Removed
Categorization Resulting From Expert Panel Votes on First Categorization
(Continued on next page)
4
7
9
3
5
1
4
0
2
2
3
5
1
4
6
8
Expert Panel Votes on First Categorization: Exclude Votes
Stockwell et al J Patient Saf • Volume 00, Number 00, Month 2014
© 2014 Lippincott Williams & Wilkins
NEC
Specialty consult
Abrupt drop in Hgb or Hct (>25%)
Administration of general anesthesia
Admission to intensive care postoperatively
Arterial or central line insertion during surgery (unanticipated)
Cancelled elective procedure, delayed discharge
Change in planned procedure
Complication of procedure or treatment
Diagnosis change preoperatively/postoperatively
Injury, repair, or removal of organ during operative procedure
Intraoperative IV epinephrine, norepinephrine
Intubation or reintubation or use of BiPAP in PACU
Mechanical ventilation > 24 h postoperatively
Operative time > 6 h (in noncardiac patient)
Pathology report normal or unrelated to diagnosis
Postoperative increase in troponin levels > 1.5 ng/ml
Unplanned return to surgery
NICU
NICU
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
NICU
Abnormal cranial imaging/IVH grade 3 or 4 head bleed in CICU or NICU Bilirubin (total) > 25
NICU 1.3 1.0 (1.0–2.0) 2.4 2.0 (2.0–3.0) 4.3 4.5 (4.0–5.0) 2.5 2.0 (1.8–3.3) 3.6 4.0 (2.0–5.0) 2.3 2.0 (1.8–3.0) 2.1 2.0 (2.0–2.3) 3.3 3.0 (2.0–4.3) 2.0 2.0 (2.0–2.0) 2.5 2.0 (2.0–3.0) 2.5 2.5 (1.8–3.3) 1.4 1.0 (1.0–2.0) 1.7 2.0 (1.0–2.0) 2.0 2.0 (2.0–2.0) 2.6 2.5 (2.0–3.0) 2.2 2.0 (2.0–3.0) 2.1 2.0 (1.0–3.0) 1.0 1.0 (1.0–1.0) 2.3 2.0 (1.8–3.0)
2.4 2.5 (1.8–3.0) 4.2 4.0 (3.8–5.0) 4.3 4.0 (4.0–5.0) 2.2 2.5 (1.0–3.0) 3.8 4.0 (2.8–5.0) 2.7 3.0 (1.0–3.5) 3.5 4.0 (2.8–4.0) 3.3 3.0 (3.0–4.0) 2.5 2.0 (2.0–3.3) 3.0 2.5 (2.0–4.3) 3.6 3.5 (3.0–4.0) 3.1 3.0 (2.0–4.3) 4.5 4.5 (4.0–5.0) 4.0 4.0 (3.0–5.0) 3.6 4.0 (3.0–4.0) 3.3 3.0 (3.0–4.0) 3.5 3.5 (3.0–4.0) 3.2 3.0 (3.0–3.5) 3.5 4.0 (2.0–5.0) 4.3 4.0 (4.0–5.0)
4.7 5.0 (4.0–5.0) 4.5 5.0 (4.8–5.0) 2.7 3.0 (1.8–3.3) 3.4 3.5 (2.5–5.0) 4.0 4.0 (3.8–5.0) 3.0 3.0 (1.0–5.0) 3.8 4.0 (2.8–5.0) 2.5 2.5 (1.0–4.0) 2.6 2.5 (1.8–3.3) 2.7 3.0 (1.0–4.0) 2.1 2.0 (1.0–3.0) 2.8 2.5 (2.0–4.0) 2.7 2.5 (1.8–3.5) 3.9 4.5 (2.8–5.0) 2.9 2.5 (2.0–4.0) 3.1 3.0 (2.0–4.0) 3.6 4.0 (2.8–4.3) 2.5 2.0 (1.8–4.0) 3.8 4.5 (2.0–5.0) 2.6 2.0 (1.8–3.5)
3.4 3.0 (2.0–5.0)
© 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Remove
Ambiguous
Remove
Ambiguous
Retain
Remove
Ambiguous
Remove
Remove
Remove
Remove
Remove
Remove
Ambiguous
Retain
Retain
Ambiguous
Remove
Ambiguous
Ambiguous
9
4
3
6
8
8
6
8
5
8
7
3
6
2
3
8
1
7
7
7
Manual retained
Removed
Removed
Retained
Retained
Manual retained
Retained
Manual retained
Manual retained
Manual retained
Manual retained
Removed
Manual retained
Removed
Removed
Retained
Removed
Manual retained
Retained
Retained
(Continued on next page)
1
6
7
4
2
2
4
2
5
2
3
7
4
7
7
2
9
2
2
2 J Patient Saf • Volume 00, Number 00, Month 2014 Pediatric All-Cause Harm Measurement Tool
www.journalpatientsafety.com
7
J Patient Saf • Volume 00, Number 00, Month 2014
8
X-ray intraoperatively or in PACU Perioperative
www.journalpatientsafety.com
ADT, admission, diagnosis, and transfer; ALI, acute lung injury; ALT, alanine transminase; aPTT, activated partial thromboplastin time; ARDS, adult respiratory distress syndrome; BiPAP, bilevel positive airway pressure; CF, cystic fibrosis; CICU, cardiac intensive care unit; CVC, central venous catheter; CXR, chest x-ray; D-dimer, dimerized plasmin fragment D; DVT, deep venous thrombosis; ED, emergency department; ETT, endotracheal tube; HAI, healthcare associated infection; Hct, hematocrit; Hgb, hemoglobin; INR, international normalized ratio; IQR, interquartile range; IV, intravenous; IVH, intraventricular hemorrhage; LMWH, low-molecular-weight heparin; MRSA, methicillin-resistant Staphylococcus aureus; NEC, necrotizing enterocolitis; PACU, postanesthesia care unit; SIRS, systemic inflammatory response syndrome; VRE, vancomycin-resistant enterococci.
Retained 5 Ambiguous
5
Manual retained 0 10 Remove
2.4 2.0 (1.0–4.3) 3.5 3.5 (2.0–5.0) 4.7 5.0 (4.8–5.0) 2.7 3.0 (2.0–3.3) 1.5 1.0 (1.0–1.3) 2.2 2.0 (2.0–3.0) Wrong site, wrong procedure, wrong patient Perioperative
Median (IQR) Median (IQR) Median (IQR) Group
TABLE 1. (Continued)
Trigger
Mean Mean
Mean
Severity (Not at All Serious, 1, to Very Serious, 5) Frequency (Rarely, 1, to Very Common, 5)
Detectability (Not at All Easy, 1, to Very Easy, 5)
Categorization as a Result of Delphi Round
Categorization Resulting From Expert Panel Votes on First Categorization Expert Panel Votes on First Categorization: Exclude Votes Expert Panel Votes on First Categorization: Retain Votes
Stockwell et al
electronic specificity for the qualifying triggers. For example, if there was a laboratory value–based trigger, the experts were asked to develop the value that would be the set point for the trigger. After each panel member had the opportunity to view the candidate trigger definitions and developed logic, proposals for each were vetted on conference calls, and their appropriateness for inclusion was determined by vote. Expert feedback from this exercise yielded design descriptors and thresholds for further testing in a paper-based and EMR environment.
Step 5 (Finalization) Proposals for each trigger were shared and discussed with all panelists, and consensus on the final pediatric global trigger tool for pilot testing was reached via voting for retention or removal by majority vote.
RESULTS Eleven content experts participated in each step of the developmental process described above. The literature review and trigger collection (step 1) yielded 108 triggers. Step 2 (prioritization) resulted in 21 harms and associated triggers being placed in the remove group (scored 3 in detectability but 25 mg/dL Hyperglycemia (14 mmol/l or 250 mg/dL) Hypernatremia (>160 mEq/l) Hypoglycemia (2 mmol/l or 40 mg/dl) Hyponatremia ( 100 s Anti-Xa > 1.5 Warfarin: INR > 6 Warfarin: vitamin K administration after warfarin administration Protamine administration Naloxone administration Flumazenil administration Ongoing or intermittent laxative use Digibind administration Hyperkalemia (K+ > 6.0 mEq/l) and sodium polystyrene administration Elevated drug levels (antiepileptics): phenytoin (>30 μg/ml) Elevated drug levels (antiepileptics): valproic acid (>170 μg/ml) Elevated drug levels (antiepileptics): carbamazepine (>20 μg/ml) Elevated drug levels (antiepileptics): oxcarbamazepine (>45 μg/ml) Elevated drug levels (antiepileptics): phenobarbital (>30 μg/ml) Elevated drug levels (antibiotics): vancomycin (trough > 25 μg/ml) Elevated drug levels (antibiotics): gentamicin (trough > 4 μg/ml) Elevated drug levels (antibiotics): tobramycin (trough > 4 μg/ml) Elevated drug levels (antibiotics): amikacin (trough > 20 μg/ml) Racemic epinephrine administration (within 24 h after endotracheal extubation)
© 2014 Lippincott Williams & Wilkins
51 triggers that constitute the proposed Pediatric All-Cause Harm Measurement Tool (Tables 2–7).
DISCUSSION By integrating the findings of multiple prior pediatric-specific trigger tool studies, the experience and expertise of 11 pediatric trigger tool experts, the strong project management infrastructure of the CHA, as well as the strengths of a modified Delphi method, we believe that our final tool is the most sophisticated pediatric all-cause harm measurement tool ever constructed. Our decision to use this modified Delphi approach resulted in a final tool that harnessed the collective intelligence of the 11 thought leaders and the findings in the literature to date. Although 2 other pediatric global trigger tools exist, neither was constructed with the benefit of existing literature or such a cadre of trigger experts and neither integrated the practical requirement of detectability in an EMR environment. Our tool, once tested, should emerge as the state of the science in pediatric all-cause harm measurement. There are several limitations to this study. First, there is a distinct lack of published investigations for specific triggers, packages of triggers, and utility of those packages in the pediatric setting. A recent publication evaluated the use of the IHI Adult Global Trigger Tool in a pediatric hospital24 and revealed that many of the adult-focused triggers were not relevant in the pediatric environment. Second, expert opinion was used extensively to construct the tool. Opinions, however expert they are, are subject to biases that can nevertheless impact trigger/harm selection and final tool creation in uncertain ways. These biases were minimized by our use of a modified Delphi method. Third, the final tool was constructed as a subset from an original collection of triggers identified in the literature or via experience of the authors. Hence, any unpublished triggers or those used outside the authorship experience were, by definition, excluded from the final tool. Finally, we minimized inclusion of multielement triggers (for example, TABLE 5. Perioperative Triggers P1 P2 P3 P4 P5 P6
Abrupt drop of >25% in hemoglobin or hematocrit Unanticipated insertion of arterial or central venous line during surgery Intraoperative epinephrine, norepinephrine, or phenylephrine (noncardiac patients) Mechanical ventilation > 48 h postoperatively Operative time > 6 h (noncardiac patients) X-ray intraoperatively or in postanesthesia care unit
www.journalpatientsafety.com
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
9
J Patient Saf • Volume 00, Number 00, Month 2014
Stockwell et al
TABLE 6. Readmission Triggers
7. Resar RK, Rozich JD, Classen DC. Methodology and rationale for the measurement of harm with trigger tools. Qual Saf Health Care. 2003;12(suppl 2):ii39–ii45.
R1 R2 R3
8. Classen DC, Resar R, Griffin F, et al. ‘Global trigger tool’ shows that harms in hospitals may be ten times greater than previously measured. Health Aff. 2011;30:581–589.
Readmission to ICU within 24 h from discharge/transfer Hospital readmission within 30 d Readmission to the emergency department within 48 h
TABLE 7. Resuscitation/Death Triggers RES1 RES2 RES3
Any code or arrest or rapid response team activation Transfer to higher level of care All inpatient deaths
creatinine doubling in 24 h and urine output decreasing by 50% in 24 h), purposefully sacrificing potential increased specificity in favor of the efficiency of single-element triggers. Future triggers are likely to include additional complexity as the ability to query for multielement triggers becomes increasingly possible in more sophisticated electronic environments. This study identified 51 pediatric triggers for testing in a pediatric inpatient environment. Pilot testing of this initial list is the next phase of the work. This pilot test phase will identify positive predictive values for each trigger, suggest more appropriate set points, and capture harm prevalence before the third phase of broad-scale validation testing as well as allow for some comparisons with recent similar research with the Canadian Pediatric Trigger Tool.25 The broad scale validation testing phase will establish positive predictive values of each trigger, establish the sensitivity and specificity of the entire tool, and will most importantly describe the epidemiology of all-cause harm in pediatric inpatients. Finally, the pediatric all-cause harm tool will be embedded and tested in the electronic environment, which should further enhance the sensitivity of the findings, expand the populations evaluated, and improve the efficiency of the local review process. With a pediatric all-cause harm measurement tool constructed to maximize the opportunities provided by the EMR, the goal of real-time harm detection, and the ability to mitigate harms as a result, is moving rapidly toward a reality. REFERENCES 1. Landrigan CP, Parry GJ, Bones CB, et al. Temporal trends in rates of patient harm resulting from medical care. N Engl J Med. 2010;363:2124–2134. 2. Cullen DJ, Bates DW, Small SD, et al. The incident reporting system does not detect adverse drug events: a problem for quality improvement. Jt Comm J Qual Improv. 1995;21:541–548. 3. Stockwell DC, Slonim AD. Quality and safety in critical care units. Intensive Care Med. 2006;21:199–210. 4. Office of the Inspector General. Harms in Hospitals: Methods for Identifying Events. Washington, DC: Department of Health and Human Services; 2010. OEI-06-08-00221. Available at: http://www.oig.hhs.gov/oei/reports/ oei-06-08-00221.pdf. Accessed November 25, 2012.
9. Dickerman MJ, Jacobs BR, Vinodrao H, et al. Recognizing hypoglycemia in children through automated harm detection. Pediatrics. 2011;127: e1035–e1041. 10. Ferranti J, Horvath MM, Cozart H, et al. Reevaluating the safety profile of pediatrics: a comparison of computerized adverse drug event surveillance and voluntary reporting in the pediatric environment. Pediatrics. 2008;121:e1201–e1207. 11. Kilbridge PM, Noirot LA, Reichley RM, et al. Computerized surveillance for adverse drug events in a pediatric hospital. J Am Med Inform Assoc. 2009;16:607–612. 12. Takata GS, Mason W, Taketomo C, et al. Development, testing, and findings of a pediatric-focused trigger tool to identify medication-related harm in US children's hospitals. Pediatrics. 2008;121: e927–e935. 13. Sharek PJ, Horbar JD, Mason W, et al. Harms in the neonatal intensive care unit: development, testing, and findings of a NICU-focused trigger tool to identify harm in North American NICUs. Pediatrics. 2006;118: 1332–1340. 14. Agarwal S, Classen DC, Larsen GL, et al. Incidence of harms in pediatric intensive care units in the United States. Pediatr Crit Care Med. 2010;11: 568–578. 15. Sharek PJ, Parry G, Goldmann DA, et al. Performance characteristics of a methodology to quantify adverse events over time in hospitalized patients. Health Serv Res. 2011;46:654–678. 16. Matlow AG, Cronin CM, Flintoft V, et al. Description of the development and validation of the Canadian Paediatric Trigger Tool. BMJ Qual Saf. 2011;20:416–423. 17. Franklin BD, Birch S, Schachter M, Barber N. Testing a trigger tool as a method of detecting harm from medication errors in a UK hospital: a pilot study. Int J Pharm Pract. 2010;18:305–311. 18. Classen DC, Pestotnik SL, Evans RS, et al. Computerized surveillance of adverse drug events in hospital patients. JAMA. 1991;266:2847–2851. 19. Jha AK, Kuperman GJ, Teich JM, et al. Identifying harms: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc. 1998;5:305–314. 20. Bates DW, Evans RS, Murff H, et al. Detecting harms using information technology. J Am Med Inform Assoc. 2003;10:115–128. 21. Sittig DF, Singh H. Electronic health records and national patient-safety goals. N Engl J Med. 2012;367:1854–1860. 22. Williams PL, Webb C. The Delphi technique: a methodological discussion. J Adv Nurs. 1994;19:180–186. 23. Mull HJ, Nebeker JR, Shimada SL, et al. Consensus building for development of outpatient adverse drug event triggers. J Patient Saf. 2011; 7:66–71.
5. Griffin FA, Resar RK. IHI Global Trigger Tool for Measuring Harms (Second Edition). IHI Innovation Series White Paper. Cambridge, MA: Institute for Healthcare Improvement; 2009.
24. Kirkendall E, Kloppenborg E, Papp J, et al. Measuring harms and levels of harm in pediatric inpatients with the Global Trigger Tool. Pediatrics. 2012; 130:e1206–e1214.
6. Rozich JD, Haraden CR, Resar RK. Adverse drug event trigger tool: a practical methodology for measuring medication related harm. Qual Saf Health Care. 2003;12:194–200.
25. Matlow AG, Baker GR, Flintoft V, et al. Adverse events among children in Canadian hospitals: the Canadian Paediatric Adverse Events Study. CMAJ. 2012;184:E709–E718.
10
www.journalpatientsafety.com
© 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Development of an Electronic Pediatric All-Cause Harm Measurement Tool Using a Modified Delphi Method David Christopher Stockwell, MD, MBA,*† Hema Bisarya, MHSA, RD,‡ David C. Classen, MD,§k Eric S. Kirkendall, MD,¶# Peter I. Lachman, MBBS,** Anne G. Matlow, MD,†† Eric Tham, MD,‡‡§§ Dan Hyman, MD,‡‡kk Samuel M. Lehman, MD,¶¶ Elizabeth Searles, RN,## Stephen E. Muething, MD,# and Paul J. Sharek, MD***††† Objectives: To have impact on reducing harm in pediatric inpatients, an efficient and reliable process for harm detection is needed. This work describes the first step toward the development of a pediatric all-cause harm measurement tool by recognized experts in the field. Methods: An international group of leaders in pediatric patient safety and informatics were charged with developing a comprehensive pediatric inpatient all-cause harm measurement tool using a modified Delphi technique. The process was conducted in 5 distinct steps: (1) literature review of triggers (elements from a medical record that assist in identifying patient harm) for inclusion; (2) translation of triggers to likely associated harm, improving the ability for expert prioritization; (3) 2 applications of a modified Delphi selection approach with consensus criteria using severity and frequency of harm as well as detectability of the associated trigger as criteria to rate each trigger and associated harm; (4) developing specific trigger logic and relevant values when applicable; and (5) final vetting of the entire trigger list for pilot testing. Results: Literature and expert panel review identified 108 triggers and associated harms suitable for consideration (steps 1 and 2). This list was pared to 64 triggers and their associated harms after the first of the 2 independent expert reviews. The second independent expert review led to further refinement of the trigger package, resulting in 46 items for inclusion (step 3). Adding in specific trigger logic expanded the list. Final review and voting resulted in a list of 51 triggers (steps 4 and 5). Conclusions: Application of a modified Delphi method on an expertconstructed list of 108 triggers, focusing on severity and frequency of harms as well as detectability of triggers in an electronic medical record, resulted in a final list of 51 pediatric triggers. Pilot testing this list of pediatric triggers to identify all-cause harm for pediatric inpatients is the next step to establish the appropriateness of each trigger for inclusion in a global pediatric safety measurement tool. From the *Division of Critical Care Medicine, Department of Pediatrics, School of Medicine, The George Washington University, Washington, DC; †Center for Quality and Improvement Science, Children's National Medical Center, Washington, DC; ‡Children's Hospital Association, Overland Park, KS; §Department of Infectious Disease, School of Medicine, University of Utah, Salt Lake City, UT; kPascal Metrics, Washington, DC; ¶Division of Biomedical Informatics, Division of Hospital Medicine, Department of Pediatrics, University of Cincinnati, Cincinnati, OH; #James M. Anderson Center for Health Systems Excellence, Cincinnati Children's Hospital Medical Center, Cincinnati, OH; **Medical Directors Office, Quality and Safety, Great Ormond Street Hospital NHS Foundation Trust, London, England; ††Departments of Paediatrics and Medicine and Centre for Patient Safety, University of Toronto, Toronto, ON; ‡‡Department of Pediatrics, School of Medicine, University of Colorado, Aurora, CO; §§Research Institute, and kkDepartment of Quality and Patient Safety, Children's Hospital Colorado, Aurora, CO; ¶¶Department of Anesthesia and Critical Care Medicine, and ##Department of Quality, Children's Hospital Central California, Madera, CA; ***Division of General Pediatrics, Department of Pediatrics, School of Medicine, Stanford University palo Alto, CA; and †††Center for Quality and Clinical Effectiveness, Lucile Packard Children's Hospital, Palo Alto, CA. Correspondence: David Christopher Stockwell, MD, MBA, Center for Quality and Improvement Science, Children's National Medical Center, 111 Michigan Ave, NW, Suite M-4800, Washington, DC 20010 (e‐mail: [email protected]). The authors disclose no conflict of interest. Supported by the Children's Hospital Association. Copyright © 2014 by Lippincott Williams & Wilkins
J Patient Saf • Volume 00, Number 00, Month 2014
Key Words: harms, patient safety, harm, trigger, Delphi method, adverse event (J Patient Saf 2014;00: 00–00)
A
ccording to a recent study of adult inpatients in North Carolina, rates of medically induced harm of hospitalized adults have not improved in the last 10 years since the release of the Institute of Medicine's To Err Is Human report in 1999.1 This study provided multiple significant conclusions including the following: (1) despite major efforts to improve patient safety, harm rates have not decreased over time, and (2) previously used harm detection techniques have not produced consistent approaches to measure harm. A robust and reliable method of harm detection is needed to have consistent measurement across hospitals, as well as to provide reliable evidence of change in harm rates over time. The mainstay of harm detection in most hospitals is voluntary incident reporting. Unfortunately, voluntary reports have been shown to capture only 2% to 8% of all harms that occur.2 Several other detection methods have been implemented with various successes and limitations.3 Recently, the U.S. Office of the Inspector General commissioned work to compare voluntary reporting and the Institute for Healthcare Improvement (IHI) Adult Global Trigger Tool.4,5 A trigger is defined as an “occurrence, prompt, or flag found on manual review of the medical chart that ‘triggers’ further investigation to determine the presence or absence of an adverse event.”6,7 The findings suggested that the trigger tool approach is significantly more sensitive than voluntary reporting for identifying harm.4 Another study of the IHI Global Trigger Tool revealed 10-fold higher harm rates compared with either the administrative coding-based Agency for Healthcare Research and Quality's (AHRQ) Patient Safety Indicators or hospital voluntary reporting.8 Evidence consistently reveals the trigger tool methodology to be the most sensitive “all-cause harm” detection method presently available. Numerous pediatric investigators have attempted to augment voluntary reporting by using triggers similar to those in the adult-focused IHI Global Trigger Tool used in the North Carolina Patient Safety Study, the U.S. Office of the Inspector General report, and others.9–14 Pediatric-focused trigger tools have been created for the neonatal intensive care unit (ICU), the pediatric ICU, and the general pediatric inpatient environments as harm detection tools. Similar to those focused on adult populations, pediatricfocused trigger tools have been found to be more efficient and effective at identifying harms compared with traditional methods.9–14 Trigger tool–derived harm rates in the pediatric population have ranged from 11 drug-related harms per 100 patients in the pediatric acute care environment to 203 all-cause harms per 100 patients in the pediatric ICU.12,14 Our ability to successfully pursue and demonstrate reduction of all-cause harm in pediatric inpatients will require having an efficient and reliable process for harm identification that can be implemented across disparate sites.15 Building on the work of the www.journalpatientsafety.com
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
1
J Patient Saf • Volume 00, Number 00, Month 2014
Stockwell et al
Canadian and European pediatric trigger tools, we sought to develop a pediatric version of the IHI's Global Trigger Tool to reliably measure all-cause harm across pediatric institutions.16,17 Because the Canadian and European pediatric global trigger tools did not focus on electronic detection of triggers, did not include a large number of experts in trigger use and development, and had limited validity testing, further development and validation of a pediatric trigger tool are warranted. Most significantly, to minimize the time and resource burden of manual chart review as well as leverage the rapid transition toward electronic health records, special attention was given to developing triggers that are easily extractable from an electronic medical record (EMR).12–15 Using the EMR in this manner is an efficient application of triggers and is likely to be recommended as a national patient safety goal in the future.18–21
METHODS A modified Delphi method22 was used to develop and identify the pediatric triggers in this study (Fig. 1). Specifically, our modified Delphi approach mirrored that described by Williams and Webb22 as follows: “the technique consists of questioning a panel of experts on specific questions or issues. Information concerning the issue is posted individually to each expert, who then responds to the researcher. This procedure is anonymous and confidential. The individual responses of the panel are scrutinized and collated by the researcher, who next compiles a comprehensive list for resubmission to the panel. At this stage, the experts are asked to reconsider the list and respond by post again, indicating their agreement or disagreement with items. The replies are collated once more and the process repeated until consensus is reached.” A panel of 11 international leaders (DS, DC, EK, PL, AM, ET, DH, SM, ES, SM, PS) in the fields of patient safety; informatics; as well as pediatric trigger tool creation, use, and research served as experts. The Children's Hospital Association (CHA), formerly known as the Child Health Corporation of America, provided facilitation of our modified Delphi method, consistent with previously published U.S. pediatric trigger tools.12–14 The model for this approach to trigger tool development was based on earlier CHA work in trigger development and testing12–14 combined with a recent description of adult outpatient trigger development.23
Step 1 (Development) The panel conducted an exhaustive literature review, synthesis, and reconciliation of all established pediatric trigger tools in active use and included the adult patient–based IHI Global Trigger Tool to create a trigger compendium.9–14,16,17 Each trigger was translated into associated harms (e.g., the trigger of “creatinine doubling” translated to the harm “renal injury,” and hyponatremia was translated to seizures related to electrolyte imbalance). This translation and linkage were performed so that prioritization of potential harm would be more cognitively accessible and are consistent with earlier pediatric trigger development efforts.12–14 All obstetric-related triggers/harms were removed.
Step 2 (Prioritization) Once each trigger was linked to a potential harm, the panelists independently reviewed and scored each trigger and associated harm according to 3 prioritization categories (frequency of the harm, severity of the harm, and electronic detectability of the associated trigger). This prioritization survey was developed as a framework to initiate an iterative Delphi review process of the triggers and corresponding harm as a package. Scoring placed triggers and their harm into 3 separate categories throughout the entire process: a “retain” group, an “ambiguous” group, or a “remove” group. All 3 categories scored used a 1 (low frequency, low severity, or low electronic detectability) to 5 (high frequency, high severity, or high electronic detectability) scale. Surveys were e-mailed to each expert panelist, including instructions on how to apply the criteria and rating scales, for completion during a 2-week review period. Results from this first round of scoring were analyzed and anonymously reported back to the work group by the CHA. Because the emphasis of this work was to generate a compendium of pediatric triggers amenable to an automated/EMR approach and the more traditional manual approach, high emphasis was placed on electronic detectability. Therefore, harms and their associated triggers needed to have a mean score of 3 or higher in each of the 3 categories to be retained after the first independent review as part of their consensus criteria for this round. If detectability of the trigger and associated harm was lower than a mean score of 3, the trigger was placed in the remove group; if equal to or higher than 3, it was then assessed for frequency and severity. If both frequency and severity had scores at or higher than 3, they were placed in the retain group; if both were lower than 3, they were placed in the remove group. If either the frequency or the severity category was at or higher than 3 or the other was not, they were placed in the ambiguous group.
Step 3 (Refinement) A second round of assessment of the triggers and associated harms in the remove, ambiguous, and retain categories was then undertaken. Each of the triggers within each category was assessed independently by each expert to determine appropriateness for permanent removal or inclusion in the next step. This was undertaken to enable the experts to confirm and revise former answers in light of the replies of their peer panelists, with consensus criteria being majority opinion. This repeated process allowed convergence toward a “short list” of triggers through consensus building.
Step 4 (Adding Specification) The list of prioritized triggers and associated harms was then reviewed using the working experience and knowledge of all panelists to provide greater detail on the required logic specific to each trigger. The goal was to enhance the anticipated positive predictive value of each trigger, building on the expert opinion of the convened panel. A series of guiding questions were provided to the panelists for consideration as part of the process of building
FIGURE 1. Pediatric patient all-cause harm measurement tool development multistep approach.
2
www.journalpatientsafety.com
© 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
© 2014 Lippincott Williams & Wilkins
Death
Hospital readmissions/ICU readmissions/ICU admissions
Complication related to CVC
Infiltration/phlebitis/phentolamine or hyaluronidase administration
Order to “pull back or push in ETT” or CXR with right/left mainstem intubation Dissatisfaction with care and/or evidence of complaint lodged
Elevated pain score (equivalent to or greater than a score of 6/10)
Extreme temperature of 41°C
Fall
Hypotension
Hypoxia, O2 saturation of 6 h
Diagnostic imaging for embolus or thrombus confirmation
Cares: iatrogenic thrombus
ED
D-dimer (positive by local laboratory normal values)
Cares: iatrogenic thrombus
Readmission to the ED within 48 h
Health care–associated infections: new positive blood culture, urine culture, or viral testing; SIRS criteria; and Clostridium difficile, VRE, MRSA, etc Anticoagulant use (LMWH, warfarin, or heparin drip)
Cares: HAI
ED
Antibiotic use
Cares: HAI
CXR with pulmonary edema or pleural effusion or pneumothorax Racemic epinephrine administration
Transfusion, use of blood products
Cares: general
Critical care
Seizures
Trigger
Cares: general
Group
TABLE 1. (Continued)
2.7 2.0 (2.0–3.3) 2.8 3.0 (2.0–3.0) 3.7 4.0 (3.0–4.3)
2.6 2.0 (2.0–4.0) 2.2 2.0 (1.8–3.0) 1.9 2.0 (1.8–2.0) 1.2 1.0 (1.0–1.3) 2.9 3.0 (2.0–3.0) 2.5 3.0 (2.0–3.0) 1.9 2.0 (1.8–2.0) 3.2 3.0 (3.0–4.0)
3.1 2.5 (2.0–4.3) 2.8 3.0 (2.0–4.0) 4.3 5.0 (3.5–5.0) 3.5 3.5 (2.8–4.3)
3.2 3.0 (2.0–4.0) 3.1 3.0 (2.0–4.0) 2.4 2.0 (1.8–3.0)
3.7 4.0 (3.0–4.0) 3.5 4.0 (3.0–4.0) 3.9 4.0 (3.8–4.0) 4.8 5.0 (4.8–5.0) 3.0 3.0 (2.8–4.0) 4.3 4.5 (3.8–5.0) 4.6 5.0 (4.0–5.0) 3.7 4.0 (3.0–4.0)
3.1 3.0 (3.0–4.0) 2.9 3.0 (2.0–3.3) 2.9 3.0 (2.0–4.0) 3.6 3.5 (3.0–4.3)
Median (IQR)
Mean
Mean Median (IQR)
Severity (Not at All Serious, 1, to Very Serious, 5)
Frequency (Rarely, 1, to Very Common, 5)
4.6 5.0 (4.0–5.0) 4.4 4.5 (4.0–5.0) 4.2 5.0 (3.5–5.0)
3.6 3.5 (2.8–5.0) 3.7 4.0 (2.0–5.0) 3.3 3.5 (2.0–4.3) 3.0 3.0 (2.0–4.3) 3.0 3.0 (2.0–4.0) 3.1 3.0 (2.0–4.3) 2.4 2.0 (2.0–3.3) 2.9 3.0 (2.0–3.5)
2.2 2.0 (1.8–3.0) 3.7 4.0 (2.8–5.0) 3.2 4.0 (1.5–5.0) 3.7 4.0 (3.0–4.0)
Median (IQR)
Mean
Detectability (Not at All Easy, 1, to Very Easy, 5)
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Ambiguous
Retain
Retain
Remove
Remove
Retain
Retain
Ambiguous
Ambiguous
Ambiguous
Retain
Retain
Retain
Retain
Remove
Categorization as a Result of Delphi Round
3
8
7
3
1
8
4
4
4
3
6
9
3
7
3
Expert Panel Votes on First Categorization: Retain Votes
Removed
Retained
Retained
Removed
Removed
Retained
Removed
Removed
Removed
Removed
Retained
Retained
Removed
Retained
Removed
Categorization Resulting From Expert Panel Votes on First Categorization
(Continued on next page)
6
2
3
7
9
2
6
6
6
6
4
1
7
3
7
Expert Panel Votes on First Categorization: Exclude Votes
Stockwell et al J Patient Saf • Volume 00, Number 00, Month 2014
© 2014 Lippincott Williams & Wilkins
© 2014 Lippincott Williams & Wilkins
2.7 2.5 (2.0–3.0) 1.1 1.0 (1.0–1.0) 1.9 1.5 (1.0–2.3) 2.0 2.0 (1.0–2.3) 2.8 2.0 (2.0–4.0) 2.9 2.5 (2.0–4.0)
Oversedation/lethargy/fall
Terbutaline use
Anti-Xa > 1.5
aPTT > 100
INR elevated, vitamin K (excluding newborns), vitamin K + warfarin Platelet count < 50,000
Acute kidney injury/dialysis/ creatinine elevation
ALT rising and taken isoniazid, phenytoin, cyclosporine, methotrexate, or warfarin during the past 7 d
Medications: general
3.5 3.5 (2.0–5.0) 2.7 2.0 (2.0–4.0) 2.0 2.0 (1.0–3.0)
2.8 3.0 (2.0–3.0)
Naloxone
Abrupt medication stop
1.7 2.0 (1.0–2.0)
N-acetylcysteine
1.5 1.5 (1.0–2.0)
3.3 3.0 (2.8–4.3)
2.0 2.0 (1.0–3.0)
Methadone administration
Protamine
3.9 4.0 (3.0–4.3)
1.8 2.0 (1.0–2.0)
Flumazenil (Romazicon)
2.9 3.0 (2.0–4.0) 3.5 4.0 (2.8–4.0) 3.3 3.0 (3.0–4.0)
3.3 3.5 (3.0–4.0 )
3.6 4.0 (2.75-4.0)
3.6 4.0 (3.0–4.0)
3.3 3.0 (3.0–4.0)
3.5 3.5 (3.0–4.0)
4.2 4.0 (3.8–5.0)
3.7 4.0 (2.8–4.3)
3.1 3.0 (2.0–4.0)
4.1 4.0 (3.0–5.0)
4.6 5.0 (4.0–5.0)
1.0 1.0 (1.0–1.0)
Digibind administration
3.2 3.0 (3.0–4.0)
1.5 1.0 (1.0–2.0)
Benztropine administration
Medications: general
High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: reversal agents High-risk medications: antithrombotics High-risk medications: antithrombotics High-risk medications: antithrombotics High-risk medications: antithrombotics High-risk medications: antithrombotics Medications: general 2.9 3.5 (1.0–4.25) 4.3 4.5 (4.0–5.0) 4.0 4.0 (3.8–5.0)
4.6 5.0 (4.0–5.0)
4.4 5.0 (3.8–5.0)
4.6 5.0 (4.0–5.0)
4.6 5.0 (4.0–5.0)
4.4 5.0 (3.8–5.0)
4.1 5.0 (3.5–5.0)
2.1 2.0 (1.8–2.3)
4.7 5.0 (4.0–5.0)
4.3 5.0 (3.8–5.0)
4.3 5.0 (3.8–5.0)
4.7 5.0 (4.0–5.0)
4.6 5.0 (4.0–5.0)
4.3 5.0 (3.0–5.0)
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Ambiguous
Retain
Remove
Ambiguous
Retain
Retain
Ambiguous
Ambiguous
Ambiguous
Remove
Retain
Ambiguous
Ambiguous
Ambiguous
Ambiguous
Ambiguous
3
10
4
8
7
9
7
6
1
1
10
3
2
8
6
3
www.journalpatientsafety.com
Removed
Retained
Removed
Retained
Retained
Retained
Retained
Retained
Removed
Removed
Retained
Removed
Removed
Retained
Retained
Removed
(Continued on next page)
6
0
5
1
3
1
2
3
8
9
0
5
7
1
3
6
J Patient Saf • Volume 00, Number 00, Month 2014 Pediatric All-Cause Harm Measurement Tool
5
6
www.journalpatientsafety.com
Drug level out of range
Gentamicin/tobramycin: (except CF patients) trough > 2 mg/l or peak > 10 mg/l Hypercalcemia
Medications: general
Medications: general
Hyperkalemia/sodium polystyrene administration
Hypernatremia
Hypoglycemia/glucagon administration/dextrose bolus + insulin Hypokalemia
Medications: general
Medications: general
Medications: general
Hyponatremia
Laxative or stool softeners
New allergy added after 24 h
Phenobarbital
Rash
STAT orders
Medications: general
Medications: general
Medications: general
Medications: general
Medications: general
Medications: general
Medications: general
Hyperglycemia
Medications: general
Medications: general
Antihistamines
Medications: general
Trigger
Antiemetic administration
Medications: general
Group
TABLE 1. (Continued)
3.0 3.0 (2.0–4.0) 2.9 3.0 (2.0–4.0) 3.9 3.5 (3.0–5.0) 3.0 2.5 (2.0–4.3) 1.8 2.0 (1.0–2.0) 4.0 4.0 (3.0–5.0) 3.6 3.5 (2.8–5.0)
2.6 3.0 (2.0–3.0) 3.6 4.0 (2.6–4.3) 2.5 2.0 (2.0–3.3) 2.3 2.0 (1.3-3.0) 3.0 3.0 (2.0–4.0)
4.1 4.5 (3.0–5.0) 4.0 4.0 (4.0–4.5) 3.7 3.0 (3.0–5.0) 2.7 2.0 (2.0–3.3)
2.6 2.5 (2.0–3.3) 3.4 3.0 (3.0–4.0) 2.0 2.0 (1.8–2.3) 2.7 3.0 (2.0–3.0) 3.1 3.0 (2.8–4.0) 1.8 2.0 (1.0–2.0) 2.5 3.0 (1.8–3.0)
3.0 3.0 (2.8–4.0) 2.8 3.0 (2.0–3.3) 3.5 4.0 (2.8–4.0) 3.1 3.0 (2.0–4.0) 3.8 4.0 (3.0–4.3)
2.1 2.0 (1.8–2.3) 2.1 2.0 (2.0–2.5) 2.8 3.0 (2.0–3.0) 3.0 3.0 (2.0–4.0)
Median (IQR)
Mean
Mean Median (IQR)
Severity (Not at All Serious, 1, to Very Serious, 5)
Frequency (Rarely, 1, to Very Common, 5)
4.5 5.0 (4.0–5.0) 4.5 5.0 (4.0–5.0) 4.3 5.0 (4.0–5.0) 4.0 4.0 (3.0–5.0) 4.0 5.0 (2.0–5.0) 2.8 3.0 (2.0–4.0) 3.6 4.0 (2.0–5.0)
4.3 5.0 (3.8–5.0) 4.2 4.5 (3.8–5.0) 4.5 5.0 (4.0–5.0) 4.4 5.0 (4.0–5.0) 4.5 5.0 (4.0–5.0)
4.0 4.5 (3.0–5.0) 4.3 5.0 (3.5–5.0) 4.0 4.0 (3.8–5.0) 4.7 5.0 (4.0–5.0)
Median (IQR)
Mean
Detectability (Not at All Easy, 1, to Very Easy, 5)
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Retain
Remove
Ambiguous
Retain
Ambiguous
Retain
Retain
Retain
Ambiguous
Retain
Retain
Retain
Retain
Retain
Ambiguous
Ambiguous
Categorization as a Result of Delphi Round
6
3
0
6
5
8
6
10
6
8
6
5
9
6
3
1
Expert Panel Votes on First Categorization: Retain Votes
Retained
Removed
Removed
Retained
Retained
Retained
Retained
Retained
Retained
Retained
Retained
Retained
Retained
Retained
Removed
Removed
Categorization Resulting From Expert Panel Votes on First Categorization
(Continued on next page)
4
7
9
3
5
1
4
0
2
2
3
5
1
4
6
8
Expert Panel Votes on First Categorization: Exclude Votes
Stockwell et al J Patient Saf • Volume 00, Number 00, Month 2014
© 2014 Lippincott Williams & Wilkins
NEC
Specialty consult
Abrupt drop in Hgb or Hct (>25%)
Administration of general anesthesia
Admission to intensive care postoperatively
Arterial or central line insertion during surgery (unanticipated)
Cancelled elective procedure, delayed discharge
Change in planned procedure
Complication of procedure or treatment
Diagnosis change preoperatively/postoperatively
Injury, repair, or removal of organ during operative procedure
Intraoperative IV epinephrine, norepinephrine
Intubation or reintubation or use of BiPAP in PACU
Mechanical ventilation > 24 h postoperatively
Operative time > 6 h (in noncardiac patient)
Pathology report normal or unrelated to diagnosis
Postoperative increase in troponin levels > 1.5 ng/ml
Unplanned return to surgery
NICU
NICU
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
Perioperative
NICU
Abnormal cranial imaging/IVH grade 3 or 4 head bleed in CICU or NICU Bilirubin (total) > 25
NICU 1.3 1.0 (1.0–2.0) 2.4 2.0 (2.0–3.0) 4.3 4.5 (4.0–5.0) 2.5 2.0 (1.8–3.3) 3.6 4.0 (2.0–5.0) 2.3 2.0 (1.8–3.0) 2.1 2.0 (2.0–2.3) 3.3 3.0 (2.0–4.3) 2.0 2.0 (2.0–2.0) 2.5 2.0 (2.0–3.0) 2.5 2.5 (1.8–3.3) 1.4 1.0 (1.0–2.0) 1.7 2.0 (1.0–2.0) 2.0 2.0 (2.0–2.0) 2.6 2.5 (2.0–3.0) 2.2 2.0 (2.0–3.0) 2.1 2.0 (1.0–3.0) 1.0 1.0 (1.0–1.0) 2.3 2.0 (1.8–3.0)
2.4 2.5 (1.8–3.0) 4.2 4.0 (3.8–5.0) 4.3 4.0 (4.0–5.0) 2.2 2.5 (1.0–3.0) 3.8 4.0 (2.8–5.0) 2.7 3.0 (1.0–3.5) 3.5 4.0 (2.8–4.0) 3.3 3.0 (3.0–4.0) 2.5 2.0 (2.0–3.3) 3.0 2.5 (2.0–4.3) 3.6 3.5 (3.0–4.0) 3.1 3.0 (2.0–4.3) 4.5 4.5 (4.0–5.0) 4.0 4.0 (3.0–5.0) 3.6 4.0 (3.0–4.0) 3.3 3.0 (3.0–4.0) 3.5 3.5 (3.0–4.0) 3.2 3.0 (3.0–3.5) 3.5 4.0 (2.0–5.0) 4.3 4.0 (4.0–5.0)
4.7 5.0 (4.0–5.0) 4.5 5.0 (4.8–5.0) 2.7 3.0 (1.8–3.3) 3.4 3.5 (2.5–5.0) 4.0 4.0 (3.8–5.0) 3.0 3.0 (1.0–5.0) 3.8 4.0 (2.8–5.0) 2.5 2.5 (1.0–4.0) 2.6 2.5 (1.8–3.3) 2.7 3.0 (1.0–4.0) 2.1 2.0 (1.0–3.0) 2.8 2.5 (2.0–4.0) 2.7 2.5 (1.8–3.5) 3.9 4.5 (2.8–5.0) 2.9 2.5 (2.0–4.0) 3.1 3.0 (2.0–4.0) 3.6 4.0 (2.8–4.3) 2.5 2.0 (1.8–4.0) 3.8 4.5 (2.0–5.0) 2.6 2.0 (1.8–3.5)
3.4 3.0 (2.0–5.0)
© 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Remove
Ambiguous
Remove
Ambiguous
Retain
Remove
Ambiguous
Remove
Remove
Remove
Remove
Remove
Remove
Ambiguous
Retain
Retain
Ambiguous
Remove
Ambiguous
Ambiguous
9
4
3
6
8
8
6
8
5
8
7
3
6
2
3
8
1
7
7
7
Manual retained
Removed
Removed
Retained
Retained
Manual retained
Retained
Manual retained
Manual retained
Manual retained
Manual retained
Removed
Manual retained
Removed
Removed
Retained
Removed
Manual retained
Retained
Retained
(Continued on next page)
1
6
7
4
2
2
4
2
5
2
3
7
4
7
7
2
9
2
2
2 J Patient Saf • Volume 00, Number 00, Month 2014 Pediatric All-Cause Harm Measurement Tool
www.journalpatientsafety.com
7
J Patient Saf • Volume 00, Number 00, Month 2014
8
X-ray intraoperatively or in PACU Perioperative
www.journalpatientsafety.com
ADT, admission, diagnosis, and transfer; ALI, acute lung injury; ALT, alanine transminase; aPTT, activated partial thromboplastin time; ARDS, adult respiratory distress syndrome; BiPAP, bilevel positive airway pressure; CF, cystic fibrosis; CICU, cardiac intensive care unit; CVC, central venous catheter; CXR, chest x-ray; D-dimer, dimerized plasmin fragment D; DVT, deep venous thrombosis; ED, emergency department; ETT, endotracheal tube; HAI, healthcare associated infection; Hct, hematocrit; Hgb, hemoglobin; INR, international normalized ratio; IQR, interquartile range; IV, intravenous; IVH, intraventricular hemorrhage; LMWH, low-molecular-weight heparin; MRSA, methicillin-resistant Staphylococcus aureus; NEC, necrotizing enterocolitis; PACU, postanesthesia care unit; SIRS, systemic inflammatory response syndrome; VRE, vancomycin-resistant enterococci.
Retained 5 Ambiguous
5
Manual retained 0 10 Remove
2.4 2.0 (1.0–4.3) 3.5 3.5 (2.0–5.0) 4.7 5.0 (4.8–5.0) 2.7 3.0 (2.0–3.3) 1.5 1.0 (1.0–1.3) 2.2 2.0 (2.0–3.0) Wrong site, wrong procedure, wrong patient Perioperative
Median (IQR) Median (IQR) Median (IQR) Group
TABLE 1. (Continued)
Trigger
Mean Mean
Mean
Severity (Not at All Serious, 1, to Very Serious, 5) Frequency (Rarely, 1, to Very Common, 5)
Detectability (Not at All Easy, 1, to Very Easy, 5)
Categorization as a Result of Delphi Round
Categorization Resulting From Expert Panel Votes on First Categorization Expert Panel Votes on First Categorization: Exclude Votes Expert Panel Votes on First Categorization: Retain Votes
Stockwell et al
electronic specificity for the qualifying triggers. For example, if there was a laboratory value–based trigger, the experts were asked to develop the value that would be the set point for the trigger. After each panel member had the opportunity to view the candidate trigger definitions and developed logic, proposals for each were vetted on conference calls, and their appropriateness for inclusion was determined by vote. Expert feedback from this exercise yielded design descriptors and thresholds for further testing in a paper-based and EMR environment.
Step 5 (Finalization) Proposals for each trigger were shared and discussed with all panelists, and consensus on the final pediatric global trigger tool for pilot testing was reached via voting for retention or removal by majority vote.
RESULTS Eleven content experts participated in each step of the developmental process described above. The literature review and trigger collection (step 1) yielded 108 triggers. Step 2 (prioritization) resulted in 21 harms and associated triggers being placed in the remove group (scored 3 in detectability but 25 mg/dL Hyperglycemia (14 mmol/l or 250 mg/dL) Hypernatremia (>160 mEq/l) Hypoglycemia (2 mmol/l or 40 mg/dl) Hyponatremia ( 100 s Anti-Xa > 1.5 Warfarin: INR > 6 Warfarin: vitamin K administration after warfarin administration Protamine administration Naloxone administration Flumazenil administration Ongoing or intermittent laxative use Digibind administration Hyperkalemia (K+ > 6.0 mEq/l) and sodium polystyrene administration Elevated drug levels (antiepileptics): phenytoin (>30 μg/ml) Elevated drug levels (antiepileptics): valproic acid (>170 μg/ml) Elevated drug levels (antiepileptics): carbamazepine (>20 μg/ml) Elevated drug levels (antiepileptics): oxcarbamazepine (>45 μg/ml) Elevated drug levels (antiepileptics): phenobarbital (>30 μg/ml) Elevated drug levels (antibiotics): vancomycin (trough > 25 μg/ml) Elevated drug levels (antibiotics): gentamicin (trough > 4 μg/ml) Elevated drug levels (antibiotics): tobramycin (trough > 4 μg/ml) Elevated drug levels (antibiotics): amikacin (trough > 20 μg/ml) Racemic epinephrine administration (within 24 h after endotracheal extubation)
© 2014 Lippincott Williams & Wilkins
51 triggers that constitute the proposed Pediatric All-Cause Harm Measurement Tool (Tables 2–7).
DISCUSSION By integrating the findings of multiple prior pediatric-specific trigger tool studies, the experience and expertise of 11 pediatric trigger tool experts, the strong project management infrastructure of the CHA, as well as the strengths of a modified Delphi method, we believe that our final tool is the most sophisticated pediatric all-cause harm measurement tool ever constructed. Our decision to use this modified Delphi approach resulted in a final tool that harnessed the collective intelligence of the 11 thought leaders and the findings in the literature to date. Although 2 other pediatric global trigger tools exist, neither was constructed with the benefit of existing literature or such a cadre of trigger experts and neither integrated the practical requirement of detectability in an EMR environment. Our tool, once tested, should emerge as the state of the science in pediatric all-cause harm measurement. There are several limitations to this study. First, there is a distinct lack of published investigations for specific triggers, packages of triggers, and utility of those packages in the pediatric setting. A recent publication evaluated the use of the IHI Adult Global Trigger Tool in a pediatric hospital24 and revealed that many of the adult-focused triggers were not relevant in the pediatric environment. Second, expert opinion was used extensively to construct the tool. Opinions, however expert they are, are subject to biases that can nevertheless impact trigger/harm selection and final tool creation in uncertain ways. These biases were minimized by our use of a modified Delphi method. Third, the final tool was constructed as a subset from an original collection of triggers identified in the literature or via experience of the authors. Hence, any unpublished triggers or those used outside the authorship experience were, by definition, excluded from the final tool. Finally, we minimized inclusion of multielement triggers (for example, TABLE 5. Perioperative Triggers P1 P2 P3 P4 P5 P6
Abrupt drop of >25% in hemoglobin or hematocrit Unanticipated insertion of arterial or central venous line during surgery Intraoperative epinephrine, norepinephrine, or phenylephrine (noncardiac patients) Mechanical ventilation > 48 h postoperatively Operative time > 6 h (noncardiac patients) X-ray intraoperatively or in postanesthesia care unit
www.journalpatientsafety.com
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
9
J Patient Saf • Volume 00, Number 00, Month 2014
Stockwell et al
TABLE 6. Readmission Triggers
7. Resar RK, Rozich JD, Classen DC. Methodology and rationale for the measurement of harm with trigger tools. Qual Saf Health Care. 2003;12(suppl 2):ii39–ii45.
R1 R2 R3
8. Classen DC, Resar R, Griffin F, et al. ‘Global trigger tool’ shows that harms in hospitals may be ten times greater than previously measured. Health Aff. 2011;30:581–589.
Readmission to ICU within 24 h from discharge/transfer Hospital readmission within 30 d Readmission to the emergency department within 48 h
TABLE 7. Resuscitation/Death Triggers RES1 RES2 RES3
Any code or arrest or rapid response team activation Transfer to higher level of care All inpatient deaths
creatinine doubling in 24 h and urine output decreasing by 50% in 24 h), purposefully sacrificing potential increased specificity in favor of the efficiency of single-element triggers. Future triggers are likely to include additional complexity as the ability to query for multielement triggers becomes increasingly possible in more sophisticated electronic environments. This study identified 51 pediatric triggers for testing in a pediatric inpatient environment. Pilot testing of this initial list is the next phase of the work. This pilot test phase will identify positive predictive values for each trigger, suggest more appropriate set points, and capture harm prevalence before the third phase of broad-scale validation testing as well as allow for some comparisons with recent similar research with the Canadian Pediatric Trigger Tool.25 The broad scale validation testing phase will establish positive predictive values of each trigger, establish the sensitivity and specificity of the entire tool, and will most importantly describe the epidemiology of all-cause harm in pediatric inpatients. Finally, the pediatric all-cause harm tool will be embedded and tested in the electronic environment, which should further enhance the sensitivity of the findings, expand the populations evaluated, and improve the efficiency of the local review process. With a pediatric all-cause harm measurement tool constructed to maximize the opportunities provided by the EMR, the goal of real-time harm detection, and the ability to mitigate harms as a result, is moving rapidly toward a reality. REFERENCES 1. Landrigan CP, Parry GJ, Bones CB, et al. Temporal trends in rates of patient harm resulting from medical care. N Engl J Med. 2010;363:2124–2134. 2. Cullen DJ, Bates DW, Small SD, et al. The incident reporting system does not detect adverse drug events: a problem for quality improvement. Jt Comm J Qual Improv. 1995;21:541–548. 3. Stockwell DC, Slonim AD. Quality and safety in critical care units. Intensive Care Med. 2006;21:199–210. 4. Office of the Inspector General. Harms in Hospitals: Methods for Identifying Events. Washington, DC: Department of Health and Human Services; 2010. OEI-06-08-00221. Available at: http://www.oig.hhs.gov/oei/reports/ oei-06-08-00221.pdf. Accessed November 25, 2012.
9. Dickerman MJ, Jacobs BR, Vinodrao H, et al. Recognizing hypoglycemia in children through automated harm detection. Pediatrics. 2011;127: e1035–e1041. 10. Ferranti J, Horvath MM, Cozart H, et al. Reevaluating the safety profile of pediatrics: a comparison of computerized adverse drug event surveillance and voluntary reporting in the pediatric environment. Pediatrics. 2008;121:e1201–e1207. 11. Kilbridge PM, Noirot LA, Reichley RM, et al. Computerized surveillance for adverse drug events in a pediatric hospital. J Am Med Inform Assoc. 2009;16:607–612. 12. Takata GS, Mason W, Taketomo C, et al. Development, testing, and findings of a pediatric-focused trigger tool to identify medication-related harm in US children's hospitals. Pediatrics. 2008;121: e927–e935. 13. Sharek PJ, Horbar JD, Mason W, et al. Harms in the neonatal intensive care unit: development, testing, and findings of a NICU-focused trigger tool to identify harm in North American NICUs. Pediatrics. 2006;118: 1332–1340. 14. Agarwal S, Classen DC, Larsen GL, et al. Incidence of harms in pediatric intensive care units in the United States. Pediatr Crit Care Med. 2010;11: 568–578. 15. Sharek PJ, Parry G, Goldmann DA, et al. Performance characteristics of a methodology to quantify adverse events over time in hospitalized patients. Health Serv Res. 2011;46:654–678. 16. Matlow AG, Cronin CM, Flintoft V, et al. Description of the development and validation of the Canadian Paediatric Trigger Tool. BMJ Qual Saf. 2011;20:416–423. 17. Franklin BD, Birch S, Schachter M, Barber N. Testing a trigger tool as a method of detecting harm from medication errors in a UK hospital: a pilot study. Int J Pharm Pract. 2010;18:305–311. 18. Classen DC, Pestotnik SL, Evans RS, et al. Computerized surveillance of adverse drug events in hospital patients. JAMA. 1991;266:2847–2851. 19. Jha AK, Kuperman GJ, Teich JM, et al. Identifying harms: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc. 1998;5:305–314. 20. Bates DW, Evans RS, Murff H, et al. Detecting harms using information technology. J Am Med Inform Assoc. 2003;10:115–128. 21. Sittig DF, Singh H. Electronic health records and national patient-safety goals. N Engl J Med. 2012;367:1854–1860. 22. Williams PL, Webb C. The Delphi technique: a methodological discussion. J Adv Nurs. 1994;19:180–186. 23. Mull HJ, Nebeker JR, Shimada SL, et al. Consensus building for development of outpatient adverse drug event triggers. J Patient Saf. 2011; 7:66–71.
5. Griffin FA, Resar RK. IHI Global Trigger Tool for Measuring Harms (Second Edition). IHI Innovation Series White Paper. Cambridge, MA: Institute for Healthcare Improvement; 2009.
24. Kirkendall E, Kloppenborg E, Papp J, et al. Measuring harms and levels of harm in pediatric inpatients with the Global Trigger Tool. Pediatrics. 2012; 130:e1206–e1214.
6. Rozich JD, Haraden CR, Resar RK. Adverse drug event trigger tool: a practical methodology for measuring medication related harm. Qual Saf Health Care. 2003;12:194–200.
25. Matlow AG, Baker GR, Flintoft V, et al. Adverse events among children in Canadian hospitals: the Canadian Paediatric Adverse Events Study. CMAJ. 2012;184:E709–E718.
10
www.journalpatientsafety.com
© 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
