International Journal of Cardiology 179 (2015) 358–359
Contents lists available at ScienceDirect
International Journal of Cardiology journal homepage: www.elsevier.com/locate/ijcard
Letter to the Editor
External validation of the GRACE Freedom from Events score in an emergency department ‘rule out ACS’ chest pain cohort Anne-Maree Kelly a,b,⁎, Sharon Klim a, Kean Soon b,c a b c
Joseph Epstein Centre for Emergency Medicine at Western Health, Melbourne, Australia The University Of Melbourne, Melbourne, Australia Centre for Cardiovascular Therapeutics at Western Health, Melbourne, Australia
a r t i c l e
i n f o
Article history: Received 5 November 2014 Accepted 6 November 2014 Available online 7 November 2014 Keywords: Chest pain Risk stratification Emergency Acute coronary syndrome
Coronary artery disease (CAD) is common and the major challenges facing emergency departments (ED) are determining which patients presenting with chest pain have an acute coronary syndrome (ACS) and which are at low risk of adverse events and suitable for early discharge. As part of the GRACE registry project, the GRACE Freedom-fromEvent score (GFFES) was developed and internally validated [1]. It included patients admitted to a hospital with suspected acute coronary syndrome (ACS) and aimed to identify patients with a low risk of adverse in-hospital events who might be suitable for treatment in less resource intensive environments than coronary care units (CCUs). This score has undergone limited external validation [2,3] and has not been validated in an ED ‘rule-out ACS’ cohort. We aimed to externally validate the GFFES in an ED ‘rule-out ACS’ cohort in the hope that it might be a useful tool to aid ED disposition decision-making. This was a planned sub-study of a prospective observational study of consecutive adult patients presenting to the ED of a community teaching hospital. Patients were eligible if they presented with chest pain. Exclusion criteria were traumatic chest pain, aged b18 years, no chest pain within 24 h of the index ED visit, chest pain lasting b10 min, no ECG or no troponin assay performed within 24 h of index ED visit, a
⁎ Corresponding author at: JECEMR, Sunshine Hospital, Furlong Road, St Albans, 3021, Australia. E-mail addresses: [email protected], [email protected] (A.-M. Kelly), [email protected] (S. Klim), [email protected] (K. Soon).
http://dx.doi.org/10.1016/j.ijcard.2014.11.076 0167-5273/© 2014 Elsevier Ireland Ltd. All rights reserved.
clear alternative diagnosis at an initial medical officer assessment, evidence of acute ischaemia on initial ECG, haemodynamic instability, advanced terminal disease, inability to communicate in English and declined/unavailable for follow-up. We collected demographics, cardiac risk factors, history of CAD, cardiac failure, atrial fibrillation or peripheral vascular disease, clinical features, medications, biochemical analyses including cardiac biomarkers, ECG findings, interventions during hospitalization, clinical course and occurrence of defined major adverse cardiovascular events (MACE; death, new myocardial infarction (MI), survived cardiac arrest or significant arrhythmia (ventricular tachycardia, atrial fibrillation, symptomatic bradycardia)) within 30 days. MI was defined as troponin elevation N99th centile in combination with treating cardiologist's final diagnosis of MI. In cases where only one of these criteria was present, the final diagnosis was adjudicated by independent cardiologists with access to all clinical data. The troponin assay used by the laboratory was TnI-Ultra (Siemens Diagnostics). The primary outcome of interest was the predictive performance of the GFFES for MACE within 30 days (ROC curve, sensitivity, negative predictive value (NPV)) using the previously published cut-off for low risk of GFFES ≥287 [1]. The study was approved by the institutional ethics panel and was registered with the Australia and New Zealand Clinical Trials Registry (ACTRN12612000990820). Patients provided verbal consent to telephone follow-up. 1076 patients were included. Demographic and clinical features are shown in Table 1. 157 patients (14.6%) had a final diagnosis of MI. 14 patients (1.3%, 95% CI 0.8–2.2%) suffered a MACE. 721 patients (67%, 95% CI 64.1–69.8%) were classified as low risk by the GFFES. Area under the ROC curve (AUC) for prediction of MACE at 30 days was 0.8 (95% CI 0.72–0.89) (Fig. 1). Sensitivity was 92.8% (95% CI 64.2–99.6%), specificity was 67.8% (95% CI 64.9–70.6%) and NPV was 99.9% (95% CI 99.1–100%). The one patient that was missed at the published cut-off was a 40-year-old male with known CAD and a previous episode of AF who had negative serial biomarkers and developed a further episode of AF in the follow-up period. No patient had MI, cardiac arrest or death. 286 patients were admitted to a cardiology service (26.5%, 95% CI 24– 29.3%). Of these 132 had a low risk GFFES (46.2%, 95% CI 40.5–51.9%). One patient in this group (0.8%, 95% CI 0.1–4.2%) suffered a MACE at 30 days — the previously described patient with an episode of AF.
A.-M. Kelly et al. / International Journal of Cardiology 179 (2015) 358–359
359
Table 1 Sample characteristics. Variable
Overall cohort (N = 1076)
GFFES high risk group (n = 355)
GFFES low risk group (n = 721)
Gender (N male, %) Age (years, median, IQR) Ambulance arrival (N, %) Risk factors Hypertension (N, %) Diabetes (N, %) Current smoker (N, %) Known renal impairment (N, %) Family history (N, %) Hypercholesterolaemia (N, %) Known CAD (N, %) Risk scores TIMI (median, IQR) GRACE risk score (median, IQR) GRACE Freedom from Events score (median, IQR) Disposition (home, N, %) Discharge diagnosis Myocardial infarction (N, %) Non-MI ACS (N, %) Chest pain for investigation (N, %) Non-cardiac (N, %) Cardiac — non-ACS (N, %) MACE at 30 days
614, 57% 59 (48–70) 851, 79%
208, 59% 72 (64–81) 308, 87%
406, 56% 53 (45–61) 543, 75%
621, 58% 268, 25% 254, 24% 87, 8% 359, 33% 598, 56% 251, 23%
273, 77% 148, 42% 52, 15% 52, 15% 112, 32% 241, 68% 122, 34%
348, 48% 120, 17% 202, 28% 35, 5% 247, 34% 357, 50% 129, 18%
2 (1–3) 95 (76–118) 305 (274–324) 741, 69%
3 (2–4) 128 (110–148) 259 (223–273) 175, 49.3%
1 (0–3) 81 (67–98) 317 (305–330) 566, 78.5%
157, 15% 64, 6% 783, 73% 11, 1% 61, 6% 14, 1.3%
115, 32% 26, 7% 179, 50% 10, 3% 25, 7% 13, 3.7%
42, 6% 38, 5% 604, 84% 1, 0.1% 36, 5% 1, 0.1%
IQR = inter-quartile range.
The GFFES [1] was developed to identify patients with a low risk of adverse in-hospital events who might be suitable for treatment in less resource intensive environments. To date, external validation of this score has been limited with two validations reporting AUC for prediction of defined adverse events of 0.69 (95% CI 0.60 to 0.79) [2] and 0.74 (95% CI 0.62–0.86) [3]. One study reported clinical predictive performance for MACE with sensitivity of 100% (95% CI 71.7–100%), specificity of 34.7% (95% CI 28.5–41.3%) and NPV of 100% (95% CI 94.2%–100%) [3]. This study is the first validation in an ED chest pain population. Our finding of 99.9% NPV for MACE (i.e. that for low risk patients MACE was very rare) suggests that this tool may have a role in disposition decisionmaking in ED. If GFFES low risk patients without a TnI pattern consistent
1 0.9
True positive rate (Sensitivity)
0.8
with MI in ED had been treated in the ED observation unit or as outpatients, there would have been 104 fewer admissions, a 36% reduction in admission rate. The potential benefit of the GFFES score to aid ED disposition and reduce CCU admission rate was considerable in our study but could be even greater in health systems with high ward admission rates for ED chest pain patients. Application of the GFFES in routine practice faces some barriers. In particular, it has a high number of variables and a complex range of points are allocated based on data within those variable bands. An online calculator or app would assist implementation. There are some limitations to this study. While patients were identified prospectively, some data was collected from the medical record with the inherent weaknesses associated. It was conducted at a single site so it may not be generalisable to other settings. Determination of risk factors and past history was by patient self-report. No attempt was made to confirm the information provided, reflecting the ‘real world’ ED setting. It is also possible that there were some cases of missed MI, principally because of atypical presentations, in particular lack of chest pain. In summary, the GFFES showed good discrimination for MACE. These results support a potential role for this score to assist disposition decision-making for ED chest pain patients.
0.7
Conflicts of interest 0.6 0.5
The authors report no relationships that could be construed as a conflict of interest.
0.4
Acknowledgement
0.3
We would like to thank Dr Nadim Shah for assistance with adjudication of diagnosis and our team of data collectors.
0.2
References 0.1 0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
False positive rate (1-Specificity) Fig. 1. ROC curve of predictive performance of GFFES.
0.9
1
[1] D. Brieger, K.A.A. Fox, G. FitzGerald, et al., Predicting freedom from clinical events in non-ST-elevation acute coronary syndromes: the Global Registry of Acute Coronary Events, Heart 95 (2009) 888–894. [2] M. Söderholm, M.M. Deligani, M. Choudhary, J. Björk, U. Ekelund, Ability of risk scores to predict a low complication risk in patients admitted for suspected acute coronary syndrome, Emerg. Med. J. 29 (2012) 644–649. [3] A.M. Kelly, P. Dabee, S. Klim, K. Soon, External Validation of the GRACE Freedom from Events score, Heart Lung Circ. 21 (2012) 582–585.
Contents lists available at ScienceDirect
International Journal of Cardiology journal homepage: www.elsevier.com/locate/ijcard
Letter to the Editor
External validation of the GRACE Freedom from Events score in an emergency department ‘rule out ACS’ chest pain cohort Anne-Maree Kelly a,b,⁎, Sharon Klim a, Kean Soon b,c a b c
Joseph Epstein Centre for Emergency Medicine at Western Health, Melbourne, Australia The University Of Melbourne, Melbourne, Australia Centre for Cardiovascular Therapeutics at Western Health, Melbourne, Australia
a r t i c l e
i n f o
Article history: Received 5 November 2014 Accepted 6 November 2014 Available online 7 November 2014 Keywords: Chest pain Risk stratification Emergency Acute coronary syndrome
Coronary artery disease (CAD) is common and the major challenges facing emergency departments (ED) are determining which patients presenting with chest pain have an acute coronary syndrome (ACS) and which are at low risk of adverse events and suitable for early discharge. As part of the GRACE registry project, the GRACE Freedom-fromEvent score (GFFES) was developed and internally validated [1]. It included patients admitted to a hospital with suspected acute coronary syndrome (ACS) and aimed to identify patients with a low risk of adverse in-hospital events who might be suitable for treatment in less resource intensive environments than coronary care units (CCUs). This score has undergone limited external validation [2,3] and has not been validated in an ED ‘rule-out ACS’ cohort. We aimed to externally validate the GFFES in an ED ‘rule-out ACS’ cohort in the hope that it might be a useful tool to aid ED disposition decision-making. This was a planned sub-study of a prospective observational study of consecutive adult patients presenting to the ED of a community teaching hospital. Patients were eligible if they presented with chest pain. Exclusion criteria were traumatic chest pain, aged b18 years, no chest pain within 24 h of the index ED visit, chest pain lasting b10 min, no ECG or no troponin assay performed within 24 h of index ED visit, a
⁎ Corresponding author at: JECEMR, Sunshine Hospital, Furlong Road, St Albans, 3021, Australia. E-mail addresses: [email protected], [email protected] (A.-M. Kelly), [email protected] (S. Klim), [email protected] (K. Soon).
http://dx.doi.org/10.1016/j.ijcard.2014.11.076 0167-5273/© 2014 Elsevier Ireland Ltd. All rights reserved.
clear alternative diagnosis at an initial medical officer assessment, evidence of acute ischaemia on initial ECG, haemodynamic instability, advanced terminal disease, inability to communicate in English and declined/unavailable for follow-up. We collected demographics, cardiac risk factors, history of CAD, cardiac failure, atrial fibrillation or peripheral vascular disease, clinical features, medications, biochemical analyses including cardiac biomarkers, ECG findings, interventions during hospitalization, clinical course and occurrence of defined major adverse cardiovascular events (MACE; death, new myocardial infarction (MI), survived cardiac arrest or significant arrhythmia (ventricular tachycardia, atrial fibrillation, symptomatic bradycardia)) within 30 days. MI was defined as troponin elevation N99th centile in combination with treating cardiologist's final diagnosis of MI. In cases where only one of these criteria was present, the final diagnosis was adjudicated by independent cardiologists with access to all clinical data. The troponin assay used by the laboratory was TnI-Ultra (Siemens Diagnostics). The primary outcome of interest was the predictive performance of the GFFES for MACE within 30 days (ROC curve, sensitivity, negative predictive value (NPV)) using the previously published cut-off for low risk of GFFES ≥287 [1]. The study was approved by the institutional ethics panel and was registered with the Australia and New Zealand Clinical Trials Registry (ACTRN12612000990820). Patients provided verbal consent to telephone follow-up. 1076 patients were included. Demographic and clinical features are shown in Table 1. 157 patients (14.6%) had a final diagnosis of MI. 14 patients (1.3%, 95% CI 0.8–2.2%) suffered a MACE. 721 patients (67%, 95% CI 64.1–69.8%) were classified as low risk by the GFFES. Area under the ROC curve (AUC) for prediction of MACE at 30 days was 0.8 (95% CI 0.72–0.89) (Fig. 1). Sensitivity was 92.8% (95% CI 64.2–99.6%), specificity was 67.8% (95% CI 64.9–70.6%) and NPV was 99.9% (95% CI 99.1–100%). The one patient that was missed at the published cut-off was a 40-year-old male with known CAD and a previous episode of AF who had negative serial biomarkers and developed a further episode of AF in the follow-up period. No patient had MI, cardiac arrest or death. 286 patients were admitted to a cardiology service (26.5%, 95% CI 24– 29.3%). Of these 132 had a low risk GFFES (46.2%, 95% CI 40.5–51.9%). One patient in this group (0.8%, 95% CI 0.1–4.2%) suffered a MACE at 30 days — the previously described patient with an episode of AF.
A.-M. Kelly et al. / International Journal of Cardiology 179 (2015) 358–359
359
Table 1 Sample characteristics. Variable
Overall cohort (N = 1076)
GFFES high risk group (n = 355)
GFFES low risk group (n = 721)
Gender (N male, %) Age (years, median, IQR) Ambulance arrival (N, %) Risk factors Hypertension (N, %) Diabetes (N, %) Current smoker (N, %) Known renal impairment (N, %) Family history (N, %) Hypercholesterolaemia (N, %) Known CAD (N, %) Risk scores TIMI (median, IQR) GRACE risk score (median, IQR) GRACE Freedom from Events score (median, IQR) Disposition (home, N, %) Discharge diagnosis Myocardial infarction (N, %) Non-MI ACS (N, %) Chest pain for investigation (N, %) Non-cardiac (N, %) Cardiac — non-ACS (N, %) MACE at 30 days
614, 57% 59 (48–70) 851, 79%
208, 59% 72 (64–81) 308, 87%
406, 56% 53 (45–61) 543, 75%
621, 58% 268, 25% 254, 24% 87, 8% 359, 33% 598, 56% 251, 23%
273, 77% 148, 42% 52, 15% 52, 15% 112, 32% 241, 68% 122, 34%
348, 48% 120, 17% 202, 28% 35, 5% 247, 34% 357, 50% 129, 18%
2 (1–3) 95 (76–118) 305 (274–324) 741, 69%
3 (2–4) 128 (110–148) 259 (223–273) 175, 49.3%
1 (0–3) 81 (67–98) 317 (305–330) 566, 78.5%
157, 15% 64, 6% 783, 73% 11, 1% 61, 6% 14, 1.3%
115, 32% 26, 7% 179, 50% 10, 3% 25, 7% 13, 3.7%
42, 6% 38, 5% 604, 84% 1, 0.1% 36, 5% 1, 0.1%
IQR = inter-quartile range.
The GFFES [1] was developed to identify patients with a low risk of adverse in-hospital events who might be suitable for treatment in less resource intensive environments. To date, external validation of this score has been limited with two validations reporting AUC for prediction of defined adverse events of 0.69 (95% CI 0.60 to 0.79) [2] and 0.74 (95% CI 0.62–0.86) [3]. One study reported clinical predictive performance for MACE with sensitivity of 100% (95% CI 71.7–100%), specificity of 34.7% (95% CI 28.5–41.3%) and NPV of 100% (95% CI 94.2%–100%) [3]. This study is the first validation in an ED chest pain population. Our finding of 99.9% NPV for MACE (i.e. that for low risk patients MACE was very rare) suggests that this tool may have a role in disposition decisionmaking in ED. If GFFES low risk patients without a TnI pattern consistent
1 0.9
True positive rate (Sensitivity)
0.8
with MI in ED had been treated in the ED observation unit or as outpatients, there would have been 104 fewer admissions, a 36% reduction in admission rate. The potential benefit of the GFFES score to aid ED disposition and reduce CCU admission rate was considerable in our study but could be even greater in health systems with high ward admission rates for ED chest pain patients. Application of the GFFES in routine practice faces some barriers. In particular, it has a high number of variables and a complex range of points are allocated based on data within those variable bands. An online calculator or app would assist implementation. There are some limitations to this study. While patients were identified prospectively, some data was collected from the medical record with the inherent weaknesses associated. It was conducted at a single site so it may not be generalisable to other settings. Determination of risk factors and past history was by patient self-report. No attempt was made to confirm the information provided, reflecting the ‘real world’ ED setting. It is also possible that there were some cases of missed MI, principally because of atypical presentations, in particular lack of chest pain. In summary, the GFFES showed good discrimination for MACE. These results support a potential role for this score to assist disposition decision-making for ED chest pain patients.
0.7
Conflicts of interest 0.6 0.5
The authors report no relationships that could be construed as a conflict of interest.
0.4
Acknowledgement
0.3
We would like to thank Dr Nadim Shah for assistance with adjudication of diagnosis and our team of data collectors.
0.2
References 0.1 0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
False positive rate (1-Specificity) Fig. 1. ROC curve of predictive performance of GFFES.
0.9
1
[1] D. Brieger, K.A.A. Fox, G. FitzGerald, et al., Predicting freedom from clinical events in non-ST-elevation acute coronary syndromes: the Global Registry of Acute Coronary Events, Heart 95 (2009) 888–894. [2] M. Söderholm, M.M. Deligani, M. Choudhary, J. Björk, U. Ekelund, Ability of risk scores to predict a low complication risk in patients admitted for suspected acute coronary syndrome, Emerg. Med. J. 29 (2012) 644–649. [3] A.M. Kelly, P. Dabee, S. Klim, K. Soon, External Validation of the GRACE Freedom from Events score, Heart Lung Circ. 21 (2012) 582–585.
