ORIGINAL ARTICLE Clinical utility of inappropriate positron emission tomography myocardial perfusion imaging: Test results and cardiovascular events David E. Winchester, MD, MS,a,b Ryan J. Chauffe, DO,b Ryan Meral, BA,b Daniel Nguyen, MD,b Scott Ryals, MD,b Raman Dusaj, MD,a,b Leslee Shaw, PhD,c and Rebecca J. Beyth, MD MSca,b a b c
Malcom Randall VA Medical Center, Gainesville, FL College of Medicine, University of Florida, Gainesville, FL Emory University School of Medicine, Atlanta, GA
Received Jan 1, 2014; revised May 15, 2014; accepted May 16, 2014 doi:10.1007/s12350-014-9925-1
Background. Appropriate use criteria for myocardial perfusion imaging (MPI) were developed to categorize scenarios where MPI might be beneficial (appropriate) or not (inappropriate). Few investigations have evaluated the clinical utility of this categorization strategy, particularly with positron emission tomography (PET) MPI. Methods and Results. We conducted this retrospective cohort investigation in a Veterans Affairs (VA) medical center, on predominantly male subjects who underwent PET-MPI. We correlated appropriateness to test result and cardiovascular events. Of 521 subjects, 414 (79.5%) were appropriate, 54 (10.4%) were uncertain, and 53 (10.2%) were inappropriate. PET-MPI was abnormal more often when appropriate or uncertain (28% and 34.6%, respectively, vs 7.7% for inappropriate, P 5 .003). Among abnormal inappropriate tests, none detected occult ischemia. By Cox regression, summed difference score ‡5 (HR 5.06, 95% CI 2.72-9.44) and an abnormal test result (HR 4.48, 95% CI 2.19-9.14) were associated with higher likelihood of catheterization. Log-rank analysis demonstrated similar likelihood of catheterization when comparing abnormal vs normal test result (P < .0001) and between appropriate, uncertain, and inappropriate tests (P 5 .024). Conclusions. Inappropriate PET-MPI was rarely abnormal, associated with low catheterization rates, and failed to detect occult ischemia for any subjects. The clinical utility of inappropriate PET-MPI is negligible. (J Nucl Cardiol 2015;22:9–15.) Key Words: Myocardial perfusion imaging: PET Æ appropriate use criteria Æ outcome
See related editorial, pp. 16–21 INTRODUCTION After decades of worldwide use, nuclear myocardial perfusion imaging (MPI) has amassed an extensive literature demonstrating a robust capacity to stratify risk
Reprint requests: David E. Winchester, MD, MS, Malcom Randall VA Medical Center, Gainesville, FL 32608; [email protected] 1071-3581/$34.00 Copyright Ó 2014 American Society of Nuclear Cardiology.
for clinically relevant cardiovascular outcomes. As a result, MPI is commonly used in the United States for the assessment of coronary ischemia. This widespread use has driven some to question whether the test is used inappropriately. Concerns about inappropriate use are one potential factor contributing to recent decreases in reimbursement by federal regulators and third-party payers. Professional societies have suggested that attention should be focused on reducing MPI performed in settings where patients are not likely to benefit from the test, rather than reduce reimbursement globally. To inform this discussion, these societies have developed appropriate use criteria (AUC) 9
10
Winchester et al Clinical utility of inappropriate PET-MPI
for MPI with the initial publication in 20051 and revised in 2009.2 The central premise of the AUC is that appropriate tests are those ‘‘in which the expected incremental information, combined with the clinical judgment, exceeds the negative consequences.’’ Determinations about AUC categories are made using existing research on the value of MPI for various patient populations and clinical scenarios. Data are limited, however, regarding the use of AUC to directly measure the clinical utility of MPI, particularly the effects on patient management and patient outcomes. Ideally, AUC would be able to reliably and prospectively identify patient populations and clinical scenarios, where MPI is not clinically useful which could then be used to prospectively discourage unnecessary testing. We conducted this investigation to better understand the clinical utility of inappropriate MPI, specifically positron emission tomography (PET), which has greater sensitivity than single-photon emission computed tomography (SPECT) for detecting coronary artery disease (CAD).3 We hypothesize that inappropriate PET-MPI will provide less clinical utility than appropriate or uncertain PET-MPI as determined by the rate of abnormal testing and subsequent catheterization. METHODS Patient Selection and Data Collection We examined PET-MPI interpreted in a single Veterans Affairs (VA) medical center and conducted a retrospective cohort analysis of appropriateness by applying the AUC. The investigation was reviewed by our institutional review board which waived the requirement for seeking informed consent. We used the VA Computerized Patient Record System to identify subjects who underwent PET-MPI from December 2010 to July 2011. Testing was performed utilizing rubidium82 PET and regadenoson as the pharmacologic stress agent. PET-MPI could be ordered by any clinician or in the outlying clinics that refer to our laboratory for testing, primarily cardiologists, hospitalists, primary care providers, and emergency medicine providers. We used the documentation from the encounter during which PET-MPI was ordered to determine AUC category. Demographic information and medical history were extracted from the same document if possible, and from prior notes if necessary. Data were recorded using a standardized paper form and then entered into an electronic database for statistical analysis. Appropriate use was determined using the 2009 criteria2 and a paper instrument endorsed by professional societies and the Imaging in FOCUS (Formation of Optimal Cardiovascular Utilization Strategies) initiative.4 Four authors (RC, RM, DN, and SR) extracted the data. If the primary extractors had any doubt about AUC classification, or if any subjects met more than one AUC indication, then the primary
Journal of Nuclear CardiologyÒ January/February 2015
author (DEW) adjudicated the result. No formal assessments of interrater or intrarater variability for AUC indications were performed. When clinicians ordered PET-MPI for subjects with symptoms, the clinician was presumed to be concerned that ischemic heart disease was the cause of those symptoms. Diabetes mellitus was defined as hemoglobin A1C greater than 8% or the documented prescription of oral/injected medications for treatment of diabetes. CAD was defined as prior myocardial infarction (MI) or revascularization. Data gathered for each PET-MPI included the summed stress score (SSS), summed rest score (SRS), summed difference score (SDS), and test conclusion. PET-MPI was classified as normal or abnormal based on the conclusion of the clinical report; normal and probably normal results were combined as were abnormal and probably abnormal. We only used the original interpretation of the PET-MPI for these determinations and did not perform any secondary interpretation of the PET-MPI. For each subject, outcomes were assessed 1 year after which the PET-MPI was performed. Cardiovascular events (CE) including death, MI, coronary revascularization, and cardiac catheterization were recorded.
Outcomes and Statistical Analysis To demonstrate the clinical utility of inappropriate PETMPI, the primary outcome of this investigation was the proportion of PET-MPI categorized as abnormal. At our facilities, the proportion of abnormal PET-MPI is approximately 20%. We estimated from our prior investigation in this population that the rate of inappropriate PET-MPI to be at least 10%.5 Assuming that appropriate tests are more likely to be abnormal and inappropriate tests less, an unequal sample of 500 (with 10% inappropriate) would have greater than 80% power to detect a 20% absolute difference in the proportion of PET-MPI, which are abnormal. Power analysis was performed using G*Power 3.6 The secondary outcome was to determine the occurrence of CE as listed above and estimate the likelihood of catheterization based on patient and test characteristics. Analysis of baseline characteristics and CE at 1 year was performed across three subject categories (appropriate, uncertain, and inappropriate). For calculation of univariate associations and Cox regression, appropriate and uncertain studies were combined, thus stratifying the study cohort into an appropriate or uncertain group and an inappropriate group. One-way ANOVA was used to compare means of continuous variables. Chi-square and Fisher’s exact test were used to compare proportions in categorical variables as appropriate. Univariate correlation and Cox proportional hazards model were used to calculate odds ratio (OR) or hazard ratio (HR) and 95% confidence intervals (CI) to estimate the association between catheterization and symptoms, test results, and other patient characteristics with statistically significant univariate correlations. Kaplan-Meier plots and log-rank tests were used to demonstrate the time-dependent likelihood of catheterization. Statistical analysis was performed using SPSS version 21 (IBM; Armonk, NY). We applied the Strengthening the Reporting of Observational Studies in Epidemiology method in design of our investigation.7
Journal of Nuclear CardiologyÒ Volume 22, Number 1;9–15
Winchester et al Clinical utility of inappropriate PET-MPI
RESULTS Baseline Characteristics We identified 521 subjects; 414 (79.5%) of PETMPI were appropriate, 54 (10.4%) were uncertain, and 53 (10.2%) were inappropriate, consistent with our prior report from this population.5 We excluded 8 cases from our analysis which could not be classified by the AUC. The subjects were predominantly male with high prevalence of overweight, diabetes mellitus, hypertension, hyperlipidemia, and CAD as shown in Table 1.
Primary Outcome: Abnormal Test Results As demonstrated in Table 1, abnormal tests were more common in subjects with appropriate and uncertain PET-MPI, while inappropriate PET-MPI was rarely abnormal (28.0% and 34.6%, respectively, vs 7.7%, P = .003). Only 4 inappropriate tests were abnormal, and details for each subject with both inappropriate and abnormal PET-MPI are summarized in Table 2. Two inappropriate abnormal PET-MPI demonstrated fixed defects in patients with known CAD. One was a false positive PET-MPI confirmed by computed tomography coronary angiography and one detected ischemia in a patient with a clinically evident acute coronary syndrome. Significant ischemia (SDS C 5) was detected in
11
54 (13.7%) subjects with appropriate testing, 10 (19.6%) subjects with uncertain testing, and only 1 (2%) patient with inappropriate testing (P = .02). The only inappropriate test with significant ischemia was performed on the aforementioned subject with definite acute coronary syndrome prior to undergoing PET-MPI (2009 AUC Indication #10 Definite ACS). Secondary Outcome: Cardiovascular Events CE occurred more frequently in subjects with appropriate or uncertain PET-MPI (Table 3). The overall rate of death or MI was low, approximately 3.5%. Cardiac catheterization was the most common event observed and the only one significantly different across the groups (P = .02). Several factors were associated with catheterization in univariate correlation, as noted in Table 4. Significant correlations were seen for abnormal PET-MPI result (OR 14.31, 95% CI 8.00-25.64), inappropriate PET-MPI (OR 0.10, 95% CI 0.01-0.74), SDS C5 (OR 25.10, 95% CI 13.36-47.14), diabetes mellitus (OR 1.66, 95% CI 1.02-2.70), hyperlipidemia (OR 2.69, 95% CI 1.30-5.56), and CAD (OR 3.56, 95% CI 2.135.97). The presence or absence of symptoms was not significantly associated with catheterization. We created a Cox proportional hazards model of catheterization likelihood using the following variables: age, CAD, DM, chest pain, no symptoms, abnormal test, inappropriate
Table 1. Baseline characteristics of 521 patients undergoing PET-MPI
Age BMI Female Overweight Obese Diabetes Hypertension Hyperlipidemia CAD Current smoker Chest pain Dyspnea Fatigue No symptoms Abnormal test SDS C5
Appropriate (n 5 414)
Uncertain (n 5 54)
Inappropriate (n 5 53)
P value
62.7 ± 9.1 33.4 ± 7.2 13 (3.1%) 369 (89.3%) 281 (68.0%) 187 (45.2%) 340 (82.1%) 321 (77.5%) 170 (41.1%) 111 (26.8%) 249 (60.1%) 178 (43.0%) 47 (11.4%) 57 (13.8%) 114 (28.0%) 54 (13.7%)
63.4 ± 8.3 34.8 ± 7.4 4 (7.4%) 54 (100%) 36 (66.7%) 18 (33.3%) 51 (94.4%) 43 (79.6%) 33 (61.1%) 14 (25.9%) 5 (9.3%) 12 (22.2%) 2 (3.7%) 28 (51.9%) 18 (34.6%) 10 (19.6%)
57.6 ± 10.6 33.7 ± 6.6 3 (5.7%) 48 (90.6%) 38 (71.7%) 7 (13.2%) 36 (67.9%) 30 (56.6%) 10 (18.9%) 10 (18.9%) 5 (9.4%) 22 (41.5%) 2 (3.8%) 24 (45.3%) 4 (7.7%) 1 (2.0%)
\.0001 .364 .236 .042 .835 \.0001 .002 .003 \.0001 .462 \.0001 .014 .061 \.0001 .003 .024
Values are reported as mean ± standard deviation and n (%) BMI, body mass index; CAD, coronary artery disease; MPI, myocardial perfusion imaging; SDS summed difference score
12
Winchester et al Clinical utility of inappropriate PET-MPI
Journal of Nuclear CardiologyÒ January/February 2015
Table 2. Clinical history for subjects with MPI that was both inappropriate and abnormal
SSS Without CE 1 Asymptomatic patient less than 2 years after PCI, fixed defect on MPI 2 Asymptomatic patient with intermediate risk, MPI detected ischemia, follow-up cardiac CTCA was negative 3 Asymptomatic patient less than 2 years after PCI, fixed defect on MPI With CE 1 Patient with clinical ACS, MPI abnormal, had MI after bypass surgery
SDS
AUC indication
13
1
59
4
4
13
25
1
59
16
15
10
ACS, acute coronary syndrome; AUC, appropriate use criteria; CE, cardiovascular events; CTCA, computed tomography coronary angiography; MI, myocardial infarction; MPI, myocardial perfusion imaging; PCI, percutaneous coronary intervention; SDS, summed difference score; SSS, summed stress score
Table 3. Cardiovascular events at 1 year
Appropriate (n 5 414) Catheterization Revascularization MI Death Death or MI
68 31 3 13 15
Uncertain (n 5 54)
(16.4%) (7.5%) (0.7%) (3.1%) (3.6%)
7 2 1 1 2
Inappropriate (n 5 53)
(13.0%) (3.7%) (1.9%) (1.9%) (3.7%)
1 1 1 0 1
(1.9%) (1.9%) (1.9%) (0%) (1.9%)
P value .02 .20 .56 .38 .80
Values reported as n (%) MI, myocardial infarction
test, and SDS C 5. Factors retained in the model of catheterization likelihood were an abnormal test result (HR 4.48, 95% CI 2.19-9.14), and SDS C 5 (HR 5.06, 95% CI 2.72-9.44). The likelihood of catheterization was compared for appropriate vs uncertain vs inappropriate PET-MPI and normal vs abnormal PET-MPI by the log-rank test and in Kaplan-Meier plots (Figures 1 and 2). A similar stratification of catheterization likelihood was observed for both normal vs abnormal PET-MPI (log rank, P \ .001) and inappropriate vs appropriate/uncertain PET-MPI (log rank, P = .024). Log-rank analysis was also performed for death and MI alone, but no significant differences were observed (appropriate/uncertain vs inappropriate PET-MPI, P = .81; normal vs abnormal PET-MPI, P = .16). DISCUSSION We observed that inappropriate PET-MPI was rarely abnormal, and PET-MPI that was both inappropriate and abnormal did not detect occult ischemia.
Table 4. Univariate correlation of factors related to catheterization
Female Overweight Obese Diabetes Hypertension Hyperlipidemia CAD Current smoker Chest pain Dyspnea Fatigue No symptoms Abnormal test Inappropriate test SDS C5
OR
95% CI
P value
0.30 2.03 0.82 1.66 1.21 2.69 3.56 1.20 1.22 1.46 1.10 0.75 14.31 0.10 25.10
0.39–2.27 0.71–5.82 0.49–1.36 1.02–2.70 0.62–2.33 1.30–5.56 2.13–5.97 0.70–2.05 0.75–1.99 0.90–2.39 0.50–2.44 0.40–1.42 8.00–25.64 0.01–0.74 13.36–47.14
.22 .18 .44 .04 .58 .006 \.0001 .51 .424 .125 .815 .376 \.0001 .006 \.0001
CAD, coronary artery disease; CI, confidence interval; OR, odds ratio; SDS, summed difference score
Journal of Nuclear CardiologyÒ Volume 22, Number 1;9–15
Winchester et al Clinical utility of inappropriate PET-MPI
13
Figure 1. Survival free of catheterization by test result—This Kaplan-Meier plot demonstrates significantly lower survival free of catheterization for patients with abnormal PET-MPI as compared to those with normal PET-MPI.
Figure 2. Survival free of catheterization by appropriateness—This Kaplan-Meier plot demonstrates significantly lower survival free of catheterization for patients with appropriate PET-MPI or uncertain PET-MPI as compared to those with inappropriate PET-MPI.
Cardiovascular events were rare in patients undergoing inappropriate PET-MPI. A similar difference in catheterization likelihood was observed when comparing
appropriate/uncertain vs inappropriate PET-MPI as with normal vs abnormal PET-MPI. This suggests that inappropriate PET-MPI, even in a population with high
14
Winchester et al Clinical utility of inappropriate PET-MPI
prevalence of CAD and its risk factors, offer minimal clinical utility. Further demonstrating the minimal clinical utility of inappropriate PET-MPI, we observed that occult ischemia was not detected in any subject with inappropriate PET-MPI. The only subject with demonstrable ischemia was categorized as definite acute coronary syndrome, and likely should have gone directly for cardiac catheterization instead of noninvasive testing. Furthermore, this subject was the only one in the inappropriate group to undergo catheterization and revascularization. He underwent bypass surgery for multi-vessel CAD and suffered a post-operative MI, making him the only subject with inappropriate testing to have MI or death during follow-up. Data regarding the results of inappropriate MPI and subsequent cardiac events are limited, but expanding. Our results demonstrate a lower likelihood of catheterization with inappropriate testing, seen in log-rank analysis, but not maintained on Cox regression. These results, drawn from a sample utilizing PET-MPI, are consistent with other data which suggest that inappropriate SPECT-MPI fails to detect patients at elevated risk of cardiovascular events, such as Aldweib et al who found that subjects tested inappropriately tend to have both low cardiovascular risk at baseline and have favorable outcomes.8-10 Khawaja et al11 found that inappropriate PET-MPI was less likely to be abnormal and less likely to result in angiography or revascularization, while Doukky et al9 demonstrated that the prognostic value of inappropriate PET-MPI is less than that of appropriate PET-MPI. Peer-reviewed manuscripts have demonstrated the failure to risk stratify in specific categories of inappropriate testing, such as perioperative assessment12 and syncope.13 While similar data have been reported for stress echocardiography,14 our investigation is one of the few to examine all inappropriate MPI indications and is the only study that we are aware of using strictly rubidium-82 PET and demonstrating minimal clinical utility. Another unique element of our investigation is the VA setting, which suggests that utility is limited even in a population with higher than average cardiovascular risk factors (42% diabetics, 26% current smokers, 90% overweight, and 82% hypertensive). Demonstrating that inappropriate PET-MPI performs poorly for detecting ischemia and cardiovascular events is not an unexpected finding. In creating the AUC, experts were asked to weigh the totality of existing evidence to select patient populations and clinical scenarios, where MPI is most likely to result in net benefit to the patient. The importance of these findings lies in the fact that appropriateness categorization can be determined a priori, as opposed to the results
Journal of Nuclear CardiologyÒ January/February 2015
of the MPI which must clearly be determined after which the test is completed. As comparative effectiveness becomes more of a focus for research and clinical application, AUC may play a key role.15 Indeed, initiatives such as those described above are beginning to gain traction. The Choosing Wisely campaign encourages both patients and physicians to avoid tests and procedures which are overused.16 Both the American College of Cardiology and the American Society of Nuclear Cardiology discourage the use of MPI in low-risk or asymptomatic patients, in accord with AUC. Practice improvement modules such as the Formation of Optimal Cardiovascular Utilization Strategies demonstrate that physician-led quality improvement can reduce inappropriate use.17 Multiple vendors are developing and implementing web-based and electronic medical record system-based tools, some of which will allow providers to assess appropriateness at the point of care.18,19 Some insurance companies have partnered with the American College of Cardiology utilizing AUC in lieu of radiology benefits managers for prior authorization.20 LIMITATIONS Our investigation is not without limitations. Our data were gathered retrospectively and are, therefore, dependent on the available clinical documentation of each encounter to provide adequate detail for determining appropriateness. AUC determination was not automated and was performed by un-blinded reviewers; our rate of inappropriate studies was on the low end of the reported range, possibly reflecting bias. We relied on the original clinical interpretation of the PET-MPI rather than perform a fully quantitative interpretation by a reader blinded to clinical, stress ECG, and outcome data because images for all studies not easily available for direct review. NEW KNOWLEDGE GAINED Our investigation adds to the growing body of literature demonstrating that MPI classified as inappropriate based on AUC developed by professional societies are rarely abnormal and offer little clinical benefit to patients. Specifically, we have demonstrated this in a population of only rubidium-82 PET-MPI.
CONCLUSION Inappropriate PET-MPI was rarely abnormal and failed to detect occult ischemia. The likelihood of undergoing cardiac catheterization diverged between
Journal of Nuclear CardiologyÒ Volume 22, Number 1;9–15
appropriateness categories in a pattern similar to that observed for normal to abnormal PET-MPI. These findings suggest that inappropriate rubidium-82 PETMPI offers negligible clinical benefit to patients.
Acknowledgments This work was supported by resources provided by the North Florida/South Georgia Veterans Health System. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.
Disclosure This investigation was supported by NIH T35 Training Grant: T35-HL007489-28. Dr Winchester had full access to the data and accepts responsibility for its integrity.
References 1. Brindis RG, Douglas PS, Hendel RC, Peterson ED, Wolk MJ, Allen JM, et al. ACCF/ASNC appropriateness criteria for singlephoton emission computed tomography myocardial perfusion imaging (SPECT MPI): A report of the American College of Cardiology Foundation Quality Strategic Directions Committee Appropriateness Criteria Working Group and the American Society of Nuclear Cardiology endorsed by the American Heart Association. J Am Coll Cardiol 2005;46(8):1587-605. 2. Hendel RC, Berman DS, Di Carli MF, Heidenreich PA, Henkin RE, Pellikka PA, et al. ACCF/ASNC/ACR/AHA/ASE/SCCT/ SCMR/SNM 2009 appropriate use criteria for cardiac radionuclide imaging: A report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, the American Society of Nuclear Cardiology, the American College of Radiology, the American Heart Association, the American Society of Echocardiography, the Society of Cardiovascular Computed Tomography, the Society for Cardiovascular Magnetic Resonance, and the Society of Nuclear Medicine. Circulation 2009;119(22):e561-87. 3. Parker MW, Iskandar A, Limone B, Perugini A, Kim H, Jones C, et al. Diagnostic accuracy of cardiac positron emission tomography versus single photon emission computed tomography for coronary artery disease: A bivariate meta-analysis. Circ Cardiovasc Imaging 2012;5(6):700-7. 4. FOCUS Initiative, http://www.cardiosource.org/Science-AndQuality/Quality-Programs/Imaging-in-FOCUS/. Accessed August 10, 2012. 5. Winchester DE, Meral R, Ryals S, Beyth RJ, Shaw LJ. Appropriate use of myocardial perfusion imaging in a veteran population: Profit motives and professional liability concerns. JAMA Intern Med 2013;173(14):1381-3.
Winchester et al Clinical utility of inappropriate PET-MPI
15
6. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007;39(2):175-91. 7. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: Guidelines for reporting observational studies. Ann Intern Med 2007;147(8):573-7. 8. DeMuth KFJ, Hernandez M, Rodriguez J, Schwartz R, Lambiris I, Asher C, et al. Evaluation and Improvement of SPECT appropriateness criteria with clinical outcomes: A long-term follow-up study. J Am Coll Cardiol 2013;61(10):E989. 9. Doukky R, Hayes K, Frogge N, Balakrishnan G, Dontaraju VS, Rangel MO, et al. Impact of appropriate use on the prognostic value of single-photon emission computed tomography myocardial perfusion imaging. Circulation 2013;128(15):1634-43. 10. Aldweib N, Negishi K, Seicean S, Jaber WA, Hachamovitch R, Cerqueira M, et al. Appropriate test selection for single-photon emission computed tomography imaging: Association with clinical risk, posttest management, and outcomes. Am Heart J 2013;166(3): 581-8. 11. Khawaja FJ, Jouni H, Miller TD, Hodge DO, Gibbons RJ. Downstream clinical implications of abnormal myocardial perfusion single-photon emission computed tomography based on appropriate use criteria. J Nucl Cardiol 2013;20(6):1041-8. 12. Koh AS, Flores JL, Keng FY, Tan RS, Chua TS. Correlation between clinical outcomes and appropriateness grading for referral to myocardial perfusion imaging for preoperative evaluation prior to non-cardiac surgery. J Nucl Cardiol 2012;19(2):277-84. 13. Aljaroudi WA, Alraies MC, Wazni O, Cerqueira MD, Jaber WA. Yield and diagnostic value of stress myocardial perfusion imaging in patients without known coronary artery disease presenting with syncope. Circ Cardiovasc Imaging 2013;6(3):384-91. 14. Cortigiani L, Bigi R, Bovenzi F, Molinaro S, Picano E, Sicari R. Prognostic implication of appropriateness criteria for pharmacologic stress echocardiography performed in an outpatient clinic [corrected]. Circ Cardiovasc Imaging 2012;5(3):298-305. 15. Shaw LJ, Hage FG, Berman DS, Hachamovitch R, Iskandrian A. Prognosis in the era of comparative effectiveness research: Where is nuclear cardiology now and where should it be? J Nucl Cardiol 2012;19(5):1026-43. 16. Rao VM, Levin DC. The overuse of diagnostic imaging and the Choosing Wisely initiative. Ann Intern Med. 2012;157(8):574-6. 17. Saifi S, Taylor AJ, Allen J, Hendel R. The use of a learning community and online evaluation of utilization for SPECT myocardial perfusion imaging. JACC Cardiovasc Imaging 2013;6(7): 823-9. 18. Allen J. Abstract P231: A National Initiative to Improve Utilization of Cardiac Imaging: The FOCUS Learning Community and Performance Improvement Module. Circulation 2011;4(6 Supplement): AP231. 19. Apps4Docs. http://www.appsdocs.com/. Accessed May 21, 2013. 20. American College of Cardiology. FOCUS for Health Plans: Cardiovascular Imaging Strategies. http://www.cardiosource.org/ Science-And-Quality/Quality-Programs/Imaging-in-FOCUS/FOCUSfor-Health-Plans-and-Cardiovascular-Imaging-Strategies.aspx. Accessed May 21, 2013.
Malcom Randall VA Medical Center, Gainesville, FL College of Medicine, University of Florida, Gainesville, FL Emory University School of Medicine, Atlanta, GA
Received Jan 1, 2014; revised May 15, 2014; accepted May 16, 2014 doi:10.1007/s12350-014-9925-1
Background. Appropriate use criteria for myocardial perfusion imaging (MPI) were developed to categorize scenarios where MPI might be beneficial (appropriate) or not (inappropriate). Few investigations have evaluated the clinical utility of this categorization strategy, particularly with positron emission tomography (PET) MPI. Methods and Results. We conducted this retrospective cohort investigation in a Veterans Affairs (VA) medical center, on predominantly male subjects who underwent PET-MPI. We correlated appropriateness to test result and cardiovascular events. Of 521 subjects, 414 (79.5%) were appropriate, 54 (10.4%) were uncertain, and 53 (10.2%) were inappropriate. PET-MPI was abnormal more often when appropriate or uncertain (28% and 34.6%, respectively, vs 7.7% for inappropriate, P 5 .003). Among abnormal inappropriate tests, none detected occult ischemia. By Cox regression, summed difference score ‡5 (HR 5.06, 95% CI 2.72-9.44) and an abnormal test result (HR 4.48, 95% CI 2.19-9.14) were associated with higher likelihood of catheterization. Log-rank analysis demonstrated similar likelihood of catheterization when comparing abnormal vs normal test result (P < .0001) and between appropriate, uncertain, and inappropriate tests (P 5 .024). Conclusions. Inappropriate PET-MPI was rarely abnormal, associated with low catheterization rates, and failed to detect occult ischemia for any subjects. The clinical utility of inappropriate PET-MPI is negligible. (J Nucl Cardiol 2015;22:9–15.) Key Words: Myocardial perfusion imaging: PET Æ appropriate use criteria Æ outcome
See related editorial, pp. 16–21 INTRODUCTION After decades of worldwide use, nuclear myocardial perfusion imaging (MPI) has amassed an extensive literature demonstrating a robust capacity to stratify risk
Reprint requests: David E. Winchester, MD, MS, Malcom Randall VA Medical Center, Gainesville, FL 32608; [email protected] 1071-3581/$34.00 Copyright Ó 2014 American Society of Nuclear Cardiology.
for clinically relevant cardiovascular outcomes. As a result, MPI is commonly used in the United States for the assessment of coronary ischemia. This widespread use has driven some to question whether the test is used inappropriately. Concerns about inappropriate use are one potential factor contributing to recent decreases in reimbursement by federal regulators and third-party payers. Professional societies have suggested that attention should be focused on reducing MPI performed in settings where patients are not likely to benefit from the test, rather than reduce reimbursement globally. To inform this discussion, these societies have developed appropriate use criteria (AUC) 9
10
Winchester et al Clinical utility of inappropriate PET-MPI
for MPI with the initial publication in 20051 and revised in 2009.2 The central premise of the AUC is that appropriate tests are those ‘‘in which the expected incremental information, combined with the clinical judgment, exceeds the negative consequences.’’ Determinations about AUC categories are made using existing research on the value of MPI for various patient populations and clinical scenarios. Data are limited, however, regarding the use of AUC to directly measure the clinical utility of MPI, particularly the effects on patient management and patient outcomes. Ideally, AUC would be able to reliably and prospectively identify patient populations and clinical scenarios, where MPI is not clinically useful which could then be used to prospectively discourage unnecessary testing. We conducted this investigation to better understand the clinical utility of inappropriate MPI, specifically positron emission tomography (PET), which has greater sensitivity than single-photon emission computed tomography (SPECT) for detecting coronary artery disease (CAD).3 We hypothesize that inappropriate PET-MPI will provide less clinical utility than appropriate or uncertain PET-MPI as determined by the rate of abnormal testing and subsequent catheterization. METHODS Patient Selection and Data Collection We examined PET-MPI interpreted in a single Veterans Affairs (VA) medical center and conducted a retrospective cohort analysis of appropriateness by applying the AUC. The investigation was reviewed by our institutional review board which waived the requirement for seeking informed consent. We used the VA Computerized Patient Record System to identify subjects who underwent PET-MPI from December 2010 to July 2011. Testing was performed utilizing rubidium82 PET and regadenoson as the pharmacologic stress agent. PET-MPI could be ordered by any clinician or in the outlying clinics that refer to our laboratory for testing, primarily cardiologists, hospitalists, primary care providers, and emergency medicine providers. We used the documentation from the encounter during which PET-MPI was ordered to determine AUC category. Demographic information and medical history were extracted from the same document if possible, and from prior notes if necessary. Data were recorded using a standardized paper form and then entered into an electronic database for statistical analysis. Appropriate use was determined using the 2009 criteria2 and a paper instrument endorsed by professional societies and the Imaging in FOCUS (Formation of Optimal Cardiovascular Utilization Strategies) initiative.4 Four authors (RC, RM, DN, and SR) extracted the data. If the primary extractors had any doubt about AUC classification, or if any subjects met more than one AUC indication, then the primary
Journal of Nuclear CardiologyÒ January/February 2015
author (DEW) adjudicated the result. No formal assessments of interrater or intrarater variability for AUC indications were performed. When clinicians ordered PET-MPI for subjects with symptoms, the clinician was presumed to be concerned that ischemic heart disease was the cause of those symptoms. Diabetes mellitus was defined as hemoglobin A1C greater than 8% or the documented prescription of oral/injected medications for treatment of diabetes. CAD was defined as prior myocardial infarction (MI) or revascularization. Data gathered for each PET-MPI included the summed stress score (SSS), summed rest score (SRS), summed difference score (SDS), and test conclusion. PET-MPI was classified as normal or abnormal based on the conclusion of the clinical report; normal and probably normal results were combined as were abnormal and probably abnormal. We only used the original interpretation of the PET-MPI for these determinations and did not perform any secondary interpretation of the PET-MPI. For each subject, outcomes were assessed 1 year after which the PET-MPI was performed. Cardiovascular events (CE) including death, MI, coronary revascularization, and cardiac catheterization were recorded.
Outcomes and Statistical Analysis To demonstrate the clinical utility of inappropriate PETMPI, the primary outcome of this investigation was the proportion of PET-MPI categorized as abnormal. At our facilities, the proportion of abnormal PET-MPI is approximately 20%. We estimated from our prior investigation in this population that the rate of inappropriate PET-MPI to be at least 10%.5 Assuming that appropriate tests are more likely to be abnormal and inappropriate tests less, an unequal sample of 500 (with 10% inappropriate) would have greater than 80% power to detect a 20% absolute difference in the proportion of PET-MPI, which are abnormal. Power analysis was performed using G*Power 3.6 The secondary outcome was to determine the occurrence of CE as listed above and estimate the likelihood of catheterization based on patient and test characteristics. Analysis of baseline characteristics and CE at 1 year was performed across three subject categories (appropriate, uncertain, and inappropriate). For calculation of univariate associations and Cox regression, appropriate and uncertain studies were combined, thus stratifying the study cohort into an appropriate or uncertain group and an inappropriate group. One-way ANOVA was used to compare means of continuous variables. Chi-square and Fisher’s exact test were used to compare proportions in categorical variables as appropriate. Univariate correlation and Cox proportional hazards model were used to calculate odds ratio (OR) or hazard ratio (HR) and 95% confidence intervals (CI) to estimate the association between catheterization and symptoms, test results, and other patient characteristics with statistically significant univariate correlations. Kaplan-Meier plots and log-rank tests were used to demonstrate the time-dependent likelihood of catheterization. Statistical analysis was performed using SPSS version 21 (IBM; Armonk, NY). We applied the Strengthening the Reporting of Observational Studies in Epidemiology method in design of our investigation.7
Journal of Nuclear CardiologyÒ Volume 22, Number 1;9–15
Winchester et al Clinical utility of inappropriate PET-MPI
RESULTS Baseline Characteristics We identified 521 subjects; 414 (79.5%) of PETMPI were appropriate, 54 (10.4%) were uncertain, and 53 (10.2%) were inappropriate, consistent with our prior report from this population.5 We excluded 8 cases from our analysis which could not be classified by the AUC. The subjects were predominantly male with high prevalence of overweight, diabetes mellitus, hypertension, hyperlipidemia, and CAD as shown in Table 1.
Primary Outcome: Abnormal Test Results As demonstrated in Table 1, abnormal tests were more common in subjects with appropriate and uncertain PET-MPI, while inappropriate PET-MPI was rarely abnormal (28.0% and 34.6%, respectively, vs 7.7%, P = .003). Only 4 inappropriate tests were abnormal, and details for each subject with both inappropriate and abnormal PET-MPI are summarized in Table 2. Two inappropriate abnormal PET-MPI demonstrated fixed defects in patients with known CAD. One was a false positive PET-MPI confirmed by computed tomography coronary angiography and one detected ischemia in a patient with a clinically evident acute coronary syndrome. Significant ischemia (SDS C 5) was detected in
11
54 (13.7%) subjects with appropriate testing, 10 (19.6%) subjects with uncertain testing, and only 1 (2%) patient with inappropriate testing (P = .02). The only inappropriate test with significant ischemia was performed on the aforementioned subject with definite acute coronary syndrome prior to undergoing PET-MPI (2009 AUC Indication #10 Definite ACS). Secondary Outcome: Cardiovascular Events CE occurred more frequently in subjects with appropriate or uncertain PET-MPI (Table 3). The overall rate of death or MI was low, approximately 3.5%. Cardiac catheterization was the most common event observed and the only one significantly different across the groups (P = .02). Several factors were associated with catheterization in univariate correlation, as noted in Table 4. Significant correlations were seen for abnormal PET-MPI result (OR 14.31, 95% CI 8.00-25.64), inappropriate PET-MPI (OR 0.10, 95% CI 0.01-0.74), SDS C5 (OR 25.10, 95% CI 13.36-47.14), diabetes mellitus (OR 1.66, 95% CI 1.02-2.70), hyperlipidemia (OR 2.69, 95% CI 1.30-5.56), and CAD (OR 3.56, 95% CI 2.135.97). The presence or absence of symptoms was not significantly associated with catheterization. We created a Cox proportional hazards model of catheterization likelihood using the following variables: age, CAD, DM, chest pain, no symptoms, abnormal test, inappropriate
Table 1. Baseline characteristics of 521 patients undergoing PET-MPI
Age BMI Female Overweight Obese Diabetes Hypertension Hyperlipidemia CAD Current smoker Chest pain Dyspnea Fatigue No symptoms Abnormal test SDS C5
Appropriate (n 5 414)
Uncertain (n 5 54)
Inappropriate (n 5 53)
P value
62.7 ± 9.1 33.4 ± 7.2 13 (3.1%) 369 (89.3%) 281 (68.0%) 187 (45.2%) 340 (82.1%) 321 (77.5%) 170 (41.1%) 111 (26.8%) 249 (60.1%) 178 (43.0%) 47 (11.4%) 57 (13.8%) 114 (28.0%) 54 (13.7%)
63.4 ± 8.3 34.8 ± 7.4 4 (7.4%) 54 (100%) 36 (66.7%) 18 (33.3%) 51 (94.4%) 43 (79.6%) 33 (61.1%) 14 (25.9%) 5 (9.3%) 12 (22.2%) 2 (3.7%) 28 (51.9%) 18 (34.6%) 10 (19.6%)
57.6 ± 10.6 33.7 ± 6.6 3 (5.7%) 48 (90.6%) 38 (71.7%) 7 (13.2%) 36 (67.9%) 30 (56.6%) 10 (18.9%) 10 (18.9%) 5 (9.4%) 22 (41.5%) 2 (3.8%) 24 (45.3%) 4 (7.7%) 1 (2.0%)
\.0001 .364 .236 .042 .835 \.0001 .002 .003 \.0001 .462 \.0001 .014 .061 \.0001 .003 .024
Values are reported as mean ± standard deviation and n (%) BMI, body mass index; CAD, coronary artery disease; MPI, myocardial perfusion imaging; SDS summed difference score
12
Winchester et al Clinical utility of inappropriate PET-MPI
Journal of Nuclear CardiologyÒ January/February 2015
Table 2. Clinical history for subjects with MPI that was both inappropriate and abnormal
SSS Without CE 1 Asymptomatic patient less than 2 years after PCI, fixed defect on MPI 2 Asymptomatic patient with intermediate risk, MPI detected ischemia, follow-up cardiac CTCA was negative 3 Asymptomatic patient less than 2 years after PCI, fixed defect on MPI With CE 1 Patient with clinical ACS, MPI abnormal, had MI after bypass surgery
SDS
AUC indication
13
1
59
4
4
13
25
1
59
16
15
10
ACS, acute coronary syndrome; AUC, appropriate use criteria; CE, cardiovascular events; CTCA, computed tomography coronary angiography; MI, myocardial infarction; MPI, myocardial perfusion imaging; PCI, percutaneous coronary intervention; SDS, summed difference score; SSS, summed stress score
Table 3. Cardiovascular events at 1 year
Appropriate (n 5 414) Catheterization Revascularization MI Death Death or MI
68 31 3 13 15
Uncertain (n 5 54)
(16.4%) (7.5%) (0.7%) (3.1%) (3.6%)
7 2 1 1 2
Inappropriate (n 5 53)
(13.0%) (3.7%) (1.9%) (1.9%) (3.7%)
1 1 1 0 1
(1.9%) (1.9%) (1.9%) (0%) (1.9%)
P value .02 .20 .56 .38 .80
Values reported as n (%) MI, myocardial infarction
test, and SDS C 5. Factors retained in the model of catheterization likelihood were an abnormal test result (HR 4.48, 95% CI 2.19-9.14), and SDS C 5 (HR 5.06, 95% CI 2.72-9.44). The likelihood of catheterization was compared for appropriate vs uncertain vs inappropriate PET-MPI and normal vs abnormal PET-MPI by the log-rank test and in Kaplan-Meier plots (Figures 1 and 2). A similar stratification of catheterization likelihood was observed for both normal vs abnormal PET-MPI (log rank, P \ .001) and inappropriate vs appropriate/uncertain PET-MPI (log rank, P = .024). Log-rank analysis was also performed for death and MI alone, but no significant differences were observed (appropriate/uncertain vs inappropriate PET-MPI, P = .81; normal vs abnormal PET-MPI, P = .16). DISCUSSION We observed that inappropriate PET-MPI was rarely abnormal, and PET-MPI that was both inappropriate and abnormal did not detect occult ischemia.
Table 4. Univariate correlation of factors related to catheterization
Female Overweight Obese Diabetes Hypertension Hyperlipidemia CAD Current smoker Chest pain Dyspnea Fatigue No symptoms Abnormal test Inappropriate test SDS C5
OR
95% CI
P value
0.30 2.03 0.82 1.66 1.21 2.69 3.56 1.20 1.22 1.46 1.10 0.75 14.31 0.10 25.10
0.39–2.27 0.71–5.82 0.49–1.36 1.02–2.70 0.62–2.33 1.30–5.56 2.13–5.97 0.70–2.05 0.75–1.99 0.90–2.39 0.50–2.44 0.40–1.42 8.00–25.64 0.01–0.74 13.36–47.14
.22 .18 .44 .04 .58 .006 \.0001 .51 .424 .125 .815 .376 \.0001 .006 \.0001
CAD, coronary artery disease; CI, confidence interval; OR, odds ratio; SDS, summed difference score
Journal of Nuclear CardiologyÒ Volume 22, Number 1;9–15
Winchester et al Clinical utility of inappropriate PET-MPI
13
Figure 1. Survival free of catheterization by test result—This Kaplan-Meier plot demonstrates significantly lower survival free of catheterization for patients with abnormal PET-MPI as compared to those with normal PET-MPI.
Figure 2. Survival free of catheterization by appropriateness—This Kaplan-Meier plot demonstrates significantly lower survival free of catheterization for patients with appropriate PET-MPI or uncertain PET-MPI as compared to those with inappropriate PET-MPI.
Cardiovascular events were rare in patients undergoing inappropriate PET-MPI. A similar difference in catheterization likelihood was observed when comparing
appropriate/uncertain vs inappropriate PET-MPI as with normal vs abnormal PET-MPI. This suggests that inappropriate PET-MPI, even in a population with high
14
Winchester et al Clinical utility of inappropriate PET-MPI
prevalence of CAD and its risk factors, offer minimal clinical utility. Further demonstrating the minimal clinical utility of inappropriate PET-MPI, we observed that occult ischemia was not detected in any subject with inappropriate PET-MPI. The only subject with demonstrable ischemia was categorized as definite acute coronary syndrome, and likely should have gone directly for cardiac catheterization instead of noninvasive testing. Furthermore, this subject was the only one in the inappropriate group to undergo catheterization and revascularization. He underwent bypass surgery for multi-vessel CAD and suffered a post-operative MI, making him the only subject with inappropriate testing to have MI or death during follow-up. Data regarding the results of inappropriate MPI and subsequent cardiac events are limited, but expanding. Our results demonstrate a lower likelihood of catheterization with inappropriate testing, seen in log-rank analysis, but not maintained on Cox regression. These results, drawn from a sample utilizing PET-MPI, are consistent with other data which suggest that inappropriate SPECT-MPI fails to detect patients at elevated risk of cardiovascular events, such as Aldweib et al who found that subjects tested inappropriately tend to have both low cardiovascular risk at baseline and have favorable outcomes.8-10 Khawaja et al11 found that inappropriate PET-MPI was less likely to be abnormal and less likely to result in angiography or revascularization, while Doukky et al9 demonstrated that the prognostic value of inappropriate PET-MPI is less than that of appropriate PET-MPI. Peer-reviewed manuscripts have demonstrated the failure to risk stratify in specific categories of inappropriate testing, such as perioperative assessment12 and syncope.13 While similar data have been reported for stress echocardiography,14 our investigation is one of the few to examine all inappropriate MPI indications and is the only study that we are aware of using strictly rubidium-82 PET and demonstrating minimal clinical utility. Another unique element of our investigation is the VA setting, which suggests that utility is limited even in a population with higher than average cardiovascular risk factors (42% diabetics, 26% current smokers, 90% overweight, and 82% hypertensive). Demonstrating that inappropriate PET-MPI performs poorly for detecting ischemia and cardiovascular events is not an unexpected finding. In creating the AUC, experts were asked to weigh the totality of existing evidence to select patient populations and clinical scenarios, where MPI is most likely to result in net benefit to the patient. The importance of these findings lies in the fact that appropriateness categorization can be determined a priori, as opposed to the results
Journal of Nuclear CardiologyÒ January/February 2015
of the MPI which must clearly be determined after which the test is completed. As comparative effectiveness becomes more of a focus for research and clinical application, AUC may play a key role.15 Indeed, initiatives such as those described above are beginning to gain traction. The Choosing Wisely campaign encourages both patients and physicians to avoid tests and procedures which are overused.16 Both the American College of Cardiology and the American Society of Nuclear Cardiology discourage the use of MPI in low-risk or asymptomatic patients, in accord with AUC. Practice improvement modules such as the Formation of Optimal Cardiovascular Utilization Strategies demonstrate that physician-led quality improvement can reduce inappropriate use.17 Multiple vendors are developing and implementing web-based and electronic medical record system-based tools, some of which will allow providers to assess appropriateness at the point of care.18,19 Some insurance companies have partnered with the American College of Cardiology utilizing AUC in lieu of radiology benefits managers for prior authorization.20 LIMITATIONS Our investigation is not without limitations. Our data were gathered retrospectively and are, therefore, dependent on the available clinical documentation of each encounter to provide adequate detail for determining appropriateness. AUC determination was not automated and was performed by un-blinded reviewers; our rate of inappropriate studies was on the low end of the reported range, possibly reflecting bias. We relied on the original clinical interpretation of the PET-MPI rather than perform a fully quantitative interpretation by a reader blinded to clinical, stress ECG, and outcome data because images for all studies not easily available for direct review. NEW KNOWLEDGE GAINED Our investigation adds to the growing body of literature demonstrating that MPI classified as inappropriate based on AUC developed by professional societies are rarely abnormal and offer little clinical benefit to patients. Specifically, we have demonstrated this in a population of only rubidium-82 PET-MPI.
CONCLUSION Inappropriate PET-MPI was rarely abnormal and failed to detect occult ischemia. The likelihood of undergoing cardiac catheterization diverged between
Journal of Nuclear CardiologyÒ Volume 22, Number 1;9–15
appropriateness categories in a pattern similar to that observed for normal to abnormal PET-MPI. These findings suggest that inappropriate rubidium-82 PETMPI offers negligible clinical benefit to patients.
Acknowledgments This work was supported by resources provided by the North Florida/South Georgia Veterans Health System. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.
Disclosure This investigation was supported by NIH T35 Training Grant: T35-HL007489-28. Dr Winchester had full access to the data and accepts responsibility for its integrity.
References 1. Brindis RG, Douglas PS, Hendel RC, Peterson ED, Wolk MJ, Allen JM, et al. ACCF/ASNC appropriateness criteria for singlephoton emission computed tomography myocardial perfusion imaging (SPECT MPI): A report of the American College of Cardiology Foundation Quality Strategic Directions Committee Appropriateness Criteria Working Group and the American Society of Nuclear Cardiology endorsed by the American Heart Association. J Am Coll Cardiol 2005;46(8):1587-605. 2. Hendel RC, Berman DS, Di Carli MF, Heidenreich PA, Henkin RE, Pellikka PA, et al. ACCF/ASNC/ACR/AHA/ASE/SCCT/ SCMR/SNM 2009 appropriate use criteria for cardiac radionuclide imaging: A report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, the American Society of Nuclear Cardiology, the American College of Radiology, the American Heart Association, the American Society of Echocardiography, the Society of Cardiovascular Computed Tomography, the Society for Cardiovascular Magnetic Resonance, and the Society of Nuclear Medicine. Circulation 2009;119(22):e561-87. 3. Parker MW, Iskandar A, Limone B, Perugini A, Kim H, Jones C, et al. Diagnostic accuracy of cardiac positron emission tomography versus single photon emission computed tomography for coronary artery disease: A bivariate meta-analysis. Circ Cardiovasc Imaging 2012;5(6):700-7. 4. FOCUS Initiative, http://www.cardiosource.org/Science-AndQuality/Quality-Programs/Imaging-in-FOCUS/. Accessed August 10, 2012. 5. Winchester DE, Meral R, Ryals S, Beyth RJ, Shaw LJ. Appropriate use of myocardial perfusion imaging in a veteran population: Profit motives and professional liability concerns. JAMA Intern Med 2013;173(14):1381-3.
Winchester et al Clinical utility of inappropriate PET-MPI
15
6. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007;39(2):175-91. 7. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: Guidelines for reporting observational studies. Ann Intern Med 2007;147(8):573-7. 8. DeMuth KFJ, Hernandez M, Rodriguez J, Schwartz R, Lambiris I, Asher C, et al. Evaluation and Improvement of SPECT appropriateness criteria with clinical outcomes: A long-term follow-up study. J Am Coll Cardiol 2013;61(10):E989. 9. Doukky R, Hayes K, Frogge N, Balakrishnan G, Dontaraju VS, Rangel MO, et al. Impact of appropriate use on the prognostic value of single-photon emission computed tomography myocardial perfusion imaging. Circulation 2013;128(15):1634-43. 10. Aldweib N, Negishi K, Seicean S, Jaber WA, Hachamovitch R, Cerqueira M, et al. Appropriate test selection for single-photon emission computed tomography imaging: Association with clinical risk, posttest management, and outcomes. Am Heart J 2013;166(3): 581-8. 11. Khawaja FJ, Jouni H, Miller TD, Hodge DO, Gibbons RJ. Downstream clinical implications of abnormal myocardial perfusion single-photon emission computed tomography based on appropriate use criteria. J Nucl Cardiol 2013;20(6):1041-8. 12. Koh AS, Flores JL, Keng FY, Tan RS, Chua TS. Correlation between clinical outcomes and appropriateness grading for referral to myocardial perfusion imaging for preoperative evaluation prior to non-cardiac surgery. J Nucl Cardiol 2012;19(2):277-84. 13. Aljaroudi WA, Alraies MC, Wazni O, Cerqueira MD, Jaber WA. Yield and diagnostic value of stress myocardial perfusion imaging in patients without known coronary artery disease presenting with syncope. Circ Cardiovasc Imaging 2013;6(3):384-91. 14. Cortigiani L, Bigi R, Bovenzi F, Molinaro S, Picano E, Sicari R. Prognostic implication of appropriateness criteria for pharmacologic stress echocardiography performed in an outpatient clinic [corrected]. Circ Cardiovasc Imaging 2012;5(3):298-305. 15. Shaw LJ, Hage FG, Berman DS, Hachamovitch R, Iskandrian A. Prognosis in the era of comparative effectiveness research: Where is nuclear cardiology now and where should it be? J Nucl Cardiol 2012;19(5):1026-43. 16. Rao VM, Levin DC. The overuse of diagnostic imaging and the Choosing Wisely initiative. Ann Intern Med. 2012;157(8):574-6. 17. Saifi S, Taylor AJ, Allen J, Hendel R. The use of a learning community and online evaluation of utilization for SPECT myocardial perfusion imaging. JACC Cardiovasc Imaging 2013;6(7): 823-9. 18. Allen J. Abstract P231: A National Initiative to Improve Utilization of Cardiac Imaging: The FOCUS Learning Community and Performance Improvement Module. Circulation 2011;4(6 Supplement): AP231. 19. Apps4Docs. http://www.appsdocs.com/. Accessed May 21, 2013. 20. American College of Cardiology. FOCUS for Health Plans: Cardiovascular Imaging Strategies. http://www.cardiosource.org/ Science-And-Quality/Quality-Programs/Imaging-in-FOCUS/FOCUSfor-Health-Plans-and-Cardiovascular-Imaging-Strategies.aspx. Accessed May 21, 2013.
