Int J Cardiovasc Imaging DOI 10.1007/s10554-014-0582-x
ORIGINAL PAPER
The association of abnormal findings on transthoracic echocardiography with 2011 Appropriate Use Criteria and clinical impact Thomas P. Koshy • Anand Rohatgi • Sandeep R. Das • Angela L. Price • Andres deLuna • Nicholas Reimold • Kyle Willett • Sharon C. Reimold • Susan A. Matulevicius
Received: 29 September 2014 / Accepted: 18 December 2014 Ó Springer Science+Business Media Dordrecht 2015
Abstract Transthoracic echocardiography (TTE) Appropriate Use Criteria (AUC) were developed to promote high-value care. We describe the prevalence of clinically significant abnormal TTE findings overall and in subgroups defined by appropriate and inappropriate AUC, and their association with clinical impact. 548 consecutive TTEs at an academic medical center were retrospectively reviewed for AUC, clinical impact, and TTE abnormalities. TTE reports within 1 year of the index TTE were reviewed to determine if abnormalities were new, unchanged, or resolved. Clinical impact was classified into no change, active change, or continuation of care. 91 % of TTEs were appropriate, 5 % were inappropriate, and 4 % were uncertain by AUC. 46 % of all TTEs and 57 % of first-time TTEs had no significant TTE abnormalities. Appropriate TTEs had a higher prevalence of C1 TTE abnormality than inappropriate TTEs (56 vs. 33 %, p = 0.029). Among repeat TTEs, 72 % had C1 TTE abnormality, however only 25 % had a new abnormality. The prevalence of a new abnormality was similar between inappropriate and appropriate repeat TTEs (25 vs. 26 %, p = 1.0). The prevalence of C1 abnormality was similar between TTEs that resulted in active change and no change in care (70 vs.
T. P. Koshy A. Rohatgi S. R. Das A. L. Price A. deLuna S. C. Reimold S. A. Matulevicius (&) Department of Medicine, University of Texas Southwestern Medical Center, 5909 Harry Hines Boulevard, Dallas, TX 75390-9047, USA e-mail: [email protected] N. Reimold Northwestern University, Evanston, IL, USA K. Willett University of Texas Austin, Austin, TX, USA
64 %, p = 0.06). Although most TTEs were appropriate as defined by AUC, the majority had no significant abnormalities. Although most TTEs were appropriate by AUC, [50 % of all TTEs and 25 % of repeat TTEs had no significant abnormalities. Appropriate TTEs had a higher prevalence of abnormalities, however the prevalence of abnormalities was similar between TTEs that resulted in active change versus no change in care. Keywords Echocardiography Appropriate use criteria Health services Clinical decision-making Retrospective study
Introduction Health care expenditures on imaging have grown dramatically over the last few decades throughout the world. United States (US) Health care spending was nearly 2.1 trillion dollars in 2010, and health expenditures are expected to double in the next 30 years [1]. Spending for non-invasive imaging has grown faster than any other form of healthcare and represents over three quarters of costs in cardiology with echocardiography representing more than half of cardiovascular diagnostic imaging in the US [2, 3]. Similar increases in echocardiography utilization have been seen in Canada, where age- and sex-adjusted rates of echocardiography grew at an annual rate of 5.5 % from 39.1 per 1,000 persons in 2001 to 59.9 per 1,000 persons in 2009 [4]. In the US, 29 % of Medicare beneficiaries have had a TTE, with 31 % of those patients undergoing a repeat TTE within 1 year and 55 % within 3 years [5]. In Canada, repeat TTEs increased at a rate of 10.6 % per year and accounted for 25.3 % of all TTEs in 2009 as compared to 18.5 % in 2002 [4].
123
Int J Cardiovasc Imaging
Materials and methods
impact of each TTE through retrospective review of the complete electronic medical record (EMR). Clinical impact was empirically defined a priori into one of three mutually exclusive categories: (1) active change in care, (2) continuation of current care, or (3) no change in care as previously described [12]. Briefly, active change was assigned if there was any escalation or de-escalation of care resulting from the TTE including medication changes, subspecialty consultation, surgery/invasive procedures, further diagnostic testing, transfer to a different level of care, or cancellation of an initially planned procedure or intervention. Continuation of care was assigned if there was no escalation or de-escalation of current care but there was clear evidence of direct communication to the patients and/ or documentation of the TTE results in the chart by the providers. No change in care was assigned if neither of the above two categories could be satisfied and there was an absence of evidence that the TTE findings either confirmed or changed clinical care. Any disagreements in clinical impact were resolved by consensus. If consensus could not be reached, a third blinded non-invasive cardiologist (S.R.) reviewed the cases and made a final assignment of clinical impact.
Study population
TTE abnormal clinical findings determination
As previously described [12], all TTEs ordered from April 1st 2011 to April 30th 2011 at the University of Texas Southwestern Medical Center were retrospectively reviewed for AUC, clinical impact, and clinically significant abnormal TTE findings. A TTE was excluded from review if: (1) it was not performed, (2) there were no clinical data for review, or (3) it was performed post-cardiac transplant or post-left ventricular assist device. The study protocol and waiver of consent were approved by the UT Southwestern Institutional Review Board.
TTE data were abstracted from the TTE reports in the EMR. The TTE laboratory at UT Southwestern is an Intersocietal Accreditation Commission accredited echocardiography laboratory in which all TTEs are performed and interpreted in accordance with recommendations of the American Society of Echocardiography (ASE) by level II or III and/or board-certified echocardiographers [13, 14]. Quantitative methods (Teichholz or Simpson’s biplane or single plane method of discs) were used to assess left ventricular (LV) ejection fraction (EF). The interpreting physicians quantified valvular regurgitation severity through a variety of methods, including visual estimation, vena contracta measurement, effective regurgitant orifice area, and pressure-half time [15]. Valvular stenosis was assessed through Doppler determination of valvular gradients, aortic valve gradients, and Doppler velocity index calculation [16]. A noninvasive cardiologist (S.M.) blinded to the appropriateness adjudication oversaw the abstraction and determined the prevalence of clinically significant abnormalities for each TTE. Clinically significant abnormalities were defined as: (1) LVEF \ 45 %; (2) any wall motion abnormality; (3) Grade III diastolic dysfunction; (4) right ventricular (RV) dysfunction; (5) RV dilatation; (6) presence of an intracardiac shunt; (7) presence of a vegetation or cardiac mass; (8) moderate or greater valvular regurgitation; (9) moderate or greater valvular stenosis (valve area \ 1.5 cm2); (10) RV systolic pressure
Because of concerns about overutilization of echocardiography in the US [3], the American College of Cardiology Foundation in association with other imaging societies, including the American Society of Echocardiography (ASE), developed appropriateness use criteria (AUC) ‘‘to respond to the need for the rational use of imaging services in the delivery of high quality care’’ [6–8]. Some studies have suggested that AUC associate with prevalent abnormal echocardiographic findings, suggesting clinical utility [9– 11]. However, despite the fact that the majority of TTEs are appropriate, we found that less than a third of TTEs at our academic medical center resulted in active change in clinical care [12]. No prior study has examined the association between abnormal TTE findings and clinical impact. To help elucidate this important issue, we describe the prevalence of clinically significant abnormal TTE findings overall and in subgroups defined by appropriate and inappropriate AUC as well as describe the association between clinically significant abnormal TTE findings and clinical impact in our cohort.
AUC adjudication Two independent general cardiologists (A.D., A.P.) reviewed the electronic medical record (EMR) dated prior to the TTE being performed and classified each TTE according to the 2011 AUC as previously described [6, 12]. The reviewers were blinded to the results of the TTE and to the clinical course subsequent to the TTE. Disagreements in AUC classification were resolved by consensus. Cases where consensus could not be reached underwent definitive adjudication by a third cardiologist (S.D.) blinded as above. Clinical impact adjudication Two noninvasive cardiologists (S.M., A.R.), blinded to AUC classification, independently assessed the clinical
123
Int J Cardiovasc Imaging
[40 mmHg above right atrial pressure; (11) dilated aorta; (12) aortic coarctation; (13) moderate or greater pericardial effusion; or (14) presence of pericardial disease, in accordance with previously defined clinically significant TTE abnormalities in the literature [9, 11]. Prior TTE reports in the EMR within the past year from the initial TTE were reviewed to determine if abnormal findings were new, unchanged, or if a prior abnormal finding had resolved. Statistical analysis Agreement between reviewers in appropriateness grading after consensus was attempted was assessed with weighted kappa analysis. The proportions of TTEs with C1 clinically significant TTE abnormality were calculated for the overall cohort, by AUC grade, and by clinical impact. Differences in frequency distributions were statistically compared using the Chi square test and differences in continuous variables were compared using ANOVA. All reported p values are 2-tailed with p values \ 0.05 considered statistically significant. All statistical analyses were performed with commercially available software (SAS, version 9.2; SAS Institute Inc., Research Triangle Park, NC, USA).
Results Overall cohort A total of 617 TTEs were reviewed and 548 were included in this study. AUC was unable to be classified for 1 subject, there was insufficient clinical data in the EMR to assess clinical impact in 3 subjects, and 65 subjects had an LVAD or were post-cardiac transplant and were therefore excluded from final analysis. The study population was 59 % female, 55 % Caucasian, 21 % African American and 8 % Hispanic. Internists (38 %) and cardiologists (31 %) ordered the majority of TTEs (Table 1). Of the TTEs reviewed, 58 % had no prior TTE, 55 % were inpatient TTEs, and arbitration for AUC assignment by a third cardiologist was required for 24 (3.9 %, weighted kappa 0.8, p \ 0.0001). AUC classification Based on 2011 AUC, 91 % (n = 498) of TTEs were appropriate, 5 % (n = 27) were inappropriate and 4 % (n = 23) uncertain. There was no statistically significant difference in appropriateness by ordering provider specialty (p = 0.78). Although the majority of TTEs were appropriate, fewer outpatient TTEs were appropriate compared with inpatient TTEs (85 vs. 96 %, p \ 0.0001).
The ten most frequent AUCs, all classified as appropriate, accounted for 66 % of all TTEs (Table 2). Clinically significant abnormal TTE findings Of 548 TTEs reviewed, 54 % had C1 clinically significant TTE abnormality. The most common abnormalities were wall motion abnormalities (25 %), elevated RVSP [ 40 mmHg (20 %), moderate or greater valvular regurgitation (15 %), and grade III diastolic dysfunction (14 %; Table 3). There was no difference in the prevalence of C1 clinically significant TTE abnormality among outpatient versus inpatient TTEs (50 vs. 58 %, p = 0.07).There was a higher prevalence of C1 clinically significant TTE abnormality among appropriate versus inappropriate TTEs (56 vs. 33 %, p = 0.029). There was a non-statistically significant trend towards a higher prevalence of C1 clinically significant TTE abnormality among appropriate versus inappropriate TTE in inpatients (59 vs. 27 %, p = 0.058) and outpatients (51 vs. 38 %, p = 0.31). Repeat TTEs (n = 232) had increased proportion of C1 clinically significant TTE abnormality compared with firsttime (n = 316) TTEs (70 vs. 43 %, p \ 0.0001). There was a higher prevalence of C1 clinically significant TTE abnormality among appropriate versus inappropriate repeat TTEs (72 vs. 42 %, p = 0.03). No differences were seen for appropriate versus inappropriate first-time TTEs (44 vs. 27 %, p = 0.09), however the number of inappropriate TTEs was small in both groups (first-time TTEs n = 15, repeat TTEs n = 12). Among repeat TTEs, a new clinically significant TTE abnormality was found in only 25 % and resolution of a previous TTE abnormality was found in 10 %. The prevalence of a new clinically significant TTE abnormality was similar between appropriate and inappropriate repeat TTEs (26 vs. 25 %, p = 1.0). The prevalence of clinically significant TTE abnormalities among individual appropriate AUCs varied significantly, with some appropriate AUCs being associated with a high prevalence (C2/3) of abnormality (AUC #73, #5, #15, #18, #19) while other appropriate AUCs were associated with a low prevalence (\1/2) of abnormality (AUC #1, #34, #91; Table 3). Similarly, some inappropriate AUCs were associated with a high prevalence of abnormality (C2/3; AUC #16, #74) while others were associated with no abnormal findings (#10, #53, #68; Table 2). Clinical impact The prevalence of C1 clinically significant TTE abnormality was lower in TTEs that led to continuation of current care than TTEs that led to no change or active change (continuation of care vs. no change vs. active change: 40 vs. 64 vs. 70 %). The prevalence of C1 clinically
123
Int J Cardiovasc Imaging Table 1 Cohort characteristics Overall cohort (n = 548)
Normal echo (n = 250)
C1 Echo abnormality (n = 298)
Age (years)
58 ± 17
56 ± 17
60 ± 18
Male (%)
41
31
49
Female (%)
59
69
51
Inpatient (%)
55
49
59
Outpatient (%)
45
51
41
White (%)
55
53
57
Hispanic (%)
8
9
8
Black (%)
21
21
21
Other or unknown (%)
15
17
15
Cardiology (%) Pulmonary/critical care (%)
31 15
26 8
36 20
Internal medicine (%)
38
49
29
Surgery (%)
10
10
10
Ordering provider
Neurology (%)
3
6
2
Other (%)
3
2
3
significant TTE abnormality was not statistically significantly different between TTEs that lead to no change versus active change in care (64 vs. 70 %, p = 0.06). Similarly, there was no statistically significant difference between the proportion of active change versus no change in TTEs with C1 clinically significant TTE abnormality (active change vs. no change: 40 vs. 24 %, p = 0.30).
Discussion Although echocardiography is an extremely helpful clinical tool, there is growing concern regarding its overuse as a diagnostic modality. Our study found that nearly half (46 %) of all TTEs and over half (57 %) of first-time TTEs performed had no clinically significant TTE abnormalities and were therefore essentially normal studies. Among repeat TTEs, only a minority had a new clinically significant abnormality. Although appropriate TTEs had a higher prevalence of C1 clinically significant TTE abnormality, the prevalence of abnormal TTE findings was similar between TTEs that resulted in active change versus no change suggesting that clinical decision-making is driven by a mixture of patient, center, and physician factors rather than by TTE alone. Few TTEs were inappropriate by AUC but a majority of TTEs had no clinically significant findings suggesting AUC is overly broad and that TTEs are being over-utilized. In our cohort, 91 % of TTEs were appropriate, 5 % were inappropriate, and 4 % were uncertain similar to recently published data of 92 % appropriate, 2 % inappropriate,
123
\1 % uncertain, 5.5 % unclassifiable [17]. Other studies report some variation in the proportion of appropriate TTEs, which may in part be due to the granularity of the data in the clinical chart to determine AUC, differences in patient acuity, differences in evaluation of appropriateness in each study, and timing relative to publication of AUC criteria [10, 18–20]. The prevalence for appropriate TTEs ranged from 77 to 82 % in these studies, suggesting that with dissemination of the guidelines and incorporation of the AUC into practice, appropriateness may have increased. From an international perspective, when AUC were applied in the United Kingdom and in Italy, similar results were found with 86 % of TTEs performed in Wales [21] and 80 % of TTEs performed in Italian centers [18] were classified as appropriate. In all studies, the majority of TTEs are appropriate which is in agreement with our current findings. Despite the fact that over 90 % of all TTEs were appropriate in our study, only about had abnormal TTE findings. Appropriate TTEs were associated with a higher prevalence of abnormal findings compared with inappropriate TTEs (51 vs. 38 %, p = 0.013). These findings are similar to a recent study by Bhatia et al. [9] that found that appropriate TTEs had a higher prevalence of TTE abnormalities than inappropriate or uncertain studies (42 vs. 24 vs. 18 %, p = 0.08). In terms of repeat TTEs, a previous study by Ghatak et al. [22] evaluated the appropriateness and incidence of new TTE findings in a community teaching inpatient hospital setting and showed that 56 % of repeat TTEs performed within 1 year of the index TTE (n = 104) had a new TTE abnormality defined as a 20 %
Int J Cardiovasc Imaging Table 2 The most frequent appropriate use criteria and their association with clinically significant abnormal transthoracic echocardiogram abnormalities AUC
AUC description
C1 Echo abnormality (%)
No. of TTEs
36
151
Most frequent appropriate AUCs 1.
Symptoms or conditions potentially related to suspected cardiac etiology including but not limited to chest pain, shortness of breath, palpitations, transient ischemic attack, stroke, or peripheral embolic event
73.
Re-evaluation of known heart failure to guide therapy
88
34
2.
Prior testing that is concerning for heart disease or structural abnormality including but not limited to chest X-ray, scout images of stress echocardiogram, electrocardiogram, cardiac biomarkers
54
28
5.
Sustained or non-sustained atrial fibrillation, supraventricular tachycardia, or ventricular tachycardia
67
27
91. 15.
Baseline and serial re-evaluations in a patient undergoing therapy with cardiotoxic agents Evaluation of suspected pulmonary hypertension including right ventricular function and pulmonary artery pressure estimation
33 78
24 23
34.
Initial evaluation when there is a reasonable suspicion of valvular or structural heart disease
41
22
18.
Re-evaluation of known pulmonary hypertension if change in clinical status or cardiac exam or to guide therapy
85
20
19.
Hypotension or hemodynamic instability of uncertain or suspected cardiac etiology
80
20
52.
Initial evaluation of suspected infective endocarditis with positive blood culture or a murmur
60
15
Most frequent inappropriate AUCs 13.
Routine perioperative evaluation of ventricular function with no symptoms or signs of cardiovascular disease
25
4
16.
Routine surveillance (\1 year) of known pulmonary hypertension without change in clinical status or cardiac exam
67
3
74.
Routine surveillance (\1 year) of HF (systolic or diastolic) when there is no change in clinical status or cardiac exam
67
3
6. 10.
Asymptomatic isolated sinus bradycardia Initial evaluation of ventricular function (e.g., screening) with no symptoms or signs of cardiovascular disease
50 0
2 2
53.
Transient fever without evidence of bacteremia or a new murmur
0
2
68.
Routine evaluation of systemic hypertension without symptoms or signs of hypertensive heart disease
0
2
change in LVEF, change in wall motion, new valvular regurgitation that increased by one grade, or a 20 % change in pulmonary pressures. In our study, 42 % of patients had repeat TTEs within 1 year, of whom 25 % had a new clinically significant TTE finding on the repeat TTE, 10 % had resolution of a previously demonstrated TTE abnormality, and 65 % had no new TTE findings. Our lower prevalence of new TTE abnormalities in repeat TTEs may be partially due to our mix of both inpatient and outpatient TTEs compared to the exclusively inpatient TTEs in the previous study as well as our stricter definition of a clinically significant TTE abnormality. Another study [11] found that new clinically important TTE findings were found in 32 % of their cohort and that this did not differ between appropriate and inappropriate TTEs. We found that appropriate and inappropriate TTEs had similar prevalence of new major TTE abnormalities (26 vs. 25 %) in repeat TTEs, however the number of inappropriate repeat TTEs was small (n = 12) limiting our power to demonstrate a statistically significant difference. In either case,
the yield of new diagnostic information from these repeat TTEs within 1 year is low. Factors besides TTE abnormalities likely drive the decision to actively respond to TTE findings and need to be further investigated. Abnormal TTE findings have been suggested as a surrogate endpoint to assess the clinical utility of AUC [23]. Indeed, our study found appropriate TTEs had a higher prevalence of C1 clinically significant TTE abnormality than inappropriate TTEs. However, the definitions of appropriate AUC often are specific to a TTE-defined disease process, resulting in a high pre-test probability of a TTE abnormality while inappropriate AUC definitions often result in a low pre-test probability of a TTE abnormality. For example, indication #73, ‘‘re-evaluation of known heart failure to guide therapy’’ has a prevalence of C1 major TTE abnormality of 91 % since these patients had to have known heart failure to meet this criterion. In contrast, inappropriate indication #10, ‘‘Initial evaluation of ventricular function (e.g., screening) with no symptoms or signs of cardiovascular disease’’ selects for patients with
123
Int J Cardiovasc Imaging Table 3 Prevalence of transthoracic echocardiogram abnormalities Echo abnormality
All [(n = 548) (%)]
Appropriate [(n = 498) (%)]
Inappropriate [(n = 27) (%)]
LVEF \ 45 %
16
16
11
Wall motion abnormalities
25
25
22
Grade III diastolic dysfunction
14
15
4
RVSP [ 40 mmHg
20
20
7
Dilated RV
14
14
11
RV dysfunction
10
10
4
Cardiac shunt
3
3
4
Vegetation or cardiac mass
2
2
0
CModerate valvular regurgitation
15
15
4
CModerate valvular stenosis
3
3
0
Dilated aorta Aortic coarctation
2 \1
2 \1
4 0
CModerate pericardial effusion
2
2
0
Pericardial disease
\1
\1
0
LVEF left ventricular ejection fraction, RV right ventricle, RVSP right ventricular systolic pressure
a very low pre-test probability of having a cardiac structural or functional abnormality. Patients meeting this criterion had a 0 % prevalence of C1 major TTE abnormality in our cohort. As the definitions of an individual AUC becomes less specific (ex. appropriate AUC #1, ‘‘Symptoms or conditions potentially related to suspected cardiac etiology including but not limited to chest pain, shortness of breath, palpitations, transient ischemic attack, stroke, or peripheral embolic event’’), the prevalence of C1 major TTE abnormality decreases (36 %). Therefore, prevalence of TTE abnormalities track with the language of AUC definitions and will likely be of limited utility as a metric for AUC clinical effectiveness. Similarly, in inappropriate AUCs that have a high pre-test probability of a TTE abnormality, like # 74, ‘‘Routine surveillance (\1 year) of HF (systolic or diastolic) when there is no change in clinical status or cardiac exam’’, the prevalence of C1 major TTE abnormality was also high (67 %). Moreover, even though a bias introduced by potentially leading definitions of AUC would magnify the association between appropriateness and abnormal findings, our overall prevalence of clinically significant echocardiographic abnormalities of 54 % was low. Defining an appropriate metric to assess the efficacy of noninvasive imaging in the provision of care is a difficult task. In the 1990s, Dr. Fryback and Thornbury [24] described a 6-tiered hierarchal framework for assessing technology efficacy, which was recently endorsed by the American College of Cardiology, in collaboration with multiple subspecialty societies, in a health policy statement on noninvasive imaging use [25]. In this framework, efficacy is assessed by considering the technical efficacy
123
(spatial resolution, imaging reproducibility), diagnostic efficacy (sensitivity, specificity, accuracy), diagnostic thinking efficacy (incremental value of testing information on diagnosis), therapeutic efficacy (changes in patient management related to testing), patient outcome efficacy (safety concerns, improved patient outcomes, including quality of life, morbidity and mortality as a result of testing), and society efficacy (cost versus overall societal benefit of testing) of testing in clinical care. The detection of TTE abnormalities according to AUC classification would be consistent with level 2 (diagnostic efficacy) of this framework) while trying to assess clinical impact of testing is consistent with level 4. A prime tenet of the RAND process that was used to develop the AUC is that the assignment of appropriateness needs to be formally tested to see if it led to improved outcomes and modified if it did not [26]. For TTE AUC, this would require validating if criteria rated as appropriate accomplish the ultimate objective of impacting patient care. Developing metrics that not only capture diagnostic efficacy but attempt to assess the higher levels of the technology efficacy framework are needed in order to maximize the societal benefit of TTE or any diagnostic testing modality. There are several limitations to this study. This study relies on retrospective chart review, which may have led to misclassification of AUC. The performance of this study in a tertiary care academic institution may limit its generalizability to other patient care settings. Without clinical outcomes, the value of TTE findings on patient care cannot be determined. The low prevalence of inappropriate TTEs limits statistical power to detect a difference in the clinically significant abnormal TTE findings between
Int J Cardiovasc Imaging
appropriate and inappropriate TTEs in subgroups; however the low overall prevalence of clinically significant abnormal findings suggests that efforts to further maximize the utility of TTE are needed. Finally, clinical decision-making is a dynamic process that involves interactions between clinicians, patients and external factors. Our study attempts to place this dynamic process into categorical groups which can be challenging in the setting of an EMR where we are limited to information that was documented.
Conclusions Appropriate TTEs are associated with a higher prevalence of clinically significant TTE abnormalities than inappropriate TTEs, in part due to the AUC definitions. TTEs that actively impact care had a similar prevalence of abnormal TTE findings compared with TTEs that led to no change in care, suggesting limited clinical utility in using abnormal findings to assess the clinical effectiveness of TTE. Development and validation of metrics reflecting improved therapeutic efficacy and patient outcomes may allow for targeted efforts that can maximize diagnostic and therapeutic yield of TTE. Acknowledgments Supported in part by grant UL1TR000451 from the UT-STAR, National Center for Advancing Translational Sciences, National Institutes of Health. Susan Matulevicius, M.D. had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. The content is solely the responsibility of the authors and does not necessarily represent the official views of UT-STAR, UT Southwestern Medical Center and its affiliated academic and health care centers, the National Center for Advancing Translational Sciences, or the National Institutes of Health. Conflict of interest
None.
References 1. Medicare Payment Advisory Commission (2012) A Data Book: health care spending and the medicare program. Medicare Payment Advisory Commission, Washington, DC 2. Pearlman AS, Ryan T, Picard MH, Douglas PS (2007) Evolving trends in the use of echocardiography: a study of medicare beneficiaries. J Am Coll Cardiol 49(23):2283–2291. doi:10.1016/j. jacc.2007.02.048 3. Andrus BW, Welch HG (2012) Medicare services provided by cardiologists in the United States: 1999–2008. Circ Cardiovasc Qual Outcomes 5(1):31–36. doi:10.1161/CIRCOUTCOMES.111. 961813 4. Blecker S, Bhatia RS, You JJ, Lee DS, Alter DA, Wang JT, Wong HJ, Tu JV (2013) Temporal trends in the utilization of echocardiography in Ontario, 2001 to 2009. JACC Cardiovasc Imaging 6(4):515–522. doi:10.1016/j.jcmg.2012.10.026 5. Welch HHKJFC (2012) Repeat testing among medicare beneficiaries. Arch Intern Med 172(22):1745–1751. doi:10.1001/2013. jamainternmed.727
6. Douglas PS, Garcia MJ, Haines DE, Lai WW, Manning WJ, Patel AR, Picard MH, Polk DM, Ragosta M, Parker Ward R, Weiner RB (2011) ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/ SCCT/SCMR 2011 appropriate use criteria for echocardiography. J Am Soc Echocardiogr 24(3):229–267. doi:10.1016/j.echo.2010. 12.008 7. Douglas PS, Khandheria B, Stainback RF, Weissman NJ, Brindis RG, Patel MR, Alpert JS, Fitzgerald D, Heidenreich P, Martin ET, Messer JV, Miller AB, Picard MH, Raggi P, Reed KD, Rumsfeld JS, Steimle AE, Tonkovic R, Vijayaraghavan K, Yeon SB, Hendel RC, Peterson E, Wolk MJ, Allen JM (2007) ACCF/ ASE/ACEP/ASNC/SCAI/SCCT/SCMR 2007 appropriateness criteria for transthoracic and transesophageal echocardiography: a report of the American College of Cardiology Foundation Quality Strategic Directions Committee Appropriateness Criteria Working Group, American Society of Echocardiography, American College of Emergency Physicians, American Society of Nuclear Cardiology, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, and the Society for Cardiovascular Magnetic Resonance endorsed by the American College of Chest Physicians and the Society of Critical Care Medicine. J Am Coll Cardiol 50(2):187–204. doi:10.1016/j.jacc.2007.05.003 8. Patel MR, Spertus JA, Brindis RG, Hendel RC, Douglas PS, Peterson ED, Wolk MJ, Allen JM, Raskin IE (2005) ACCF proposed method for evaluating the appropriateness of cardiovascular imaging. J Am Coll Cardiol 46(8):1606–1613. doi:10. 1016/j.jacc.2005.08.030 9. Bhatia RS, Carne DM, Picard MH, Weiner RB (2012) Comparison of the 2007 and 2011 appropriate use criteria for transthoracic echocardiography in various clinical settings. J Am Soc Echocardiogr 25(11):1162–1169. doi:10.1016/j.echo. 2012.07.018 10. Mansour IN, Razi RR, Bhave NM, Ward RP (2012) Comparison of the updated 2011 appropriate use criteria for echocardiography to the original criteria for transthoracic, transesophageal, and stress echocardiography. J Am Soc Echocardiogr 25(11):1153– 1161. doi:10.1016/j.echo.2012.08.008 11. Kirkpatrick JN, Ky B, Rahmouni HW, Chirinos JA, Farmer SA, Fields AV, Ogbara J, Eberman KM, Ferrari VA, Silvestry FE, Keane MG, Opotowsky AR, Sutton MS, Wiegers SE (2009) Application of appropriateness criteria in outpatient transthoracic echocardiography. J Am Soc Echocardiogr 22(1):53–59. doi:10. 1016/j.echo.2008.10.020 12. Matulevicius SA, Rohatgi A, Das SR, Price AL, Deluna A, Reimold SC (2013) Appropriate use and clinical impact of transthoracic echocardiography. JAMA Intern Med 173(17): 1600–1607. doi:10.1001/jamainternmed.2013.8972 13. Lang RM, Bierig M, Devereux RB, Flachskampf FA, Foster E, Pellikka PA, Picard MH, Roman MJ, Seward J, Shanewise JS, Solomon SD, Spencer KT, Sutton MS, Stewart WJ, Chamber Quantification Writing G, American Society of Echocardiography’s G, Standards C, European Association of E (2005) Recommendations for chamber quantification: a report from the American Society of Echocardiography’s Guidelines and standards committee and the chamber quantification writing group, developed in conjunction with the European Association of Echocardiography, a branch of the European Society of Cardiology. J Am Soc Echocardiogr 18(12):1440–1463. doi:10.1016/j. echo.2005.10.005 14. Picard MH, Adams D, Bierig SM, Dent JM, Douglas PS, Gillam LD, Keller AM, Malenka DJ, Masoudi FA, McCulloch M, Pellikka PA, Peters PJ, Stainback RF, Strachan GM, Zoghbi WA (2011) American Society of Echocardiography recommendations for quality echocardiography laboratory operations. J Am Soc Echocardiogr 24(1):1–10. doi:10.1016/j.echo.2010.11.006
123
Int J Cardiovasc Imaging 15. Zoghbi WA, Enriquez-Sarano M, Foster E, Grayburn PA, Kraft CD, Levine RA, Nihoyannopoulos P, Otto CM, Quinones MA, Rakowski H, Stewart WJ, Waggoner A, Weissman NJ, American Society of E (2003) Recommendations for evaluation of the severity of native valvular regurgitation with two-dimensional and Doppler echocardiography. J Am Soc Echocardiogr 16(7):777–802. doi:10.1016/S0894-7317(03)00335-3 16. Baumgartner H, Hung J, Bermejo J, Chambers JB, Evangelista A, Griffin BP, Iung B, Otto CM, Pellikka PA, Quinones M, American Society of E, European Association of E (2009) Echocardiographic assessment of valve stenosis: EAE/ASE recommendations for clinical practice. J Am Soc Echocardiogr 22(1):1–23. doi:10.1016/ j.echo.2008.11.029 quiz 101–102 17. Parikh PB, Asheld J, Kort S (2012) Does the revised appropriate use criteria for echocardiography represent an improvement over the initial criteria? A comparison between the 2011 and the 2007 appropriateness use criteria for echocardiography. J Am Soc Echocardiogr 25(2):228–233. doi:10.1016/j.echo.2011.09.017 18. Ballo P, Bandini F, Capecchi I, Chiodi L, Ferro G, Fortini A, Giuliani G, Landini G, Laureano R, Milli M, Nenci G, Pizzarelli F, Santoro GM, Vannelli P, Cappelletti C, Zuppiroli A (2012) Application of 2011 American College of Cardiology Foundation/American Society of Echocardiography appropriateness use criteria in hospitalized patients referred for transthoracic echocardiography in a community setting. J Am Soc Echocardiogr 25(6):589–598. doi:10.1016/j.echo.2012.03.006 19. Alqarqaz M, Koneru J, Mahan M, Ananthasubramaniam K (2012) Applicability, limitations and downstream impact of echocardiography utilization based on the appropriateness use criteria for transthoracic and transesophageal echocardiography. Int J Cardiovasc Imaging 28(8):1951–1958. doi:10.1007/s10554012-0008-6 20. Willens HJ, Hendel RC, Inhaber FR, Chakko SC, Postel C, Hasan T, Mohammed F (2011) Appropriateness use criteria for
123
21.
22.
23.
24. 25.
26.
transthoracic echocardiography: relationship with radiology benefit managers preauthorization determination and comparison of the new (2010) criteria to the original (2007) criteria. Am Heart J 162(4):772–779. doi:10.1016/j.ahj.2011.07.009 Gurzun MM, Ionescu A (2014) Appropriateness of use criteria for transthoracic echocardiography: are they relevant outside the USA? Eur Heart J Cardiovasc Imaging 15(4):450–455. doi:10. 1093/ehjci/jet186 Ghatak A, Pullatt R, Vyse S, Silverman DI (2011) Appropriateness criteria are an imprecise measure for repeat echocardiograms. Echocardiography 28(2):131–135. doi:10.1111/j.15408175.2010.01302.x Dudzinski DM, Bhatia RS, Weiner RB (2014) Assessing the clinical impact of appropriate echocardiograms. JAMA Intern Med 174(7):1195–1196. doi:10.1001/jamainternmed.2014.1440 Fryback DG, Thornbury JR (1991) The efficacy of diagnostic imaging. Med Decis Making 11(2):88–94 Mark DB, Anderson JL, Brinker JA, Brophy JA, Casey DE Jr, Cross RR, Edmundowicz D, Hachamovitch R, Hlatky MA, Jacobs JE, Jaskie S, Kett KG, Malhotra V, Masoudi FA, McConnell MV, Rubin GD, Shaw LJ, Sherman ME, Stanko S, Ward RP (2014) ACC/AHA/ASE/ASNC/HRS/IAC/Mended Hearts/NASCI/RSNA/SAIP/SCAI/SCCT/SCMR/SNMMI 2014 health policy statement on use of noninvasive cardiovascular imaging: a report of the American College of Cardiology Clinical Quality Committee. J Am Coll Cardiol 63(7):698–721. doi:10. 1016/j.jacc.2013.02.002 Fitch K, Bernstein SJ, Aguilar MS, Burnand B, LaCalle JR, Lazaro P, van het Loo M, McDonnell J, Vader J, Kahan JP (2001) The RAND/UCLA appropriateness method user’s manual. RAND Corporation, Santa Monica, CA
ORIGINAL PAPER
The association of abnormal findings on transthoracic echocardiography with 2011 Appropriate Use Criteria and clinical impact Thomas P. Koshy • Anand Rohatgi • Sandeep R. Das • Angela L. Price • Andres deLuna • Nicholas Reimold • Kyle Willett • Sharon C. Reimold • Susan A. Matulevicius
Received: 29 September 2014 / Accepted: 18 December 2014 Ó Springer Science+Business Media Dordrecht 2015
Abstract Transthoracic echocardiography (TTE) Appropriate Use Criteria (AUC) were developed to promote high-value care. We describe the prevalence of clinically significant abnormal TTE findings overall and in subgroups defined by appropriate and inappropriate AUC, and their association with clinical impact. 548 consecutive TTEs at an academic medical center were retrospectively reviewed for AUC, clinical impact, and TTE abnormalities. TTE reports within 1 year of the index TTE were reviewed to determine if abnormalities were new, unchanged, or resolved. Clinical impact was classified into no change, active change, or continuation of care. 91 % of TTEs were appropriate, 5 % were inappropriate, and 4 % were uncertain by AUC. 46 % of all TTEs and 57 % of first-time TTEs had no significant TTE abnormalities. Appropriate TTEs had a higher prevalence of C1 TTE abnormality than inappropriate TTEs (56 vs. 33 %, p = 0.029). Among repeat TTEs, 72 % had C1 TTE abnormality, however only 25 % had a new abnormality. The prevalence of a new abnormality was similar between inappropriate and appropriate repeat TTEs (25 vs. 26 %, p = 1.0). The prevalence of C1 abnormality was similar between TTEs that resulted in active change and no change in care (70 vs.
T. P. Koshy A. Rohatgi S. R. Das A. L. Price A. deLuna S. C. Reimold S. A. Matulevicius (&) Department of Medicine, University of Texas Southwestern Medical Center, 5909 Harry Hines Boulevard, Dallas, TX 75390-9047, USA e-mail: [email protected] N. Reimold Northwestern University, Evanston, IL, USA K. Willett University of Texas Austin, Austin, TX, USA
64 %, p = 0.06). Although most TTEs were appropriate as defined by AUC, the majority had no significant abnormalities. Although most TTEs were appropriate by AUC, [50 % of all TTEs and 25 % of repeat TTEs had no significant abnormalities. Appropriate TTEs had a higher prevalence of abnormalities, however the prevalence of abnormalities was similar between TTEs that resulted in active change versus no change in care. Keywords Echocardiography Appropriate use criteria Health services Clinical decision-making Retrospective study
Introduction Health care expenditures on imaging have grown dramatically over the last few decades throughout the world. United States (US) Health care spending was nearly 2.1 trillion dollars in 2010, and health expenditures are expected to double in the next 30 years [1]. Spending for non-invasive imaging has grown faster than any other form of healthcare and represents over three quarters of costs in cardiology with echocardiography representing more than half of cardiovascular diagnostic imaging in the US [2, 3]. Similar increases in echocardiography utilization have been seen in Canada, where age- and sex-adjusted rates of echocardiography grew at an annual rate of 5.5 % from 39.1 per 1,000 persons in 2001 to 59.9 per 1,000 persons in 2009 [4]. In the US, 29 % of Medicare beneficiaries have had a TTE, with 31 % of those patients undergoing a repeat TTE within 1 year and 55 % within 3 years [5]. In Canada, repeat TTEs increased at a rate of 10.6 % per year and accounted for 25.3 % of all TTEs in 2009 as compared to 18.5 % in 2002 [4].
123
Int J Cardiovasc Imaging
Materials and methods
impact of each TTE through retrospective review of the complete electronic medical record (EMR). Clinical impact was empirically defined a priori into one of three mutually exclusive categories: (1) active change in care, (2) continuation of current care, or (3) no change in care as previously described [12]. Briefly, active change was assigned if there was any escalation or de-escalation of care resulting from the TTE including medication changes, subspecialty consultation, surgery/invasive procedures, further diagnostic testing, transfer to a different level of care, or cancellation of an initially planned procedure or intervention. Continuation of care was assigned if there was no escalation or de-escalation of current care but there was clear evidence of direct communication to the patients and/ or documentation of the TTE results in the chart by the providers. No change in care was assigned if neither of the above two categories could be satisfied and there was an absence of evidence that the TTE findings either confirmed or changed clinical care. Any disagreements in clinical impact were resolved by consensus. If consensus could not be reached, a third blinded non-invasive cardiologist (S.R.) reviewed the cases and made a final assignment of clinical impact.
Study population
TTE abnormal clinical findings determination
As previously described [12], all TTEs ordered from April 1st 2011 to April 30th 2011 at the University of Texas Southwestern Medical Center were retrospectively reviewed for AUC, clinical impact, and clinically significant abnormal TTE findings. A TTE was excluded from review if: (1) it was not performed, (2) there were no clinical data for review, or (3) it was performed post-cardiac transplant or post-left ventricular assist device. The study protocol and waiver of consent were approved by the UT Southwestern Institutional Review Board.
TTE data were abstracted from the TTE reports in the EMR. The TTE laboratory at UT Southwestern is an Intersocietal Accreditation Commission accredited echocardiography laboratory in which all TTEs are performed and interpreted in accordance with recommendations of the American Society of Echocardiography (ASE) by level II or III and/or board-certified echocardiographers [13, 14]. Quantitative methods (Teichholz or Simpson’s biplane or single plane method of discs) were used to assess left ventricular (LV) ejection fraction (EF). The interpreting physicians quantified valvular regurgitation severity through a variety of methods, including visual estimation, vena contracta measurement, effective regurgitant orifice area, and pressure-half time [15]. Valvular stenosis was assessed through Doppler determination of valvular gradients, aortic valve gradients, and Doppler velocity index calculation [16]. A noninvasive cardiologist (S.M.) blinded to the appropriateness adjudication oversaw the abstraction and determined the prevalence of clinically significant abnormalities for each TTE. Clinically significant abnormalities were defined as: (1) LVEF \ 45 %; (2) any wall motion abnormality; (3) Grade III diastolic dysfunction; (4) right ventricular (RV) dysfunction; (5) RV dilatation; (6) presence of an intracardiac shunt; (7) presence of a vegetation or cardiac mass; (8) moderate or greater valvular regurgitation; (9) moderate or greater valvular stenosis (valve area \ 1.5 cm2); (10) RV systolic pressure
Because of concerns about overutilization of echocardiography in the US [3], the American College of Cardiology Foundation in association with other imaging societies, including the American Society of Echocardiography (ASE), developed appropriateness use criteria (AUC) ‘‘to respond to the need for the rational use of imaging services in the delivery of high quality care’’ [6–8]. Some studies have suggested that AUC associate with prevalent abnormal echocardiographic findings, suggesting clinical utility [9– 11]. However, despite the fact that the majority of TTEs are appropriate, we found that less than a third of TTEs at our academic medical center resulted in active change in clinical care [12]. No prior study has examined the association between abnormal TTE findings and clinical impact. To help elucidate this important issue, we describe the prevalence of clinically significant abnormal TTE findings overall and in subgroups defined by appropriate and inappropriate AUC as well as describe the association between clinically significant abnormal TTE findings and clinical impact in our cohort.
AUC adjudication Two independent general cardiologists (A.D., A.P.) reviewed the electronic medical record (EMR) dated prior to the TTE being performed and classified each TTE according to the 2011 AUC as previously described [6, 12]. The reviewers were blinded to the results of the TTE and to the clinical course subsequent to the TTE. Disagreements in AUC classification were resolved by consensus. Cases where consensus could not be reached underwent definitive adjudication by a third cardiologist (S.D.) blinded as above. Clinical impact adjudication Two noninvasive cardiologists (S.M., A.R.), blinded to AUC classification, independently assessed the clinical
123
Int J Cardiovasc Imaging
[40 mmHg above right atrial pressure; (11) dilated aorta; (12) aortic coarctation; (13) moderate or greater pericardial effusion; or (14) presence of pericardial disease, in accordance with previously defined clinically significant TTE abnormalities in the literature [9, 11]. Prior TTE reports in the EMR within the past year from the initial TTE were reviewed to determine if abnormal findings were new, unchanged, or if a prior abnormal finding had resolved. Statistical analysis Agreement between reviewers in appropriateness grading after consensus was attempted was assessed with weighted kappa analysis. The proportions of TTEs with C1 clinically significant TTE abnormality were calculated for the overall cohort, by AUC grade, and by clinical impact. Differences in frequency distributions were statistically compared using the Chi square test and differences in continuous variables were compared using ANOVA. All reported p values are 2-tailed with p values \ 0.05 considered statistically significant. All statistical analyses were performed with commercially available software (SAS, version 9.2; SAS Institute Inc., Research Triangle Park, NC, USA).
Results Overall cohort A total of 617 TTEs were reviewed and 548 were included in this study. AUC was unable to be classified for 1 subject, there was insufficient clinical data in the EMR to assess clinical impact in 3 subjects, and 65 subjects had an LVAD or were post-cardiac transplant and were therefore excluded from final analysis. The study population was 59 % female, 55 % Caucasian, 21 % African American and 8 % Hispanic. Internists (38 %) and cardiologists (31 %) ordered the majority of TTEs (Table 1). Of the TTEs reviewed, 58 % had no prior TTE, 55 % were inpatient TTEs, and arbitration for AUC assignment by a third cardiologist was required for 24 (3.9 %, weighted kappa 0.8, p \ 0.0001). AUC classification Based on 2011 AUC, 91 % (n = 498) of TTEs were appropriate, 5 % (n = 27) were inappropriate and 4 % (n = 23) uncertain. There was no statistically significant difference in appropriateness by ordering provider specialty (p = 0.78). Although the majority of TTEs were appropriate, fewer outpatient TTEs were appropriate compared with inpatient TTEs (85 vs. 96 %, p \ 0.0001).
The ten most frequent AUCs, all classified as appropriate, accounted for 66 % of all TTEs (Table 2). Clinically significant abnormal TTE findings Of 548 TTEs reviewed, 54 % had C1 clinically significant TTE abnormality. The most common abnormalities were wall motion abnormalities (25 %), elevated RVSP [ 40 mmHg (20 %), moderate or greater valvular regurgitation (15 %), and grade III diastolic dysfunction (14 %; Table 3). There was no difference in the prevalence of C1 clinically significant TTE abnormality among outpatient versus inpatient TTEs (50 vs. 58 %, p = 0.07).There was a higher prevalence of C1 clinically significant TTE abnormality among appropriate versus inappropriate TTEs (56 vs. 33 %, p = 0.029). There was a non-statistically significant trend towards a higher prevalence of C1 clinically significant TTE abnormality among appropriate versus inappropriate TTE in inpatients (59 vs. 27 %, p = 0.058) and outpatients (51 vs. 38 %, p = 0.31). Repeat TTEs (n = 232) had increased proportion of C1 clinically significant TTE abnormality compared with firsttime (n = 316) TTEs (70 vs. 43 %, p \ 0.0001). There was a higher prevalence of C1 clinically significant TTE abnormality among appropriate versus inappropriate repeat TTEs (72 vs. 42 %, p = 0.03). No differences were seen for appropriate versus inappropriate first-time TTEs (44 vs. 27 %, p = 0.09), however the number of inappropriate TTEs was small in both groups (first-time TTEs n = 15, repeat TTEs n = 12). Among repeat TTEs, a new clinically significant TTE abnormality was found in only 25 % and resolution of a previous TTE abnormality was found in 10 %. The prevalence of a new clinically significant TTE abnormality was similar between appropriate and inappropriate repeat TTEs (26 vs. 25 %, p = 1.0). The prevalence of clinically significant TTE abnormalities among individual appropriate AUCs varied significantly, with some appropriate AUCs being associated with a high prevalence (C2/3) of abnormality (AUC #73, #5, #15, #18, #19) while other appropriate AUCs were associated with a low prevalence (\1/2) of abnormality (AUC #1, #34, #91; Table 3). Similarly, some inappropriate AUCs were associated with a high prevalence of abnormality (C2/3; AUC #16, #74) while others were associated with no abnormal findings (#10, #53, #68; Table 2). Clinical impact The prevalence of C1 clinically significant TTE abnormality was lower in TTEs that led to continuation of current care than TTEs that led to no change or active change (continuation of care vs. no change vs. active change: 40 vs. 64 vs. 70 %). The prevalence of C1 clinically
123
Int J Cardiovasc Imaging Table 1 Cohort characteristics Overall cohort (n = 548)
Normal echo (n = 250)
C1 Echo abnormality (n = 298)
Age (years)
58 ± 17
56 ± 17
60 ± 18
Male (%)
41
31
49
Female (%)
59
69
51
Inpatient (%)
55
49
59
Outpatient (%)
45
51
41
White (%)
55
53
57
Hispanic (%)
8
9
8
Black (%)
21
21
21
Other or unknown (%)
15
17
15
Cardiology (%) Pulmonary/critical care (%)
31 15
26 8
36 20
Internal medicine (%)
38
49
29
Surgery (%)
10
10
10
Ordering provider
Neurology (%)
3
6
2
Other (%)
3
2
3
significant TTE abnormality was not statistically significantly different between TTEs that lead to no change versus active change in care (64 vs. 70 %, p = 0.06). Similarly, there was no statistically significant difference between the proportion of active change versus no change in TTEs with C1 clinically significant TTE abnormality (active change vs. no change: 40 vs. 24 %, p = 0.30).
Discussion Although echocardiography is an extremely helpful clinical tool, there is growing concern regarding its overuse as a diagnostic modality. Our study found that nearly half (46 %) of all TTEs and over half (57 %) of first-time TTEs performed had no clinically significant TTE abnormalities and were therefore essentially normal studies. Among repeat TTEs, only a minority had a new clinically significant abnormality. Although appropriate TTEs had a higher prevalence of C1 clinically significant TTE abnormality, the prevalence of abnormal TTE findings was similar between TTEs that resulted in active change versus no change suggesting that clinical decision-making is driven by a mixture of patient, center, and physician factors rather than by TTE alone. Few TTEs were inappropriate by AUC but a majority of TTEs had no clinically significant findings suggesting AUC is overly broad and that TTEs are being over-utilized. In our cohort, 91 % of TTEs were appropriate, 5 % were inappropriate, and 4 % were uncertain similar to recently published data of 92 % appropriate, 2 % inappropriate,
123
\1 % uncertain, 5.5 % unclassifiable [17]. Other studies report some variation in the proportion of appropriate TTEs, which may in part be due to the granularity of the data in the clinical chart to determine AUC, differences in patient acuity, differences in evaluation of appropriateness in each study, and timing relative to publication of AUC criteria [10, 18–20]. The prevalence for appropriate TTEs ranged from 77 to 82 % in these studies, suggesting that with dissemination of the guidelines and incorporation of the AUC into practice, appropriateness may have increased. From an international perspective, when AUC were applied in the United Kingdom and in Italy, similar results were found with 86 % of TTEs performed in Wales [21] and 80 % of TTEs performed in Italian centers [18] were classified as appropriate. In all studies, the majority of TTEs are appropriate which is in agreement with our current findings. Despite the fact that over 90 % of all TTEs were appropriate in our study, only about had abnormal TTE findings. Appropriate TTEs were associated with a higher prevalence of abnormal findings compared with inappropriate TTEs (51 vs. 38 %, p = 0.013). These findings are similar to a recent study by Bhatia et al. [9] that found that appropriate TTEs had a higher prevalence of TTE abnormalities than inappropriate or uncertain studies (42 vs. 24 vs. 18 %, p = 0.08). In terms of repeat TTEs, a previous study by Ghatak et al. [22] evaluated the appropriateness and incidence of new TTE findings in a community teaching inpatient hospital setting and showed that 56 % of repeat TTEs performed within 1 year of the index TTE (n = 104) had a new TTE abnormality defined as a 20 %
Int J Cardiovasc Imaging Table 2 The most frequent appropriate use criteria and their association with clinically significant abnormal transthoracic echocardiogram abnormalities AUC
AUC description
C1 Echo abnormality (%)
No. of TTEs
36
151
Most frequent appropriate AUCs 1.
Symptoms or conditions potentially related to suspected cardiac etiology including but not limited to chest pain, shortness of breath, palpitations, transient ischemic attack, stroke, or peripheral embolic event
73.
Re-evaluation of known heart failure to guide therapy
88
34
2.
Prior testing that is concerning for heart disease or structural abnormality including but not limited to chest X-ray, scout images of stress echocardiogram, electrocardiogram, cardiac biomarkers
54
28
5.
Sustained or non-sustained atrial fibrillation, supraventricular tachycardia, or ventricular tachycardia
67
27
91. 15.
Baseline and serial re-evaluations in a patient undergoing therapy with cardiotoxic agents Evaluation of suspected pulmonary hypertension including right ventricular function and pulmonary artery pressure estimation
33 78
24 23
34.
Initial evaluation when there is a reasonable suspicion of valvular or structural heart disease
41
22
18.
Re-evaluation of known pulmonary hypertension if change in clinical status or cardiac exam or to guide therapy
85
20
19.
Hypotension or hemodynamic instability of uncertain or suspected cardiac etiology
80
20
52.
Initial evaluation of suspected infective endocarditis with positive blood culture or a murmur
60
15
Most frequent inappropriate AUCs 13.
Routine perioperative evaluation of ventricular function with no symptoms or signs of cardiovascular disease
25
4
16.
Routine surveillance (\1 year) of known pulmonary hypertension without change in clinical status or cardiac exam
67
3
74.
Routine surveillance (\1 year) of HF (systolic or diastolic) when there is no change in clinical status or cardiac exam
67
3
6. 10.
Asymptomatic isolated sinus bradycardia Initial evaluation of ventricular function (e.g., screening) with no symptoms or signs of cardiovascular disease
50 0
2 2
53.
Transient fever without evidence of bacteremia or a new murmur
0
2
68.
Routine evaluation of systemic hypertension without symptoms or signs of hypertensive heart disease
0
2
change in LVEF, change in wall motion, new valvular regurgitation that increased by one grade, or a 20 % change in pulmonary pressures. In our study, 42 % of patients had repeat TTEs within 1 year, of whom 25 % had a new clinically significant TTE finding on the repeat TTE, 10 % had resolution of a previously demonstrated TTE abnormality, and 65 % had no new TTE findings. Our lower prevalence of new TTE abnormalities in repeat TTEs may be partially due to our mix of both inpatient and outpatient TTEs compared to the exclusively inpatient TTEs in the previous study as well as our stricter definition of a clinically significant TTE abnormality. Another study [11] found that new clinically important TTE findings were found in 32 % of their cohort and that this did not differ between appropriate and inappropriate TTEs. We found that appropriate and inappropriate TTEs had similar prevalence of new major TTE abnormalities (26 vs. 25 %) in repeat TTEs, however the number of inappropriate repeat TTEs was small (n = 12) limiting our power to demonstrate a statistically significant difference. In either case,
the yield of new diagnostic information from these repeat TTEs within 1 year is low. Factors besides TTE abnormalities likely drive the decision to actively respond to TTE findings and need to be further investigated. Abnormal TTE findings have been suggested as a surrogate endpoint to assess the clinical utility of AUC [23]. Indeed, our study found appropriate TTEs had a higher prevalence of C1 clinically significant TTE abnormality than inappropriate TTEs. However, the definitions of appropriate AUC often are specific to a TTE-defined disease process, resulting in a high pre-test probability of a TTE abnormality while inappropriate AUC definitions often result in a low pre-test probability of a TTE abnormality. For example, indication #73, ‘‘re-evaluation of known heart failure to guide therapy’’ has a prevalence of C1 major TTE abnormality of 91 % since these patients had to have known heart failure to meet this criterion. In contrast, inappropriate indication #10, ‘‘Initial evaluation of ventricular function (e.g., screening) with no symptoms or signs of cardiovascular disease’’ selects for patients with
123
Int J Cardiovasc Imaging Table 3 Prevalence of transthoracic echocardiogram abnormalities Echo abnormality
All [(n = 548) (%)]
Appropriate [(n = 498) (%)]
Inappropriate [(n = 27) (%)]
LVEF \ 45 %
16
16
11
Wall motion abnormalities
25
25
22
Grade III diastolic dysfunction
14
15
4
RVSP [ 40 mmHg
20
20
7
Dilated RV
14
14
11
RV dysfunction
10
10
4
Cardiac shunt
3
3
4
Vegetation or cardiac mass
2
2
0
CModerate valvular regurgitation
15
15
4
CModerate valvular stenosis
3
3
0
Dilated aorta Aortic coarctation
2 \1
2 \1
4 0
CModerate pericardial effusion
2
2
0
Pericardial disease
\1
\1
0
LVEF left ventricular ejection fraction, RV right ventricle, RVSP right ventricular systolic pressure
a very low pre-test probability of having a cardiac structural or functional abnormality. Patients meeting this criterion had a 0 % prevalence of C1 major TTE abnormality in our cohort. As the definitions of an individual AUC becomes less specific (ex. appropriate AUC #1, ‘‘Symptoms or conditions potentially related to suspected cardiac etiology including but not limited to chest pain, shortness of breath, palpitations, transient ischemic attack, stroke, or peripheral embolic event’’), the prevalence of C1 major TTE abnormality decreases (36 %). Therefore, prevalence of TTE abnormalities track with the language of AUC definitions and will likely be of limited utility as a metric for AUC clinical effectiveness. Similarly, in inappropriate AUCs that have a high pre-test probability of a TTE abnormality, like # 74, ‘‘Routine surveillance (\1 year) of HF (systolic or diastolic) when there is no change in clinical status or cardiac exam’’, the prevalence of C1 major TTE abnormality was also high (67 %). Moreover, even though a bias introduced by potentially leading definitions of AUC would magnify the association between appropriateness and abnormal findings, our overall prevalence of clinically significant echocardiographic abnormalities of 54 % was low. Defining an appropriate metric to assess the efficacy of noninvasive imaging in the provision of care is a difficult task. In the 1990s, Dr. Fryback and Thornbury [24] described a 6-tiered hierarchal framework for assessing technology efficacy, which was recently endorsed by the American College of Cardiology, in collaboration with multiple subspecialty societies, in a health policy statement on noninvasive imaging use [25]. In this framework, efficacy is assessed by considering the technical efficacy
123
(spatial resolution, imaging reproducibility), diagnostic efficacy (sensitivity, specificity, accuracy), diagnostic thinking efficacy (incremental value of testing information on diagnosis), therapeutic efficacy (changes in patient management related to testing), patient outcome efficacy (safety concerns, improved patient outcomes, including quality of life, morbidity and mortality as a result of testing), and society efficacy (cost versus overall societal benefit of testing) of testing in clinical care. The detection of TTE abnormalities according to AUC classification would be consistent with level 2 (diagnostic efficacy) of this framework) while trying to assess clinical impact of testing is consistent with level 4. A prime tenet of the RAND process that was used to develop the AUC is that the assignment of appropriateness needs to be formally tested to see if it led to improved outcomes and modified if it did not [26]. For TTE AUC, this would require validating if criteria rated as appropriate accomplish the ultimate objective of impacting patient care. Developing metrics that not only capture diagnostic efficacy but attempt to assess the higher levels of the technology efficacy framework are needed in order to maximize the societal benefit of TTE or any diagnostic testing modality. There are several limitations to this study. This study relies on retrospective chart review, which may have led to misclassification of AUC. The performance of this study in a tertiary care academic institution may limit its generalizability to other patient care settings. Without clinical outcomes, the value of TTE findings on patient care cannot be determined. The low prevalence of inappropriate TTEs limits statistical power to detect a difference in the clinically significant abnormal TTE findings between
Int J Cardiovasc Imaging
appropriate and inappropriate TTEs in subgroups; however the low overall prevalence of clinically significant abnormal findings suggests that efforts to further maximize the utility of TTE are needed. Finally, clinical decision-making is a dynamic process that involves interactions between clinicians, patients and external factors. Our study attempts to place this dynamic process into categorical groups which can be challenging in the setting of an EMR where we are limited to information that was documented.
Conclusions Appropriate TTEs are associated with a higher prevalence of clinically significant TTE abnormalities than inappropriate TTEs, in part due to the AUC definitions. TTEs that actively impact care had a similar prevalence of abnormal TTE findings compared with TTEs that led to no change in care, suggesting limited clinical utility in using abnormal findings to assess the clinical effectiveness of TTE. Development and validation of metrics reflecting improved therapeutic efficacy and patient outcomes may allow for targeted efforts that can maximize diagnostic and therapeutic yield of TTE. Acknowledgments Supported in part by grant UL1TR000451 from the UT-STAR, National Center for Advancing Translational Sciences, National Institutes of Health. Susan Matulevicius, M.D. had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. The content is solely the responsibility of the authors and does not necessarily represent the official views of UT-STAR, UT Southwestern Medical Center and its affiliated academic and health care centers, the National Center for Advancing Translational Sciences, or the National Institutes of Health. Conflict of interest
None.
References 1. Medicare Payment Advisory Commission (2012) A Data Book: health care spending and the medicare program. Medicare Payment Advisory Commission, Washington, DC 2. Pearlman AS, Ryan T, Picard MH, Douglas PS (2007) Evolving trends in the use of echocardiography: a study of medicare beneficiaries. J Am Coll Cardiol 49(23):2283–2291. doi:10.1016/j. jacc.2007.02.048 3. Andrus BW, Welch HG (2012) Medicare services provided by cardiologists in the United States: 1999–2008. Circ Cardiovasc Qual Outcomes 5(1):31–36. doi:10.1161/CIRCOUTCOMES.111. 961813 4. Blecker S, Bhatia RS, You JJ, Lee DS, Alter DA, Wang JT, Wong HJ, Tu JV (2013) Temporal trends in the utilization of echocardiography in Ontario, 2001 to 2009. JACC Cardiovasc Imaging 6(4):515–522. doi:10.1016/j.jcmg.2012.10.026 5. Welch HHKJFC (2012) Repeat testing among medicare beneficiaries. Arch Intern Med 172(22):1745–1751. doi:10.1001/2013. jamainternmed.727
6. Douglas PS, Garcia MJ, Haines DE, Lai WW, Manning WJ, Patel AR, Picard MH, Polk DM, Ragosta M, Parker Ward R, Weiner RB (2011) ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/ SCCT/SCMR 2011 appropriate use criteria for echocardiography. J Am Soc Echocardiogr 24(3):229–267. doi:10.1016/j.echo.2010. 12.008 7. Douglas PS, Khandheria B, Stainback RF, Weissman NJ, Brindis RG, Patel MR, Alpert JS, Fitzgerald D, Heidenreich P, Martin ET, Messer JV, Miller AB, Picard MH, Raggi P, Reed KD, Rumsfeld JS, Steimle AE, Tonkovic R, Vijayaraghavan K, Yeon SB, Hendel RC, Peterson E, Wolk MJ, Allen JM (2007) ACCF/ ASE/ACEP/ASNC/SCAI/SCCT/SCMR 2007 appropriateness criteria for transthoracic and transesophageal echocardiography: a report of the American College of Cardiology Foundation Quality Strategic Directions Committee Appropriateness Criteria Working Group, American Society of Echocardiography, American College of Emergency Physicians, American Society of Nuclear Cardiology, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, and the Society for Cardiovascular Magnetic Resonance endorsed by the American College of Chest Physicians and the Society of Critical Care Medicine. J Am Coll Cardiol 50(2):187–204. doi:10.1016/j.jacc.2007.05.003 8. Patel MR, Spertus JA, Brindis RG, Hendel RC, Douglas PS, Peterson ED, Wolk MJ, Allen JM, Raskin IE (2005) ACCF proposed method for evaluating the appropriateness of cardiovascular imaging. J Am Coll Cardiol 46(8):1606–1613. doi:10. 1016/j.jacc.2005.08.030 9. Bhatia RS, Carne DM, Picard MH, Weiner RB (2012) Comparison of the 2007 and 2011 appropriate use criteria for transthoracic echocardiography in various clinical settings. J Am Soc Echocardiogr 25(11):1162–1169. doi:10.1016/j.echo. 2012.07.018 10. Mansour IN, Razi RR, Bhave NM, Ward RP (2012) Comparison of the updated 2011 appropriate use criteria for echocardiography to the original criteria for transthoracic, transesophageal, and stress echocardiography. J Am Soc Echocardiogr 25(11):1153– 1161. doi:10.1016/j.echo.2012.08.008 11. Kirkpatrick JN, Ky B, Rahmouni HW, Chirinos JA, Farmer SA, Fields AV, Ogbara J, Eberman KM, Ferrari VA, Silvestry FE, Keane MG, Opotowsky AR, Sutton MS, Wiegers SE (2009) Application of appropriateness criteria in outpatient transthoracic echocardiography. J Am Soc Echocardiogr 22(1):53–59. doi:10. 1016/j.echo.2008.10.020 12. Matulevicius SA, Rohatgi A, Das SR, Price AL, Deluna A, Reimold SC (2013) Appropriate use and clinical impact of transthoracic echocardiography. JAMA Intern Med 173(17): 1600–1607. doi:10.1001/jamainternmed.2013.8972 13. Lang RM, Bierig M, Devereux RB, Flachskampf FA, Foster E, Pellikka PA, Picard MH, Roman MJ, Seward J, Shanewise JS, Solomon SD, Spencer KT, Sutton MS, Stewart WJ, Chamber Quantification Writing G, American Society of Echocardiography’s G, Standards C, European Association of E (2005) Recommendations for chamber quantification: a report from the American Society of Echocardiography’s Guidelines and standards committee and the chamber quantification writing group, developed in conjunction with the European Association of Echocardiography, a branch of the European Society of Cardiology. J Am Soc Echocardiogr 18(12):1440–1463. doi:10.1016/j. echo.2005.10.005 14. Picard MH, Adams D, Bierig SM, Dent JM, Douglas PS, Gillam LD, Keller AM, Malenka DJ, Masoudi FA, McCulloch M, Pellikka PA, Peters PJ, Stainback RF, Strachan GM, Zoghbi WA (2011) American Society of Echocardiography recommendations for quality echocardiography laboratory operations. J Am Soc Echocardiogr 24(1):1–10. doi:10.1016/j.echo.2010.11.006
123
Int J Cardiovasc Imaging 15. Zoghbi WA, Enriquez-Sarano M, Foster E, Grayburn PA, Kraft CD, Levine RA, Nihoyannopoulos P, Otto CM, Quinones MA, Rakowski H, Stewart WJ, Waggoner A, Weissman NJ, American Society of E (2003) Recommendations for evaluation of the severity of native valvular regurgitation with two-dimensional and Doppler echocardiography. J Am Soc Echocardiogr 16(7):777–802. doi:10.1016/S0894-7317(03)00335-3 16. Baumgartner H, Hung J, Bermejo J, Chambers JB, Evangelista A, Griffin BP, Iung B, Otto CM, Pellikka PA, Quinones M, American Society of E, European Association of E (2009) Echocardiographic assessment of valve stenosis: EAE/ASE recommendations for clinical practice. J Am Soc Echocardiogr 22(1):1–23. doi:10.1016/ j.echo.2008.11.029 quiz 101–102 17. Parikh PB, Asheld J, Kort S (2012) Does the revised appropriate use criteria for echocardiography represent an improvement over the initial criteria? A comparison between the 2011 and the 2007 appropriateness use criteria for echocardiography. J Am Soc Echocardiogr 25(2):228–233. doi:10.1016/j.echo.2011.09.017 18. Ballo P, Bandini F, Capecchi I, Chiodi L, Ferro G, Fortini A, Giuliani G, Landini G, Laureano R, Milli M, Nenci G, Pizzarelli F, Santoro GM, Vannelli P, Cappelletti C, Zuppiroli A (2012) Application of 2011 American College of Cardiology Foundation/American Society of Echocardiography appropriateness use criteria in hospitalized patients referred for transthoracic echocardiography in a community setting. J Am Soc Echocardiogr 25(6):589–598. doi:10.1016/j.echo.2012.03.006 19. Alqarqaz M, Koneru J, Mahan M, Ananthasubramaniam K (2012) Applicability, limitations and downstream impact of echocardiography utilization based on the appropriateness use criteria for transthoracic and transesophageal echocardiography. Int J Cardiovasc Imaging 28(8):1951–1958. doi:10.1007/s10554012-0008-6 20. Willens HJ, Hendel RC, Inhaber FR, Chakko SC, Postel C, Hasan T, Mohammed F (2011) Appropriateness use criteria for
123
21.
22.
23.
24. 25.
26.
transthoracic echocardiography: relationship with radiology benefit managers preauthorization determination and comparison of the new (2010) criteria to the original (2007) criteria. Am Heart J 162(4):772–779. doi:10.1016/j.ahj.2011.07.009 Gurzun MM, Ionescu A (2014) Appropriateness of use criteria for transthoracic echocardiography: are they relevant outside the USA? Eur Heart J Cardiovasc Imaging 15(4):450–455. doi:10. 1093/ehjci/jet186 Ghatak A, Pullatt R, Vyse S, Silverman DI (2011) Appropriateness criteria are an imprecise measure for repeat echocardiograms. Echocardiography 28(2):131–135. doi:10.1111/j.15408175.2010.01302.x Dudzinski DM, Bhatia RS, Weiner RB (2014) Assessing the clinical impact of appropriate echocardiograms. JAMA Intern Med 174(7):1195–1196. doi:10.1001/jamainternmed.2014.1440 Fryback DG, Thornbury JR (1991) The efficacy of diagnostic imaging. Med Decis Making 11(2):88–94 Mark DB, Anderson JL, Brinker JA, Brophy JA, Casey DE Jr, Cross RR, Edmundowicz D, Hachamovitch R, Hlatky MA, Jacobs JE, Jaskie S, Kett KG, Malhotra V, Masoudi FA, McConnell MV, Rubin GD, Shaw LJ, Sherman ME, Stanko S, Ward RP (2014) ACC/AHA/ASE/ASNC/HRS/IAC/Mended Hearts/NASCI/RSNA/SAIP/SCAI/SCCT/SCMR/SNMMI 2014 health policy statement on use of noninvasive cardiovascular imaging: a report of the American College of Cardiology Clinical Quality Committee. J Am Coll Cardiol 63(7):698–721. doi:10. 1016/j.jacc.2013.02.002 Fitch K, Bernstein SJ, Aguilar MS, Burnand B, LaCalle JR, Lazaro P, van het Loo M, McDonnell J, Vader J, Kahan JP (2001) The RAND/UCLA appropriateness method user’s manual. RAND Corporation, Santa Monica, CA
