REFERENCES [1] Nashef SA, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon R. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999;16:9–13. [2] Chalmers J, Pullan M, Fabri B, McShane J, Shaw M, Mediratta N et al. Validation of EuroSCORE II in a modern cohort of patients undergoing cardiac surgery. Eur J Cardiothorac Surg 2013;43:688–94. [3] Nashef SA, Roques F, Sharples LD, Nilsson J, Smith C, Goldstone AR et al. EuroSCORE II. Eur J Cardiothorac Surg 2012;41:734–44. [4] Shanmugam G, West M, Berg G. Additive and logistic EuroSCORE performance in high risk patients. Interact CardioVasc Thorac Surg 2005;4: 299–303. [5] Michel P, Roques F, Nashef SA. Logistic or additive EuroSCORE for highrisk patients? Eur J Cardiothorac Surg 2003;23:684–7. [6] Wendt D, Osswald BR, Kayser K, Thielmann M, Tossios P, Massoudy P et al. Society of thoracic surgeons score is superior to the EuroSCORE determining mortality in high risk patients undergoing isolated aortic valve replacement. Ann Thorac Surg 2009;88:468–74. [7] Austin PC. A comparison of regression trees, logistic regression, generalized additive models, and multivariate adaptive regression splines for predicting AMI mortality. Stat Med 2007;26:2937–57. [8] Kleinman LC, Norton EC. What’s the risk? A simple approach for estimating adjusted risk measures from nonlinear models including logistic regression. Health Serv Res 2009;44:288–302. [9] Poullis M. Introducing change (science into the operating room): quality improvement versus experimentation. J Extra Corpor Technol 2009;41:11–5. [10] Poullis M. Has Microsoft left behind risk modeling in cardiac and thoracic surgery? J Extra Corpor Technol 2011;43:2–9. [11] O’Boyle F, Mediratta N, Fabri B, Pullan M, Chalmers J, McShane J et al. Long-term survival after coronary artery bypass surgery stratified by EuroSCORE. Eur J Cardiothorac Surg 2012;42:101–6. [12] Pagano D, Freemantle N, Bridgewater B, Howell N, Ray D, Jackson M et al. Social deprivation and prognostic benefits of cardiac surgery: observational study of 44 902 patients from five hospitals over 10 years. BMJ 2009; 338:b902. [13] Hosmer DW, Lemeshow S. Assessing the fit of the Model. Applied Logistic Regression. NJ: Wiley-Blackwell, 2012, 143–203. [14] Chalmers J, Mediratta N, McShane J, Shaw M, Pullan M, Poullis M. The long-term effects of developing renal failure post-coronary artery bypass surgery, in patients with normal preoperative renal function. Eur J Cardiothorac Surg 2013;43:555–9. [15] Warwick R, Mediratta N, Chalmers J, Pullan M, Shaw M, McShane J et al. Is single-unit blood transfusion bad post-coronary artery bypass surgery? Interact CardioVasc Thorac Surg 2013;16:765–71. [16] O’Boyle F, Mediratta N, Chalmers J, Warwick R, Shaw M, McShane J et al. Long-term survival of non-smokers undergoing coronary artery bypass surgery. Eur J Cardiothorac Surg 2014;45:445–51. [17] Bhatti F, Grayson AD, Grotte G, Fabri BM, Au J, Jones M et al. The logistic EuroSCORE in cardiac surgery: how well does it predict operative risk? Heart 2006;92:1817–20. [18] Nilsson J, Ohlsson M, Thulin L, Hoglund P, Nashef SA, Brandt J. Risk factor identification and mortality prediction in cardiac surgery using artificial neural networks. J Thorac Cardiovasc Surg 2006;132:12–9. [19] Edwards FH, Peterson RF, Bridges C, Ceithaml EL. 1988: use of a Bayesian statistical model for risk assessment in coronary artery surgery. Updated in 1995. Ann Thorac Surg 1995;59:1611–2. [20] Stojadinovic A, Eberhardt J, Brown TS, Hawksworth JS, Gage F, Tadaki DK et al. Development of a Bayesian model to estimate health care outcomes in the severely wounded. J Multidiscip Health 2010;3:125–35.

eComment. EuroSCORE II - corrected in-hospital mortality rate in a modern cohort of patients undergoing cardiac surgery Author: Dusko G. Nezic "Dedinje" Cardiovascular Institute, Belgrade, Serbia doi: 10.1093/icvts/ivu313 © The Author 2014. Published by Oxford University Press on behalf of the European Association for Cardio-Thoracic Surgery. All rights reserved. Although both additive and logistic versions of the EuroSCORE have retained very good discriminatory power, suspicions have developed concerning whether the

733

model may now be inappropriately calibrated for current cardiac surgery [1]. Therefore, the old EuroSCORE has recently been adapted into the EuroSCORE II in order to optimize its discriminatory power and, particularly, its calibration [1]. Furthermore, any new adaptations, including the use of new methodologies [2], are welcome in order to improve the Hosmer-Lemeshow statistics. However, I would like to use this eComment to clarify data in Table 1 (Patient characteristics) from M. Poullis’ manuscript [2]. It is obvious that Poullis [2] has used the same patient cohort that has previously been reported by Chalmers et al. [3], that is, patients who have undergone isolated coronary artery bypass grafting (CABG), isolated aortic valve replacement (AVR), isolated mitral valve surgery (MVR), combined CABG and AVR surgery, aortic surgery and miscellaneous cardiac procedures between January 2006 and March 2010 in the Liverpool Heart and Chest Hospital, Liverpool, UK. In Table 1 of the Poullis’ manuscript [2], in the last row, we can find data that in-hospital mortality was 2.2% [101 patients out of all 5576 patients (second column) who were operated during aforementioned period, with EuroSCORE II predicted mortality of 2.0% - i.e. 112 patients]. However, those are misleading data. Indeed, 101 patients died but only in the subgroups of CABG surgery, isolated AVR surgery, isolated MVR surgery and AVR + CABG surgery. Data about mortality in aortic surgery and mortality in miscellaneous procedures (24 patients out of 350 who had aortic surgery died, and 66 patients out of 642 who had miscellaneous procedures performed [3]) were not included in overall mortality of the entire cohort (all 5576 who were operated). Therefore, overall mortality is 191 patients out of 5576 who were operated during the period from January 2006 and March 2010. Thus corrected, in-hospital mortality of the reported cohort [2, 3] appears to be 3.43%, not 2.2%. It has recently been suggested that the calibration of the EuroSCORE II should be assessed by the observed/expected (O/E) ratio of mortality. Ideally, this ratio equals 1.0 (the observed mortality equals expected mortality, thus the predictive model is perfectly calibrated). A value above 1.0 means that the model underestimates mortality, a value below 1.0 means that model overestimates mortality. If the 95% confidence interval (CI) of the O/E ratio excludes the value 1.0, it may be considered statistically significant [4]. This leads to the conclusion that the observed to expected mortality ratio (191/112) is 1.71 (95% CI 1.47-1.95), thus not including value of 1.0. Therefore, an overall poor calibration of the EuroSCORE II model in the presented cohort of patients [2, 3] has been confirmed with the O/E ratio of mortality, as well as with Hosmer-Lemeshow statistics (P < 0.001). Conflict of interest: none declared. References [1] Nashef S, Roques F, Sharples L, Nillson J, Smith C, Goldstone A et al. EuroSCORE II. Eur J Cardiothorac Surg 2012;41:734–745. [2] Poullis M. Recursive and non-linear logistic regression: moving on from the original EuroSCORE and EuroSCORE II methodologies. Interact CardioVasc Thorac Surg 2014;19:726–33. [3] Chalmers J, Pullan M, Fabri B, McShane J, Shaw M, Mediratta N et al. Validation of EUROSCORE II in a modern cohort of patients undergoing cardiac surgery. Eur J Cardiothorac Surg 2013;43:688–694. [4] Nezic D, Borzanovic M, Spasic T, Vukovic P. Calibration of the EuroSCORE II risk stratification model: is the Hosmer-Lemeshow test acceptable any more? Eur J Cardiothorac Surg 2013;43:206.

eComment. EuroSCORE II and its Achilles’ heel Authors: Kyriakos Spiliopoulos, Oliver Deutsch, Walter Eichinger and Brigitte Gansera Department of Cardiovascular Surgery, Klinikum München Bogenhausen GmbH, Munich, Germany doi: 10.1093/icvts/ivu339 © The Author 2014. Published by Oxford University Press on behalf of the European Association for Cardio-Thoracic Surgery. All rights reserved. We read with great interest the article by Poullis [1] proposing that a recursive and non-linear regression model utilizing the EuroSCORE II risk model improves both its ROC and Hosmer-Lemeshow statistics. The European System for Cardiac Operative Risk Evaluation II (EuroSCORE II) is an updated tool for the prediction of in-hospital mortality after cardiac surgery, recently launched to replace the older additive and logistic EuroSCOREs developed in the late 1990s [2]. These previous versions have been extensively used in the last decade and have performed well, showing acceptable applicability to different populations of cardiac surgical patients [3]. However, the improvements in surgical techniques, the impact of open reporting of clinical

NEW IDEAS

M. Poullis / Interactive CardioVascular and Thoracic Surgery

734

M. Poullis / Interactive CardioVascular and Thoracic Surgery

outcome on medical behaviours and quality of practices, as well as changes in case mix have led to the loss of their predictive power and calibration. The new version is a result of refinement and modification of some of the established risk factors and the way the model evaluates them [2]. Its internal validation has showed a better calibration associated with a constant optimal discrimination and the few currently existing external validation studies may attest to an accuracy similar to the previous EuroSCORE and a much improved calibration. On the other hand, these studies have also expressed concern regarding an underestimation in the prediction of mortality, especially in high risk patients, or in those undergoing combined procedures [4]. Poullis’ suggestion [1] for mathematical adjustments to the model using new methodologies certainly contributes to its performance improvement, but on the other hand, this confirms the observation that the updated EuroSCORE version seems to improve but not solve the original limitations of EuroSCORE I. However, it should not be overlooked, that the main measured variable by the model, namely operative mortality, is defined and registered differently among researchers, countries and national health systems. Poullis defined, in accordance to the original EuroSCORE II study by Nashef et al. [3], operative mortality as death at the institution where the surgery was performed. This may be convenient in order to assess most of the available information, but it does not reflect the "real" early operative-related mortality as perceived by either the patients or many cardiac surgeons in their everyday practice. Although a realistic approach seems to be the registration of death events occurring either within 30 days of cardiac surgery, and/or in the hospital prior to discharge to home, regardless of length of the hospital stay, the most reasonable follow-up time period to quantify early mortality for coronary bypass patients after adult cardiac surgery extends up to 3 months or more for valve or combined procedures patients

[5]. As long as there is a lack of uniformity in what the model measures and what the surgical community measures, it is rather wise to first go one step back in defining a widely-accepted early operative-related mortality and then two steps forward by using mathematical adjustments like those suggested by Poullis [1]. Conflict of interest: none declared References [1] Poullis M. Recursive and non-linear logistic regression: moving on from the original EuroSCORE and EuroSCORE II methodologies. Interact CardioVasc Thorac Surg 2014;19:726–33. [2] Nashef SA, Roques F, Sharples LD, Nilsson J, Smith C, Goldstone AR et al. EuroSCORE II. Eur J Cardiothorac Surg 2012;41:734–45. [3] Nashef SA, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon R. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999;16:9–13. [4] Spiliopoulos K, Bagiatis V, Deutsch O, Kemkes BM, Antonopoulos N, Karangelis D et al. Performance of EuroSCORE II compared to EuroSCORE I in predicting operative and mid-term mortality of patients from a single center after combined coronary artery bypass grafting and aortic valve replacement. Gen Thorac Cardiovasc Surg 2014;62:103–11. [5] Osswald BR, Blackstone EH, Tochtermann U, Thomas G, Vahl CF, Hagl S. The meaning of early mortality after CABG. Eur J Cardiothorac Surg 1999;15:401–7.

eComment. EuroSCORE II and its Achilles' heel.

eComment. EuroSCORE II and its Achilles' heel. - PDF Download Free
50KB Sizes 0 Downloads 6 Views