Annals of Surgery Volume 265, Number 4, April 2017
Letters to the Editor
rectal cancer restaging after concurrent neoadjuvant chemo- and radiation therapy, it is better than nothing, especially considering a higher morbidity rate and a longer hospital stay after ELAPE. In summary, we conclude that ELAPE may still be effective in highly selective situations. We think the Danish investigation will not be the prohibition sign of ELPAE in routine colorectal surgical practice but a great milestone of conducting future studies to develop the guidelines for such an aggressive powerful procedure. Disclosure: The authors declare no conflicts of interest and no funding.
REFERENCES
ISGLS 0.093 1.000
50−50 0.796
PeakBili>7
ISGLS 0.068 0.499
50−50 0.730
PeakBili>7
ISGLS 0.266 0.280
50−50 1.000
PeakBili>7
ISGLS 0.527 0.284
50−50 1.000
PeakBili>7 2
4
Mortality within 10d
e56 | www.annalsofsurgery.com
A
Mortality POD5
1. Klein M, Fischer A, Rosenberg J, et al. Extralevatory abdominoperineal excision (ELAPE) does not result in reduced rate of tumor perforation or rate of positive circumferential resection margin: a nationwide database study [published online ahead of print September 10, 2014]. Ann Surg. doi:10.1097/ SLA.0000000000000910. 2. Nagtegaal ID, van de Velde CJ, Marijnen CA, et al. Low rectal cancer: a call for a change of approach in abdominoperineal resection. J Clin Oncol. 2005;23:9257–9264. 3. West NP, Anderin C, Smith KJ, et al. Multicentre experience with extralevator abdominoperineal excision for low rectal cancer. Br J Surg. 2010;97:588–599. 4. Martijnse IS, Dudink RL, West NP, et al. Focus on extralevator perineal dissection in supine position for low rectal cancer has led to better quality of surgery and oncologic outcome. Ann Surg Oncol. 2012;19:786–793. 5. de Campos-Lobato LF, Stocchi L, Dietz DW, et al. Prone or lithotomy positioning during an abdominoperineal resection for rectal cancer results in comparable oncologic outcomes. Dis Colon Rectum. 2011;54:939–946. 6. Asplund D, Haglind E, Angenete E. Outcome of extralevator abdominoperineal excision compared with standard surgery: results from a single centre. Colorectal Dis. 2012;14:1191–1196. 7. Ortiz H, Ciga MA, Armendariz P, et al. Multicentre propensity score-matched analysis of conventional
To the Editor: lthough external validation of predictive clinical scoring systems is essential, it is
Morbidity within 10d
Cun Wang, MD, PhD Lie Yang, MD, PhD Department of Gastrointestinal Surgery West China Hospital Sichuan University Chengdu, China. [email protected]
Comparison of Binary Predictive Scoring Systems of Posthepatectomy Liver Failure
infrequently performed. Skrzypczyk and colleagues1 should be congratulated for undertaking a comparison of posthepatectomy liver failure (PHLF) definitions and their utility in predicting clinical outcomes. The authors conclude that the International Study Group of Liver Surgery (ISGLS) PHLF definition was ‘‘less discriminatory than the ‘50–50’ and ‘PeakBili >7’ criteria in identifying patients at risk of posthepatectomy major complications or death,’’ but this does not appear to be supported by the data presented. The predictive accuracy of binary diagnostic tests can be compared in a number of ways. The Cochrane Collaboration are currently producing guidelines for comparison and meta-analysis of studies of diagnostic accuracy.2 A common initial strategy is to compare the discordance in sensitivity and specificity (using McNemar’s test), though concerns have been raised about this approach and modifications suggested.3
Morbidity POD5
Yuan-Yi Rui, MD Zong-Guang Zhou, MD, PhD, FACS Department of Gastrointestinal Surgery West China Hospital Sichuan University Chengdu, China Institute of Digestive Surgery West China Hospital Sichuan University Chengdu, China
versus extended abdominoperineal excision for low rectal cancer. Br J Surg. 2014;101:874–882. 8. Han JG, Wang ZJ, Wei GH, et al. Randomized clinical trial of conventional versus cylindrical abdominoperineal resection for locally advanced lower rectal cancer. Am J Surg. 2012;204:274–282. 9. Huang A, Zhao H, Ling T, et al. Oncological superiority of extralevator abdominoperineal resection over conventional abdominoperineal resection: a meta-analysis. Int J Colorectal Dis. 2014;29:321–327.
6
logOR
FIGURE 1. j Comparison of log odds ratios (95% confidence intervals) of 3 scoring systems in predicting morbidity and mortality at 2 time points after hepatectomy. Comparisons were performed using available data, z-tests and are corrected for the 3 contrasts made within each outcome. ß
2017 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved.
Annals of Surgery Volume 265, Number 4, April 2017
Skrzypczyk et al show that the ISGLS definition is significantly more sensitive than the 50–50 and PeakBili >7 criteria, but less specific for predicting major morbidity. An alternative approach is to determine overall predictive performance, for instance, by comparing diagnostic odds ratios or likelihood ratios.4 The authors take a similar approach but do not report a statistical comparison of results. We have compared the reported odds ratios and confidence intervals between the 3 scoring systems for predicting morbidity or mortality. Although a numerical difference between scores exists, this fails to achieve traditional measures of statistical significance (Fig. 1). Maximizing sensitivity and specificity improves overall predictive performance but is not always desirable. In PHLF, it may be useful to increase the true detection rate (sensitivity) at the expense of more false positives (specificity). This is not a population-screening tool, where the minimization of false positives is important to avoid unnecessary anxiety, diagnostic tests, and invasive procedures. We argue that a patient termed as having PHLF who does not go on to have a complication or die has few negative consequences of having been labeled such. Yet, there may be much to be gained by increasing the ‘‘pickup rate’’ of patients at risk of poor outcomes. We agree with the authors’ assertion that in this cohort all 3 definitions were poor at identifying patients who go on to have a complication or die: the association between these blood markers and outcome was not strong. We were less clear, however, as to why the 3-point grading system associated with the ISGLS score had not been used in the assessment of test performance?5 Determining the accuracy and utility of predictive scores is important and studies should be described using standardized reporting methods.6 An alternative conclusion to this study is that there is weak evidence for any difference in test performance, but that weighting true detection rate over the avoidance of false positives may be more appropriate for this condition. Disclosure: The authors declare no conflicts of interest. Ewen M. Harrison, FRCS Stephen O’Neill, MB BCh BAO Stephen J. Wigmore, FRCS O. James Garden, FRCS Department of Clinical Surgery University of Edinburgh Royal Infirmary of Edinburgh Edinburgh, UK ß
Letters to the Editor
REFERENCES 1. Skrzypczyk C, Truant S, Duhamel A, et al. Relevance of the ISGLS definition of posthepatectomy liver failure in early prediction of poor outcome after liver resection: study on 680 hepatectomies. Ann Surg. 2014;260:865–870. 2. Handbook for DTA Reviews j Diagnostic Test Accuracy Working Group [Internet]. http://srdta. cochrane.org/handbook-dta-reviews. November 21, 2014. 3. Trajman A, Luiz RR. McNemar x2 (2 test revisited: comparing sensitivity and specificity of diagnostic examinations. Scand J Clin Lab Invest. 2008;68: 77–80. 4. Nofuentes JAR, del Castillo Jde DL. Comparison of the likelihood ratios of two binary diagnostic tests in paired designs. Stat Med. 2007;26:4179– 4201. 5. Rahbari NN, Garden OJ, Padbury R, et al. Posthepatectomy liver failure: a definition and grading by the International Study Group of Liver Surgery (ISGLS). Surgery. 2011;149:713–724. 6. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ. 2003;326:41–44.
Comparison of Binary Predictive Scoring Systems of Posthepatectomy Liver Failure: A Response Reply: e thank Harrison et al for their relevant comments on our article1 that undertook a comparison of posthepatectomy liver failure (PHLF) definitions (International Study Group of Liver Surgery [ISGLS] definition2 vs 50–50 criteria3 and PeakBili >74) and their relevance in predicting clinical outcomes. On the basis of the low restrictive pattern of the ISGLS definition of PHLF (defined as an increased International Normalized Ratio and concomitant hyperbilirubinemia—according to the normal limits of the local laboratory—on or after postoperative day 5), we initially assumed that this definition would be fulfilled by too many patients to be relevant in clinical practice. Indeed within a study population of 680 patients undergoing hepatic resection, 79 patients fulfilled ISGLS definition compared with 24 for 50–50 and 44 for PeakBili greater than 7 criteria. As a consequence, ISGLS definition showed high sensitivity—namely, detecting most patients with ‘‘PHLF’’— though at the cost of a low value in identifying patients with clinically significant PHLF (ie, at risk of severe complications or death). As stated by Harrison et al, several procedures can be used to compare the predictive
W
2017 Wolters Kluwer Health, Inc. All rights reserved.
accuracy of diagnostic tests. For that purpose, the sensitivity and the specificity are usually reported and compared by the mean of an adequate statistical test. When the data are paired, one must use a test that takes into account the dependence between the data, and the McNemar x2 test is currently used for this purpose.5 Harrison et al suggested taking into account modifications of this approach with reference to Nofuentes and del Castillo.6 But in the cited article,6 this point is clearly emphasized: ‘‘In paired designs, the comparison of the sensitivity and specificity of the two diagnostic tests is carried out through the classic McNemar test.’’ This is the reason why we used this latter test. Thus, the ISGLS definition showed the highest sensitivity among the 3 scores whatever the postoperative time point (on POD5 and within 10 days) but at the price of a greatly lower positive predictive value (PPV) in comparison with the 2 other scores. Besides sensitivity/specificity and odds ratios that are statistical parameters useful in series, positive or negative predictive values are the most useful in clinical practice in predicting the risk of death or major complications (or not) for a particular patient. They have the particularity to depend on the prevalence of the event of interest. Our different results and especially the morbidity and mortality rates of the studied population are quite consistent with those reported in the literature so that we were allowed to use PPV and Negative Predictive Value. In patients with a positive score, the ISGLS definition was the least relevant to predict major complications and mortality (PPVof 49.4% and 21.8% vs 79.2% and 47.8% for 50–50 and 61.4% and 40.5% for PeakBili >7 criteria). Moreover, the relevance of ISGLS definition was not better in subgroups of patients at higher risk of PHLF (major hepatectomy, portal vein occlusion, high blood loss, vascular clamping, or cirrhosis). The multifactorial and complex nature of PHLF could partly explain the low relevance of the purely biological definitions of PHLF that reflect only the consequences of PHLF. Otherwise, Harrison et al compared our reported odds ratios and confidence intervals between the 3 scoring systems for predicting morbidity or mortality by using a ‘‘Z test’’ with a Bonferroni correction and they failed to find statistical difference. We studied the diagnostic odds ratio because the use of a measure that summarizes the diagnostic accuracy as a single value is recommended in terms of statistical method.7 We did not perform the comparisons because the diagnostic odds ratios were computed on a single sample and the data were consequently dependent. The ‘‘Z test’’ is not adapted in this case because it assumes that the data are independent. Therefore, the www.annalsofsurgery.com | e57
Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved.
Letters to the Editor
rectal cancer restaging after concurrent neoadjuvant chemo- and radiation therapy, it is better than nothing, especially considering a higher morbidity rate and a longer hospital stay after ELAPE. In summary, we conclude that ELAPE may still be effective in highly selective situations. We think the Danish investigation will not be the prohibition sign of ELPAE in routine colorectal surgical practice but a great milestone of conducting future studies to develop the guidelines for such an aggressive powerful procedure. Disclosure: The authors declare no conflicts of interest and no funding.
REFERENCES
ISGLS 0.093 1.000
50−50 0.796
PeakBili>7
ISGLS 0.068 0.499
50−50 0.730
PeakBili>7
ISGLS 0.266 0.280
50−50 1.000
PeakBili>7
ISGLS 0.527 0.284
50−50 1.000
PeakBili>7 2
4
Mortality within 10d
e56 | www.annalsofsurgery.com
A
Mortality POD5
1. Klein M, Fischer A, Rosenberg J, et al. Extralevatory abdominoperineal excision (ELAPE) does not result in reduced rate of tumor perforation or rate of positive circumferential resection margin: a nationwide database study [published online ahead of print September 10, 2014]. Ann Surg. doi:10.1097/ SLA.0000000000000910. 2. Nagtegaal ID, van de Velde CJ, Marijnen CA, et al. Low rectal cancer: a call for a change of approach in abdominoperineal resection. J Clin Oncol. 2005;23:9257–9264. 3. West NP, Anderin C, Smith KJ, et al. Multicentre experience with extralevator abdominoperineal excision for low rectal cancer. Br J Surg. 2010;97:588–599. 4. Martijnse IS, Dudink RL, West NP, et al. Focus on extralevator perineal dissection in supine position for low rectal cancer has led to better quality of surgery and oncologic outcome. Ann Surg Oncol. 2012;19:786–793. 5. de Campos-Lobato LF, Stocchi L, Dietz DW, et al. Prone or lithotomy positioning during an abdominoperineal resection for rectal cancer results in comparable oncologic outcomes. Dis Colon Rectum. 2011;54:939–946. 6. Asplund D, Haglind E, Angenete E. Outcome of extralevator abdominoperineal excision compared with standard surgery: results from a single centre. Colorectal Dis. 2012;14:1191–1196. 7. Ortiz H, Ciga MA, Armendariz P, et al. Multicentre propensity score-matched analysis of conventional
To the Editor: lthough external validation of predictive clinical scoring systems is essential, it is
Morbidity within 10d
Cun Wang, MD, PhD Lie Yang, MD, PhD Department of Gastrointestinal Surgery West China Hospital Sichuan University Chengdu, China. [email protected]
Comparison of Binary Predictive Scoring Systems of Posthepatectomy Liver Failure
infrequently performed. Skrzypczyk and colleagues1 should be congratulated for undertaking a comparison of posthepatectomy liver failure (PHLF) definitions and their utility in predicting clinical outcomes. The authors conclude that the International Study Group of Liver Surgery (ISGLS) PHLF definition was ‘‘less discriminatory than the ‘50–50’ and ‘PeakBili >7’ criteria in identifying patients at risk of posthepatectomy major complications or death,’’ but this does not appear to be supported by the data presented. The predictive accuracy of binary diagnostic tests can be compared in a number of ways. The Cochrane Collaboration are currently producing guidelines for comparison and meta-analysis of studies of diagnostic accuracy.2 A common initial strategy is to compare the discordance in sensitivity and specificity (using McNemar’s test), though concerns have been raised about this approach and modifications suggested.3
Morbidity POD5
Yuan-Yi Rui, MD Zong-Guang Zhou, MD, PhD, FACS Department of Gastrointestinal Surgery West China Hospital Sichuan University Chengdu, China Institute of Digestive Surgery West China Hospital Sichuan University Chengdu, China
versus extended abdominoperineal excision for low rectal cancer. Br J Surg. 2014;101:874–882. 8. Han JG, Wang ZJ, Wei GH, et al. Randomized clinical trial of conventional versus cylindrical abdominoperineal resection for locally advanced lower rectal cancer. Am J Surg. 2012;204:274–282. 9. Huang A, Zhao H, Ling T, et al. Oncological superiority of extralevator abdominoperineal resection over conventional abdominoperineal resection: a meta-analysis. Int J Colorectal Dis. 2014;29:321–327.
6
logOR
FIGURE 1. j Comparison of log odds ratios (95% confidence intervals) of 3 scoring systems in predicting morbidity and mortality at 2 time points after hepatectomy. Comparisons were performed using available data, z-tests and are corrected for the 3 contrasts made within each outcome. ß
2017 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved.
Annals of Surgery Volume 265, Number 4, April 2017
Skrzypczyk et al show that the ISGLS definition is significantly more sensitive than the 50–50 and PeakBili >7 criteria, but less specific for predicting major morbidity. An alternative approach is to determine overall predictive performance, for instance, by comparing diagnostic odds ratios or likelihood ratios.4 The authors take a similar approach but do not report a statistical comparison of results. We have compared the reported odds ratios and confidence intervals between the 3 scoring systems for predicting morbidity or mortality. Although a numerical difference between scores exists, this fails to achieve traditional measures of statistical significance (Fig. 1). Maximizing sensitivity and specificity improves overall predictive performance but is not always desirable. In PHLF, it may be useful to increase the true detection rate (sensitivity) at the expense of more false positives (specificity). This is not a population-screening tool, where the minimization of false positives is important to avoid unnecessary anxiety, diagnostic tests, and invasive procedures. We argue that a patient termed as having PHLF who does not go on to have a complication or die has few negative consequences of having been labeled such. Yet, there may be much to be gained by increasing the ‘‘pickup rate’’ of patients at risk of poor outcomes. We agree with the authors’ assertion that in this cohort all 3 definitions were poor at identifying patients who go on to have a complication or die: the association between these blood markers and outcome was not strong. We were less clear, however, as to why the 3-point grading system associated with the ISGLS score had not been used in the assessment of test performance?5 Determining the accuracy and utility of predictive scores is important and studies should be described using standardized reporting methods.6 An alternative conclusion to this study is that there is weak evidence for any difference in test performance, but that weighting true detection rate over the avoidance of false positives may be more appropriate for this condition. Disclosure: The authors declare no conflicts of interest. Ewen M. Harrison, FRCS Stephen O’Neill, MB BCh BAO Stephen J. Wigmore, FRCS O. James Garden, FRCS Department of Clinical Surgery University of Edinburgh Royal Infirmary of Edinburgh Edinburgh, UK ß
Letters to the Editor
REFERENCES 1. Skrzypczyk C, Truant S, Duhamel A, et al. Relevance of the ISGLS definition of posthepatectomy liver failure in early prediction of poor outcome after liver resection: study on 680 hepatectomies. Ann Surg. 2014;260:865–870. 2. Handbook for DTA Reviews j Diagnostic Test Accuracy Working Group [Internet]. http://srdta. cochrane.org/handbook-dta-reviews. November 21, 2014. 3. Trajman A, Luiz RR. McNemar x2 (2 test revisited: comparing sensitivity and specificity of diagnostic examinations. Scand J Clin Lab Invest. 2008;68: 77–80. 4. Nofuentes JAR, del Castillo Jde DL. Comparison of the likelihood ratios of two binary diagnostic tests in paired designs. Stat Med. 2007;26:4179– 4201. 5. Rahbari NN, Garden OJ, Padbury R, et al. Posthepatectomy liver failure: a definition and grading by the International Study Group of Liver Surgery (ISGLS). Surgery. 2011;149:713–724. 6. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ. 2003;326:41–44.
Comparison of Binary Predictive Scoring Systems of Posthepatectomy Liver Failure: A Response Reply: e thank Harrison et al for their relevant comments on our article1 that undertook a comparison of posthepatectomy liver failure (PHLF) definitions (International Study Group of Liver Surgery [ISGLS] definition2 vs 50–50 criteria3 and PeakBili >74) and their relevance in predicting clinical outcomes. On the basis of the low restrictive pattern of the ISGLS definition of PHLF (defined as an increased International Normalized Ratio and concomitant hyperbilirubinemia—according to the normal limits of the local laboratory—on or after postoperative day 5), we initially assumed that this definition would be fulfilled by too many patients to be relevant in clinical practice. Indeed within a study population of 680 patients undergoing hepatic resection, 79 patients fulfilled ISGLS definition compared with 24 for 50–50 and 44 for PeakBili greater than 7 criteria. As a consequence, ISGLS definition showed high sensitivity—namely, detecting most patients with ‘‘PHLF’’— though at the cost of a low value in identifying patients with clinically significant PHLF (ie, at risk of severe complications or death). As stated by Harrison et al, several procedures can be used to compare the predictive
W
2017 Wolters Kluwer Health, Inc. All rights reserved.
accuracy of diagnostic tests. For that purpose, the sensitivity and the specificity are usually reported and compared by the mean of an adequate statistical test. When the data are paired, one must use a test that takes into account the dependence between the data, and the McNemar x2 test is currently used for this purpose.5 Harrison et al suggested taking into account modifications of this approach with reference to Nofuentes and del Castillo.6 But in the cited article,6 this point is clearly emphasized: ‘‘In paired designs, the comparison of the sensitivity and specificity of the two diagnostic tests is carried out through the classic McNemar test.’’ This is the reason why we used this latter test. Thus, the ISGLS definition showed the highest sensitivity among the 3 scores whatever the postoperative time point (on POD5 and within 10 days) but at the price of a greatly lower positive predictive value (PPV) in comparison with the 2 other scores. Besides sensitivity/specificity and odds ratios that are statistical parameters useful in series, positive or negative predictive values are the most useful in clinical practice in predicting the risk of death or major complications (or not) for a particular patient. They have the particularity to depend on the prevalence of the event of interest. Our different results and especially the morbidity and mortality rates of the studied population are quite consistent with those reported in the literature so that we were allowed to use PPV and Negative Predictive Value. In patients with a positive score, the ISGLS definition was the least relevant to predict major complications and mortality (PPVof 49.4% and 21.8% vs 79.2% and 47.8% for 50–50 and 61.4% and 40.5% for PeakBili >7 criteria). Moreover, the relevance of ISGLS definition was not better in subgroups of patients at higher risk of PHLF (major hepatectomy, portal vein occlusion, high blood loss, vascular clamping, or cirrhosis). The multifactorial and complex nature of PHLF could partly explain the low relevance of the purely biological definitions of PHLF that reflect only the consequences of PHLF. Otherwise, Harrison et al compared our reported odds ratios and confidence intervals between the 3 scoring systems for predicting morbidity or mortality by using a ‘‘Z test’’ with a Bonferroni correction and they failed to find statistical difference. We studied the diagnostic odds ratio because the use of a measure that summarizes the diagnostic accuracy as a single value is recommended in terms of statistical method.7 We did not perform the comparisons because the diagnostic odds ratios were computed on a single sample and the data were consequently dependent. The ‘‘Z test’’ is not adapted in this case because it assumes that the data are independent. Therefore, the www.annalsofsurgery.com | e57
Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved.
