LETTER TO Considerations in the Early Termination of Clinical Trials in Surgery To the Editor: thics and statistics collide in decisions relating to the early termination of clinical trials. Investigators have a fundamental responsibility to stop a trial where an excess of harm is seen in one of the arms. Decisions on stopping are not straightforward and must balance the potential risk to trial patients against the likelihood that in fact there is no difference in outcome between groups. Indeed, in early termination, the potential loss of generalizable knowledge may itself harm future patients. We therefore read with interest the article by Van Buren and colleagues1 and congratulate the authors on the first multicenter randomized trial on the controversial topic of surgical drains after pancreaticoduodenectomy. As the authors report, the trial was stopped by the Data Safety Monitoring Board after only 18% recruitment due to a numerical excess of deaths in the ‘‘no-drain’’ arm. We would be interested in learning from the process that led to the decision to terminate the trial. A common method to monitor adverse events advocated by the CONSORT group is to define formal sequential stopping rules based on the limit of acceptable adverse event rates.2 These guidelines suggest that authors report the number of planned ‘‘looks’’ at the data, the statistical methods used including any formal stopping rules, and whether these were planned before trial commencement. This information is often not included in published trial reports, even when early termination has occurred.3 We feel that in the context of important surgical trials, these guidelines should be adhered to. Early termination can reduce the statistical power of a trial. This can be addressed by examining results as data accumulate, preferably by an independent data monitoring committee. However, performing multiple statistical examinations of accumulating data without appropriate correction can lead to erroneous results and interpretation.4 For example, if accumulating data from a trial are examined at 5 interim analyses that use a P value of 0.05, the overall false-positive rate is nearer to 19% than to the nominal 5%. Several group sequential statistical methods are available to adjust for multiple
E
Disclosure: The authors declare no conflicts of interest. Copyright ß 2015 Wolters Kluwer Health, Inc. All rights reserved. ISSN: 0003-4932/14/26105-0821 DOI: 10.1097/SLA.0000000000001000
e74 | www.annalsofsurgery.com
THE
EDITOR
analyses5,6 and their use should be prespecified in the trial protocol. Stopping rules may be formed by 2 broad methods, either using a Bayesian approach to evaluate the proportion of patients with adverse effects or using a hypothesis testing approach with a sequential probability ratio test to determine whether the acceptable adverse effects rate has been exceeded. Data are compared at each interim analysis and decisions based on prespecified criteria. As an example, stopping rules for harm from a recent study used modified Haybittle-Peto boundaries of 3 SDs in the first half of the study and 2 SDs in the second half.7 The study of Van Buren and colleagues is reported to have been stopped after 18% recruitment due to an excess of 6 deaths in the ‘‘no-drain’’ arm. The relative risk of death at 90 days in the ‘‘no-drain’’ group versus the ‘‘drain’’ group was 3.94 (95% confidence interval, 0.87–17.90), equivalent to a difference of 1.78 SD. The primary outcome measure was any grade 2 complication or more and had a relative risk of 1.32 (5% confidence interval, 1.00–1.75), or 1.95 SD. The decision to terminate a trial early is not based on statistics alone. Judgments must be made using all the available evidence, including the biological and clinical plausibility of harm and the findings of previous studies. Statistical considerations should therefore be used as a starting point for decisions, rather than a definitive rule. The Data Safety Monitoring Board for the study of Van Buren and colleagues clearly felt that there was no option other than to terminate the trial. However, at least on statistical grounds, this occurred very early in the trial using conservative criteria. The question remains therefore is the totality of evidence convincing that the question posed has been unequivocally answered? We would suggest that this is not the case. In general terms, stopping a clinical trial early is a rare event that sends out a message that, because of the ‘‘sensational’’ effect, may have greater impact on the medical community than intended, making future studies in that area challenging. Tom K. Gallagher, MCh, FRCSI O. James Garden, FRCS Stephen J. Wigmore, MD, FRCS Ewen M. Harrison, PhD, FRCS Hepatobiliary and Transplant Surgery Royal Infirmary Edinburgh Edinburgh, England [email protected]
REFERENCES 1. Van Buren G, Bloomston M, Hughes SJ, et al. A randomised prospective multicenter trial of
2.
3.
4.
5. 6.
7.
pancreaticoduodenectomy with and without routine intraperitoneal drainage. Ann Surg. 2014;259:605– 612. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trial. BMJ. 2010;340:c869. Montori VM, Devereaux PJ, Adhikari NK, et al. Randomized trials stopped early for benefit: a systematic review. JAMA. 2005;294:2203–2209. Geller NL, Pocock SJ. Interim analyses in randomized clinical trials: ramifications and guidelines for practitioners. Biometrics. 1987;43:213–223. Pocock SJ. When to stop a clinical trial. BMJ. 1992;305:235–240. Berry DA. Interim analyses in clinical trials: classical vs. Bayesian approaches. Stat Med. 1985;4:521– 526. Connolly SJ, Pogue J, Hart RG, et al. Effect of clopidogrel added to aspirin in patients with atrial fibrillation. N Engl J Med. 2009;360:2066–2078.
Pancreaticoduodenectomy Without Drains: Interpretation of the Evidence Reply: e would like to thank the authors for their insightful and thoughtful responses to our article, ‘‘A Randomized Prospective Multicenter Trial of Pancreaticoduodenectomy With and Without Routine Intraperitoneal Drainage.’’1 We are pleased that the surgical community has taken such interest in our contribution to the literature. There has been a wide range of opinions on performing a multicenter randomized controlled trial of pancreatectomy without drains and the subsequent interpretations of the results. We have responded to each of these letters to address the salient points in each. Dr Bohara questions whether it is ethical to randomize patients with known risk factors for postoperative fistula to a no-drain group and subject them to possibly increased morbidity and mortality. As the commenter points out, before our study, there was a growing body of literature implying that there was no adverse outcome to performing pancreaticoduodenectomy without routine intraperitoneal drainage. The author’s concern reflects a bias toward drains not previously supported by the literature before our trial. A growing number of experienced surgeons who were adopting a no-drain strategy were biased to believe that a
W
Disclosure: The authors declare no conflicts of interest. Copyright ß 2015 Wolters Kluwer Health, Inc. All rights reserved. ISSN: 0003-4932/14/26105-0821 DOI: 10.1097/SLA.0000000000001152
Annals of Surgery Volume 263, Number 5, May 2016
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
E
Disclosure: The authors declare no conflicts of interest. Copyright ß 2015 Wolters Kluwer Health, Inc. All rights reserved. ISSN: 0003-4932/14/26105-0821 DOI: 10.1097/SLA.0000000000001000
e74 | www.annalsofsurgery.com
THE
EDITOR
analyses5,6 and their use should be prespecified in the trial protocol. Stopping rules may be formed by 2 broad methods, either using a Bayesian approach to evaluate the proportion of patients with adverse effects or using a hypothesis testing approach with a sequential probability ratio test to determine whether the acceptable adverse effects rate has been exceeded. Data are compared at each interim analysis and decisions based on prespecified criteria. As an example, stopping rules for harm from a recent study used modified Haybittle-Peto boundaries of 3 SDs in the first half of the study and 2 SDs in the second half.7 The study of Van Buren and colleagues is reported to have been stopped after 18% recruitment due to an excess of 6 deaths in the ‘‘no-drain’’ arm. The relative risk of death at 90 days in the ‘‘no-drain’’ group versus the ‘‘drain’’ group was 3.94 (95% confidence interval, 0.87–17.90), equivalent to a difference of 1.78 SD. The primary outcome measure was any grade 2 complication or more and had a relative risk of 1.32 (5% confidence interval, 1.00–1.75), or 1.95 SD. The decision to terminate a trial early is not based on statistics alone. Judgments must be made using all the available evidence, including the biological and clinical plausibility of harm and the findings of previous studies. Statistical considerations should therefore be used as a starting point for decisions, rather than a definitive rule. The Data Safety Monitoring Board for the study of Van Buren and colleagues clearly felt that there was no option other than to terminate the trial. However, at least on statistical grounds, this occurred very early in the trial using conservative criteria. The question remains therefore is the totality of evidence convincing that the question posed has been unequivocally answered? We would suggest that this is not the case. In general terms, stopping a clinical trial early is a rare event that sends out a message that, because of the ‘‘sensational’’ effect, may have greater impact on the medical community than intended, making future studies in that area challenging. Tom K. Gallagher, MCh, FRCSI O. James Garden, FRCS Stephen J. Wigmore, MD, FRCS Ewen M. Harrison, PhD, FRCS Hepatobiliary and Transplant Surgery Royal Infirmary Edinburgh Edinburgh, England [email protected]
REFERENCES 1. Van Buren G, Bloomston M, Hughes SJ, et al. A randomised prospective multicenter trial of
2.
3.
4.
5. 6.
7.
pancreaticoduodenectomy with and without routine intraperitoneal drainage. Ann Surg. 2014;259:605– 612. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trial. BMJ. 2010;340:c869. Montori VM, Devereaux PJ, Adhikari NK, et al. Randomized trials stopped early for benefit: a systematic review. JAMA. 2005;294:2203–2209. Geller NL, Pocock SJ. Interim analyses in randomized clinical trials: ramifications and guidelines for practitioners. Biometrics. 1987;43:213–223. Pocock SJ. When to stop a clinical trial. BMJ. 1992;305:235–240. Berry DA. Interim analyses in clinical trials: classical vs. Bayesian approaches. Stat Med. 1985;4:521– 526. Connolly SJ, Pogue J, Hart RG, et al. Effect of clopidogrel added to aspirin in patients with atrial fibrillation. N Engl J Med. 2009;360:2066–2078.
Pancreaticoduodenectomy Without Drains: Interpretation of the Evidence Reply: e would like to thank the authors for their insightful and thoughtful responses to our article, ‘‘A Randomized Prospective Multicenter Trial of Pancreaticoduodenectomy With and Without Routine Intraperitoneal Drainage.’’1 We are pleased that the surgical community has taken such interest in our contribution to the literature. There has been a wide range of opinions on performing a multicenter randomized controlled trial of pancreatectomy without drains and the subsequent interpretations of the results. We have responded to each of these letters to address the salient points in each. Dr Bohara questions whether it is ethical to randomize patients with known risk factors for postoperative fistula to a no-drain group and subject them to possibly increased morbidity and mortality. As the commenter points out, before our study, there was a growing body of literature implying that there was no adverse outcome to performing pancreaticoduodenectomy without routine intraperitoneal drainage. The author’s concern reflects a bias toward drains not previously supported by the literature before our trial. A growing number of experienced surgeons who were adopting a no-drain strategy were biased to believe that a
W
Disclosure: The authors declare no conflicts of interest. Copyright ß 2015 Wolters Kluwer Health, Inc. All rights reserved. ISSN: 0003-4932/14/26105-0821 DOI: 10.1097/SLA.0000000000001152
Annals of Surgery Volume 263, Number 5, May 2016
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
