CE: ; ANNSURG-D-15-00531; Total nos of Pages: 1;
ANNSURG-D-15-00531
RESPONSE Statistical Tools for Propensity Score Matching Reply: e thank Wilson et al for their interest in our article and we have carefully considered their comments. We disagree with their remarks suggesting that the statistical methods employed in our article, which studies the impact of neoadjuvant chemoradiotherapy (NCRT) on postoperative outcomes after esophageal cancer resection, may lead to misinformation.1 However, we recognize that the statistical section of our article lacks detail because of word count limitation, and we thank Wilson et al for giving us the opportunity to clarify this. All variables included in the propensity score were selected a priori on the basis of their potential relationship with anastomotic leak, as indicated in the discussion. This is in accordance with Brookhart et al,2 recommending that the optimal model should include all variables related to the outcome (anastomotic leak) regardless of whether they are related to the treatment (NCRT or primary surgery). Because the propensity score is the probability of treatment assignment conditionally on observed baseline characteristics (known to be related to the outcome), it was calculated on the basis of 2080 patients (593 with NCRT); this does not constitute a small sample. Consequently, we disagree with the statement of Wilson et al, claiming that ‘‘by using all variables to potentially influence rates of AL, the authors may have introduced bias.’’ Regarding the matching method, there are 2 types of matching algorithm: local optimal or global optimal. The nearest available neighbor matching and caliper matching are the most popular local optimum algorithms. The former guarantees that a match is always found for all cases (NCRT in our study), but it is possible that the propensity scores are not close. In the caliper algorithm, the control (primary surgery in our study) is matched to a case only if the control’s pro-
W
TO
EDITOR
pensity score is within a certain distance (caliper distance). However, in this method, it is possible that a case cannot be matched to a control. As recommended by Rosenbaum3 and by Ming and Rosenbaum,4 we used the global optimal algorithm (neighbors matching without replacement). Malnutrition, emphasized as a major confounder by Wilson et al, was not included in the calculation of propensity score, as 21% of data for this variable was missing. All matched comparisons were done before and after adjustment for this confounder to minimize its confounding bias. Finally, we read with great interest the recent article on the use of bootstrap resampling methods with regard to a propensityscore matching approach.5 The motivation for this article is the debate on the choice of statistical methods that do or do not account for the matched nature of the sample for estimating the standard error of the estimated treatment effect. First, we note that this article was published in 2014 as our article was. Second, we believe that the interpretation of this article by Wilson et al is not exactly accurate. In fact, Austin and Small5 have shown that ‘‘simple bootstrap approach (bootstrapping matched pairs from the original propensity-score matched sample) tended to result in variance estimates very similar to those obtained from a parametric variance estimate that accounted for the matched nature of sample.’’ Interestingly, they have shown that a complex bootstrap approach (where bootstrap samples were drawn from the original unmatched sample and matching algorithm was applied to each bootstrap sample) tended to have inferior performance than single bootstrap approach. As a result, Austin and Small5 ‘‘suggest to use parametric-based estimators of the standard error that accounted for the matched nature of sample,’’ as used in our study. In fact, we used a generalized linear mixed model to take into account the matched pairs. So, we disagree with the last statement by Wilson et al that there is concern about the accuracy of the inferences reported in our article.
Alain Duhamel, PhD Department of Biostatistics EA 2694 University Hospital of Lille Lille, France University of Lille—Nord de France Lille, France SIRIC ONCOLille, Rue Polonovski Lille Cedex, France Julien Labreuche, Bst Department of Biostatistics EA 2694 University Hospital of Lille Lille, France Caroline Gronnier, MD, PhD University of Lille—Nord de France Lille, France Department of Digestive and Oncological Surgery, University Hospital of Lille Lille, France Christophe Mariette, MD, PhD University of Lille—Nord de France Lille, France Department of Digestive and Oncological Surgery, University Hospital of Lille Lille, France SIRIC ONCOLille, Rue Polonovski Lille Cedex, France [email protected]
REFERENCES 1. Gronnier C, Tre´chot B, Duhamel A, et al. Impact of neoadjuvant chemoradiotherapy on postoperative outcomes after esophageal cancer resection: results of a European multicenter study. Ann Surg. 2014;260:764–771. 2. Brookhart A, Schneeweiss S, Rothman KJ, et al. Variable selection for propensity score models. Am J Epidemiol. 2006;163:1149–1156. 3. Rosenbaum PR. Optimal matching for observational studies. J Am Statist Assoc. 1989;84:1024– 1032. 4. Ming K, Rosenbaum PR. A note on optimal matching with variable controls using the assignment algorithm. J Comput Graph Stat. 2001;10: 455–463. 5. Austin PC, Small DS. The use of bootstrapping when using propensity-score matching without replacement: a simulation study. Stat Med. 2014;33:4306–4319.
Disclosure: The authors declare no conflicts of interest. Copyright ß 2015 Wolters Kluwer Health, Inc. All rights reserved. ISSN: 0003-4932/14/26105-0821 DOI: 10.1097/SLA.0000000000001312
Annals of Surgery Volume XX, Number X, Month 2015
www.annalsofsurgery.com | 1
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
ANNSURG-D-15-00531
RESPONSE Statistical Tools for Propensity Score Matching Reply: e thank Wilson et al for their interest in our article and we have carefully considered their comments. We disagree with their remarks suggesting that the statistical methods employed in our article, which studies the impact of neoadjuvant chemoradiotherapy (NCRT) on postoperative outcomes after esophageal cancer resection, may lead to misinformation.1 However, we recognize that the statistical section of our article lacks detail because of word count limitation, and we thank Wilson et al for giving us the opportunity to clarify this. All variables included in the propensity score were selected a priori on the basis of their potential relationship with anastomotic leak, as indicated in the discussion. This is in accordance with Brookhart et al,2 recommending that the optimal model should include all variables related to the outcome (anastomotic leak) regardless of whether they are related to the treatment (NCRT or primary surgery). Because the propensity score is the probability of treatment assignment conditionally on observed baseline characteristics (known to be related to the outcome), it was calculated on the basis of 2080 patients (593 with NCRT); this does not constitute a small sample. Consequently, we disagree with the statement of Wilson et al, claiming that ‘‘by using all variables to potentially influence rates of AL, the authors may have introduced bias.’’ Regarding the matching method, there are 2 types of matching algorithm: local optimal or global optimal. The nearest available neighbor matching and caliper matching are the most popular local optimum algorithms. The former guarantees that a match is always found for all cases (NCRT in our study), but it is possible that the propensity scores are not close. In the caliper algorithm, the control (primary surgery in our study) is matched to a case only if the control’s pro-
W
TO
EDITOR
pensity score is within a certain distance (caliper distance). However, in this method, it is possible that a case cannot be matched to a control. As recommended by Rosenbaum3 and by Ming and Rosenbaum,4 we used the global optimal algorithm (neighbors matching without replacement). Malnutrition, emphasized as a major confounder by Wilson et al, was not included in the calculation of propensity score, as 21% of data for this variable was missing. All matched comparisons were done before and after adjustment for this confounder to minimize its confounding bias. Finally, we read with great interest the recent article on the use of bootstrap resampling methods with regard to a propensityscore matching approach.5 The motivation for this article is the debate on the choice of statistical methods that do or do not account for the matched nature of the sample for estimating the standard error of the estimated treatment effect. First, we note that this article was published in 2014 as our article was. Second, we believe that the interpretation of this article by Wilson et al is not exactly accurate. In fact, Austin and Small5 have shown that ‘‘simple bootstrap approach (bootstrapping matched pairs from the original propensity-score matched sample) tended to result in variance estimates very similar to those obtained from a parametric variance estimate that accounted for the matched nature of sample.’’ Interestingly, they have shown that a complex bootstrap approach (where bootstrap samples were drawn from the original unmatched sample and matching algorithm was applied to each bootstrap sample) tended to have inferior performance than single bootstrap approach. As a result, Austin and Small5 ‘‘suggest to use parametric-based estimators of the standard error that accounted for the matched nature of sample,’’ as used in our study. In fact, we used a generalized linear mixed model to take into account the matched pairs. So, we disagree with the last statement by Wilson et al that there is concern about the accuracy of the inferences reported in our article.
Alain Duhamel, PhD Department of Biostatistics EA 2694 University Hospital of Lille Lille, France University of Lille—Nord de France Lille, France SIRIC ONCOLille, Rue Polonovski Lille Cedex, France Julien Labreuche, Bst Department of Biostatistics EA 2694 University Hospital of Lille Lille, France Caroline Gronnier, MD, PhD University of Lille—Nord de France Lille, France Department of Digestive and Oncological Surgery, University Hospital of Lille Lille, France Christophe Mariette, MD, PhD University of Lille—Nord de France Lille, France Department of Digestive and Oncological Surgery, University Hospital of Lille Lille, France SIRIC ONCOLille, Rue Polonovski Lille Cedex, France [email protected]
REFERENCES 1. Gronnier C, Tre´chot B, Duhamel A, et al. Impact of neoadjuvant chemoradiotherapy on postoperative outcomes after esophageal cancer resection: results of a European multicenter study. Ann Surg. 2014;260:764–771. 2. Brookhart A, Schneeweiss S, Rothman KJ, et al. Variable selection for propensity score models. Am J Epidemiol. 2006;163:1149–1156. 3. Rosenbaum PR. Optimal matching for observational studies. J Am Statist Assoc. 1989;84:1024– 1032. 4. Ming K, Rosenbaum PR. A note on optimal matching with variable controls using the assignment algorithm. J Comput Graph Stat. 2001;10: 455–463. 5. Austin PC, Small DS. The use of bootstrapping when using propensity-score matching without replacement: a simulation study. Stat Med. 2014;33:4306–4319.
Disclosure: The authors declare no conflicts of interest. Copyright ß 2015 Wolters Kluwer Health, Inc. All rights reserved. ISSN: 0003-4932/14/26105-0821 DOI: 10.1097/SLA.0000000000001312
Annals of Surgery Volume XX, Number X, Month 2015
www.annalsofsurgery.com | 1
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
