The
n e w e ng l a n d j o u r na l
Because claims data lack detailed clinical information, I cannot ascertain whether the decrease in active surveillance among self-referring urologists was associated with potentially inappropriate treatment. Jacobs et al. question whether the low rate of IMRT use in the control groups may have overstated the magnitude of the change in IMRT use. They report findings from SEER–Medicare showing 22% IMRT use in 2005 and 28% in 2007. Comparing results from my article with IMRT use according to SEER–Medicare data is problematic for several reasons. First, many of the self-referring groups and matched controls in my study were located in states that do not participate in the SEER–Medicare program. These states include New York, Pennsylvania, Maryland, Virginia, Florida, Texas, Indiana, Illinois, Colorado, Texas, Alabama, and Oregon. Thus, rates of use derived from SEER–Medicare are not based on a geographically representative sample. Second, the SEER–Medicare use rates do not control for financial incentives associated with IMRT ownership. Third, my study distinguished among five possible primary treatment options for newly diagnosed nonmetastatic prostate cancer: IMRT
of
m e dic i n e
only, brachytherapy, prostatectomy, hormone therapy, and active surveillance. These categories are mutually exclusive. Some patients who underwent either brachytherapy or prostatectomy also received IMRT as adjuvant therapy, but it was not the primary treatment. Most patients who received IMRT as adjuvant therapy typically received 10 to 15 treatments. In contrast, patients in the IMRT-only group received 35 or more IMRT treatments The higher rates of IMRT use based on the SEER data probably do not distinguish between whether IMRT was used a primary treatment or an adjuvant treatment. Finally, in my study, identical criteria were applied to classify both the self-referring and non–self-referring groups according to type of treatment. Therefore, if there was any selection effect, it would apply equally to both self-referring and non–selfreferring groups. Jean M. Mitchell, Ph.D. Georgetown University Washington, DC [email protected] Since publication of her article, the author reports no further potential conflict of interest. DOI: 10.1056/NEJMc1314524
Effects of Bracing in Adolescents with Idiopathic Scoliosis To the Editor: Weinstein and colleagues (Oct. 17 issue)1 reported a study evaluating the effect of bracing to decrease curve progression in patients with adolescent idiopathic scoliosis. In the primary analysis, the authors combined data from randomized and preference cohorts and performed a propensity-score analysis to evaluate the effect of bracing. The resulting estimated effect in the primary analysis, which included both cohorts (odds ratio for a successful outcome, 1.93), appears quite different, at least numerically, from that in the randomized cohort alone (odds ratio, 4.11). This suggests either that the model used to create the propensity score may not appropriately capture factors affecting the decision to use bracing or not in each of the two cohorts, or that the underlying true effect of bracing might differ between the two cohorts. In our opinion, the two cohorts should be analyzed separately, providing one odds ratio for the ran-
680
domized cohort (which was reported) and one for the preference cohort (not reported). Then, with the use of meta-analysis, these two estimates might be combined with appropriate caution if there are no concerns about the heterogeneity of the effect between cohorts, or if there are such concerns, potential sources of heterogeneity should be evaluated. Hajime Uno, Ph.D. Lee-Jen Wei, Ph.D. Michael Hughes, Ph.D. Harvard University Boston, MA [email protected] No potential conflict of interest relevant to this letter was reported. 1. Weinstein SL, Dolan LA, Wright JG, Dobbs MB. Effects of
bracing in adolescents with idiopathic scoliosis. N Engl J Med 2013;369:1512-21.
DOI: 10.1056/NEJMc1314229
n engl j med 370;7 nejm.org february 13, 2014
The New England Journal of Medicine Downloaded from nejm.org at NYU WASHINGTON SQUARE CAMPUS on May 31, 2015. For personal use only. No other uses without permission. Copyright © 2014 Massachusetts Medical Society. All rights reserved.
correspondence
The Authors Reply: Uno and colleagues suggest that the odds ratio from the primary analysis cohort did not approximate that from the randomized cohort, possibly because selection bias was not removed through propensity-score adjustment, because the treatment effect was not the same in the two cohorts, or both. As acknowledged in the article, propensityscore adjustment reduces but may not eliminate selection bias. Furthermore, there are other possible reasons for the difference in these estimates. The odds ratio from the randomized cohort is an unadjusted estimate of the effect of receiving a prescription for a brace, whereas the odds ratio from the primary analysis is an adjusted estimate of the effect of actually receiving a brace. Two covariates were included in the primary analysis: the propensity score and the length of follow-up. Length of follow-up (time between consent and study end point) is a function of multiple interacting variables, including baseline factors contributing to the rate of curve progression and maturation (i.e., sex, maturity, Cobb angle, and curve type) and their interaction with treatment factors (including the “dose” of bracing received). Thus, it is perhaps not surprising that the estimates of treatment effect from the primary analysis and the randomized analysis
would differ, even if all selection bias had been removed. In response to their inquiry: we calculated odds ratios associated with receiving a prescription for a brace in both the primary (combined cohort) and preference cohorts, adjusting for propensity scores but not length of follow-up or other covariates. The odds ratio in the combined cohort was 2.45 (95% confidence interval [CI], 1.40 to 4.29), and the odds ratio in the preference cohort was 2.68 (95% CI, 1.01 to 7.11). Thus, although there remains some variation depending on which cohort and which model is used, these findings also affirm the study conclusion that bracing reduces the rates of progression of scoliosis. Lori A. Dolan, Ph.D. University of Iowa Iowa City, IA
James G. Wright, M.D., M.P.H. Hospital for Sick Children Toronto, ON, Canada
Stuart L. Weinstein, M.D. University of Iowa Iowa City, IA [email protected] Since publication of their article, the authors report no further potential conflict of interest. DOI: 10.1056/NEJMc1314229
The Randomized Registry Trial To the Editor: Lauer and D’Agostino (Oct. 24 issue)1 noted the “intellectual trap” between randomized trials that lack external validity (generalizability) and observational studies that lack internal validity owing to unmeasured confounders. They propose registry-based randomized trials to achieve both internal and external validity. Unfortunately, such trials are not feasible if there is no registry or if randomization is unacceptable. For short-term end points, a much simpler, although less rigorous, alternative is the paired availability design.2-4 This method can be used to estimate treatment effect through comparison of outcomes in all patients before and after increased availability of the new treatment, thereby avoiding treatment-selection bias from unmeasured confounders. The paired availability design increases generalizability by including multiple
centers, with appropriate adjustment for different availabilities of new treatment among centers. The main requirements are geographically or institutionally isolated centers, stable care other than the specific treatment intervention under study, and reasonable assumptions about the receipt of treatment if availability has changed. In a study of the effect of epidural analgesia on the rate of cesarean section, the results of a paired availability design agreed with those of a meta-analysis of randomized trials and differed from those of a multivariable observational study.4 Stuart G. Baker, Sc.D. Barnett S. Kramer, M.D. National Institutes of Health Bethesda, MD [email protected]
n engl j med 370;7 nejm.org february 13, 2014
681
The New England Journal of Medicine Downloaded from nejm.org at NYU WASHINGTON SQUARE CAMPUS on May 31, 2015. For personal use only. No other uses without permission. Copyright © 2014 Massachusetts Medical Society. All rights reserved.
n e w e ng l a n d j o u r na l
Because claims data lack detailed clinical information, I cannot ascertain whether the decrease in active surveillance among self-referring urologists was associated with potentially inappropriate treatment. Jacobs et al. question whether the low rate of IMRT use in the control groups may have overstated the magnitude of the change in IMRT use. They report findings from SEER–Medicare showing 22% IMRT use in 2005 and 28% in 2007. Comparing results from my article with IMRT use according to SEER–Medicare data is problematic for several reasons. First, many of the self-referring groups and matched controls in my study were located in states that do not participate in the SEER–Medicare program. These states include New York, Pennsylvania, Maryland, Virginia, Florida, Texas, Indiana, Illinois, Colorado, Texas, Alabama, and Oregon. Thus, rates of use derived from SEER–Medicare are not based on a geographically representative sample. Second, the SEER–Medicare use rates do not control for financial incentives associated with IMRT ownership. Third, my study distinguished among five possible primary treatment options for newly diagnosed nonmetastatic prostate cancer: IMRT
of
m e dic i n e
only, brachytherapy, prostatectomy, hormone therapy, and active surveillance. These categories are mutually exclusive. Some patients who underwent either brachytherapy or prostatectomy also received IMRT as adjuvant therapy, but it was not the primary treatment. Most patients who received IMRT as adjuvant therapy typically received 10 to 15 treatments. In contrast, patients in the IMRT-only group received 35 or more IMRT treatments The higher rates of IMRT use based on the SEER data probably do not distinguish between whether IMRT was used a primary treatment or an adjuvant treatment. Finally, in my study, identical criteria were applied to classify both the self-referring and non–self-referring groups according to type of treatment. Therefore, if there was any selection effect, it would apply equally to both self-referring and non–selfreferring groups. Jean M. Mitchell, Ph.D. Georgetown University Washington, DC [email protected] Since publication of her article, the author reports no further potential conflict of interest. DOI: 10.1056/NEJMc1314524
Effects of Bracing in Adolescents with Idiopathic Scoliosis To the Editor: Weinstein and colleagues (Oct. 17 issue)1 reported a study evaluating the effect of bracing to decrease curve progression in patients with adolescent idiopathic scoliosis. In the primary analysis, the authors combined data from randomized and preference cohorts and performed a propensity-score analysis to evaluate the effect of bracing. The resulting estimated effect in the primary analysis, which included both cohorts (odds ratio for a successful outcome, 1.93), appears quite different, at least numerically, from that in the randomized cohort alone (odds ratio, 4.11). This suggests either that the model used to create the propensity score may not appropriately capture factors affecting the decision to use bracing or not in each of the two cohorts, or that the underlying true effect of bracing might differ between the two cohorts. In our opinion, the two cohorts should be analyzed separately, providing one odds ratio for the ran-
680
domized cohort (which was reported) and one for the preference cohort (not reported). Then, with the use of meta-analysis, these two estimates might be combined with appropriate caution if there are no concerns about the heterogeneity of the effect between cohorts, or if there are such concerns, potential sources of heterogeneity should be evaluated. Hajime Uno, Ph.D. Lee-Jen Wei, Ph.D. Michael Hughes, Ph.D. Harvard University Boston, MA [email protected] No potential conflict of interest relevant to this letter was reported. 1. Weinstein SL, Dolan LA, Wright JG, Dobbs MB. Effects of
bracing in adolescents with idiopathic scoliosis. N Engl J Med 2013;369:1512-21.
DOI: 10.1056/NEJMc1314229
n engl j med 370;7 nejm.org february 13, 2014
The New England Journal of Medicine Downloaded from nejm.org at NYU WASHINGTON SQUARE CAMPUS on May 31, 2015. For personal use only. No other uses without permission. Copyright © 2014 Massachusetts Medical Society. All rights reserved.
correspondence
The Authors Reply: Uno and colleagues suggest that the odds ratio from the primary analysis cohort did not approximate that from the randomized cohort, possibly because selection bias was not removed through propensity-score adjustment, because the treatment effect was not the same in the two cohorts, or both. As acknowledged in the article, propensityscore adjustment reduces but may not eliminate selection bias. Furthermore, there are other possible reasons for the difference in these estimates. The odds ratio from the randomized cohort is an unadjusted estimate of the effect of receiving a prescription for a brace, whereas the odds ratio from the primary analysis is an adjusted estimate of the effect of actually receiving a brace. Two covariates were included in the primary analysis: the propensity score and the length of follow-up. Length of follow-up (time between consent and study end point) is a function of multiple interacting variables, including baseline factors contributing to the rate of curve progression and maturation (i.e., sex, maturity, Cobb angle, and curve type) and their interaction with treatment factors (including the “dose” of bracing received). Thus, it is perhaps not surprising that the estimates of treatment effect from the primary analysis and the randomized analysis
would differ, even if all selection bias had been removed. In response to their inquiry: we calculated odds ratios associated with receiving a prescription for a brace in both the primary (combined cohort) and preference cohorts, adjusting for propensity scores but not length of follow-up or other covariates. The odds ratio in the combined cohort was 2.45 (95% confidence interval [CI], 1.40 to 4.29), and the odds ratio in the preference cohort was 2.68 (95% CI, 1.01 to 7.11). Thus, although there remains some variation depending on which cohort and which model is used, these findings also affirm the study conclusion that bracing reduces the rates of progression of scoliosis. Lori A. Dolan, Ph.D. University of Iowa Iowa City, IA
James G. Wright, M.D., M.P.H. Hospital for Sick Children Toronto, ON, Canada
Stuart L. Weinstein, M.D. University of Iowa Iowa City, IA [email protected] Since publication of their article, the authors report no further potential conflict of interest. DOI: 10.1056/NEJMc1314229
The Randomized Registry Trial To the Editor: Lauer and D’Agostino (Oct. 24 issue)1 noted the “intellectual trap” between randomized trials that lack external validity (generalizability) and observational studies that lack internal validity owing to unmeasured confounders. They propose registry-based randomized trials to achieve both internal and external validity. Unfortunately, such trials are not feasible if there is no registry or if randomization is unacceptable. For short-term end points, a much simpler, although less rigorous, alternative is the paired availability design.2-4 This method can be used to estimate treatment effect through comparison of outcomes in all patients before and after increased availability of the new treatment, thereby avoiding treatment-selection bias from unmeasured confounders. The paired availability design increases generalizability by including multiple
centers, with appropriate adjustment for different availabilities of new treatment among centers. The main requirements are geographically or institutionally isolated centers, stable care other than the specific treatment intervention under study, and reasonable assumptions about the receipt of treatment if availability has changed. In a study of the effect of epidural analgesia on the rate of cesarean section, the results of a paired availability design agreed with those of a meta-analysis of randomized trials and differed from those of a multivariable observational study.4 Stuart G. Baker, Sc.D. Barnett S. Kramer, M.D. National Institutes of Health Bethesda, MD [email protected]
n engl j med 370;7 nejm.org february 13, 2014
681
The New England Journal of Medicine Downloaded from nejm.org at NYU WASHINGTON SQUARE CAMPUS on May 31, 2015. For personal use only. No other uses without permission. Copyright © 2014 Massachusetts Medical Society. All rights reserved.
