Menopause: The Journal of The North American Menopause Society Vol. 21, No. 8, pp. 786/787 DOI: 10.1097/gme.0000000000000289 * 2014 by The North American Menopause Society
EDITORIAL But does it work in women like me?
n this issue of Menopause, Kornstein et al1 present the results of a post hoc analysis of a 10-week, double-blind, placebo-controlled randomized trial that assessed the efficacy of desvenlafaxine 50 mg/day among perimenopausal and postmenopausal women (aged 40-70 y) with major depressive disorder. The analysis examined whether drug effects were different between perimenopausal (n = 135) and postmenopausal (n = 291) women. The primary outcome was change from baseline in 17-item Hamilton Depression Rating Scale scores. Secondary outcomes included scores on the Sheehan Disability Scale, the Menopause Rating Scale, and the Clinical Global Impressions scale. Kornstein et al1 reported that among perimenopausal and postmenopausal women, 17-item Hamilton Depression Rating Scale scores improved significantly more among women assigned to desvenlafaxine than among those assigned to placebo, although the differences were modest. Compared with placebo, desvenlafaxine was associated with significant improvements in the Sheehan Disability Scale only among perimenopausal women and was associated with improvements in Menopause Rating Scale and Clinical Global Impressions scores only among postmenopausal women. What do post hoc subgroup analyses such as these contribute? Critics argue that subgroup analyses are almost always underpowered and sometimes misguided. Most trials are funded to have sufficient size and power to answer only the primary questions. For example, in the study by Kornstein et al,1 the number of African-American women was too small for subgroup analysis even though there is interest in whether there are differences in response by race/ethnicity. Furthermore, randomization may be compromised in subgroup analyses unless randomization is stratified by subgroups of interest, which was not the case in this study. Nevertheless, the value of wellconducted subgroup analyses lies in suggesting areas of heterogeneous responses to an intervention and in informing answers to the question that individuals ask, BDoes this study apply to people like me?[ For many reasons, including stringent entry criteria and the willingness and ability of persons to subject themselves to the rigors of study participation, participants in randomized trials are typically a highly select group and are not representative of the general population. Their rigor is therefore both a strength and a weakness of randomized controlled trials (RCTs). The old and the young, persons with comorbid conditions, those of lower socioeconomic class, those likely to be less compliant with therapy, and, until recently, women (particularly premenopausal women) were underrepresented in or absent from most
I
786
large RCTs. This leaves individuals and clinicians to do the best they can with the incomplete information at hand. There are legitimate concerns that subgroup analyses may find an absence of effect when they are underpowered, delaying care in some subgroups until later, adequately powered studies show that the therapy is indeed effective.2 A second concern is that if numerous post hoc analyses are conducted, those that reach significance gather attention.2 Because of these concerns, it is suggested that subgroup analyses be hypothesis driven, defined a priori, limited to a few clinically important questions, and adequately powered.2 Stratified randomization should be considered for groups of highest importance (eg, age, race/ ethnicity, menopause status, or clinically important subgroups). It is also suggested that tests for interaction be used to evaluate subgroup effects because of their rigor and that findings of heterogeneity of response be considered cautiously and confirmed by subsequent trials.3 When these guidelines are applied, subgroup analyses can provide valuable information to guide future studies and clinical care.4 Kornstein et al1 had reason to suspect that responses to antidepressants might differ by age and menopause status; thus, their analysis was hypothesis driven, although they did not say if the decision to conduct the analysis was made during the design of the original trial. Unfortunately, it is probable that many studies suggested by subgroup analyses will never be performed owing to cost and feasibility. Reactions to this situation were demonstrated in a recent series of focus groups at Group Health conducted among women aged 40 to 55 years who were using estrogen for menopause symptoms. They shared their deep frustration that trial results were being used to guide their care even though they believed that the trials did not include women like them. A response to concerns about the lack of generalizability in RCTs is the growing interest in pragmatic clinical trials,5,6 in comparative effectiveness research,7 and in involving stakeholders in study design and execution.6,7 Pragmatic trials focus on whether a program or intervention works under usualVrather than idealVconditions. Pragmatic studies differ from efficacy studies in that they take the perspective of policy makers, practitioners, and individuals; are conducted in heterogeneous realworld settings; minimize exclusion criteria; and use comparisons based on real-world alternatives such as the current standard of care.7 This approach leads to more broadly applicable findings. The Patient-Centered Outcomes Research Institute is pushing this point further by insisting that the comparative effectiveness studies it funds engage and collaborate with individuals and other stakeholders throughout the research process, from
Menopause, Vol. 21, No. 8, 2014
Copyright © 2014 The North American Menopause Society. Unauthorized reproduction of this article is prohibited.
EDITORIAL
defining the study question through dissemination of results.6,7 The research is designed to examine outcomes that are most relevant to individuals, caregivers, and clinicians.7 Pragmatic trials and comparative effectiveness studies may be more likely to provide answers to the question, BDoes it work in people like me?[ Returning to Kornstein et al,1 their study is important because new and recurrent mood disorders are associated with the menopausal transition.8,9 This study confirmed the benefit of desvenlafaxine 50 mg/day in perimenopausal and postmenopausal women. Understanding these selective responses will help guide therapy for women on both sides of the menopausal transition.
2.
3.
4. 5.
6.
Financial disclosure/conflicts of interest: None reported. 7.
Katherine M. Newton, PhD Group Health Research Institute Seattle, Washington
8.
REFERENCES 9. 1. Kornstein SG, Clayton A, Bao W, Guico-Pabia CJ. Post hoc analysis of the efficacy and safety of desvenlafaxine 50 mg/day in a randomized, placebo-
controlled study of perimenopausal and postmenopausal women with major depressive disorder. Menopause 2014;21:799-806. Rothwell PM. Treating individuals 2. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet 2005; 365:176-186. Pocock SJ, Assmann SE, Enos LE, Kasten LE. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med 2002;21:2917-2930. Feinstein AR. The problem of cogent subgroups: a clinicostatistical tragedy. J Clin Epidemiol 1998;51:297-299. Glasgow R. What does it mean to be pragmatic? Pragmatic methods, measures, and models to facilitate research translation. Health Educ Behav 2013; 40:257-267. Selby J. Interview: Patient-Centered Outcomes Research Institute seeks to find out what works best by involving Fend-users_ from the beginning. J Comp Eff Res 2014;3:125-129. Fluerence R, Selby J, Odom-Walker K, et al. How the Patient-Centered Outcomes Research Institute is engaging patients and others in shaping its research agenda. Health Aff 2013;32:2393-2400. Cohen LS, Soares CN, Vitonis AF, Otto MW, Harlow BL. Risk for new onset of depression during the menopausal transition: the Harvard Study of Moods and Cycles. Arch Gen Psychiatry 2006;63:385-390. Freeman EW, Sammel MD, Boorman DW, Zhang R. Longitudinal pattern of depressive symptoms around natural menopause. JAMA Psychiatry 2014; 71:36-43.
Menopause, Vol. 21, No. 8, 2014
Copyright © 2014 The North American Menopause Society. Unauthorized reproduction of this article is prohibited.
787
EDITORIAL But does it work in women like me?
n this issue of Menopause, Kornstein et al1 present the results of a post hoc analysis of a 10-week, double-blind, placebo-controlled randomized trial that assessed the efficacy of desvenlafaxine 50 mg/day among perimenopausal and postmenopausal women (aged 40-70 y) with major depressive disorder. The analysis examined whether drug effects were different between perimenopausal (n = 135) and postmenopausal (n = 291) women. The primary outcome was change from baseline in 17-item Hamilton Depression Rating Scale scores. Secondary outcomes included scores on the Sheehan Disability Scale, the Menopause Rating Scale, and the Clinical Global Impressions scale. Kornstein et al1 reported that among perimenopausal and postmenopausal women, 17-item Hamilton Depression Rating Scale scores improved significantly more among women assigned to desvenlafaxine than among those assigned to placebo, although the differences were modest. Compared with placebo, desvenlafaxine was associated with significant improvements in the Sheehan Disability Scale only among perimenopausal women and was associated with improvements in Menopause Rating Scale and Clinical Global Impressions scores only among postmenopausal women. What do post hoc subgroup analyses such as these contribute? Critics argue that subgroup analyses are almost always underpowered and sometimes misguided. Most trials are funded to have sufficient size and power to answer only the primary questions. For example, in the study by Kornstein et al,1 the number of African-American women was too small for subgroup analysis even though there is interest in whether there are differences in response by race/ethnicity. Furthermore, randomization may be compromised in subgroup analyses unless randomization is stratified by subgroups of interest, which was not the case in this study. Nevertheless, the value of wellconducted subgroup analyses lies in suggesting areas of heterogeneous responses to an intervention and in informing answers to the question that individuals ask, BDoes this study apply to people like me?[ For many reasons, including stringent entry criteria and the willingness and ability of persons to subject themselves to the rigors of study participation, participants in randomized trials are typically a highly select group and are not representative of the general population. Their rigor is therefore both a strength and a weakness of randomized controlled trials (RCTs). The old and the young, persons with comorbid conditions, those of lower socioeconomic class, those likely to be less compliant with therapy, and, until recently, women (particularly premenopausal women) were underrepresented in or absent from most
I
786
large RCTs. This leaves individuals and clinicians to do the best they can with the incomplete information at hand. There are legitimate concerns that subgroup analyses may find an absence of effect when they are underpowered, delaying care in some subgroups until later, adequately powered studies show that the therapy is indeed effective.2 A second concern is that if numerous post hoc analyses are conducted, those that reach significance gather attention.2 Because of these concerns, it is suggested that subgroup analyses be hypothesis driven, defined a priori, limited to a few clinically important questions, and adequately powered.2 Stratified randomization should be considered for groups of highest importance (eg, age, race/ ethnicity, menopause status, or clinically important subgroups). It is also suggested that tests for interaction be used to evaluate subgroup effects because of their rigor and that findings of heterogeneity of response be considered cautiously and confirmed by subsequent trials.3 When these guidelines are applied, subgroup analyses can provide valuable information to guide future studies and clinical care.4 Kornstein et al1 had reason to suspect that responses to antidepressants might differ by age and menopause status; thus, their analysis was hypothesis driven, although they did not say if the decision to conduct the analysis was made during the design of the original trial. Unfortunately, it is probable that many studies suggested by subgroup analyses will never be performed owing to cost and feasibility. Reactions to this situation were demonstrated in a recent series of focus groups at Group Health conducted among women aged 40 to 55 years who were using estrogen for menopause symptoms. They shared their deep frustration that trial results were being used to guide their care even though they believed that the trials did not include women like them. A response to concerns about the lack of generalizability in RCTs is the growing interest in pragmatic clinical trials,5,6 in comparative effectiveness research,7 and in involving stakeholders in study design and execution.6,7 Pragmatic trials focus on whether a program or intervention works under usualVrather than idealVconditions. Pragmatic studies differ from efficacy studies in that they take the perspective of policy makers, practitioners, and individuals; are conducted in heterogeneous realworld settings; minimize exclusion criteria; and use comparisons based on real-world alternatives such as the current standard of care.7 This approach leads to more broadly applicable findings. The Patient-Centered Outcomes Research Institute is pushing this point further by insisting that the comparative effectiveness studies it funds engage and collaborate with individuals and other stakeholders throughout the research process, from
Menopause, Vol. 21, No. 8, 2014
Copyright © 2014 The North American Menopause Society. Unauthorized reproduction of this article is prohibited.
EDITORIAL
defining the study question through dissemination of results.6,7 The research is designed to examine outcomes that are most relevant to individuals, caregivers, and clinicians.7 Pragmatic trials and comparative effectiveness studies may be more likely to provide answers to the question, BDoes it work in people like me?[ Returning to Kornstein et al,1 their study is important because new and recurrent mood disorders are associated with the menopausal transition.8,9 This study confirmed the benefit of desvenlafaxine 50 mg/day in perimenopausal and postmenopausal women. Understanding these selective responses will help guide therapy for women on both sides of the menopausal transition.
2.
3.
4. 5.
6.
Financial disclosure/conflicts of interest: None reported. 7.
Katherine M. Newton, PhD Group Health Research Institute Seattle, Washington
8.
REFERENCES 9. 1. Kornstein SG, Clayton A, Bao W, Guico-Pabia CJ. Post hoc analysis of the efficacy and safety of desvenlafaxine 50 mg/day in a randomized, placebo-
controlled study of perimenopausal and postmenopausal women with major depressive disorder. Menopause 2014;21:799-806. Rothwell PM. Treating individuals 2. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet 2005; 365:176-186. Pocock SJ, Assmann SE, Enos LE, Kasten LE. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med 2002;21:2917-2930. Feinstein AR. The problem of cogent subgroups: a clinicostatistical tragedy. J Clin Epidemiol 1998;51:297-299. Glasgow R. What does it mean to be pragmatic? Pragmatic methods, measures, and models to facilitate research translation. Health Educ Behav 2013; 40:257-267. Selby J. Interview: Patient-Centered Outcomes Research Institute seeks to find out what works best by involving Fend-users_ from the beginning. J Comp Eff Res 2014;3:125-129. Fluerence R, Selby J, Odom-Walker K, et al. How the Patient-Centered Outcomes Research Institute is engaging patients and others in shaping its research agenda. Health Aff 2013;32:2393-2400. Cohen LS, Soares CN, Vitonis AF, Otto MW, Harlow BL. Risk for new onset of depression during the menopausal transition: the Harvard Study of Moods and Cycles. Arch Gen Psychiatry 2006;63:385-390. Freeman EW, Sammel MD, Boorman DW, Zhang R. Longitudinal pattern of depressive symptoms around natural menopause. JAMA Psychiatry 2014; 71:36-43.
Menopause, Vol. 21, No. 8, 2014
Copyright © 2014 The North American Menopause Society. Unauthorized reproduction of this article is prohibited.
787
