0 145-6008/90/ 14054736$2.00/0 ALCOHOLISM: CLINICAL AND EXPERIMENTAL RESEARCH
Vol. 14, No. 5
September/October 1990
Subject Selection Bias in Alcoholics Volunteering for a Treatment Study David B. Strohmetz, Arthur I. Alterman, and Dan Walter
Baseline differencesin alcoholism problem severity were compared between alcoholics who did and did not volunteer to participatein a treatment effectiveness study. A positive relationship was found between self-reports of alcohol-related problems and the degree of research participation. Group differences were also revealed in the rate of treatment completion. Possible explanations and solutions for this volunteer bias are discussed. Key Words: Subject Selection Bias, Treatment Outcome Studies, Randomized Assignment, Volunteer Bias.
ECENTLY A DEBATE has arisen over the comparative effectiveness of inpatient versus outpatient treatment of substance abuse patients. According to usual scientific practices, a way to resolve this issue is through a randomization study of treatment effectiveness using patients who volunteer as research subjects. However, there is substantial evidence suggesting that people who volunteer to participate in research studies may not be reprea result, relisentative of the population of i n t e r e ~ t .As ~’~ ance on volunteers as subjects for behavioral research can raise questions concerning the validity and generalization of research findings. In an extensive review of research on this volunteer issue, Rosenthal and Rosnow2 concluded that the nature of the research study can influence the type of person who volunteers to participate in that study. For example, they found that medical and psychiatric research tended to attract subjects who were clinically defined as being less well adjusted than the population of interest as a whole. The authors hypothesized that maladjusted individuals may be more apt to volunteer for clinical studies out of an explicit or implicit hope of receiving the most help for their problems. It is reasonable then to suggest that volunteers for treatment effectiveness studies may also not be representative of the target populations, especially concerning the severity of the problems of interest and the need for treatment of those problems. There has been little empirical research specifically examining whether volunteers for alcoholism research differ from nonvolunfeers. In the only relevant
R
’
From Temple University8and P 2 Medical Center, and University of Pennsylvania School of Medicine, Philadelphia, Pennsylvania. Receivedfor publication August 29, 1989: accepted May 16. 1990. This research was supported by the Medical Research and H R S D Services (No. CP86-098) of the Department of Veterans Affairs, as well as National Institute of Drug Abuse Research Center Grant DAO5186 and National Institute on Alcohol Abuse and Alcoholism Grant No. IROI AAO725 7. Reprint requests: Dr. Arthur I. Alterman (116), 116 VA Medical Center, University & Woodland Avenue, Philadelphia, PA 19104. Copyright 0 I990 by The Research Society on Alcoholism. 736
investigation known to us, Taylor et aL4 found that alcoholics who volunteered for research were more intelligent, more externally oriented, and had more years of problem drinking, more binge-oriented drinking, and more physical symptoms that are commonly associated with problem drinking. However, this paper did not consider alcoholics volunteering for a treatment study and therefore cannot address the conceptualization put forward by Rosenthal and Rosnow.2 Thus, the question still remains whether a truly randomized research study is possible in a clinical setting. We have recently become interested in this possible volunteer bias in our ongoing random assignment comparative treatment study of inpatient and outpatient rehabilitation for alcoholics. Since we were able to identify different degrees of participation in our study and obtain problem severity data on both study participants and nonparticipants, we were able to address this question. This ongoing study compares the effectiveness of alcoholism treatment in a day hospital versus an inpatient rehabilitation program. There were two levels of research participation which comprised two of the three comparison groups for the present study. One group of alcoholics volunteered for full participation in the study which included random assignment to either the inpatient or outpatient treatment program. Both of these treatment programs were designed to provide similar treatment over a 1-month period. The major difference between the two programs was that the inpatient treatment took place in a collaborating VA facility 40 miles from the outpatient treatment site. Patients who participated on a random assignment basis were provided with funds for transportation to the program and were given $15 for completing the research measures obtained during treatment. Those assigned to the outpatient program also received lunch money while attending treatment. A second group of patients consisted of partial volunteers who agreed to complete all the research measures, but insisted on receiving outpatient treatment. Family or employment reasons were most often cited by this group for refusing the inpatient treatment alternative. The only compensation this partial volunteer group received was $15 for completing the research measures. The third comparison group in this study, the nonvolunteer group, also insisted on outpatient treatment, but refused to participate in any of the ongoing studies at the Addiction Recovery Unit (ARU). Using Rosenthal and Rosnow’s2 conceptualization, we hypothesized that there was a relationship between a patient’s level of involvement in our treatment study and AlcoholClin Exp Res, VoI 14,No 5, 1990: pp 736-738
SELECTION BIAS IN TREATMENT STUDY VOLUNTEER ALCOHOLICS
the number of alcohol problems and severity of those problems as reported by the patients. METHOD Subjects A total of 190male patients sought rehabilitation treatment for alcohol dependence at the Philadelphia Veterans Administration Medical Center's outpatient ARU between September 1987 and May 1989. Based on psychiatric evaluation, none of the patients was found to be dependent on any drug other than marijuana or nicotine. Those patients who qualified for the treatment study (i.e., eligible for day hospital rehabilitation; under 60 years of age; did not have a serious psychiatricor mental disorder requiring inpatient care; and did not live outside of the Philadelphia metropolitan area) were recruited for our study by a research technician after completing the ARU's intake procedure. Twenty-five patients gave voluntary consent to full randomized participation in the study. Thirty-one patients agreed to limited participation in the study which consisted of completing the baseline and followup research measures. A cursory search of the charts of the remaining patients undergoing treatment in the 1-month day hospital program yielded 38 subjects who had refused to volunteer for any of the ARU's ongoing research studies. These subjects constituted the nonvolunteer group. Procedure Before any patient was recruited for a research project, he was administered the Addiction Severity Index (ASI)5.6by a trained technician as part of the standard intake procedure at the ARU. The MI is a structured interview that provides sociodemographic information and assesses the severity of current and lifetime problems in seven areas of functioning typically afFected by substance abuse: medical, employment, alcohol use, drug use, legal, familial/social, and psychiatric. In addition to individual items, weighted composite (factor-like) scores are derived for each section providing an indication of the patient's level of functioning in that area. Higher scores indicate more severe problems. The AS1 has been shown to be a reliable and valid measure of problem severity in substance abuse treatment settings6 Only the data drawn from the alcohol use section were used to form the basis for our evaluation. Program completion rates for the treatment program were also available for the three groups. It should be noted that for the full volunteers, only the completion data for day hospital patients was used, since differential completion rates for the day hospital and inpatient programs made the inclusion of inpatients in this analysis inappropriate.
RESULTS
Planned contrasts using the Bonferroni correction7were used to test for linear trends for selected items in the AS1 alcohol use section. The nonvolunteer, partial volunteer, and full volunteer groups were given weights of - 1, 0, +I. x2 analysis was used for categorical data. The Bonferroni correction was applied to post-hoc multiple comparisons. As Table 1 indicates, the three groups were fairly homogeneous with respect to sociodemographic characteristics. The one exception was that significant group differences were found ( p < 0.05) in the proportion of subjects who were married. However, none of the between group comparisons achieved statistical significance. Table 1. Mean (and SD) Demographic Characteristicsof the Groups Full Partial volunteer volunteer Nonvdunteer n = 25 n = 31 n = 38 Variables Age (in years) Race (% black) Education (in years) Days Worked7 Religion (% Protestant) Marital status (YOManiedlt
40.6 (7.2)' 84.0 12.2(1.8) 9.0(9.1) 50.0 4.0
41.5(7.3) 67.7 12.5 (1.7) 9.2(10.3) 54.8 38.7
Mean and standard deviation.
t During the 30 days prior to treatment request. S p c 0.05.
42.4(8.2) 70.0 12.0 (2.3) 8.2 (9.5) 60.0 26.3
131
With respect to alcohol problems reported at intake, several significant linear trends were found (see Table 2). With increased study participation the overall severity of alcohol problems as indicated by the alcohol use composite score [F(1,89) = 14.69,p < 0.01, eta = 0.381 increased. In addition, there was a significant linear relationship between a patient's level of participation and the number of days he used alcohol [F(1,91) = 9.18, p < 0.01, eta = 0.301 and days he was intoxicated during the 30 days prior to treatment request [F(1,91) = 10.40, p < 0.01, eta = 0.321. There was no difference between the three groups with respect to how many days during the prior 30 they reported experiencing alcohol-related problems, nor concerning how bothered they were by these problems and their perceived need for treatment. Finally, significant group differences were revealed in completion rates for the one month day hospital treatment program (x2= 16.79,2 dl; p = < 0.001). Post-hoc between group comparisons revealed a significantlyhigher completion rate (x2= 16.26, 1 dJ;p < 0.001) for partial volunteers (27 of 3 1 , or 87. I %) than for nonvolunteers (15 of 38, or 39.5%). The completion rate for the full volunteer group (9 of 13, or 69.2%) was not found to differ significantly from that of either of the other two groups (Table 3). DISCUSSION
The results are consistent with the hypothesis that a volunteering bias may be introduced into alcoholism treatment research. The level of a patient's participation in the research was related to the severity of alcohol problems reported during the 30 days prior to treatment entry. It should be emphasized that the intake interviews were administered before research recruitment was initiated. Group differences were also revealed in program completion rates. Two plausible explanations come to mind for the positive association between the extent of research participation and the severity of alcohol-related problems upon Means (and so) for Selected AS1 Alcohol Use Items Full Partial Volunteer volunteer Nonvdunteer n=25 n=31 n=38 Variables
Table 2.
Alcohol-related' Days drankt Days drank to intoxication* Alcohol problem days* Bothered by alcohol problems§ Perceived need for treatment5
p
0.68(0.2)t 0.59(0.2) 0.49(0.2)
Vol. 14, No. 5
September/October 1990
Subject Selection Bias in Alcoholics Volunteering for a Treatment Study David B. Strohmetz, Arthur I. Alterman, and Dan Walter
Baseline differencesin alcoholism problem severity were compared between alcoholics who did and did not volunteer to participatein a treatment effectiveness study. A positive relationship was found between self-reports of alcohol-related problems and the degree of research participation. Group differences were also revealed in the rate of treatment completion. Possible explanations and solutions for this volunteer bias are discussed. Key Words: Subject Selection Bias, Treatment Outcome Studies, Randomized Assignment, Volunteer Bias.
ECENTLY A DEBATE has arisen over the comparative effectiveness of inpatient versus outpatient treatment of substance abuse patients. According to usual scientific practices, a way to resolve this issue is through a randomization study of treatment effectiveness using patients who volunteer as research subjects. However, there is substantial evidence suggesting that people who volunteer to participate in research studies may not be reprea result, relisentative of the population of i n t e r e ~ t .As ~’~ ance on volunteers as subjects for behavioral research can raise questions concerning the validity and generalization of research findings. In an extensive review of research on this volunteer issue, Rosenthal and Rosnow2 concluded that the nature of the research study can influence the type of person who volunteers to participate in that study. For example, they found that medical and psychiatric research tended to attract subjects who were clinically defined as being less well adjusted than the population of interest as a whole. The authors hypothesized that maladjusted individuals may be more apt to volunteer for clinical studies out of an explicit or implicit hope of receiving the most help for their problems. It is reasonable then to suggest that volunteers for treatment effectiveness studies may also not be representative of the target populations, especially concerning the severity of the problems of interest and the need for treatment of those problems. There has been little empirical research specifically examining whether volunteers for alcoholism research differ from nonvolunfeers. In the only relevant
R
’
From Temple University8and P 2 Medical Center, and University of Pennsylvania School of Medicine, Philadelphia, Pennsylvania. Receivedfor publication August 29, 1989: accepted May 16. 1990. This research was supported by the Medical Research and H R S D Services (No. CP86-098) of the Department of Veterans Affairs, as well as National Institute of Drug Abuse Research Center Grant DAO5186 and National Institute on Alcohol Abuse and Alcoholism Grant No. IROI AAO725 7. Reprint requests: Dr. Arthur I. Alterman (116), 116 VA Medical Center, University & Woodland Avenue, Philadelphia, PA 19104. Copyright 0 I990 by The Research Society on Alcoholism. 736
investigation known to us, Taylor et aL4 found that alcoholics who volunteered for research were more intelligent, more externally oriented, and had more years of problem drinking, more binge-oriented drinking, and more physical symptoms that are commonly associated with problem drinking. However, this paper did not consider alcoholics volunteering for a treatment study and therefore cannot address the conceptualization put forward by Rosenthal and Rosnow.2 Thus, the question still remains whether a truly randomized research study is possible in a clinical setting. We have recently become interested in this possible volunteer bias in our ongoing random assignment comparative treatment study of inpatient and outpatient rehabilitation for alcoholics. Since we were able to identify different degrees of participation in our study and obtain problem severity data on both study participants and nonparticipants, we were able to address this question. This ongoing study compares the effectiveness of alcoholism treatment in a day hospital versus an inpatient rehabilitation program. There were two levels of research participation which comprised two of the three comparison groups for the present study. One group of alcoholics volunteered for full participation in the study which included random assignment to either the inpatient or outpatient treatment program. Both of these treatment programs were designed to provide similar treatment over a 1-month period. The major difference between the two programs was that the inpatient treatment took place in a collaborating VA facility 40 miles from the outpatient treatment site. Patients who participated on a random assignment basis were provided with funds for transportation to the program and were given $15 for completing the research measures obtained during treatment. Those assigned to the outpatient program also received lunch money while attending treatment. A second group of patients consisted of partial volunteers who agreed to complete all the research measures, but insisted on receiving outpatient treatment. Family or employment reasons were most often cited by this group for refusing the inpatient treatment alternative. The only compensation this partial volunteer group received was $15 for completing the research measures. The third comparison group in this study, the nonvolunteer group, also insisted on outpatient treatment, but refused to participate in any of the ongoing studies at the Addiction Recovery Unit (ARU). Using Rosenthal and Rosnow’s2 conceptualization, we hypothesized that there was a relationship between a patient’s level of involvement in our treatment study and AlcoholClin Exp Res, VoI 14,No 5, 1990: pp 736-738
SELECTION BIAS IN TREATMENT STUDY VOLUNTEER ALCOHOLICS
the number of alcohol problems and severity of those problems as reported by the patients. METHOD Subjects A total of 190male patients sought rehabilitation treatment for alcohol dependence at the Philadelphia Veterans Administration Medical Center's outpatient ARU between September 1987 and May 1989. Based on psychiatric evaluation, none of the patients was found to be dependent on any drug other than marijuana or nicotine. Those patients who qualified for the treatment study (i.e., eligible for day hospital rehabilitation; under 60 years of age; did not have a serious psychiatricor mental disorder requiring inpatient care; and did not live outside of the Philadelphia metropolitan area) were recruited for our study by a research technician after completing the ARU's intake procedure. Twenty-five patients gave voluntary consent to full randomized participation in the study. Thirty-one patients agreed to limited participation in the study which consisted of completing the baseline and followup research measures. A cursory search of the charts of the remaining patients undergoing treatment in the 1-month day hospital program yielded 38 subjects who had refused to volunteer for any of the ARU's ongoing research studies. These subjects constituted the nonvolunteer group. Procedure Before any patient was recruited for a research project, he was administered the Addiction Severity Index (ASI)5.6by a trained technician as part of the standard intake procedure at the ARU. The MI is a structured interview that provides sociodemographic information and assesses the severity of current and lifetime problems in seven areas of functioning typically afFected by substance abuse: medical, employment, alcohol use, drug use, legal, familial/social, and psychiatric. In addition to individual items, weighted composite (factor-like) scores are derived for each section providing an indication of the patient's level of functioning in that area. Higher scores indicate more severe problems. The AS1 has been shown to be a reliable and valid measure of problem severity in substance abuse treatment settings6 Only the data drawn from the alcohol use section were used to form the basis for our evaluation. Program completion rates for the treatment program were also available for the three groups. It should be noted that for the full volunteers, only the completion data for day hospital patients was used, since differential completion rates for the day hospital and inpatient programs made the inclusion of inpatients in this analysis inappropriate.
RESULTS
Planned contrasts using the Bonferroni correction7were used to test for linear trends for selected items in the AS1 alcohol use section. The nonvolunteer, partial volunteer, and full volunteer groups were given weights of - 1, 0, +I. x2 analysis was used for categorical data. The Bonferroni correction was applied to post-hoc multiple comparisons. As Table 1 indicates, the three groups were fairly homogeneous with respect to sociodemographic characteristics. The one exception was that significant group differences were found ( p < 0.05) in the proportion of subjects who were married. However, none of the between group comparisons achieved statistical significance. Table 1. Mean (and SD) Demographic Characteristicsof the Groups Full Partial volunteer volunteer Nonvdunteer n = 25 n = 31 n = 38 Variables Age (in years) Race (% black) Education (in years) Days Worked7 Religion (% Protestant) Marital status (YOManiedlt
40.6 (7.2)' 84.0 12.2(1.8) 9.0(9.1) 50.0 4.0
41.5(7.3) 67.7 12.5 (1.7) 9.2(10.3) 54.8 38.7
Mean and standard deviation.
t During the 30 days prior to treatment request. S p c 0.05.
42.4(8.2) 70.0 12.0 (2.3) 8.2 (9.5) 60.0 26.3
131
With respect to alcohol problems reported at intake, several significant linear trends were found (see Table 2). With increased study participation the overall severity of alcohol problems as indicated by the alcohol use composite score [F(1,89) = 14.69,p < 0.01, eta = 0.381 increased. In addition, there was a significant linear relationship between a patient's level of participation and the number of days he used alcohol [F(1,91) = 9.18, p < 0.01, eta = 0.301 and days he was intoxicated during the 30 days prior to treatment request [F(1,91) = 10.40, p < 0.01, eta = 0.321. There was no difference between the three groups with respect to how many days during the prior 30 they reported experiencing alcohol-related problems, nor concerning how bothered they were by these problems and their perceived need for treatment. Finally, significant group differences were revealed in completion rates for the one month day hospital treatment program (x2= 16.79,2 dl; p = < 0.001). Post-hoc between group comparisons revealed a significantlyhigher completion rate (x2= 16.26, 1 dJ;p < 0.001) for partial volunteers (27 of 3 1 , or 87. I %) than for nonvolunteers (15 of 38, or 39.5%). The completion rate for the full volunteer group (9 of 13, or 69.2%) was not found to differ significantly from that of either of the other two groups (Table 3). DISCUSSION
The results are consistent with the hypothesis that a volunteering bias may be introduced into alcoholism treatment research. The level of a patient's participation in the research was related to the severity of alcohol problems reported during the 30 days prior to treatment entry. It should be emphasized that the intake interviews were administered before research recruitment was initiated. Group differences were also revealed in program completion rates. Two plausible explanations come to mind for the positive association between the extent of research participation and the severity of alcohol-related problems upon Means (and so) for Selected AS1 Alcohol Use Items Full Partial Volunteer volunteer Nonvdunteer n=25 n=31 n=38 Variables
Table 2.
Alcohol-related' Days drankt Days drank to intoxication* Alcohol problem days* Bothered by alcohol problems§ Perceived need for treatment5
p
0.68(0.2)t 0.59(0.2) 0.49(0.2)
