Another V i e w of Active-Controlled Trials A. Lawrence Gould, PhD Merck, Sharp, and Dohme Research Laboratories, West Point, Pennsylvania.
ABSTRACT: Placebo-controlled efficacy trials may become more difficult to carry out with the increasing availability of effective therapies, especially for serious illnesses where denial of effective therapy may be objectionable ethically. Active-controlled trials aimed at establishing efficacy by demonstration of "equivalence" to "standard" therapy have potentially serious interpretational problems, and do not necessarily encourage good experimental practice. This article describes an alternative approach to the analysis of data from active-controlled trials using the information that makes an active-controlled trial necessary or desirable, namely a large, valid body of information about the consequence of using placebo. The approach uses information about placebo responses and also active agent responses from prior placebo-controlled trials to determine the likelihood of a significant active-placebo difference in an active-controlled trial, or in a trial with a vestigial placebo group. The sensitivity of the treatment comparisons depends directly on the quality of the design and execution of the active-controlled trial. The method is illustrated with data from trials of an H2-receptor antagonist in the treatment of acute duodenal ulcer. KEY WORDS: Empirical Bayes, predictive distributions, clinical trials, meta-analysis INTRODUCTION Establishing the effectiveness of a therapy for treating an illness ordinarily requires demonstrating that the therapy provides a superior clinical result to some alternative course of treatment such as n o t h i n g (or placebo to control for the act of treating) or an alternative agent k n o w n to be effective. The latter option could require impracticably large sample sizes if the standard a n d test therapies have similar effects. Effectiveness also could be established by demonstrating a clear relationship between dose a n d clinical response. In largescale clinical trials there is a real operational advantage to keeping the sample size as small as possible a n d the experimental design as simple as possible to assure that the trial will be carried out as the protocol requires. Consequently, placebo-controlled trials have long been the m e t h o d of choice. Placebo-controlled efficacy trials become more difficult to carry out with the increasing availability of effective therapies, especially in serious illnesses where withholding effective therapy m a y be medically inappropriate. Institutional Review Boards a n d treating physicians will have ethical objections to d e n y i n g active treatment to patients w h e n the consequences m a y be se-
Address reprint requests to: A. L. Gould, PhD, BL3-2, Merck, Sharp, and Dohme Research Laboratories, West Point, PA 19486. Received June 11, 1990; revised March 19, 1991.
474 0197-245(~91/$3.50
Controlled Clinical Trials 12:474--485 (1991) © Elsevier Science Publishing Co., Inc. 1991 655 Avenue of the Americas, New York, New York 10010
Active-Controlled Trials
475
rious, e.g., prophylactic treatment of patients recovering from a myocardial infarction, long-term control of hypertension. Once properly informed, patients may be reluctant to participate in a trial in which they have a substantial chance of receiving an inactive therapy when they know that feasible active therapies exist. Active-controlled trials, in which no patients receive placebo, therefore seem to be inevitable, at least in some circumstances. Regulatory approval for the marketing of a n e w drug requires unequivocal demonstration of its efficacy in at least (usually more than) one well-controlled trial. Active-controlled trials aimed at establishing the efficacy of a new agent by demonstrating its "equivalence" to the effect of some "standard" therapy tend to be unacceptable for several reasons [1]: 1. No generally accepted statistical methods exist for establishing "equivalence" or even "similarity." 2. "Equivalence" cannot be distinguished from lack of power for detecting a clinically meaningful difference (small or sloppy studies are unlikely to show differences). 3. " . . . positive-control studies do not provide the incentives toward conducting an excellent study that a placebo-controlled t r i a l . . , provides." 4. The expectation that everyone should do well in the trial introduces a bias that may be inconsistent with careful observation and measurement. 5. "Showing that two drugs are equivalent in a study does not demonstrate that either is effective; it shows that both were effective or that neither was." Equivalence also could be defined using confidence intervals for treatment effect difference instead of lack of statistical significance in the conventional sense. This is common practice in clinical pharmacology trials aimed at establishing bioequivalence, but also applies to clinical equivalence [2]. Careful execution is essential to the success of these trials. Confidence-based approaches can establish equivalence, but not necessarily effectiveness. Instead of demonstrating "equivalence," one can establish efficacy by exploiting what makes an active-controlled trial imperative: a b o d y of convincing evidence that the condition cannot go untreated for the duration of the study, and the existence of at least one feasible, effective therapy. With this information one can compute the likelihood that the n e w therapy would have proved more effective than placebo had the tri~l included a placebo-treated group. This is not the definitive evidence that a placebo-controlled trial would provide, but under the circumstances may be the best that can be done. The approach is very different from attempting to establish efficacy of a test therapy by demonstrating its "equivalence" to an active control. Regarding active-controlled trials in this light should encourage careful trial execution, since small, sloppy trials are unlikely to provide positive results. If the trial can include a small group of patients on placebo, then the demonstration of efficacy can be strengthened. What follows is an overview of the methodology as it applies to dichotomous outcomes; a more comprehensive discussion appears in Ref. 3. BASIC PROCEDURE
The outcomes of active-controlled trials can be evaluated usefully and realistically in terms of the likelihood that the outcomes observed for the patients
476
A.L. Gould on the "active" agents would have differed significantly from the outcome for a placebo-treated group had there been one. Because these trials lack placebo-treated groups, one cannot know what differences would have been observed. At best, one can determine the likelihood (or confidence) of a significant difference using the results of the trial outcome and the results observed for similar kinds of patients treated with placebo in similar trials. Of course, this requires assuming that the process generating the placebotreated group outcomes in the previous trials also would have generated the outcomes for a placebo-treated group in the active-controlled trial. The quality of design and execution of the trials providing the data to which the calculations are applied govern the credibility of the inferences this approach provides. This also is true for the trials providing information about the result of placebo treatment that the active-controlled trial lacks. This information must reflect the experience of similar kinds of patients in trials where the measurements are made similarly in the same time frame, so that the spectrum of potential placebo group outcomes realistically reflects what might have occurred had the active-controlled trial included a placebo group. Obtaining this information is not a simple task and requires careful attention to details [4]. Badly designed or executed trials will not yield credible conclusions. This is particularly true for active-controlled trials in view of regulatory concerns about their interpretation [1,5,6]. The prior information about placebo group responses must be carefully documented, scientifically valid, and relevant to the target population and experimental conditions of the active-controlled trial. Meta-analysis [7,8] provides a convenient and coherent w a y to accumulate this information.
Incorporating Prior Placebo Information The following example illustrates the process. A multiclinic trial was carried out to compare three dosage regimens of a new active agent with a standard agent in the treatment of acute duodenal ulcer. The key response variable was the proportion of patients whose ulcer was healed after four weeks of treatment; healing was defined as replacement of the ulceration by scar tissue or normal mucosa. The results for each regimen are shown in Table 1. No placebo-treated group was included in the trial. None of the between-group differences is significant. A high spontaneous healing rate in this trial cannot be ruled out in favor of treatment efficacy as an explanation for these findings, although the sample sizes suggest that efficacy is the more likely explanation. Understanding how a placebo-treated group might have performed can help in drawing an appropriate inference from these outcomes.
Table 1
Comparison of Three Dosage Regimens of a N e w Active Agent With a Standard Agent for Treatment of Duodenal Ulcer Regimen 1 Regimen 2 Regimen 3 Standard Number of patients 240 247 247 246 Number with healed ulcer 164 191 201 186 Healing rate (%) 68 77 81 76
477
Active-Controlled Trials
The first step in the analysis is to obtain information about previous trials of similar design, target population, and execution that included a placebo treatment. Table 2 summarizes the findings from a number of trials, some fairly large, comparing various active agents against placebo in the treatment of acute duodenal ulcer. In general, the trials were double-masked and included outpatients with duodenal ulcers endoscopically proven within 2-3 days before starting their (randomly assigned) treatment. As far as could be determined, the published trials and the present trial were conducted similarly. These 23 outcomes are consistent with the hypothesis that they are random outcomes from a single process; see "Checking assumptions," below. The outcomes from these 23 trials therefore appear to comprise a sample of outcomes that might be expected from groups of patients with acute duodenal ulcer receiving placebo for 4 weeks in double-masked trials. These 23 outcomes can be used to determine the likelihood that the outcomes from the present trial would have been significantly superior to the outcome for a placebo-treated group had one been included. The variation in the healing rates among the 23 trials is greater than can be accounted for by sampling variation from a parent binomial distribution with a single healing rate parameter. Thus, instead of a universal "true" healing rate for placebo-treated patients, it is more likely that each trial has its own "true" placebo healing rate that depends on its populations' demographics, diets, stress levels, medical characteristics, etc. Since neither the form of this dependence nor all of the test populations' characteristics are knowable, it certainly is more practical, and appears to be more realistic, to assume that some process generates trial-specific "true" rates at random. This would be true, for example, even if there were a common distribution for the responses of all possible placebo-treated patients: variations among trials in the apparent "true" rate would occur because of variations in the samples drawn for each trial. Table 2
Numbers and Percentages of Patients With Healed Duodenal Ulcers After 4 Weeks on Placebo in Double-Blind Controlled Trials a Healed at 4 Weeks
Healed at 4 Weeks
Site
N
N
(%)
Site
N
N
(%)
Belgium/Holland Czechoslovakia France France Hong Kong Hungary Ireland Italy Italy Norway Norway Switzerland
73 55 87 99 24 20 35 758 166 72 20 164
29 33 49 54 4 12 14 242 60 36 12 93
(40) (60) (56) (55) (17) (60) (40) (32) (36) (50) (60) (57)
Switzerland U.K. U.K. U.S.A. U.S.A. W. Germany W. Germany
106 143 151 340 195 123 101
61 41 47 167 80 74 63
(58) (29) (31) (49) (41) (60) (62)
Italy [10] Italy [11] Norway [12] U.S.A. [13]
17 80 24 168
5 23 11 76
(29) (29) (46) (45)
aAll results are from Ref. 9 except as otherwise noted.
478
A.L. Gould
Marginal Distribution of Outcomes Suppose that for any trial, the "true" placebo healing rate, p, is a random realization from a [5 distribution with parameters (a,b). Once a value of p has been "generated" for a trial, the number of patients healed out of n treated has a binomial (n,p) distribution. The joint probability function for the outcome and the value of p is the product of these two distributions, h(x, p ; n, a, b) = ( n ) B - ' ( a ,
b)lY +~-I ( 1 -
p)b+ . . . .
1.
(1)
The relationship between the outcome (x) and the process generating the "true" response rate for the trial follows from integrating (1) with respect to P, k(x;n,a,b)
= (n)B_~(a,b)B(a+x,b+
n-x),
(2)
a [5-binomial density [14]. This is the a priori likelihood function for the trial, i.e., the probability function of the outcome (x) in terms of the process that generates the (trial-specific) "true" response rates. The joint likelihood for m trials is the product of quantities like Eq. 2, L(x ; n, a, b ) = B - " ( a , b ) f i
(ni'~ B(a + xi, b + ni - xi) i= 1 \ / X i
Likelihood of a Significant Difference from Placebo Suppose that x patients responded out of m treated with an active agent. Also, let ~ denote the critical value for a chi-square variable with 1 degree of freedom corresponding to a two-tail test at some significance level (e.g., for a 5% level test, ~ = 3.84). Given x and m, an outcome of y responses out of n patients treated with placebo would provide a chi-square statistic value exceeding ~ if y >! y~ ~ yu m n ( 2 N x - 2~v. + NI;) + N{m 2 n2~ 2 + 4mnU~lx(m - x)} °5 2 m ( m N + n~) or
y ~ Y'{ ~ yL =
m n ( 2 N x - 2~x + NI~) - N ( m 2 n2~ 2 + 4 m n ~ N x ( m - x)}°'5 2 m ( m N + nl~)
where N = m + n , = smallest integer >- yu, and y~ = largest integer -< yL. T h e confidence that a sample of n patients on placebo would yield an outcome significantly different from that observed for the active agent (y responses out of m tested) is the probability that y ~ y[ or y >-- y~ using (2) as the probability function for y, Conf{Sign. Diff. [ m, x, n} = "~ k(y; n, a, b) + "~ k(y; n, a, b)
(3)
479
Active-Controlled Trials
Example For the 23 trials, a = 9.3 and b = 11.2. The expected average response rate is E(p) = a/(a + b) = 0.453, about 45%, so that almost half the patients on placebo could expect healed ulcers after 4 weeks. Substituting these values of a and b into (2) gives the overall estimated probability of the observed number of responses (to placebo) among n patients treated. This probability is used as the confidence distribution for the outcomes that might have been observed had a placebo-treated group been present. Table 3 displays the outcomes from the present trial and the results of performing the calculations described above. The confidence level exceeds 95% for three of the four groups even if a placebo-treated group of less than half the size of any of the active treatment groups is used. Therefore, if a placebo-treated group had been included, its healing rate very likely would have been significantly less than the healing rate for most, if not all, of the four active treatment groups.
Checking Assumptions Regarding the placebo outcomes from the 23 trials as realizations from some process generating "true" healing rates can be justified statistically (C. L. Mallows, personal communication). Each trial's outcome is a sample from a "known" distribution of potential outcomes for placebo-treated patients in that trial. Replacing the outcome by its corresponding distribution function value yields a realization from a uniform distribution, so the points representing these cdf values plotted against their relative ranks [corresponding values of the cdf for a uniform (0,1) distribution], will fall on or near the line connecting the points (0,0) and (1,1). When the points are obtained by using assumed rather than known study-specific placebo value distributions, the degree to which they coincide with this straight line reflects the degree to which the assumptions are consistent with the data. If the cdf values based on the assumed distributions deviate greatly from the line (as assessed by a Kolmogorov goodness-of-fit test, for example), then the data are unlikely to have arisen from the assumed distributions. Assuming that each of the 23 outcomes comes from a distribution of the form (2) yields a maximum absolute difference between the (ordered) calculated cdfs and the corresponding uniform (0,1) order statistics of 0.137. This
Table 3
Confidence of Significant Active-Placebo Differences Outcomes from present trial Placebo-treated group sample size" No.
Treatment Group Test, Reg. 1 Test, Reg. 2 Test, Reg. 3 Standard
N 240 247 247 246
Healed (%) 164 (68) 191 (77) 201 (81) 186 (76)
100 0.844 0.968 0.987 0.954
200 0.888 0.983 0.995 0.973
220 0.897 0.983 0.995 0.975
aConfidence levels corresponding to various placebo-treated group sample sizes.
240 0.898 0.984 0.995 0.976
260 0.899 0.986 0.996 0.978
480
A.L. Gould is less than the 20% critical value for a Kolmogorov test based on 23 observations [15]. The 23 findings therefore can be regarded as having been generated by a common process.
COMPENSATING FOR ACTIVE CONTROL BIAS
The patients' responses could be affected by their knowing that whatever treatment they receive is "active." This expectation or active control bias may cause a spuriously large (or small) apparent difference between the test agent's effect and the estimated placebo effect. The presence of active control bias can be detected and compensated for when the previous placebo-controlled trials include outcomes for the standard active agent used in the active-controlled trial. If the outcome for the standard fails well within the range of its outcomes in the placebo-controlled trials, e.g., within the interval containing the middle 80% of them, then expectation bias probably has not affected the active-controlled trials outcomes materially, and no adjustment should be necessary. Adjustment for a material expectation bias proceeds by applying the method described above to a measure of the difference between the treatments. If the bias acts to shift the response rate, i.e., ~(observed response) = treatment effect + bias, where the bias shift is the same for both treatments, then the arithmetic difference between the observed responses will estimate the arithmetic difference between the treatment effects. If the bias affects the response multiplicatively, i.e., ~{(observed response) - treatment effect × bias, where the bias factor is the same for both treatments, then the ratio of the actual responses will estimate the ratio for the treatment effects. The results from the placebo-controlled trials can be used to construct a confidence distribution for the difference between (or the ratio of) the response rates for patients treated with the standard active agent or placebo. Thus, if the activecontrolled trial had included a placebo-treated group, then a test of the null hypothesis of the equivalence of the test (e) and placebo (p) effects could be expressed as, "Reject H0 if t~p > c', where t~ denotes the difference between (or ratio of) the responses observed for the test active and placebo-treated groups. The test also could be expressed as "Reject Ho if tes + tsv > c (or tes × t~p > c)", where te~ and t~p are obvious analogues of t~p, or as "Reject/4o if tsv > c - te~ (or tsv > c/tes)."
(4)
If the trial does not contain a placebo-treated group, tsv cannot be observed. At best, one can use the findings from similar previous trials to construct a con-
481
Active-Controlled Trials
fidence distribution for t~p and use this distribution to determine the probability content of the rejection region (4). The result is a statement of the confidence, based on the prior information, that the outcome for the test active agent would have differed significantly from the outcome for a placebo-treated group if one had been included in the trial. This is very much like the approach for incorporating just the responses to placebo in previous trials, differing only in the variate whose confidence distribution is computed, and in the fact that considerably less information may be available. For example, only 4 of the 23 placebocontrolled trials in acute duodenal ulcer compared the active control from the current trial with placebo, so only the outcomes for those trials could be used to construct a confidence distribution for the difference between (or the ratio of) the response rates for the standard and placebo. Correcting an additive bias with dichotomous responses proceeds as follows. The correction when the bias affects the responses multiplicatively is similar; Ref. 3 provides details. For either the placebo or the standard active group in any of the prior trials, the joint likelihood for the observations and rate parameter has the form (1). Integrating the response rate parameters out of the product of these separate likelihoods yields the joint marginal likelihood for the responses,
(m)
( y ) B-~(a, b) B - l ( c , d ) B(a + x , b + m - x) B(c + y , d + n - y).
(5) In (5), m, x, a, and b refer to the active group and n, y, c, and d refer to the placebo-treated group. The parameters in Eq. 5 are estimated separately for the placebo and standard active groups using data from the previous trials containing both groups as described above (assuming two response categories). Given estimates of a, b, c, and d, the confidence or predictive distribution of tsp can be determined from (5) and used to compute the confidence associated with (4). The confidence or predictive distribution of tsp is determined by rewriting (5) as a function of t~p and y, and then summing over the range of y values corresponding to the value of tsp. If t = x/m - y/n, then the joint marginal likelihood has the form
fit, y lm, n,a,b,c,d)
(6)
m ! ( y ) X B(a + m(t + y/n), b + m ( 1 -
t-y/n))B(c+
y, d + n - y )
F(m(t + y/n) + 1)F(m(1 - t - y/n) + 1)B(a,b)B(c,d) where t takes rational values between - 1 and 1 and the range of y depends on t as follows when m is a multiple of n (the case most likely to be useful in practice):
t = k/m
~y
t = -k/m~y
= 0, 1. . . . .
[(m - k)/s],k = 0, 1. . . . .
= {k/s},{k/s} + 1. . . . .
n , k = 1. . . . .
m, m,
where Ix] denotes the largest integer -< x, s = re~n, and {x} denotes the smallest integer -> x. Summing this marginal joint likelihood over the values of y
482
A.L. Gould corresponding to a given value of t yields the marginal distribution of t (given m, n, a, b, c, and d),
f(tlm,
n,a,b,c,d
) = ~f(t,y[m,n,a,b,c,d). Y
The next step is to incorporate this information about t into the rule for accepting or rejecting the null hypothesis. The usual uncorrected chi-square test with 1 degree of freedom based on (4) w h e n the treatment group differences are expressed as the arithmetic differences of the observed response proportions can be reduced, with some algebra, to Reject H0 when tsp >>-Q - tes o r tsp ~
--Q - tes,
(7)
where Q = X/~qil - q)(1/ne + 1/n), q = (xe + y)/(n~ + n), ~ is an appropriate critical value for a chi-square distribution with I degree of freedom (e.g., 3.84 for a 5% level test), x~ denotes the number of patients responding to the test active agent out of ne treated, and y denotes the number of patients responding to placebo out of n treated. The confidence that H0 would be rejected given the results from the studies comparing the standard active agent to placebo is the probability that (7) would be satisfied; this is given by summing (6) over the region of (t[ = tsp], y) values satisfying (7), Yl
Prob(RejectH0) = ~
f(t, y l m ,
~
n,a,b,c,d)
y~O t~Q;--tes
(8)
i
+
~,
[(t, y l m ,
n,a,b,c,d),
y~y2 t-~-(Q+tes)
where yl denotes the largest value of y for which (1 - y/n) --- Q - t~, and y2 denotes the smallest value of y for which y/n >- Q + te~.In practice, one would set m = n = number of patients treated with the standard active agent in the active-controlled trial.
Example Four of the 23 trials compared the active control of the present trial against placebo. The results follow in Table 4. For the active treatment ~ = 171 and b = 49.9; for the placebo-treated group, ~ = 26.5 and d = 42.7. The values of ~ and d reflect less study-to-study variation in the observed healing rate
Table 4 Comparison of Active Control of Present Trial against Placebos in Four of 23 Trials Active Study Study D Study P Study B Study H
Number Treated 18 84 25 187
Placebo Number Healed 15 65 23 137
Number Treated 17 80 24 168
Number Healed 5 23 11 76
483
Active-Controlled Trials
for these four trials than for all 23 trials. For the present active-controlled trial, 246 patients were observed on the active standard and 164 out of 240 patients healed on the first regimen of the test agent. The value of (8) for a 5% level (1% level) test is 0.989 (0.971), so that, as before, the outcome of a placebotreated group, had one been included, very likely would have been significantly inferior to the outcome from any of the active agents in the trial. INCLUDING A SMALL PLACEBO-TREATED GROUP Sometimes it may be possible to include a small placebo-treated group in an active-controlled trial. This could happen, for example, in conditions such as acute duodenal ulcer where the consequences of 4 weeks of placebo are unlikely to be serious, so that withholding effective medication does not pose a major ethical problem. Having at least some patients on placebo, even if fewer than on the active agents, could reduce the likelihood of expectation bias because not all patients would receive an "active" treatment. Treatment-placebo differences large enough to reach statistical significance for key efficacy parameters should comprise evidence for efficacy on a par with a conventional placebo-controlled trial. The efficacy of the test agent can be evaluated in more than one w a y if the treatment-placebo differences do not reach statistical significance. We consider two approaches when the differences are not significant. The first approach is analogous to the argument leading to expression (3). If (i) the failure to reach significance is due to an insufficient placebo group size, and (ii) the data from prior trials provide a reasonable guide to the distribution of placebo responses the present trial might have obtained, then expression (3) provides the confidence that a significant treatment-placebo difference would have been found if there had been n observations on placebo instead of, say, n': Conf{Sign. Diff. I m , x , n , n ' , y ' } =
~
k(y;n - n',a,b) +
Y~YL _yt
~
k(y;n - n',a,b).
(9)
y~yU_yt
The values of a and b would be obtained from the prior information as before. If there is a lot of prior evidence, as there should be, then the current trial values (n', y') should not affect the estimates, so whether they are included or not will not matter greatly. If there is little prior information, then it is not at all clear that a placebo-controlled trial should not be done. The second approach combines the placebo data from the current trial with information about the difference between the active control and placebo from the prior trials. Suppose that n' patients received placebo in the current trial and that y' of them responded; also, suppose that xe patients responded out of me on the test therapy, and that xs out of ms responded on the standard agent. The objective is to determine the likelihood of rejecting H0 if there had been n (> n') placebo observations (there were y' responses among the n'; the distribution of y, the number of responses among the n, would be determined from the prior information). Let 0 = n'/n and t*~ = x d m , - Oy'/n' - (1 - O)xdm,
484
A.L. Gould Then the argument used to derive (7) also provides an analogue that takes account of the placebo information provided by the trial: Reject H0 when tsp >- {Q - t*s}/O o r tsp ~ --{Q + t*~}/O,
(10)
where Q = X/~q(1 - q)[1/me + 1/n], q = (me + y + y')/(me + n), and ~ is an appropriate critical value for a chi-square distribution with 1 degree of freedom. The corresponding analogue of (8) follows immediately: Yl
Prob(Reject Ho) = ~ y =0
f(t, Y l m, n, a, b, c, d)
~ t ~ ( Q - tes)/o n
+ ~
~
(f(t, y l m , n , a , b , c , d ) .
(11)
Y=Y2 t-(Q+tes)/O~:
Ordinarily, the value of n will be approximately that of me or m~. Further research is needed to provide operationally useful guidelines for 0 (= n'/n). DISCUSSION The methods described here represent applications of empirical Bayes and prediction theory. Using information from the previous studies to estimate the parameters of the prior distribution is routine empirical Bayes. Equations (3), (8), (9), and (11) are expressions for the tail areas of appropriate predictive distributions. The methods described here are related procedures for incorporating historical information using Bayesian principles [16,17] but differ in h o w this information is used. Here, the information is used to determine the likelihood of a significant difference between the response rates of the patients on the active treatment(s) and the response rates that could have been provided by the patients on placebo, had there been any. This likelihood is calculated as a function of the results for placebo-treated patients from similar prior trials and the number of patients that might have been included in the activecontrolled trial. In other applications the historical placebo information is used to provide a single significance level, essentially as a function of the expected placebo-treated outcome [16,17] or are used to determine optimal allocation of experimental resources to test and control groups [16,18]. Incorporating external information into the decision-making process does not compromise the conduct of the trial. The approach described here does not benefit from a finding of "no difference", so there is no more advantage to conducting a sloppy or inadequate trial than there would be if a placebotreated group had been included. Demonstrating efficacy by significant improvement relative to placebo or a known active agent in the current trial has a long history of use and acceptance. In circumstances where a placebo-controlled trial can be done, it should be; if an active control can be included as well, so much the better. However, when a large, or possibly any, placebo-treated group is not a practical option, the approach described here provides a sensible context for designing, executing, and analyzing active-controlled trials. The comments by the Editor and reviewers have been very helpful in clarifying a number of points in this article. I am grateful for their insights.
Active-Controlled Trials
485
REFERENCES
1. Temple R: Difficulties in Evaluating Positive Control Trials. Proc A m Statist Assoc (Biopharmaceutical Sect):1-7, 1983 2. Durrleman S, Simon R: Planning and monitoring of equivalence studies. Biometrics 46:329-336, 1990 3. Gould AL: Placebo comparisons in active-controlled trials. Proc A m Statist Assoc (Biopharmaceutical Sect):255-265, 1987 4. Boissel J-P, Blanchard J, Panak E, Peyrieux J-C, Sacks H: Considerations for the meta-analysis of randomized clinical trials. Controlled Clin Trials 10:2,54-281, 1989 5. Pledger GW: Active control trials: Do they address the efficacy issue? (with discussion) Proc Am Statist Assoc (Biopharmaceutical Sect):1-10, 1986 6. Temple R: Government viewpoint of clinical trials. Drug Inform J 16:10--17, 1982 7. Light RJ, Pillemer DB: Summing Up: The Science of Reviewing Research. Cambridge, Harvard University Press, 1984 8. Hedges LV, Olkin I: Statistical Methods for Meta-Analysis. New York, Academic, 1985 9. Wormsley KG: Short-term treatment of duodenal ulceration. In: Cimetidine in the '80s, Baron JG, Ed. Edinburgh, Churchill Livingstone, 1981, pp 3--8 10. Dobrilla G, de Pretis G, Felter M, Chilovi F: Endoscopic double-blind clinical trial on ranitidine versus placebo in the short-term treatment of duodenal ulcer. Hepatogastroenterology 28:49-52, 1981 11. Porro GB, Petrillo M, Lazzaroni M: Ranitidine in the short-term treatment of duodenal ulcer: a multicenter endoscopic double-blind trial. In: The Clinical Use of Ranitidine. Proceedings of the Second International Symposium on Ranitidine, Misiewicz JJ, Wormsley KG, Eds. Oxford, Medicine Publishing Foundation, 1983 12. Berstad A, Kett K, Aadland E, Carlsen E, Frislid K, Saxhaug K, Kruse-Jensen A: Treatment of duodenal ulcer with ranitidine, a new histamine H2-receptor antagonist. Scand J Gastroenterol 15:637-639, 1980 13. Hirschowitz BI: Lessons from the U.S. multicenter trial of ranitidine treatments for duodenal ulcer. J (]In Gastroenterol 5 (suppl 1):115--122, 1983 14. Skellam JG: A probability distribution derived from the binomial distribution by regarding the probability of success as variable between the sets of trials. J R Stat Soc B 10:257-261, 1948 15. Miller LH: Table of percentage points of Kolmogorov statistics. J Am Stat Assoc 51:111-121, 1956 16. Pocock SJ: The combination of randomized and historical controls in clinical studies. J Citron Dis 29:175--188, 1976 17. Tarone RE: The use of historical control information in testing for a trend in proportion. Biometrics 38:215-220, 1982 18. Thall RF, Simon R: Incorporating historical control in planning phase II clinical trials. Star Med 9:215-228, 1990
ABSTRACT: Placebo-controlled efficacy trials may become more difficult to carry out with the increasing availability of effective therapies, especially for serious illnesses where denial of effective therapy may be objectionable ethically. Active-controlled trials aimed at establishing efficacy by demonstration of "equivalence" to "standard" therapy have potentially serious interpretational problems, and do not necessarily encourage good experimental practice. This article describes an alternative approach to the analysis of data from active-controlled trials using the information that makes an active-controlled trial necessary or desirable, namely a large, valid body of information about the consequence of using placebo. The approach uses information about placebo responses and also active agent responses from prior placebo-controlled trials to determine the likelihood of a significant active-placebo difference in an active-controlled trial, or in a trial with a vestigial placebo group. The sensitivity of the treatment comparisons depends directly on the quality of the design and execution of the active-controlled trial. The method is illustrated with data from trials of an H2-receptor antagonist in the treatment of acute duodenal ulcer. KEY WORDS: Empirical Bayes, predictive distributions, clinical trials, meta-analysis INTRODUCTION Establishing the effectiveness of a therapy for treating an illness ordinarily requires demonstrating that the therapy provides a superior clinical result to some alternative course of treatment such as n o t h i n g (or placebo to control for the act of treating) or an alternative agent k n o w n to be effective. The latter option could require impracticably large sample sizes if the standard a n d test therapies have similar effects. Effectiveness also could be established by demonstrating a clear relationship between dose a n d clinical response. In largescale clinical trials there is a real operational advantage to keeping the sample size as small as possible a n d the experimental design as simple as possible to assure that the trial will be carried out as the protocol requires. Consequently, placebo-controlled trials have long been the m e t h o d of choice. Placebo-controlled efficacy trials become more difficult to carry out with the increasing availability of effective therapies, especially in serious illnesses where withholding effective therapy m a y be medically inappropriate. Institutional Review Boards a n d treating physicians will have ethical objections to d e n y i n g active treatment to patients w h e n the consequences m a y be se-
Address reprint requests to: A. L. Gould, PhD, BL3-2, Merck, Sharp, and Dohme Research Laboratories, West Point, PA 19486. Received June 11, 1990; revised March 19, 1991.
474 0197-245(~91/$3.50
Controlled Clinical Trials 12:474--485 (1991) © Elsevier Science Publishing Co., Inc. 1991 655 Avenue of the Americas, New York, New York 10010
Active-Controlled Trials
475
rious, e.g., prophylactic treatment of patients recovering from a myocardial infarction, long-term control of hypertension. Once properly informed, patients may be reluctant to participate in a trial in which they have a substantial chance of receiving an inactive therapy when they know that feasible active therapies exist. Active-controlled trials, in which no patients receive placebo, therefore seem to be inevitable, at least in some circumstances. Regulatory approval for the marketing of a n e w drug requires unequivocal demonstration of its efficacy in at least (usually more than) one well-controlled trial. Active-controlled trials aimed at establishing the efficacy of a new agent by demonstrating its "equivalence" to the effect of some "standard" therapy tend to be unacceptable for several reasons [1]: 1. No generally accepted statistical methods exist for establishing "equivalence" or even "similarity." 2. "Equivalence" cannot be distinguished from lack of power for detecting a clinically meaningful difference (small or sloppy studies are unlikely to show differences). 3. " . . . positive-control studies do not provide the incentives toward conducting an excellent study that a placebo-controlled t r i a l . . , provides." 4. The expectation that everyone should do well in the trial introduces a bias that may be inconsistent with careful observation and measurement. 5. "Showing that two drugs are equivalent in a study does not demonstrate that either is effective; it shows that both were effective or that neither was." Equivalence also could be defined using confidence intervals for treatment effect difference instead of lack of statistical significance in the conventional sense. This is common practice in clinical pharmacology trials aimed at establishing bioequivalence, but also applies to clinical equivalence [2]. Careful execution is essential to the success of these trials. Confidence-based approaches can establish equivalence, but not necessarily effectiveness. Instead of demonstrating "equivalence," one can establish efficacy by exploiting what makes an active-controlled trial imperative: a b o d y of convincing evidence that the condition cannot go untreated for the duration of the study, and the existence of at least one feasible, effective therapy. With this information one can compute the likelihood that the n e w therapy would have proved more effective than placebo had the tri~l included a placebo-treated group. This is not the definitive evidence that a placebo-controlled trial would provide, but under the circumstances may be the best that can be done. The approach is very different from attempting to establish efficacy of a test therapy by demonstrating its "equivalence" to an active control. Regarding active-controlled trials in this light should encourage careful trial execution, since small, sloppy trials are unlikely to provide positive results. If the trial can include a small group of patients on placebo, then the demonstration of efficacy can be strengthened. What follows is an overview of the methodology as it applies to dichotomous outcomes; a more comprehensive discussion appears in Ref. 3. BASIC PROCEDURE
The outcomes of active-controlled trials can be evaluated usefully and realistically in terms of the likelihood that the outcomes observed for the patients
476
A.L. Gould on the "active" agents would have differed significantly from the outcome for a placebo-treated group had there been one. Because these trials lack placebo-treated groups, one cannot know what differences would have been observed. At best, one can determine the likelihood (or confidence) of a significant difference using the results of the trial outcome and the results observed for similar kinds of patients treated with placebo in similar trials. Of course, this requires assuming that the process generating the placebotreated group outcomes in the previous trials also would have generated the outcomes for a placebo-treated group in the active-controlled trial. The quality of design and execution of the trials providing the data to which the calculations are applied govern the credibility of the inferences this approach provides. This also is true for the trials providing information about the result of placebo treatment that the active-controlled trial lacks. This information must reflect the experience of similar kinds of patients in trials where the measurements are made similarly in the same time frame, so that the spectrum of potential placebo group outcomes realistically reflects what might have occurred had the active-controlled trial included a placebo group. Obtaining this information is not a simple task and requires careful attention to details [4]. Badly designed or executed trials will not yield credible conclusions. This is particularly true for active-controlled trials in view of regulatory concerns about their interpretation [1,5,6]. The prior information about placebo group responses must be carefully documented, scientifically valid, and relevant to the target population and experimental conditions of the active-controlled trial. Meta-analysis [7,8] provides a convenient and coherent w a y to accumulate this information.
Incorporating Prior Placebo Information The following example illustrates the process. A multiclinic trial was carried out to compare three dosage regimens of a new active agent with a standard agent in the treatment of acute duodenal ulcer. The key response variable was the proportion of patients whose ulcer was healed after four weeks of treatment; healing was defined as replacement of the ulceration by scar tissue or normal mucosa. The results for each regimen are shown in Table 1. No placebo-treated group was included in the trial. None of the between-group differences is significant. A high spontaneous healing rate in this trial cannot be ruled out in favor of treatment efficacy as an explanation for these findings, although the sample sizes suggest that efficacy is the more likely explanation. Understanding how a placebo-treated group might have performed can help in drawing an appropriate inference from these outcomes.
Table 1
Comparison of Three Dosage Regimens of a N e w Active Agent With a Standard Agent for Treatment of Duodenal Ulcer Regimen 1 Regimen 2 Regimen 3 Standard Number of patients 240 247 247 246 Number with healed ulcer 164 191 201 186 Healing rate (%) 68 77 81 76
477
Active-Controlled Trials
The first step in the analysis is to obtain information about previous trials of similar design, target population, and execution that included a placebo treatment. Table 2 summarizes the findings from a number of trials, some fairly large, comparing various active agents against placebo in the treatment of acute duodenal ulcer. In general, the trials were double-masked and included outpatients with duodenal ulcers endoscopically proven within 2-3 days before starting their (randomly assigned) treatment. As far as could be determined, the published trials and the present trial were conducted similarly. These 23 outcomes are consistent with the hypothesis that they are random outcomes from a single process; see "Checking assumptions," below. The outcomes from these 23 trials therefore appear to comprise a sample of outcomes that might be expected from groups of patients with acute duodenal ulcer receiving placebo for 4 weeks in double-masked trials. These 23 outcomes can be used to determine the likelihood that the outcomes from the present trial would have been significantly superior to the outcome for a placebo-treated group had one been included. The variation in the healing rates among the 23 trials is greater than can be accounted for by sampling variation from a parent binomial distribution with a single healing rate parameter. Thus, instead of a universal "true" healing rate for placebo-treated patients, it is more likely that each trial has its own "true" placebo healing rate that depends on its populations' demographics, diets, stress levels, medical characteristics, etc. Since neither the form of this dependence nor all of the test populations' characteristics are knowable, it certainly is more practical, and appears to be more realistic, to assume that some process generates trial-specific "true" rates at random. This would be true, for example, even if there were a common distribution for the responses of all possible placebo-treated patients: variations among trials in the apparent "true" rate would occur because of variations in the samples drawn for each trial. Table 2
Numbers and Percentages of Patients With Healed Duodenal Ulcers After 4 Weeks on Placebo in Double-Blind Controlled Trials a Healed at 4 Weeks
Healed at 4 Weeks
Site
N
N
(%)
Site
N
N
(%)
Belgium/Holland Czechoslovakia France France Hong Kong Hungary Ireland Italy Italy Norway Norway Switzerland
73 55 87 99 24 20 35 758 166 72 20 164
29 33 49 54 4 12 14 242 60 36 12 93
(40) (60) (56) (55) (17) (60) (40) (32) (36) (50) (60) (57)
Switzerland U.K. U.K. U.S.A. U.S.A. W. Germany W. Germany
106 143 151 340 195 123 101
61 41 47 167 80 74 63
(58) (29) (31) (49) (41) (60) (62)
Italy [10] Italy [11] Norway [12] U.S.A. [13]
17 80 24 168
5 23 11 76
(29) (29) (46) (45)
aAll results are from Ref. 9 except as otherwise noted.
478
A.L. Gould
Marginal Distribution of Outcomes Suppose that for any trial, the "true" placebo healing rate, p, is a random realization from a [5 distribution with parameters (a,b). Once a value of p has been "generated" for a trial, the number of patients healed out of n treated has a binomial (n,p) distribution. The joint probability function for the outcome and the value of p is the product of these two distributions, h(x, p ; n, a, b) = ( n ) B - ' ( a ,
b)lY +~-I ( 1 -
p)b+ . . . .
1.
(1)
The relationship between the outcome (x) and the process generating the "true" response rate for the trial follows from integrating (1) with respect to P, k(x;n,a,b)
= (n)B_~(a,b)B(a+x,b+
n-x),
(2)
a [5-binomial density [14]. This is the a priori likelihood function for the trial, i.e., the probability function of the outcome (x) in terms of the process that generates the (trial-specific) "true" response rates. The joint likelihood for m trials is the product of quantities like Eq. 2, L(x ; n, a, b ) = B - " ( a , b ) f i
(ni'~ B(a + xi, b + ni - xi) i= 1 \ / X i
Likelihood of a Significant Difference from Placebo Suppose that x patients responded out of m treated with an active agent. Also, let ~ denote the critical value for a chi-square variable with 1 degree of freedom corresponding to a two-tail test at some significance level (e.g., for a 5% level test, ~ = 3.84). Given x and m, an outcome of y responses out of n patients treated with placebo would provide a chi-square statistic value exceeding ~ if y >! y~ ~ yu m n ( 2 N x - 2~v. + NI;) + N{m 2 n2~ 2 + 4mnU~lx(m - x)} °5 2 m ( m N + n~) or
y ~ Y'{ ~ yL =
m n ( 2 N x - 2~x + NI~) - N ( m 2 n2~ 2 + 4 m n ~ N x ( m - x)}°'5 2 m ( m N + nl~)
where N = m + n , = smallest integer >- yu, and y~ = largest integer -< yL. T h e confidence that a sample of n patients on placebo would yield an outcome significantly different from that observed for the active agent (y responses out of m tested) is the probability that y ~ y[ or y >-- y~ using (2) as the probability function for y, Conf{Sign. Diff. [ m, x, n} = "~ k(y; n, a, b) + "~ k(y; n, a, b)
(3)
479
Active-Controlled Trials
Example For the 23 trials, a = 9.3 and b = 11.2. The expected average response rate is E(p) = a/(a + b) = 0.453, about 45%, so that almost half the patients on placebo could expect healed ulcers after 4 weeks. Substituting these values of a and b into (2) gives the overall estimated probability of the observed number of responses (to placebo) among n patients treated. This probability is used as the confidence distribution for the outcomes that might have been observed had a placebo-treated group been present. Table 3 displays the outcomes from the present trial and the results of performing the calculations described above. The confidence level exceeds 95% for three of the four groups even if a placebo-treated group of less than half the size of any of the active treatment groups is used. Therefore, if a placebo-treated group had been included, its healing rate very likely would have been significantly less than the healing rate for most, if not all, of the four active treatment groups.
Checking Assumptions Regarding the placebo outcomes from the 23 trials as realizations from some process generating "true" healing rates can be justified statistically (C. L. Mallows, personal communication). Each trial's outcome is a sample from a "known" distribution of potential outcomes for placebo-treated patients in that trial. Replacing the outcome by its corresponding distribution function value yields a realization from a uniform distribution, so the points representing these cdf values plotted against their relative ranks [corresponding values of the cdf for a uniform (0,1) distribution], will fall on or near the line connecting the points (0,0) and (1,1). When the points are obtained by using assumed rather than known study-specific placebo value distributions, the degree to which they coincide with this straight line reflects the degree to which the assumptions are consistent with the data. If the cdf values based on the assumed distributions deviate greatly from the line (as assessed by a Kolmogorov goodness-of-fit test, for example), then the data are unlikely to have arisen from the assumed distributions. Assuming that each of the 23 outcomes comes from a distribution of the form (2) yields a maximum absolute difference between the (ordered) calculated cdfs and the corresponding uniform (0,1) order statistics of 0.137. This
Table 3
Confidence of Significant Active-Placebo Differences Outcomes from present trial Placebo-treated group sample size" No.
Treatment Group Test, Reg. 1 Test, Reg. 2 Test, Reg. 3 Standard
N 240 247 247 246
Healed (%) 164 (68) 191 (77) 201 (81) 186 (76)
100 0.844 0.968 0.987 0.954
200 0.888 0.983 0.995 0.973
220 0.897 0.983 0.995 0.975
aConfidence levels corresponding to various placebo-treated group sample sizes.
240 0.898 0.984 0.995 0.976
260 0.899 0.986 0.996 0.978
480
A.L. Gould is less than the 20% critical value for a Kolmogorov test based on 23 observations [15]. The 23 findings therefore can be regarded as having been generated by a common process.
COMPENSATING FOR ACTIVE CONTROL BIAS
The patients' responses could be affected by their knowing that whatever treatment they receive is "active." This expectation or active control bias may cause a spuriously large (or small) apparent difference between the test agent's effect and the estimated placebo effect. The presence of active control bias can be detected and compensated for when the previous placebo-controlled trials include outcomes for the standard active agent used in the active-controlled trial. If the outcome for the standard fails well within the range of its outcomes in the placebo-controlled trials, e.g., within the interval containing the middle 80% of them, then expectation bias probably has not affected the active-controlled trials outcomes materially, and no adjustment should be necessary. Adjustment for a material expectation bias proceeds by applying the method described above to a measure of the difference between the treatments. If the bias acts to shift the response rate, i.e., ~(observed response) = treatment effect + bias, where the bias shift is the same for both treatments, then the arithmetic difference between the observed responses will estimate the arithmetic difference between the treatment effects. If the bias affects the response multiplicatively, i.e., ~{(observed response) - treatment effect × bias, where the bias factor is the same for both treatments, then the ratio of the actual responses will estimate the ratio for the treatment effects. The results from the placebo-controlled trials can be used to construct a confidence distribution for the difference between (or the ratio of) the response rates for patients treated with the standard active agent or placebo. Thus, if the activecontrolled trial had included a placebo-treated group, then a test of the null hypothesis of the equivalence of the test (e) and placebo (p) effects could be expressed as, "Reject H0 if t~p > c', where t~ denotes the difference between (or ratio of) the responses observed for the test active and placebo-treated groups. The test also could be expressed as "Reject Ho if tes + tsv > c (or tes × t~p > c)", where te~ and t~p are obvious analogues of t~p, or as "Reject/4o if tsv > c - te~ (or tsv > c/tes)."
(4)
If the trial does not contain a placebo-treated group, tsv cannot be observed. At best, one can use the findings from similar previous trials to construct a con-
481
Active-Controlled Trials
fidence distribution for t~p and use this distribution to determine the probability content of the rejection region (4). The result is a statement of the confidence, based on the prior information, that the outcome for the test active agent would have differed significantly from the outcome for a placebo-treated group if one had been included in the trial. This is very much like the approach for incorporating just the responses to placebo in previous trials, differing only in the variate whose confidence distribution is computed, and in the fact that considerably less information may be available. For example, only 4 of the 23 placebocontrolled trials in acute duodenal ulcer compared the active control from the current trial with placebo, so only the outcomes for those trials could be used to construct a confidence distribution for the difference between (or the ratio of) the response rates for the standard and placebo. Correcting an additive bias with dichotomous responses proceeds as follows. The correction when the bias affects the responses multiplicatively is similar; Ref. 3 provides details. For either the placebo or the standard active group in any of the prior trials, the joint likelihood for the observations and rate parameter has the form (1). Integrating the response rate parameters out of the product of these separate likelihoods yields the joint marginal likelihood for the responses,
(m)
( y ) B-~(a, b) B - l ( c , d ) B(a + x , b + m - x) B(c + y , d + n - y).
(5) In (5), m, x, a, and b refer to the active group and n, y, c, and d refer to the placebo-treated group. The parameters in Eq. 5 are estimated separately for the placebo and standard active groups using data from the previous trials containing both groups as described above (assuming two response categories). Given estimates of a, b, c, and d, the confidence or predictive distribution of tsp can be determined from (5) and used to compute the confidence associated with (4). The confidence or predictive distribution of tsp is determined by rewriting (5) as a function of t~p and y, and then summing over the range of y values corresponding to the value of tsp. If t = x/m - y/n, then the joint marginal likelihood has the form
fit, y lm, n,a,b,c,d)
(6)
m ! ( y ) X B(a + m(t + y/n), b + m ( 1 -
t-y/n))B(c+
y, d + n - y )
F(m(t + y/n) + 1)F(m(1 - t - y/n) + 1)B(a,b)B(c,d) where t takes rational values between - 1 and 1 and the range of y depends on t as follows when m is a multiple of n (the case most likely to be useful in practice):
t = k/m
~y
t = -k/m~y
= 0, 1. . . . .
[(m - k)/s],k = 0, 1. . . . .
= {k/s},{k/s} + 1. . . . .
n , k = 1. . . . .
m, m,
where Ix] denotes the largest integer -< x, s = re~n, and {x} denotes the smallest integer -> x. Summing this marginal joint likelihood over the values of y
482
A.L. Gould corresponding to a given value of t yields the marginal distribution of t (given m, n, a, b, c, and d),
f(tlm,
n,a,b,c,d
) = ~f(t,y[m,n,a,b,c,d). Y
The next step is to incorporate this information about t into the rule for accepting or rejecting the null hypothesis. The usual uncorrected chi-square test with 1 degree of freedom based on (4) w h e n the treatment group differences are expressed as the arithmetic differences of the observed response proportions can be reduced, with some algebra, to Reject H0 when tsp >>-Q - tes o r tsp ~
--Q - tes,
(7)
where Q = X/~qil - q)(1/ne + 1/n), q = (xe + y)/(n~ + n), ~ is an appropriate critical value for a chi-square distribution with I degree of freedom (e.g., 3.84 for a 5% level test), x~ denotes the number of patients responding to the test active agent out of ne treated, and y denotes the number of patients responding to placebo out of n treated. The confidence that H0 would be rejected given the results from the studies comparing the standard active agent to placebo is the probability that (7) would be satisfied; this is given by summing (6) over the region of (t[ = tsp], y) values satisfying (7), Yl
Prob(RejectH0) = ~
f(t, y l m ,
~
n,a,b,c,d)
y~O t~Q;--tes
(8)
i
+
~,
[(t, y l m ,
n,a,b,c,d),
y~y2 t-~-(Q+tes)
where yl denotes the largest value of y for which (1 - y/n) --- Q - t~, and y2 denotes the smallest value of y for which y/n >- Q + te~.In practice, one would set m = n = number of patients treated with the standard active agent in the active-controlled trial.
Example Four of the 23 trials compared the active control of the present trial against placebo. The results follow in Table 4. For the active treatment ~ = 171 and b = 49.9; for the placebo-treated group, ~ = 26.5 and d = 42.7. The values of ~ and d reflect less study-to-study variation in the observed healing rate
Table 4 Comparison of Active Control of Present Trial against Placebos in Four of 23 Trials Active Study Study D Study P Study B Study H
Number Treated 18 84 25 187
Placebo Number Healed 15 65 23 137
Number Treated 17 80 24 168
Number Healed 5 23 11 76
483
Active-Controlled Trials
for these four trials than for all 23 trials. For the present active-controlled trial, 246 patients were observed on the active standard and 164 out of 240 patients healed on the first regimen of the test agent. The value of (8) for a 5% level (1% level) test is 0.989 (0.971), so that, as before, the outcome of a placebotreated group, had one been included, very likely would have been significantly inferior to the outcome from any of the active agents in the trial. INCLUDING A SMALL PLACEBO-TREATED GROUP Sometimes it may be possible to include a small placebo-treated group in an active-controlled trial. This could happen, for example, in conditions such as acute duodenal ulcer where the consequences of 4 weeks of placebo are unlikely to be serious, so that withholding effective medication does not pose a major ethical problem. Having at least some patients on placebo, even if fewer than on the active agents, could reduce the likelihood of expectation bias because not all patients would receive an "active" treatment. Treatment-placebo differences large enough to reach statistical significance for key efficacy parameters should comprise evidence for efficacy on a par with a conventional placebo-controlled trial. The efficacy of the test agent can be evaluated in more than one w a y if the treatment-placebo differences do not reach statistical significance. We consider two approaches when the differences are not significant. The first approach is analogous to the argument leading to expression (3). If (i) the failure to reach significance is due to an insufficient placebo group size, and (ii) the data from prior trials provide a reasonable guide to the distribution of placebo responses the present trial might have obtained, then expression (3) provides the confidence that a significant treatment-placebo difference would have been found if there had been n observations on placebo instead of, say, n': Conf{Sign. Diff. I m , x , n , n ' , y ' } =
~
k(y;n - n',a,b) +
Y~YL _yt
~
k(y;n - n',a,b).
(9)
y~yU_yt
The values of a and b would be obtained from the prior information as before. If there is a lot of prior evidence, as there should be, then the current trial values (n', y') should not affect the estimates, so whether they are included or not will not matter greatly. If there is little prior information, then it is not at all clear that a placebo-controlled trial should not be done. The second approach combines the placebo data from the current trial with information about the difference between the active control and placebo from the prior trials. Suppose that n' patients received placebo in the current trial and that y' of them responded; also, suppose that xe patients responded out of me on the test therapy, and that xs out of ms responded on the standard agent. The objective is to determine the likelihood of rejecting H0 if there had been n (> n') placebo observations (there were y' responses among the n'; the distribution of y, the number of responses among the n, would be determined from the prior information). Let 0 = n'/n and t*~ = x d m , - Oy'/n' - (1 - O)xdm,
484
A.L. Gould Then the argument used to derive (7) also provides an analogue that takes account of the placebo information provided by the trial: Reject H0 when tsp >- {Q - t*s}/O o r tsp ~ --{Q + t*~}/O,
(10)
where Q = X/~q(1 - q)[1/me + 1/n], q = (me + y + y')/(me + n), and ~ is an appropriate critical value for a chi-square distribution with 1 degree of freedom. The corresponding analogue of (8) follows immediately: Yl
Prob(Reject Ho) = ~ y =0
f(t, Y l m, n, a, b, c, d)
~ t ~ ( Q - tes)/o n
+ ~
~
(f(t, y l m , n , a , b , c , d ) .
(11)
Y=Y2 t-(Q+tes)/O~:
Ordinarily, the value of n will be approximately that of me or m~. Further research is needed to provide operationally useful guidelines for 0 (= n'/n). DISCUSSION The methods described here represent applications of empirical Bayes and prediction theory. Using information from the previous studies to estimate the parameters of the prior distribution is routine empirical Bayes. Equations (3), (8), (9), and (11) are expressions for the tail areas of appropriate predictive distributions. The methods described here are related procedures for incorporating historical information using Bayesian principles [16,17] but differ in h o w this information is used. Here, the information is used to determine the likelihood of a significant difference between the response rates of the patients on the active treatment(s) and the response rates that could have been provided by the patients on placebo, had there been any. This likelihood is calculated as a function of the results for placebo-treated patients from similar prior trials and the number of patients that might have been included in the activecontrolled trial. In other applications the historical placebo information is used to provide a single significance level, essentially as a function of the expected placebo-treated outcome [16,17] or are used to determine optimal allocation of experimental resources to test and control groups [16,18]. Incorporating external information into the decision-making process does not compromise the conduct of the trial. The approach described here does not benefit from a finding of "no difference", so there is no more advantage to conducting a sloppy or inadequate trial than there would be if a placebotreated group had been included. Demonstrating efficacy by significant improvement relative to placebo or a known active agent in the current trial has a long history of use and acceptance. In circumstances where a placebo-controlled trial can be done, it should be; if an active control can be included as well, so much the better. However, when a large, or possibly any, placebo-treated group is not a practical option, the approach described here provides a sensible context for designing, executing, and analyzing active-controlled trials. The comments by the Editor and reviewers have been very helpful in clarifying a number of points in this article. I am grateful for their insights.
Active-Controlled Trials
485
REFERENCES
1. Temple R: Difficulties in Evaluating Positive Control Trials. Proc A m Statist Assoc (Biopharmaceutical Sect):1-7, 1983 2. Durrleman S, Simon R: Planning and monitoring of equivalence studies. Biometrics 46:329-336, 1990 3. Gould AL: Placebo comparisons in active-controlled trials. Proc A m Statist Assoc (Biopharmaceutical Sect):255-265, 1987 4. Boissel J-P, Blanchard J, Panak E, Peyrieux J-C, Sacks H: Considerations for the meta-analysis of randomized clinical trials. Controlled Clin Trials 10:2,54-281, 1989 5. Pledger GW: Active control trials: Do they address the efficacy issue? (with discussion) Proc Am Statist Assoc (Biopharmaceutical Sect):1-10, 1986 6. Temple R: Government viewpoint of clinical trials. Drug Inform J 16:10--17, 1982 7. Light RJ, Pillemer DB: Summing Up: The Science of Reviewing Research. Cambridge, Harvard University Press, 1984 8. Hedges LV, Olkin I: Statistical Methods for Meta-Analysis. New York, Academic, 1985 9. Wormsley KG: Short-term treatment of duodenal ulceration. In: Cimetidine in the '80s, Baron JG, Ed. Edinburgh, Churchill Livingstone, 1981, pp 3--8 10. Dobrilla G, de Pretis G, Felter M, Chilovi F: Endoscopic double-blind clinical trial on ranitidine versus placebo in the short-term treatment of duodenal ulcer. Hepatogastroenterology 28:49-52, 1981 11. Porro GB, Petrillo M, Lazzaroni M: Ranitidine in the short-term treatment of duodenal ulcer: a multicenter endoscopic double-blind trial. In: The Clinical Use of Ranitidine. Proceedings of the Second International Symposium on Ranitidine, Misiewicz JJ, Wormsley KG, Eds. Oxford, Medicine Publishing Foundation, 1983 12. Berstad A, Kett K, Aadland E, Carlsen E, Frislid K, Saxhaug K, Kruse-Jensen A: Treatment of duodenal ulcer with ranitidine, a new histamine H2-receptor antagonist. Scand J Gastroenterol 15:637-639, 1980 13. Hirschowitz BI: Lessons from the U.S. multicenter trial of ranitidine treatments for duodenal ulcer. J (]In Gastroenterol 5 (suppl 1):115--122, 1983 14. Skellam JG: A probability distribution derived from the binomial distribution by regarding the probability of success as variable between the sets of trials. J R Stat Soc B 10:257-261, 1948 15. Miller LH: Table of percentage points of Kolmogorov statistics. J Am Stat Assoc 51:111-121, 1956 16. Pocock SJ: The combination of randomized and historical controls in clinical studies. J Citron Dis 29:175--188, 1976 17. Tarone RE: The use of historical control information in testing for a trend in proportion. Biometrics 38:215-220, 1982 18. Thall RF, Simon R: Incorporating historical control in planning phase II clinical trials. Star Med 9:215-228, 1990
