LESSONS LEARNED FROM LARGE-SCALE CLINICAL TRIALS*** DANIEL DEYKINt BOSTON
Over the past five years, it has been my pleasure to direct the Clinical Trials Program of the Department of Veterans Affairs. In that capacity, I have reviewed over 100 applications to fund clinical trials. I have observed the planning and peer review of at least 60 of those clinical trial applications, and I have participated in the administration of the 25 that have been funded by us as well as by other agencies including the National Institutes of Health and the Department of Defense. From that vantage point, as well as from experience with the trials (and tribulations) of anticoagulant therapy, (1) I have learned several lessons that I should like to describe under five headings: 1) when to do (or not do) a large clinical trial; 2) how selection of subjects affects outcome; 3) the cathedral effect-the drift to complexity; 4) alpha terror vs beta phobia-the subgroup dilemma; and 5) do overviews (meta-analysis) provide the "truth"? WHEN TO DO LARGE-SCALE INTERVENTION TRIALS. We need to consider a few basic elements that determine the size of clinical trials. The event rate in the proportion of the population under study who serve as controls is Pc, and the event rate in the proportion of the population who receive the intervention to be tested is Pi. The therapeutic benefit of the intervention is (Pc - Pi)/Pc. We consider the therapeutic benefit to be extreme if it is 0.60 or greater, large if it lies between 0.40 and 0.60, and moderate if it lies between 0.25 and 0.60. We also need to define the alpha error (Type I)-that is saying a difference exists between Pc and Pi when in fact there is no difference. The beta error (Type II) consists of saying there is no difference between Pc and Pi when in fact there is a difference. We calculate a sample size sufficiently large to reduce the chance of making the Type I error to be less than 1 in 20, or
Over the past five years, it has been my pleasure to direct the Clinical Trials Program of the Department of Veterans Affairs. In that capacity, I have reviewed over 100 applications to fund clinical trials. I have observed the planning and peer review of at least 60 of those clinical trial applications, and I have participated in the administration of the 25 that have been funded by us as well as by other agencies including the National Institutes of Health and the Department of Defense. From that vantage point, as well as from experience with the trials (and tribulations) of anticoagulant therapy, (1) I have learned several lessons that I should like to describe under five headings: 1) when to do (or not do) a large clinical trial; 2) how selection of subjects affects outcome; 3) the cathedral effect-the drift to complexity; 4) alpha terror vs beta phobia-the subgroup dilemma; and 5) do overviews (meta-analysis) provide the "truth"? WHEN TO DO LARGE-SCALE INTERVENTION TRIALS. We need to consider a few basic elements that determine the size of clinical trials. The event rate in the proportion of the population under study who serve as controls is Pc, and the event rate in the proportion of the population who receive the intervention to be tested is Pi. The therapeutic benefit of the intervention is (Pc - Pi)/Pc. We consider the therapeutic benefit to be extreme if it is 0.60 or greater, large if it lies between 0.40 and 0.60, and moderate if it lies between 0.25 and 0.60. We also need to define the alpha error (Type I)-that is saying a difference exists between Pc and Pi when in fact there is no difference. The beta error (Type II) consists of saying there is no difference between Pc and Pi when in fact there is a difference. We calculate a sample size sufficiently large to reduce the chance of making the Type I error to be less than 1 in 20, or
