PAPERS
2'Cholesterol lowering trials in coronary heart disease: frequency of citation and outcome /( Ul
vnskov
Abstract
Objective-To see if the claim that lowering cholesterol values prevents coronary heart disease is true or if it is based on citation of supportive trials only. Design-Comparison of frequency ofcitation with outcome of all controlled cholesterol lowering trials using coronary heart disease or death, or both, as end point. Subjects-22 controlled cholesterol lowering trials. Results-Trials considered by their directors as supportive of the contention were cited almost six times more often than others, according to Science Citation Index. Apart from trials discontinued because of alleged side effects of treatment, unsupportive trials were not cited after 1970, although their number almost equalled the number considered supportive. In three supportive reviews the outcome of the selected trials was more favourable than the outcome of the excluded and ignored trials. In the 22 controlled cholesterol lowering trials studied total and coronary heart disease mortality was not changed significantly either overall or in any subgroup. A statistically significant 0-32% reduction in non-fatal coronary heart disease seemed to be due to bias as event frequencies were unrelated to trial length and to mean net reduction in cholesterol value; individual changes in cholesterol values were unsystematically or not related to outcome; and after correction for a small but significant increase in non-medical deaths in the intervention groups total mortality remained unchanged (odds ratio 1-02). Conclusion-Lowering serum cholesterol concentrations does not reduce mortality and is unlikely to prevent coronary heart disease. Claims of the opposite are based on preferential citation of supportive trials.
Lund, Sweden U Ravnskov, Dozent
Correspondence to: Rabygatan 2, S-22361 Lund, Sweden. BMJ 1992;305:15-9
BMJ
VOLUME
305
Introduction A causal association between dietary fat, serum cholesterol concentration, and coronary heart disease is reportedly proved beyond doubt,' and trial reviewers have concluded that the risk of coronary heart disease is reduced substantially when the serum cholesterol concentration is lowered.24 However, counter views5"8 have been presented, and in two reviews of cholesterol lowering trials only a nonsignificant lowering of coronary mortality9 or no evidence at all'0 was found. As all five reviews were incomplete the disagreement may be due either to the proponents having based their idea on preferential citation of supportive trials or to their critics having ignored such trials. This study is an attempt to determine which of these opposing views is true by
4 JULY 1992
analysing all controlled trials and comparing their frequency of citation with their outcome. Methods TRIAL SELECTION
The criteria for including a controlled trial were designed and successful lowering of cholesterol concentrations aimed at preventing coronary heart disease, and total mortality or incidence of coronary heart disease reported as end points. Compiling a fair selection of blind trials was not feasible because most cholesterol lowering drugs have typical and frequent side effects and laboratory records may reflect treatment. Both open and blind trials were therefore accepted. Trials using angiography were excluded. The incidence of coronary heart disease was defined as including fatal myocardial infarctions and sudden deaths and definite, non-fatal myocardial infarctions in patients surviving to the end of the trial. Intention to treat data were used when available. A total of 22 trials satisfied these criteria (table I). As different drug treatments were used in the five large branches of the coronary drug project'-24 they were analysed separately, making a total of 26 trials. In the trial of Marmorston et aP2 three different oestrogen preparations were used but because of their small size the three groups were treated as one. In five trials the participants were stratified into intervention and control groups by age, sex, previous coronary heart disease, other diseases, or various laboratory test results" 3 25 35; in m one trial cluster allocation was used32; in the rest random allocation was used. Two trials'3 26 did not give mortality from coronary heart disease, four'4 15 202' did not give total mortality, and three'2'326 did not give the number of non-fatal cases of coronary heart disease. Five trials included both sexes, but in three02'2m it was not possible to extract all relevant figures for each sex and event separately. Thus the number of people at risk for each event differed slightly in the subgroup calculations. CALCULATIONS
For each event odds ratios and 95% confidence intervals for treatment groups versus control groups were calculated. Odds ratios were the ratio between events and non-events in the treatment group and events and non-events in the control group. Confidence intervals were derived by the logit method. To test whether an odds ratio differed significantly from unity the difference between the observed (0) and expected (E) numbers of each event in each trial was calculated. E was dx n/N, where d was the total number of events, n the number of subjects in the treatment group, and N the number of subjects in both groups together. The Z statistic was derived from
15
100 -
0
0
0
D
A
*' 410
0
Unsupportive
Suppor tive
trials
trial s
1-Mean annual citations of trials or trial branches conside,red supportive (right) by authors a2s
(O-E)/VV with two tailed p=0(05 corresponding to Z= 1 96, and the variance (V) of each finding was derived from E(I -n/N)(N-d)/(N- 1). With the Mantel-Haenszel test as used by Yusuf et aP6 mean weighted odds ratios with 95% confidence intervals and Z scores were calculated for all trials together and for various constellations of trials-that is, unifactorial and multifactorial, drug and dietary, primary and secondary preventive trials, and trials of varying length. The mean annual number of citations of the trials according to Science Citation Index was calculated when possible up to five years after their publication. The number of mutual citations between the trials was also recorded and analysed. Only citations which appeared in the main report of the trial were considered. The mean weighted odds ratio was calculated for the trials selected for the mentioned reviews2"490 and for those excluded or overlooked.
FIG
regards non-fatal coronary hetart disease and of trials consideretd unsupportive (left).
Reut
esuts CITATION ANALYSIS
To study the influence of trial outcome on frequency of citation the trials were divided into two groupsthose (n= 14) regarded by the authors as supportive and those (n= 10) considered unsupportive. (Most regarded by the authors as trials supportive were not supportive by conventional statistics (table I)). The multiple risk factors intervention trial29 was excluded from this analysis as owing to its small effect on serum cholesterol concentration it was considered inconclusive (fig 1). The mean annual number of citations of the suppor-
O =Intervention group 10G10 subjects. Open symbols=Primary prevention Closed symbols=secondary prevention.
tive trials was 40 and of the unsupportive trials 7-4. Frequent citation of a paper was not correlated with trial size (fig 1), nor was it due to its having been published in a major journal-that is, the New England Journal ofMedicine, Lancet, BMJ, orJAMA. Supportive trials published in major journals (n= 8) were cited on average 61 times a year and unsupportive trials (n= 10) eight times a year. This difference was also evident when comparing papers published in the same journal. Thus in the first four years after its publication the supportive Lipid Research Clinics trial30 was cited 109, 121, 202, and 180 times each year whereas the unsupportive trial of Miettinen et aP' was cited six, five, three times, and once each year. Even minor differences in outcome were reflected in citation frequency. The Newcastle trial2' with an odds ratio of 0-58 (morbidity and mortality from coronary heart disease combined) was cited 12 times a year whereas the Scottish trial20 with an odds ratio of 0-82, published in the same issue of the journal, was cited only nine times each year. The total number of mutual citations was 72:49 to supportive or inconclusive (multiple risk factor intervention) trials, 23 to unsupportive ones (fig 2). Nine of the citations to unsupportive trials concerned the discontinued branches of the coronary drug project.'92223 As these trials were considered as choice failures by the investigators (wrong drug) and not real treatment failures only 14 citations concerned trials which were considered unsupportive. Only four supportive trial reports cited unsupportive trials.'213'68 Thus 102021242527283032-34 of 14 reports of trials considered supportive or inconclusive did not mention any unsupportive trial.
TABLE I -Pertinent data from trials No of
Fatal coronary heart disease
All deaths
Non-fatal coronary heart disease
investigated
Trials Oliver and Boyd" Marmorston etal"' Starnler et al"
Type of study
No individuals Mean intervention (intervention group/ Duration annual group/ Odds ratio controls) (years) citations controls
SUDr SUDr SUDr SUDi SUDi
Leren" MRC soyabean"
SUDi
5 5 5 3 2 5
Dayton et all' Coronary drug project,
PUDi
4 7
SUDr SUDr Men
1-5 5
Researchcommittee" Rose et al"
5mgoestrogen" Scottish Society"
SUIDi
6t 8t 7 7
15t 9
1St 12
9t
Women SUDr
Newcastle2'
12
50/50 285/147 156/119 123/129 28/52 206/206 199/194 424/422 1119/2789 350/367 288/305
Frantztal" Frantz et al"
0o61
41/56 28/31 174/177
0-67 0-86
91/193
1-19
0-58
6 19
95
20/24 5/4 37/50 25/25 41/50
0-85 2 60 0-68 0-97 0-80
26/24t 20/26
1-17 0-77 0-74 0-72
6-76 6-84 7 70 7 07 6 06
8-3 8-8 13 9 13 5 12 7
67/133 34/35
1-27 1-02
56/76 20/37
1-88** 0 54
7-10
16
27/37
0-73
23/38
2/611
0-61 0-26
6 71 6 37 7 02
11 10 15
119/274
1-13
78/175
1-16
6 50
12
3/7 24/31
160/339
4-7
8
1101/2789
219/525
1-07
162/410
1 00
94/242
0-98
62
78t
1119/2789
273/709
0-95
203/535
0 93
84/304
0-66**
6-55
99
6-2 2
78
281/709 17/27
1-00
0-62
0 90 0 40 1-08
114/304 13/24 22/19
0-94 0-53 1-13
39/28
0-92 1 60
195/535 9/22 10/9
2'Cholesterol lowering trials in coronary heart disease: frequency of citation and outcome /( Ul
vnskov
Abstract
Objective-To see if the claim that lowering cholesterol values prevents coronary heart disease is true or if it is based on citation of supportive trials only. Design-Comparison of frequency ofcitation with outcome of all controlled cholesterol lowering trials using coronary heart disease or death, or both, as end point. Subjects-22 controlled cholesterol lowering trials. Results-Trials considered by their directors as supportive of the contention were cited almost six times more often than others, according to Science Citation Index. Apart from trials discontinued because of alleged side effects of treatment, unsupportive trials were not cited after 1970, although their number almost equalled the number considered supportive. In three supportive reviews the outcome of the selected trials was more favourable than the outcome of the excluded and ignored trials. In the 22 controlled cholesterol lowering trials studied total and coronary heart disease mortality was not changed significantly either overall or in any subgroup. A statistically significant 0-32% reduction in non-fatal coronary heart disease seemed to be due to bias as event frequencies were unrelated to trial length and to mean net reduction in cholesterol value; individual changes in cholesterol values were unsystematically or not related to outcome; and after correction for a small but significant increase in non-medical deaths in the intervention groups total mortality remained unchanged (odds ratio 1-02). Conclusion-Lowering serum cholesterol concentrations does not reduce mortality and is unlikely to prevent coronary heart disease. Claims of the opposite are based on preferential citation of supportive trials.
Lund, Sweden U Ravnskov, Dozent
Correspondence to: Rabygatan 2, S-22361 Lund, Sweden. BMJ 1992;305:15-9
BMJ
VOLUME
305
Introduction A causal association between dietary fat, serum cholesterol concentration, and coronary heart disease is reportedly proved beyond doubt,' and trial reviewers have concluded that the risk of coronary heart disease is reduced substantially when the serum cholesterol concentration is lowered.24 However, counter views5"8 have been presented, and in two reviews of cholesterol lowering trials only a nonsignificant lowering of coronary mortality9 or no evidence at all'0 was found. As all five reviews were incomplete the disagreement may be due either to the proponents having based their idea on preferential citation of supportive trials or to their critics having ignored such trials. This study is an attempt to determine which of these opposing views is true by
4 JULY 1992
analysing all controlled trials and comparing their frequency of citation with their outcome. Methods TRIAL SELECTION
The criteria for including a controlled trial were designed and successful lowering of cholesterol concentrations aimed at preventing coronary heart disease, and total mortality or incidence of coronary heart disease reported as end points. Compiling a fair selection of blind trials was not feasible because most cholesterol lowering drugs have typical and frequent side effects and laboratory records may reflect treatment. Both open and blind trials were therefore accepted. Trials using angiography were excluded. The incidence of coronary heart disease was defined as including fatal myocardial infarctions and sudden deaths and definite, non-fatal myocardial infarctions in patients surviving to the end of the trial. Intention to treat data were used when available. A total of 22 trials satisfied these criteria (table I). As different drug treatments were used in the five large branches of the coronary drug project'-24 they were analysed separately, making a total of 26 trials. In the trial of Marmorston et aP2 three different oestrogen preparations were used but because of their small size the three groups were treated as one. In five trials the participants were stratified into intervention and control groups by age, sex, previous coronary heart disease, other diseases, or various laboratory test results" 3 25 35; in m one trial cluster allocation was used32; in the rest random allocation was used. Two trials'3 26 did not give mortality from coronary heart disease, four'4 15 202' did not give total mortality, and three'2'326 did not give the number of non-fatal cases of coronary heart disease. Five trials included both sexes, but in three02'2m it was not possible to extract all relevant figures for each sex and event separately. Thus the number of people at risk for each event differed slightly in the subgroup calculations. CALCULATIONS
For each event odds ratios and 95% confidence intervals for treatment groups versus control groups were calculated. Odds ratios were the ratio between events and non-events in the treatment group and events and non-events in the control group. Confidence intervals were derived by the logit method. To test whether an odds ratio differed significantly from unity the difference between the observed (0) and expected (E) numbers of each event in each trial was calculated. E was dx n/N, where d was the total number of events, n the number of subjects in the treatment group, and N the number of subjects in both groups together. The Z statistic was derived from
15
100 -
0
0
0
D
A
*' 410
0
Unsupportive
Suppor tive
trials
trial s
1-Mean annual citations of trials or trial branches conside,red supportive (right) by authors a2s
(O-E)/VV with two tailed p=0(05 corresponding to Z= 1 96, and the variance (V) of each finding was derived from E(I -n/N)(N-d)/(N- 1). With the Mantel-Haenszel test as used by Yusuf et aP6 mean weighted odds ratios with 95% confidence intervals and Z scores were calculated for all trials together and for various constellations of trials-that is, unifactorial and multifactorial, drug and dietary, primary and secondary preventive trials, and trials of varying length. The mean annual number of citations of the trials according to Science Citation Index was calculated when possible up to five years after their publication. The number of mutual citations between the trials was also recorded and analysed. Only citations which appeared in the main report of the trial were considered. The mean weighted odds ratio was calculated for the trials selected for the mentioned reviews2"490 and for those excluded or overlooked.
FIG
regards non-fatal coronary hetart disease and of trials consideretd unsupportive (left).
Reut
esuts CITATION ANALYSIS
To study the influence of trial outcome on frequency of citation the trials were divided into two groupsthose (n= 14) regarded by the authors as supportive and those (n= 10) considered unsupportive. (Most regarded by the authors as trials supportive were not supportive by conventional statistics (table I)). The multiple risk factors intervention trial29 was excluded from this analysis as owing to its small effect on serum cholesterol concentration it was considered inconclusive (fig 1). The mean annual number of citations of the suppor-
O =Intervention group 10G10 subjects. Open symbols=Primary prevention Closed symbols=secondary prevention.
tive trials was 40 and of the unsupportive trials 7-4. Frequent citation of a paper was not correlated with trial size (fig 1), nor was it due to its having been published in a major journal-that is, the New England Journal ofMedicine, Lancet, BMJ, orJAMA. Supportive trials published in major journals (n= 8) were cited on average 61 times a year and unsupportive trials (n= 10) eight times a year. This difference was also evident when comparing papers published in the same journal. Thus in the first four years after its publication the supportive Lipid Research Clinics trial30 was cited 109, 121, 202, and 180 times each year whereas the unsupportive trial of Miettinen et aP' was cited six, five, three times, and once each year. Even minor differences in outcome were reflected in citation frequency. The Newcastle trial2' with an odds ratio of 0-58 (morbidity and mortality from coronary heart disease combined) was cited 12 times a year whereas the Scottish trial20 with an odds ratio of 0-82, published in the same issue of the journal, was cited only nine times each year. The total number of mutual citations was 72:49 to supportive or inconclusive (multiple risk factor intervention) trials, 23 to unsupportive ones (fig 2). Nine of the citations to unsupportive trials concerned the discontinued branches of the coronary drug project.'92223 As these trials were considered as choice failures by the investigators (wrong drug) and not real treatment failures only 14 citations concerned trials which were considered unsupportive. Only four supportive trial reports cited unsupportive trials.'213'68 Thus 102021242527283032-34 of 14 reports of trials considered supportive or inconclusive did not mention any unsupportive trial.
TABLE I -Pertinent data from trials No of
Fatal coronary heart disease
All deaths
Non-fatal coronary heart disease
investigated
Trials Oliver and Boyd" Marmorston etal"' Starnler et al"
Type of study
No individuals Mean intervention (intervention group/ Duration annual group/ Odds ratio controls) (years) citations controls
SUDr SUDr SUDr SUDi SUDi
Leren" MRC soyabean"
SUDi
5 5 5 3 2 5
Dayton et all' Coronary drug project,
PUDi
4 7
SUDr SUDr Men
1-5 5
Researchcommittee" Rose et al"
5mgoestrogen" Scottish Society"
SUIDi
6t 8t 7 7
15t 9
1St 12
9t
Women SUDr
Newcastle2'
12
50/50 285/147 156/119 123/129 28/52 206/206 199/194 424/422 1119/2789 350/367 288/305
Frantztal" Frantz et al"
0o61
41/56 28/31 174/177
0-67 0-86
91/193
1-19
0-58
6 19
95
20/24 5/4 37/50 25/25 41/50
0-85 2 60 0-68 0-97 0-80
26/24t 20/26
1-17 0-77 0-74 0-72
6-76 6-84 7 70 7 07 6 06
8-3 8-8 13 9 13 5 12 7
67/133 34/35
1-27 1-02
56/76 20/37
1-88** 0 54
7-10
16
27/37
0-73
23/38
2/611
0-61 0-26
6 71 6 37 7 02
11 10 15
119/274
1-13
78/175
1-16
6 50
12
3/7 24/31
160/339
4-7
8
1101/2789
219/525
1-07
162/410
1 00
94/242
0-98
62
78t
1119/2789
273/709
0-95
203/535
0 93
84/304
0-66**
6-55
99
6-2 2
78
281/709 17/27
1-00
0-62
0 90 0 40 1-08
114/304 13/24 22/19
0-94 0-53 1-13
39/28
0-92 1 60
195/535 9/22 10/9
