Review

A multistudy analysis investigating systematic differences in cardiovascular trial results between Europe and Asia Louise C Hartley,1,2 Alan J Girling,2 Russell J Bowater,3 Richard J Lilford2 1

Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK 2 School of Health and Population Sciences, University of Birmingham, Birmingham, UK 3 Faculty of Engineering, Universidad Autónoma de Querétaro, Cerro de las Campanas, Santiago de Querétaro, Qro, Mexico Correspondence to Dr Louise C Hartley, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry CV4 7AL, UK; [email protected] Received 15 November 2013 Revised 15 October 2014 Accepted 13 November 2014 Published Online First 5 December 2014

ABSTRACT Objective To assess whether there are differences in the results of cardiovascular trials between Europe and Asia using data from an extensive collection of randomised controlled trials. Study design and setting All meta-analyses containing randomised controlled trials (RCT’s) for the treatment or prevention of cardiovascular diseases were searched for in The Cochrane Library (2000–2008) and MEDLINE (2005–2008). Analysis was then conducted within and over each meta-analysis which satisfied given criteria. Separate estimates of treatment effect were calculated for Europe and Asia in each meta-analysis and then compared. Estimates of a common intercontinental difference over all meta-analyses were also calculated and meta-regression was performed. This was performed for both fatal and non-fatal end points. Results The literature search identified 59 metaanalyses that satisfied the inclusion criteria. After exclusion, the number of meta-analyses reporting greater effect sizes in Asia than in Europe was significantly more than would be expected by chance (fatal 12/14, p=0.013; non-fatal 23/32, p=0.020). Conclusions This study provides some evidence that for cardiovascular interventions treatment effect estimation differs between Europe and Asia, with respect to both fatal and non-fatal end points.

INTRODUCTION

To cite: Hartley LC, Girling AJ, Bowater RJ, et al. J Epidemiol Community Health 2015;69:397–404.

Clinical trials are a fundamental part of the development of safe and effective healthcare interventions and are seen as the best way to evaluate their effects.1 As a consequence of globalisation, they are now no longer conducted in selected countries but are carried out worldwide. While this has many advantages, such as broadening the representativeness of those who will receive the intervention, it also means that the trial results of any country, even those from countries inexperienced in conducting trials, can influence the clinical practice of another country. It is imperative to consider the transferability of clinical trial data between countries since some countries may differ to others in important facets2 that could influence treatment effect estimation and/or the reporting of results. In some countries, for example, less emphasis may be placed on good trial design and conduct, which may lead to exaggerated treatment effect estimates and bring the reliability of their results into question. One study looking at this assessed the quality of reporting and conduct in randomised controlled trials conducted

in China.3 Results showed that the standard of reporting in Chinese trial was in the main poor with many studies not reporting randomisation techniques, allocation concealment methods or blinding procedures. Furthermore, there were low drop-out rates in the majority of studies and ethical issues were generally inadequately reported. Authors concluded that until there is substantial improvements in trial reporting, trial conduct cannot be clearly ascertained. Another study found that trials conducted in China and India scored considerably lower on trial quality than those from western countries with Chinese trials being found to be associated with an increased rate of positivei trial results.4 Countries may also differ in the amount and sources of bias that influence the dissemination of their trial results. Researchers in some countries may be more likely to publish their research in their native language5 or in local journals,6 especially if their results are negative. In one study, pairs of randomised controlled trials were compared to assess whether authors were more likely to report statistically significant trial results in English or German. It was found that positive trial results were more likely to be reported in English language journals and negative results in German national journals.7 Furthermore, researchers in some countries may only report the outcomes that produce favourable results even though other outcome measures were recorded.7 For instance, one Chinese trial reported only six outcomes in the results section of their paper even though 22 outcomes were indicated in the methods section. Of these six reported outcomes, five had significant results.8 Some trial researchers may also face hurdles in getting negative findings published in certain countries. Evidence also suggests that some countries only publish positive trial results.2 4 9 In one recent study, for instance, trials conducted in less developed countries reported more favourable results than trials conducted in developed countries and produced more favourable treatment effect estimates.2 This is supported by previous work which compared the results of trials from Russia, China, Japan and Taiwan to those published in England. Even though 75% of the English trials found the

i

In this paper, a positive trial result refers to where the intervention is beneficial compared to the control. A negative trial result refers to where the intervention is equal or inferior to the control.

Hartley LC, et al. J Epidemiol Community Health 2015;69:397–404. doi:10.1136/jech-2013-203646

397

Review intervention to be superior to the control, 99% of Chinese trials, 97% of Russian trials, 95% of Taiwanese trials and 85% of Japanese trials reported this to be the case. Furthermore, none of the Chinese or Russian trials reported that the intervention was ineffective.9 There may also be real between-country differences in treatment effect. Indeed, a belief that the effectiveness of interventions differs between countries may make those developing clinical guidelines skeptical about using foreign clinical data.10 Evidence to suggest that there may be systematic differences in trial results between regions comes from a study which examined whether treatment effect estimation differed between Europe and North America in a large set of cardiovascular trials. This study found that for non-fatal end points, interventions, relative to controls, were more favoured in trials conducted in Europe than in North America. However, for fatal end points, no evidence was found to suggest that there were inter-continental differences in treatment effect.11 Regional differences in treatment effect may be the consequence of between-country differences in adjunctive treatments, genetic profiles, and resources to support intervention delivery, among others. It may also be the case that one country has recruited patients at higher risk compared to the other country and while the intervention benefit on a ratio scale would be consistent across these two countries, on an absolute scale the benefit would be greater in the country with the higher risk participants. The extrapolation of foreign clinical data for use in other countries is increasing in clinical healthcare research. Therefore, studies should try to quantify whether the results of clinical trials published in one region differ from those published in another region. This study aims to examine whether treatment effect estimation differs for cardiovascular disease interventions between Europe and Asia, continents with different levels of experience in trial conduct, trial regulation and dissemination of trial results.

METHODS Databases searched A literature search was conducted in MEDLINE ( January 2005–December 2008) and The Cochrane Library ( January 2000–December 2008) for published meta-analyses of RCT’s indexed by the MESH term Cardiovascular Diseases (CVD).

Inclusion criteria Inclusion criteria for the meta-analyses were: 1. Involve RCT’s of adults who had CVD or were at high risk of CVD. 2. Involve an intervention aimed at treating a problem with the heart or blood vessels in the chest cavity. 3. For at least one of the outcomes the meta-analysis should include at least one trial published in or after 1990 conducted in Europe and at least one trial published in or after 1990 conducted in Asia. 4. No language restrictions were placed on the trials included in this study.

Data extraction and management Data extraction was conducted by one reviewer but independently checked by a second reviewer. For each systematic review reference data were extracted for each study as well as information on the intervention investigated, the comparator treatment, CVD explored, and country and/or continent information. Information on the continent where each trial was conducted 398

was initially sought from the systematic reviews. Where this was not specified, the RCT’s abstract was examined, and, if information about the continent was still not found, the full text of the original trial report was consulted. However, information about the continent was not always reported and so in these cases trial authors were contacted directly. In most cases authors did not respond or provide the information required and so continent was determined by the addresses of the authors themselves. The countries making up Europe and Asia were determined according to the definition provided by the statistics division of the United Nations.12 Outcome data for a fatal and/or non-fatal end point was also extracted from the systematic review. Recording fatal and nonfatal details was relatively easy in some cases as the systematic review only contained one fatal and/or non-fatal meta-analysis. In other systematic reviews, however, more than one fatal and/ or non-fatal meta-analysis was reported. In these cases, the most commonly reported fatal and/or non-fatal end point in each systematic review was chosen, that is, the fatal (or non-fatal) end point whose meta-analysis had combined the greatest number of trials. For instance, where there are two fatal end points and one meta-analysis combined nine trials and the other five trials, then the former was identified as the most commonly reported fatal end point. Ties in the number of trials combined were settled by choosing the end point that contained the highest number of participants.

Data synthesis and analysis In each meta-analysis, results for the fatal (or non-fatal) end points were separately combined in Europe and in Asia to provide individual estimates of treatment effect for each continent. This meant that no truly new meta-analysis was conducted but that new subgroups for existing meta-analyses were produced. If an end point was event based and event rates were provided the trial results for each continent were combined over relative risks (RR). However, if event rates were not reported and the original meta-analysis reported effect estimates using ORs then trial results were combined over ORs. It should also be pointed out that beneficial outcomes were coded in the negative direction reflecting a decreased risk of fatal or non-fatal events. All ratio summary statistics underwent natural log transformations before being used in analysis. Trials were combined using the random-effects model (DerSimonian and Laird). The difference between each continent was then calculated, on the log scale, by subtracting Asia’s treatment effect estimate from that of Europe. A positive (+) or negative (−) symbol was used to indicate the continent in which the intervention performed best because if Asia had a larger treatment effect estimate than Europe (when Asia’s estimate was subtracted from Europe’s estimate), the sign of the calculated difference would be negative. The total number of meta-analyses favouring each continent was then calculated. A binomial sign test was used to see if there was a significant difference in the number of meta-analyses favouring each continent. Estimates for a common inter-continental effect were also produced. These showed the average between-continent difference in treatment effect over all of the included meta-analyses. They were calculated by combining the estimates of betweencontinent difference of all the included meta-analyses, using the random-effects model. This was performed for each type of effect measure, for example, an estimate was calculated for all meta-analyses where RR had been used, while a separate estimate was calculated for those meta-analyses that had provided

Hartley LC, et al. J Epidemiol Community Health 2015;69:397–404. doi:10.1136/jech-2013-203646

Review OR. These were estimated on the log scale and were calculated for both fatal and non-fatal end points. Lastly, as an attempt to explain any between study heterogeneity, the continent of each study was included in a meta-regression. A random-effects meta-regression was conducted for both fatal and non-fatal end points and these were carried out using Stata.

RESULTS Fatal end points The literature search identified 59 meta-analyses that satisfied the inclusion criteria. Of these, only 20 contained at least one European and one Asian trial with fatal end point data (see table 1). Six of these meta-analyses did not report which was the intervention and which the control. In such cases it was not possible to calculate which continent favoured the intervention (see table 2 for study details). Of the 14 meta-analyses that identified their intervention and control groups, 12 showed interventions to perform best in Asia. To find the significance of this, a binomial sign test was conducted under the null hypothesis that the effect of interventions, relative to controls, was the same in both Europe and Asia. This showed that the effectiveness of interventions, compared to controls, was statistically significantly different between Europe and Asia (2-sided p=0.013). From the estimates fora common inter-continental effect, it was found that for RR (13 meta-analyses) the mean log difference over all the meta-analyses was 0.222 (95% CI 0.028 to 0.416) while for OR (1 meta-analysis) the mean log difference was 0.791 (95% CI −7.55 to 9.13). The estimate of intercontinental effect for RR was statistically significant at the 5% level, suggesting that when treatment effect was measured using RR, there was a statistically significant difference in the effect of interventions between Europe and Asia, with, on average, interventions performing better, relative to controls, in Asia than in Europe. The result for OR was inconclusive. The random effects meta-analysis summary estimate for fatal end points was −0.71 (95% CI −1.13 to −0.29) with an I2 of 21% and heterogeneity test p value of 0.16. To attempt to explain some of the between study heterogeneity we included the continent of each study in a meta-regression. This gave the random effects summary estimate for Europe as −0.62 (95% CI −1.25 to 0.01) and for those in Asia as −0.80 (95% CI −1.43 to −0.17). Performing each meta-analysis separately revealed substantial heterogeneity in the European studies (I2=48%) but not in the Asian studies (I2=0%). Although there were no two meta-analyses based on exactly the same set of trials there were some addressing similar questions which could be seen as ‘overlapping’. By overlapping, it is meant that any one meta-analysis shares some trial results with at least one other included meta-analysis to the extent that the estimates produced may be regarded as being to some degree not independent of one another. Given that overlapping meta-analyses were included, it may be that the result above is an artefact of their inclusion. Therefore, the impact of overlapping was investigated and six meta-analyses with fatal end points were considered to be overlapping. The percentage of meta-analyses where intervention was favoured in Asia rather than Europe was 83% (5/6) for overlapping meta-analyses but 88% (7/8) for non-overlapping meta-analyses. This suggested that the proportion of meta-analyses favouring Asia was not exaggerated as a consequence of including overlapping meta-analyses.

Non-fatal end points Of the 59 meta-analyses identified by the literature search, 39 contained at least one European and one Asian trial with nonfatal data (see table 3). Seven of these meta-analyses did not report which was the intervention and which the control. In such cases it was not possible to calculate which continent favoured the intervention (see table 4). Of the 32 meta-analyses that identified their intervention and control groups, 23 showed the intervention to perform better in Asia than in Europe. A binomial sign test was conducted under the null hypothesis that the effect of interventions, relative to controls, was the same in Europe and Asia. This showed that the effectiveness of interventions, compared to controls, was statistically significantly different between Europe and Asia (2-sided p=0.020). From the estimates of inter-continental effect, it was found that for RR (24 meta-analyses) the mean log difference was 0.161 (95% CI 0.021 to 0.301), while for OR (1 meta-analysis) the mean log difference was 0.548 (95% CI −3.08 to 4.18). For continuous end points, the mean difference (MD) over all the data was found to be −1.34 (95% CI −4.7 to 2.01). The estimate of inter-continental effect for RR was statistically significant at the 5% level which suggested that when treatment effect was measured using RR there was a significant difference in the effectiveness of interventions between Europe and Asia, with interventions on average performing better (relative to controls) in Asia than in Europe. The findings for both OR and MD did not reach statistical significance and so no conclusions can be drawn. The random effects meta-analysis summary estimate for nonfatal end points was −0.68 (95% CI −1.19 to −0.19) with an I2=73% and heterogeneity test p value of 0.00. To try to explain some of the between study heterogeneity we included the continent of each study in a meta-regression. This gave the random effects summary estimate for Europe as −0.83 (95% CI −1.55 to −0.11) and for those in Asia as −0.53 (95% CI −1.25 to 0.18). Performing each meta-analysis separately revealed considerable heterogeneity in the European studies (I2=81%) and substantial heterogeneity in the Asian studies (I2=54%). Twenty meta-analyses were identified as overlapping and 12 considered non-overlapping. The percentage of meta-analyses where the intervention performed better in Asia than Europe was 65% (13/20) for overlapping meta-analyses but 83% (10/ 12) for non-overlapping meta-analyses, suggesting that the proportion of meta-analyses favouring Asia over Europe had not been exaggerated as a consequence of including overlapping meta-analyses.

DISCUSSION This study has aimed to investigate the existence of treatment effect differences between Europe and Asia for CVD interventions. It showed that there was some evidence for differences in treatment effect between Europe and Asia when investigating within and over meta-analyses for both fatal and non-fatal end points. For instance, there was a statistically significant difference in the number of meta-analyses whose interventions, relative to controls, performed better in Asia than in Europe. A major source of bias in any meta-analysis is publication bias, and this may help to account for the findings of this study. It may be the case that trials that favour the intervention are more likely to be published in Asia than in Europe. It may be, however, that the potential biases in the source data did not

Hartley LC, et al. J Epidemiol Community Health 2015;69:397–404. doi:10.1136/jech-2013-203646

399

Article reference number

Intervention

Control

End point

13

Phosphodiesterase III inhibitors for HF

Placebo

14

Vasopressin for cardiac arrest Antiarrhythmics after cardioversion of AF Intravenous magnesium for acute MI

Epinephrine Placebo, drugs for rate control or no treatment Placebo

Cardiovascular mortality Death within 24 h All-cause mortality

G-CSF therapy for acute MI NPPV for Cardiogeneic pulmonary oedema Adjunctive mechanical devices for acute MI PCI for late reperfusion after MI MIDCAB for proximal stenosis of the left anterior descending artery Intracoronary cell therapy for acute MI Late PCI for acute MI Thrombectomy or embolic protection devices with PCI for acute MI Stem cell treatment for acute MI Amiodarone for prevention of AF

15

16

17 18

19

20 21

22

Hartley LC, et al. J Epidemiol Community Health 2015;69:397–404. doi:10.1136/jech-2013-203646

23 24

25 26

Number of trials (Europe/Asia)

Treatment effect measure

Treatment effect estimate Europe (95% CI)

Treatment effect estimate Asia (95% CI)

Best Region* (p value of between-continent difference)

2/1

Relative risk

1.562 (1.02 to 2.40)

0.51 (0.05 to 5.60)

Asia (0.369)

2/1 5/1

Relative risk Relative risk

0.991 (0.95 to 1.03) 1.019 (0.79 to 1.31)

0.83 (0.68 to 1.01) 0.62 (0.03 to 14.3)

Asia (0.086) Asia (0.759) Asia (0.475)

5/1

Relative risk

0.776 (0.61 to 0.99)

0.34 (0.04 to 3.21)

Placebo Standard medical care

Mortality by time of admission Death Hospital Mortality

3/1 6/4

Relative risk Relative risk

1.377 (0.28 to 6.89) 0.587 (0.37 to 0.92)

3.63 (0.16 to 84.11) 0.40 (0.18 to 0.87)

Standard care or stenting

30-day mortality

5/2

Relative risk

1.00 (0.47 to 2.15)

0.73 (0.26 to 2.07)

Asia (0.628)

Standard therapy PCI

Death Mortality

2/1 3/1

Relative risk Relative risk

0.96 (0.41 to 2.23) 0.93 (0.45 to 1.91)

0.22 (0.03 to 1.82) 0.12 (0.01 to 2.43)

Asia (0.204) Asia (0.194)

Standard care Usual management PCI alone

Death Death Mortality

3/1 5/1 9/2

Relative risk Relative risk Relative risk

0.51 (0.15 to 1.66) 0.45 (0.22 to 0.92) 0.70 (0.16 to 2.98)

1.07 (0.07 to 16.33) 0.18 (0.02 to 1.45) 0.26 (0.00 to 281.07)

Europe (0.619) Asia (0.4007) Asia (0.786)

No treatment or Placebo Placebo or routine treatment

Mortality Death

4/1 3/1

Relative risk OR

0.59 (0.21 to 1.68) 1.79 (0.00 to 1169.4)

0.33 (0.01 to 7.81) 0.81 (0.00 to 153.8)

Asia (0.739) Asia (0.853)

Europe (0.590) Asia (0.402)

*Region where intervention performs best relative to the control (according to the data collected). The p value is for the difference in the estimates of treatment efficacy (eg, relative risk, OR or mean difference) between Europe and Asia (under the null hypothesis of no difference). AF, atrial fibrillation; G-CSF, granulocyte-colony stimulating factor; HF, heart failure; MIDCAB, minimally invasive direct coronary artery bypass MI, myocardial infarction; NPPV, non-invasive positive pressure ventilation PCI, percutaneous coronary intervention.

Table 2 Article reference number 27 28 29 30 31

32

Details of meta-analyses with fatal end points that do not identify which is the intervention group and which is the control

Intervention 1

Intervention 2

End point

Stenting for Small Vessel CAD Stenting for Acute MI PCI LMWH for STEMI SES for Coronary artery disease

PTCA Balloon angioplasty CABG surgery UFH PES

Minimally invasive left internal thoracic artery bypass for isolated lesions of left anterior descending artery

Percutaneous coronary artery stenting

Death 30-day mortality All-cause mortality 30-day mortality Patients experiencing death Mortality at maximum follow-up

Number of trials (Europe/Asia)

Treatment effect Measure

Treatment effect estimate Europe (95% CI)

Treatment effect estimate Asia (95% CI)

Best Region* (p value of between-continent difference)

6/1 3/2 7/1 1/1 7/3

Relative risk Relative risk Relative risk Relative risk Relative risk

0.859 1.259 0.574 0.570 1.010

0.12 0.40 1.18 0.80 1.04

NA NA NA NA NA

3/1

Relative risk

(0.46 to (0.37 to (0.32 to (0.26 to (0.75 to

1.61) 4.31) 1.03) 1.25) 1.36)

2.32 (0.51 to 10.48)

(0.01 (0.10 (0.11 (0.39 (0.48

to 2.13) to 1.68) to 12.74) to 1.64) to 2.26)

1 (0.15 to 6.82)

(0.187) (0.234) (0.567) (0.526) (0.944)

NA (0.500)

*Region where intervention performs best relative to the control (according to the data collected). The p value is for the difference in the estimates of treatment efficacy (eg, relative risk, OR or mean difference) between Europe and Asia (under the null hypothesis of no difference). CABG, coronary artery bypass graft; CAD, coronary artery disease; LMWH, low-molecular-weight heparin; MI, myocardial infarction; NA, not applicable; PCI, percutaneous coronary intervention; PES, paclitaxel-eluting stents; PTCA, percutaneous transluminal coronary angioplasty; SES, sirolimus-eluting stents; UFH, unfractionated heparin.

Review

400

Table 1 Details of meta-analyses with fatal end points

Hartley LC, et al. J Epidemiol Community Health 2015;69:397–404. doi:10.1136/jech-2013-203646

Table 3 Details of meta-analyses with non-fatal end points Article reference number 13

14

15

16

17 18

19

20 22 23 24

25 33

34 35

36

37

38 39 40 41

42 43 44

Treatment effect measure

Treatment effect estimate Europe (95% CI)

Treatment effect estimate Asia (95% CI)

Best Region* (p value of between-continent difference)

Intervention

Control

End point

Phosphodiesterase III inhibitors for HF Vasopressin for cardiac arrest

Placebo

Worsening heart failure

3/2

Relative risk

1.52 (0.77 to 3.01)

0.42 (0.11 to 1.62)

Asia (0.100)

Epinephrine

2/1

Relative risk

0.79 (0.36 to 1.74)

0.59 (0.38 to 0.91)

Asia (0.524)

Antiarrhythmics after cardioversion of AF Intravenous magnesium for acute MI G-CSF therapy NNPV for cardiogenic pulmonary oedema Adjunctive mechanical devices for acute MI PCI for late reperfusion after MI Intracoronary cell therapy Late PCI for acute MI Thrombectomy or embolic protection devices with PCI for acute MI Stem cell treatment for acute MI Vitamin-mineral supplements for CAD Calcium channel antagonists Glycoprotein IIb/IIIa inhibitors for PCI Prophylactic steroids for cardiopulmonary bypass Glycoprotein IIb/IIIa inhibitors for PCI ACEI or ARB Off pump CABG surgery G-CSF administration ACEI or ARB for prevention of AF

Placebo, drugs for rate control or no treatment Placebo

Failure of return of spontaneous circulation AF recurrence

11/2

Relative risk

0.70 (0.61 to 0.80)

0.80 (0.64 to 1.01)

Europe (0.332)

Heart failure

4/1

Relative risk

0.79 (0.65 to 0.97)

0.66 (0.39 to 1.10)

Asia (0.512)

Placebo Standard Medical care

Instent restenosis Endotracheal intubation rate

4/1 4/1

Relative risk Relative risk

1.96 (−0.31 to 4.25) 0.86 (0.58 to 1.28)

7.79 (4.67 to 11.1) 0.13 (0.01 to 2.38)

Europe (0.003) Asia (0.206)

Standard care or stenting

Distal embolisation

4/2

Relative risk

0.60 (0.27 to 1.37)

0.40 (0.21 to 0.75)

Asia (0.432)

Standard therapy Standard medical therapy Usual management PCI alone

MI Target vessel revascularisation Non-fatal MI Target vessel revascularisation

2/1 6/1 5/1 6/2

Relative Relative Relative Relative

1.72 (0.52 1.09 (0.67 0.71 (0.30 0.94 (0.30

0.38 (0.11 0.36 (0.02 0.38 (0.11 0.66 (0.02

Asia (0.092) Asia (0.494) Asia (0.427) Asia (0.853)

No treatment or Placebo Placebo

Incidence of restenosis Restenosis

5/1 2/1

Relative risk Relative risk

1.15 (0.71 to 1.87) 0.84 (0.34 to 2.06)

0.20 (0.01 to 3.97) 0.56 (0.06 to 5.31)

Asia (0.256) Asia (0.740)

Adenosine Placebo or standard care

Reversion rate MI within 30 days

2/1 4/1

Relative risk Relative risk

1.70 (0.31 to 9.35) 0.86 (0.58 to 1.28)

0.97 (0.39 to 2.41) 0.13 (0.01 to 2.38)

Asia (0.600) Asia (0.206)

Placebo or standard care

New onset AF

3/1

Relative risk

0.84 (0.57 to 1.25)

1.25 (0.49 to 3.16)

Europe (0.450)

Placebo

30-day urgent revascularisation AF Stroke Binary restenosis rate Development of AF

5/1

Relative risk

0.56 (0.35 to 0.88)

2.74 (0.12 to 63.63)

Europe (0.327)

6/1 9/1 4/1 6/1

Relative Relative Relative Relative

0.83 (0.67 0.41 (0.18 0.97 (0.61 0.83 (0.67

0.60 (0.37 0.35 (0.01 2.75 (0.24 0.60 (0.10

Asia (0.234) Asia (0.928) Europe (0.410) Asia (0.736)

Off pump CABG surgery G-CSF for acute MI Amiodarone prophylaxis for cardiothoracic surgery Amiodarone for prevention of AF

Placebo or alternative therapy On pump CABG surgery Placebo or no treatment Placebo, other antihypertensive therapy or treatment drug plus irbesartan On pump CABG surgery Placebo Placebo Placebo, Metoprolol or routine treatment

MI Angiographic restenosis Postoperative AF

15/1 4/2 6/1

Prevention of postcardiac surgery AF

5/1

risk risk risk risk

risk risk risk risk

to to to to

to to to to

5.71) 1.76) 1.67) 2.95)

1.02) 0.95) 1.53) 1.02)

to to to to

to to to to

1.37) 8.41) 1.37) 21.39)

0.97) 8.56) 31.72) 3.76)

Relative risk Relative risk Relative risk

0.74 (0.46 to 1.19) 1.00 (0.63 to 1.59) 0.72 (0.56 to 0.93)

1.06 (0.15 to 7.36) 0.85 (0.23 to 3.10) 0.36 (0.18 to 0.71)

Europe (0.721) Europe (0.809) Asia (0.058)

OR

0.47 (0.10 to 2.12)

0.27 (0.01 to 7.34)

Asia (0.767) Continued

401

Review

45

Number of trials (Europe/Asia)

Article reference number 46

47

48 49

50 51

52

Number of trials (Europe/Asia)

Treatment effect measure

Treatment effect estimate Europe (95% CI)

Treatment effect estimate Asia (95% CI)

Best Region* (p value of between-continent difference)

−0.36 (−4.19 to 3.47)

Asia (0.853)

Intervention

Control

End point

β-Blocker therapy for heart failure ACE inhibition

Placebo

Mean change in QoL score

1/1

Mean difference

−0.28 (−1.69 to 1.13)

Diastolic volume

6/1

Mean difference

−2.37 (−15.95 to 11.21)

Autologous bone marrow cells Bone marrow derived cell transplantation

Placebo, conventional treatment or no treatment Standard medical therapy Placebo, optimal therapy or no treatment

2/1 7/5

Mean difference Mean difference

G-CSF therapy PCI

Placebo or blank control Optimal treatment

4/3 1/1

Bone marrow cells

Placebo or no treatment

Change in EF Difference in mean improvement in left ventricular ejection fraction Change in LVEF Change in EF from baseline to follow-up LVEF change at follow-up

5/1

−18 (−88.43 to 52.53)

Asia (0.669)

3.41 (−11.33 to 18.17) 2.10 (−2.26 to 6.45)

13 (4.96 to 21.04) 8.15 (−0.19 to 16.48)

Europe (0.264) Asia (0.207)

Mean difference Mean difference

0.47 (−7.24 to 8.18) 4 (−1.55 to 9.55)

3.33 (−9.17 to 15.83) 4 (−7.41 to 15.41)

Asia (0.703) None (1)

Mean difference

2.60 (−2.25 to 7.45)

6.7 (−11.61 to 25.01)

Asia (0.671)

Hartley LC, et al. J Epidemiol Community Health 2015;69:397–404. doi:10.1136/jech-2013-203646

*Region where intervention performs best relative to the control (according to the data collected). The p value is for the difference in the estimates of treatment efficacy (eg, relative risk, OR or mean difference) between Europe and Asia (under the null hypothesis of no difference). AF, atrial fibrillation; CABG, coronary artery bypass graft; CAD, coronary artery disease; G-CSF, granulocyte-colony stimulating factor; HF, heart failure; LMWH, low-molecular-weight heparin; LVEF, left ventricular ejection fraction; MI, myocardial infarction; PCI, percutaneous coronary intervention.

Table 4 Article reference number 27 28 29 30 31 32

53

Details of meta-analyses with non-fatal end points that do not identify which group is the intervention and which the control Number of trials (Europe/Asia)

Intervention 1

Intervention 2

End point

Stenting for small vessel CAD Stenting for acute MI PCI LMWH for STEMI SES Minimally invasive left internal thoracic artery bypass Drugs, behavioural therapy or exercise for secondary prevention of Coronary artery disease

PTCA Balloon angioplasty CABG surgery UFH PES Percutaneous coronary artery stenting Detraining, low intensity exercise, rest or no treatment

Repeat revascularisation Revascularisation at 30 days Stroke 30-day reinfarction Myocardial infarction Repeat revascularisation

6/2 3/2 6/1 1/1 7/3 4/2

Treatment outcome for time domain heart rate variability indices

13/3

Treatment effect measure

Treatment effect estimate Europe (95% CI)

Treatment effect estimate Asia (95% CI)

Best Region* (p value of between-continent difference)

Relative risk Relative risk Relative risk Relative risk Relative risk Relative risk

0.74 (0.55 0.27 (0.04 0.36 (0.11 0.74 (0.45 0.87 (0.61 0.26 (0.15

0.72 0.32 0.20 1.29 0.73 0.25

NA NA NA NA NA NA

Mean difference

0.43 (−0.16 to 1.01)

to to to to to to

1.00) 1.73) 1.23) 1.23) 1.22) 0.47)

(0.41 to (0.07 to (0.01 to (2.45 to (0.47 to (0.06 to

1.26) 1.56) 4.78) 0.68) 1.16) 1.04)

0.33 (−0.80 to 1.46)

(0.924) (0.893) (0.725) (0.183) (0.572) (0.930)

NA (0.878)

*Region where intervention performs best relative to the control (according to the data collected). The p value is for the difference in the estimates of treatment efficacy (eg, relative risk, OR or mean difference) between Europe and Asia (under the null hypothesis of no difference). CABG, coronary artery bypass graft; CAD, coronary artery disease; MI, myocardial infarction; NA, not applicable; PCI, percutaneous coronary intervention; STEMI, ST segment elevation myocardial infarction.

Review

402

Table 3 Continued

Review greatly affect the results of this study, and there is therefore a ‘genuine’ inter-continental difference with interventions actually performing better in Asia than in Europe. It may also be the case that some countries have a tendency to conduct RCTs of new treatments only on subgroups of patients who may be expected to benefit more from new treatment than the type of patients who generally participate in other countries clinical trials. Furthermore, it may be that in Asia less emphasis is placed on good trial design and conduct compared to Europe, leading to exaggerated treatment effect estimates and the differences in treatment effect between Europe and Asia found in this study. However, further exploration is required in order to ascertain the factors contributing to inter-continental differences in treatment effect. Few studies have directly assessed differences in treatment effect between continents or countries but several studies have highlighted an interaction between country and treatment effect54 which this study’s results support. This study’s results also suggest that trials from Asia tend to produce more positive results (results that favour the intervention) when compared to Europe, a finding consistent with others studies.4–5 9

Limitations As with all meta-analytic techniques, there are potential sources of bias in this study. While a comprehensive search strategy was used in this study not all published and unpublished systematic reviews may have been located, and some relevant data may therefore have been excluded. Furthermore, this study only included systematic reviews published in English. This may have created a language bias since systematic reviews not published in English would not have been identified and used. Another limitation of this study is the inclusion of overlapping meta-analyses. However, as the raw data used in this study was taken from meta-analyses, the inclusion of duplicated trial results is inevitable. This study also only investigated whether a difference in treatment effect estimation between Europe and Asia existed and did not examine the magnitude of this difference or investigate the true source of the variation found. However, factors that may cause inter-continental differences would have been hard to identify for this study due to the large aggregation of data involved. Lastly, this study was unable to conduct analysis at country level. This was because few of the included trials provided country information and authors were not forthcoming about the countries in which their trials were conducted.

Strengths Since the analysis has its foundations in the meta-analytic approach, it has all of the strengths of this method. First, it followed a structured methodology involving meticulous review and analysis of all trials contained in the included meta-analyses. This overcame the biases that can be associated with basing conclusions on only a few trials. Second, this method permitted small trials, or those with non-significant effects, to be included in the analysis so that they contributed to the overall results of this study. In doing so, few data were wasted and the impact of bias that could have resulted from not including such trials (that could have distorted the results of this study) was reduced. This study also conducted a detailed search to determine the countries and continents where each trial originated. This search involved the examination of the original trial reports and entailed contacting authors to obtain country and continent information. Without this, a thorough examination of intercontinental differences in treatment effectiveness could not have

been conducted, as details such as these are rarely presented in systematic reviews themselves.

CONCLUSION Notwithstanding some weaknesses, this study has provided some evidence for the existence of inter-continental differences in treatment effect. It has shown that treatment effect estimation differs between Europe and Asia for both fatal and non-fatal end points. However, this study is unable to say anything about the magnitude of this difference or anything about its source. Further exploration is required in order to ascertain the factors contributing to inter-continental differences in treatment effect and to determine the magnitude of such differences. It is hoped that by providing a better understanding of intercontinental differences, improvements can be made in the way that trial data is extrapolated between continents. Furthermore, on the basis of this study’s findings it is hoped that improvements can be made to the evidence used as the foundation for guideline recommendations and in intervention approval processes and that this will impact on the quality of support, services and interventions patients receive and the outcomes they are able to achieve.

What is already known on this subject ▸ Some countries may produce more favourable trial results than others. However, few studies have specifically investigated whether this is due to genuine differences in treatment effect.

What this study adds ▸ This study provides some evidence that treatment effect differs between regions. ▸ Intercontinental differences in treatment effect should be taken into account when clinical data is transferred between regions.

Contributors The acquisition of data was carried out by LCH but was checked by RJB. Data analysis and interpretation was conducted by LCH and AJG. LCH drafted the paper with all authors providing comments. All authors contributed to the concept and design of the study. Final approval was given by all authors. Funding This work was supported by the Engineering and Physical Sciences Research Council of the UK through the Multidisciplinary Assessment of Technology Centre for Healthcare (MATCH) programme (GR/S29874/01). Competing interests None. Provenance and peer review Not commissioned; externally peer reviewed.

REFERENCES 1

2

3 4

Wood L, Egger M, Gluud LL, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 2008;336:601–5. Panagiotou OA, Contopoulos-Ioannidis DG, Ioannidis JP, et al. Comparative effect sizes in randomised trials from less developed and more developed countries: meta-epidemiological assessment. BMJ 2013;346:1–19. Zhang D, Freemantle N, Jordan R, et al. A critical evaluation of randomized controlled trials conducted in China. Trials 2008;9:22. Zhang D, Freemantle N, Cheng KK. Are randomized trials conducted in China or India biased? A comparative empirical analysis. J Clin Epidemiol 2011;64:90–5.

Hartley LC, et al. J Epidemiol Community Health 2015;69:397–404. doi:10.1136/jech-2013-203646

403

Review 5

6 7

8

9

10

11

12

13 14 15

16 17

18

19

20

21

22

23

24

25 26

27

28

29

404

Pan Z, Trikalinos TA, Kavvoura FK, et al. Local literature bias in genetic epidemiology: an empirical evaluation of the Chinese literature. PLoS Med 2005;2:e334. Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ 1994;309:1286–91. Egger M, Dickersin K, Davey Smith G. Problems and limitations in conducting systematic reviews. In: Egger M, Smith GD, Altman DG, eds. Systematic reviews in health care: meta-analysis in context. 2nd edn. London, UK: BMJ Publishing Group, 2008. Lu L, Zeng M, Mao Y, et al. Oxymatrine in the treatment of chronic hepatitis B for one year: a multicentre random double-blind placebo-controlled trial. Chin J Hepatol 2004;12:597–600. Vickers A, Goyal N, Harland R, et al. Do certain countries produce only positive results? A Systematic review of controlled trials. Control Clin Trials 1998;19:159–66. European Medicines Agency. Ethnic factors in the acceptability of foreign clinical data. In: Note for guidance on ethnic factors in the acceptability of foreign clinical data. European Medicines Agency, 2006. http://www.ema.europa.eu/docs/enGB/ documentlibrary/Scientificguideline/2009/09/WC500002842.pdf (accessed 21 Apr 2009). Bowater RJ, Hartley LC, Lilford RJ. Are cardiovascular trial results systematically different between North America and Europe? A study based on intra-meta-analysis comparisons. Arch Cardiovasc Dis 2014:pii: S1875-2136(14)00061-8. Published Online First: Jul 2014. doi:10.1016/j.acvd.2014.03.005 United Nations Statistics Division. Standard country or area codes and geographical regions for statistical use. United Nations. 2008. http://unstats.un.org/unsd/methods/ m49/m49.htm (accessed 21 Apr 2009). Amsallem E, Kasparian C, Haddour G, et al. Phosphodiesterase III inhibitors for heart failure. Cochrane Database Syst Rev 2005;(1):CD002230. Aung K & Htay T. Vasopressin for cardiac arrest: a systematic review and meta-analysis. Arch Intern Med 2005;165:17–24. Lafuente-Lafuente C, Mouly S, Longas-Tejero MA, et al. Antiarrhythmics for maintaining sinus rhythm after cardioversion of atrial fibrillation. Cochrane Database Syst Rev 2007;(4):CD005049. Li J, Zhang Q, Zhang M, et al. Intravenous magnesium for acute myocardial infarction. Cochrane Database Syst Rev 2007;(2):CD002755. Abdel-Latif A, Bolli R, Zuba-Surma EK, et al. Granulocyte colony-stimulating factor therapy for cardiac repair after acute myocardial infarction: a systematic review and meta-analysis of randomized controlled trials. Am Heart J 2008;156:216–26. Vital FMR, Saconato H, Ladeira MT, et al. Non-invasive positive pressure ventilation (CPAP or bilevel NPPV) for cardiogenic pulmonary edema. Cochrane Database Syst Rev 2008;(3):CD005351. De Luca G, Suryapranata H, Stone GW, et al. Adjunctive mechanical devices to prevent distal embolization in patients undergoing mechanical revascularization for acute myocardial infarction: a meta-analysis of randomized trials. Am Heart J 2007;153:343–53. Ioannidis JPA, Katritsis DG. Percutaneous coronary intervention for late reperfusion after myocardial infarction in stable patients. Am Heart J 2007;154:1065–71. Jaffery Z, Kowalski M, Weaver WD, et al. A meta-analysis of randomized control trials comparing minimally invasive direct coronary bypass grafting versus percutaneous coronary intervention for stenosis of the proximal left anterior descending artery. Eur J Cardiothorac Surg 2007;31:691–7. Lipinski MJ, Biondi-Zoccai GGL, Abbate A, et al. Impact of intracoronary cell therapy on left ventricular function in the setting of acute myocardial infarction: a collaborative systematic review and meta-analysis of controlled clinical trials. JACC 2007;50:1761–7. Abbate A, Biondi-Zoccai GGL, Appleton DL, et al. Survival and cardiac remodeling benefits in patients undergoing late percutaneous coronary intervention of the infarct-related artery: evidence from a meta-analysis of randomized controlled trials. JACC 2008;51:956–4. Bavry AA, Kumbhani DJ, Bhatt DL. Role of adjunctive thrombectomy and embolic protection devices in acute myocardial infarction: a comprehensive meta-analysis of randomized trials. Eur Heart J 2008;29:2989–3001. Martin-Rendon E, Brunskill S, Doree C, et al. Stem cell treatment for acute myocardial infarction. Cochrane Database Syst Rev 2008;(4):CD006536. Patel AA, White CM, Gillespie EL, et al. Safety of amiodarone in the prevention of postoperative atrial fibrillation: a meta-analysis. Am J Health Syst Pharm 2006;63:829–37. Agostoni P, Biondi-Zoccai GGL, Gasparini GL, et al. Is bare-metal stenting superior to balloon angioplasty for small vessel coronary artery disease? Evidence from a meta-analysis of randomized trials. Eur Heart J 2005;26:881–9. Nordmann AJ, Bucher HC, Hengstler P, et al. Primary stenting versus primary balloon angioplasty for treating acute myocardial infarction. Cochrane Database Syst Rev 2005;(2):CD005313. Bravata DM, Gienger AL, McDonald KM, et al. Systematic review: the comparative effectiveness of percutaneous coronary interventions and coronary artery bypass graft surgery. Ann Intern Med 2007;147:703–16.

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49 50

51

52

53

54

De Luca G, Marino P. Adjunctive benefits from low-molecular-weight heparins as compared to unfractionated heparin among patients with ST-segment elevation myocardial infarction treated with thrombolysis. A meta-analysis of the randomized trials. Am Heart J 2007;154:1085. Schomig A, Dibra A, Windecker S, et al. A meta-analysis of 16 randomized trials of sirolimus-eluting stents versus paclitaxel-eluting stents in patients with coronary artery disease. JACC 2007;50:1373–80. Aziz O, Rao C, Panesar SS, et al. Meta-analysis of minimally invasive internal thoracic artery bypass versus percutaneous revascularisation for isolated lesions of the left anterior descending artery. BMJ 2007;334:617. Bleys J, Miller ER, Pastor-Barriuso R, et al. Vitamin-mineral supplementation and the progression of atherosclerosis: a meta-analysis of randomized controlled trials. Am J Clin Nutr 2006;84:880–7. Holdgate A, Foo A. Adenosine versus intravenous calcium channel antagonists for the treatment of supraventricular tachycardia in adults. Cochrane Database Syst Rev 2006;(4):CD005154. Labinaz M, Ho C, Banerjee S, et al. Meta-analysis of clinical efficacy and bleeding risk with intravenous glycoprotein IIb/IIIa antagonists for percutaneous coronary intervention. Can J Cardiol 2007;23:963–70. Whitlock RP, Chan S, Devereaux PJ, et al. Clinical benefit of steroid use in patients undergoing cardiopulmonary bypass: a meta-analysis of randomized trials. Eur Heart J 2008;29:2592–600. Bosch X, Loma-Osorio P, Marrugat J. Platelet glycoprotein IIb/IIIa blockers for percutaneous coronary intervention and the initial treatment of non-ST segment elevation acute coronary syndromes. Cochrane Database Syst Rev 2001;(4):CD002130. Healey JS, Baranchuk A, Crystal E, et al. Prevention of atrial fibrillation with angiotensin-converting enzyme inhibitors and angiotensin receptor blockers: a meta-analysis. JACC 2005;45:1832–9. Sedrakyan A, Wu AW, Parashar A, et al. Off-pump surgery is associated with reduced occurrence of stroke and other morbidity as compared with traditional coronary artery bypass grafting: a meta-analysis of systematically reviewed trials. Stroke 2006;37:2759–69. Ince H, Valgimigli M, Petzsch M, et al. Cardiovascular events and re-stenosis following administration of G-CSF in acute myocardial infarction: systematic review and meta-analysis. Heart 2008;94:610–16. Jibrini M, Molnar J, Arora R. Prevention of atrial fibrillation by way of abrogation of the renin-angiotensin system: a systematic review and meta-analysis. Am J Ther 2008;15:36–43. Moller CH, Penninga L, Wetterslev J, et al. Clinical outcomes in randomized trials of off- vs. on-pump coronary artery bypass surgery: systematic review with meta-analyses and trial sequential analyses. Eur Heart J 2008;29:2601–16. Zohlnhofer D, Dibra A, Koppara T, et al. Stem cell mobilization by granulocyte colony-stimulating factor for myocardial recovery after acute myocardial infarction: a meta-analysis. JACC 2008;51:1429–37. Gillespie EL, Coleman CI, Sander S, et al. Effect of prophylactic amiodarone on clinical and economic outcomes after cardiothoracic surgery: a meta-analysis. Ann Pharmaco ther 2005;39:1409–15. Bagshaw SM, Galbraith PD, Mitchell LB, et al. Prophylactic amiodarone for prevention of atrial fibrillation after cardiac surgery: a meta-analysis. Ann Thorac Surg 2006;82:1927–37. Dobre D, van Jaarsveld CHM, de Jongste MJL, et al. The effect of beta-blocker therapy on quality of life in heart failure patients: a systematic review and meta-analysis. Pharmaco Epidemiol Drug Saf 2007;16:152–9. Abdulla J, Barlera S, Latini R, et al. A systematic review: effect of angiotensin converting enzyme inhibition on left ventricular volumes and ejection fraction in patients with a myocardial infarction and in patients with left ventricular dysfunction. Eur J Heart Fail 2007;9:129–35. Hristov M, Heussen N, Schober A, et al. Intracoronary infusion of autologous bone marrow cells and left ventricular function after acute myocardial infarction: a meta-analysis. J Cell Mol Med 2006;10:727–33. Abdel-Latif A, Bolli R, Tleyjeh IM, et al. Adult bone marrow-derived cells for cardiac repair: a systematic review and meta-analysis. Arch Intern Med 2007;167:989–7. Kang S, Yang Y, Li CJ, et al. Effectiveness and tolerability of administration of granulocyte colony-stimulating factor on left ventricular function in patients with myocardial infarction: a meta-analysis of randomized controlled trials. Clin Ther 2007;29:2406–18. Appleton DL, Abbate A, Biondi-Zoccai GGL. Late percutaneous coronary intervention for the totally occluded infarct-related artery: a meta-analysis of the effects on cardiac function and remodeling. Catheter Cardiovasc Interv 2008;71:772–81. Kang S, Yang Y, Li C, et al. Effects of intracoronary autologous bone marrow cells on left ventricular function in acute myocardial infarction: a systematic review and meta-analysis for randomized controlled trials. Coron Artery Dis 2008;19:327–35. Nolan R, Jong P, Barry-Bianchi S, et al. Effects of drug, biobehavioral and exercise therapies on heart rate variability in coronary artery disease: a systematic review. Eur J Cardiovasc Prev Rehabil 2008;15:386–96. O’Shea JC, Califf RM. International differences in treatment effects in cardiovascular clinical trials. Am Heart J 2001;141:875–80.

Hartley LC, et al. J Epidemiol Community Health 2015;69:397–404. doi:10.1136/jech-2013-203646

Copyright of Journal of Epidemiology & Community Health is the property of BMJ Publishing Group and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.