A systematic appraisal of field synopses in genetic epidemiology: a HuGE review.

American Journal of Epidemiology Advance Access published December 11, 2014 American Journal of Epidemiology © The Author 2014. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: [email protected].

DOI: 10.1093/aje/kwu249

Human Genome Epidemiology (HuGE) Review A Systematic Appraisal of Field Synopses in Genetic Epidemiology: A HuGE Review

* Correspondence to Dr. Evangelos Evangelou, Department of Hygiene and Epidemiology, University of Ioannina Medical School, Ioannina 45110, Greece (e-mail: [email protected]).

Initially submitted March 24, 2014; accepted for publication August 20, 2014.

Evidence from genetic association studies is accumulating rapidly. Field synopses have recently arisen as an unbiased way of systematically synthesizing this evidence. We performed a systematic review and appraisal of published field synopses in genetic epidemiology and assessed their main findings and methodological characteristics. We identified 61 eligible field synopses, published between January 1, 2007, and October 31, 2013, on 52 outcomes reporting 734 significant associations at the P < 0.05 level. The median odds ratio for these associations was 1.25 (interquartile range, 1.15–1.43). Egger’s test was the most common method (n = 30 synopses) of assessing publication bias. Only 12 synopses (20%) used the Venice criteria to evaluate the epidemiologic credibility of their findings (n = 449 variants). Eleven synopses (18%) were accompanied by an online database that has been regularly updated. These synopses received more citations (P = 0.01) and needed a larger research team (P = 0.02) than synopses without an online database. Overall, field synopses are becoming a valuable tool for the identification of common genetic variants, especially when researchers follow relevant methodological guidelines. Our work provides a summary of the current status of the field synopses published to date and may help interested readers efficiently identify the online resources containing the relevant genetic evidence. epidemiologic methods; field synopsis; genetic associations; genetic epidemiology; genome, human; meta-analysis

Abbreviations: GWAS, genome-wide association studies; HuGE, Human Genome Epidemiology; IQR, interquartile range.

Editor’s note: This article also appears on the website of the Human Genome Epidemiology Network (http://www.cdc. gov/genomics/hugenet/default.htm).

which has led to the adoption of stringent significance thresholds (5). A scientific enterprise with so many false-positive findings can lose credibility or impede progress in research and translation (6). Knowledge synthesis and integration of findings from candidate gene studies and GWAS came to be regarded as essential, especially after the recommendations of the Human Genome Epidemiology Network (HuGENet) were published (7, 8). Although similar approaches may have existed before, they became more apparent and standardized after 2007, when the first field synopses were published (9–12). Field synopses constitute a snapshot of the current state of knowledge on genetic associations in a particular field of research defined by a disease or trait (7) and provide a summary of the cumulative evidence on the genetic etiology of complex diseases and normal traits. Most importantly, one of their main aspects, according to the proposed framework, is the

The number of genetic association studies for complex traits has increased exponentially in the last several years (1, 2). As of November 2013, the Human Genome Epidemiology (HuGE) Navigator database (3) had cataloged more than 85,000 articles pertaining to both candidate gene studies and genome-wide association studies (GWAS). Despite their large number, the majority of the proposed candidate gene associations were not replicated in subsequent studies as a result of limited statistical power, as well as selective reporting and other biases (4). Moreover, the low prior probabilities for large numbers of comparisons among all candidate gene studies and in GWAS create the risk of false-positive findings, 1

Downloaded from http://aje.oxfordjournals.org/ at University of California, San Francisco on December 12, 2014

Lazaros Belbasis, Orestis A. Panagiotou, Vasilios Dosis, and Evangelos Evangelou*

2 Belbasis et al.

evaluation of the epidemiologic credibility of their findings using various approaches. To date, several field synopses on various phenotypes have been published (10, 13, 14). However, to the best of our knowledge, no systematic overview of these studies has been performed. We conducted a systematic appraisal of the published field synopses, and here we present the results, as well as the reported findings and statistical and methodological approaches used in the eligible articles. METHODS

We searched PubMed (US National Library of Medicine, Bethesda, Maryland) for field synopses pertaining to genetic associations with a specific disease or trait that had been published between January 1, 2007, and October 31, 2013. We used the algorithm (“systematic review” OR meta-analysis OR “field synopsis” OR “umbrella review”) AND (gene OR genetic* OR polymorphism*). Following the HuGENet recommendations (7), we considered 2007 to be the year in which the first synopsis was published (10). Additionally, to ensure that we included all potentially eligible synopses, we complemented our PubMed search by searching HuGE Navigator (http://www.hugenavigator.net/), a regularly updated database focusing exclusively on genetic association studies and meta-analyses thereof (3), using the term “field synopsis.” Eligibility criteria

Eligible articles were considered on the basis of the definition adopted by Khoury et al. (7), which describes a field synopsis as a regularly updated snapshot of the current state of knowledge about genetic associations and common diseases. Ideally, a field synopsis should be freely available, should use online databases, should use objective and transparent criteria for grading the credibility of cumulative evidence, and should summarize and update the information in peer-reviewed articles on a regular basis (7). We considered as study units all field synopses that had provided a systematic review of genetic associations between a disease or normal trait and variants in any gene and had additionally synthesized the available data using meta-analytical techniques. We excluded reports that did not systematically review the literature; that a priori restricted their search to specific genes and/or polymorphisms; that did not pertain to DNA polymorphisms (e.g., RNA sequence); that focused on pharmacogenetics; that were limited to the presentation of technical characteristics of published databases of genetic associations; that focused exclusively on specific gene families or on multiple variants of a specific gene; or that appraised already published meta-analyses. Two investigators (L.B., V.D.) independently screened and identified eligible synopses. Disagreements were resolved by consensus after discussions with the other 2 investigators. Data extraction

The data were recorded from the main reports and from the supplementary material when needed. We extracted

Data analysis

We calculated the median numbers (and their interquartile ranges) of nominally significant variants, eligible studies, data sets, authors, and citations. We also calculated the median effect sizes (and their corresponding interquartile ranges) of all significant associations for binary outcomes. We accepted the threshold used by the field synopses’ authors to claim statistical significance. We converted the effect size so as to


Search strategy

information on all nonoverlapping nominally significant associations reported by the authors of the eligible papers in main analyses and subgroup analyses carried out in various ethnic groups. When more than 1 genetic model had been used for the analysis, we captured the significant associations from the most powerful model, using the following order: per-allele, additive, dominant, recessive, and other models. When a significant association was available in main and subgroup analyses, we extracted the data corresponding to the main analyses. If a significant association was present in multiple subgroup analyses but not in the main analysis, we included the association from the analysis with the greatest statistical power (largest sample size). When an association was reported as significant under both fixed-effects and random-effects meta-analysis models, we extracted the effect sizes corresponding to the random-effects model for associations with high heterogeneity and to the fixed-effects model for associations with low heterogeneity, as specified by the authors of the synopses. From each paper we extracted information on the first author, journal and year of publication, outcome, number of authors, main and subgroup analyses performed, number of eligible studies, total number of variants identified, number of polymorphisms claimed to be statistically significant, effect estimates and measures of uncertainty thereof (standard errors or confidence intervals), total sample size, and total number of data sets per significant association used for metaanalysis. We also recorded the criteria for performing metaanalysis, as well as the methods used (i.e., fixed-effects, random-effects, other). Additionally, we recorded the methods used for the assessment of between-study heterogeneity, small-study effects, publication bias, and/or other biases; the epidemiologic credibility of the identified associations (15); and adjustments for multiple comparisons (16). We also recorded whether evaluation of Hardy-Weinberg equilibrium had been performed; the threshold of statistical significance used by the authors to deem an identified association significant; the sensitivity analyses performed; and whether any GWAS had been included. We assumed that the significance threshold was set at 0.05 if the authors did not specify otherwise. We accessed the online “Catalog of Published GWAS” (http://www.genome.gov/gwastudies/) (17) to record whether there had been any published GWAS pertaining to the same outcomes as those in field synopses by the ending date of the synopses’ literature search and to determine whether the synopses’ authors had included all available GWAS data. Two investigators (L.B., V.D.) independently extracted the data. Consensus was reached in cases of disagreement among the authors.

Synopsis of Field Synopses 3

9,221 Articles Reviewed by Title Screening Reasons for Exclusion (n = 9,064) Meta-analyses of single variants or gene families (n = 3,357) Articles not evaluating genetic associations (n = 2,874) Single candidate gene studies or GWAS (n = 1,692) Nonresearch articles (n = 538) Articles not associated with humans (n = 288) Pharmacogenetic studies (n = 254) Articles without quantitative synthesis (n = 61)

Reasons for Exclusion (n = 87) Articles without quantitative synthesis (n = 24) Meta-analyses of single variants or gene families (n = 19) Nonresearch articles (n = 19) Articles not associated with humans (n = 12) Articles not evaluating genetic associations (n = 10) Articles appraising already published meta-analyses (n = 2) Pharmacogenetic studies (n = 1)

70 Articles Reviewed by Full-Text Screening

Reasons for Exclusion (n = 9) Meta-analyses of single variants or gene families (n = 5) Articles without quantitative synthesis (n = 3) Articles appraising already published meta-analyses (n = 1)

61 Eligible Field Synopses Published From January 1, 2007, to October 31, 2013

Figure 1. PubMed literature search used to locate and select field synopses of genetic association studies published between January 1, 2007, and October 31, 2013. GWAS, genome-wide association studies.

always reflect the risk allele for disease-related traits. We also calculated descriptive statistics for any sensitivity and subgroup analyses performed, and we summarized the methods and criteria used for meta-analysis and for assessment of publication bias/small-study effects. Finally, we estimated the number and percentage of associations that survived after the application of a more stringent statistical significance threshold. Whenever the published reports mentioned an accompanying online database, we accessed the website, verified the database’s existence, and assessed its frequency of data updates

and format. We checked whether the databases had been updated at least once in the subsequent 24 months after their creation and recorded the date of their last update. On December 28, 2013, we accessed ISI Web of Science (Thomson Reuters, New York, New York) to record the latest impact factor of the journals in which the field synopses had been published and the number of citations each field synopsis had received since its publication. We assessed whether the existence of a database affected the number of citations received and the target journal (based on the journal impact factor), using the Wilcoxon rank-sum test.


157 Articles Reviewed by Abstract Screening

4 Belbasis et al.

Table 1. Characteristics of Field Synopses of Genetic Association Studies Published Between January 1, 2007, and October 31, 2013 First Author, Year (Reference No.)

Outcome

No. of Eligible Articles

No. of Nominally Significant Associations

GWAS Used?

No. of Citations

Online Database (Website)

Abhary, 2009 (59)

Diabetic retinopathy

82

2

No

37

Alg, 2013 (28)

Intracranial aneurysms

66

19

Yes

0

No

Allen, 2008 (13)

Schizophrenia

1,179

24

No

464

Ariyaratnam, 2007 (9)

Ischemic stroke

60

5

No

60

Bertram, 2007 (10)

Alzheimer’s disease

789

24

No

653

Brouwer, 2010 (60)

Meningococcal disease

28

3

No

22

No

Brouwer, 2009 (21)

Pneumonococcal disease, meningococcal disease

76

2

No

72

No

Buurma, 2013 (61)

Preeclampsia

Calati, 2013 (39)

Borderline personality disorder

Castaldi, 2010 (62)

No SZGene (http://www. szgene.org/) No AlzGene (http://www. alzgene.org/)

7

No

1

No

28

0

No

0

No

COPD

108

4

No

39

COPDGene (http:// copdgene.org)

Chatzinasiou, 2011 (27)

Cutaneous melanoma

145

47

Yes

22

MelGene (http://www. melgene.org/)

Chen, 2012 (63)

Polypoidal choroidal vasculopathy

33

8

No

5

Debette, 2009 (64)

Cervical artery dissection

Dolan, 2010 (65)

Preterm birth

Dwyer, 2013 (66)

Vascular cognitive impairment

Gizer, 2009 (67)

Attention-deficit/hyperactivity disorder

No

7

1

No

25

No

61

5

No

20

PTBGene (http://ric. einstein.yu.edu/ ptbgene/)

104

3

No

0

No

8

No

231

No

NR

Gohil, 2009 (68)

Venous thromboembolism

173

10

No

56

No

Hamzi, 2011 (69)

Ischemic stroke

32

2

No

3

No

He, 2013 (40)

Henoch-Schonlein purpura, Henoch-Schonlein purpura nephritis

45

6

No

0

No

Jiang, 2012 (29)

Rheumatoid arthritis

216

43

Yes

2

DRAP (http://210.46.85. 180/DRAP/)

Jin, 2011 (70)

Hepatocellular carcinoma

26

6

No

8

No

Kitsios, 2007 (11)

Ischaemic heart failure

22

1

No

29

No

Lee, 2012 (71)

Fibromyalgia

18

1

No

14

No No

Li, 2010 (72)

Asthma

119

7

No

7

Lill, 2012 (14)

Parkinson’s disease

828

212

Yes

55

PDGene (http://www. pdgene.org/)

Loh, 2009 (73)

Gastric cancer

203

19

No

39

No

López-León, 2008 (74)

Major depressive disorder

183

5

No

122

No

Ma, 2014a (25)

Colorectal cancer

945

62

Yes

NA

No

Marjot, 2011 (75)

Cerebral venous thrombosis

26

2

No

10

No

Maxwell, 2011 (42)

Brain microbleeds

10

1

No

8

No

Mazaki, 2011 (76)

Pancreatic cancer

32

2

No

5

No Table continues

For this analysis, we excluded journals that are not indexed in ISI Web of Science. We examined whether the association between the number of citations and the existence of a database was altered when we adjusted for the number of months since publication, performing a generalized linear model analysis with the number of citations as the

dependent variable and the number of months since publication and the existence of a database as the independent variables. For recently published field synopses not yet indexed in ISI Web of Science, we performed a main analysis assuming that the number of citations these field synopses had received and the number of months since publication


542


Table 1. Continued First Author, Year (Reference No.)

McColgan, 2010 (77)

Outcome

Intracranial aneurysms, subarachnoid hemorrhage



30

3

GWAS Used?

No

No. of Citations

13

Online Database (Website)

No

Mooyaart, 2011 (78)

Diabetic nephropathy

132

24

No

34

No

Paternoster, 2010 (18)

Carotid intima-media thickness

95

1

No

15

No


White matter hyperintensities

45

1

No

25

No

Peck, 2008 (79)

Hemorrhagic stroke, subarachnoid hemorrhage, intracranial aneurysms

85

5

No

27

No

Metabolic syndrome Cerebral amyloid angiopathy

Rao, 2009 (81)

Ischemic stroke

Romero-Gomez, 2011 (82) Hepatitis C

88

5

No

21

No

136

1

No

1

No

NR

0

No

18

No

NR

1

No

17

No

Sava, 2014a (48)

Chronic lymphocytic leukemia

36

4

No

NA

No

Schild, 2013 (83)

Suicidal behavior

107

0

No

0

No

Seifuddin, 2012 (24)

Bipolar disorder

487

4

Yes

6

No

Smolonska, 2009 (84)

COPD

69

8

No

54

No

Srivastava, 2011 (85)

Gallbladder cancer

44

1

No

10

No

Staines-Urias, 2012 (23)

Preeclampsia

192

5

No

4

No

Taylor, 2013 (22)

Obsessive-compulsive disorder

113

3

No

1

No

Theodoratou, 2012 (26)

Colorectal cancer

635

42

Yes

11

Tüttelmann, 2007 (12)

Male infertility

7

2

No

94

No

Varvarigou, 2011 (86)

Obstructive sleep apnea

36

1

No

10

No

Vassos, 2014a (49)

Violence and aggression

185

0

No

NA

No

Wu, 2011 (87)

Cerebral palsy

11

1

No

8

No

Xin, 2009 (41)

Adult early-onset ischemic stroke

51

3

No

19

No

Yadav, 2013 (88)

Ischemic stroke

42

4

No

2

No

Zaffanello, 2011 (43)

Renal scar formation after urinary tract infections

18

2

No

6

No No

CRCgene (http://www.cphs. mvm.ed.ac.uk/projects/ CRCgene/)

Zdoukopoulos, 2008 (89)

Placental abruption

22

4

No

19

Zhang, 2013 (90)

Gestational diabetes mellitus

29

9

No

0

No

Zhang, 2011 (91)

Breast cancer

1,059

51

No

49

No

Zintzaras, 2010 (46)

Osteoarthritis

61

4

No

7


Chronic lymphocytic leukemia

81

0

No

11

CUMAGAS-CLL (http:// biomath.med.uth.gr/)


Peripheral arterial disease

41

4

No

21

CUMAGAS-PAD (http:// biomath.med.uth.gr/)

CUMAGAS-OSTEO (http://biomath.med.uth.gr/)

Abbreviations: COPD, chronic obstructive pulmonary disease; CUMAGAS, Cumulative Meta-Analysis of Genetic Association Studies; DRAP, Database of Rheumatoid Arthritis Related Polymorphisms; GWAS, genome-wide association studies; NA, not available; NR, not reported. a Article was published online (electronic publication) before November 2013; year refers to the year of print publication.

were both equal to zero and a sensitivity analysis excluding these synopses. Furthermore, we assessed whether the number of authors was higher in field synopses that were accompanied by an online database, using the Wilcoxon rank-sum test. We also examined the association between the existence of a database and the receipt of funding for

conducting the synopsis, using Fisher’s exact test. Additionally, to explore any potential factors influencing the existence of an online database, we performed a logistic regression analysis with the existence of a database as the dependent variable and the number of authors and the existence of funding as independent variables.


Povel, 2011 (80) Rannikmäe, 2013 (20)

6 Belbasis et al.

Statistical analyses were performed in STATA, version 12.0 (StataCorp LP, College Station, Texas). All P values presented are 2-tailed. RESULTS Eligible field synopses

Characteristics of published field synopses

Of the 61 eligible synopses, 58 pertained to binary outcomes and used the odds ratio as the effect-size metric; 1 synopsis (18) pertained to a continuous outcome and used standardized mean differences; and 2 synopses (19, 20) included both binary and continuous outcomes. Of the 58 synopses with binary outcomes, 55 used an α level of 0.05 and estimated 95% confidence intervals, 2 synopses (21, 22) used an α level of 0.01 and estimated 99% confidence intervals, and 1 synopsis (23) used an α level of 0.05 and estimated 99% confidence intervals. The median number of eligible studies across the included synopses was 68 (interquartile range (IQR), 32–143). In total, 734 associations were nominally significant based on the thresholds used by the authors of the synopses. Eighty-one associations, from 5 field synopses, were subjected to a more stringent threshold set by their authors using the Bonferroni correction, and 68 (84%) of them remained significant after this correction. The median number of nominally significant associations in each synopsis was 4 (IQR, 1–8), whereas the median number of data sets used for the meta-analyses of the 734 significant associations was 5 (IQR, 3–11). The median odds ratio for all nominally significant variants was 1.25 (IQR, 1.15– 1.43). GWAS were available at the time of completion of the literature search for 22 synopses, and 7 of them included these GWAS data in their analyses. These synopses pertained to bipolar disorder (24), colorectal cancer (25, 26), cutaneous melanoma (27), intracranial aneurysm (28), Parkinson’s disease (14), and rheumatoid arthritis (29). The decision on whether to conduct a meta-analysis (Table 2) was based on at least 1 of the following criteria: 1) the number of data sets available per variant (n = 30); 2) the number of subjects per polymorphism (n = 4); and 3) the minor allele frequency in healthy controls (n = 1). Three synopses used only fixed-effects models for synthesis of the available data, 24 synopses used only random-effects models, 33 synopses used both random-effects and fixed-effects models, and the authors of 1 synopsis did not report the meta-

Reporting on the credibility of the findings and adjustment for multiple comparisons

Twelve field synopses used the Venice criteria (15) to assess the epidemiologic credibility of the 449 variants included in the respective analyses, whereas 1 of them also used the Bayes factor (36) (212 variants). Only 155 of the 449 associations were found to have evidence of strong epidemiologic credibility (overall grade A) by application of the Venice criteria (15). Eight field synopses used at least 1 method to adjust for multiple comparisons. These methods included the Bonferroni correction (n = 5 synopses; 81 assessed variants), the falsepositive report probability (37) (n = 3 synopses; 64 assessed variants), or the Bayesian false-discovery probability (38) (n = 1 synopses; 42 assessed variants). Additionally, 3 field synopses evaluated the included studies using some quality score, including the Newcastle-Ottawa scale (n = 2) (39, 40) or other qualitative criteria defined by the research team (n = 1) (41). Moreover, 2 field synopses used the STrengthening the REporting of Genetic Association Studies (STREGA) statement, which aims to improve reporting in genetic association studies (42–44). More details are presented in Table 2.


Our literature search yielded 9,221 studies, of which 61 studies met the inclusion criteria, as shown in Figure 1. The eligible synopses were published between January 1, 2007, and October 31, 2013. These synopses evaluated genetic risk factors for 52 outcomes (Table 1). For 44 outcomes, there was only 1 field synopsis; for 1 outcome (ischemic stroke), there were 5 field synopses; for another outcome (intracranial aneurysms), there were 3 synopses; and for each of the remaining 6 outcomes (chronic lymphocytic leukemia, chronic obstructive pulmonary disease, colorectal cancer, meningococcal disease, preeclampsia, and subarachnoid hemorrhage), there were 2 synopses.

analysis method applied (Table 2). The most common criterion for performing a meta-analysis was the presence of at least 3 independent data sets per variant (n = 22), while 30 synopses did not apply any criteria for conducting a metaanalysis. Fifty-six synopses assessed heterogeneity, using Cochran’s Q statistic (n = 6), the I 2 statistic (n = 7), both Q and I 2 (n = 41), or Woolf’s test (n = 2). Thirty-eight field synopses evaluated Hardy-Weinberg equilibrium using P = 0.05 as the threshold (Table 2). To explore potential sources of heterogeneity, the authors of 34 field synopses performed a total of 47 subgroup analyses, whereas the authors of 29 synopses performed a total of 45 sensitivity analyses. Specifically, the subgroup analyses were based on ethnicity (n = 30 synopses), disease characteristics (n = 9), and population characteristics (n = 8). Similarly, the sensitivity analyses were based on the exclusion of studies violating Hardy-Weinberg equilibrium in controls (n = 14), the exclusion of the first published study (n = 9), sample size (n = 7), study characteristics (n = 5), the exclusion of the first positive study (n = 3), or an iterative exclusion of studies (n = 7) (Table 2). Five more field synopses excluded a priori all studies that were characterized by violation of Hardy-Weinberg equilibrium in the control group, as it was set by their inclusion criteria for the meta-analysis. Finally, the authors of 6 synopses performed a meta-regression to further explore potential sources of heterogeneity. A total of 47 synopses used 1 or more methods to evaluate publication bias. Evaluation with graphical tools included funnel plots (n = 25 synopses) and Galbraith plots (n = 1). The authors of 43 synopses performed at least 1 statistical test or applied at least 1 method for the assessment of publication bias and/or small-study effects, including Egger’s test (n = 30) (30), Begg’s test (n = 11) (31), Harbord’s test (n = 11) (32), the test for an excess of statistical significance (n = 5) (33), the fail-safe N method (n = 2) (34), or Duval and Tweedie’s trim-and-fill analysis (n = 1) (35) (Table 2).

Table 2. Methodological Details on Field Synopses of Genetic Association Studies Published Between January 1, 2007, and October 31, 2013

First Author, Year (Reference No.)

Meta-Analysis Model

Criteria for Conducting Meta-Analysis

Subgroup Analyses

Sensitivity Analyses

Evaluation of Heterogeneity

Assessment of HWE?

Evaluation of Credibility and Adjustment for Multiple Comparisons

Assessment of Publication Bias/ Small-Study Effects

Abhary, 2009 (59)

RE

≥5 data sets per variant

Ethnicity, disease characteristics

None

Qa

No

None

Funnel plots, Egger’s test

Alg, 2013 (28)

RE, FE

None

Ethnicity

Iterative exclusion of studies

Q, I 2

Yes

Bonferroni correction


Allen, 2008 (13)

RE, FE

Minor allele frequency ≥0.01 in healthy controls, ≥4 data sets per variant

Ethnicity

EFPS, HWEvb

Q, I 2

Yes

Venice criteria

Funnel plots, Harbord’s test, ESS test


RE

≥3 different data sets per variant

Ethnicity

Sample size

Q, I 2

No

None


Bertram, 2007 (10)

RE


Ethnicity

EFPS, HWEv

Q, I 2

Yes

None

Begg’s test, Egger’s test

Brouwer, 2010 (60)

RE, FE

None

Disease characteristics

None

Q

No

None

None

RE, FE

None

None

None

Q

Yes

None

None

Buurma, 2013 (61)

RE

None

None

Sample size

I2

No

None

Funnel plots, Egger’s test, Begg’s test

Calati, 2013 (39)

RE, FE


None

None

Q, I 2

Yes

None


Castaldi, 2010 (62)

RE


Ethnicity, population EFPS, HWEv, study characteristics characteristics

Q, I 2

Yes

None

Egger’s test

Chatzinasiou, 2011 (27)

RE


Ethnicity

EFPS, HWEv

Q, I 2

Yes

Venice criteria

Harbord’s test, ESS test

Chen, 2012 (63)

RE, FE

None

None

None

Q

Yes

None

Egger’s test

Debette, 2009 (64)

FE

None

None

None

Q, I 2

No

None

None

Dolan, 2010 (65)

RE, FE


Ethnicity

EFPS, HWEv

Q, I 2

Yes

Venice criteria

None

Dwyer, 2013 (66)

RE, FE


Ethnicity

None

None

Yes

None

Egger’s test

Gizer, 2009 (67)

RE, FE

None

None

None

Q, I 2

No

None

Funnel plots

Gohil, 2009 (68)

RE

None

Ethnicity

None

Q, I 2

No

None


Hamzi, 2011 (69)

FE

None

None

None

None

No

None

None

He, 2013 (40)

RE, FE

None

None

None

None

No

None

Egger’s test

Jiang, 2012 (29)

RE, FE

None

Ethnicity

None

Q, I 2

No

None


Jin, 2011 (70)

RE, FE


None

None

Q, I

Yes

Venice criteria, FPRP

Egger’s test, Begg’s test Table continues


Brouwer, 2009 (21)



Kitsios, 2007 (11)

Meta-Analysis Model

RE, FE


None

Subgroup Analyses

None


HWEv


Assessment of HWE?



Q, I 2

Yes

None

None

2

Yes

None


Lee, 2012 (71)

RE, FE

None

None

None

Q, I

Li, 2010 (72)

RE, FE


Population characteristics

None

Q, I 2

Yes

None


Lill, 2012 (14)

RE


Ethnicity

EFPS, HWEv

I2

Yes

Bayes factor, Venice criteria

Harbord’s test

Loh, 2009 (73)

RE, FE

None

Ethnicity

None

Q, I 2

No

None

Funnel plots, Begg’s test

López-León, 2008 (74)

RE, FE


None

EFPS

I2

Yes

None

Funnel plots

Ma, 2014 (25)c

RE


Ethnicity

EFPS, exclusion of the first positive study, HWEv

Q, I 2

Yes

Venice criteria, FPRP, Bonferroni correction


Marjot, 2011 (75)

RE

None

None


Q, I 2

No

None


Maxwell, 2011 (42)

FE

>100 subjects per variant

Ethnicity, population Sample size characteristics

Q, I 2

Yes

None

Fail-safe N method

Mazaki, 2011 (76)

RE, FE

None

Ethnicity, disease characteristics, population characteristics

None

Q, I 2

Yes

None

None

McColgan, 2010 (77)

RE, FE

None

None


Q

No

None


Mooyaart, 2011 (78)

RE

None


None

I2

No

None



RE

≥3 different data sets per gene, >5,000 subjects per gene

Ethnicity, population Sample size characteristics

Q, I 2

Yes

None

Funnel plots


RE, FE

>2,000 subjects per None gene

None

Q, I 2

Yes

None

None

Peck, 2008 (79)

RE, FE


None

Q, I 2

No

None


Povel, 2011 (80)

RE

>4,000 subjects per Ethnicity variant, ≥3 different data sets per variant

None

I2

No

None


Rannikmäe, 2013 (20)

RE, FE

None

Study characteristics

Q, I 2

No

None

Funnel plots, fail-safe N method

Disease characteristics


Table continues

8 Belbasis et al.

Table 2. Continued


Table 2. Continued


Meta-Analysis Model


Subgroup Analyses



Assessment of HWE?



Rao, 2009 (81)

RE, FE


None


Q

No

None

Funnel plots

Romero-Gomez, 2011 (82)

RE, FE

None

None

None

None

No

None

None

Sava, 2014 (48)c

RE, FE

None

None

None

Q, I 2

Yes

FPRP

Egger’s test

Schild, 2013 (83)

RE

None

Ethnicity


Q, I 2

No

None

Egger’s test, trim-and-fill analysis

Seifuddin, 2012 (24)

RE, FE


None

None

Woolf’s test

Yes


None


RE, FE


Ethnicity

None

Woolf’s test

Yes

None

None

Srivastava, 2011 (85)

RE, FE


None


Q, I 2

Yes

Venice criteria

Egger’s test, Begg’s test


RE


Ethnicity

Exclusion of the first positive study, HWEv, sample size, study characteristics

Q, I 2

Yes

Venice criteria

Funnel plots, Galbraith plots, Harbord’s test, ESS test

Taylor, 2013 (22)

RE


None

None

I2

Yes


Egger’s test

Theodoratou, 2012 (26)

RE, FE


Ethnicity

None

Q, I 2

No

Venice criteria, BFDP

Harbord’s test

NR

None

None

None

None

Yes

None

None

RE


Population characteristics

None

Q, I 2

Yes

None

Harbord’s test, Egger’s test

Vassos, 2014 (49)c

RE



Study characteristics

Q, I 2

No

Venice criteria


Wu, 2011 (87)

RE, FE

None

None

None

Q, I 2

Yes



Xin, 2009 (41)

RE, FE


None

HWEv

Q, I 2

Yes

None


Yadav, 2013 (88)

RE

None

None

None

I2

No

None


Zaffanello, 2011 (43)

RE, FE

None


Study characteristics, sample size

Q, I 2

Yes

Venice criteria

None

Zdoukopoulos, 2008 (89)

RE, FE

None

None

None

Q, I 2

Yes

None

None

Zhang, 2013 (90)

RE

None

None


Q, I 2

Yes

None

Egger’s test Table continues


Tüttelmann, 2007 (12) Varvarigou, 2011 (86)


Abbreviations: BFDP, Bayesian false-discovery probability; EFPS, exclusion of first published study; ESS, excess of statistical significance; FE, fixed effects; FPRP, false-positive report probability; HWE, Hardy-Weinberg equilibrium; HWEv, violation of Hardy-Weinberg equilibrium; NR, not reported; RE, random effects. a Cochran’s Q test. b Exclusion of studies violating Hardy-Weinberg equilibrium. c Article was published online (electronic publication) before November 2013; year refers to the year of print publication.

Harbord’s test Yes Ethnicity Zintzaras, 2009 (47)

RE

None

HWEv

Q, I 2

None

Harbord’s test Yes Ethnicity Zintzaras, 2009 (45)

RE


HWEv

Q, I 2

None

Harbord’s test Yes Ethnicity, disease characteristics Zintzaras, 2010 (46)

RE


HWEv

Q, I 2

None

Funnel plots, Begg’s test, Harbord’s test, ESS test Venice criteria Yes Q, I 2 EFPS, HWEv, exclusion of first positive study, sample size Ethnicity ≥3 different data sets per variant RE Zhang, 2011 (91)

Subgroup Analyses


Evaluation of Heterogeneity Criteria for Conducting Meta-Analysis Meta-Analysis Model First Author, Year (Reference No.)

Table 2. Continued

Comparison of field synopses with overlapping outcomes

As we noted above, there were 18 synopses on 8 overlapping outcomes. With the exception of 2 synopses that evaluated genetic associations with ischemic stroke in South Asians and non-Europeans, all of the synopses considered participants of all ethnicities. The synopses on ischemic stroke, subarachnoid hemorrhage, meningococcal disease, and chronic obstructive pulmonary disease did not use any method to evaluate the credibility of the association or to adjust for multiple comparisons. Such tools were used in both studies on colorectal cancer, in 1 synopsis on chronic lymphocytic leukemia, and in 1 synopsis on preeclampsia. Assessment of publication bias or small-study effects was performed in most of the synopses (n = 14). In 5 of the 8 overlapping outcomes, not accounting for the 2 synopses limited to specific ethnicities, the last published synopsis did not have more eligible articles, suggesting that different search strategies and inclusion criteria may have been used. A detailed description of the synopses on overlapping outcomes is given in Appendix Table 1. Databases

Only 11 synopses (18%) were accompanied by online databases cataloging the findings and the studies included in the corresponding published field synopsis. In 4 databases (29, 45–47), the authors urged interested scientists to include their data in the database by granting access through a loggingin system. The median number of authors in the field synopses supported by an online database was 12, while in field synopses without an online database the median number of authors was 4 (P = 0.02, Wilcoxon rank-sum test). Funding was associated with the existence of an online database (P = 0.046, Fisher’s exact test). However, when the number of authors and the existence of funding were considered in a logistic regression model, the existence of a database was affected only by the number of authors (P = 0.007) and not by the existence of funding (P = 0.238). All tracked databases are available online and can be accessed through their initial Web address as provided by the authors. Nine of the 11 databases have been online at least 24 months, and all of them were updated at least once in the 24 months after their creation, whereas 2 databases (14, 29) have been online for less than 24 months and therefore have not been evaluated with regard to an update. Citation analysis

Three field synopses (25, 48, 49) were published recently and have not yet been indexed in ISI Web of Science. The remaining 58 synopses have received 2,512 citations in total, with a median of 13 citations (IQR, 4–29). Field synopses supported by an online database received a median of 21 citations, while those without a database received a median of 10 citations (P = 0.073, Wilcoxon ranksum test). However, this difference was significant when we adjusted for number of months since publication (P = 0.007) using generalized linear regression. There were no significant differences in the median journal impact factor between


Assessment of HWE?



10 Belbasis et al.


synopses with (7.692) and without (5.419) a database (P = 0.157). The results were similar in a sensitivity analysis where we excluded nonindexed field synopses. DISCUSSION

ACKNOWLEDGMENTS

Author affiliations: Department of Hygiene and Epidemiology, University of Ioannina Medical School, Ioannina, Greece (Lazaros Belbasis, Vasilios Dosis, Evangelos Evangelou); Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, Maryland (Orestis A. Panagiotou); and Department of Epidemiology and Biostatistics, Imperial College London, London, United Kingdom (Evangelos Evangelou).


To our knowledge, this is the first systematic appraisal of field synopses in genetic epidemiology summarizing the current status of the synopses published to date, along with their related online resources whenever available. We identified 61 published field synopses covering a wide spectrum of outcomes and more than 700 variants that were deemed nominally significant. However, only about one-third of those variants showed strong evidence of credibility for the respective associations whenever such criteria were applied. The evidence of genetic associations was mostly assessed through the Venice criteria, the Bayes factor, or other statisticsbased approaches to adjustment for multiple comparisons, such as the Bonferroni correction, the Bayesian false-discovery probability, and the false-positive report probability. In most cases, the credibility of the findings was assessed and interpreted efficiently, providing information on the variants that had weak, moderate, or strong epidemiologic credibility. However, more than half of the published field synopses failed to either provide any assessment for their associations or apply a stringent P-value threshold, and therefore these findings should be interpreted with caution to avoid falsepositive inferences. At the same time, given that the stringent genome-wide significance thresholds applied in GWAS may increase the number of false-negative findings, it is important for researchers to consider the balance between false-positive and false-negative findings (50). Regarding dissemination of the results, we found that only about one-sixth of the synopses were supported by an online database as previously suggested (7), the presence of which improved the impact and visibility of the article. Synopses accompanied by an online database seem to attract significantly more citations than synopses without a database. All available databases have been updated and have incorporated new features for data-searching and analysis over time, improving the experience for the involved investigators and clinicians and promoting the relevant scientific fields. As expected, all field synopses used meta-analytical techniques to synthesize available evidence. Meta-analysis is now a sine qua non for the discovery of common variants for complex diseases (51); it is especially valuable when applied to studies retrieved through systematic literature reviews, such as those typically performed in field synopses. In this context, meta-analytical techniques and related “diagnostic” procedures can help identify biases (e.g., small-study effects, excess significance bias, and other types of bias) and assess between-study heterogeneity and its potential sources, and in general they provide insights about the validity of the proposed associations. Overall, the current findings support the crucial role of meta-analysis in the discovery of novel genetic associations either through synthesis of published studies or through meta-analysis of individual-level data as routinely performed in GWAS consortia (52). Our study has some caveats that should be acknowledged. First, we did not include synopses focusing on gene families,

gene pathways (53), or multiple variants of a specific gene (54), even though such approaches may have been conducted rigorously. In addition, we did not include databases that were not accompanied by published articles describing the results in detail or a published article focusing on technical characteristics, such as MSGene (http://www.msgene.org/), ALSGene (55), or CADgene (56). These initiatives are expected to follow stringent criteria for the synthesis and reporting of the available data, and they add to the existing evidence of genetic associations. Such efforts should be encouraged and should follow the proper guidelines as previously reported (7, 15, 44). Furthermore, some investigators have systematically appraised the published meta-analyses of genetic association studies related to a specific disease. For example, in an overview of meta-analyses, Dong et al. (57) examined the validity of genetic associations for cancer, estimating the false-positive report probability. This methodology did not provide a complete snapshot of all published genetic associations related to cancer susceptibility, but it contributed to the identification of false-positive genetic associations. Moreover, we did not capture articles published before 2007, given that the actual definition of a field synopsis and the ideal description of a field synopsis had not been set by that time. Finally, we stress that our work summarizes and provides a snapshot of the field without assessing the methodological approaches used by the synopses’ authors. Of course, all of the approaches may have certain disadvantages, and it is possible that in some cases they may have been used inappropriately. Field synopses have been proven to be a valuable tool for identifying common variants in the past, and we expect that new findings will be continuously added in the future. This method could also be proven useful for rare variants, where single studies will mostly lack the statistical power to show a significant association. Therefore, it is obvious that the rapid evolution of the field necessitates such systematic approaches for the quantitative synthesis and critical appraisal of significant associations. However, researchers should adhere to rules and guidelines, such as those provided by the HuGENet road map (7), in order to achieve adequate quality and reporting of the identified signals. Similar approaches, such as umbrella reviews, are already being used in other fields of epidemiology to systematically appraise the validity and credibility of different factors, including biomarkers or other environmental risk factors (58). The existence of an online database and the application of statistical approaches to evaluate the cumulative evidence and to adjust for multiple comparisons may improve the reporting and impact of field synopses.

12 Belbasis et al.

The findings and conclusions in this report are those of the authors and do not necessarily represent the official views of the National Cancer Institute or the US Department of Health and Human Services. Conflict of interest: none declared.

18.

19. REFERENCES 20.

21.

22. 23. 24. 25.

26.

27.

28. 29.

30. 31. 32. 33. 34. 35. 36.


1. Ioannidis JP. Expectations, validity, and reality in omics. J Clin Epidemiol. 2010;63(9):945–949. 2. Lin BK, Clyne M, Walsh M, et al. Tracking the epidemiology of human genes in the literature: the HuGE Published Literature database. Am J Epidemiol. 2006;164(1):1–4. 3. Yu W, Gwinn M, Clyne M, et al. A navigator for human genome epidemiology. Nat Genet. 2008;40(2):124–125. 4. Ioannidis JP, Ntzani EE, Trikalinos TA, et al. Replication validity of genetic association studies. Nat Genet. 2001;29(3): 306–309. 5. Panagiotou OA, Ioannidis JP; Genome-Wide Significance Project. What should the genome-wide significance threshold be? Empirical replication of borderline genetic associations. Int J Epidemiol. 2012;41(1):273–286. 6. Khoury MJ, Gwinn M, Ioannidis JP. The emergence of translational epidemiology: from scientific discovery to population health impact. Am J Epidemiol. 2010;172(5): 517–524. 7. Khoury MJ, Bertram L, Boffetta P, et al. Genome-wide association studies, field synopses, and the development of the knowledge base on genetic variation and human diseases. Am J Epidemiol. 2009;170(3):269–279. 8. Ioannidis JP, Gwinn M, Little J, et al. A road map for efficient and reliable human genome epidemiology. Nat Genet. 2006; 38(1):3–5. 9. Ariyaratnam R, Casas JP, Whittaker J, et al. Genetics of ischaemic stroke among persons of non-European descent: a meta-analysis of eight genes involving approximately 32,500 individuals. PLoS Med. 2007;4(4):e131. 10. Bertram L, McQueen MB, Mullin K, et al. Systematic meta-analyses of Alzheimer disease genetic association studies: the AlzGene database. Nat Genet. 2007;39(1):17–23. 11. Kitsios G, Zintzaras E. Genetic variation associated with ischemic heart failure: a HuGE review and meta-analysis. Am J Epidemiol. 2007;166(6):619–633. 12. Tüttelmann F, Rajpert-De Meyts E, Nieschlag E, et al. Gene polymorphisms and male infertility—a meta-analysis and literature review. Reprod Biomed Online. 2007;15(6):643–658. 13. Allen NC, Bagade S, McQueen MB, et al. Systematic meta-analyses and field synopsis of genetic association studies in schizophrenia: the SzGene database. Nat Genet. 2008;40(7): 827–834. 14. Lill CM, Roehr JT, McQueen MB, et al. Comprehensive research synopsis and systematic meta-analyses in Parkinson’s disease genetics: the PDGene database. PLoS Genet. 2012;8(3): e1002548. 15. Ioannidis JP, Boffetta P, Little J, et al. Assessment of cumulative evidence on genetic associations: interim guidelines. Int J Epidemiol. 2008;37(1):120–132. 16. Attia J, Ioannidis JP, Thakkinstian A, et al. How to use an article about genetic association: B: are the results of the study valid? JAMA. 2009;301(2):191–197. 17. Hindorff LA, Sethupathy P, Junkins HA, et al. Potential etiologic and functional implications of genome-wide

association loci for human diseases and traits. Proc Natl Acad Sci U S A. 2009;106(23):9362–9367. Paternoster L, Martinez-Gonzalez NA, Charleton R, et al. Genetic effects on carotid intima-media thickness: systematic assessment and meta-analyses of candidate gene polymorphisms studied in more than 5000 subjects. Circ Cardiovasc Genet. 2010;3(1):15–21. Paternoster L, Chen W, Sudlow CL. Genetic determinants of white matter hyperintensities on brain scans: a systematic assessment of 19 candidate gene polymorphisms in 46 studies in 19,000 subjects. Stroke. 2009;40(6):2020–2026. Rannikmäe K, Samarasekera N, Martînez-Gonzâlez NA, et al. Genetics of cerebral amyloid angiopathy: systematic review and meta-analysis. J Neurol Neurosurg Psychiatry. 2013;84(8): 901–908. Brouwer MC, de Gans J, Heckenberg SG, et al. Host genetic susceptibility to pneumococcal and meningococcal disease: a systematic review and meta-analysis. Lancet Infect Dis. 2009; 9(1):31–44. Taylor S. Molecular genetics of obsessive-compulsive disorder: a comprehensive meta-analysis of genetic association studies. Mol Psychiatry. 2013;18(7):799–805. Staines-Urias E, Paez MC, Doyle P, et al. Genetic association studies in pre-eclampsia: systematic meta-analyses and field synopsis. Int J Epidemiol. 2012;41(6):1764–1775. Seifuddin F, Mahon PB, Judy J, et al. Meta-analysis of genetic association studies on bipolar disorder. Am J Med Genet B Neuropsychiatr Genet. 2012;159B(5):508–518. Ma X, Zhang B, Zheng W. Genetic variants associated with colorectal cancer risk: comprehensive research synopsis, meta-analysis, and epidemiological evidence. Gut. 2014;63(2): 326–336. Theodoratou E, Montazeri Z, Hawken S, et al. Systematic meta-analyses and field synopsis of genetic association studies in colorectal cancer. J Natl Cancer Inst. 2012;104(19): 1433–1457. Chatzinasiou F, Lill CM, Kypreou K, et al. Comprehensive field synopsis and systematic meta-analyses of genetic association studies in cutaneous melanoma. J Natl Cancer Inst. 2011; 103(16):1227–1235. Alg VS, Sofat R, Houlden H, et al. Genetic risk factors for intracranial aneurysms: a meta-analysis in more than 116,000 individuals. Neurology. 2013;80(23):2154–2165. Jiang Y, Zhang R, Zheng J, et al. Meta-analysis of 125 rheumatoid arthritis-related single nucleotide polymorphisms studied in the past two decades. PLoS One. 2012;7(12):e51571. Egger M, Davey Smith G, Schneider M, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997; 315(7109):629–634. Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50(4): 1088–1101. Harbord RM, Egger M, Sterne JA. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Stat Med. 2006;25(20):3443–3457. Ioannidis JP, Trikalinos TA. An exploratory test for an excess of significant findings. Clin Trials. 2007;4(3):245–253. Rosenthal R. The file drawer problem and tolerance for null results. Psychol Bull. 1979;86(3):638–641. Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56(2):455–463. Goodman SN. Toward evidence-based medical statistics. 2: the Bayes factor. Ann Intern Med. 1999;130(12):1005–1013.


56. Liu H, Liu W, Liao Y, et al. CADgene: a comprehensive database for coronary artery disease genes. Nucleic Acids Res. 2011;39(database issue):D991–D996. 57. Dong LM, Potter JD, White E, et al. Genetic susceptibility to cancer: the role of polymorphisms in candidate genes. JAMA. 2008;299(20):2423–2436. 58. Ioannidis JP. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. CMAJ. 2009; 181(8):488–493. 59. Abhary S, Hewitt AW, Burdon KP, et al. A systematic meta-analysis of genetic association studies for diabetic retinopathy. Diabetes. 2009;58(9):2137–2147. 60. Brouwer MC, Read RC, van de Beek D. Host genetics and outcome in meningococcal disease: a systematic review and meta-analysis. Lancet Infect Dis. 2010;10(4):262–274. 61. Buurma AJ, Turner RJ, Driessen JH, et al. Genetic variants in pre-eclampsia: a meta-analysis. Hum Reprod Update. 2013; 19(3):289–303. 62. Castaldi PJ, Cho MH, Cohn M, et al. The COPD genetic association compendium: a comprehensive online database of COPD genetic associations. Hum Mol Genet. 2010;19(3): 526–534. 63. Chen H, Liu K, Chen LJ, et al. Genetic associations in polypoidal choroidal vasculopathy: a systematic review and meta-analysis. Mol Vis. 2012;18:816–829. 64. Debette S, Markus HS. The genetics of cervical artery dissection: a systematic review. Stroke. 2009;40(6):e459–e466. 65. Dolan SM, Hollegaard MV, Merialdi M, et al. Synopsis of preterm birth genetic association studies: the Preterm Birth Genetics Knowledge Base (PTBGene). Public Health Genomics. 2010;13(7-8):514–523. 66. Dwyer R, Skrobot OA, Dwyer J, et al. Using Alzgene-like approaches to investigate susceptibility genes for vascular cognitive impairment. J Alzheimers Dis. 2013;34(1):145–154. 67. Gizer IR, Ficks C, Waldman ID. Candidate gene studies of ADHD: a meta-analytic review. Hum Genet. 2009;126(1): 51–90. 68. Gohil R, Peck G, Sharma P. The genetics of venous thromboembolism. A meta-analysis involving approximately 120,000 cases and 180,000 controls. Thromb Haemost. 2009; 102(2):360–370. 69. Hamzi K, Tazzite A, Nadifi S. Large-scale meta-analysis of genetic studies in ischemic stroke: five genes involving 152,797 individuals. Indian J Hum Genet. 2011;17(3): 212–217. 70. Jin F, Xiong WJ, Jing JC, et al. Evaluation of the association studies of single nucleotide polymorphisms and hepatocellular carcinoma: a systematic review. J Cancer Res Clin Oncol. 2011; 137(7):1095–1104. 71. Lee YH, Choi SJ, Ji JD, et al. Candidate gene studies of fibromyalgia: a systematic review and meta-analysis. Rheumatol Int. 2012;32(2):417–426. 72. Li X, Zhang Y, Zhang J, et al. Asthma susceptible genes in Chinese population: a meta-analysis. Respir Res. 2010;11: 129–150. 73. Loh M, Koh KX, Yeo BH, et al. Meta-analysis of genetic polymorphisms and gastric cancer risk: variability in associations according to race. Eur J Cancer. 2009;45(14): 2562–2568. 74. López-León S, Janssens AC, González-Zuloeta Ladd AM, et al. Meta-analyses of genetic studies on major depressive disorder. Mol Psychiatry. 2008;13(8):772–785. 75. Marjot T, Yadav S, Hasan N, et al. Genes associated with adult cerebral venous thrombosis. Stroke. 2011;42(4):913–918.


37. Wacholder S, Chanock S, Garcia-Closas M, et al. Assessing the probability that a positive report is false: an approach for molecular epidemiology studies. J Natl Cancer Inst. 2004; 96(6):434–442. 38. Wakefield J. A Bayesian measure of the probability of false discovery in genetic epidemiology studies. Am J Hum Genet. 2007;81(2):208–227. 39. Calati R, Gressier F, Balestri M, et al. Genetic modulation of borderline personality disorder: systematic review and meta-analysis. J Psychiatr Res. 2013;47(10):1275–1287. 40. He X, Yu C, Zhao P, et al. The genetics of Henoch-Schönlein purpura: a systematic review and meta-analysis. Rheumatol Int. 2013;33(6):1387–1395. 41. Xin XY, Song YY, Ma JF, et al. Gene polymorphisms and risk of adult early-onset ischemic stroke: a meta-analysis. Thromb Res. 2009;124(5):619–624. 42. Maxwell SS, Jackson CA, Paternoster L, et al. Genetic associations with brain microbleeds: systematic review and meta-analyses. Neurology. 2011;77(2):158–167. 43. Zaffanello M, Tardivo S, Cataldi L, et al. Genetic susceptibility to renal scar formation after urinary tract infection: a systematic review and meta-analysis of candidate gene polymorphisms. Pediatr Nephrol. 2011;26(7):1017–1029. 44. Little J, Higgins JP, Ioannidis JP, et al. STrengthening the REporting of Genetic Association Studies (STREGA): an extension of the STROBE Statement. PLoS Med. 2009;6(2): e22. 45. Zintzaras E, Kitsios GD. Synopsis and synthesis of candidate-gene association studies in chronic lymphocytic leukemia: the CUMAGAS-CLL information system. Am J Epidemiol. 2009;170(6):671–678. 46. Zintzaras E, Kitsios GD, Ziogas DC, et al. Field synopsis and synthesis of genetic association studies in osteoarthritis: the CUMAGAS-OSTEO information system. Am J Epidemiol. 2010;171(8):851–858. 47. Zintzaras E, Zdoukopoulos N. A field synopsis and meta-analysis of genetic association studies in peripheral arterial disease: the CUMAGAS-PAD database. Am J Epidemiol. 2009;170(1):1–11. 48. Sava GP, Speedy HE, Houlston RS. Candidate gene association studies and risk of chronic lymphocytic leukemia: a systematic review and meta-analysis. Leuk Lymphoma. 2014;55(1): 160–167. 49. Vassos E, Collier DA, Fazel S. Systematic meta-analyses and field synopsis of genetic association studies of violence and aggression. Mol Psychiatry. 2014;19(4):471–477. 50. Ioannidis JP, Tarone R, McLaughlin JK. The false-positive to false-negative ratio in epidemiologic studies. Epidemiology. 2011;22(4):450–456. 51. Evangelou E, Ioannidis JP. Meta-analysis methods for genome-wide association studies and beyond. Nat Rev Genet. 2013;14(6):379–389. 52. Panagiotou OA, Willer CJ, Hirschhorn JN, et al. The power of meta-analysis in genome-wide association studies. Annu Rev Genomics Hum Genet. 2013;14:441–465. 53. Vineis P, Manuguerra M, Kavvoura FK, et al. A field synopsis on low-penetrance variants in DNA repair genes and cancer susceptibility. J Natl Cancer Inst. 2009;101(1):24–36. 54. Gao J, Pan Z, Jiao Z, et al. TPH2 gene polymorphisms and major depression—a meta-analysis. PLoS One. 2012;7(5): e36721. 55. Lill CM, Abel O, Bertram L, et al. Keeping up with genetic discoveries in amyotrophic lateral sclerosis: the ALSoD and ALSGene databases. Amyotroph Lateral Scler. 2011;12(4): 238–249.

14 Belbasis et al.

84. Smolonska J, Wijmenga C, Postma DS, et al. Meta-analyses on suspected chronic obstructive pulmonary disease genes: a summary of 20 years’ research. Am J Respir Crit Care Med. 2009;180(7):618–631. 85. Srivastava K, Srivastava A, Sharma KL, et al. Candidate gene studies in gallbladder cancer: a systematic review and meta-analysis. Mutat Res. 2011;728(1-2):67–79. 86. Varvarigou V, Dahabreh IJ, Malhotra A, et al. A review of genetic association studies of obstructive sleep apnea: field synopsis and meta-analysis. Sleep. 2011;34(11): 1461–1468. 87. Wu D, Zou YF, Xu XY, et al. The association of genetic polymorphisms with cerebral palsy: a meta-analysis. Dev Med Child Neurol. 2011;53(3):217–225. 88. Yadav S, Hasan N, Marjot T, et al. Detailed analysis of gene polymorphisms associated with ischemic stroke in South Asians. PLoS One. 2013;8(3):e57305. 89. Zdoukopoulos N, Zintzaras E. Genetic risk factors for placental abruption: a HuGE review and meta-analysis. Epidemiology. 2008;19(2):309–323. 90. Zhang C, Bao W, Rong Y, et al. Genetic variants and the risk of gestational diabetes mellitus: a systematic review. Hum Reprod Update. 2013;19(4):376–390. 91. Zhang B, Beeghly-Fadiel A, Long J, et al. Genetic variants associated with breast-cancer risk: comprehensive research synopsis, meta-analysis, and epidemiological evidence. Lancet Oncol. 2011;12(5):477–488.

(Appendix follows)


76. Mazaki T, Masuda H, Takayama T. Polymorphisms and pancreatic cancer risk: a meta-analysis. Eur J Cancer Prev. 2011;20(3):169–183. 77. McColgan P, Thant KZ, Sharma P. The genetics of sporadic ruptured and unruptured intracranial aneurysms: a genetic meta-analysis of 8 genes and 13 polymorphisms in approximately 20,000 individuals. J Neurosurg. 2010;112(4): 714–721. 78. Mooyaart AL, Valk EJ, van Es LA, et al. Genetic associations in diabetic nephropathy: a meta-analysis. Diabetologia. 2011; 54(3):544–553. 79. Peck G, Smeeth L, Whittaker J, et al. The genetics of primary haemorrhagic stroke, subarachnoid haemorrhage and ruptured intracranial aneurysms in adults. PLoS One. 2008;3(11):e3691. 80. Povel CM, Boer JM, Reiling E, et al. Genetic variants and the metabolic syndrome: a systematic review. Obes Rev. 2011; 12(11):952–967. 81. Rao R, Tah V, Casas JP, et al. Ischaemic stroke subtypes and their genetic basis: a comprehensive meta-analysis of small and large vessel stroke. Eur Neurol. 2009;61(2):76–86. 82. Romero-Gomez M, Eslam M, Ruiz A, et al. Genes and hepatitis C: susceptibility, fibrosis progression and response to treatment. Liver Int. 2011;31(4):443–460. 83. Schild AH, Pietschnig J, Tran US, et al. Genetic association studies between SNPs and suicidal behavior: a meta-analytical field synopsis. Prog Neuropsychopharmacol Biol Psychiatry. 2013;46:36–42.

Appendix Table 1. Characteristics of Field Synopses of Genetic Association Studies With Overlapping Outcomes, January 1, 2007–October 31, 2013 First Author, Year (Reference No.)

Date of Last Literature Search

Database(s)a Searched

Racial/Ethnic Population Covered






Ischemic Stroke Yadav, 2013 (88)

August 2012

PubMed, EMBASE, Google Scholar

South Asians

42

4

None

None


Hamzi, 2011 (69)

January 2010

PubMed, EMBASE, Scopus

All ethnicities

32

2

None

None

None

Rao, 2009 (81)

January 2007


All ethnicities

NR

0


None

Funnel plots

Xin, 2009b (41)

December 2008 PubMed, EMBASE, Google Scholar

All ethnicities

51

3


None



January 2005

Non-Europeans

60

5


None


Alg, 2013 (28)

December 2012 PubMed, EMBASE, Google Scholar

All ethnicities

66

19

None



McColgan, 2010 (77)

June 2008

PubMed, EMBASE, Google Scholar, Yahoo!

All ethnicities

30

3

None

None


Peck, 2008 (79)

March 2007


All ethnicities

85

5


None


Sava, 2014c (48)

October 2012

PubMed

All ethnicities

36

4

None

FPRP

Egger’s test


February 2009

PubMed, HuGE Navigator

All ethnicities

81

0


None

Harbord’s test

Castaldi, 2010 (62)

July 2008


All ethnicities

108

4


None

Egger’s test


NR

PubMed

All ethnicities

69

8


None

None


Intracranial Aneurysms

Chronic Lymphocytic Leukemia

Colorectal Cancer Ma, 2014 (25)

December 2012 PubMed, Google Scholar

Theodoratou, 2012 (26) March 2011


All ethnicities

945

62


Venice criteria, FPRP, Bonferroni correction


All ethnicities

635

42


Venice criteria, BFDP

Harbord’s test Table continues


c


Chronic Obstructive Pulmonary Disease

16 Belbasis et al.

Appendix Table 1. Continued First Author, Year (Reference No.)

Date of Last Literature Search

Database(s)a Searched

Racial/Ethnic Population Covered






Meningococcal Disease Brouwer, 2010 (60)

October 2009

PubMed

All ethnicities

28

3

None

None

None

Brouwer, 2009 (21)

August 2008

PubMed

All ethnicities

76

2

None

None

None

Preeclampsia Buurma, 2013 (61)

February 2012

PubMed, EMBASE, ISI Web of Science

All ethnicities

542

7

None

None



January 2011

PubMed, EMBASE, KoreaMed

All ethnicities

192

5


Venice criteria

Funnel plots, Galbraith plots, Harbord’s test, ESS test

McColgan, 2010 (77)

June 2008


All ethnicities

30

3

None

None


Peck, 2008 (79)

March 2007


All ethnicities

85

5


None


Subarachnoid Hemorrhage

Abbreviations: BFDP, Bayesian false-discovery probability; EMBASE, Excerpta Medica Database; ESS, excess of statistical significance; FPRP, false-positive report probability; HuGE, Human Genome Epidemiology; ISI, Institute for Scientific Information; NR, not reported. a PubMed: US National Library of Medicine, Bethesda, Maryland; ISI Web of Science: Thomson Reuters, New York, New York; Google Scholar: Google Inc., Mountain View, California; EMBASE: Elsevier B.V., Amsterdam, the Netherlands; Scopus: Elsevier B.V.; Yahoo!: Yahoo! Inc., Sunnyvale, California; HuGE Navigator: Centers for Disease Control and Prevention, Atlanta, Georgia; KoreaMed: Korean Association of Medical Journal Editors, Seoul, South Korea. b The outcome was defined as adult early-onset ischemic stroke. c Article was published online (electronic publication) before November 2013; year refers to the year of print publication.


Epidemiology and genetic diversity of Taenia asiatica: a systematic review.

Genetic epidemiology of pelvic organ prolapse: a systematic review.

Global epidemiology of yaws: a systematic review.

Acute hypertension: a systematic review and appraisal of guidelines.

Critical appraisal of implant impression accuracies: A systematic review.

Appraisal tools for clinical practice guidelines: a systematic review.

Epidemiology of Japanese encephalitis in the Philippines: a systematic review.

Epidemiology of multiple sclerosis in Iran: a systematic review.

Epidemiology of molluscum contagiosum in children: a systematic review.

Childhood asthma and physical activity: a systematic review with meta-analysis and Graphic Appraisal Tool for Epidemiology assessment.

A critical appraisal of the systematic review process: systematic reviews of zirconia single crowns.

Epidemiology of sport-related spinal cord injuries: A systematic review.

Epidemiology of work-related traumatic brain injury: a systematic review.

A HuGE Review and Meta-Analyses of Genetic Associations in New Onset Diabetes after Kidney Transplantation.

A systematic review of Rift Valley Fever epidemiology 1931-2014.

Epidemiology and burden of alopecia areata: a systematic review.

Infective endocarditis epidemiology over five decades: a systematic review.

Multiple Sclerosis Relapses: Epidemiology, Outcomes and Management. A Systematic Review.

Clostridium difficile: epidemiology, diagnostic and therapeutic possibilities-a systematic review.

An appraisal of oral cancer and pre-cancer screening programmes in Europe: a systematic review.

Analyzing communication in genetic consultations--a systematic review.

Economic evaluations in bipolar disorder: a systematic review and critical appraisal.

Use of Systematic Review and Meta-Analysis in Environmental Health Epidemiology: a Systematic Review and Comparison with Guidelines.

A synopsis of the synopses, 2013-2014.