EDITORIAL
Meta-analyses: editor’s dream or nightmare? Greater numbers of meta-analyses are finding their way into medical journals (Figure 1), including the International Journal of Clinical Practice (Figure 2). China has surpassed the USA as the current top producer of meta-analyses; the annual number of metaanalyses from China increased 40-fold between 2003 and 2011 (1). There is greater interest in ‘getting the big picture’ and having information that supports evidence-based practice. Another reason for the proliferation of meta-analytic papers in the scientific literature has been the development of automated tools that perform the analyses, produce the forest plots in a format suitable for publication, and provide a template for the authors to fill in the blanks. It is also true that constructing and publishing a meta-analytical report is less time-consuming and certainly substantially less expensive than conducting a clinical
trial that sets out to address the clinical question at hand. Add a push from one’s University or Institution to ‘publish or perish’, and the end result is a flood of journal article submissions of papers that do little to help clinicians in their work with patients. When properly executed, meta-analyses have the potential to inform clinical practice and are often done in conjunction with systematic reviews. This is no easy task and requires thoughtful care in determining the best way to differentiate between the utility of competing interventions, laboratory tests and other procedures. Meta-analyses are often necessary because head-to-head studies are unlikely to have been conducted for all competing scenarios (2). As noted by many, including this editor (3), systematic meta-analytic reviews of randomised trials sit atop the hierarchy of strength of evidence for
6000 5000 4000 3000
Prior to submission, authors need to ask themselves if their contribution can potentially impact medical decision-making
2000 1000 0
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
Figure 1 Number of articles with ‘meta-analysis’ in the title published by journals indexed on PubMed, by year, 2003–2012
12 10 8 6 4 2 0
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
Figure 2 Number of articles with ‘meta-analysis’ in the title published by the International Journal of Clinical Practice, by
year, 2003–2012 ª 2013 John Wiley & Sons Ltd Int J Clin Pract, November 2013, 67, 11, 1069–1075
1069
1070
Editorials
treatment decisions, bested only by the often mythical N of 1 randomised clinical trial (4). Systematic meta-analytic reviews of randomised trials are generally considered to be superior to the usual single randomised trial or to observational studies. However, not all meta-analyses are of sufficient quality to inform clinical care. Several resources are available to readers so that they may be better able to understand and interpret meta-analyses (5,6). It behoves the clinician to understand what meta-analyses can and cannot tell us. Applying the results of a meta-analysis to one’s clinical practice entails verifying that the outcomes that were measured are relevant to you and your patients (3). Other questions to ask include: Would your patient have been eligible for any of the studies included in the meta-analysis? Is the dose of the medication that was studied similar to the dose you are contemplating? Is the measured outcome, such as reduction in symptoms to below a set threshold, commonly achievable? Moreover, the magnitude of the treatment effect must be clinically significant. The International Journal of Clinical Practice is eager to publish well-thought-out, and clinically relevant meta-analyses that answer questions that are of general interest. The clinical research questions must be of the sort that actually requires the analysis of multiple studies in order to arrive at a generalisable and reasonably precise answer, or conclude that such answers are not yet available. Unfortunately, when there are too few quality studies, with too few subjects, a meta-analysis becomes a mere academic exercise. An additional issue is that there are many methodological considerations in constructing a proper meta-analysis, from the appropriate selection criteria for studies to be included, to tests of heterogeneity, strategies to enhance precision and the interpretative procedures. Universal agreement is absent regarding
References 1 Ioannidis JP, Chang CQ, Lam TK, Schully SD, Khoury MJ. The geometric increase in meta-analyses from China in the genomic era. PLoS ONE 2013; 8: e65602. 2 Edwards S, Clarke M, Wordsworth S, Borrill J. Indirect comparisons of treatments based on systematic reviews of randomised controlled trials. Int J Clin Pract 2009; 63: 841–54.
which ways are best. Competing academics often have the urge to point out these methodological issues in letters to the editor, the content of which hold little interest to the general reader. An effort is being made by the International Journal of Clinical Practice to steer away from publishing this correspondence and urging the submitting party to communicate directly with the author of the metaanalysis in question. Next steps: The International Journal of Clinical Practice urges authors to consult the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement available through http:// www.prisma-statement.org/. The PRISMA statement can help authors improve their reporting of systematic reviews and meta-analyses via a 27-item checklist and a four-phase flow diagram. Other resources can be found in (7). However, guidelines are no substitute for common sense. Prior to submission, authors need to ask themselves if general readers will find the topic interesting, timely, that the paper covers it in enough depth, and whether their contribution can potentially impact medical decision-making.
Disclosures In the past 36 months, Leslie Citrome has engaged in collaborative research with, or received consulting or speaking fees, from: Alexza, Alkermes, AstraZeneca, Avanir, Bristol-Myers Squibb, Eli Lilly, Envivo, Forest, Genentech, Janssen, Lundbeck, Merck, Mylan, Novartis, Noven, Otsuka, Pfizer, Reckitt Benckiser, ReViva, Shire, Sunovion, Takeda and Valeant.
3 Citrome L. Systematic reviews: much ado about a lot. Int J Clin Pract 2009; 63: 832–3. 4 Guyatt GH, Rennie D. Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice. Chicago, IL: AMA Press, 2001. Table 1A-1, page 7. 5 Khoshdel A, Attia J, Carney SL. Basic concepts in meta-analysis: a primer for clinicians. Int J Clin Pract 2006; 60: 1287–94.
L. Citrome New York Medical College, Valhalla, NY, USA Email: [email protected]
6 Lam RW, Kennedy SH. Using metaanalysis to evaluate evidence: practical tips and traps. Can J Psychiatry 2005; 50: 167–74. 7 Simera I, Altman DG. Reporting medical research. Int J Clin Pract 2013; 67: 710–6. doi: 10.1111/ijcp.12324
ª 2013 John Wiley & Sons Ltd Int J Clin Pract, November 2013, 67, 11, 1069–1075
Meta-analyses: editor’s dream or nightmare? Greater numbers of meta-analyses are finding their way into medical journals (Figure 1), including the International Journal of Clinical Practice (Figure 2). China has surpassed the USA as the current top producer of meta-analyses; the annual number of metaanalyses from China increased 40-fold between 2003 and 2011 (1). There is greater interest in ‘getting the big picture’ and having information that supports evidence-based practice. Another reason for the proliferation of meta-analytic papers in the scientific literature has been the development of automated tools that perform the analyses, produce the forest plots in a format suitable for publication, and provide a template for the authors to fill in the blanks. It is also true that constructing and publishing a meta-analytical report is less time-consuming and certainly substantially less expensive than conducting a clinical
trial that sets out to address the clinical question at hand. Add a push from one’s University or Institution to ‘publish or perish’, and the end result is a flood of journal article submissions of papers that do little to help clinicians in their work with patients. When properly executed, meta-analyses have the potential to inform clinical practice and are often done in conjunction with systematic reviews. This is no easy task and requires thoughtful care in determining the best way to differentiate between the utility of competing interventions, laboratory tests and other procedures. Meta-analyses are often necessary because head-to-head studies are unlikely to have been conducted for all competing scenarios (2). As noted by many, including this editor (3), systematic meta-analytic reviews of randomised trials sit atop the hierarchy of strength of evidence for
6000 5000 4000 3000
Prior to submission, authors need to ask themselves if their contribution can potentially impact medical decision-making
2000 1000 0
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
Figure 1 Number of articles with ‘meta-analysis’ in the title published by journals indexed on PubMed, by year, 2003–2012
12 10 8 6 4 2 0
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
Figure 2 Number of articles with ‘meta-analysis’ in the title published by the International Journal of Clinical Practice, by
year, 2003–2012 ª 2013 John Wiley & Sons Ltd Int J Clin Pract, November 2013, 67, 11, 1069–1075
1069
1070
Editorials
treatment decisions, bested only by the often mythical N of 1 randomised clinical trial (4). Systematic meta-analytic reviews of randomised trials are generally considered to be superior to the usual single randomised trial or to observational studies. However, not all meta-analyses are of sufficient quality to inform clinical care. Several resources are available to readers so that they may be better able to understand and interpret meta-analyses (5,6). It behoves the clinician to understand what meta-analyses can and cannot tell us. Applying the results of a meta-analysis to one’s clinical practice entails verifying that the outcomes that were measured are relevant to you and your patients (3). Other questions to ask include: Would your patient have been eligible for any of the studies included in the meta-analysis? Is the dose of the medication that was studied similar to the dose you are contemplating? Is the measured outcome, such as reduction in symptoms to below a set threshold, commonly achievable? Moreover, the magnitude of the treatment effect must be clinically significant. The International Journal of Clinical Practice is eager to publish well-thought-out, and clinically relevant meta-analyses that answer questions that are of general interest. The clinical research questions must be of the sort that actually requires the analysis of multiple studies in order to arrive at a generalisable and reasonably precise answer, or conclude that such answers are not yet available. Unfortunately, when there are too few quality studies, with too few subjects, a meta-analysis becomes a mere academic exercise. An additional issue is that there are many methodological considerations in constructing a proper meta-analysis, from the appropriate selection criteria for studies to be included, to tests of heterogeneity, strategies to enhance precision and the interpretative procedures. Universal agreement is absent regarding
References 1 Ioannidis JP, Chang CQ, Lam TK, Schully SD, Khoury MJ. The geometric increase in meta-analyses from China in the genomic era. PLoS ONE 2013; 8: e65602. 2 Edwards S, Clarke M, Wordsworth S, Borrill J. Indirect comparisons of treatments based on systematic reviews of randomised controlled trials. Int J Clin Pract 2009; 63: 841–54.
which ways are best. Competing academics often have the urge to point out these methodological issues in letters to the editor, the content of which hold little interest to the general reader. An effort is being made by the International Journal of Clinical Practice to steer away from publishing this correspondence and urging the submitting party to communicate directly with the author of the metaanalysis in question. Next steps: The International Journal of Clinical Practice urges authors to consult the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement available through http:// www.prisma-statement.org/. The PRISMA statement can help authors improve their reporting of systematic reviews and meta-analyses via a 27-item checklist and a four-phase flow diagram. Other resources can be found in (7). However, guidelines are no substitute for common sense. Prior to submission, authors need to ask themselves if general readers will find the topic interesting, timely, that the paper covers it in enough depth, and whether their contribution can potentially impact medical decision-making.
Disclosures In the past 36 months, Leslie Citrome has engaged in collaborative research with, or received consulting or speaking fees, from: Alexza, Alkermes, AstraZeneca, Avanir, Bristol-Myers Squibb, Eli Lilly, Envivo, Forest, Genentech, Janssen, Lundbeck, Merck, Mylan, Novartis, Noven, Otsuka, Pfizer, Reckitt Benckiser, ReViva, Shire, Sunovion, Takeda and Valeant.
3 Citrome L. Systematic reviews: much ado about a lot. Int J Clin Pract 2009; 63: 832–3. 4 Guyatt GH, Rennie D. Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice. Chicago, IL: AMA Press, 2001. Table 1A-1, page 7. 5 Khoshdel A, Attia J, Carney SL. Basic concepts in meta-analysis: a primer for clinicians. Int J Clin Pract 2006; 60: 1287–94.
L. Citrome New York Medical College, Valhalla, NY, USA Email: [email protected]
6 Lam RW, Kennedy SH. Using metaanalysis to evaluate evidence: practical tips and traps. Can J Psychiatry 2005; 50: 167–74. 7 Simera I, Altman DG. Reporting medical research. Int J Clin Pract 2013; 67: 710–6. doi: 10.1111/ijcp.12324
ª 2013 John Wiley & Sons Ltd Int J Clin Pract, November 2013, 67, 11, 1069–1075
