BMJ 2014;348:g3259 doi: 10.1136/bmj.g3259 (Published 16 May 2014)
Page 1 of 2
Editorials
EDITORIALS Delayed publication of vaccine trials Why are we waiting? Sponsors, authors, and editors all contribute to delays 1
Christopher W Jones attending physician , Timothy F Platts-Mills assistant professor
2
Department of Emergency Medicine, Cooper Medical School of Rowan University, Camden, NJ 08103, USA; 2Departments of Emergency Medicine and Anesthesiology, University of North Carolina Chapel Hill, Chapel Hill, NC, USA 1
Among medical interventions to improve human health, vaccination has been and remains one of the most important.1 Given the huge number of deaths from influenza pandemics in recent history, the ability to rapidly develop effective vaccines for new strains of influenza is particularly critical. Making and testing a new influenza vaccine that can be administered to the public takes six months or less.2 For example, the pandemic A/H1N1 2009 influenza strain was first identified in April of 2009; four vaccines were approved by the US Food and Drug Administration in September. After the approval of a vaccine, however, important questions remain regarding dosage, effectiveness, and safety. These questions are best answered by randomized clinical trials, and getting complete results from these trials to policy makers, clinicians, and the general public in a timely manner is essential.
The problems of delayed publication and non-publication of clinical trials have been described in a variety of settings.3-5 In this issue of The BMJ, Manzoli and colleagues (doi:10.1136/ bmj.g3058) examine delays to publication and non-publication for the vitally important area of vaccine trials.6 The authors searched for clinical trials registered from 2006 through 2012 that assessed the effects of vaccines to human papillomavirus, H1N1, meningococcus, pneumococcus, and rotavirus. They then searched for published reports corresponding to these registered trials and reviewed the ClinicalTrials.gov database for non-peer reviewed but publicly available results. The authors found a median delay of 26 months between completion of the trial and publication. More than a quarter of trial results remained unpublished after four years. Although previous studies have documented similar delays to publication,7 given the time sensitivity of vaccine related trial data, it is disappointing to see delays of more than two years. For H1N1 vaccine trials, the median time from trial completion to publication was 22 months, slightly better than for the other vaccine trials studied. However, the 22 month delay still stands in stark contrast with the six months needed to develop the vaccine and the median trial duration observed by Manzoli and colleagues of only seven months. Given the potential impact on public health, where was the urgency in making these results available?
Several reasons for delays between the completion of a trial and publication exist. Analyzing data and drafting a manuscript take time. Sponsors may choose to delay the release of unfavorable results, especially for a vaccine that has been approved and is profitable.
Even if the sponsor supports timely publication, the current peer review process is time consuming, particularly when authors’ first few submissions are to high impact journals with low acceptance rates. Journal policies also contribute to the delay, by prohibiting authors from submitting simultaneously to multiple journals. In short, sponsors, authors, and journal editors all have interests, and none of these interests align exactly with the public’s interests in prompt access to trial results and publication of peer reviewed manuscripts.
Protection of the public’s interest in timely and comprehensive access to the results of clinical trials is likely to be most effectively accomplished through changes in public policy. The 2007 US FDA Amendments Act requires that results from most US trials be made available in ClinicalTrials.gov within one year of trial completion. This requirement applies to most trials of vaccines manufactured in the United States or to trials with US sites. But the requirement has not been aggressively enforced, and fewer than 25% of trials meeting the Amendments Act criteria comply with the one year deadline.8
The Trial and Experimental Studies Transparency (TEST) Act, which would close many of the existing loopholes in reporting requirements, was introduced to the US Congress in 2012 and again in 2013, but has since stalled. Meanwhile, the AllTrials campaign9 and the European parliament have achieved important legislative progress in Europe through passage in April 2014 of the new Clinical Trials Regulation. Among other key requirements, this legislation mandates that summary study results be released within one year of trial completion for most European clinical trials. Passage of the law complements the recent commitment from the European Medicines Agency to release detailed clinical study reports for all future trials submitted to the EMA. Though these developments are promising, the experience of the United States highlights the importance of developing clear guidelines for compliance, and robust enforcement mechanisms.
Correspondence to: C W. Jones [email protected] For personal use only: See rights and reprints http://www.bmj.com/permissions
Subscribe: http://www.bmj.com/subscribe
BMJ 2014;348:g3259 doi: 10.1136/bmj.g3259 (Published 16 May 2014)
Page 2 of 2
EDITORIALS
Changes to the peer review process could also expedite the publication of peer reviewed trial manuscripts. Many journals allow authors to submit reviewers’ comments from other journals, but in our experience authors rarely do so. The result is that peer reviewers and editors end up replicating previous work, which takes time.
By incentivizing or requiring authors to include previously obtained reviews at the time of manuscript submission, journals could expedite the peer review process. Allowing simultaneous submission of a manuscript to multiple journals might also reduce delays to publication by incentivizing journals to deliver publication decisions rapidly and by allowing manuscript evaluations to occur in parallel among different journals.10
WHO, CDC, and European Influenza Surveillance Networks invest hundreds of millions of dollars annually in testing centers and infrastructure to enable the detection of new strains of influenza within weeks and to support rapid vaccine development. Large investments are also made each year by governments and industry to develop and test vaccines for other infectious diseases, such as pneumococcus and rotavirus that are responsible for the deaths of an estimated 800 000 children a year worldwide.11 These investments are wise; accepting a system that leaves the majority of trial results unavailable after two years is not. Competing interests: We have read and understood the BMJ Group policy on declaration of interests and declare the following interests:
For personal use only: See rights and reprints http://www.bmj.com/permissions
CWJ’s department has received funding for participating in clinical trials sponsored by Roche Diagnostics. TFP-M is supported by National Institute on Aging of the National Institutes of Health under a No K23AG038548. Provenance and peer review: Commissioned; not externally peer reviewed. 1 2 3 4 5 6 7 8 9 10 11
Centers for Disease Control and Prevention. Impact of vaccines universally recommended for children—United States, 1990-1998. MMWR 1999;48:243. World Health Organization. Pandemic influenza vaccine manufacturing process and timeline; 2009. www.who.int/csr/disease/swineflu/notes/h1n1_vaccine_20090806/en/. Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 1998;279:281-6. Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ 2011;344:d7292. Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, Platts-Mills TF. Non-publication of large randomized clinical trials: cross sectional analysis. BMJ 2013;347:f6104. Manzoli L, Flacco ME, D’Addario M, Capasso L, De Vito C, Marzuillo C, et al. Non-publication and delayed publication of randomized trials on vaccines: survey. BMJ 2014;348:g3058. Gordon D, Taddei-Peters W, Mascette A, Antman M, Kaufmann PG, Lauer MS. Publication of trials funded by the National Heart, Lung, and Blood Institute. N Engl J Med 2013;369:1926-34. Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ 2012;344:d7373. The AllTrials Campaign. All trials registered, all results reported. 2013. www.alltrials.net. Torgerson DJ, Adamson J, Cockayne S, Dumville J, Petherick E. Submission to multiple journals: a method of reducing time to publication? BMJ 2005;330:305-7. Walker CL, Rudan I, Liu L, Nair H, Theodoratou E, Bhutta ZA, et al. Global burden of childhood pneumonia and diarrhoea. Lancet 2013;381:1405-16.
Cite this as: BMJ 2014;348:g3259 © BMJ Publishing Group Ltd 2014
Subscribe: http://www.bmj.com/subscribe
Page 1 of 2
Editorials
EDITORIALS Delayed publication of vaccine trials Why are we waiting? Sponsors, authors, and editors all contribute to delays 1
Christopher W Jones attending physician , Timothy F Platts-Mills assistant professor
2
Department of Emergency Medicine, Cooper Medical School of Rowan University, Camden, NJ 08103, USA; 2Departments of Emergency Medicine and Anesthesiology, University of North Carolina Chapel Hill, Chapel Hill, NC, USA 1
Among medical interventions to improve human health, vaccination has been and remains one of the most important.1 Given the huge number of deaths from influenza pandemics in recent history, the ability to rapidly develop effective vaccines for new strains of influenza is particularly critical. Making and testing a new influenza vaccine that can be administered to the public takes six months or less.2 For example, the pandemic A/H1N1 2009 influenza strain was first identified in April of 2009; four vaccines were approved by the US Food and Drug Administration in September. After the approval of a vaccine, however, important questions remain regarding dosage, effectiveness, and safety. These questions are best answered by randomized clinical trials, and getting complete results from these trials to policy makers, clinicians, and the general public in a timely manner is essential.
The problems of delayed publication and non-publication of clinical trials have been described in a variety of settings.3-5 In this issue of The BMJ, Manzoli and colleagues (doi:10.1136/ bmj.g3058) examine delays to publication and non-publication for the vitally important area of vaccine trials.6 The authors searched for clinical trials registered from 2006 through 2012 that assessed the effects of vaccines to human papillomavirus, H1N1, meningococcus, pneumococcus, and rotavirus. They then searched for published reports corresponding to these registered trials and reviewed the ClinicalTrials.gov database for non-peer reviewed but publicly available results. The authors found a median delay of 26 months between completion of the trial and publication. More than a quarter of trial results remained unpublished after four years. Although previous studies have documented similar delays to publication,7 given the time sensitivity of vaccine related trial data, it is disappointing to see delays of more than two years. For H1N1 vaccine trials, the median time from trial completion to publication was 22 months, slightly better than for the other vaccine trials studied. However, the 22 month delay still stands in stark contrast with the six months needed to develop the vaccine and the median trial duration observed by Manzoli and colleagues of only seven months. Given the potential impact on public health, where was the urgency in making these results available?
Several reasons for delays between the completion of a trial and publication exist. Analyzing data and drafting a manuscript take time. Sponsors may choose to delay the release of unfavorable results, especially for a vaccine that has been approved and is profitable.
Even if the sponsor supports timely publication, the current peer review process is time consuming, particularly when authors’ first few submissions are to high impact journals with low acceptance rates. Journal policies also contribute to the delay, by prohibiting authors from submitting simultaneously to multiple journals. In short, sponsors, authors, and journal editors all have interests, and none of these interests align exactly with the public’s interests in prompt access to trial results and publication of peer reviewed manuscripts.
Protection of the public’s interest in timely and comprehensive access to the results of clinical trials is likely to be most effectively accomplished through changes in public policy. The 2007 US FDA Amendments Act requires that results from most US trials be made available in ClinicalTrials.gov within one year of trial completion. This requirement applies to most trials of vaccines manufactured in the United States or to trials with US sites. But the requirement has not been aggressively enforced, and fewer than 25% of trials meeting the Amendments Act criteria comply with the one year deadline.8
The Trial and Experimental Studies Transparency (TEST) Act, which would close many of the existing loopholes in reporting requirements, was introduced to the US Congress in 2012 and again in 2013, but has since stalled. Meanwhile, the AllTrials campaign9 and the European parliament have achieved important legislative progress in Europe through passage in April 2014 of the new Clinical Trials Regulation. Among other key requirements, this legislation mandates that summary study results be released within one year of trial completion for most European clinical trials. Passage of the law complements the recent commitment from the European Medicines Agency to release detailed clinical study reports for all future trials submitted to the EMA. Though these developments are promising, the experience of the United States highlights the importance of developing clear guidelines for compliance, and robust enforcement mechanisms.
Correspondence to: C W. Jones [email protected] For personal use only: See rights and reprints http://www.bmj.com/permissions
Subscribe: http://www.bmj.com/subscribe
BMJ 2014;348:g3259 doi: 10.1136/bmj.g3259 (Published 16 May 2014)
Page 2 of 2
EDITORIALS
Changes to the peer review process could also expedite the publication of peer reviewed trial manuscripts. Many journals allow authors to submit reviewers’ comments from other journals, but in our experience authors rarely do so. The result is that peer reviewers and editors end up replicating previous work, which takes time.
By incentivizing or requiring authors to include previously obtained reviews at the time of manuscript submission, journals could expedite the peer review process. Allowing simultaneous submission of a manuscript to multiple journals might also reduce delays to publication by incentivizing journals to deliver publication decisions rapidly and by allowing manuscript evaluations to occur in parallel among different journals.10
WHO, CDC, and European Influenza Surveillance Networks invest hundreds of millions of dollars annually in testing centers and infrastructure to enable the detection of new strains of influenza within weeks and to support rapid vaccine development. Large investments are also made each year by governments and industry to develop and test vaccines for other infectious diseases, such as pneumococcus and rotavirus that are responsible for the deaths of an estimated 800 000 children a year worldwide.11 These investments are wise; accepting a system that leaves the majority of trial results unavailable after two years is not. Competing interests: We have read and understood the BMJ Group policy on declaration of interests and declare the following interests:
For personal use only: See rights and reprints http://www.bmj.com/permissions
CWJ’s department has received funding for participating in clinical trials sponsored by Roche Diagnostics. TFP-M is supported by National Institute on Aging of the National Institutes of Health under a No K23AG038548. Provenance and peer review: Commissioned; not externally peer reviewed. 1 2 3 4 5 6 7 8 9 10 11
Centers for Disease Control and Prevention. Impact of vaccines universally recommended for children—United States, 1990-1998. MMWR 1999;48:243. World Health Organization. Pandemic influenza vaccine manufacturing process and timeline; 2009. www.who.int/csr/disease/swineflu/notes/h1n1_vaccine_20090806/en/. Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 1998;279:281-6. Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ 2011;344:d7292. Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, Platts-Mills TF. Non-publication of large randomized clinical trials: cross sectional analysis. BMJ 2013;347:f6104. Manzoli L, Flacco ME, D’Addario M, Capasso L, De Vito C, Marzuillo C, et al. Non-publication and delayed publication of randomized trials on vaccines: survey. BMJ 2014;348:g3058. Gordon D, Taddei-Peters W, Mascette A, Antman M, Kaufmann PG, Lauer MS. Publication of trials funded by the National Heart, Lung, and Blood Institute. N Engl J Med 2013;369:1926-34. Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ 2012;344:d7373. The AllTrials Campaign. All trials registered, all results reported. 2013. www.alltrials.net. Torgerson DJ, Adamson J, Cockayne S, Dumville J, Petherick E. Submission to multiple journals: a method of reducing time to publication? BMJ 2005;330:305-7. Walker CL, Rudan I, Liu L, Nair H, Theodoratou E, Bhutta ZA, et al. Global burden of childhood pneumonia and diarrhoea. Lancet 2013;381:1405-16.
Cite this as: BMJ 2014;348:g3259 © BMJ Publishing Group Ltd 2014
Subscribe: http://www.bmj.com/subscribe
