Requirements for trial registration and adherence to reporting guidelines in critical care journals: a meta-epidemiological study of journals' instructions for authors.

CE: Swati; IJEBH-D-17-00013; Total nos of Pages: 11;

IJEBH-D-17-00013

ORIGINAL RESEARCH

Requirements for trial registration and adherence to reporting guidelines in critical care journals: a meta-epidemiological study of journals’ instructions for authors Matthew T. Sims BS, Jake X. Checketts BS, Cole Wayant BS and Matt Vassar PhD Oklahoma State University Center for Health Sciences, Tulsa, Oklahoma, USA

ABSTRACT Background: The purpose of this study was to investigate the policies of critical care journals with regard to guideline adoption and clinical trial registration to understand the extent to which journals use these mechanisms to improve reporting practices. Methods: The current study’s sample comprised 37 critical care journals cataloged in the Expanded Science Citation Index of the 2015 Journal Citation Reports and Google Scholar Metrics h5-index critical care subcategory. A webbased data abstraction was performed to identify which journals required, recommended, or made no mention of 17 different reporting guidelines. We also extracted whether journals required or recommended trial registration. Authors were blinded to one another’s ratings until completion of the data validation. Cross tabulations and descriptive statistics were calculated by using STATA 13. Results: Of the 37 critical care journals, 15 (15/37, 40.5%) did not mention a single guideline within their instructions for authors, whereas the remaining 22 (22/37, 59.5%) mentioned one or more guidelines. The Quality of Reporting of Meta-analyses statement and Standards for Reporting Qualitative Research were not mentioned by any journals, whereas the International Committee of Medical Journal Editors Uniform Requirements for Manuscripts (26/37, 70.3%) and Consolidated Standards of Reporting Trials statement (17/37, 45.9%) were mentioned most often. Of the 37 critical care journals, 21 (21/37, 56.8%) did not mention trial or review registration, but the remaining 16 (16/37, 43.2%) mentioned at least one of the two. Trial registration through ClinicalTrials.gov was mentioned by six (6/37, 16.2%) journals, whereas the WHO registry was mentioned by five (5/37, 13.5%). Sixteen (16/37, 43.2%) journals mentioned trial registration through a registry platform. Conclusion: Nearly half of the journals in our sample did not mention a reporting guideline, and only a small percentage of journals required the registration of clinical trials as a condition for publication. Implementing these two mechanisms may limit bias, and their adoption should be considered by journal editors in critical care. Trial registration: UMIN000024081. Key words: clinical trial registry, ClinicalTrials.gov, Consolidated Standards of Reporting Trials, Enhancing the Quality of Transparency of Health Research Network, International Committee of Medical Journal Editors, Preferred Reporting Items for Systematic Reviews and Meta-Analyses, reporting guidelines, Standards for Reporting Diagnostic Accuracy Studies, Strengthening the Reporting of Observational Studies in Epidemiology, WHO Int J Evid Based Healthc 2017; 15:000–000.

Correspondence: Matthew T. Sims, BS, Oklahoma State University Center for Health Sciences, Tulsa, Oklahoma, USA. E-mail: [email protected] DOI: 10.1097/XEB.0000000000000120

Background

C

ritical care research forms the knowledge base that clinicians use to make evidence-based clinical decisions. Well-designed and well-conducted studies

International Journal of Evidence-Based Healthcare ß 2017 University of Adelaide, Joanna Briggs Institute

1

©2017 University of Adelaide, Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.


IJEBH-D-17-00013

MT Sims et al.

are needed for an adequate evaluation of the validity of study results.1 To enhance validity, critical care researchers must make every effort to minimize bias when planning, conducting, and reporting a study. One method of minimizing bias is the use of reporting guidelines. Reporting guidelines facilitate full disclosure of information, limit exclusion of critical information (e.g., adverse events), and minimize inaccurate interpretations of results.2 Latronico et al.3 evaluated the reporting quality of randomized trials in intensive care medicine and concluded that stronger adherence to Consolidated Standards of Reporting Trials (CONSORT) recommendations was needed; in particular, the random allocation process, description of blinding, and correct reporting of statistically nonsignificant results were found to be in need of improvement. The Enhancing the Quality of Transparency of Health Research (EQUATOR) Network was developed to advance high-quality reporting of health research studies and promote sound reporting practices.4 To date, more than 308 guidelines have been cataloged, including CONSORT for randomized controlled trials, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for systematic reviews, and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) for observational studies. The first aim of this study is to investigate the policies of critical care journals on use of reporting guidelines. Clinical trial registration is another mechanism for minimizing study bias.5,6 For example, outcome inconsistencies between the registration of a clinical trial and final publication of its results can be a significant source of bias. This bias potentially influences treatment decisions, especially when outcomes are modified based on the statistical significance of research findings rather than true differences between treatment and control groups.7 Trial registration promotes transparency and accountability and is thus a viable approach to limiting bias.8 To this end, the US government implemented the Food and Drug Administration (FDA) Amendments Act that mandates that clinical trials (involving FDA-approved drugs, biologics, and devices) be registered on ClinicalTrials.gov prior to patient enrollment, with summary results submitted within 1 year of trial completion.8,9 Such narrow time periods may limit manipulation of results and improve reporting. Trial registration is widely supported by the WHO,10 the European Union,11 and the International Committee of Medical Journal Editors (ICMJE). The ICMJE requires the registration of clinical trials as a prerequisite for publication among member journals.8 Our second aim is to investigate whether journals have requirements for clinical trial registration. 2

Methods We conducted a survey of journal policies and their instructions for authors concerning guideline adherence and trial registration requirements. The current study did not meet the regulatory definition of human subject research as defined in 45 CFR 46.102(d) and (f) of the Department of Health and Human Services’ Code of Federal Regulations and, therefore, was not subject to Institutional Review Board oversight. We applied relevant Statistical Analyses and Methods in the Published Literature guidelines for reporting descriptive statistics12 and a modified version of PRISMA guidelines was used for reporting meta-epidemiological studies (Supplemental Table 1, http://links.lww.com/IJEBH/A12).13 The current study is registered on the University Hospital Medical Information Network Clinical Trial Registry (UMIN-CTR, UMIN000024081) and data from this study is publicly available on figshare (https://doi.org/10.6084/ m9.figshare.4517033.v4). We conducted a search of critical care journals indexed in the Expanded Science Citation Index of the 2015 Journal Citation Reports14–23 (Thomson Reuters; New York, New York; accessed on 11 July 2016) and Google Scholar Metrics h5-index24–27 critical care subcategory (Google Inc.; Mountain View, California, USA; accessed on 16 June 2017). A total of 37 journals were selected between the two journal indexing databases (Fig. 1). The first author (M.T.S.) performed web-based searches for each journal and located their instructions for authors (performed 22 July 2016 and 16 June 2017). M.T.S. and the last author (M.V.) together reviewed each journal’s website to determine the types of articles accepted for publication. In many cases, the descriptions were vague and required further clarification. These authors emailed the editor-in-chief of each journal to inquire about the types of study designs considered for publication (animal research, case reports, clinical trials, diagnostic accuracy studies, economic evaluations, observational studies in epidemiology, qualitative research studies, quality improvement studies, study protocols, and systematic reviews/meta-analyses) (Table 1). For nonresponding editors, we emailed three times at 1-week intervals in an attempt to obtain this information as suggested by Dillman et al.28 to improve response rates (RRs). Prior to the study, all authors met to plan the study design and discuss how to resolve any anticipated problems. Follow-up meetings were held to pilot test the process and address any issues that arose after evaluating a subset of journal guidelines. Necessary adjustments were made by consensus, and a final extraction manual was created to standardize the process.




IJEBH-D-17-00013

ORIGINAL RESEARCH

Thomson Reuters Journal Citation Reports 2015 (N = 33) Lancet Respiratory Medicine American Journal of Respiratory and Critical Care Medicine Intensive Care Medicine Critical Care Medicine Chest Resuscitation Critical Care Annals of Intensive Care Journal of Neurotrauma Critical Care and Resuscitation Journal of Intensive Care Medicine Shock Human Gene Therapy Clinical Development Journal of Trauma and Acute Care Surgery Current Opinion in Critical Care Pediatric Critical Care Medicine Neurocritical Care Journal of Critical Care Seminars in Respiratory and Critical Care Medicine Anasthesiologie und Intensivmedizin American Journal of Critical Care Minerva Anestesiologica Respiratory Care Injury – International Journal of the Care of the Injured Burns Critical Care Clinics Australian Critical Care Therapeutic Hypothermia and Temperature Management Critical Care Nurse Anaesthesia and Intensive Care Medicina Intensiva Journal of Trauma Nursing Anasthesiologie Intensivmedizin Notfallmedizin Schmerztherapie

Google Scholar Metrics h5-index (N = 20)

Records remaining after duplicates removed (N = 5)

Records after exclusions (N = 4)

Exclusion (with reason) (N = 1) Annual Update in Intensive Care and Emergency Medicine (book)

Journals added to final sample (N = 4) Nursing in Critical Care Intensive and Critical Care Nursing Critical Care Research and Practice Indian Journal of Critical Care Medicine

Final sample size (N = 37)

Figure 1. Journal selection flowchart. We conducted a search of two journal indexing databases, Thomson Reuters Journal Citation Reports 2015 and Google Scholar Metrics h5-index, to produce a final sample size of 37 journals.

For each journal, M.T.S. cataloged the title, impact factor, and geographic location (defined by the primary location of the journal’s editorial office as indexed in the Expanded Science Citation Index). We followed the classification of Meerpohl et al.21 and classified geographic location as North America (United States and Canada), the United Kingdom, Europe (without the United Kingdom), or other countries (Australia and India). Next, M.T.S. used a combination of keyword

searches (e.g., ‘PRISMA’, ‘CONSORT’, and ‘STROBE’) and reviewed the full-text versions of the instructions for authors, submission guidelines, editorial policies, and other relevant sections related to the article submission (hereafter collectively referred to as instructions for authors). The policy statements for guidelines listed in Table 2 were extracted. Information regarding clinical trial registration and systematic review registration requirements and acceptable or recommended trials


3


4 U U U

U U

U U U U

U

U U

U U

U

U

U U U U U U U U U

U U U U

U U

U U U U U U

U U U U U

U U U U

U U U U U U U U U U U U U U U U U

U

U

U U U U U U

U U U U U

U U U U U


Diagnostic accuracy studies

Animal trials

U ¼ accepted study type, ‘Box filled with red’ ¼study type not accepted by the journal.

Journal Lancet Respiratory Medicine American Journal of Respiratory and Critical Care Medicine Intensive Care Medicine Critical Care Medicine Chest Resuscitation Critical Care Annals of Intensive Care Journal of Neurotrauma Critical Care and Resuscitation Journal of Intensive Care Medicine Shock Human Gene Therapy Clinical Development Journal of Trauma and Acute Care Surgery Current Opinion in Critical Care Pediatric Critical Care Medicine Neurocritical Care Journal of Critical Care Seminars in Respiratory and Critical Care Medicine Anasthesiologie und Intensivmedizin American Journal of Critical Care Minerva Anestesiologica Respiratory Care Injury - International Journal of The Care of The Injured Burns Critical Care Clinics Nursing in Critical Care Australian Critical Care Therapeutic Hypothermia and Temperature Management Critical Care Nurse Anaesthesia and Intensive Care Intensive and Critical Care Nursing Medicina Intensiva Journal of Trauma Nursing Anasthesiologie Intensivmedizin Notfallmedizin Schmerztherapie Critical Care Research and Practice Indian Journal of Critical Care Medicine

Systematic review and meta-analysis U U

Table 1. Accepted study types by journal

U U

U U U U U U

U U U U U

U U U U


Epidemiology U U

U U

U U U U U U

U U U U U

U

U U

U

U U U

U U U U U

U

U U

Case reports U

U U

U U U U U U

U U U U U

U U U U


Clinical trials U U

U U

U U U U U U

U U U U U

U U U U

U U U U U U U U U U U

U U U U U

Economic evaluations U U

U U

U U U U U U

U U U U U

U U U U


Qualitative research U U

U U

U U U U U U

U U U U U

U U U U

U

U U U U U U U U U U U U U U U

U

Quality improvement studies

U U

U

U U U U

U

U U

U

U U

U

U U U

U U U U U U U U U U U

Study protocols


IJEBH-D-17-00013

MT Sims et al.




IJEBH-D-17-00013

ORIGINAL RESEARCH Table 2. Reporting guidelines by study type Animal research

Animal Research: Reporting of In Vivo Experiments (ARRIVE) Guidelines

Case reports

Case Reports (CARE) Guidelines

Clinical trials

Consolidated Standards of Reporting Trials (CONSORT) Statement

Diagnostic accuracy studies

Standards for Reporting Diagnostic Accuracy Studies (STARD) Transparent Reporting of a Multivariate Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) Statement

Economic evaluations

Consolidated Health Economic Evaluation Reporting Standards (CHEERS) Statement

Observational studies in epidemiology (Meta-Analysis)

Meta-Analysis of Observational Studies in Epidemiology (MOOSE) Statement Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement

Qualitative research

Consolidated Criteria for Reporting Qualitative Research (COREQ) Checklist Standards for Reporting Qualitative Research (SRQR)

Quality improvement studies

Standards for Quality Improvement Reporting Excellence (SQUIRE) Checklist

Study protocols

Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) Statement Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Statement

Systematic reviews and meta-analyses

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement Quality of Reporting of Meta-Analyses (QUOROM) Checklist

and review registries, when provided, were also extracted. Statements that were extracted in a language other than English (e.g., German and Spanish) were translated both by Google Translate (Google Inc.) and native speakers who were biomedical faculty members of our university. Following data extraction, M.T.S. and C.W. examined each extracted statement and, by using a Google Form, rated whether the journal required, recommended, or failed to mention each reporting guideline. Statements about trial registration were rated the same way. During the pilot session, it was agreed that words or phrases such as ‘should’, ‘prefer’, ‘encourage’, or ‘in accordance to the recommendation of’ would be rated as recommended, whereas words or phrases such as ‘must’, ‘need’, or ‘manuscripts won’t be considered for publication unless’ would be regarded as required. If M.T.S. and C.W. were unable to determine whether a journal required or recommended a particular guideline, then the statement was marked as unclear. M.T.S. and C.W. were blinded to each other’s ratings. M.T.S. and C.W. met after completing the rating process to compare scores and resolve any disagreements. Cross-tabulations and descriptive statistics were calculated by using STATA 13 (StataCorp LP, College Station, Texas, USA). If a journal did not publish a particular type of study, then it was not considered when computing proportions. For example, if a journal did not publish preclinical animal studies, then the Animal Research: Reporting of In Vivo Experiments (ARRIVE) Guidelines were not relevant to that journal.

Results The impact factors of these journals in our sample ranged from 0.325 to 15.328 [x¯ ¼ 3.521 (SD ¼ 3.346)]. Editorial offices were located in North America (21/37, 56.8%), Europe (6/37, 16.2%), the United Kingdom (7/37, 18.9%), Australia (2/37, 5.4%), and India (1/37, 2.7%). For each study type, the appropriate guideline was identified (Table 2). Following a review of instructions for authors and editor-in-chief email inquiries (RR ¼ 20/37, 54.1%), the following reporting guidelines were removed from computing proportions as their study type was not accepted by the journal: Standards for Reporting Diagnostic Accuracy Studies (1/37, 2.7%), Transparent Reporting of a Multivariate Prediction Model for Individual Prognosis or Diagnosis (1/37, 2.7%), Meta-Analysis of Observational Studies in Epidemiology (1/37, 2.7%), STROBE (1/37, 2.7%), Consolidated Criteria for Reporting Qualitative Research (1/37, 2.7%), Standards for Reporting Qualitative Research (SRQR) (1/37, 2.7%), Standards for Quality Improvement Reporting Excellence (3/37, 8.1%), Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (9/37, 24.3%), and Standard Protocol Items: Recommendations for Interventional Trials (9/37, 24.3%) (Table 3).

Reporting guidelines The instructions for authors of nine (9/37, 24.3%) journals referenced the EQUATOR Network. The authors’ guidelines of 26 (26/37, 70.3%) journals referenced the ICMJE Uniform Requirements for Manuscripts. Of the 37 critical


5


15.328 13.118 10.125 7.422 6.136 5.414 4.950 4.529 4.377 3.317 3.262 3.048 2.889 2.802 2.706 2.659 2.488 2.445 2.274 2.055 2.053 2.036 1.922 1.910 1.904 1.782 (1.492) 1.479 1.415 1.333 1.283 (1.214) 1.193 0.556 0.325 N/A N/A

Journal

Geographical Zone UK NA NA NA NA EU UK EU NA AU NA NA NA NA NA NA NA NA NA EU NA EU NA UK UK NA UK NA NA NA AU UK EU NA EU UK IN

Equator Network Yes No Yes No Yes Yes Yes Yes No No No No No Yes No No No No No No No No No No No No Yes Yes No No No No No No No No No

ICMJE Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes No No Yes No Yes Yes Yes No No No Yes Yes No Yes No Yes Yes No Yes No

ARRIVE U

U

U

U U

U

U

U U

CARE

CONSORT U

U

U U

U

U

U

U U U U

U

STARD U

U

U U

TRIPOD U U

U U

U

STROBE U

U

U U

U

COREQ U

U U

SQUIRE U U

PRISMA-P U U

SPIRIT U U

PRISMA U

U

U

U U U U

ClinicalTrials.gov ?

U

U

WHO U U

U

Registry

?

PROSPERO U

U U

U

U U

Other/Generic (Review)

QUOROM

SRQR

¼ required, U ¼ recommended, ? ¼ unclear, box filled with gray ¼ journal does not accept the study design requiring these guidelines, box filled with red ¼ journal does not accept the study design yet the instructions for authors mention the guideline, ¼ editor-in-chief did not respond to email inquiry. (Impact factor), impact factor reported via the journal itself; AU, Australia; EU, Europe (excluding the United Kingdom); IN, India; N/A, no impact factor available; NA, North America (United States and Canada); UK, United Kingdom.

Impact Factor

Lancet Respiratory Medicine American Journal of Respiratory and Critical Care Medicine Intensive Care Medicine Critical Care Medicine Chest Resuscitation Critical Care Annals of Intensive Care Journal of Neurotrauma Critical Care and Resuscitation Journal of Intensive Care Medicine Shock Human Gene Therapy Clinical Development Journal of Trauma and Acute Care Surgery Current Opinion in Critical Care Pediatric Critical Care Medicine Neurocritical Care Journal of Critical Care Seminars in Respiratory and Critical Care Medicine Anasthesiologie und Intensivmedizin American Journal of Critical Care Minerva Anestesiologica Respiratory Care Injury-International Journal of the Care of the Injured Burns Critical Care Clinics Nursing in Critical Care Australian Critical Care Therapeutic Hypothermia and Temperature Management Critical Care Nurse Anaesthesia and Intensive Care Intensive and Critical Care Nursing Medicina Intensiva Journal of Trauma Nursing Anasthesiologie Intensivmedizin Notfallmedizin Schmerztherapie Critical Care Research and Practice Indian Journal of Critical Care Medicine

CHEERS

Guidelines

MOOSE

6 Other/Generic (Trial)

Table 3. Use of reporting guidelines and study registration by journal


IJEBH-D-17-00013

MT Sims et al.




IJEBH-D-17-00013

ORIGINAL RESEARCH Clinical trial and systematic review registration Of the 37 critical care journals, 21 (21/37, 56.8%) did not mention trial or review registration, whereas the remaining 16 (16/37, 43.2%) at least mentioned one of the two. Trial registration through ClinicalTrials.gov was mentioned by six (6/37, 16.2%) journals: it was required by three (3/6, 50.0%) journals and recommended by two (2/ 6, 33.3%) journals, and one (1/6, 16.7%) journal’s statement was unclear. Registration through WHO was mentioned by five (5/37, 13.5%) journals: it was required by two (2/5, 40.0%) journals and recommended by three (3/ 5, 60.0%) journals. Fifteen (15/37, 40.5%) journals required trial registration but did not specify the specific trial registry. Systematic review registration was recommended by three (3/37, 8.1%) journals with the PROSPERO registry mentioned most frequently (Fig. 3).

care journals, 15 (15/37, 40.5%) did not mention a single guideline within their instructions to authors, whereas the remaining 22 (22/37, 59.5%) mentioned one or more guidelines. Across reporting guidelines, the CONSORT statement (17/37, 45.9%) was most frequently required (5/17, 29.4%) or recommended (12/17, 70.6%) by journals in our sample (Table 3). The PRISMA guidelines (9/37, 24.3%) were the second most frequently required or recommended followed by the ARRIVE guidelines (7/37, 18.9%). The Quality of Reporting of Meta-analyses statement and SRQR were not mentioned by any journals (Fig. 2). Some journals mentioned various reporting guidelines in their instructions for authors’ information; however, the editor-in-chiefs of these journals corresponded that their journal did not accept the type of study for which the guideline was intended. These discrepancies are as follows: The ARRIVE guidelines were mentioned by seven journals: two (2/7, 28.6%) required adherence and five (5/7, 71.4%) recommended adherence. The CARE guidelines were mentioned by four journals: all (4/4, 100.0%) journals recommended adherence. The CONSORT statement was mentioned by 17 journals: five (5/ 17, 29.4%) required adherence and 12 (12/17, 70.6%) recommended adherence. The Consolidated Health Economic Evaluation Reporting Standards checklist was mentioned by two journals: all (2/2, 100.0%) journals recommended adherence (Table 3).

Discussion We evaluated reporting guidelines adherence and clinical trial registration of critical care journals. We found that nearly half did not mention a single reporting guideline. The 2001 CONSORT revision states, ‘. . . inadequate reporting borders on unethical practice when biased results receive false credibility’.29 The case could be made that researchers and reviewers have the moral and ethical duty to report their findings as openly as possible to provide readers with the ability to accurately judge the validity of the results.16,29 Inadequate

ARRIVE CARE CONSORT STARD TRIPOD

Guidelines

CHEERS MOOSE STROBE COREQ SRQR SQUIRE PRISMA-P SPIRIT PRISMA QUOROM 0

3

6

9 Required

12

15

18

Recommended

21

24

27

30

33

36

Not mentioned

Figure 2. Frequency of reporting guideline adherence across journals. Each guideline reviewed within this study and its level of adherence is displayed in this figure. In the scenario that a study type was not accepted by journal, the associated guideline was removed when computing proportions. International Journal of Evidence-Based Healthcare ß 2017 University of Adelaide, Joanna Briggs Institute

7



IJEBH-D-17-00013

MT Sims et al.

Registry platform

Other/Generic (Review)

PROSPERO

Other/Generic (Trial)

WHO

ClinicalTrials.gov 0 Required

3

6

9

Recommended

12

15

18

Unclear

21

24

27

30

33

36

Not mentioned

Figure 3. Frequency of registration adherence across journals. Each type of trial and review registration platform is depicted in this figure.

reporting hinders the readers’ ability to judge the reliability of results, subsequently preventing their accurate interpretation.4 Reporting guidelines help researchers compose their work in a fashion that will maximize the value of the findings to others. This transparency is a focal point of well-executed scientific research; without it, authors may be blinding their readers, which could impair scientific inquiry.28–30 Some evidence supports the effectiveness of reporting guidelines on the completeness of reporting.30 – 35 For instance, following a change in the International Journal of Surgery’s policy on reporting guideline adherence, a follow-up study indicated an increased adherence to CONSORT (50–70%), PRISMA (48–76%), and STROBE (12%).31 A Cochrane review on the effects of journal endorsement of CONSORT in relation to completeness of reporting found that 69 meta-analyses showed a relative benefit from CONSORT endorsement on completeness of reporting.30 In gastroenterology and hepatology journals endorsing PRISMA, improvements in the completeness of reporting were also found.32 A systematic review that collated studies investigating journals’ endorsement of reporting guidelines and the completeness of reporting of their published reports concluded that there was insufficient evidence to make a conclusive judgment about this relationship.33 Another study found no evidence of improvement in an analysis of reporting completeness prior to and after publication of the STROBE guidelines.34 The higher rates of adherence to specific guidelines, such as CONSORT and PRISMA, may be partially explained by their greater familiarity in the research community.35 This greater familiarity may account for researchers more frequently consulting these guidelines when planning, conducting, and reporting studies.35 8

Our study also examined trial registration requirements among critical care journals. Only a small percentage of journals in our sample require clinical trial registration as a condition for publication. Recently, medical journal editors have expressed firm conviction that requiring trial registration is the single most valuable method for ensuring unbiased research reporting36 as it allows for the identification of outcome bias or other deviations from the study protocol.36 In 2007, clinical trial registration became a requirement of investigators in accordance with US law [Public law 110–85, Title VIII, also called the FDA Amendments Act (FDAAA) of 2007, Section 801]. The FDAAA requires that all phases 2–4, US-based studies evaluating FDA-approved drugs, devices, or biologics be registered prior to study commencement; it also requires results to be reported within 1 year of trial completion.37,38 However, a significant number of researchers and journal editors have failed to fully endorse trial registration.39–41 This is perhaps due to the increased demands on journal resources, as journals requiring trial registration would need to carefully regulate this policy. Prayle et al.42 found that only 22% of studies adhered to the FDAAA’s requirements. In addition, Mathieu et al.43 reported that only 45.5% of studies were properly registered before trial completion, and one-quarter of published studies were never registered despite US legislation. Overall, many published randomized trials are retrospectively registered or unregistered, and rates of registration vary drastically between medical specialties.43–53 Improving these practices may depend on developing and adhering to policies that require registration of a trial prior to journal submission. Our study revealed that the majority of critical care journals have no such policies. Physical medicine and rehabilitation journals have formed a collaboration to ensure that high-quality research reporting standards are created and upheld.54




IJEBH-D-17-00013

ORIGINAL RESEARCH As of 2014, 28 major journals from this discipline have collaborated to require adherence to reporting guidelines to enhance research reporting quality in each journal and in the field as a whole. Given that many critical care journals do not promote reporting guidelines, it would be reasonable for critical care journal editors to consider forming such collaborations, especially if such collaborations are shown to improve the quality of research reporting. Many of the questions surrounding reporting guideline usage are centered on the factors that support or deter researchers’ and editors’ use of reporting guidelines. In a recent survey of 116 health research journals, Hirst and Altman55 identified three reasons for not using reporting guidelines: a lack of awareness of their existence, uncertainty about how they can improve research practices, and uncertainty about how reporting guidelines should be incorporated in the peer review process. In a more recent study, a combination of individual, professional culture, environmental, and practical factors accounted for authors adhering to reporting guidelines.55,56 When an author or editor had multiple factors contributing to their use of reporting guidelines, they incorporated reporting guidelines despite any perceived disadvantages.56 Research indicates that editors of highly reputable journals should require reporting guideline adherence at the time of submission to promote more completeness of reporting throughout the field.56 Fuller et al.56 suggested that if authors perceive adherence to reporting guidelines as increasing their potential for publishing in a highly reputable journal within their field, it will ultimately promote the completeness of reporting.

Study limitations In our study, we sought to contact editors to obtain information when the instructions for authors were unclear. Some did not respond to our inquiries and we were not able to verify or clarify whether these journals published certain types of study designs. Conclusion Critical care journals infrequently require or recommend reporting guidelines. In addition, few journals require clinical trial registration. Implementing these two mechanisms could limit bias and journal editors in critical care should consider adopting them. As a step toward adoption, critical care journals might recommend rather than require these mechanisms. Additional research is needed to determine the effectiveness of these mechanisms and to examine the future role that reporting guidelines and clinical trial registration could play in writing, peer review, and the publication process.

Acknowledgements We would like to thank Dr. Gerwald A. Ko¨hler, PhD and Dr. Lea Figueroa-Hall, PhD for their assistance in accurately translating journal adherence statements into foreign languages. Ethics approval and consent to participate: The current study did not meet the regulatory definition of human patient research as defined in 45 CFR 46.102(d) and (f) of the Department of Health and Human Services’ Code of Federal Regulations and therefore, was not subject to Institutional Review Board oversight. Consent for publication: Not applicable. Availability of data and material: Data from this study is publically available on figshare (https://doi.org/10.6084/ m9.figshare.4517033.v4). Funding: This project was not funded. Author’s contributions: M.T.S., C.W., and M.V. made substantial contributions to the conception and design of the work and the acquisition, analysis, and interpretation of data for the work. M.T.S., J.X.C., C.W., and M.V. contributed to drafting the work and revising it critically for important intellectual content. M.T.S., J.X.C., C.W., and M.V. approved the final version to be published. M.T.S., J.X.C., C.W., and M.V. agreed to be accountable for all aspects of the work to ensure that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Conflicts of interest There are no conflicts of interest.

References 1. Moher D, Weeks L, Ocampo M, et al. Describing reporting guidelines for health research: a systematic review. J Clin Epidemiol 2011; 64: 718–42. 2. Simera I, Moher D, Hoey J, et al. The EQUATOR Network and reporting guidelines: helping to achieve high standards in reporting health research studies. Maturitas 2009; 63: 4–6. 3. Latronico N, Metelli M, Turin M, et al. Quality of reporting of randomized controlled trials published in Intensive Care Medicine from 2001 to 2010. Intensive Care Med 2013; 39: 1386–95. 4. Simera I, Moher D, Hirst A, et al. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Med 2010; 8: 24. 5. Bonati M, Pandolfini C. Trial registration, the ICMJE statement, and paediatric journals. Arch Dis Child 2006; 91: 93. 6. Rivara FP. Registration of clinical trials. Arch Pediatr Adolesc Med 2005; 159: 685. 7. International Committee of Medical Journal Editors (ICMJE). Clinical Trial Registration. http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/clinical-trialregistration.html. [Accessed 18 March 2016].


9



IJEBH-D-17-00013

MT Sims et al. 8. ClinicalTrials.gov. Why should I register and submit results? Available at https://clinicaltrials.gov/ct2/manage-recs/ background. [Accessed 16 November 2016]. 9. Food and Drug Administration. Public law 110–85-Sept 27, 2007. 121 STAT 823–978. 2007. Available at https://www.gpo. gov/fdsys/pkg/PLAW-110publ85/pdf/PLAW-110publ85.pdf. [Accessed 3 January 2017]. 10. World Health Organization (WHO); International Clinical Trials Registry Platform (ICTRP). WHO Statement on public disclosure of clinical trial results. 2015. Available at http:// www.who.int/ictrp/results/WHO_Statement_results_reporting_clinical_trials.pdf?ua=1. [Accessed 4 January 2017]. 11. Regulation (EU) No 536/2014 of the European Parliament and of the Council of 16 April 2014 on clinical trials on medicinal products for human use, and repealing Directive 2001/20/ED, Stat. 536/2014. 2014. Available at http://ec. europa.eu/health//sites/health/files/files/eudralex/vol-1/reg_ 2014_536/reg_2014_536_en.pdf. [Accessed 3 January 2017]. 12. Lang TA, Altman DG. Basic statistical reporting for articles published in biomedical journals: the ‘Statistical Analyses and Methods in the Published Literature’ or the ‘SAMPL Guidelines’. In: Smart P, Maisonneuve H, Polderman A, editors. Science editors’ handbook. Oxford: European Association of Science Editors, 2013. 13. Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research Evid Based Med 2017; 22: 132–42. 14. Onishi A, Furukawa TA. Publication bias is underreported in systematic reviews published in high-impact-factor journals: metaepidemiologic study. J Clin Epidemiol 2014; 67: 1320–6. 15. Bonnot B, Yavchitz A, Mantz J, et al. Selective primary outcome reporting in high-impact journals of anaesthesia and pain. Br J Anaesth 2016; 117: 542–3. 16. Helfer B, Prosser A, Samara MT, et al. Recent meta-analyses neglect previous systematic reviews and meta-analyses about the same topic: a systematic examination. BMC Med 2015; 13: 82. 17. Uhlig C, Krause H, Koch T, et al. Anesthesia and monitoring in small laboratory mammals used in anesthesiology, respiratory and critical care research: a systematic review on the current reporting in top-10 impact factor ranked journals. PLoS One 2015; 10: e0134205. 18. Tao K-M, Li X-Q, Zhou Q-H, et al. From QUOROM to PRISMA: a survey of high-impact medical journals’ instructions to authors and a review of systematic reviews in anesthesia literature. PLoS One 2011; 6: e27611. 19. Tunis AS, McInnes MD, Hanna R, Esmail K. Association of study quality with completeness of reporting: have completeness of reporting and quality of systematic reviews and meta-analyses in major radiology journals changed since publication of the PRISMA statement? Radiology 2013; 269: 413–26. 20. Bigna JJ, Um LN, Nansseu JR. A comparison of quality of abstracts of systematic reviews including meta-analysis of randomized controlled trials in high-impact general

10

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

34.

medicine journals before and after the publication of PRISMA extension for abstracts: a systematic review and meta-analysis. Syst Rev 2016; 5: 174. Meerpohl JJ, Wolff RF, Niemeyer CM, et al. Editorial policies of pediatric journals: survey of instructions for authors. Arch Pediatr Adolesc Med 2010; 164: 268–72. Smith TA, Kulatilake P, Brown LJ, et al. Do surgery journals insist on reporting by CONSORT and PRISMA? A follow-up survey of ‘instructions to authors’. Ann Med Surg (Lond) 2015; 4: 17–21. Moher D, Schulz KF, Altman DG. CONSORT. The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials. BMC Med Res Methodol 2001; 1: 2. Delgado-Lo´pez-Co´zar E, Cabezas-Clavijo A´. Ranking journals: could Google Scholar Metrics be an alternative to Journal Citation Reports and Scimago Journal Rank? Learn Publ 2013; 26: 101–14. Delgado ERR. The impact of scientific journals of communication: comparing Google Scholar Metrics, Web of Science and Scopus. Comunicar 2013; 21: 45–52. Truex D, Cuellar M, Takeda H. Assessing scholarly influence: using the Hirsch indices to reframe the discourse. J Assoc Inf Syst 2009; 10: 560–94. Mingers J, Macri F, Petrovici D. Using the h-index to measure the quality of journals in the field of business and management. Inf Process Manag 2012; 48: 234–41. Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys: the tailored design method. Hoboken: John Wiley & Sons; 2014. Altman DG, Simera I. Using reporting guidelines effectively to ensure good reporting of health research (chapter 4). In: Moher D, Altman D, Schulz K, Simera I, Wager E, editors. Guidelines for reporting health research: a user’s manual. Hoboken, New Jersey: John Wiley & Sons, Ltd, 2014. Turner L, Shamseer L, Altman DG, et al. Does the CONSORT Statement influence the quality of reporting of RCTs published in medical journals? Cochrane Database Syst Rev 2012; 11: MR000030. Agha RA, Fowler AJ, Limb C, et al. Impact of the mandatory implementation of reporting guidelines on reporting quality in a surgical journal: a before and after study. Int J Surg 2016; 30: 169–72. Panic N, Leoncini E, de Belvis G, et al. Evaluation of the endorsement of the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement on the quality of published systematic review and meta-analyses. PLoS One 2013; 8: e83138. Stevens A, Shamseer L, Weinstein E, et al. Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: systematic review. BMJ 2014; 348: g3804. Bastuji-Garin S, Sbidian E, Gaudy-Marqueste C, et al. Impact of STROBE statement publication on quality of observational study reporting: interrupted time series versus before-after analysis. PLoS One 2013; 8: e64733.




IJEBH-D-17-00013

ORIGINAL RESEARCH 35. Fleming PS, Koletsi D, Pandis N. Blinded by PRISMA: are systematic reviewers focusing on PRISMA and ignoring other guidelines? PLoS One 2014; 9: e96407. 36. Weber WEJ, Merino JG, Loder E. Trial registration 10 years on. BMJ 2015; 351: h3572. 37. National Institute of Health (NIH); National Institute of Neurological Disorders and Stroke. ClinicalTrials.gov Registration and Reporting. Modified on September 18, 2015. Available at http://www.ninds.nih.gov/research/clinical_ research/basics/clinicaltrials_required_registration.htm. [Accessed 18 September 2016]. 38. ClinicalTrials.gov. FDAAA 801 Requirements. Modified on November 1, 2015. Available at https://clinicaltrials.gov/ct2/ manage-recs/fdaaa. [Accessed 26 May 2016]. 39. Malicki M, Marusic A. OPEN (to Overcome failure to Publish nEgative fiNdings) Consortium. Is there a solution to publication bias? Researchers call for changes in dissemination of clinical research results. J Clin Epidemiol 2014; 67: 1103–10. 40. Reveiz L, Krleza-Jeric´ K, Chan A-W, de Aguiar S. Do trialists endorse clinical trial registration? Survey of a PubMed sample. Trials 2007; 8: 30. 41. Wager E, Williams P. Project Overcome failure to Publish nEgative fiNdings Consortium. ‘Hardly worth the effort’? Medical journals’ policies and their editors’ and publishers’ views on trial registration and publication bias: quantitative and qualitative study. BMJ 2013; 347: f5248. 42. Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ 2012; 344: d7373. 43. Mathieu S, Boutron I, Moher D, et al. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA 2009; 302: 977–84. 44. van de Wetering FT, Scholten RJPM, Haring T, et al. Trial registration numbers are underreported in biomedical publications. PLoS One 2012; 7: e49599. 45. Hardt JLS, Metzendorf M-I, Meerpohl JJ. Surgical trials and trial registers: a cross-sectional study of randomized controlled trials published in journals requiring trial registration in the author instructions. Trials 2013; 14: 407.

46. Killeen S, Sourallous P, Hunter IA, et al. Registration rates, adequacy of registration, and a comparison of registered and published primary outcomes in randomized controlled trials published in surgery journals. Ann Surg 2014; 259: 193–6. 47 Hooft L, Korevaar DA, Molenaar N, et al. Endorsement of ICMJE’s clinical trial registration policy: a survey among journal editors. Neth J Med 2014; 72: 349–55. 48. Anand V, Scales DC, Parshuram CS, Kavanagh BP. Registration and design alterations of clinical trials in critical care: a cross-sectional observational study. Intensive Care Med 2014; 40: 700–22. 49. Kunath F, Grobe HR, Keck B, et al. Do urology journals enforce trial registration? A cross-sectional study of published trials. BMJ Open 2011; 1: e000430. 50. Mann E, Nguyen N, Fleischer S, Meyer G. Compliance with trial registration in five core journals of clinical geriatrics: a survey of original publications on randomised controlled trials from 2008 to 2012. Age Ageing 2014; 43: 872–6. 51. Huser V, Cimino JJ. Evaluating adherence to the International Committee of Medical Journal Editors’ policy of mandatory, timely clinical trial registration. J Am Med Inform Assoc 2013; 20: e169–74. 52. Wiebe J, Detten G, Scheckel C, et al. The heart of the matter: outcome reporting bias and registration status in cardiothoracic surgery. Int J Cardiol 2017; 227: 299–304. 53. Sims MT, Sanchez ZC, Herrington JM, et al. Shoulder arthroplasty trials are infrequently registered: a systematic review of trials. PLoS One 2016; 11: e0164984. 54. The EQUATOR Network. Collaborative initiative involving 28 rehabilitation and disability journals. Oxford: The Equator Network; 2016. Available at http://www.equator-network. org/2014/04/09/collaborative-initiative-involving-28-rehabilitationand-disability-journals/. [Accessed 26 July 2016]. 55. Hirst A, Altman DG. Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals. PLoS One 2012; 7: e35621. 56. Fuller T, Pearson M, Peters J, Anderson R. What affects authors’ and editors’ use of reporting guidelines? Findings from an online survey and qualitative interviews. PLoS One 2015; 10: e0121585.


11


Upgrading instructions for authors of scholarly journals.

Pediatric case reports: assessing recommendations from journals' instructions to authors.

Research Reporting Guidelines and the New Annals Instructions for Authors.

Do surgery journals insist on reporting by CONSORT and PRISMA? A follow-up survey of 'instructions to authors'.

Clinical trial registration in oral health journals.

EVALUATION OF ETHICAL IN INSTRUCTIONS TO AUTHORS OF BRAZILIAN SURGICAL JOURNALS.

Requirements of Clinical Journals for Authors' Disclosure of Financial and Non-Financial Conflicts of Interest: A Cross Sectional Study.

The Content of Statistical Requirements for Authors in Biomedical Research Journals.

Relation of completeness of reporting of health research to journals' endorsement of reporting guidelines: systematic review.

The explicit mentioning of reporting guidelines in urogynecology journals in 2013: A bibliometric study.

Reporting guidelines: how can they be implemented by veterinary journals?

Do Croatian open access journals support ethical research? Content analysis of instructions to authors.

Update on the endorsement of CONSORT by high impact factor journals: a survey of journal "Instructions to Authors" in 2014.

Endorsement of the CONSORT guidelines, trial registration, and the quality of reporting randomised controlled trials in leading nursing journals: A cross-sectional analysis.

Quality of reporting in infertility journals.

Are Scholarly Journals in Search for Top Quality Articles or Authors in Search for High Quality Journals?

Guidance in author instructions of hematology and oncology journals: A cross sectional and longitudinal study.

Response to letter to the editor, "Clinical trial registration in oral health journals".

Requirements of health policy and services journals for authors to disclose financial and non-financial conflicts of interest: a cross-sectional study.

Letter to the editor, "Clinical trial registration in oral health journals".

CONSORT item reporting quality in the top ten ranked journals of critical care medicine in 2011: a retrospective analysis.

Journals for developing countries.

Journals for developing countries.

Critical appraisal of reporting randomized clinical trials published in Iranian dental journals during 2003-2010.