Evaluation of observational research reports published in Turkish nursing journals Z. Karaçam1 PhD, E. S¸en2 PhD & B. Yildirim3 PhD 1 Professor, 3 Assistant Professor, Aydın School of Health, 2 Assistant Professor, Söke School of Health, Adnan Menderes University, Aydın, Turkey
KARAÇAM Z., S¸EN E. & YILDIRIM B. (2015) Evaluation of observational research reports published in Turkish nursing journals. International Nursing Review 62, 394–403 Aim: The aim of this literature-based descriptive study was to examine the reporting of the observational research studies published in peer-reviewed nursing journals in Turkey. Methods: Eleven peer-reviewed nursing journals printed on a regular basis in Turkey between 2007 and 2012 were selected. These journals were searched for observational research studies, and 502 studies were selected and examined by using the Strengthening the Reporting of Observational Studies in Epidemiology Statement. Results: Of the 502 studies, 495 were cross-sectional, 3 were cohort, and 4 were case controlled. Summary and introduction and aim sections were sufficient in most of the studies. The methods sections of the reports were mostly not reported: 64.3% of the reports did not indicate eligibility/inclusion criteria; sampling method, 67.0%; possible sources of bias, 99.2%; ways to reach sample size, 92.6%. In the results section, the number of individuals participating in each stage of the studies (44.0%) and in other analyses made (39.2%) was not reported. In the discussion section, a main comment about research findings was partly made (97.4%), and limitations of the studies and possible sources of bias were not written in 99.0% of the studies. Conclusions: This study clearly revealed that the observational research studies published in nursing journals in Turkey did not fulfil the important criteria and needed to be improved. Implications for nursing and health policy: Information obtained from this study can contribute to improvement of the quality of reporting observational studies in nursing and thus using obtained findings in practice. Keywords: Journal Article, Nursing, Nursing Research, Observational Study, Turkey
Introduction Solutions for many health challenges are sought in observational research studies, accounting for an important portion of Correspondence address: Dr Zekiye Karaçam, Aydın Sag˘lık Yüksekokulu, Adnan Menderes Üniversitesi, Aydın 09100, Turkey; Tel: (90)-256-2148041; Fax: (90)-256-2124219; E-mail: [email protected]; [email protected].
Source of funding statement No grants were obtained in relation to this study. Conflict of interest No conflict of interest has been declared by the authors.
© 2015 International Council of Nurses
studies about positive and negative effects of medical interventions. Many studies about aetiology of diseases are cohort, case-controlled and cross-sectional observational studies. Experimental studies may not provide answers to all questions about a given intervention. Conducting observational studies could be more appropriate in investigating rare and long-term side effects of a treatment modality and in revealing data about medical practices implemented daily. However, only a limited amount of information acquired from observational studies is reported due to deficiencies in the reporting (von Elm et al. 2007; Simera et al. 2012).
394
bs_bs_banner
Nursing Research
Evaluation of observational research reports
A very well-written report reveals the strengths and weaknesses of a study and facilitates interpretation and implementation of research findings (Vandenbroucke et al. 2007). Reliability of a study depends on a critical evaluation of its design and conduction and the strengths and weaknesses of analyses by readers. Therefore, authors should present sufficient information clearly so that readers can make critical evaluations of what was planned and conducted, new information obtained and reliability of findings. The readers should also be able to understand the conclusions made. A poorly written research report restricts making use of important findings in clinical practice and further research and causes loss of valuable information. In addition, poor reporting creates an ethical challenge about how the findings were obtained and whether the study was really necessary if a report does not include this piece of information (Altman & Simera 2010; von Elm et al. 2007; Simera et al. 2012). In recent years, undertaking systematic reviews has led to the development of clear quality criteria to assess research studies. These criteria might reveal many deficiencies which affect the quality of publications in terms of the possibility of bias, presentation of findings in summaries and throughout manuscripts, original study protocols, details of interventions, inclusion and exclusion criteria and missing data about negative effects of interventions (Altman & Simera 2010; Simera et al. 2012). These deficiencies in the reporting of studies make difficult the evaluation of the quality of studies, decrease credibility of presented findings and make them hard to use in creating evidence and developing clinical guides. As a result, the findings cannot be beneficial in patient care. In addition, selectively reporting positive findings about a given treatment may lead to overdependence on that treatment, which may result in high health costs without improving patient care outcomes. Moreover, overemphasis on positive effects of a treatment alternative or inability to present all findings may cause harm to patients who will be included in further studies. Insufficient presentation or not presenting harmful effects of a given alternative at all, especially in experimental studies, is a serious challenge and an unethical practice (Glasziou et al. 2008; Ioannidis 2009; Simera et al. 2010; Song et al. 2010). Studies in which research reports about health-related topics have been examined have shown serious deficiencies including insufficient reporting of study designs and statistical analyses (Rangel et al. 2003). This includes not reporting the logic behind selection of possible confounding factors (Pocock et al. 2004), not explaining methods used to determine study and control groups (Lee et al. 2007), and not mentioning inclusion/ exclusion criteria (Tooth et al. 2005). In addition, examination of research reports might also reveal some discrepancies such as
© 2015 International Council of Nurses
395
including late or not reporting negative findings, selectively reporting positive findings or conversely changing findings in accordance with study protocols, not writing important information about methods and treatments, and offering insufficient information or misinterpreting findings in summaries and insufficient or misleading reports of side effects (Altman & Simera 2010). In one study, 300 observational studies published between 1997 and 2002 were shown to fulfil fewer than half of the criteria for writing research reports (Rangel et al. 2003). The term observational study describes a wide range of study designs including prospective and retrospective cohort studies, case-control studies and cross-sectional studies, a defining feature of which is that any intervention studied is determined by clinical practice and not by the protocol. Data from large, prospective observational studies provide information about the safety and efficacy of medicines in daily clinical use. Such observational studies are generally carried out once the researchers have received approval from regulatory agencies. Observational trials have inherent limitations in terms of their susceptibility to bias and confounding factors, restricting their ability to define causality. However, their strengths include their ability to reflect daily clinical practice more closely than randomized controlled trials (RCTs), both in terms of the heterogeneous patient populations that are included, and the medical interventions that they receive. Therefore, observational trials can provide clinically relevant information that is not necessarily provided by RCTs. Given the limitations of an observational study approach, it is important to optimize study designs to maximize the validity, and thus in particular known causes of bias and confounding factors should be measured (Home 2010; Norris et al. 2011; Yang et al. 2010). An important role for editors is to ensure that research articles published in their journals are clear, complete, transparent and as free as possible from bias. Medical investigators also have a direct or indirect contractual responsibility to the funder of their study to ensure that they accurately report safety and efficacy in their research outcomes according to agreed procedures. In an effort to uphold high standards, journal editors may feel the need to endorse multiple reporting guidelines without knowledge of their rigour or ability to improve reporting. The CONSORT (Consolidated Standards of Reporting Trials) statement is a well-known reporting guideline that has been extensively evaluated (Moher et al. 2010). In addition, any publications or presentations of the results should be within guidelines for appropriate reporting of observational trials, such as those published by the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement. Observational studies give the medical community an opportunity to share latest clinical data/practice and to report insights
396
Z. Karaçam et al.
from their own clinical practice (Stevens et al. 2014; Yang et al. 2010). Most of the articles (80.7–84.5%) published in Turkish nursing journals are observational studies (Çalıs¸ır et al. 2013; Ünsal & Sökmen 2011). Based on the experience of our individual Turkish nursing journal editors and reviewers, it is also known that there are important challenges with preparation of studies for publication in both national and international journals in Turkey. However, there has not been any research on this subject, the types and severity of these challenges are not known, and research-based information about them is required. In this context, an examination of observational studies in nursing based on the STROBE Statements Checklist of items should be included in reports of observational studies which may help reveal what problems exist and determine what sections of reports one should focus on. Therefore, the aim of this study was to examine the observational research reports published in nursing journals in Turkey. We examined only nursing research done in Turkey and published in Turkish nursing journals. It is anticipated that information obtained from this study will contribute to the improvement of the quality of reporting observational studies in nursing and thus using the obtained findings in practice.
Methods A literature-based descriptive study was performed on publications included in nursing journals in Turkey from 2007 to 2012. The study was carried out between January 2013 and July 2013. The articles included were based on quantitative methods mainly reported observational studies. Furthermore, in this study a quantitative approach was used. Journals available and selection of journals
All of nursing journals published in Turkey were available from the website of Koç University Suna Kıraç Library (http:// libguides.ku.edu.tr/content.php?pid=40495&sid=298033). There were 19 nursing journals published in Turkey. The study included 11 nursing journals having been published as peerreviewed journals since 2007 and are still being published. All articles of these journals were accessed through the Internet as full text. Selection of studies
Selection and recording of observational studies were carried out by two researchers independently. Eleven journals included were accessed, and cohort, case-controlled and cross-sectional studies were recorded. The studies, the designs of which could not be exactly determined, were also recorded. The studies determined independently by the researchers were compared in
© 2015 International Council of Nurses
one session, and disagreements about the studies were eliminated and an agreement was reached about the studies whose methods were not mentioned. All the studies included were listed and coded by naming them with sequential numbers and the first letters of the titles of the journals in which they were published. There were a total of 1045 articles published in 11 nursing journals between 2007 and 2012 and satisfying the inclusion criteria. Both quantitative and qualitative studies are published in these journals. Of these 1045 articles, 664 were research, 354 were traditional reviews, 15 were letters to editors, and 12 were case reports. Of the research articles, 502 were observational, 102 were experimental/quasi-experimental, 45 were methodological, 12 were qualitative, and 3 were systematic reviews. A total of 502 observational research reports were included in this study (Fig. 1). Inclusion and exclusion criteria
The study was performed on the observational research reports from peer-reviewed nursing journals in Turkey, and these reports were examined according to the STROBE Statements Checklist. Because observational research is used widely in Turkish nursing research (Çalıs¸ır et al. 2013; Ünsal & Sökmen 2011), the observational studies (cohort, case controlled, crosssectional and descriptive) with a quantitative design were
The number of nursing journals in total N = 19
The number of journals published regularly between 2007 and 2012 N = 11
The number of studies reported in journals published regularly between 2007 and 2012 N = 1045 Excluded studies 354 Traditional reviews 102 Experimental/quasi-experimental studies 45 Methodological studies 15 Letters to editors 12 Qualitative studies 12 Case reports 3 Systematic reviews The number of studies included in this study N = 502
Fig. 1 Flow chart for studies included in the study.
Evaluation of observational research reports
397
included. To limit the scope of the study, experimental and quasi-experimental studies, traditional reviews, systematic reviews, methodological studies, qualitative studies, case reports and letters to editors were excluded.
informed about the study, and their studies could not be included in the references. In addition, information about the authors of the examined studies was concealed.
Data extraction
Results
In the first stage, a Turkish version of the STROBE Statements Checklist, including items required in reporting observational studies, was developed. The validity and reliability of the Checklist was achieved by linguistic validity with the methods of group translation and expert opinion. In the second stage, a questionnaire about information available in the STROBE Statements Checklist was created by making use of the study by Langan et al. (2010) (Table 1). In the next stage, the reports to be included in the study were determined and examined according to the STROBE Statements Checklist. Then, to standardize the research assessment skills of researchers with the STROBE Statements Checklist, a pilot study was conducted in a session in which all the researchers in a journal not included in the main study participated. Differences of information and opinion of researchers were eliminated about how items of the STROBE Statements Checklist in the article were determined.
In this study, 502 observation research reports published in 11 nursing journals in Turkey between 2007 and 2012 were examined according to the STROBE Statements Checklist. Of these 502 studies, 98.6% were cross-sectional (n = 495/502), 0.6% were cohort (n = 3/502) and 0.8% were case controlled (n = 4/ 502). The examination of titles and abstracts of the studies revealed that study designs were mentioned in the summary sections in 79.1% (n = 397/502) of the reports and that sufficient information explaining what was investigated and what was found was evidenced in 81.1% (n = 407/502) of the reports (Table 1). All of the reports included a study design in their abstracts. Available scientific evidence and impetus for research were presented in all the reports. Aims of the studies were mentioned in 97.6% of the reports, and hypotheses or research questions were provided in 17.5% of the reports. Of these reports, 14 presented aims of the studies and 6 presented research questions and hypotheses in the methods sections (Table 1). The methods sections of the reports mostly included study designs (90.5%), settings (56.6%), time of data collection (78.9%), sources of data collected and data collection tools (96.0%), ways of analysing quantitative variables (92.0%) and methods of statistical analyses (92.2%). Out of all the study reports, 52 supplied study designs in their introduction sections. In a considerable proportion of the cross-sectional studies, 64.3 and 67.0% did not indicate the eligibility/inclusion criteria and the methods of selection of participants, respectively. Furthermore, attempts to show possible sources of bias and ways to reach sample sizes were 99.2 and 92.6% respectively incomplete in the majority of the studies. Up to 42.1% of the reports of the studies did not explain subgroup analyses and their interactions (Table 1). In the result sections of the reports, mostly characteristics of participants were presented (91.4%), and the results expressed in numbers or data obtained were summarized (cross-sectional studies: 97.4%). Few reports presented the number of participants included in each stage of the studies (37.1%), causes of declining to participate (0.8%; n = 4/502), the number of participants required for each variable (1.0%; n = 5/502), limits of groups when continuous variables were grouped (10.6%; n = 53/502) and other analyses made (10.4%; n = 52/502) (Table 1).
Examination of research reports
The research reports included in the study were examined by using a questionnaire developed based on a Turkish version of the STROBE Statements Checklist which includes 22 items. The examination of the research reports was performed independently by the two researchers to determine whether the reports fulfilled the criteria expressed in each question in the questionnaire. Then results of the examination made independently were compared. Relevant parts of the reports for which the researchers marked different choices were re-examined. An agreement was reached and a single examination form was filled in. Evaluation of data
Whether the reports included satisfied the criterion in each question in the questionnaire was determined by marking ‘yes’, ‘partly yes’, ‘no’, ‘unclear’ or ‘not applicable’. Obtained data were analysed with frequencies and percentages in a Statistical Package for the Social Sciences (SPSS) computer software program 15.0 (PASW Inc., Chicago, IL, USA). Ethical considerations
Extant information on this study was given to editors of the 11 journals by means of email. Because the total number of authors in this study was high (n = 502), they could not be
© 2015 International Council of Nurses
398
Z. Karaçam et al.
Table 1 Results of the examination of observational research reports according to the STROBE Statements Checklist (N = 502) Original STROBE Item No.
1
2 3
4 5
6
7
8
9 10 11
12
13
Questions prepared according to the STROBE Statements Checklist
Titles and abstract (a) Is study design expressed in the title or the abstract a commonly used term? (b) Is there sufficient explanatory information about what was done and what was found in the study in the abstract? Introduction Is the scientific background and rationale for the study reported? Are hypotheses or research questions reported? Is the aim of the study reported? Methods Are key parts related to the study design presented? Is the study setting reported? Are the dates during which data were collected reported? Are the dates of follow-up reported? (only used for three cohort studies) Are the eligibility/inclusion criteria given? Are the sources of participants given? Are the methods of selection explained? Are methods of follow-up described? (only used for three cohort studies) Are all variables explained clearly? Are all exposures, if appropriate, described? (only used for four case-controlled studies) Are all predictors, if appropriate, described? Are possible confounders, if appropriate, described? Are effect modifiers, if appropriate, described? Are diagnostic criteria, if appropriate, reported? Are sources of data for each variable and details about data collection tools reported? If there is more than one group, is comparability of data collection tools described? (Is whether methods of measurement are valid and reliable in differentiation of the groups reported?) Are attempts to show possible sources of bias described? Is how the sample size is reached (the statistical method used to determine the sample size) described? Is how quantitative variables are analysed explained? If appropriate, did which groupings were made for quantitative variables report? If appropriate, are the reasons for grouping for quantitative variables reported? Are all statistical tests used, including those used to control confounding factors, described? Are all methods used to investigate subgroups and their interactions explained? Is how missing data are dealt with explained? If appropriate, is how cases lost to follow-up dealt with explained? (only used for three cohort studies) Are sensitivity analyses described? Results Is the number of participants included in each stage of the study reported? [e.g. the size of the sample likely to be achieved, eligible participants (inclusion criteria), participants who were found to be eligible and approved, participants included, participants completing their follow-up and participants included into analyses] Are reasons for non-participation in all stages of the study given? Is a flow diagram, if appropriate, used? (only used for case-controlled and cohort studies)
© 2015 International Council of Nurses
No (%) Yes
Partly
No
Unclear
Not applicable§
397 (79.1) 407 (81.1)
– 95 (18.9)
105 (20.9) –
– –
– –
502 (100.0) 88 (17.5)* 490 (97.6)†
– – –
– 414 (82.5) 12 (2.4)
– – –
– – –
454 (90.4)‡ 284 (56.6) 396 (78.9) – 153 (30.5) 331 (65.9) 82 (16.3) 2 (0.4) 7 (1.4) 2 (0.4)
–
48 (9.6) 84 (16.7) 106 (21.1) 3 (0.6) 323 (64.3) 65 (13.0) 336 (67.0) 1 (0.2) – 2 (0.4)
– 134 (26.7) – – – 106 (21.1) 84 (16.7) – – –
– – – 499 (99.4) – – – 499 (99.4) – 498 (99.2)
– – –
3 (0.6) 3 (0.6) 2 (0.4) 4 (0.8) 482 (96.0)
– – 26 (5.2) – – – 495 (98.6) – 1 (0.2) 1 (0.2) – – 12 (2.4)
2 (0.4)
2 (0.4) 37 (7.4)
2 (0.4) –
11 (2.2) 11 (2.2) 13 (2.6) 5 (1.0) 8 (1.6)
–
487 (97.0) 487 (97.0) 487 (97.0) 490 (97.6) –
24 (4.8)
–
476 (94.8)
498 (99.2) 465 (92.6)
– –
– –
33 (6.6) 12 (2.4) 17 (3.4) 18 (3.6)
– – – –
– 476 (94.8) 476 (94.8) –
3 (0.6)
462 (92.0) 14 (2.8) 9 (1.8) 463 (92.2)
7 (1.4) – – 21 (4.2)
51 (10.2) 4 (0.8) –
– –
161 (32.1) – 3 (0.6)
12 (2.4) – –
276 (55.0) 498 (99.2) 499 (99.4)
–
–
–
502 (100.0)
–
221 (44.0)
–
–
–
489 (99.2)
– 495 (98.6)
186 (37.1)
4 (0.8)
2 (0.4)
95 (18.9)
–
7 (1.4)
Evaluation of observational research reports
Table 1
399
Continued
Original STROBE Item No.
Questions prepared according to the STROBE Statements Checklist
14
Is information about characteristics of participants (e.g. demographic, clinical and social characteristics), exposures to treatments and possible contributing factors described? Is the number of participants for each variable together with missing data indicated? Is information about duration of follow-up (mean and total duration of follow-up) reported? Is the number of outcome measures over time reported or are measurements made over time summarized? (only used for three cohort studies) Is the number of each exposure category reported or are measurements concerning exposures summarized? (only used for four case-controlled studies) Is the number of outcomes reported or are measurements summarized? (only used for 495 cross-sectional studies) Are adjusted values of confounding factors and their sensitivity (e.g. 95% CI) explained? Are confounding factors which were corrected and reasons for their inclusion explained? Are category boundaries of groups when continuous variables are categorized reported? If appropriate, is conversion of relative risk estimates for a significant period of time to absolute risk taken into consideration? Are other analyses made reported? (e.g. subgroup analyses and interactions and sensitivity analyses) Discussion Are the key results summarized by referring to the aims of the study? Are limitations of the study discussed taking account of possible sources of bias? Are both the size and the direction of each source of bias discussed? Is a general interpretation of the results made by taking account of aims, limitations, variety of analyses, findings from similar studies and other relevant evidence? Is generalizability (external validity or applicability) discussed? Other information Are sources of funding and roles of the individuals contributing to the study explained?
15
16
17
18 19 20
21 22
No (%) Yes
459 (91.4)
5 (1.0) –
Partly
20 (4.0)
–
No
23 (4.6)
–
–
3 (0.6)
Unclear
Not applicable§
–
–
–
497 (99.0)
–
499 (99.4)
3 (0.6)
–
–
–
499 (99.4)
4 (0.8)
–
–
–
498 (99.2)
–
–
7 (1.4)
–
487 (97.0)
489 (97.4)
6 (1.2)
6 (1.2)
–
9 (1.8)
1 (0.2)
–
14 (2.8)
53 (10.6) –
79 (15.7) –
181 (36.0) 15 (3.0)
487 (97.0) 2 (0.4)
187 (37.3)
–
487 (97.0)
52 (10.4)
11 (2.2)
197 (39.2)
33 (6.6)
209 (41.6)
29 (5.8) 6 (1.2) 4 (0.8) 12 (2.4)
472 (94.0) 2 (0.4) 1 (0.2) 489 (97.4)
1 (0.2) 494 (98.4) 497 (99.0) 1 (0.2)
– – – –
– – – –
14 (2.8)
–
488 (97.2)
–
–
6 (1.2)
–
496 (98.8)
–
–
*Hypotheses and research questions of six studies have been reported in methods. †Aims of 14 studies have been reported in methods. ‡Study designs of 52 studies have been reported in introduction. §Not applicable: If an item is not essential for study, it is evaluated as ‘Not Applicable’.
In the discussion sections of the reports, primary findings were partly summarized by referring to aims of the studies (94.0%) and a general interpretation of research findings was partly made (97.4%). In almost all the studies, limitations and possible sources of bias or inconsistency (98.4%) and both the size and the direction of each bias (99.0%) were not mentioned in the discussion sections. Generalizations about
© 2015 International Council of Nurses
findings were made only in 2.8% of the reports and a source of funding was mentioned only in 1.2% of the reports (Table 1).
Discussion In this literature-based study, 502 observational research reports from 11 nursing journals, regularly published in Turkey
400
Z. Karaçam et al.
between 2007 and 2012, were examined according to the STROBE Statements Checklist. The study revealed that sufficient information about study designs, settings, time of data collection, study populations, sources of data obtained, data collection tools, ways of analysing numerical variables and statistical methods used were found to be provided in their titles and summaries. In addition, the introduction and methods sections, characteristics of participants, findings expressed in numbers and summaries of findings obtained were sufficiently reported. However, the researchers found that the hypotheses tested or research questions, inclusion/exclusion criteria, selection of participants, possible sources of bias, ways to reach a given sample size, subgroup analyses, causes of declining to participate in each stage of a study, the number of participants for each variable, limits of groups when continuous variables are grouped and other analyses made, sources of funding and discussion sections were either not reported at all or were inadequately reported. These findings are of great importance since they reveal the quality of reporting the observational studies in nursing journals recently published in Turkey in terms of international standards. Consistent with the results of a study by Langan et al. (2010) and Jeelani et al. (2014), titles and summaries of the studies were found to be well reported. Unlike relevant international practices, study designs were reported in the summary sections but not in the titles. Although it is acceptable in terms of the STROBE Statements Checklist (von Elm et al. 2007), it reflects Turkish nursing researchers’ tendency to report study designs. A favourable finding was that all the reports presented relevant scientific evidence available in the literature and implications for further studies. This can be explained by the fact that reviewers and editors in Turkey place great emphasis on this criterion. However, answers to the question whether the available relevant literature and the impetus for the study were supplied could not be evaluated since information provided in the answers was beyond the scope of the researchers’ domain knowledge. Therefore, further research is needed to evaluate the answers to this question. Although aims of the studies were presented in almost all the reports, hypotheses tested and research questions were presented in few studies. In addition, aims, hypotheses and research questions were found to be provided in the methods sections in several studies although they are supposed to be written in the introduction sections. Similarly, Langan et al. (2010) found that hypotheses and research questions were presented only in a small proportion of the studies (9%). This can be associated with a similarity in attitudes between editors, reviewers and researchers.
© 2015 International Council of Nurses
Methods sections are of importance in that they reveal details about how studies are conducted and the quality of their findings totally depends on the quality of the methods sections (Ajithkumar 2011; Hesselbach et al. 2012). In this study, the methods sections were found to provide information about the study designs and settings, time of data collection, study populations, sources of data collected, data collection tools, ways of analysing numerical variables and statistical methods used in most of the reports, which was satisfactory. However, in a high proportion of the reports, inclusion/exclusion criteria, selection of participants, possible sources of bias, how a given sample size was reached and subgroup analyses were not presented. Jeelani et al. (2014) also reported similar results. In a study evaluating the quality of methodological reports of 408 case-controlled studies published in general psychiatric journals, the reports turned out to be poorly written and information about inclusion of the participants was frequently not reported (Lee et al. 2007). Grindstaff & Saliba (2012) also reported that ‘not including a sample size estimate’ was one of the ten most frequent mistakes made. Likewise, several other studies revealed important deficiencies in the methods sections of research reports (Guglielminotti et al. 2014; Pocock et al. 2004; Rangel et al. 2003; Tooth et al. 2005). Missing information in methods sections may affect statistical power and reliability of studies and generalizability of the results. In addition, in this study, study designs were found to be described in the introduction sections in several reports, which were considered acceptable according to the STROBE Statements Checklist (Vandenbroucke et al. 2007). Furthermore, the lack of explanations about how missing data were dealt with in general and tendency to exclude the participants and not providing answers to all questions in the data collection tools can be due to cancelling forms with missing data and not viewing it as important to report this condition in Turkey. Results of studies together with their statistical analyses should be presented in an order of their significance without their interpretations by using text, tables and figures (Ajithkumar 2011; Grindstaff & Saliba 2012). Consistent with the literature, characteristics of participants were found to be reported and summarized in a satisfactory frequency. However, it was noted that the number of individuals participating in each stage of a study, causes of declining to participate, the number of participants for each variable, limits of groups when continuous variables were grouped and other analyses made were not reported, which was consistent with the results of a study by Langan et al. (2010) on an examination of observational studies published in dermatology journals. Not
Evaluation of observational research reports
reporting information about reasons of declining to participate in all stages of studies can be associated with exclusion of individuals declining to participate and acting as if they were never invited to participate in the study. It has been reported in the literature that discussion sections are the most difficult to write (Grindstaff & Saliba 2012; Jenicek 2006). In fact, this study showed that the discussion sections were written quite poorly in all the reports examined. Summarizing primary findings by referring to aims of the studies and a general interpretation of all findings were partly made and limitations and sources of bias and generalizability of results were not reported in the discussion sections, which was not comparable with the results of a study by Langan et al. (2010). It is expected that findings should be interpreted in discussion sections in the light of the evidence available in the literature and that after incorporating the new knowledge obtained, the research problem presented at the beginning of the report should be understood and presented again by synthesizing it in the light of expert opinions and the literature (Akhgari et al. 2010; Bates College 2013). In addition, authors are expected to express their opinions about their studies as a whole, to explain the meaning and importance of findings and underlying mechanisms, to compare study results with those from the literature by using a sufficient number of sources, to discuss sources of bias and limitations, strengths and weaknesses of their studies, and to report what should be done in the future (Davidson et al. 2012; Jenicek 2006; Polton & Grant 2012). This study showed that sources of funding were reported only in six studies. Similarly, Jeelani et al. (2014) found that source of funding were presented in a small proportion of the studies (8%). It may be that instructions to authors in the journals do not include a requirement for supplying sources of funding and that the reviewers and the editors do not pay attention to this information.
Limitations In this study, the research reports were examined by the researchers experienced in observational research on gynecology and obstetrics nursing and having articles published in international journals. Nevertheless, the results obtained may differ from those obtained by an epidemiologist. Furthermore, the examination of the research reports on subjects different from the researchers’ self-specializing topics might have created bias. Therefore, further research should be conducted by groups of researchers specializing in different topics and epidemiologists together. The results of this study are representative of the observational studies examined, and therefore they cannot be generalized to other contexts.
© 2015 International Council of Nurses
401
Implications for nursing and health policies The study showed that study designs and settings, time of data collection, study populations, data collection tools, ways of analysing numerical variables and statistical tests used were generally well reported in the titles, summaries and the introduction sections of the studies. Characteristics of participants and obtained findings were summarized satisfactorily in the results sections. However, hypotheses or research questions were generally not reported. In methods section, inclusion/exclusion criteria, selection of participants, possible sources of bias, ways to reach sample sizes and subgroup analyses, and causes of declining to participate in each stage of the studies were reported in an insufficient way or not reported. In results section, the number of participants per variable, limits of groups when continuous variables were grouped and other analyses made, discussion section and sources of funding were not insufficiently reported or not reported. These outcomes clearly revealed that the observational research studies published in nursing journals in Turkey did not fulfil important criteria and needed to be improved. Furthermore, better reporting is required to synthesize available evidence and to create reliable scientific knowledge which could be used in clinical practice and health policies. Therefore, it can be recommended to nurse researchers, nurse educators and nursing journal editors in Turkey that international guides like the STROBE Statements Checklist should be utilized to prepare higher quality observational research reports (Grindstaff & Saliba 2012). These are the recommendations: (1) Turkish government and universities should support to researchers’ works on the quality of nursing research reporting with present research fund, (2) health services agencies and nursing services should compose support units to guide and educate about research process included in research reporting for researchers, (3) editors should provide guidelines of their journals to authors based on international guidelines, (4) nursing educators should spend more time to the research reporting in undergraduate and post-graduate nursing education, and (5) the researchers should consider international research reporting guidelines during research process, and participate more in research reporting during education programme. In addition, to improve the quality of nursing research reporting, Turkish government, health services agencies and nursing services should generate together a special strategy on the developing of nursing research reporting and put it into practice immediately. Information about the use of the STROBE Statements Checklist can be accessed through the website http://www.strobe-statement.org and through relevant studies (von Elm et al. 2007; Vandenbroucke et al. 2007). The STROBE Statements and other guides for reporting other types of research are provided by Enhancing the Quality and Trans-
402
Z. Karaçam et al.
parency of Health Research (EQUATOR) and available on its website http://www.equator-network.org. In addition, further research should be conducted to continue monitoring the quality of reporting observational studies and to reveal more details and gaps which could not be revealed in this study.
Conclusion This study revealed that reporting of observational research published in nursing journals in Turkey did not fulfil some important items listed in the STROBE Statements. Using of the STROBE Statements and/or other information sources about research reporting might improve the quality of reporting observational research studies and other research for better patient outcomes.
Acknowledgements The authors are truly grateful to Denis Walsh, Associate Professor, Nottingham University, and to Nursel Duransoy, Lecturer, Yeditepe University, for their assistance in editing.
Author contributions ZK, ES¸ and BY: Study design, data collection, data analyses and manuscript preparation.
References Ajithkumar, K. (2011) Scientific writing: How to write a manuscript – an introduction. Kerala Journal of Orthopaedics, 24 (1), 1–3. Akhgari, M., Asadi, M., Lee, E. & Nasirova, D. (2010) Scientific paper writing guidelines: writing a scientific research article. Retrieved 19 January 2010. Available at: http://www.columbia.edu/cu/biology/ug/ research/paper.html (accessed 15 January 2013). Altman, D. & Simera, I. (2010) Responsible reporting of health research studies: transparent, complete, accurate and timely. Journal of Antimicrobial Chemotherapy, 65 (1), 1–3. Bates College. (2013) How to write a paper in scientific journal style and format (pdf). Available at: http://abacus.bates.edu/∼ganderso/biology/ resources/writing/HTWsections.html (accessed 15 January 2013). Çalıs¸ır, H., Karabudak, S.S., S¸en, E. & Karaçam, Z. (2013) Research in nursing in Turkey. From Research to Publication, Lecture Material of 5th International Research School, College of Nursing Jesenice, Jesenice, pp. 15–20. Davidson, A., McD Taylor, D. & Babl, F.E. (2012) Review article: a primer for clinical researchers in the emergency department: part III: how to write a scientific paper. Emergency Medicine Australasia, 24 (4), 357–362. von Elm, E., et al. (2007) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet, 370 (9596), 1453–1457. Glasziou, P., Meats, E., Heneghan, C. & Shepperd, S. (2008) What is missing from descriptions of treatment in trials and reviews? BMJ (Clinical Research Ed.), 336 (7659), 1472–1474.
© 2015 International Council of Nurses
Grindstaff, T.L. & Saliba, S.A. (2012) Avoiding manuscript mistakes. International Journal of Sports Physical Therapy, 7 (5), 518–524. Guglielminotti, J., et al. (2014) Reporting and methodology of multivariable analyses in prognostic observational studies published in 4 anesthesiology journals: a methodological descriptive review. Anesthesia & Analgesia, doi: 10.1213/ane.0000000000000517. Hesselbach, R.A., et al. (2012) A guide to writing a scientific paper: a focus on high school through graduate level student research. Zebrafish, 9 (4), 246–249. Home, P.D. (2010) How can observational trials inform and improve clinical practice? Diabetes Research and Clinical Practice, 88 (S1), S1–S2. Ioannidis, J.P. (2009) Adverse events in randomized trials: neglected, restricted, distorted and silenced. Archives of Internal Medicine, 169 (19), 1737–1739. Jeelani, A., et al. (2014) Cross-sectional studies published in Indian Journal of Community Medicine: evaluation of adherence to strengthening the reporting of observational studies in epidemiology statement. Annals of Medical & Health Sciences Research, 4 (6), 875–878. Jenicek, M. (2006) How to read, understand, and write ‘Discussion’ sections in medical articles. An exercise in critical thinking. Medical Science Monitor, 12 (6), SR28–SR36. Langan, S., et al. (2010) The reporting of observational research studies in dermatology journals: a literature-based study. Archives of Dermatology, 146 (5), 534–541. Lee, W., et al. (2007) Bias in psychiatric case-control studies: literature survey. British Journal of Psychiatry, 190 (3), 204–209. Moher, D., et al. (2010) CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ (Clinical Research Ed.), 340, c869. Norris, S.L., et al. (2011) Observational studies in systemic reviews of comparative effectiveness: AHRQ and the Effective Health Care Program. Journal of Clinical Epidemiology, 64 (11), 1178–1186. Pocock, S.J., et al. (2004) Issues in the reporting of epidemiological studies: a survey of recent practice. BMJ (Clinical Research Ed.), 329 (7471), 883– 887. Polton, G. & Grant, I. (2012) The subtleties of scientific writing. Journal of Small Animal Practice, 53 (6), 311–312. Rangel, S.J., Kelsey, J., Henry, C.W.M. & Moss, R.L. (2003) Critical analysis of clinical research reporting in pediatric surgery: justifying the need for a new Standard. Journal of Pediatric Surgery, 38 (12), 1739–1743. Simera, I., et al. (2010) Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Medicine, 8 (4), 24. Simera, I., Kirtley, S. & Altman, D.G. (2012) Reporting clinical research: guidance to encourage accurate and transparent research reporting. Maturitas, 72 (1), 84–87. Song, F., et al. (2010) Dissemination and publication of research findings: an updated review of related biases. Health Technology Assessment Journal, 14 (8), 1–193, iii, ix–xi. Stevens, A., et al. (2014) Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: systematic review. BMJ (Clinical Research Ed.), 348, g3804.
Evaluation of observational research reports
Tooth, L., et al. (2005) Quality of reporting of observational longitudinal research. American Journal of Epidemiology, 161 (3), 280–288. Ünsal, A. & Sökmen, S. (2011) Characteristics of the articles and authors in The Journal of Atatürk University School of Nursing. Journal of Anatolia Nursing and Health Sciences, 14 (4), 10–16.
© 2015 International Council of Nurses
403
Vandenbroucke, J.P., et al. (2007) Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Medicine, 4 (10), 1628–1654. Yang, W., et al. (2010) Observational studies: going beyond the boundaries of randomized controlled trials. Diabetes Research and Clinical Practice, 88 (S1), S3–S9.
KARAÇAM Z., S¸EN E. & YILDIRIM B. (2015) Evaluation of observational research reports published in Turkish nursing journals. International Nursing Review 62, 394–403 Aim: The aim of this literature-based descriptive study was to examine the reporting of the observational research studies published in peer-reviewed nursing journals in Turkey. Methods: Eleven peer-reviewed nursing journals printed on a regular basis in Turkey between 2007 and 2012 were selected. These journals were searched for observational research studies, and 502 studies were selected and examined by using the Strengthening the Reporting of Observational Studies in Epidemiology Statement. Results: Of the 502 studies, 495 were cross-sectional, 3 were cohort, and 4 were case controlled. Summary and introduction and aim sections were sufficient in most of the studies. The methods sections of the reports were mostly not reported: 64.3% of the reports did not indicate eligibility/inclusion criteria; sampling method, 67.0%; possible sources of bias, 99.2%; ways to reach sample size, 92.6%. In the results section, the number of individuals participating in each stage of the studies (44.0%) and in other analyses made (39.2%) was not reported. In the discussion section, a main comment about research findings was partly made (97.4%), and limitations of the studies and possible sources of bias were not written in 99.0% of the studies. Conclusions: This study clearly revealed that the observational research studies published in nursing journals in Turkey did not fulfil the important criteria and needed to be improved. Implications for nursing and health policy: Information obtained from this study can contribute to improvement of the quality of reporting observational studies in nursing and thus using obtained findings in practice. Keywords: Journal Article, Nursing, Nursing Research, Observational Study, Turkey
Introduction Solutions for many health challenges are sought in observational research studies, accounting for an important portion of Correspondence address: Dr Zekiye Karaçam, Aydın Sag˘lık Yüksekokulu, Adnan Menderes Üniversitesi, Aydın 09100, Turkey; Tel: (90)-256-2148041; Fax: (90)-256-2124219; E-mail: [email protected]; [email protected].
Source of funding statement No grants were obtained in relation to this study. Conflict of interest No conflict of interest has been declared by the authors.
© 2015 International Council of Nurses
studies about positive and negative effects of medical interventions. Many studies about aetiology of diseases are cohort, case-controlled and cross-sectional observational studies. Experimental studies may not provide answers to all questions about a given intervention. Conducting observational studies could be more appropriate in investigating rare and long-term side effects of a treatment modality and in revealing data about medical practices implemented daily. However, only a limited amount of information acquired from observational studies is reported due to deficiencies in the reporting (von Elm et al. 2007; Simera et al. 2012).
394
bs_bs_banner
Nursing Research
Evaluation of observational research reports
A very well-written report reveals the strengths and weaknesses of a study and facilitates interpretation and implementation of research findings (Vandenbroucke et al. 2007). Reliability of a study depends on a critical evaluation of its design and conduction and the strengths and weaknesses of analyses by readers. Therefore, authors should present sufficient information clearly so that readers can make critical evaluations of what was planned and conducted, new information obtained and reliability of findings. The readers should also be able to understand the conclusions made. A poorly written research report restricts making use of important findings in clinical practice and further research and causes loss of valuable information. In addition, poor reporting creates an ethical challenge about how the findings were obtained and whether the study was really necessary if a report does not include this piece of information (Altman & Simera 2010; von Elm et al. 2007; Simera et al. 2012). In recent years, undertaking systematic reviews has led to the development of clear quality criteria to assess research studies. These criteria might reveal many deficiencies which affect the quality of publications in terms of the possibility of bias, presentation of findings in summaries and throughout manuscripts, original study protocols, details of interventions, inclusion and exclusion criteria and missing data about negative effects of interventions (Altman & Simera 2010; Simera et al. 2012). These deficiencies in the reporting of studies make difficult the evaluation of the quality of studies, decrease credibility of presented findings and make them hard to use in creating evidence and developing clinical guides. As a result, the findings cannot be beneficial in patient care. In addition, selectively reporting positive findings about a given treatment may lead to overdependence on that treatment, which may result in high health costs without improving patient care outcomes. Moreover, overemphasis on positive effects of a treatment alternative or inability to present all findings may cause harm to patients who will be included in further studies. Insufficient presentation or not presenting harmful effects of a given alternative at all, especially in experimental studies, is a serious challenge and an unethical practice (Glasziou et al. 2008; Ioannidis 2009; Simera et al. 2010; Song et al. 2010). Studies in which research reports about health-related topics have been examined have shown serious deficiencies including insufficient reporting of study designs and statistical analyses (Rangel et al. 2003). This includes not reporting the logic behind selection of possible confounding factors (Pocock et al. 2004), not explaining methods used to determine study and control groups (Lee et al. 2007), and not mentioning inclusion/ exclusion criteria (Tooth et al. 2005). In addition, examination of research reports might also reveal some discrepancies such as
© 2015 International Council of Nurses
395
including late or not reporting negative findings, selectively reporting positive findings or conversely changing findings in accordance with study protocols, not writing important information about methods and treatments, and offering insufficient information or misinterpreting findings in summaries and insufficient or misleading reports of side effects (Altman & Simera 2010). In one study, 300 observational studies published between 1997 and 2002 were shown to fulfil fewer than half of the criteria for writing research reports (Rangel et al. 2003). The term observational study describes a wide range of study designs including prospective and retrospective cohort studies, case-control studies and cross-sectional studies, a defining feature of which is that any intervention studied is determined by clinical practice and not by the protocol. Data from large, prospective observational studies provide information about the safety and efficacy of medicines in daily clinical use. Such observational studies are generally carried out once the researchers have received approval from regulatory agencies. Observational trials have inherent limitations in terms of their susceptibility to bias and confounding factors, restricting their ability to define causality. However, their strengths include their ability to reflect daily clinical practice more closely than randomized controlled trials (RCTs), both in terms of the heterogeneous patient populations that are included, and the medical interventions that they receive. Therefore, observational trials can provide clinically relevant information that is not necessarily provided by RCTs. Given the limitations of an observational study approach, it is important to optimize study designs to maximize the validity, and thus in particular known causes of bias and confounding factors should be measured (Home 2010; Norris et al. 2011; Yang et al. 2010). An important role for editors is to ensure that research articles published in their journals are clear, complete, transparent and as free as possible from bias. Medical investigators also have a direct or indirect contractual responsibility to the funder of their study to ensure that they accurately report safety and efficacy in their research outcomes according to agreed procedures. In an effort to uphold high standards, journal editors may feel the need to endorse multiple reporting guidelines without knowledge of their rigour or ability to improve reporting. The CONSORT (Consolidated Standards of Reporting Trials) statement is a well-known reporting guideline that has been extensively evaluated (Moher et al. 2010). In addition, any publications or presentations of the results should be within guidelines for appropriate reporting of observational trials, such as those published by the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement. Observational studies give the medical community an opportunity to share latest clinical data/practice and to report insights
396
Z. Karaçam et al.
from their own clinical practice (Stevens et al. 2014; Yang et al. 2010). Most of the articles (80.7–84.5%) published in Turkish nursing journals are observational studies (Çalıs¸ır et al. 2013; Ünsal & Sökmen 2011). Based on the experience of our individual Turkish nursing journal editors and reviewers, it is also known that there are important challenges with preparation of studies for publication in both national and international journals in Turkey. However, there has not been any research on this subject, the types and severity of these challenges are not known, and research-based information about them is required. In this context, an examination of observational studies in nursing based on the STROBE Statements Checklist of items should be included in reports of observational studies which may help reveal what problems exist and determine what sections of reports one should focus on. Therefore, the aim of this study was to examine the observational research reports published in nursing journals in Turkey. We examined only nursing research done in Turkey and published in Turkish nursing journals. It is anticipated that information obtained from this study will contribute to the improvement of the quality of reporting observational studies in nursing and thus using the obtained findings in practice.
Methods A literature-based descriptive study was performed on publications included in nursing journals in Turkey from 2007 to 2012. The study was carried out between January 2013 and July 2013. The articles included were based on quantitative methods mainly reported observational studies. Furthermore, in this study a quantitative approach was used. Journals available and selection of journals
All of nursing journals published in Turkey were available from the website of Koç University Suna Kıraç Library (http:// libguides.ku.edu.tr/content.php?pid=40495&sid=298033). There were 19 nursing journals published in Turkey. The study included 11 nursing journals having been published as peerreviewed journals since 2007 and are still being published. All articles of these journals were accessed through the Internet as full text. Selection of studies
Selection and recording of observational studies were carried out by two researchers independently. Eleven journals included were accessed, and cohort, case-controlled and cross-sectional studies were recorded. The studies, the designs of which could not be exactly determined, were also recorded. The studies determined independently by the researchers were compared in
© 2015 International Council of Nurses
one session, and disagreements about the studies were eliminated and an agreement was reached about the studies whose methods were not mentioned. All the studies included were listed and coded by naming them with sequential numbers and the first letters of the titles of the journals in which they were published. There were a total of 1045 articles published in 11 nursing journals between 2007 and 2012 and satisfying the inclusion criteria. Both quantitative and qualitative studies are published in these journals. Of these 1045 articles, 664 were research, 354 were traditional reviews, 15 were letters to editors, and 12 were case reports. Of the research articles, 502 were observational, 102 were experimental/quasi-experimental, 45 were methodological, 12 were qualitative, and 3 were systematic reviews. A total of 502 observational research reports were included in this study (Fig. 1). Inclusion and exclusion criteria
The study was performed on the observational research reports from peer-reviewed nursing journals in Turkey, and these reports were examined according to the STROBE Statements Checklist. Because observational research is used widely in Turkish nursing research (Çalıs¸ır et al. 2013; Ünsal & Sökmen 2011), the observational studies (cohort, case controlled, crosssectional and descriptive) with a quantitative design were
The number of nursing journals in total N = 19
The number of journals published regularly between 2007 and 2012 N = 11
The number of studies reported in journals published regularly between 2007 and 2012 N = 1045 Excluded studies 354 Traditional reviews 102 Experimental/quasi-experimental studies 45 Methodological studies 15 Letters to editors 12 Qualitative studies 12 Case reports 3 Systematic reviews The number of studies included in this study N = 502
Fig. 1 Flow chart for studies included in the study.
Evaluation of observational research reports
397
included. To limit the scope of the study, experimental and quasi-experimental studies, traditional reviews, systematic reviews, methodological studies, qualitative studies, case reports and letters to editors were excluded.
informed about the study, and their studies could not be included in the references. In addition, information about the authors of the examined studies was concealed.
Data extraction
Results
In the first stage, a Turkish version of the STROBE Statements Checklist, including items required in reporting observational studies, was developed. The validity and reliability of the Checklist was achieved by linguistic validity with the methods of group translation and expert opinion. In the second stage, a questionnaire about information available in the STROBE Statements Checklist was created by making use of the study by Langan et al. (2010) (Table 1). In the next stage, the reports to be included in the study were determined and examined according to the STROBE Statements Checklist. Then, to standardize the research assessment skills of researchers with the STROBE Statements Checklist, a pilot study was conducted in a session in which all the researchers in a journal not included in the main study participated. Differences of information and opinion of researchers were eliminated about how items of the STROBE Statements Checklist in the article were determined.
In this study, 502 observation research reports published in 11 nursing journals in Turkey between 2007 and 2012 were examined according to the STROBE Statements Checklist. Of these 502 studies, 98.6% were cross-sectional (n = 495/502), 0.6% were cohort (n = 3/502) and 0.8% were case controlled (n = 4/ 502). The examination of titles and abstracts of the studies revealed that study designs were mentioned in the summary sections in 79.1% (n = 397/502) of the reports and that sufficient information explaining what was investigated and what was found was evidenced in 81.1% (n = 407/502) of the reports (Table 1). All of the reports included a study design in their abstracts. Available scientific evidence and impetus for research were presented in all the reports. Aims of the studies were mentioned in 97.6% of the reports, and hypotheses or research questions were provided in 17.5% of the reports. Of these reports, 14 presented aims of the studies and 6 presented research questions and hypotheses in the methods sections (Table 1). The methods sections of the reports mostly included study designs (90.5%), settings (56.6%), time of data collection (78.9%), sources of data collected and data collection tools (96.0%), ways of analysing quantitative variables (92.0%) and methods of statistical analyses (92.2%). Out of all the study reports, 52 supplied study designs in their introduction sections. In a considerable proportion of the cross-sectional studies, 64.3 and 67.0% did not indicate the eligibility/inclusion criteria and the methods of selection of participants, respectively. Furthermore, attempts to show possible sources of bias and ways to reach sample sizes were 99.2 and 92.6% respectively incomplete in the majority of the studies. Up to 42.1% of the reports of the studies did not explain subgroup analyses and their interactions (Table 1). In the result sections of the reports, mostly characteristics of participants were presented (91.4%), and the results expressed in numbers or data obtained were summarized (cross-sectional studies: 97.4%). Few reports presented the number of participants included in each stage of the studies (37.1%), causes of declining to participate (0.8%; n = 4/502), the number of participants required for each variable (1.0%; n = 5/502), limits of groups when continuous variables were grouped (10.6%; n = 53/502) and other analyses made (10.4%; n = 52/502) (Table 1).
Examination of research reports
The research reports included in the study were examined by using a questionnaire developed based on a Turkish version of the STROBE Statements Checklist which includes 22 items. The examination of the research reports was performed independently by the two researchers to determine whether the reports fulfilled the criteria expressed in each question in the questionnaire. Then results of the examination made independently were compared. Relevant parts of the reports for which the researchers marked different choices were re-examined. An agreement was reached and a single examination form was filled in. Evaluation of data
Whether the reports included satisfied the criterion in each question in the questionnaire was determined by marking ‘yes’, ‘partly yes’, ‘no’, ‘unclear’ or ‘not applicable’. Obtained data were analysed with frequencies and percentages in a Statistical Package for the Social Sciences (SPSS) computer software program 15.0 (PASW Inc., Chicago, IL, USA). Ethical considerations
Extant information on this study was given to editors of the 11 journals by means of email. Because the total number of authors in this study was high (n = 502), they could not be
© 2015 International Council of Nurses
398
Z. Karaçam et al.
Table 1 Results of the examination of observational research reports according to the STROBE Statements Checklist (N = 502) Original STROBE Item No.
1
2 3
4 5
6
7
8
9 10 11
12
13
Questions prepared according to the STROBE Statements Checklist
Titles and abstract (a) Is study design expressed in the title or the abstract a commonly used term? (b) Is there sufficient explanatory information about what was done and what was found in the study in the abstract? Introduction Is the scientific background and rationale for the study reported? Are hypotheses or research questions reported? Is the aim of the study reported? Methods Are key parts related to the study design presented? Is the study setting reported? Are the dates during which data were collected reported? Are the dates of follow-up reported? (only used for three cohort studies) Are the eligibility/inclusion criteria given? Are the sources of participants given? Are the methods of selection explained? Are methods of follow-up described? (only used for three cohort studies) Are all variables explained clearly? Are all exposures, if appropriate, described? (only used for four case-controlled studies) Are all predictors, if appropriate, described? Are possible confounders, if appropriate, described? Are effect modifiers, if appropriate, described? Are diagnostic criteria, if appropriate, reported? Are sources of data for each variable and details about data collection tools reported? If there is more than one group, is comparability of data collection tools described? (Is whether methods of measurement are valid and reliable in differentiation of the groups reported?) Are attempts to show possible sources of bias described? Is how the sample size is reached (the statistical method used to determine the sample size) described? Is how quantitative variables are analysed explained? If appropriate, did which groupings were made for quantitative variables report? If appropriate, are the reasons for grouping for quantitative variables reported? Are all statistical tests used, including those used to control confounding factors, described? Are all methods used to investigate subgroups and their interactions explained? Is how missing data are dealt with explained? If appropriate, is how cases lost to follow-up dealt with explained? (only used for three cohort studies) Are sensitivity analyses described? Results Is the number of participants included in each stage of the study reported? [e.g. the size of the sample likely to be achieved, eligible participants (inclusion criteria), participants who were found to be eligible and approved, participants included, participants completing their follow-up and participants included into analyses] Are reasons for non-participation in all stages of the study given? Is a flow diagram, if appropriate, used? (only used for case-controlled and cohort studies)
© 2015 International Council of Nurses
No (%) Yes
Partly
No
Unclear
Not applicable§
397 (79.1) 407 (81.1)
– 95 (18.9)
105 (20.9) –
– –
– –
502 (100.0) 88 (17.5)* 490 (97.6)†
– – –
– 414 (82.5) 12 (2.4)
– – –
– – –
454 (90.4)‡ 284 (56.6) 396 (78.9) – 153 (30.5) 331 (65.9) 82 (16.3) 2 (0.4) 7 (1.4) 2 (0.4)
–
48 (9.6) 84 (16.7) 106 (21.1) 3 (0.6) 323 (64.3) 65 (13.0) 336 (67.0) 1 (0.2) – 2 (0.4)
– 134 (26.7) – – – 106 (21.1) 84 (16.7) – – –
– – – 499 (99.4) – – – 499 (99.4) – 498 (99.2)
– – –
3 (0.6) 3 (0.6) 2 (0.4) 4 (0.8) 482 (96.0)
– – 26 (5.2) – – – 495 (98.6) – 1 (0.2) 1 (0.2) – – 12 (2.4)
2 (0.4)
2 (0.4) 37 (7.4)
2 (0.4) –
11 (2.2) 11 (2.2) 13 (2.6) 5 (1.0) 8 (1.6)
–
487 (97.0) 487 (97.0) 487 (97.0) 490 (97.6) –
24 (4.8)
–
476 (94.8)
498 (99.2) 465 (92.6)
– –
– –
33 (6.6) 12 (2.4) 17 (3.4) 18 (3.6)
– – – –
– 476 (94.8) 476 (94.8) –
3 (0.6)
462 (92.0) 14 (2.8) 9 (1.8) 463 (92.2)
7 (1.4) – – 21 (4.2)
51 (10.2) 4 (0.8) –
– –
161 (32.1) – 3 (0.6)
12 (2.4) – –
276 (55.0) 498 (99.2) 499 (99.4)
–
–
–
502 (100.0)
–
221 (44.0)
–
–
–
489 (99.2)
– 495 (98.6)
186 (37.1)
4 (0.8)
2 (0.4)
95 (18.9)
–
7 (1.4)
Evaluation of observational research reports
Table 1
399
Continued
Original STROBE Item No.
Questions prepared according to the STROBE Statements Checklist
14
Is information about characteristics of participants (e.g. demographic, clinical and social characteristics), exposures to treatments and possible contributing factors described? Is the number of participants for each variable together with missing data indicated? Is information about duration of follow-up (mean and total duration of follow-up) reported? Is the number of outcome measures over time reported or are measurements made over time summarized? (only used for three cohort studies) Is the number of each exposure category reported or are measurements concerning exposures summarized? (only used for four case-controlled studies) Is the number of outcomes reported or are measurements summarized? (only used for 495 cross-sectional studies) Are adjusted values of confounding factors and their sensitivity (e.g. 95% CI) explained? Are confounding factors which were corrected and reasons for their inclusion explained? Are category boundaries of groups when continuous variables are categorized reported? If appropriate, is conversion of relative risk estimates for a significant period of time to absolute risk taken into consideration? Are other analyses made reported? (e.g. subgroup analyses and interactions and sensitivity analyses) Discussion Are the key results summarized by referring to the aims of the study? Are limitations of the study discussed taking account of possible sources of bias? Are both the size and the direction of each source of bias discussed? Is a general interpretation of the results made by taking account of aims, limitations, variety of analyses, findings from similar studies and other relevant evidence? Is generalizability (external validity or applicability) discussed? Other information Are sources of funding and roles of the individuals contributing to the study explained?
15
16
17
18 19 20
21 22
No (%) Yes
459 (91.4)
5 (1.0) –
Partly
20 (4.0)
–
No
23 (4.6)
–
–
3 (0.6)
Unclear
Not applicable§
–
–
–
497 (99.0)
–
499 (99.4)
3 (0.6)
–
–
–
499 (99.4)
4 (0.8)
–
–
–
498 (99.2)
–
–
7 (1.4)
–
487 (97.0)
489 (97.4)
6 (1.2)
6 (1.2)
–
9 (1.8)
1 (0.2)
–
14 (2.8)
53 (10.6) –
79 (15.7) –
181 (36.0) 15 (3.0)
487 (97.0) 2 (0.4)
187 (37.3)
–
487 (97.0)
52 (10.4)
11 (2.2)
197 (39.2)
33 (6.6)
209 (41.6)
29 (5.8) 6 (1.2) 4 (0.8) 12 (2.4)
472 (94.0) 2 (0.4) 1 (0.2) 489 (97.4)
1 (0.2) 494 (98.4) 497 (99.0) 1 (0.2)
– – – –
– – – –
14 (2.8)
–
488 (97.2)
–
–
6 (1.2)
–
496 (98.8)
–
–
*Hypotheses and research questions of six studies have been reported in methods. †Aims of 14 studies have been reported in methods. ‡Study designs of 52 studies have been reported in introduction. §Not applicable: If an item is not essential for study, it is evaluated as ‘Not Applicable’.
In the discussion sections of the reports, primary findings were partly summarized by referring to aims of the studies (94.0%) and a general interpretation of research findings was partly made (97.4%). In almost all the studies, limitations and possible sources of bias or inconsistency (98.4%) and both the size and the direction of each bias (99.0%) were not mentioned in the discussion sections. Generalizations about
© 2015 International Council of Nurses
findings were made only in 2.8% of the reports and a source of funding was mentioned only in 1.2% of the reports (Table 1).
Discussion In this literature-based study, 502 observational research reports from 11 nursing journals, regularly published in Turkey
400
Z. Karaçam et al.
between 2007 and 2012, were examined according to the STROBE Statements Checklist. The study revealed that sufficient information about study designs, settings, time of data collection, study populations, sources of data obtained, data collection tools, ways of analysing numerical variables and statistical methods used were found to be provided in their titles and summaries. In addition, the introduction and methods sections, characteristics of participants, findings expressed in numbers and summaries of findings obtained were sufficiently reported. However, the researchers found that the hypotheses tested or research questions, inclusion/exclusion criteria, selection of participants, possible sources of bias, ways to reach a given sample size, subgroup analyses, causes of declining to participate in each stage of a study, the number of participants for each variable, limits of groups when continuous variables are grouped and other analyses made, sources of funding and discussion sections were either not reported at all or were inadequately reported. These findings are of great importance since they reveal the quality of reporting the observational studies in nursing journals recently published in Turkey in terms of international standards. Consistent with the results of a study by Langan et al. (2010) and Jeelani et al. (2014), titles and summaries of the studies were found to be well reported. Unlike relevant international practices, study designs were reported in the summary sections but not in the titles. Although it is acceptable in terms of the STROBE Statements Checklist (von Elm et al. 2007), it reflects Turkish nursing researchers’ tendency to report study designs. A favourable finding was that all the reports presented relevant scientific evidence available in the literature and implications for further studies. This can be explained by the fact that reviewers and editors in Turkey place great emphasis on this criterion. However, answers to the question whether the available relevant literature and the impetus for the study were supplied could not be evaluated since information provided in the answers was beyond the scope of the researchers’ domain knowledge. Therefore, further research is needed to evaluate the answers to this question. Although aims of the studies were presented in almost all the reports, hypotheses tested and research questions were presented in few studies. In addition, aims, hypotheses and research questions were found to be provided in the methods sections in several studies although they are supposed to be written in the introduction sections. Similarly, Langan et al. (2010) found that hypotheses and research questions were presented only in a small proportion of the studies (9%). This can be associated with a similarity in attitudes between editors, reviewers and researchers.
© 2015 International Council of Nurses
Methods sections are of importance in that they reveal details about how studies are conducted and the quality of their findings totally depends on the quality of the methods sections (Ajithkumar 2011; Hesselbach et al. 2012). In this study, the methods sections were found to provide information about the study designs and settings, time of data collection, study populations, sources of data collected, data collection tools, ways of analysing numerical variables and statistical methods used in most of the reports, which was satisfactory. However, in a high proportion of the reports, inclusion/exclusion criteria, selection of participants, possible sources of bias, how a given sample size was reached and subgroup analyses were not presented. Jeelani et al. (2014) also reported similar results. In a study evaluating the quality of methodological reports of 408 case-controlled studies published in general psychiatric journals, the reports turned out to be poorly written and information about inclusion of the participants was frequently not reported (Lee et al. 2007). Grindstaff & Saliba (2012) also reported that ‘not including a sample size estimate’ was one of the ten most frequent mistakes made. Likewise, several other studies revealed important deficiencies in the methods sections of research reports (Guglielminotti et al. 2014; Pocock et al. 2004; Rangel et al. 2003; Tooth et al. 2005). Missing information in methods sections may affect statistical power and reliability of studies and generalizability of the results. In addition, in this study, study designs were found to be described in the introduction sections in several reports, which were considered acceptable according to the STROBE Statements Checklist (Vandenbroucke et al. 2007). Furthermore, the lack of explanations about how missing data were dealt with in general and tendency to exclude the participants and not providing answers to all questions in the data collection tools can be due to cancelling forms with missing data and not viewing it as important to report this condition in Turkey. Results of studies together with their statistical analyses should be presented in an order of their significance without their interpretations by using text, tables and figures (Ajithkumar 2011; Grindstaff & Saliba 2012). Consistent with the literature, characteristics of participants were found to be reported and summarized in a satisfactory frequency. However, it was noted that the number of individuals participating in each stage of a study, causes of declining to participate, the number of participants for each variable, limits of groups when continuous variables were grouped and other analyses made were not reported, which was consistent with the results of a study by Langan et al. (2010) on an examination of observational studies published in dermatology journals. Not
Evaluation of observational research reports
reporting information about reasons of declining to participate in all stages of studies can be associated with exclusion of individuals declining to participate and acting as if they were never invited to participate in the study. It has been reported in the literature that discussion sections are the most difficult to write (Grindstaff & Saliba 2012; Jenicek 2006). In fact, this study showed that the discussion sections were written quite poorly in all the reports examined. Summarizing primary findings by referring to aims of the studies and a general interpretation of all findings were partly made and limitations and sources of bias and generalizability of results were not reported in the discussion sections, which was not comparable with the results of a study by Langan et al. (2010). It is expected that findings should be interpreted in discussion sections in the light of the evidence available in the literature and that after incorporating the new knowledge obtained, the research problem presented at the beginning of the report should be understood and presented again by synthesizing it in the light of expert opinions and the literature (Akhgari et al. 2010; Bates College 2013). In addition, authors are expected to express their opinions about their studies as a whole, to explain the meaning and importance of findings and underlying mechanisms, to compare study results with those from the literature by using a sufficient number of sources, to discuss sources of bias and limitations, strengths and weaknesses of their studies, and to report what should be done in the future (Davidson et al. 2012; Jenicek 2006; Polton & Grant 2012). This study showed that sources of funding were reported only in six studies. Similarly, Jeelani et al. (2014) found that source of funding were presented in a small proportion of the studies (8%). It may be that instructions to authors in the journals do not include a requirement for supplying sources of funding and that the reviewers and the editors do not pay attention to this information.
Limitations In this study, the research reports were examined by the researchers experienced in observational research on gynecology and obstetrics nursing and having articles published in international journals. Nevertheless, the results obtained may differ from those obtained by an epidemiologist. Furthermore, the examination of the research reports on subjects different from the researchers’ self-specializing topics might have created bias. Therefore, further research should be conducted by groups of researchers specializing in different topics and epidemiologists together. The results of this study are representative of the observational studies examined, and therefore they cannot be generalized to other contexts.
© 2015 International Council of Nurses
401
Implications for nursing and health policies The study showed that study designs and settings, time of data collection, study populations, data collection tools, ways of analysing numerical variables and statistical tests used were generally well reported in the titles, summaries and the introduction sections of the studies. Characteristics of participants and obtained findings were summarized satisfactorily in the results sections. However, hypotheses or research questions were generally not reported. In methods section, inclusion/exclusion criteria, selection of participants, possible sources of bias, ways to reach sample sizes and subgroup analyses, and causes of declining to participate in each stage of the studies were reported in an insufficient way or not reported. In results section, the number of participants per variable, limits of groups when continuous variables were grouped and other analyses made, discussion section and sources of funding were not insufficiently reported or not reported. These outcomes clearly revealed that the observational research studies published in nursing journals in Turkey did not fulfil important criteria and needed to be improved. Furthermore, better reporting is required to synthesize available evidence and to create reliable scientific knowledge which could be used in clinical practice and health policies. Therefore, it can be recommended to nurse researchers, nurse educators and nursing journal editors in Turkey that international guides like the STROBE Statements Checklist should be utilized to prepare higher quality observational research reports (Grindstaff & Saliba 2012). These are the recommendations: (1) Turkish government and universities should support to researchers’ works on the quality of nursing research reporting with present research fund, (2) health services agencies and nursing services should compose support units to guide and educate about research process included in research reporting for researchers, (3) editors should provide guidelines of their journals to authors based on international guidelines, (4) nursing educators should spend more time to the research reporting in undergraduate and post-graduate nursing education, and (5) the researchers should consider international research reporting guidelines during research process, and participate more in research reporting during education programme. In addition, to improve the quality of nursing research reporting, Turkish government, health services agencies and nursing services should generate together a special strategy on the developing of nursing research reporting and put it into practice immediately. Information about the use of the STROBE Statements Checklist can be accessed through the website http://www.strobe-statement.org and through relevant studies (von Elm et al. 2007; Vandenbroucke et al. 2007). The STROBE Statements and other guides for reporting other types of research are provided by Enhancing the Quality and Trans-
402
Z. Karaçam et al.
parency of Health Research (EQUATOR) and available on its website http://www.equator-network.org. In addition, further research should be conducted to continue monitoring the quality of reporting observational studies and to reveal more details and gaps which could not be revealed in this study.
Conclusion This study revealed that reporting of observational research published in nursing journals in Turkey did not fulfil some important items listed in the STROBE Statements. Using of the STROBE Statements and/or other information sources about research reporting might improve the quality of reporting observational research studies and other research for better patient outcomes.
Acknowledgements The authors are truly grateful to Denis Walsh, Associate Professor, Nottingham University, and to Nursel Duransoy, Lecturer, Yeditepe University, for their assistance in editing.
Author contributions ZK, ES¸ and BY: Study design, data collection, data analyses and manuscript preparation.
References Ajithkumar, K. (2011) Scientific writing: How to write a manuscript – an introduction. Kerala Journal of Orthopaedics, 24 (1), 1–3. Akhgari, M., Asadi, M., Lee, E. & Nasirova, D. (2010) Scientific paper writing guidelines: writing a scientific research article. Retrieved 19 January 2010. Available at: http://www.columbia.edu/cu/biology/ug/ research/paper.html (accessed 15 January 2013). Altman, D. & Simera, I. (2010) Responsible reporting of health research studies: transparent, complete, accurate and timely. Journal of Antimicrobial Chemotherapy, 65 (1), 1–3. Bates College. (2013) How to write a paper in scientific journal style and format (pdf). Available at: http://abacus.bates.edu/∼ganderso/biology/ resources/writing/HTWsections.html (accessed 15 January 2013). Çalıs¸ır, H., Karabudak, S.S., S¸en, E. & Karaçam, Z. (2013) Research in nursing in Turkey. From Research to Publication, Lecture Material of 5th International Research School, College of Nursing Jesenice, Jesenice, pp. 15–20. Davidson, A., McD Taylor, D. & Babl, F.E. (2012) Review article: a primer for clinical researchers in the emergency department: part III: how to write a scientific paper. Emergency Medicine Australasia, 24 (4), 357–362. von Elm, E., et al. (2007) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet, 370 (9596), 1453–1457. Glasziou, P., Meats, E., Heneghan, C. & Shepperd, S. (2008) What is missing from descriptions of treatment in trials and reviews? BMJ (Clinical Research Ed.), 336 (7659), 1472–1474.
© 2015 International Council of Nurses
Grindstaff, T.L. & Saliba, S.A. (2012) Avoiding manuscript mistakes. International Journal of Sports Physical Therapy, 7 (5), 518–524. Guglielminotti, J., et al. (2014) Reporting and methodology of multivariable analyses in prognostic observational studies published in 4 anesthesiology journals: a methodological descriptive review. Anesthesia & Analgesia, doi: 10.1213/ane.0000000000000517. Hesselbach, R.A., et al. (2012) A guide to writing a scientific paper: a focus on high school through graduate level student research. Zebrafish, 9 (4), 246–249. Home, P.D. (2010) How can observational trials inform and improve clinical practice? Diabetes Research and Clinical Practice, 88 (S1), S1–S2. Ioannidis, J.P. (2009) Adverse events in randomized trials: neglected, restricted, distorted and silenced. Archives of Internal Medicine, 169 (19), 1737–1739. Jeelani, A., et al. (2014) Cross-sectional studies published in Indian Journal of Community Medicine: evaluation of adherence to strengthening the reporting of observational studies in epidemiology statement. Annals of Medical & Health Sciences Research, 4 (6), 875–878. Jenicek, M. (2006) How to read, understand, and write ‘Discussion’ sections in medical articles. An exercise in critical thinking. Medical Science Monitor, 12 (6), SR28–SR36. Langan, S., et al. (2010) The reporting of observational research studies in dermatology journals: a literature-based study. Archives of Dermatology, 146 (5), 534–541. Lee, W., et al. (2007) Bias in psychiatric case-control studies: literature survey. British Journal of Psychiatry, 190 (3), 204–209. Moher, D., et al. (2010) CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ (Clinical Research Ed.), 340, c869. Norris, S.L., et al. (2011) Observational studies in systemic reviews of comparative effectiveness: AHRQ and the Effective Health Care Program. Journal of Clinical Epidemiology, 64 (11), 1178–1186. Pocock, S.J., et al. (2004) Issues in the reporting of epidemiological studies: a survey of recent practice. BMJ (Clinical Research Ed.), 329 (7471), 883– 887. Polton, G. & Grant, I. (2012) The subtleties of scientific writing. Journal of Small Animal Practice, 53 (6), 311–312. Rangel, S.J., Kelsey, J., Henry, C.W.M. & Moss, R.L. (2003) Critical analysis of clinical research reporting in pediatric surgery: justifying the need for a new Standard. Journal of Pediatric Surgery, 38 (12), 1739–1743. Simera, I., et al. (2010) Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Medicine, 8 (4), 24. Simera, I., Kirtley, S. & Altman, D.G. (2012) Reporting clinical research: guidance to encourage accurate and transparent research reporting. Maturitas, 72 (1), 84–87. Song, F., et al. (2010) Dissemination and publication of research findings: an updated review of related biases. Health Technology Assessment Journal, 14 (8), 1–193, iii, ix–xi. Stevens, A., et al. (2014) Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: systematic review. BMJ (Clinical Research Ed.), 348, g3804.
Evaluation of observational research reports
Tooth, L., et al. (2005) Quality of reporting of observational longitudinal research. American Journal of Epidemiology, 161 (3), 280–288. Ünsal, A. & Sökmen, S. (2011) Characteristics of the articles and authors in The Journal of Atatürk University School of Nursing. Journal of Anatolia Nursing and Health Sciences, 14 (4), 10–16.
© 2015 International Council of Nurses
403
Vandenbroucke, J.P., et al. (2007) Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Medicine, 4 (10), 1628–1654. Yang, W., et al. (2010) Observational studies: going beyond the boundaries of randomized controlled trials. Diabetes Research and Clinical Practice, 88 (S1), S3–S9.
