Nurse Educator Vol. 38, No. 6, pp. 265-268 Copyright * 2013 Wolters Kluwer Health | Lippincott Williams & Wilkins
Nurse Educator
Effectiveness of a Reference Accuracy Strategy for Peer-Reviewed Journal Articles Kirsten E. Speck, MPH, BA & Barbara St. Pierre Schneider, DNSc, RN Dissemination of information through peer-reviewed journal articles is an important requirement of success in academia. Despite the importance of publishing articles, about 25% to 45% of articles published in nursing journals have at least 1 reference error in the reference list. The authors discuss the implementation and outcomes of an internal copyeditor strategy aimed at reducing reference inaccuracy in faculty-authored journal articles.
S
cholarly publishing is 1 of the most important requirements to be successful in academia and a common criterion upon which academic tenure and promotion is based.1,2 It is also a mechanism by which faculty achieve a scholarly reputation.3 Scholarly publishing contributes to a university’s overall performance. For example, both the QS World University Rankings (http://www.topuniversities .com/university-rankings/world-university-rankings/2012) and Academic Ranking of World Universities (http://www .arwu.org/aboutARWU.jsp) consider faculty publications, including number of citations per faculty and number of articles indexed in a scientific citation database, as indicators of performance. In a competitive higher education market, a university with a strong record of scholarly publishing is likely to garner a strong rating of performance and attract highquality junior faculty and students as well as external funding.4 One component of scholarly publishing is ‘‘crediting sources.’’5 Crediting sources includes accurately quoting or paraphrasing content and recording the citation of the sources. Although the latter may seem insignificant, reference accuracy and reference inaccuracy of published articles have implications. For example, reference accuracy aids in establishing one’s reputation as a ‘‘careful researcher.’’5 In contrast, reference inaccuracy may have negative implications such as calling into question the accuracy of other content in the paper. Reference inaccuracy is common in articles published in nursing. Approximately 25% to 45% of articles published in nursing journals have at least 1 error in the reference list.6-11 These errors may be major, such as incorrect author names or article title, or minor, such as additional or omitted author
Author Affiliations: Research Technician (Ms Speck), Associate Professor (Dr St. Pierre Schneider), School of Nursing, University of Nevada, Las Vegas. The authors declare no conflicts of interest. Correspondence: Ms Speck, School of Nursing, University of Nevada, Las Vegas, BHS 100, 4505 S Maryland Pkwy, Box 453018, Las Vegas, NV 89154-3018 ([email protected]). DOI: 10.1097/01.NNE.0000435272.47774.51
Nurse Educator
initials, or minor misspellings and subtitle omission of the article title.8 These errors can result in 2 major consequences. First, the reader may be unable to retrieve the citation.6 Today’s retrieval of published material relies primarily on the use of electronic reference options (ie, World Wide Web, literature databases), and these options require accurate information of reference elements (eg, author name) to yield results. Second, the review or publication of the manuscript may be delayed or rejected as journal reviewers and editors may question reported research results or other details.9,12 To reduce reference inaccuracy of journal articles, most journals provide specific information and examples for authors and inform authors that reference accuracy is their responsibility. Citation styles also indicate that reference accuracy is the authors’ responsibility.5 There are various strategies that authors can use to ensure reference accuracy. One recommended strategy is to use a reference manager software.13 A copy of a reference manager software program may be free to a faculty author or cost $100 to $200. Although many of these programs allow citation information to be imported directly from an online scientific citation database, this activity still requires attention to detail. For example, the PubMed scientific citation database provides the journal title in the form of the journal abbreviation. If the reference citation style of an author’s chosen journal requires the journal full name, the author has to manually insert the journal full name into the program. Another strategy for authors is to work with a copyeditor. According to Meyer,14 ‘‘A good copyeditorIcan catch a majority of reference errors.’’ On the surface, the use of a copyeditor appears to be a more costly strategy than faculty use of reference manager software ($30-$50/h15 vs a 1-time cost of $100-$200); however, many faculty do not have copyeditor skills and faculty time is limited and expensive too. In the long run, the use of a copyeditor may be the best strategy for faculty and university performance. At our school of nursing, we adopted an internal copyeditor strategy for faculty authors. This service was provided Volume 38 & Number 6 & November/December 2013
Copyright © 2013 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
265
to all faculty and consisted of correcting grammar and punctuation and minor reference style (American Psychological Association [APA] style in our case), formatting errors, and proofing.
Methods This analysis was performed retrospectively. A literature search was performed using PubMed and Scopus databases to locate journal articles authored by faculty at our school of nursing. Keywords were (a) author affiliation, (b) subject area in nursing, and (c) articles published from January 1, 2000, to September 21, 2012. Only journal articles that were peer reviewed and in which the faculty was listed as first author were used in the analysis. Exclusion criteria were journal articles that (a) contained fewer than 5 citations, (b) were not retrievable because of in press status, and/or (c) were not retrievable as a PDF version. Once articles were identified, and therefore citations, the articles were separated into a copyediting or comparison group. Articles assigned to the copyediting group were those published between January 1, 2005, and September 21, 2012, and used an internal (school of nursing) copyediting service. This service consisted of copyediting by staff who had editing experience and were knowledgeable in using reference manager software. In contrast, those articles assigned to the comparison group were those published during the time period 2000-2004, before this internal copyediting was available to faculty authors. Journal citations in the reference list of articles in both groups underwent review for errors. Web pages and books in the reference list were excluded because of lack of accessibility. For articles that had 10 or fewer journal citations, all journal citations were included in the analysis. For articles that had more than 10 journal citations, the first journal citation followed by every third citation was selected.
Article Characteristics Two characteristics of the analyzed faculty authors’ journal articles were identified. The first was assigning each article a document type based on the document types covered in Scopus.16 Therefore, analyzed articles were (a) article (opinion or original research), (b) review (nonoriginal research), or (c) other (ie, editorial, note). This information was collected because review articles are likely to consist of more references than data-based or research articles.17 Another characteristic was the reference citation style of the journal in which the articles were published. This characteristic was examined because APA is the uniformed formatting style used at most US schools of nursing; however, the required use of a different style to publish journal articles may yield more errors. The journal’s current author guidelines were the source of the style and verified by the articles. Four standard styles were identified: (a) American Medical Association (AMA); (b) APA and favored style used by the social and behavioral sciences; (c) Vancouver (similar to AMA), popular among the physical sciences; and (d) Harvard (similar to APA). An ‘‘other’’ category was used when the style could not be identified through the journal’s author guidelines or by inspection. Also, faculty-authored articles were classified in this ‘‘other’’ category if the journal used a modified version of 1 of the 4 standard styles. 266
Volume 38 & Number 6 & November/December 2013
Reference Inaccuracy Once the article characteristics were identified, the journal citations in each article’s reference list were reviewed for 3 types of reference inaccuracy: major and minor content errors and style errors. Major and Minor Content Errors Based on a modification of the approach of Oermann et al,8 major content errors were (a) author errors in which author names were misspelled, omitted, or ordered incorrectly or author initials were incorrect; (b) article title errors, where incorrect article titles were provided; (c) incorrect journal or incorrect spelling/abbreviation of the journal title; and (d) incorrect or omitted volume, issue, year, or page number/s. Minor content errors were (a) omitted or additional author initials as listed on the article, (b) article title misspellings and omissions of subtitles, (c) added or omitted words to journal title, and (d) incomplete list of page numbers.8 Based on Taylor’s18 criteria, the scoring options for each element of a citation were 0 for no errors or 1 for 1 or more errors per element. For example, if there were 2 errors in the author name, 1 error was counted. Because of differences in the number of citations reviewed between the 2 groups, the total number of errors per faculty-authored publication was divided by the total number of citations reviewed within that article to yield a weighted total error value. For example, if article 1 has 18 citations and a raw total of 7 major errors, the weighted total error value is 7 18 = 0.39 for article 1. To spot check the scoring, a second reviewer independently scored the major and minor content errors of citations from a random selection of 3 articles from each group. The second reviewer selected the citations as described above and compared the faculty author’s version with a paper copy of the article and/or through Scopus, PubMed, or EBSCOhost databases. In total, the second reviewer checked 30 and 28 citations from the comparison and copyediting groups, respectively, and the first and second reviewer agreement was 96%.
Citation Style Errors The journal citations were also assessed for style errors. Reference citation errors were categorized into the 6 elements of the journal reference format: (a) author errors were incorrect spacing, punctuation, and/or number of listed authors; (b) year errors were inclusion of month of publication; (c) article title errors were incorrect punctuation and/or grammar such as a comma instead of a period after the article title; (d) journal title errors were journal abbreviation instead of full journal name or incorrect punctuation; (e) volume/issue errors were incorrect punctuation related to the volume and journal pagination; and (f ) pages errors were incorrect spacing or page number formatting. To spot check this scoring, the second reviewer also independently scored reference style citation errors using the same citations for the content error spot check. The first and second reviewer agreement was 98%. Statistical Analysis Data were analyzed using IBM SPSS Statistics 19 (IBM Corporation, Armonk, New York). Frequencies and percentages were used to summarize document type and style. The Mann-Whitney U test was performed to compare the median Nurse Educator
Copyright © 2013 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
major and minor content errors and style errors between the comparison and copyediting groups. Alpha was set at .05.
Results A literature search retrieved 21 faculty-authored publications in the comparison group (8 faculty authors) and 18 facultyauthored publications in the copyediting group (12 faculty authors). Based on the exclusion criteria, 3 articles from the comparison group were removed from the analysis. A total of 36 peer-reviewed articles (n = 18 articles per group) were reviewed in this analysis. From these 36 articles, 120 and 186 citations were selected from the comparison and copyediting groups, respectively. However, 3 citations from 3 separate articles (2 from the comparison and 1 from the copyediting group) could not be obtained through online databases, Google search, and/or performing a journal search. In total, 118 citations from the comparison group (an average of 7 citations/article) and 185 citations (an average of 10 citations/article) from the copyediting group were reviewed for major and minor content errors and style errors.
Article Characteristics The percentages of each document type and style were similar between the comparison and copyediting groups. With each group containing 18 articles, the comparison group had 13 articles (72.2%) and 4 reviews (22.2%), whereas the copyediting group had 15 articles (83.3%) and 3 reviews (16.7%). The majority reference style seen in both groups’ articles was APA and AMA. Both groups each had 8 articles (44.4%) with APA style, whereas the comparison group had 7 articles (38.9%) with AMA style and the copyediting group had 5 articles (27.8%) with AMA style. The remaining few articles in both groups used Vancouver, Harvard, or other styles. Reference Inaccuracy Major and Minor Content Errors There were significantly fewer total weighted major errors in the copyediting group (1 error/10 citations; median, 0.08) than in the comparison group (4 errors/10 citations; median, 0.37; U = 61, P = .001). Among these errors, only weighted volume, issue, year, or page number (U = 93, P = .029) errors were significant between the 2 groups. In contrast, weighted minor content errors were not significant between the comparison (median, 0.39) and copyediting (median, 0.20) groups (U = 126.5, P = .265). Reference Citation Style There were significantly fewer total weighted style errors in the copyediting group (1 error/10 citations; median, 0.07) than in the comparison group (3 errors/10 citations; median, 0.63; U = 66, P = .002). No difference in specific element errors was detected between the 2 groups. In both groups, more than one-half of the total weighted style errors involved the volume/issue element. The most common volume/issue error was the inclusion of the journal issue number when the issue is paginated by journal when using APA style or exclusion of the journal issue number when using AMA style.
Discussion The primary significant finding of this study is that fewer major content errors and style errors occurred in facultyNurse Educator
authored journal articles that were internally copyedited than in those articles that were not internally copyedited. To our knowledge, this current analysis is the first to provide evidence that faculty author–sought copyediting is associated with a significant decrease in reference inaccuracy. Two major factors, document type and style type, can be ruled out as contributing factors for these findings. Document type may influence the occurrence of reference inaccuracy because the reference list of review articles is likely to consist of more references than the reference list of databased or research articles.17 However, all journal document types (as covered in the Scopus citation database) were included in the analysis, and the percentages of document types were similar between the 2 groups. Therefore, document type is unlikely to have influenced the finding of fewer major and minor content errors in the copyediting group than in the comparison group. Reference citation style may also influence the occurrence of reference inaccuracy. Of all the styles, nursing faculty authors are highly familiar with the APA style because most US schools of nursing adopt this style for use at all levels of nursing education. For this reason, the required use of a different style to publish journal articles may yield more errors. However, APA style was the most commonly used style of both groups, and the percentage of faculty-authored publications that used APA style was similar between the copyediting and comparison groups. Therefore, articles in the comparison group were not published in more journals that required an unfamiliar style. Although copyediting was associated with reduced reference inaccuracy in the current study, copyediting did not significantly reduce major author errors. Major author errors were common citation errors detected in previous nursing, medicine, and social science investigations,19-22 and these errors were common in the current study too. Despite the lack of statistical significance in major author errors between the 2 groups in this study, the weighted error median was nearly 4-fold less in the copyediting group. The findings of this investigation suggest that copyediting may be a resource that a school of nursing can provide to promote the quality and intellectual rigor of faculty scholarship. A potential concern of providing copyediting is the costs. As indicated earlier, the copyeditor hourly rate ranges $30 to $50,15 which calculates to an annual salary of $60,000 to $100,000. For faculty who are expected to publish and earn a salary that exceeds that of a copyeditor, the use of a copyeditor is likely to be a cost-savings for a school of nursing because, then, these faculty will have more time to perform activities that more accurately reflect their expertise, such as teaching, supervising/mentoring students, conducting research, or engaging in university/school of nursing service. For faculty who are expected to publish and earn a salary similar to that of a copyeditor, the use of a copyeditor remains a cost-effective strategy because these faculty, even with the use of a reference manager software, are likely to require more time than a copyeditor to complete the copyediting. Also, a significant number of faculty lack the expertise as evidenced by the prevalence of reference inaccuracy.6,8 In the end, more faculty time devoted to reference accuracy may translate into a higher school of nursing cost because faculty are likely to be underperforming in this situation. Volume 38 & Number 6 & November/December 2013
Copyright © 2013 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
267
Besides cost-savings, a school of nursing’s use of a copyeditor for reference accuracy may lead to other gains. Because reference accuracy may affect overall quantity and quality of journal article publications, a copyeditor for this purpose may ultimately affect the school’s success in obtaining external funding and an outstanding performance evaluation by outside evaluators. These parameters are important to attract students and junior faculty to sustain a school of nursing in a time of scarce financial resources and nursing faculty shortage.
Limitations This analysis has 3 major limitations. One limitation is that author knowledge or skill in reference style or manuscript editing was not assessed or controlled. Only 2 of the 20 faculty had authored journal articles in both the comparison and copyediting groups. (A wave of faculty departures/retirements and hires occurred over the analysis period.) Therefore, the different authors in the copyediting group could have had greater knowledge or skill in reference styling or manuscript editing than those in the comparison group. Another limitation is that this analysis did not account for the role of the journal’s copyediting staff in ensuring reference accuracy. This information was not available. However, most journals have the policy that authors are responsible for reference accuracy, and based on the authors’ publishing and/or editing experience, a journal’s copyediting staff and even journal reviewers can identify obvious content or style errors. Interestingly, findings of this study indicate that the median style error of both groups is higher than the median major or minor content error of both groups. A final major limitation is that the number of errors may be underreported. Using Taylor’s18 criteria, the highest possible error score per element was 1. However, more than 1 error per element could have been present. Therefore, the error differences, including in minor errors, between the comparison and copyediting groups may have been greater.
Conclusion Despite recent technological advances such as online databases, easily accessible search engines, and bibliographic software, reference inaccuracy is still common in faculty scholarship. The findings of this analysis suggest that internal copyediting is effective in reducing reference inaccuracy and, therefore, ensuring high-quality faculty scholarship. Therefore, for an individual faculty, using a copyeditor, even for ensuring reference accuracy, is a career investment.23 There is a greater chance for publication in a journal that has a high rejection rate if the article is ‘‘well-edited.’’23 If publishing in high-impact journals can lead to faculty tenure and promotion, then a copyeditor can lead faculty toward earning more money.23 For a school of nursing, providing a copyeditor for faculty as a scholarship resource even for reference accuracy is an investment in developing faculty scholarship and creating and sustaining a high-quality academic environment.
268
Volume 38 & Number 6 & November/December 2013
References 1. Johnson AM. Charting a course for a successful research career. A guide for early career researchers. 2011. Available at http:// cdn.elsevier.com/assets/pdf_file/0019/111835/charting-a-coursefor-a-successful-research-career.pdfc. Accessed August 2, 2013. 2. Maddux CD, Liu L. Publishing research findings: some suggestions for junior faculty. Int J Technol Teach Learn. 2005;1:55-62. 3. Dewett T, Denisi AS. Exploring scholarly reputation: it’s more than just productivity. Scientometrics. 2004;60:249-272. 4. Ioannidis JPA, Patsopoulos NA, Kavvoura FK, et al. International ranking systems for universities and institutions: a critical appraisal. BMC Med. 2007;5:30. 5. American Psychological Association. The Publication Manual of the American Psychological Association. 6th ed. Washington, DC: American Psychological Association; 2009. 6. Bahar Z, Beser A, Elcigil A, et al. Accuracy of references in eight nursing journal. HealthMED. 2012;6:2066-2073. 7. Lok CKW, Chan MTV, Martinson IM. Risk factors for citation errors in peer-reviewed nursing journals. J Adv Nurs. 2001;34:223-229. 8. Oermann MH, Cummings SL, Wilmes NA. Accuracy of references in four pediatric nursing journals. J Pediatr Nurs. 2001; 16:263-268. 9. Oermann MH, Wilmes NA. How accurate are references in nursing journals? Nurse Author Ed. 2005;15(4):1-4. 10. Oermann MH, Ziolkawski LD. Accuracy of references in three critical care nursing journals. J Perianesth Nurs. 2002;17:78-83. 11. Schulmeister L. Quotation and reference accuracy of three nursing journals. Image J Nurs Scholarsh. 1998;30:143-146. 12. Oncology Nursing Society. Oncology Nursing Forum. Author information. Available at http://www.ons.org/Publications/ONF/ AuthorInfo. Accessed August 2, 2013. 13. Smith CM, Baker B. Technology in nursing scholarship: use of citation reference managers. Int J Ment Health Nurs. 2007;16: 156-160. 14. Meyer CA. Reference accuracy: best practices for making the links. J Electron Publish. 2008;11(2). doi: http://dx.doi.org/10 .3998/3336451.0011.20. 15. Brenner E. Tip of the week: what a copyeditor earns. 2012. Available at http://www.copyediting.com/tip-week-what-copyeditorearns. Accessed August 2, 2013. 16. SciVerse. Coverage of metadata. 2013. Available at http://www .info.sciverse.com/scopus/scopus-in-detail/content-coverageguide/metadata. Accessed August 2, 2013. 17. Abt HA, Garfield E. Is the relationship between numbers of references and paper lengths the same for all sciences? J Am Soc Inform Sci Technol. 2002;53:1106-1112. 18. Taylor MK. The practical effects of errors in reference lists in nursing research journals. Nurs Res. 1998;47:300-303. 19. Adhikari P. Accuracy of references in Indian Journal of Otolaryngology and Head & Neck Surgery. Indian J Otolaryngol Head Neck Surg. 2010;62:338-341. 20. Al-Benna S, Rajgarhia P, Ahmed S, Sheikh Z. Accuracy of references in burns journals. Burns. 2009;35:677-680. 21. Awrey J, Inaba K, Barmparas G, et al. Reference accuracy in the general surgery literature. World J Surg. 2011;35:475-479. 22. Narin S, Ko0ak F, Ozalevli S, Ilgin D. Reference accuracy in four respiratory medical journals. Respiration. 2010;80:171-172. 23. Belcher W. Academic copyediting. 2011. Available at http://www .wendybelcher.com/pages/ccopyeditingrates.html. Accessed August 2, 2013.
Nurse Educator
Copyright © 2013 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Nurse Educator
Effectiveness of a Reference Accuracy Strategy for Peer-Reviewed Journal Articles Kirsten E. Speck, MPH, BA & Barbara St. Pierre Schneider, DNSc, RN Dissemination of information through peer-reviewed journal articles is an important requirement of success in academia. Despite the importance of publishing articles, about 25% to 45% of articles published in nursing journals have at least 1 reference error in the reference list. The authors discuss the implementation and outcomes of an internal copyeditor strategy aimed at reducing reference inaccuracy in faculty-authored journal articles.
S
cholarly publishing is 1 of the most important requirements to be successful in academia and a common criterion upon which academic tenure and promotion is based.1,2 It is also a mechanism by which faculty achieve a scholarly reputation.3 Scholarly publishing contributes to a university’s overall performance. For example, both the QS World University Rankings (http://www.topuniversities .com/university-rankings/world-university-rankings/2012) and Academic Ranking of World Universities (http://www .arwu.org/aboutARWU.jsp) consider faculty publications, including number of citations per faculty and number of articles indexed in a scientific citation database, as indicators of performance. In a competitive higher education market, a university with a strong record of scholarly publishing is likely to garner a strong rating of performance and attract highquality junior faculty and students as well as external funding.4 One component of scholarly publishing is ‘‘crediting sources.’’5 Crediting sources includes accurately quoting or paraphrasing content and recording the citation of the sources. Although the latter may seem insignificant, reference accuracy and reference inaccuracy of published articles have implications. For example, reference accuracy aids in establishing one’s reputation as a ‘‘careful researcher.’’5 In contrast, reference inaccuracy may have negative implications such as calling into question the accuracy of other content in the paper. Reference inaccuracy is common in articles published in nursing. Approximately 25% to 45% of articles published in nursing journals have at least 1 error in the reference list.6-11 These errors may be major, such as incorrect author names or article title, or minor, such as additional or omitted author
Author Affiliations: Research Technician (Ms Speck), Associate Professor (Dr St. Pierre Schneider), School of Nursing, University of Nevada, Las Vegas. The authors declare no conflicts of interest. Correspondence: Ms Speck, School of Nursing, University of Nevada, Las Vegas, BHS 100, 4505 S Maryland Pkwy, Box 453018, Las Vegas, NV 89154-3018 ([email protected]). DOI: 10.1097/01.NNE.0000435272.47774.51
Nurse Educator
initials, or minor misspellings and subtitle omission of the article title.8 These errors can result in 2 major consequences. First, the reader may be unable to retrieve the citation.6 Today’s retrieval of published material relies primarily on the use of electronic reference options (ie, World Wide Web, literature databases), and these options require accurate information of reference elements (eg, author name) to yield results. Second, the review or publication of the manuscript may be delayed or rejected as journal reviewers and editors may question reported research results or other details.9,12 To reduce reference inaccuracy of journal articles, most journals provide specific information and examples for authors and inform authors that reference accuracy is their responsibility. Citation styles also indicate that reference accuracy is the authors’ responsibility.5 There are various strategies that authors can use to ensure reference accuracy. One recommended strategy is to use a reference manager software.13 A copy of a reference manager software program may be free to a faculty author or cost $100 to $200. Although many of these programs allow citation information to be imported directly from an online scientific citation database, this activity still requires attention to detail. For example, the PubMed scientific citation database provides the journal title in the form of the journal abbreviation. If the reference citation style of an author’s chosen journal requires the journal full name, the author has to manually insert the journal full name into the program. Another strategy for authors is to work with a copyeditor. According to Meyer,14 ‘‘A good copyeditorIcan catch a majority of reference errors.’’ On the surface, the use of a copyeditor appears to be a more costly strategy than faculty use of reference manager software ($30-$50/h15 vs a 1-time cost of $100-$200); however, many faculty do not have copyeditor skills and faculty time is limited and expensive too. In the long run, the use of a copyeditor may be the best strategy for faculty and university performance. At our school of nursing, we adopted an internal copyeditor strategy for faculty authors. This service was provided Volume 38 & Number 6 & November/December 2013
Copyright © 2013 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
265
to all faculty and consisted of correcting grammar and punctuation and minor reference style (American Psychological Association [APA] style in our case), formatting errors, and proofing.
Methods This analysis was performed retrospectively. A literature search was performed using PubMed and Scopus databases to locate journal articles authored by faculty at our school of nursing. Keywords were (a) author affiliation, (b) subject area in nursing, and (c) articles published from January 1, 2000, to September 21, 2012. Only journal articles that were peer reviewed and in which the faculty was listed as first author were used in the analysis. Exclusion criteria were journal articles that (a) contained fewer than 5 citations, (b) were not retrievable because of in press status, and/or (c) were not retrievable as a PDF version. Once articles were identified, and therefore citations, the articles were separated into a copyediting or comparison group. Articles assigned to the copyediting group were those published between January 1, 2005, and September 21, 2012, and used an internal (school of nursing) copyediting service. This service consisted of copyediting by staff who had editing experience and were knowledgeable in using reference manager software. In contrast, those articles assigned to the comparison group were those published during the time period 2000-2004, before this internal copyediting was available to faculty authors. Journal citations in the reference list of articles in both groups underwent review for errors. Web pages and books in the reference list were excluded because of lack of accessibility. For articles that had 10 or fewer journal citations, all journal citations were included in the analysis. For articles that had more than 10 journal citations, the first journal citation followed by every third citation was selected.
Article Characteristics Two characteristics of the analyzed faculty authors’ journal articles were identified. The first was assigning each article a document type based on the document types covered in Scopus.16 Therefore, analyzed articles were (a) article (opinion or original research), (b) review (nonoriginal research), or (c) other (ie, editorial, note). This information was collected because review articles are likely to consist of more references than data-based or research articles.17 Another characteristic was the reference citation style of the journal in which the articles were published. This characteristic was examined because APA is the uniformed formatting style used at most US schools of nursing; however, the required use of a different style to publish journal articles may yield more errors. The journal’s current author guidelines were the source of the style and verified by the articles. Four standard styles were identified: (a) American Medical Association (AMA); (b) APA and favored style used by the social and behavioral sciences; (c) Vancouver (similar to AMA), popular among the physical sciences; and (d) Harvard (similar to APA). An ‘‘other’’ category was used when the style could not be identified through the journal’s author guidelines or by inspection. Also, faculty-authored articles were classified in this ‘‘other’’ category if the journal used a modified version of 1 of the 4 standard styles. 266
Volume 38 & Number 6 & November/December 2013
Reference Inaccuracy Once the article characteristics were identified, the journal citations in each article’s reference list were reviewed for 3 types of reference inaccuracy: major and minor content errors and style errors. Major and Minor Content Errors Based on a modification of the approach of Oermann et al,8 major content errors were (a) author errors in which author names were misspelled, omitted, or ordered incorrectly or author initials were incorrect; (b) article title errors, where incorrect article titles were provided; (c) incorrect journal or incorrect spelling/abbreviation of the journal title; and (d) incorrect or omitted volume, issue, year, or page number/s. Minor content errors were (a) omitted or additional author initials as listed on the article, (b) article title misspellings and omissions of subtitles, (c) added or omitted words to journal title, and (d) incomplete list of page numbers.8 Based on Taylor’s18 criteria, the scoring options for each element of a citation were 0 for no errors or 1 for 1 or more errors per element. For example, if there were 2 errors in the author name, 1 error was counted. Because of differences in the number of citations reviewed between the 2 groups, the total number of errors per faculty-authored publication was divided by the total number of citations reviewed within that article to yield a weighted total error value. For example, if article 1 has 18 citations and a raw total of 7 major errors, the weighted total error value is 7 18 = 0.39 for article 1. To spot check the scoring, a second reviewer independently scored the major and minor content errors of citations from a random selection of 3 articles from each group. The second reviewer selected the citations as described above and compared the faculty author’s version with a paper copy of the article and/or through Scopus, PubMed, or EBSCOhost databases. In total, the second reviewer checked 30 and 28 citations from the comparison and copyediting groups, respectively, and the first and second reviewer agreement was 96%.
Citation Style Errors The journal citations were also assessed for style errors. Reference citation errors were categorized into the 6 elements of the journal reference format: (a) author errors were incorrect spacing, punctuation, and/or number of listed authors; (b) year errors were inclusion of month of publication; (c) article title errors were incorrect punctuation and/or grammar such as a comma instead of a period after the article title; (d) journal title errors were journal abbreviation instead of full journal name or incorrect punctuation; (e) volume/issue errors were incorrect punctuation related to the volume and journal pagination; and (f ) pages errors were incorrect spacing or page number formatting. To spot check this scoring, the second reviewer also independently scored reference style citation errors using the same citations for the content error spot check. The first and second reviewer agreement was 98%. Statistical Analysis Data were analyzed using IBM SPSS Statistics 19 (IBM Corporation, Armonk, New York). Frequencies and percentages were used to summarize document type and style. The Mann-Whitney U test was performed to compare the median Nurse Educator
Copyright © 2013 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
major and minor content errors and style errors between the comparison and copyediting groups. Alpha was set at .05.
Results A literature search retrieved 21 faculty-authored publications in the comparison group (8 faculty authors) and 18 facultyauthored publications in the copyediting group (12 faculty authors). Based on the exclusion criteria, 3 articles from the comparison group were removed from the analysis. A total of 36 peer-reviewed articles (n = 18 articles per group) were reviewed in this analysis. From these 36 articles, 120 and 186 citations were selected from the comparison and copyediting groups, respectively. However, 3 citations from 3 separate articles (2 from the comparison and 1 from the copyediting group) could not be obtained through online databases, Google search, and/or performing a journal search. In total, 118 citations from the comparison group (an average of 7 citations/article) and 185 citations (an average of 10 citations/article) from the copyediting group were reviewed for major and minor content errors and style errors.
Article Characteristics The percentages of each document type and style were similar between the comparison and copyediting groups. With each group containing 18 articles, the comparison group had 13 articles (72.2%) and 4 reviews (22.2%), whereas the copyediting group had 15 articles (83.3%) and 3 reviews (16.7%). The majority reference style seen in both groups’ articles was APA and AMA. Both groups each had 8 articles (44.4%) with APA style, whereas the comparison group had 7 articles (38.9%) with AMA style and the copyediting group had 5 articles (27.8%) with AMA style. The remaining few articles in both groups used Vancouver, Harvard, or other styles. Reference Inaccuracy Major and Minor Content Errors There were significantly fewer total weighted major errors in the copyediting group (1 error/10 citations; median, 0.08) than in the comparison group (4 errors/10 citations; median, 0.37; U = 61, P = .001). Among these errors, only weighted volume, issue, year, or page number (U = 93, P = .029) errors were significant between the 2 groups. In contrast, weighted minor content errors were not significant between the comparison (median, 0.39) and copyediting (median, 0.20) groups (U = 126.5, P = .265). Reference Citation Style There were significantly fewer total weighted style errors in the copyediting group (1 error/10 citations; median, 0.07) than in the comparison group (3 errors/10 citations; median, 0.63; U = 66, P = .002). No difference in specific element errors was detected between the 2 groups. In both groups, more than one-half of the total weighted style errors involved the volume/issue element. The most common volume/issue error was the inclusion of the journal issue number when the issue is paginated by journal when using APA style or exclusion of the journal issue number when using AMA style.
Discussion The primary significant finding of this study is that fewer major content errors and style errors occurred in facultyNurse Educator
authored journal articles that were internally copyedited than in those articles that were not internally copyedited. To our knowledge, this current analysis is the first to provide evidence that faculty author–sought copyediting is associated with a significant decrease in reference inaccuracy. Two major factors, document type and style type, can be ruled out as contributing factors for these findings. Document type may influence the occurrence of reference inaccuracy because the reference list of review articles is likely to consist of more references than the reference list of databased or research articles.17 However, all journal document types (as covered in the Scopus citation database) were included in the analysis, and the percentages of document types were similar between the 2 groups. Therefore, document type is unlikely to have influenced the finding of fewer major and minor content errors in the copyediting group than in the comparison group. Reference citation style may also influence the occurrence of reference inaccuracy. Of all the styles, nursing faculty authors are highly familiar with the APA style because most US schools of nursing adopt this style for use at all levels of nursing education. For this reason, the required use of a different style to publish journal articles may yield more errors. However, APA style was the most commonly used style of both groups, and the percentage of faculty-authored publications that used APA style was similar between the copyediting and comparison groups. Therefore, articles in the comparison group were not published in more journals that required an unfamiliar style. Although copyediting was associated with reduced reference inaccuracy in the current study, copyediting did not significantly reduce major author errors. Major author errors were common citation errors detected in previous nursing, medicine, and social science investigations,19-22 and these errors were common in the current study too. Despite the lack of statistical significance in major author errors between the 2 groups in this study, the weighted error median was nearly 4-fold less in the copyediting group. The findings of this investigation suggest that copyediting may be a resource that a school of nursing can provide to promote the quality and intellectual rigor of faculty scholarship. A potential concern of providing copyediting is the costs. As indicated earlier, the copyeditor hourly rate ranges $30 to $50,15 which calculates to an annual salary of $60,000 to $100,000. For faculty who are expected to publish and earn a salary that exceeds that of a copyeditor, the use of a copyeditor is likely to be a cost-savings for a school of nursing because, then, these faculty will have more time to perform activities that more accurately reflect their expertise, such as teaching, supervising/mentoring students, conducting research, or engaging in university/school of nursing service. For faculty who are expected to publish and earn a salary similar to that of a copyeditor, the use of a copyeditor remains a cost-effective strategy because these faculty, even with the use of a reference manager software, are likely to require more time than a copyeditor to complete the copyediting. Also, a significant number of faculty lack the expertise as evidenced by the prevalence of reference inaccuracy.6,8 In the end, more faculty time devoted to reference accuracy may translate into a higher school of nursing cost because faculty are likely to be underperforming in this situation. Volume 38 & Number 6 & November/December 2013
Copyright © 2013 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
267
Besides cost-savings, a school of nursing’s use of a copyeditor for reference accuracy may lead to other gains. Because reference accuracy may affect overall quantity and quality of journal article publications, a copyeditor for this purpose may ultimately affect the school’s success in obtaining external funding and an outstanding performance evaluation by outside evaluators. These parameters are important to attract students and junior faculty to sustain a school of nursing in a time of scarce financial resources and nursing faculty shortage.
Limitations This analysis has 3 major limitations. One limitation is that author knowledge or skill in reference style or manuscript editing was not assessed or controlled. Only 2 of the 20 faculty had authored journal articles in both the comparison and copyediting groups. (A wave of faculty departures/retirements and hires occurred over the analysis period.) Therefore, the different authors in the copyediting group could have had greater knowledge or skill in reference styling or manuscript editing than those in the comparison group. Another limitation is that this analysis did not account for the role of the journal’s copyediting staff in ensuring reference accuracy. This information was not available. However, most journals have the policy that authors are responsible for reference accuracy, and based on the authors’ publishing and/or editing experience, a journal’s copyediting staff and even journal reviewers can identify obvious content or style errors. Interestingly, findings of this study indicate that the median style error of both groups is higher than the median major or minor content error of both groups. A final major limitation is that the number of errors may be underreported. Using Taylor’s18 criteria, the highest possible error score per element was 1. However, more than 1 error per element could have been present. Therefore, the error differences, including in minor errors, between the comparison and copyediting groups may have been greater.
Conclusion Despite recent technological advances such as online databases, easily accessible search engines, and bibliographic software, reference inaccuracy is still common in faculty scholarship. The findings of this analysis suggest that internal copyediting is effective in reducing reference inaccuracy and, therefore, ensuring high-quality faculty scholarship. Therefore, for an individual faculty, using a copyeditor, even for ensuring reference accuracy, is a career investment.23 There is a greater chance for publication in a journal that has a high rejection rate if the article is ‘‘well-edited.’’23 If publishing in high-impact journals can lead to faculty tenure and promotion, then a copyeditor can lead faculty toward earning more money.23 For a school of nursing, providing a copyeditor for faculty as a scholarship resource even for reference accuracy is an investment in developing faculty scholarship and creating and sustaining a high-quality academic environment.
268
Volume 38 & Number 6 & November/December 2013
References 1. Johnson AM. Charting a course for a successful research career. A guide for early career researchers. 2011. Available at http:// cdn.elsevier.com/assets/pdf_file/0019/111835/charting-a-coursefor-a-successful-research-career.pdfc. Accessed August 2, 2013. 2. Maddux CD, Liu L. Publishing research findings: some suggestions for junior faculty. Int J Technol Teach Learn. 2005;1:55-62. 3. Dewett T, Denisi AS. Exploring scholarly reputation: it’s more than just productivity. Scientometrics. 2004;60:249-272. 4. Ioannidis JPA, Patsopoulos NA, Kavvoura FK, et al. International ranking systems for universities and institutions: a critical appraisal. BMC Med. 2007;5:30. 5. American Psychological Association. The Publication Manual of the American Psychological Association. 6th ed. Washington, DC: American Psychological Association; 2009. 6. Bahar Z, Beser A, Elcigil A, et al. Accuracy of references in eight nursing journal. HealthMED. 2012;6:2066-2073. 7. Lok CKW, Chan MTV, Martinson IM. Risk factors for citation errors in peer-reviewed nursing journals. J Adv Nurs. 2001;34:223-229. 8. Oermann MH, Cummings SL, Wilmes NA. Accuracy of references in four pediatric nursing journals. J Pediatr Nurs. 2001; 16:263-268. 9. Oermann MH, Wilmes NA. How accurate are references in nursing journals? Nurse Author Ed. 2005;15(4):1-4. 10. Oermann MH, Ziolkawski LD. Accuracy of references in three critical care nursing journals. J Perianesth Nurs. 2002;17:78-83. 11. Schulmeister L. Quotation and reference accuracy of three nursing journals. Image J Nurs Scholarsh. 1998;30:143-146. 12. Oncology Nursing Society. Oncology Nursing Forum. Author information. Available at http://www.ons.org/Publications/ONF/ AuthorInfo. Accessed August 2, 2013. 13. Smith CM, Baker B. Technology in nursing scholarship: use of citation reference managers. Int J Ment Health Nurs. 2007;16: 156-160. 14. Meyer CA. Reference accuracy: best practices for making the links. J Electron Publish. 2008;11(2). doi: http://dx.doi.org/10 .3998/3336451.0011.20. 15. Brenner E. Tip of the week: what a copyeditor earns. 2012. Available at http://www.copyediting.com/tip-week-what-copyeditorearns. Accessed August 2, 2013. 16. SciVerse. Coverage of metadata. 2013. Available at http://www .info.sciverse.com/scopus/scopus-in-detail/content-coverageguide/metadata. Accessed August 2, 2013. 17. Abt HA, Garfield E. Is the relationship between numbers of references and paper lengths the same for all sciences? J Am Soc Inform Sci Technol. 2002;53:1106-1112. 18. Taylor MK. The practical effects of errors in reference lists in nursing research journals. Nurs Res. 1998;47:300-303. 19. Adhikari P. Accuracy of references in Indian Journal of Otolaryngology and Head & Neck Surgery. Indian J Otolaryngol Head Neck Surg. 2010;62:338-341. 20. Al-Benna S, Rajgarhia P, Ahmed S, Sheikh Z. Accuracy of references in burns journals. Burns. 2009;35:677-680. 21. Awrey J, Inaba K, Barmparas G, et al. Reference accuracy in the general surgery literature. World J Surg. 2011;35:475-479. 22. Narin S, Ko0ak F, Ozalevli S, Ilgin D. Reference accuracy in four respiratory medical journals. Respiration. 2010;80:171-172. 23. Belcher W. Academic copyediting. 2011. Available at http://www .wendybelcher.com/pages/ccopyeditingrates.html. Accessed August 2, 2013.
Nurse Educator
Copyright © 2013 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
