EDITORIAL
Peer review: the unsung heroes
F. S. Haddad From The British Editorial Society of Bone & Joint Surgery, London, United Kingdom
F. S. Haddad, MD(Res), MCh(Orth), FRCS(Orth), Professor of Orthopaedic Surgery, Editor-in-Chief The Bone & Joint Journal, 22 Buckingham Street, London, WC2N 6ET, UK. Correspondence should be sent to Professor F. S. Haddad: [email protected] ©2014 The British Editorial Society of Bone & Joint Surgery doi:10.1302/0301-620X.96B4. 33703 $2.00 Bone Joint J 2014;96-B:433–5. Received 13 March 2014; Accepted after revision 13 March 2014
VOL. 96-B, No. 4, APRIL 2014
Research is fundamental to clinical governance.1 Without high quality research, education, training, clinical effectiveness and audit would stagnate, having no evidence to base practice on, nor improve it. Consequently, among the core fundamental principles which underlie all research work are accuracy and legitimacy. Much of what is published in the name of research is false for a number of reasons, including faulty study design, bias and the nature of the research question to be addressed.2 Even in the orthopaedic literature, less than 60% of research published conforms to adequate standards of quality in either study design or statistical analysis.3 Poor research can have a profound effect on orthopaedic practice. Erroneous conclusions and recommendations may lead to inappropriate clinical interventions, particularly when published in the name of high profile clinicians in highly regarded journals. Conversely, good quality data and outcomes allow improvements in the quality of treatment.3 The clarity and integrity of the research which is published must be maintained if the orthopaedic community is to flourish. But where does the responsibility and the accountability lie? In the same way that we need continually to re-evaluate ideas and evidence by repeating studies to confirm the findings and by undertaking systematic reviews and meta-analyses, we need to re-evaluate our practice in reviewing and re-writing papers in order to maintain high quality. Peer review has become such an important tool in our learning, and the confirmation of “scientific facts” that the eighth report of the UK House of Commons’ Science and Technology Committee focuses on the process, noting the “importance of evidence-based scientific information to government”.4 The principle of review by a panel of peers is not new. It has been used in all walks of life as far back as the introduction of trial by a jury of peers in the legal system in Ancient Greece.5 It was introduced in scientific work five years after the Royal Society was founded (1660) as
‘pre-publication’ review, by Henry Oldenberg,6 the editor of the first scientific journal, The Philosophical Transactions of the Royal Society. Later, in the 18th century, the journal established a “Committee on papers”, reviewing all submitted papers,7 creating the second peer reviewed journal. The first was that of The Journal of Medical Essays and Observations8 of the Royal Society of Edinburgh, which was first published a few years earlier. It took until the middle of the 20th century, however, before peer review became essential to scientific knowledge and research.7 With the passage of time, it has become increasingly important to the wealth and breadth of knowledge.8 In this edition, Twaij et al9 eloquently describe recent developments and potential future strategies for peer review. The basic premise of peer review is to evaluate and improve the quality of papers by enrolling specialist reviewers who are forerunners in the field of study to review, criticise and suggest improvements. The process has been described as “the evaluation of scientific research findings or proposals for competence, significance and originality, by qualified experts who research and submit work for publication in the same field (peers)”.10 The process is now embedded, well beyond the assessment of journal submissions, in every step of the scientific method, including the evaluation of funding applications and the assessment of the quality of work undertaken by individuals, teams or departments.11 Fundamentally, the responsibility for the integrity of a piece of work must remain with the authors, and they must carefully check the quality and accuracy of their work following review. It must be assumed that research is undertaken in orthopaedic surgery with the primary intention of improving patient care. The submission of a manuscript must always be accompanied by a signed document confirming the originality of the content and that all the authors have contributed to it and checked it for accuracy and completeness. As the General Medical Council 433
434
F. S. HADDAD
guidelines state: “You must make sure that your conduct at all times justifies your patients’ trust in you and the public’s trust in the profession”.12 Clinicians must maintain probity and decency in everything, including research, given the potential adverse effects of false information. Thus reviewers also have a responsibility to maintain quality. Indeed, in my short tenure as Editor-in-chief of The Bone & Joint Journal (BJJ), our excellent reviewers have uncovered research fraud, an attempt at dual publication and several attempts to “salami slice” data into multiple publications. We should be grateful for their vigilance. Reviewing is sometimes regarded as a thankless task and can be an extra burden on busy clinicians. The clinical and scientific communities, however, need to be supported by the best available evidence, as determined by the highest standards of reviewing of the material which is submitted to journals. The role played by the the editorial board of the BJJ, our specialty editors and in particular by our reviewers cannot be overstated. They underpin the quality which we endeavour to deliver. Such roles need to be recognised at the national and international levels in job plans, appraisal and clinical excellence awards and should be integral to career progression. It is also important for the scientific community and journals to remain up to date and embrace new technologies. The BJJ website goes from strength to strength, as also does our sister open access online journal, Bone & Joint Research. Our reviewing process and rewriting are all undertaken online, and the interactions with our international readership are largely electronic. Moreover, we continue to develop a more interactive subspecialty international platform, using all forms of current technology, via the web, tablets and smartphone apps. There is a considerable incentive from within the scientific community and government to embrace new technologies. The recent House of Commons Committee Report, referred to above, recommends experimenting with social networking tools as a means of developing and improving the dissemination of scientific information, and this is being explored by some groups.4,14,15 I have been asked by many whether we intend to develop and improve on our peer review with open systems. Although such improvements might allow a quicker publication and distribution of knowledge with a robust platform for the delegation of labour, which classical peer review cannot, it is not currently on our agenda. Our turnaround times remain rapid with the time interval between acceptance to publication remaining usually less than four months. There have been interesting trials and projects in the use of open review systems in the scientific communities, such as arXiv,14 Nature15 and Cureus.16 The drawbacks include the concern that opening comments to all would result in unhelpful suggestions or empty praise, or that useful suggestions may end up being buried under large numbers of mediocre, poorly thought out comments or instant reactions.17 There are other risks associated with community
involvement where occasionally ‘herd mentality’ can be observed. The recent witch-hunt of the Boston bomber in 2013 highlights these dangers. Online communities thought that they recognised the picture of the bomber as Ivy League student Sunil Tripathi, who was vilified and whose parents were harassed. The student was found a week later, dead.18 We are mindful of the evolution towards more transparency in the review process and experiments with open review systems and will maintain a watching brief in that area. For the time being, we will continue to respect our readers’ and reviewers’ wishes and to evolve our existing processes. The BJJ takes great pride in our peer review process which has developed over several decades. We are fortunate to have gained access to and developed strong relationships with many of the best and most committed reviewers in the world and will continue to work with them. We also have an international Editorial Board that provides leadership and direction in all the orthopaedic specialties. The BJJ will continue to try to lead the way with its current review system and place emphasis on the education of those who wish to review through our reviewer days here at the journal offices and through presentations at conferences. Over the past year we have presented at EFORT and the British Orthopaedic Association Annual Meeting. In 2014, we will be presenting an Instructional Course at the American Academy of Orthopaedic Surgery Annual Meeting, at EFORT and at the Royal Society of Medicine on 6th May. We are currently introducing some stepwise improvements rather than planning radical change. In order to enhance the experience both for our reviewers and authors, we have appointed specialty editors who will ensure that appropriate expertise is brought to bear in their subspecialty areas. They will ensure that manuscripts are sent to the most appropriate reviewers and will facilitate the interaction with authors. To this end I will be ably supported by: Elliot Sorene (wrist and hand) James Calder (foot and ankle) Fergall Monsell (children’s orthopaedics) Sam Patton (oncology and knee) James Waddell (hip) Matt Costa (trauma) In the interim, I hope that you will all join me in saluting the heavy burden carried by our reviewers and the essential work that they undertake. The unsung heroes of the publishing process are the reviewers. We owe them a massive debt.
References 1. Pringle M. Participating in clinical governance. BMJ 2000;321:737–740. 2. Ioannidis JPA. Why most published research findings are false. PLoS Med 2005;2:124. 3. Parsons NR, Hiskens R, Price CL, Achten J, Costa ML. A systematic survey of the quality of research reporting in general orthopaedic journals. J Bone Joint Surg [Br] 2011;93-B:1154–1159. THE BONE & JOINT JOURNAL
PEER REVIEW: THE UNSUNG HEROES
4. No authors listed. House of Commons. Science and Technology Committee: peer review in scientific publications, 2011:105. http://www.publications.parliament.uk/pa/ cm201012/cmselect/cmsctech/856/85602.htm (date last accessed 14 February 2014). 5. Samons LJ, Samons LJ. The Cambridge companion to the age of pericles. Cambridge: Cambridge University Press, 2007. 6. No authors listed. Wikipedia: Peer Review. https://en.wikipedia.org/w/ index.php?title=Peer_review&oldid=549967581 (date last accessed 14 February 2014). 7. No authors listed. Rescuing science from politics: regulation and the distortion of scientific research. Cambridge: Cambridge University Press, 2006:304. 8. Benos DJ, Bashari E, Chaves JM, et al. The ups and downs of peer review. Adv Physiol Educ 2007;31:145–152. 9. Twaij H, Oussedik S, Hoffmeyer P. Peer review. Bone Joint J 2014;96-B:436–441. 10. No authors listed. Sense about science: peer review and the acceptance of new scientific ideas, 2004:61 http://www.senseaboutscience.org/data/files/resources/17/peerReview.pdf (date last accessed 14 February 2014). 11. No authors listed. Research Information Network. Peer review: a guide for researchers, 2010:16. http://www.rin.ac.uk/our-work/communicating-and-disseminatingresearch/peer-review-guide-researchers (date last accessed 14 February 2014).
VOL. 96-B, No. 4, APRIL 2014
435
12. No authors listed. General Medical Council. Good medical practice: being honest and trustworthy, 2013. http://www.gmc-uk.org/static/documents/content/ GMP_2013.pdf_51447599.pdf (date last accessed 14 February 2014). 13. No authors listed. Urban Times. Encyclopaedia Britannica vs Wikipedia: a battle already lost? 2012. http://urbantimes.co/2012/03/encyclopaedia-britannica-vs-wikipedia-a-battle-already-lost-infographic/ (date last accessed 14 February 2014). 14. No authors listed. Wikipedia: arXiv. https://en.wikipedia.org/wiki/ArXiv (date last accessed 14 February 2014). 15. No authors listed. Nature. Overview: Nature’s peer review trial, 2006. http:// www.nature.com/nature/peerreview/debate/nature05535.html (date last accessed 14 February 2014). 16. No authors listed. Cureus. http://www.cureus.com (date last accessed 14 February 2014). 17. Ware M. Peer review: benefits, perceptions and alternatives. London: Publishing Research Consortium, 2008:1–20. 18. Buncombe A. The Independent. Sunil Tripathi: the other victim of Boston’s bombs, 2013. http://www.independent.co.uk/news/world/americas/sunil-tripathi-the-othervictim-of-bostons-bombs-8585250.html (date last accessed 14 February 2014).
Peer review: the unsung heroes
F. S. Haddad From The British Editorial Society of Bone & Joint Surgery, London, United Kingdom
F. S. Haddad, MD(Res), MCh(Orth), FRCS(Orth), Professor of Orthopaedic Surgery, Editor-in-Chief The Bone & Joint Journal, 22 Buckingham Street, London, WC2N 6ET, UK. Correspondence should be sent to Professor F. S. Haddad: [email protected] ©2014 The British Editorial Society of Bone & Joint Surgery doi:10.1302/0301-620X.96B4. 33703 $2.00 Bone Joint J 2014;96-B:433–5. Received 13 March 2014; Accepted after revision 13 March 2014
VOL. 96-B, No. 4, APRIL 2014
Research is fundamental to clinical governance.1 Without high quality research, education, training, clinical effectiveness and audit would stagnate, having no evidence to base practice on, nor improve it. Consequently, among the core fundamental principles which underlie all research work are accuracy and legitimacy. Much of what is published in the name of research is false for a number of reasons, including faulty study design, bias and the nature of the research question to be addressed.2 Even in the orthopaedic literature, less than 60% of research published conforms to adequate standards of quality in either study design or statistical analysis.3 Poor research can have a profound effect on orthopaedic practice. Erroneous conclusions and recommendations may lead to inappropriate clinical interventions, particularly when published in the name of high profile clinicians in highly regarded journals. Conversely, good quality data and outcomes allow improvements in the quality of treatment.3 The clarity and integrity of the research which is published must be maintained if the orthopaedic community is to flourish. But where does the responsibility and the accountability lie? In the same way that we need continually to re-evaluate ideas and evidence by repeating studies to confirm the findings and by undertaking systematic reviews and meta-analyses, we need to re-evaluate our practice in reviewing and re-writing papers in order to maintain high quality. Peer review has become such an important tool in our learning, and the confirmation of “scientific facts” that the eighth report of the UK House of Commons’ Science and Technology Committee focuses on the process, noting the “importance of evidence-based scientific information to government”.4 The principle of review by a panel of peers is not new. It has been used in all walks of life as far back as the introduction of trial by a jury of peers in the legal system in Ancient Greece.5 It was introduced in scientific work five years after the Royal Society was founded (1660) as
‘pre-publication’ review, by Henry Oldenberg,6 the editor of the first scientific journal, The Philosophical Transactions of the Royal Society. Later, in the 18th century, the journal established a “Committee on papers”, reviewing all submitted papers,7 creating the second peer reviewed journal. The first was that of The Journal of Medical Essays and Observations8 of the Royal Society of Edinburgh, which was first published a few years earlier. It took until the middle of the 20th century, however, before peer review became essential to scientific knowledge and research.7 With the passage of time, it has become increasingly important to the wealth and breadth of knowledge.8 In this edition, Twaij et al9 eloquently describe recent developments and potential future strategies for peer review. The basic premise of peer review is to evaluate and improve the quality of papers by enrolling specialist reviewers who are forerunners in the field of study to review, criticise and suggest improvements. The process has been described as “the evaluation of scientific research findings or proposals for competence, significance and originality, by qualified experts who research and submit work for publication in the same field (peers)”.10 The process is now embedded, well beyond the assessment of journal submissions, in every step of the scientific method, including the evaluation of funding applications and the assessment of the quality of work undertaken by individuals, teams or departments.11 Fundamentally, the responsibility for the integrity of a piece of work must remain with the authors, and they must carefully check the quality and accuracy of their work following review. It must be assumed that research is undertaken in orthopaedic surgery with the primary intention of improving patient care. The submission of a manuscript must always be accompanied by a signed document confirming the originality of the content and that all the authors have contributed to it and checked it for accuracy and completeness. As the General Medical Council 433
434
F. S. HADDAD
guidelines state: “You must make sure that your conduct at all times justifies your patients’ trust in you and the public’s trust in the profession”.12 Clinicians must maintain probity and decency in everything, including research, given the potential adverse effects of false information. Thus reviewers also have a responsibility to maintain quality. Indeed, in my short tenure as Editor-in-chief of The Bone & Joint Journal (BJJ), our excellent reviewers have uncovered research fraud, an attempt at dual publication and several attempts to “salami slice” data into multiple publications. We should be grateful for their vigilance. Reviewing is sometimes regarded as a thankless task and can be an extra burden on busy clinicians. The clinical and scientific communities, however, need to be supported by the best available evidence, as determined by the highest standards of reviewing of the material which is submitted to journals. The role played by the the editorial board of the BJJ, our specialty editors and in particular by our reviewers cannot be overstated. They underpin the quality which we endeavour to deliver. Such roles need to be recognised at the national and international levels in job plans, appraisal and clinical excellence awards and should be integral to career progression. It is also important for the scientific community and journals to remain up to date and embrace new technologies. The BJJ website goes from strength to strength, as also does our sister open access online journal, Bone & Joint Research. Our reviewing process and rewriting are all undertaken online, and the interactions with our international readership are largely electronic. Moreover, we continue to develop a more interactive subspecialty international platform, using all forms of current technology, via the web, tablets and smartphone apps. There is a considerable incentive from within the scientific community and government to embrace new technologies. The recent House of Commons Committee Report, referred to above, recommends experimenting with social networking tools as a means of developing and improving the dissemination of scientific information, and this is being explored by some groups.4,14,15 I have been asked by many whether we intend to develop and improve on our peer review with open systems. Although such improvements might allow a quicker publication and distribution of knowledge with a robust platform for the delegation of labour, which classical peer review cannot, it is not currently on our agenda. Our turnaround times remain rapid with the time interval between acceptance to publication remaining usually less than four months. There have been interesting trials and projects in the use of open review systems in the scientific communities, such as arXiv,14 Nature15 and Cureus.16 The drawbacks include the concern that opening comments to all would result in unhelpful suggestions or empty praise, or that useful suggestions may end up being buried under large numbers of mediocre, poorly thought out comments or instant reactions.17 There are other risks associated with community
involvement where occasionally ‘herd mentality’ can be observed. The recent witch-hunt of the Boston bomber in 2013 highlights these dangers. Online communities thought that they recognised the picture of the bomber as Ivy League student Sunil Tripathi, who was vilified and whose parents were harassed. The student was found a week later, dead.18 We are mindful of the evolution towards more transparency in the review process and experiments with open review systems and will maintain a watching brief in that area. For the time being, we will continue to respect our readers’ and reviewers’ wishes and to evolve our existing processes. The BJJ takes great pride in our peer review process which has developed over several decades. We are fortunate to have gained access to and developed strong relationships with many of the best and most committed reviewers in the world and will continue to work with them. We also have an international Editorial Board that provides leadership and direction in all the orthopaedic specialties. The BJJ will continue to try to lead the way with its current review system and place emphasis on the education of those who wish to review through our reviewer days here at the journal offices and through presentations at conferences. Over the past year we have presented at EFORT and the British Orthopaedic Association Annual Meeting. In 2014, we will be presenting an Instructional Course at the American Academy of Orthopaedic Surgery Annual Meeting, at EFORT and at the Royal Society of Medicine on 6th May. We are currently introducing some stepwise improvements rather than planning radical change. In order to enhance the experience both for our reviewers and authors, we have appointed specialty editors who will ensure that appropriate expertise is brought to bear in their subspecialty areas. They will ensure that manuscripts are sent to the most appropriate reviewers and will facilitate the interaction with authors. To this end I will be ably supported by: Elliot Sorene (wrist and hand) James Calder (foot and ankle) Fergall Monsell (children’s orthopaedics) Sam Patton (oncology and knee) James Waddell (hip) Matt Costa (trauma) In the interim, I hope that you will all join me in saluting the heavy burden carried by our reviewers and the essential work that they undertake. The unsung heroes of the publishing process are the reviewers. We owe them a massive debt.
References 1. Pringle M. Participating in clinical governance. BMJ 2000;321:737–740. 2. Ioannidis JPA. Why most published research findings are false. PLoS Med 2005;2:124. 3. Parsons NR, Hiskens R, Price CL, Achten J, Costa ML. A systematic survey of the quality of research reporting in general orthopaedic journals. J Bone Joint Surg [Br] 2011;93-B:1154–1159. THE BONE & JOINT JOURNAL
PEER REVIEW: THE UNSUNG HEROES
4. No authors listed. House of Commons. Science and Technology Committee: peer review in scientific publications, 2011:105. http://www.publications.parliament.uk/pa/ cm201012/cmselect/cmsctech/856/85602.htm (date last accessed 14 February 2014). 5. Samons LJ, Samons LJ. The Cambridge companion to the age of pericles. Cambridge: Cambridge University Press, 2007. 6. No authors listed. Wikipedia: Peer Review. https://en.wikipedia.org/w/ index.php?title=Peer_review&oldid=549967581 (date last accessed 14 February 2014). 7. No authors listed. Rescuing science from politics: regulation and the distortion of scientific research. Cambridge: Cambridge University Press, 2006:304. 8. Benos DJ, Bashari E, Chaves JM, et al. The ups and downs of peer review. Adv Physiol Educ 2007;31:145–152. 9. Twaij H, Oussedik S, Hoffmeyer P. Peer review. Bone Joint J 2014;96-B:436–441. 10. No authors listed. Sense about science: peer review and the acceptance of new scientific ideas, 2004:61 http://www.senseaboutscience.org/data/files/resources/17/peerReview.pdf (date last accessed 14 February 2014). 11. No authors listed. Research Information Network. Peer review: a guide for researchers, 2010:16. http://www.rin.ac.uk/our-work/communicating-and-disseminatingresearch/peer-review-guide-researchers (date last accessed 14 February 2014).
VOL. 96-B, No. 4, APRIL 2014
435
12. No authors listed. General Medical Council. Good medical practice: being honest and trustworthy, 2013. http://www.gmc-uk.org/static/documents/content/ GMP_2013.pdf_51447599.pdf (date last accessed 14 February 2014). 13. No authors listed. Urban Times. Encyclopaedia Britannica vs Wikipedia: a battle already lost? 2012. http://urbantimes.co/2012/03/encyclopaedia-britannica-vs-wikipedia-a-battle-already-lost-infographic/ (date last accessed 14 February 2014). 14. No authors listed. Wikipedia: arXiv. https://en.wikipedia.org/wiki/ArXiv (date last accessed 14 February 2014). 15. No authors listed. Nature. Overview: Nature’s peer review trial, 2006. http:// www.nature.com/nature/peerreview/debate/nature05535.html (date last accessed 14 February 2014). 16. No authors listed. Cureus. http://www.cureus.com (date last accessed 14 February 2014). 17. Ware M. Peer review: benefits, perceptions and alternatives. London: Publishing Research Consortium, 2008:1–20. 18. Buncombe A. The Independent. Sunil Tripathi: the other victim of Boston’s bombs, 2013. http://www.independent.co.uk/news/world/americas/sunil-tripathi-the-othervictim-of-bostons-bombs-8585250.html (date last accessed 14 February 2014).
