Clinical Hemorheology and Microcirculation 57 (2014) 95–99 DOI 10.3233/CH-141820 IOS Press

95

The last bite was deadly – About responsibility in scientific publishing Dragan Pavlovica,∗,1 , Taras I. Usichenkob and Christian Lehmannc,d a

Ernst-Moritz-Arndt Universit¨at, Greifswald, Germany Klinik f¨ur An¨asthesiologie und Intensivmedizin, Universit¨atsmedizin Greifswald, Greifswald, Germany c Departments of Anesthesiology, Pharmacology, Microbiology and Immunology, Computer Science, Dalhousie University, Halifax, NS, Canada d Anesthesiology, Humboldt-University, Charit´e, Universit¨atsmedizin, Berlin, Germany b

Abstract. Some open access journals are believed to have devaluated the highly respected image of the scientific journal. This has been, it is claimed, verified. Yet the project we believe failed and we show why we think that it failed. The study itself was badly conducted and the report, which Science published, was itself a perfect example of “bad science”. If the article that was published in Science were to be taken as one of the “test” articles and Science as a victim journal (a perfect control though), the study would show the opposite of what author concluded in his paper: 100% of the controls (normal non-open access journals, in the present study this was Science) accepted the “bait” paper for publication, while in the experimental group only about 60% (open access journals) accepted the bait paper for publication. The conclusion is that, with respect to non-open access and open access, the probability of accepting pseudoscience is well in favor of this being done by a non-open access journal. Since this interpretation is based on some facts that were not included in the project itself, the only warranted result of this study would be that nothing could be concluded from it. It is concluded that the method that Bohannon used was heavily flawed and in addition immoral; that the report that was published by Science was inconclusive and that the act of publishing such report cannot be morally justified either. Various methods to improve the quality of published papers exist but scientific fraud with “good intentions” as a method to promote scientific publishing should be avoided. Keywords: Open access, peer review, scientific publishing, morality of publishing, Bohannon

1. Introduction It appears that scientists are under the permanent pressure of a never-ending tide of irrelevant information and its negative secondary effects. The “publish or perish” tendency sharpens; unfair competition among scientists is intense, and there is a perennial challenge posed by competitive ideologies [6, 8, 11]. Scientific publishing is an adventurous and highly responsible enterprise that makes science valuable and available through a variety of modern media. The scientists who publish frequently have been aware for some time that the number of scientific journals is not only exploding but that there are scientific journals that publish without much control over the content of the papers they publish [2]. Some open access ∗

Corresponding author: Dragan Pavlovic, Former Research Director, Ernst-Moritz-Arndt Universit¨at, Greifswald, Germany. Tel./Fax: +33 1 56 24 90 99; E-mail: [email protected]. 1 Current address: Dragan Pavlovic, 5, Place de la Sorbonne, 75005 Paris, France. 1386-0291/14/$27.50 © 2014 – IOS Press and the authors. All rights reserved

96

D. Pavlovic et al. / Responsibility in scientific publishing

journals (OA) [3] are believed to have devaluated the highly respected image of the scientific journal once established by great journals such as Nature or Science. This has been, it is claimed, verified. 2. What happened and how Recently, an article was published in Science whose author claimed to have done a study and to have identified numerous culprits, guilty of such behavior, in open access (OA) journals [3]. It looked like quite a good idea to at least have something that can help us distinguish science from pseudoscience in publishing. Yet the project we think, as do a number of others, failed [9]. Not only did the text fall short of its target, it compromised the famous journal Science, morally and scientifically. What happened and how? The plan, we presume, was to produce a text (bait) which would look like a scientific study but would be recognizably flawed and, if accepted by an open access journal (OA) (victim), would expose that OA journal as unscientific. Therefore a professional journalist, John Bohannon, was hired, and he performed the “study” that was, when finished, published as an article in Science. We describe below in some detail the bait and the text that was published to expose the OA journals. After publication, this “plot” would be still problematic because it contained a contradiction: an inference that something is the case that could not be because based on an unscientific foundation, as Bohannon’s paper was planned to be. Scientists would immediately see the problem and the discussion after the publication might become quite pernicious. Science planned therefore, to declare the paper to be merely a news item. Thereby the eventual critique could be prevented. And, we presume, no printed comments were to be published, or if some were to be allowed, they would merely be moderate critique. If some harsh critique were by accident to surface, it would be better to divert it to online comments, and not to expose it in the printed form. And then, online comments would be allowed to explode in all possible directions and this would obfuscate the result. Otherwise, the risky project could yield quite unfavorable results for Science. And this is what I at least believe to have unfortunately happened. 3. What was the result of the study The final result was much worse than expected. Bohannon’s article, which Science published [11], was itself a perfect example of “bad science” as he himself had defined it, and this was impossible to hide, i.e. the intentional flaws of Bohannon’s bait were: bad controls for the experimental groups, a wrong legend for one graph, non idiomatic English, and the conclusion contained unjustified and therefore “fallacious” recommendations. The very article by Bohannon published in Science has itself more than similar weaknesses i.e.: there is no randomization of his “experimental group”, there are no controls at all (he could have compared the OA journals to the ordinary big journals), there was elimination of non–responders and there was no application of the intention to treat principle in the analysis. Then, even worse: there were no inferential statistics and no references. Indeed, the main drawback of the bait articles was that the introduction and discussion were extremely modest. But the alleged “flaws” of the bait articles which the journal victim should not have missed, were not pertinent enough. That Fig. 1 does not really show dose – response could be excused by the fact that the agent was probably specific so that it already had its maximal effect by the first concentration tested. Then, the objection about artefacts from ethanol does not hold, because the concentrations of ethanol were too small (0.66% ethanol). Also, an increase of sensitivity to radiation (Fig. 2) may be present since the response does not start from 100%

D. Pavlovic et al. / Responsibility in scientific publishing

97

Ouroboros drawing from a late medieval Byzantine Greek alchemical manuscript. Fol. 196 of Codex Parisinus graecus 2327, a copy (made by Theodoros Pelecanos (Pelekanos) of Corfu in Khandak, Iraklio, Crete in 1478) of a lost manuscript of an early medieval tract which was attributed to Synosius (Synesius) of Cyrene (d. 412). The text of the tract is attributed to Stephanus of Alexandria (7th century). From http://en.wikipedia.org/wiki/File:Serpiente alquimica.jpg (accessed 13th Febr., 2014) .

but much lower. Finally, it is not a “fatal flaw” to conclude that the agent should be examined “in animal and human”. Therefore nothing could be concluded from the text published in Science. Bohannan’s bait articles were withdrawn from OA journals (victims), so unfortunately we do not know what type of articles they would have been if published (brief report, pilot study, field report, special report, news or something else). In the discussions that followed its publication in Science – Bohannon’s text is presented not as simple interesting “news” but as a scientific study. This is indirectly suggested also in the subtitle. What follows from Bohannon’s text is that the journal where Bohannon’s text itself was published (Science) could belong also to the “victim” group of journals characterized by Bohannon as journals that publish weak science. Science was sufficiently clever to publish Bohannon’s article as “news”, which apparently liberated the study from applying scientific method. Scientific method exists to try to establish what is and what is not, and even when it is used correctly, the conclusions are uncertain. As it is, this text would therefore imply no scientific consequences. Nevertheless, we think that it has some. 4. The last bite was deadly We must correct ourselves and the other commentators on one point. What a number of people claimed about the lack of the controls in Bohannon’s study, is not entirely true. There was a control, though a single one but nevertheless a perfect control. Here is how. Bohannon claimed that he sent 304 sting papers (baits) to open access (OA) journals (victims) and analyzed 255 responses. In fact there were 255 plus one control journal, which was the last journal that was “tested”. In that “last bite” offered to one non-OA control journal, there was no randomization, no controls at all, there was elimination of non

98

D. Pavlovic et al. / Responsibility in scientific publishing

–responders, no application of the intention to treat principle, no inferential statistics and no references. So this was somehow extreme bait, the most pseudoscientific of all bait articles so far in Bohannon’s study. The victim, which was in fact a control journal, was now non-OA, a scientific top journal, and it not only accepted the bite, but it published it. Thereby it was more than an ideal control journal – it was also an extreme victim, but in the other sense: one of the most resistant to publishing pseudoscience among normal non-OA journals. Needless to say: what was unconsciously submitted as a bait paper was Bohannon’s very report about his “study” and the victim journal that published it was Science. This is also a perfect example of how “retrograde causation” [4, 7] may work in real life: by publishing the mentioned paper, Science changed the nature of the Bohannon study that was already terminated: the Bohannon “study” had in the end a control group. The implications of the study changed as well, the study now showed the opposite of what Bohannon concluded in his paper: 100% of the controls (normal non-OA journals, in the present study this was Science) accepted the bite paper for publication, while in the experimental group only about 60% (OA journals) accepted to publish bait paper. Some would say that since the control sample was n = 1 it is not really enough to draw such a conclusion. However, since both, the bait and the control journal, were pushed to the extremes, this control experiment is much more sensitive than it would have been if an ordinary bait and an average non-OA journal were used. This super control experiment, even with n = 1 had much more weight than an ordinary single control experiment. The conclusion is that, as between non-OA and OA, the probability of accepting pseudoscience is well in favor of this being done by the non-OA journal. A result quite unexpected by the investigators. Indeed, in principle, an article either states how some matters of fact are, in the world, and thereby claims to be scientific. Or else, it is cognitively neutral and thereby just part of some form of entertainment. Since this interpretation is based on some facts that were not included in the project itself, only warranted result of this study would be that nothing could be concluded from it. Here it will be on the scientific community to decide either for the above given conclusion or for the entertainment. We would opt for the latter.

5. Morals for the end In addition, and this is the greatest problem. The act of producing and sending fraudulent articles to scientific journals is a moral failure. Such an immoral strategy to expose bad science is not necessary. (i) The scientific community already knows that such failings exist, that even „normal“ and respected journals, suffer from bad reviewing. (ii) The so called „Socal Affaire“ [12] from 1996 consisted in a similar intentional fraud to expose bad science and there was no need to repeat such an act. (iii) The very recent case involving the Romanian journal Metalurgia International has already been received without much enthusiasm [5, 10], and (iv) Scientific publications are perennially exposed to fraudulent research. Therefore, to produce a fraudulent text with “good intentions” cannot be justified in principle. It is impossible not to condemn the act on deontological grounds. Or, the principles of utility [14] would have to show that submitting 304 fraudulent articles can somehow be made acceptable because: 1) The intention was noble. 2) This is the only means through which pseudoscientific journals can be discredited and 3) The means justify the ends: Identification and elimination of pseudoscience from scientific publishing. Unfortunately, none of this was the case. The project intended to demonstrate something that was previously known, other more decent measures for the same purpose were available, and, most importantly, the act itself was an exceedingly unfair means employed to achieve doubtful end. To justify the act by using the doctrine of double effect (DDE) [1, 13, 15] would also involve the

D. Pavlovic et al. / Responsibility in scientific publishing

99

above given justifications for the use of multiple scientific frauds as sheer means, which, as we said, are inadequate. 6. Post script The idea of measuring the quality of the reviewing of open access journals was good, the motive was probably justified, the performance shabby; the result is a moral failure. Improving the reviewing procedure would most probably improve on the quality of published science. The big journals that attract through their reputation the best reviewers have lesser problems with the scientific quality of the texts that they publish. As we can see from the above discussed example, knowledge and science remain chimeras that escape most of our efforts to impose on them our rules and rigorous standards. Standards of certification for reviewers or some mechanisms of feedback that would evaluate the reviewing quality might improve the reviewing procedure. It would also help if the scientific community would pay less attention to the fame of a journal or to indicators such as the impact factor or similar indices, but would consider more directly the quality of the research itself. This could introduce some security against scientific misconduct and help improve on the quality of scientific journals. Open access journals fulfill in a certain way the dream of free knowledge. We should not spoil the dream - perhaps merely wake up. References [1] E. Anscombe, Medallist’s Address: Action, Intention and ‘Double Effect’. In: Proceedings of the American Catholic Philosophical Association 56 (1982), 12–25; Washington, D.C.: American Catholic Philosophical Association, reprinted in Woodward (ref. 15). [2] Beall’s list of predatory publishers. http://scholarlyoa.com/publishers (accessed 2013 Nov 15). For a list of the open access journals see at: http://www.doaj.org/doaj?func=byPublicationFee&uiLanguage=en (accessed 2013 Nov 15). [3] J. Bohannon, Who’s Afraid of Peer Review? A spoof paper concocted by Science reveals little or no scrutiny at many open-access journals, Science 342 (2013), 60–65. [4] M. Dummett, Can an Effect Precede its Cause”, Proceedings of the Aristotelian Society 28(Supplement) (1954), 27–44. [5] Z.D. Duri´c, B. Delilbaˇsi´c and S. Radisic, Evaluation of transformative hermeneutics heuristics for processing random data, Metalurgia International 18(6) (2013), 98–102, Free PDF available at: http://fr.scribd.com/doc/167706815/EVALUA TION-OF-TRANSFORMATIVE-HERMENEUTIC-HEURISTICS-FOR-PROCESSING-RANDOM-DATA (accessed 2013 Dec 11). [6] D.M. Eisenberg, R.B. Davis and S.L. Ettner, et al., Trends in alternative medicine use in the United States, 1990-1997: Results of a follow-up national survey, JAMA 280 (1998), 1569–1575. [7] A. Flew, Can an Effect Precede its Cause. In: Proceedings of the Aristotelian Society 28(Supplement) (1954), 45–62. [8] P.B. Fontanarosa and G.D. Lundberg, Alternative medicine meets science, JAMA 280 (1998), 1618–1619. 10 [9] Science online: http://comments.sciencemag.org/content/10.1126/science.342.6154.60/, (accessed 2013 Nov 11). [10] A Serbian Sokal? Authors spoof pub with Ron Jeremy and Michael Jackson references, Retraction Watch: http://retraction watch.com/2013/09/23/a-serbian-sokal-authors-spoof-pub-with-Ron-Jeremy-and-Michael-Jackson-references/ (accessed 2013 Dec 11). 11 [11] R. Sheldrake, The Science Delusion: Freeing the Spirit of Enquiry, Hodder and Stoughtton Ltd, (2012). [12] A. Sokal, Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity, Social Text 46/47 (1996), 217–252. [13] Thomas Aquinas, Summa Theologica (II-II, Qu. 64, Art.7). [14] B. Williams, A Critique of Utilitarianism. In: Utilitarianism: For and Against, by Smart JJC and Williams B. (1973), pp. 77–150; Cambridge: Cambridge University Press. [15] P.A. Woodward (ed.) The Doctrine of Double Effect: Philosophers Debate a Controversial Moral Principle. Notre Dame, University of Notre Dame Press, (2001), pp. 50–66.

Copyright of Clinical Hemorheology & Microcirculation is the property of IOS Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.

The last bite was deadly--about responsibility in scientific publishing.

Some open access journals are believed to have devaluated the highly respected image of the scientific journal. This has been, it is claimed, verified...
243KB Sizes 0 Downloads 3 Views