516712 research-article2014
SJP0010.1177/1403494813516712S. O. HanssonWhy and for what are clinical trials the gold standard?
Scandinavian Journal of Public Health, 2014; 42(Suppl 13): 41–48
oRIGINAL ARTICLE
Why and for what are clinical trials the gold standard?
SVEN OVE HANSSON Division of Philosophy, Royal Institute of Technology (KTH), Stockholm, Sweden
Abstract The epistemological basis of clinical trials and evidence-based medicine is investigated. Clinical trials are directly actionguiding experiments on treatment effects. This is the reason why well-performed clinical trials take precedence over all other types of studies as far as treatment effects are concerned. The efficiency of public health interventions can be studied with directly action-guiding experiments that have the same strong epistemic justification as clinical trials. However, in order to assess the causality of diseases, information from several types of studies will have to be combined. Here, no single type of studies has priority over all the others. Therefore, evidence hierarchies are less helpful in studies of causality than they are in investigations of the effects of treatments or interventions. Key Words: Action-guiding experiment, clinical trial, epistemic experiment, evidence-based medicine, public health
Introduction Clinical trials are the universally accepted gold standard of evidence-based medicine. If we have wellconducted, randomised, and double-blind clinical studies, then the recommendation is to use them as the basis both for clinical decisions and for policy decisions on healthcare priorities, without giving much weight to other types of studies such as case reports and mechanistic laboratory studies. But evidence-based medicine is continuously under debate, and the role it gives to clinical trials has been questioned. There is a need to carefully investigate the evidential status of clinical trials, both generally and more specifically in relation to the role they have in evidence-based medicine. We need, in particular, to answer the following four questions: 1. Why are clinical trials the gold standard? Many other types of studies, such as laboratory experiments and clinical case reports, contribute to the progress of medical science. Is there a convincing justification why clinical studies should have the privileged role assigned to them in evidence-based medicine?
2. For what questions can clinical trials reasonably be the gold standard? Clinical trials were developed to determine the effects of treatments. Some have wished to restrict their use by excluding certain types of treatments, whereas others have proposed their use for nontreatment issues. For what medical issues should they be used? 3. What role should clinical trials have in the total assessment of evidence? In particular, have they been given the right role in evidence-based medicine as it is usually practiced? 4. Can the same evidence hierarchy be used in public health as in treatment studies? Currently, attempts are being made to extend the scope of evidencebased medicine to new areas such as public health. Can that be done? If so, is there a need to adjust the hierarchy? Can the role of clinical trials be the same? To answer these questions, we need to see clinical trials in a more general context provided by epistemology and the philosophy of science.
Correspondence: Sven Ove Hansson, Division of Philosophy, Royal Institute of Technology (KTH), Stockholm, Sweden. E-mail: [email protected]
© 2014 the Nordic Societies of Public Health DOI: 10.1177/1403494813516712
42 S. O. Hansson Clinical trials as experiments From the viewpoint of scientific methodology, it is important to distinguish between experiments and nonexperimental observations. By an experiment (in the modern sense of the word) is meant a procedure in which some object of study is subjected to interventions (manipulations) that aim at obtaining a predictable outcome or at least predictable aspects of the outcome. Clinical trials are a special type of experiments: they are experiments in which the object of study is a group of patients, the interventions are treatments, and the outcome is the medical condition and general welfare of the patients. The requirement that the outcome of an experiment should be predictable is usually expressed as a condition of repeatability: if the same interventions are performed on similar objects of study under the specified conditions, then the same outcome should be obtained. This requirement is essential for clinical trials since we perform them in order to obtain information that can be relied on in clinical practice. Directly action-guiding experiments In addition to the distinction between experiments and nonexperimental observations we need to distinguish between two types of experiments, namely directly action-guiding and epistemic experiments, respectively [1]. An experiment is directly actionguiding if and only if it satisfies two criteria: (1) The outcome looked for should be some desired goal of human action. (2) The interventions studied should be potential candidates for being performed in a nonexperimental setting in order to achieve that goal. Both these criteria are obviously satisfied by clinical trials. For instance, in a clinical trial of an analgesic the outcome looked for is efficient pain reduction with minimal negative side effects, and the experimental intervention is a treatment that might be administered to achieve exactly that in a clinical, nonexperimental setting. Examples of directly actionguiding experiments in a nonmedical setting include agricultural field trials, many technological tests such as tests of the longevity of light bulbs, and social experiments trying out the effects of different methods of social work. An epistemic experiment aims at providing us with information about the workings of the world we live in. Therefore, the outcome looked for is one that provides such information, and the intervention is constructed to achieve such an outcome. Hence,
when photochemists measure the wavelength of electromagnetic radiation emitted from some chemical reaction, this is done in order to obtain information about the reaction, not in order to find a way to produce light with that specific wavelength.1 Nonclinical biomedical research makes abundant use of epistemic experiments (that may of course be indirectly action-guiding in various ways). The early history of experimentation The history of experiments has usually been treated as a history of scientific experiments. This has led to an exclusive focus on epistemic experiments and on the experimental tradition in modern science that began with the renaissance. However, if we include directly action-guiding experiments, then experiments turn out to have a history that is much older than that of science. This means that, contrary to what is commonly assumed, the origin of experimentation is neither academic nor curiosity-driven. Instead, it is socially widespread and driven by practical needs. Agriculture is the crucial technological innovation that made civilisation possible. Archaeological studies of ancient agriculture have shown that it was not the result of a few inventions or discoveries but of thousands of years of extensive and continuous experimentation [2]. This is exemplified in traditional Andean agriculture where the uncertainties of an unpredictable climate, pests, and other natural events were met by sophisticated risk spreading. Each farmer had many dispersed fields in different microclimates and continuously tried out different crops in different places to see what worked best. Agricultural experimentation is virtually universal in the Andes. In every Indian community there are many people who continually experiment with all plants that come their way in special fields (chacras) usually located near their houses, but some do so in a certain chacra in every production zone where they have fields. They watch how different plants “go” under different climatic conditions…. Simulation and experimentation are routine activities for the native Andean agricultor [3].
In at least one place, these experimental endeavours seem to have been conducted in a highly systematic way. In an Andean site called Moray, close to Cusco in present-day Peru, several natural sinkholes have been terraced by the Incas. There are large climatic differences between different parts of the terrace system, partly due to shadows cast by a nearby mountain. The terraces also have an irrigation system that allowed for variation of water supply between the different terraces. According to oral tradition among
Why and for what are clinical trials the gold standard? 43 local peasants, Moray served as “an Inca agricultural college” [3]. This seems to have been a site for agricultural field trials (i.e. directly action-guiding agricultural experiments). Records from other parts of the world confirm that indigenous and traditional farmers do indeed experiment [4, 5]. The Mende people in Sierra Leone have a special word, “hungoo”, for such experiments. A hungoo can for instance consist in planting two seeds in adjacent rows, and then comparing the output in order to determine which seed was best [6]. Just like farmers, craftspeople of different trades have performed directly action-guiding (i.e. technological) experiments already in prescientific times. As one example of this, extensive experiments on the composition of glass were performed in the early Islamic period in Raqqa (Ar-Raqqah) in eastern Syria. Analysis of artefacts and debris from the 8th to 11th centuries has shown that whereas some glass types had a surprisingly unvarying composition across the centuries, others were subject to much experimentation. These experiments seem to have included a so-called chemical dilution line in order to search systematically for optimal proportions of the main ingredients [7, 8]. Judging by the many types of materials and mixtures developed in the course of human history, systematic experimentation to optimise batch recipes must also have taken place in many other places and at various times throughout the world. It is, for instance, difficult to see how the optimal mixture between copper and tin to obtain as hard bronze as possible could otherwise have been discovered in short time in ancient Egypt [9]. Similarly, in order to obtain mortar of good quality, “[m]asons had to rely on experimentation or information past on orally to understand and learn the properties of the employed materials and the effects they could have produce[d] when added to a mortar” [10]. The great cathedrals that were built in the High Gothic period (around 1140–1350) are astonishing engineering products. During that period, the pillars and other construction elements of new cathedrals became slimmer and slimmer. How could that have been achieved in spite of the lack of any means to calculate the required dimensions? Extensive studies have shown that these cathedrals were in fact experimental building projects, and “the design may have been successively modified on the basis of observation of the buildings during the course of construction” [11]. When a new cathedral was planned, its construction elements could be made somewhat slimmer than those of its predecessors. During the construction period, builders had to be sensitive to signs that the building was not strong enough. One such sign was cracks that appeared in newly set
mortar when temporary construction supports were removed. The tensile strength of medieval mortar was very low, and therefore even relatively small displacements would have been observed in this way. The effects of strong winds on the construction would also have to be monitored. Since the cathedrals took several decades to build, each of them must have been subject to severe storms during construction. Fortunately, if the construction was found to be too weak, the builders did not have to tear down the cathedral and build a new, more sturdy one. Instead they could add construction elements such as flying buttresses that provided the necessary additional support. In this way, “the details of design were worked out with a crude type of experimental stress analysis performed during construction: tensile cracking observed in the weak lime mortar between the stones during the relatively long period of construction could have led to refinements in design” [12] (cf. [13]). Given the prescientific experimental traditions in agriculture and technical crafts, one should perhaps expect to find similar prescientific traditions of directly action-guiding experiments among the traditional healers who were the forerunners of physicians. The selection of herbs and other remedies for diseases can be based on experiments in the same way as the selection of a crop, an agricultural method, the composition of an alloy, or the dimensions of a column. If you want to know whether you can reduce a patient’s fever by giving her a particular herb, give feverish patients the herb and record whether their fever is abated or not. This idea was clearly expressed by Avicenna (Abd Allah ibn Sina, c.980–1037): [It] is like our judgement that the scammony plant is a purgative for bile; for since this [phenomenon] is repeated many times, one abandons that it is among the things which occur by chance, so the mind judged that it belongs to the character of scammony to purge bile and [the mind] gave into it, that is, purging bile is an intrinsic characteristic belonging to scammony [14].
However, this was an exceptional statement for its time, and Avicenna does not report whether he actually performed the experiment in question. I have not been able to find examples of directly action-guiding experiments on the treatment of diseases being performed by indigenous peoples or before the modern scientific era. It is only in the 18th century that we find reports of what we would today call clinical experiments. (The most famous of these was James Lind’s scurvy trial in 1747 [15]). These experiments were performed by physicians who were under the influence of natural science rather than prescientific experimental traditions. And it was not until after the
44 S. O. Hansson Second World War that the modern clinical trial was established as the standard method for investigating treatment effects [16–18]. It is interesting to compare this late introduction of directly action-guiding experiments in medicine with their much earlier introduction in agriculture. Farmers in different parts of the world have performed directly action-guiding experiments since long before modern science, and the introduction of field trials in agricultural science was both earlier and less conflict ridden than that of clinical trials in medical science. The only plausible explanation of this difference that I have been able to find is that the subject matter of human health tends to be much more imbued with ideology and religion than that of agriculture (or, for that matter, glassmaking or building technology). This applies to ancient as well as modern societies. Perhaps the down-to-earth nature of questions such as “can I grow this crop here?” or “will the columns support this type of roof?” facilitated the thought processes that led to the use of directly action-guiding experiments. We may be more prone to look for guidance elsewhere in questions of life or death, health or disease. A strong and theory-independent justification Let us now turn to the justification of using clinical trials to determine treatment effects. It turns out that this practice has a strong justification that is common to the general category of experiments that they belong to, namely directly action-guiding experiments. We can summarise the notion of a directly action-guiding experiment in the form of the following simple recipe: Recipe for directly action-guiding experiments If you want to know if you can achieve Y by doing X, do X and see if Y occurs.2
This recipe is in a sense self-vindicating. In order to find out whether you can achieve Y by doing X, what better method can there be than to do X and see if Y occurs? In particular, we have no problem in justifying the use of an intervention or manipulation (the X of the recipe). We want to know the effects of such an intervention, and then it is much better to actually perform it than to passively observe the workings of nature. This is a simple and indisputable argument that applies to directly action-guiding experiments but not to epistemic experiments. This justification also explains why directly action-guiding experiments should be repeatable. Since the purpose of the experiment is action-guiding, we
need to establish a connection between an intervention-type and a desired outcome. In order to be practically useful, such a connection has to appear regularly: it is not sufficient that something happened once. Someone might counter what was just said by asking: “Can this be true? Can directly action-guiding experiments really be that strongly justified? Have we not learned that all experiments are theory laden?” Well, indeed we have, but the common arguments showing the theory ladenness of experiments [19] refer to epistemic experiments, rather than to directly action-guiding ones. The specific theory ladenness of experiments seems to come with their epistemic interpretations. The following two examples should clarify the difference: 1. In the search for a new treatment against hair loss in dogs (canine follicular dysplasia), a veterinary surgeon performs an experiment on a group of affected dogs. The dogs are divided into a treatment group and a control group. The former group receives a daily dose of a newly synthesised substance. 2. In order to test a biochemical hypothesis about the regulation of cell growth, a researcher looks for an animal model in which a newly synthesised substance is expected to have an easily visible and measurable effect. She finds out that dogs with follicular dysplasia are a suitable such model. She therefore performs an experiment in which dogs with this condition are divided into an experimental group that receives a daily dose of the substance and a control group that does not receive the substance. At least in principle, these two prehistories can lead to exactly the same experiment, in terms of the study objects, the intervention performed, and how the outcome is monitored. However, there is a remarkable difference in terms of theory ladenness. In the first case, you may have whatever reason you want for believing either that the substance will improve the dogs’ health or that it will not. But when the experiment has been completed, if there is a clear difference in hair growth between the two groups, then there is not much scope for interpretation. The experiment just tells us how it is.3 The second case is different. Here, the interpretation of the experiment will have to be based on nontrivial, theoretical assumptions concerning the plausible mechanism behind any observed effect. Obviously, the strong justification of clinical trials (and other directly action-guiding experiments) does not mean that we should rely unconditionally on
Why and for what are clinical trials the gold standard? 45 them regardless of how they have been performed. They are the superior method for finding out the effects of treatments, but like all other methods they can be misapplied or misinterpreted. A whole series of safeguards, such as control groups, blinding, and randomisation, have been developed in order to ensure the reliability of the information obtained from clinical trials. Critical examinations of how clinical trials are conducted and interpreted are essential parts of clinical science. The unwelcomeness of theoryindependent information However, there is also another type of criticism that does not have this positive and essential role in the development of medical knowledge, namely general criticism against the use of clinical trials to determine the effects of medical treatments. Interestingly, such criticism does not seem to have emerged in the other areas where directly action-guiding experiments have a similarly strong standing. I have not been able to find any examples of farmers or agricultural scientists opposing the use of field trials to determine the agricultural properties of crops or farming methods. Neither have I found traces of any resistance against the use of directly actionguiding experiments in technological or engineering contexts. But as already mentioned, issues of health and disease are imbedded in various ideological belief systems that may explain the comparatively late introduction of directly action-guiding experiments in medicine. It should be no surprise that the presentation of strongly supported, theory-independent information such as that provided by good clinical trials can be quite unwelcome for the proponents of theories that may have to be revised or deserted due to that information. A person who reports strong evidence of the inadequacy of a prevalent therapeutic practice may find him- or herself in the same situation as the child in Hans Christian Andersen’s tale of the emperor’s new clothes. The emperor of that tale continued his pompous parade after being told what was open for everyone to see. Academic dignitaries and other defenders of received medical theories have been known to behave similarly in response to incontrovertible, theory-independent information from clinical trials. An interesting early example is an experiment on the treatment of pneumonia that was performed in the late 1840s by the Polish-Austrian physician Joseph Dietl (1804–1878). In those days, the general view among physicians was that pneumonia depended on an imbalance between the bodily fluids. There was
some disagreement on the nature of that imbalance, and consequently some physicians recommended bloodletting whereas others favoured the administration of an emetic. In 1849, Dietl reported an investigation in which he had compared three groups of pneumonia patients. One group had received bloodletting, the second had received an emetic, and the third had received general care but no specific treatment. Mortality among those who had received bloodletting was 20.4%, among those who had received an emetic 20.7%, and among those who had received no specific treatment only 7.4%. Dietl’s message was at first rather negatively received. His critics claimed that since disease and treatment are highly individual issues, they cannot be settled with statistics. They defended the idea of medicine as an intuitive art that goes beyond what can be determined by scientific method. But in the end, Dietl was successful. In the 1870s, the major medical textbooks advised against bloodletting of pneumonia patients [20, 21]. It took more than a century after Dietl’s paper on pneumonia until clinical trials became the generally accepted method for determining treatment effects. This was a slow process, and some pockets of resistance have persisted surprisingly long. The opposition has been strongest in psychiatry [22]. This is unsurprising due to the strong influence in that specialty of theory-based and sometimes ideologised doctrines prescribing what treatments to use. But in recent years, it has become generally accepted in psychiatry that the choice between, for instance, Freudian psychoanalysis and pharmacological treatment should not be based on ideology but instead be made separately for each diagnosis, based on directly actionguiding experiments showing what effects the different treatments have on patients with that diagnosis. Obviously, this openness to reality checks is essential for the future progress of psychiatry. The remaining pockets of resistance in the healthcare sector can now be found among so-called alternative therapists who commonly reject directly action-guiding experiments with arguments very similar to those brought forward by Joseph Dietl’s adversaries. Clinical trials in evidence-based medicine The term “evidence-based medicine” is rhetorically efficient but unfortunately rather unclear. As a first attempt, one might want to define it as medicine based on evidence. However, that would be a much too wide definition. Evidence is, generally speaking, information that should legitimately have an influence on a rational person’s degree of belief in some
46 S. O. Hansson statement. Evidence-based medicine is something much more restricted than medicine based on evidence: it is medicine based on a certain type of evidence hierarchy. Unfortunately, the idea of such a hierarchy is not easy to justify in terms of basic epistemological principles. When we have several pieces of evidence concerning some subject matter, we should ideally take them all into account in our overall assessment. If the mass of evidence is large then this cannot be done in practice, which means that we have to leave out the less important evidence from our final deliberations. This is what a court of law does in a criminal case. If there is strong evidence for instance in the form of a CCTV film showing the defendant committing the crime, then other pieces of evidence can be put aside (although they might have been relevant in the absence of that film). Similarly, if several well-performed clinical trials show that the antihypertensive drug A has a larger antihypertensive effect than its competitor B (at certain dosages), then we can consider the issue as settled, and we do not need to pay much attention to case studies, animal studies, or mechanistic considerations concerning the relative sizes of the antihypertensive effects. For some purposes, it is helpful to explain this in terms of an evidence hierarchy that puts well-performed clinical trials on top and other types of studies on various lower positions. But, however useful an evidence hierarchy is in practice, it is only a simplification or a heuristic device, not an exact model of epistemic justification. There are at least two reasons for this. First, what makes the evidence on a lower level insignificant in the presence of conclusive evidence on the higher level is the conclusiveness of that evidence, not its place in the hierarchy. If evidence from well-performed clinical trials is inconclusive concerning the size of treatment effects, then it may be unwise to disregard other types of information that could have been left out if the evidence from clinical trials were conclusive. Even more importantly, “middle-level” evidence on treatment effects, such as evidence from case–control studies, can never be as conclusive as “high-level” evidence (i.e. clinical trials). Therefore, “low-level” evidence such as case studies and mechanistic information cannot be as easily left out if the best that we have is middle-level rather than highlevel evidence. The second reason is that the relative evidential weight of different types of studies is different for different types of effects. As one example of this, the relative weight of nonblinded studies in relation to blinded ones should be much smaller for treatment effects that are known to be much influenced by the
patient’s expectations, such as pain reduction, than for other effects, such as halted tumour growth. These are just a few examples of the many types of considerations that can legitimately influence the assessment of medical evidence. Although current evidence hierarchies have been constructed to include some such considerations, it is not possible to cover them all. But although the use of fixed hierarchies cannot be justified as epistemically optimal, it can in many cases be justified as a wise compromise between what is epistemically optimal and the advantages of using fixed criteria that cannot be manipulated or unintendedly influenced by hopes and expectations. Evidence-based medicine and public health Can the principles of evidence-based medicine be transferred to public health research? In answering that question it is important to distinguish between different types of research issues in public health. Two major types of such issues are those that concern the efficiency of interventions and the causes of diseases. The efficiency of interventions can be studied with directly action-guiding experiments. Suppose that we want to know whether free distribution of mosquito nets reduces the incidence of malaria in tropical areas. The best way to investigate this is to give out mosquito nets to the population in some malaria-ridden areas but not in others and monitor the incidence of the disease in the different areas. This is not a clinical trial, but like clinical trials it is a directly action-guiding experiment. Therefore it has the same strong epistemic justification as clinical trials. If we have good intervention studies of this type, then they take precedence over all other types of information, just like clinical trials do concerning treatment effects. Questions about causes of diseases are different in this respect. Suppose that we wish to know whether a stressful situation at work increases the risk of insomnia. This is a research question that cannot be answered with directly action-guiding experiments. (We can of course choose to study the related issue whether a certain type of intervention against workplace stress reduces the risk of insomnia, but that is a distinctly different research question.) We clearly need epidemiological studies in which stress levels on the workplace, sleep parameters, and various confounding factors are followed in a sample of the population. However, even with excellent studies of that type we do not have direct access to causality in the same way as we have direct access to intervention
Why and for what are clinical trials the gold standard? 47 effects in a directly action-guiding experiment. The standard problem in studies of causality, namely that we cannot distinguish between “Y causes Z” and “some unknown underlying factor X causes both Y and Z”, cannot be avoided in this type of study. This is a major reason why such epidemiological studies need to be complemented by mechanistic investigations that help us identify plausible causal and confounding factors. (In this case such mechanistic studies may include short-term intervention studies in laboratories, observational studies with nightly EEGs, and perhaps animal studies.) An important general conclusion can be drawn from this: if there is a gold standard for evidence on disease causality, then that standard does not refer to a single type of study that has priority over all others when it is sufficiently well-performed. There is simply no such type of study. In order to draw wellfounded conclusions on the causes of diseases, we have to combine information from different types of investigations. Some of these are studies that rank low on the hierarchies of evidence-based medicine, such as studies of potential physiological or biochemical mechanisms. Their low rank on the hierarchies does not reflect low quality; instead it reflects low relevance for the specific issue of treatment effects that these hierarchies were constructed for. For issues of causality they often have high relevance. Guidelines for assessing the aetiology of disease will have to recognise that complexity, and the same applies to guidelines for risk assessment. Conclusion We have arrived at the following answers to the four questions asked in the introduction: 1. Why are clinical trials the gold standard? This is because they are directly action-guiding experiments. 2. For what questions can clinical trials reasonably be the gold standard? Only for questions about the effects of the interventions (treatments) that are tested in the trial. 3. What role should clinical trials have in the total assessment of evidence? In determining treatment effects, well-performed clinical trials have a privileged position and should take precedence over all other types of evidence. 4. Can the same evidence hierarchy be used in public health as in treatment studies? In studies of the effects of public health interventions, essentially the same hierarchy can be used, with directly action-guiding intervention experiments taking the role of clinical trials. In studies of disease causality, information
from several types of studies must be combined, and there is no single type of study that can take the supreme role on top of the hierarchy that clinical trials have in the assessment of treatment effects. Therefore, the simplified structure of an evidence hierarchy is much less suitable for causality than for intervention effects. Conflict of interest The author declares that there is no conflict of interest. Funding This work was supported by the Bank of Sweden Tercentenary Foundation, grant P11-0510:1. Notes 1. In principle, the same experiment could be performed in search of a way to construct a laser emitting some specific wavelength. Therefore, strictly speaking, we should make a distinction between directly action-guiding and epistemic purposes of experiments. 2. In some cases we have the background knowledge that without X, Y will not occur. But if such knowledge is not available, or can be put to doubt, then we need to perform a control experiment: on similar study objects we refrain from performing X, or perform some relevant alternative to X, in order to see whether Y will then happen. 3. Directly action-guiding experiments are not completely theory-independent in a philosophical sense. Like all empirical generalisations they depend on the theoretical assumptions about regularities in nature that are at play in all inductive reasoning. However, this is a minimal form of theory-dependence that we cannot avoid, in contradistinction to the dependence on more specific theories that has been shown to hold for many epistemic experiments. References [1] Hansson SO. Experiments before science – what science learned from technological experiments. In: Hansson SO (ed) The role of technology in science. Philosophical perspectives 2014. Dordrecht: Springer, in press. [2] Bray W. Ancient food for thought. Nature 2000;408: 145–6. [3] Earls J. The character of Inca and Andean agriculture. Lima: Departamento de Ciencias Sociales Pontificia Universidad Católica del Perú. Available at: http://macareo.pucp.edu.pe/~jearls/documentosPDF/theCharacter.PDF (1998, 29 July 2013). [4] Chandler PM. The indigenous knowledge of ecological processes among peasants in the People’s Republic of China. Agric Human Values 1991;8:59–66.
48 S. O. Hansson [5] Johnson AW. Individuality and experimentation in traditional agriculture. Hum Ecol 1972;1:149–59. [6] Richards P. Farmers also experiment: a neglected intellectual resource in African science. Discov Innovat 1989;1: 19–25. [7] Henderson J, Challis K, O’Hara S, et al. Experiment and innovation: early Islamic industry at al-Raqqa, Syria. Antiquity 2005;79:130–45. [8] Henderson J, McLoughlin SD and McPhail DS. Radical changes in Islamic glass technology: evidence for conservatism and experimentation with new glass recipes from early and middle Islamic Raqqa, Syria. Archaeometry 2004;46:439–68. [9] Malina J. Archaeology and experiment. Nor Archaeol Rev 1983;16:69–78. [10] Moropoulou A, Bakolas A and Anagnostopoulou S. Composite materials in ancient structures. Cement Concrete Comp 2005;27:295–300. [11] Mark R. The structural analysis of Gothic cathedrals. Sci Am 1972;227:90–9. [12] Mark R. Structural experimentation in Gothic architecture: large-scale experimentation brought Gothic cathedrals to a level of technical elegance unsurpassed until the last century. Am Sci 1978;66:542–50.
[13] Wolfe M and Mark R. Gothic cathedral buttressing: the experiment at Bourges and its influence. J Soc Archit Hist 1974;33:17–26. [14] McGinnis J. Scientific methodologies in medieval Islam. J Hist Philos 2003;41:307–27. [15] Collier R. Legumes, lemons and streptomycin: a short history of the clinical trial. Can Med Assoc J 2009;180:23–4. [16] Cooper M. Trial by accident: tort law, industrial risks and the history of medical experiment. J Cult Econ 2011;4:81–96. [17] Doll R. Controlled trials: the 1948 watershed. BMJ 1998;317:1217. [18] Medical Research Council. Streptomycin in Tuberculosis Trials Committee. Streptomycin treatment for pulmonary tuberculosis. Brit Med J 1948;ii:769–82. [19] Hanson N. Patterns of discovery: an inquiry into the conceptual foundations of science. Cambridge: Cambridge University Press, 1958. [20] Dietl J. Der Aderlass in der Lungenentzündung. Wien: Kaulfuss Witwe, Prandel & Comp. 1849. [21] Kucharz E. The life and achievements of Joseph Dietl. Clio Medica 1981;16:25–35. [22] Williams DDR and Garner J. The case against “the evidence”: a different perspective on evidence-based medicine. Brit J Psychiat 2002;180:8–12.
SJP0010.1177/1403494813516712S. O. HanssonWhy and for what are clinical trials the gold standard?
Scandinavian Journal of Public Health, 2014; 42(Suppl 13): 41–48
oRIGINAL ARTICLE
Why and for what are clinical trials the gold standard?
SVEN OVE HANSSON Division of Philosophy, Royal Institute of Technology (KTH), Stockholm, Sweden
Abstract The epistemological basis of clinical trials and evidence-based medicine is investigated. Clinical trials are directly actionguiding experiments on treatment effects. This is the reason why well-performed clinical trials take precedence over all other types of studies as far as treatment effects are concerned. The efficiency of public health interventions can be studied with directly action-guiding experiments that have the same strong epistemic justification as clinical trials. However, in order to assess the causality of diseases, information from several types of studies will have to be combined. Here, no single type of studies has priority over all the others. Therefore, evidence hierarchies are less helpful in studies of causality than they are in investigations of the effects of treatments or interventions. Key Words: Action-guiding experiment, clinical trial, epistemic experiment, evidence-based medicine, public health
Introduction Clinical trials are the universally accepted gold standard of evidence-based medicine. If we have wellconducted, randomised, and double-blind clinical studies, then the recommendation is to use them as the basis both for clinical decisions and for policy decisions on healthcare priorities, without giving much weight to other types of studies such as case reports and mechanistic laboratory studies. But evidence-based medicine is continuously under debate, and the role it gives to clinical trials has been questioned. There is a need to carefully investigate the evidential status of clinical trials, both generally and more specifically in relation to the role they have in evidence-based medicine. We need, in particular, to answer the following four questions: 1. Why are clinical trials the gold standard? Many other types of studies, such as laboratory experiments and clinical case reports, contribute to the progress of medical science. Is there a convincing justification why clinical studies should have the privileged role assigned to them in evidence-based medicine?
2. For what questions can clinical trials reasonably be the gold standard? Clinical trials were developed to determine the effects of treatments. Some have wished to restrict their use by excluding certain types of treatments, whereas others have proposed their use for nontreatment issues. For what medical issues should they be used? 3. What role should clinical trials have in the total assessment of evidence? In particular, have they been given the right role in evidence-based medicine as it is usually practiced? 4. Can the same evidence hierarchy be used in public health as in treatment studies? Currently, attempts are being made to extend the scope of evidencebased medicine to new areas such as public health. Can that be done? If so, is there a need to adjust the hierarchy? Can the role of clinical trials be the same? To answer these questions, we need to see clinical trials in a more general context provided by epistemology and the philosophy of science.
Correspondence: Sven Ove Hansson, Division of Philosophy, Royal Institute of Technology (KTH), Stockholm, Sweden. E-mail: [email protected]
© 2014 the Nordic Societies of Public Health DOI: 10.1177/1403494813516712
42 S. O. Hansson Clinical trials as experiments From the viewpoint of scientific methodology, it is important to distinguish between experiments and nonexperimental observations. By an experiment (in the modern sense of the word) is meant a procedure in which some object of study is subjected to interventions (manipulations) that aim at obtaining a predictable outcome or at least predictable aspects of the outcome. Clinical trials are a special type of experiments: they are experiments in which the object of study is a group of patients, the interventions are treatments, and the outcome is the medical condition and general welfare of the patients. The requirement that the outcome of an experiment should be predictable is usually expressed as a condition of repeatability: if the same interventions are performed on similar objects of study under the specified conditions, then the same outcome should be obtained. This requirement is essential for clinical trials since we perform them in order to obtain information that can be relied on in clinical practice. Directly action-guiding experiments In addition to the distinction between experiments and nonexperimental observations we need to distinguish between two types of experiments, namely directly action-guiding and epistemic experiments, respectively [1]. An experiment is directly actionguiding if and only if it satisfies two criteria: (1) The outcome looked for should be some desired goal of human action. (2) The interventions studied should be potential candidates for being performed in a nonexperimental setting in order to achieve that goal. Both these criteria are obviously satisfied by clinical trials. For instance, in a clinical trial of an analgesic the outcome looked for is efficient pain reduction with minimal negative side effects, and the experimental intervention is a treatment that might be administered to achieve exactly that in a clinical, nonexperimental setting. Examples of directly actionguiding experiments in a nonmedical setting include agricultural field trials, many technological tests such as tests of the longevity of light bulbs, and social experiments trying out the effects of different methods of social work. An epistemic experiment aims at providing us with information about the workings of the world we live in. Therefore, the outcome looked for is one that provides such information, and the intervention is constructed to achieve such an outcome. Hence,
when photochemists measure the wavelength of electromagnetic radiation emitted from some chemical reaction, this is done in order to obtain information about the reaction, not in order to find a way to produce light with that specific wavelength.1 Nonclinical biomedical research makes abundant use of epistemic experiments (that may of course be indirectly action-guiding in various ways). The early history of experimentation The history of experiments has usually been treated as a history of scientific experiments. This has led to an exclusive focus on epistemic experiments and on the experimental tradition in modern science that began with the renaissance. However, if we include directly action-guiding experiments, then experiments turn out to have a history that is much older than that of science. This means that, contrary to what is commonly assumed, the origin of experimentation is neither academic nor curiosity-driven. Instead, it is socially widespread and driven by practical needs. Agriculture is the crucial technological innovation that made civilisation possible. Archaeological studies of ancient agriculture have shown that it was not the result of a few inventions or discoveries but of thousands of years of extensive and continuous experimentation [2]. This is exemplified in traditional Andean agriculture where the uncertainties of an unpredictable climate, pests, and other natural events were met by sophisticated risk spreading. Each farmer had many dispersed fields in different microclimates and continuously tried out different crops in different places to see what worked best. Agricultural experimentation is virtually universal in the Andes. In every Indian community there are many people who continually experiment with all plants that come their way in special fields (chacras) usually located near their houses, but some do so in a certain chacra in every production zone where they have fields. They watch how different plants “go” under different climatic conditions…. Simulation and experimentation are routine activities for the native Andean agricultor [3].
In at least one place, these experimental endeavours seem to have been conducted in a highly systematic way. In an Andean site called Moray, close to Cusco in present-day Peru, several natural sinkholes have been terraced by the Incas. There are large climatic differences between different parts of the terrace system, partly due to shadows cast by a nearby mountain. The terraces also have an irrigation system that allowed for variation of water supply between the different terraces. According to oral tradition among
Why and for what are clinical trials the gold standard? 43 local peasants, Moray served as “an Inca agricultural college” [3]. This seems to have been a site for agricultural field trials (i.e. directly action-guiding agricultural experiments). Records from other parts of the world confirm that indigenous and traditional farmers do indeed experiment [4, 5]. The Mende people in Sierra Leone have a special word, “hungoo”, for such experiments. A hungoo can for instance consist in planting two seeds in adjacent rows, and then comparing the output in order to determine which seed was best [6]. Just like farmers, craftspeople of different trades have performed directly action-guiding (i.e. technological) experiments already in prescientific times. As one example of this, extensive experiments on the composition of glass were performed in the early Islamic period in Raqqa (Ar-Raqqah) in eastern Syria. Analysis of artefacts and debris from the 8th to 11th centuries has shown that whereas some glass types had a surprisingly unvarying composition across the centuries, others were subject to much experimentation. These experiments seem to have included a so-called chemical dilution line in order to search systematically for optimal proportions of the main ingredients [7, 8]. Judging by the many types of materials and mixtures developed in the course of human history, systematic experimentation to optimise batch recipes must also have taken place in many other places and at various times throughout the world. It is, for instance, difficult to see how the optimal mixture between copper and tin to obtain as hard bronze as possible could otherwise have been discovered in short time in ancient Egypt [9]. Similarly, in order to obtain mortar of good quality, “[m]asons had to rely on experimentation or information past on orally to understand and learn the properties of the employed materials and the effects they could have produce[d] when added to a mortar” [10]. The great cathedrals that were built in the High Gothic period (around 1140–1350) are astonishing engineering products. During that period, the pillars and other construction elements of new cathedrals became slimmer and slimmer. How could that have been achieved in spite of the lack of any means to calculate the required dimensions? Extensive studies have shown that these cathedrals were in fact experimental building projects, and “the design may have been successively modified on the basis of observation of the buildings during the course of construction” [11]. When a new cathedral was planned, its construction elements could be made somewhat slimmer than those of its predecessors. During the construction period, builders had to be sensitive to signs that the building was not strong enough. One such sign was cracks that appeared in newly set
mortar when temporary construction supports were removed. The tensile strength of medieval mortar was very low, and therefore even relatively small displacements would have been observed in this way. The effects of strong winds on the construction would also have to be monitored. Since the cathedrals took several decades to build, each of them must have been subject to severe storms during construction. Fortunately, if the construction was found to be too weak, the builders did not have to tear down the cathedral and build a new, more sturdy one. Instead they could add construction elements such as flying buttresses that provided the necessary additional support. In this way, “the details of design were worked out with a crude type of experimental stress analysis performed during construction: tensile cracking observed in the weak lime mortar between the stones during the relatively long period of construction could have led to refinements in design” [12] (cf. [13]). Given the prescientific experimental traditions in agriculture and technical crafts, one should perhaps expect to find similar prescientific traditions of directly action-guiding experiments among the traditional healers who were the forerunners of physicians. The selection of herbs and other remedies for diseases can be based on experiments in the same way as the selection of a crop, an agricultural method, the composition of an alloy, or the dimensions of a column. If you want to know whether you can reduce a patient’s fever by giving her a particular herb, give feverish patients the herb and record whether their fever is abated or not. This idea was clearly expressed by Avicenna (Abd Allah ibn Sina, c.980–1037): [It] is like our judgement that the scammony plant is a purgative for bile; for since this [phenomenon] is repeated many times, one abandons that it is among the things which occur by chance, so the mind judged that it belongs to the character of scammony to purge bile and [the mind] gave into it, that is, purging bile is an intrinsic characteristic belonging to scammony [14].
However, this was an exceptional statement for its time, and Avicenna does not report whether he actually performed the experiment in question. I have not been able to find examples of directly action-guiding experiments on the treatment of diseases being performed by indigenous peoples or before the modern scientific era. It is only in the 18th century that we find reports of what we would today call clinical experiments. (The most famous of these was James Lind’s scurvy trial in 1747 [15]). These experiments were performed by physicians who were under the influence of natural science rather than prescientific experimental traditions. And it was not until after the
44 S. O. Hansson Second World War that the modern clinical trial was established as the standard method for investigating treatment effects [16–18]. It is interesting to compare this late introduction of directly action-guiding experiments in medicine with their much earlier introduction in agriculture. Farmers in different parts of the world have performed directly action-guiding experiments since long before modern science, and the introduction of field trials in agricultural science was both earlier and less conflict ridden than that of clinical trials in medical science. The only plausible explanation of this difference that I have been able to find is that the subject matter of human health tends to be much more imbued with ideology and religion than that of agriculture (or, for that matter, glassmaking or building technology). This applies to ancient as well as modern societies. Perhaps the down-to-earth nature of questions such as “can I grow this crop here?” or “will the columns support this type of roof?” facilitated the thought processes that led to the use of directly action-guiding experiments. We may be more prone to look for guidance elsewhere in questions of life or death, health or disease. A strong and theory-independent justification Let us now turn to the justification of using clinical trials to determine treatment effects. It turns out that this practice has a strong justification that is common to the general category of experiments that they belong to, namely directly action-guiding experiments. We can summarise the notion of a directly action-guiding experiment in the form of the following simple recipe: Recipe for directly action-guiding experiments If you want to know if you can achieve Y by doing X, do X and see if Y occurs.2
This recipe is in a sense self-vindicating. In order to find out whether you can achieve Y by doing X, what better method can there be than to do X and see if Y occurs? In particular, we have no problem in justifying the use of an intervention or manipulation (the X of the recipe). We want to know the effects of such an intervention, and then it is much better to actually perform it than to passively observe the workings of nature. This is a simple and indisputable argument that applies to directly action-guiding experiments but not to epistemic experiments. This justification also explains why directly action-guiding experiments should be repeatable. Since the purpose of the experiment is action-guiding, we
need to establish a connection between an intervention-type and a desired outcome. In order to be practically useful, such a connection has to appear regularly: it is not sufficient that something happened once. Someone might counter what was just said by asking: “Can this be true? Can directly action-guiding experiments really be that strongly justified? Have we not learned that all experiments are theory laden?” Well, indeed we have, but the common arguments showing the theory ladenness of experiments [19] refer to epistemic experiments, rather than to directly action-guiding ones. The specific theory ladenness of experiments seems to come with their epistemic interpretations. The following two examples should clarify the difference: 1. In the search for a new treatment against hair loss in dogs (canine follicular dysplasia), a veterinary surgeon performs an experiment on a group of affected dogs. The dogs are divided into a treatment group and a control group. The former group receives a daily dose of a newly synthesised substance. 2. In order to test a biochemical hypothesis about the regulation of cell growth, a researcher looks for an animal model in which a newly synthesised substance is expected to have an easily visible and measurable effect. She finds out that dogs with follicular dysplasia are a suitable such model. She therefore performs an experiment in which dogs with this condition are divided into an experimental group that receives a daily dose of the substance and a control group that does not receive the substance. At least in principle, these two prehistories can lead to exactly the same experiment, in terms of the study objects, the intervention performed, and how the outcome is monitored. However, there is a remarkable difference in terms of theory ladenness. In the first case, you may have whatever reason you want for believing either that the substance will improve the dogs’ health or that it will not. But when the experiment has been completed, if there is a clear difference in hair growth between the two groups, then there is not much scope for interpretation. The experiment just tells us how it is.3 The second case is different. Here, the interpretation of the experiment will have to be based on nontrivial, theoretical assumptions concerning the plausible mechanism behind any observed effect. Obviously, the strong justification of clinical trials (and other directly action-guiding experiments) does not mean that we should rely unconditionally on
Why and for what are clinical trials the gold standard? 45 them regardless of how they have been performed. They are the superior method for finding out the effects of treatments, but like all other methods they can be misapplied or misinterpreted. A whole series of safeguards, such as control groups, blinding, and randomisation, have been developed in order to ensure the reliability of the information obtained from clinical trials. Critical examinations of how clinical trials are conducted and interpreted are essential parts of clinical science. The unwelcomeness of theoryindependent information However, there is also another type of criticism that does not have this positive and essential role in the development of medical knowledge, namely general criticism against the use of clinical trials to determine the effects of medical treatments. Interestingly, such criticism does not seem to have emerged in the other areas where directly action-guiding experiments have a similarly strong standing. I have not been able to find any examples of farmers or agricultural scientists opposing the use of field trials to determine the agricultural properties of crops or farming methods. Neither have I found traces of any resistance against the use of directly actionguiding experiments in technological or engineering contexts. But as already mentioned, issues of health and disease are imbedded in various ideological belief systems that may explain the comparatively late introduction of directly action-guiding experiments in medicine. It should be no surprise that the presentation of strongly supported, theory-independent information such as that provided by good clinical trials can be quite unwelcome for the proponents of theories that may have to be revised or deserted due to that information. A person who reports strong evidence of the inadequacy of a prevalent therapeutic practice may find him- or herself in the same situation as the child in Hans Christian Andersen’s tale of the emperor’s new clothes. The emperor of that tale continued his pompous parade after being told what was open for everyone to see. Academic dignitaries and other defenders of received medical theories have been known to behave similarly in response to incontrovertible, theory-independent information from clinical trials. An interesting early example is an experiment on the treatment of pneumonia that was performed in the late 1840s by the Polish-Austrian physician Joseph Dietl (1804–1878). In those days, the general view among physicians was that pneumonia depended on an imbalance between the bodily fluids. There was
some disagreement on the nature of that imbalance, and consequently some physicians recommended bloodletting whereas others favoured the administration of an emetic. In 1849, Dietl reported an investigation in which he had compared three groups of pneumonia patients. One group had received bloodletting, the second had received an emetic, and the third had received general care but no specific treatment. Mortality among those who had received bloodletting was 20.4%, among those who had received an emetic 20.7%, and among those who had received no specific treatment only 7.4%. Dietl’s message was at first rather negatively received. His critics claimed that since disease and treatment are highly individual issues, they cannot be settled with statistics. They defended the idea of medicine as an intuitive art that goes beyond what can be determined by scientific method. But in the end, Dietl was successful. In the 1870s, the major medical textbooks advised against bloodletting of pneumonia patients [20, 21]. It took more than a century after Dietl’s paper on pneumonia until clinical trials became the generally accepted method for determining treatment effects. This was a slow process, and some pockets of resistance have persisted surprisingly long. The opposition has been strongest in psychiatry [22]. This is unsurprising due to the strong influence in that specialty of theory-based and sometimes ideologised doctrines prescribing what treatments to use. But in recent years, it has become generally accepted in psychiatry that the choice between, for instance, Freudian psychoanalysis and pharmacological treatment should not be based on ideology but instead be made separately for each diagnosis, based on directly actionguiding experiments showing what effects the different treatments have on patients with that diagnosis. Obviously, this openness to reality checks is essential for the future progress of psychiatry. The remaining pockets of resistance in the healthcare sector can now be found among so-called alternative therapists who commonly reject directly action-guiding experiments with arguments very similar to those brought forward by Joseph Dietl’s adversaries. Clinical trials in evidence-based medicine The term “evidence-based medicine” is rhetorically efficient but unfortunately rather unclear. As a first attempt, one might want to define it as medicine based on evidence. However, that would be a much too wide definition. Evidence is, generally speaking, information that should legitimately have an influence on a rational person’s degree of belief in some
46 S. O. Hansson statement. Evidence-based medicine is something much more restricted than medicine based on evidence: it is medicine based on a certain type of evidence hierarchy. Unfortunately, the idea of such a hierarchy is not easy to justify in terms of basic epistemological principles. When we have several pieces of evidence concerning some subject matter, we should ideally take them all into account in our overall assessment. If the mass of evidence is large then this cannot be done in practice, which means that we have to leave out the less important evidence from our final deliberations. This is what a court of law does in a criminal case. If there is strong evidence for instance in the form of a CCTV film showing the defendant committing the crime, then other pieces of evidence can be put aside (although they might have been relevant in the absence of that film). Similarly, if several well-performed clinical trials show that the antihypertensive drug A has a larger antihypertensive effect than its competitor B (at certain dosages), then we can consider the issue as settled, and we do not need to pay much attention to case studies, animal studies, or mechanistic considerations concerning the relative sizes of the antihypertensive effects. For some purposes, it is helpful to explain this in terms of an evidence hierarchy that puts well-performed clinical trials on top and other types of studies on various lower positions. But, however useful an evidence hierarchy is in practice, it is only a simplification or a heuristic device, not an exact model of epistemic justification. There are at least two reasons for this. First, what makes the evidence on a lower level insignificant in the presence of conclusive evidence on the higher level is the conclusiveness of that evidence, not its place in the hierarchy. If evidence from well-performed clinical trials is inconclusive concerning the size of treatment effects, then it may be unwise to disregard other types of information that could have been left out if the evidence from clinical trials were conclusive. Even more importantly, “middle-level” evidence on treatment effects, such as evidence from case–control studies, can never be as conclusive as “high-level” evidence (i.e. clinical trials). Therefore, “low-level” evidence such as case studies and mechanistic information cannot be as easily left out if the best that we have is middle-level rather than highlevel evidence. The second reason is that the relative evidential weight of different types of studies is different for different types of effects. As one example of this, the relative weight of nonblinded studies in relation to blinded ones should be much smaller for treatment effects that are known to be much influenced by the
patient’s expectations, such as pain reduction, than for other effects, such as halted tumour growth. These are just a few examples of the many types of considerations that can legitimately influence the assessment of medical evidence. Although current evidence hierarchies have been constructed to include some such considerations, it is not possible to cover them all. But although the use of fixed hierarchies cannot be justified as epistemically optimal, it can in many cases be justified as a wise compromise between what is epistemically optimal and the advantages of using fixed criteria that cannot be manipulated or unintendedly influenced by hopes and expectations. Evidence-based medicine and public health Can the principles of evidence-based medicine be transferred to public health research? In answering that question it is important to distinguish between different types of research issues in public health. Two major types of such issues are those that concern the efficiency of interventions and the causes of diseases. The efficiency of interventions can be studied with directly action-guiding experiments. Suppose that we want to know whether free distribution of mosquito nets reduces the incidence of malaria in tropical areas. The best way to investigate this is to give out mosquito nets to the population in some malaria-ridden areas but not in others and monitor the incidence of the disease in the different areas. This is not a clinical trial, but like clinical trials it is a directly action-guiding experiment. Therefore it has the same strong epistemic justification as clinical trials. If we have good intervention studies of this type, then they take precedence over all other types of information, just like clinical trials do concerning treatment effects. Questions about causes of diseases are different in this respect. Suppose that we wish to know whether a stressful situation at work increases the risk of insomnia. This is a research question that cannot be answered with directly action-guiding experiments. (We can of course choose to study the related issue whether a certain type of intervention against workplace stress reduces the risk of insomnia, but that is a distinctly different research question.) We clearly need epidemiological studies in which stress levels on the workplace, sleep parameters, and various confounding factors are followed in a sample of the population. However, even with excellent studies of that type we do not have direct access to causality in the same way as we have direct access to intervention
Why and for what are clinical trials the gold standard? 47 effects in a directly action-guiding experiment. The standard problem in studies of causality, namely that we cannot distinguish between “Y causes Z” and “some unknown underlying factor X causes both Y and Z”, cannot be avoided in this type of study. This is a major reason why such epidemiological studies need to be complemented by mechanistic investigations that help us identify plausible causal and confounding factors. (In this case such mechanistic studies may include short-term intervention studies in laboratories, observational studies with nightly EEGs, and perhaps animal studies.) An important general conclusion can be drawn from this: if there is a gold standard for evidence on disease causality, then that standard does not refer to a single type of study that has priority over all others when it is sufficiently well-performed. There is simply no such type of study. In order to draw wellfounded conclusions on the causes of diseases, we have to combine information from different types of investigations. Some of these are studies that rank low on the hierarchies of evidence-based medicine, such as studies of potential physiological or biochemical mechanisms. Their low rank on the hierarchies does not reflect low quality; instead it reflects low relevance for the specific issue of treatment effects that these hierarchies were constructed for. For issues of causality they often have high relevance. Guidelines for assessing the aetiology of disease will have to recognise that complexity, and the same applies to guidelines for risk assessment. Conclusion We have arrived at the following answers to the four questions asked in the introduction: 1. Why are clinical trials the gold standard? This is because they are directly action-guiding experiments. 2. For what questions can clinical trials reasonably be the gold standard? Only for questions about the effects of the interventions (treatments) that are tested in the trial. 3. What role should clinical trials have in the total assessment of evidence? In determining treatment effects, well-performed clinical trials have a privileged position and should take precedence over all other types of evidence. 4. Can the same evidence hierarchy be used in public health as in treatment studies? In studies of the effects of public health interventions, essentially the same hierarchy can be used, with directly action-guiding intervention experiments taking the role of clinical trials. In studies of disease causality, information
from several types of studies must be combined, and there is no single type of study that can take the supreme role on top of the hierarchy that clinical trials have in the assessment of treatment effects. Therefore, the simplified structure of an evidence hierarchy is much less suitable for causality than for intervention effects. Conflict of interest The author declares that there is no conflict of interest. Funding This work was supported by the Bank of Sweden Tercentenary Foundation, grant P11-0510:1. Notes 1. In principle, the same experiment could be performed in search of a way to construct a laser emitting some specific wavelength. Therefore, strictly speaking, we should make a distinction between directly action-guiding and epistemic purposes of experiments. 2. In some cases we have the background knowledge that without X, Y will not occur. But if such knowledge is not available, or can be put to doubt, then we need to perform a control experiment: on similar study objects we refrain from performing X, or perform some relevant alternative to X, in order to see whether Y will then happen. 3. Directly action-guiding experiments are not completely theory-independent in a philosophical sense. Like all empirical generalisations they depend on the theoretical assumptions about regularities in nature that are at play in all inductive reasoning. However, this is a minimal form of theory-dependence that we cannot avoid, in contradistinction to the dependence on more specific theories that has been shown to hold for many epistemic experiments. References [1] Hansson SO. Experiments before science – what science learned from technological experiments. In: Hansson SO (ed) The role of technology in science. Philosophical perspectives 2014. Dordrecht: Springer, in press. [2] Bray W. Ancient food for thought. Nature 2000;408: 145–6. [3] Earls J. The character of Inca and Andean agriculture. Lima: Departamento de Ciencias Sociales Pontificia Universidad Católica del Perú. Available at: http://macareo.pucp.edu.pe/~jearls/documentosPDF/theCharacter.PDF (1998, 29 July 2013). [4] Chandler PM. The indigenous knowledge of ecological processes among peasants in the People’s Republic of China. Agric Human Values 1991;8:59–66.
48 S. O. Hansson [5] Johnson AW. Individuality and experimentation in traditional agriculture. Hum Ecol 1972;1:149–59. [6] Richards P. Farmers also experiment: a neglected intellectual resource in African science. Discov Innovat 1989;1: 19–25. [7] Henderson J, Challis K, O’Hara S, et al. Experiment and innovation: early Islamic industry at al-Raqqa, Syria. Antiquity 2005;79:130–45. [8] Henderson J, McLoughlin SD and McPhail DS. Radical changes in Islamic glass technology: evidence for conservatism and experimentation with new glass recipes from early and middle Islamic Raqqa, Syria. Archaeometry 2004;46:439–68. [9] Malina J. Archaeology and experiment. Nor Archaeol Rev 1983;16:69–78. [10] Moropoulou A, Bakolas A and Anagnostopoulou S. Composite materials in ancient structures. Cement Concrete Comp 2005;27:295–300. [11] Mark R. The structural analysis of Gothic cathedrals. Sci Am 1972;227:90–9. [12] Mark R. Structural experimentation in Gothic architecture: large-scale experimentation brought Gothic cathedrals to a level of technical elegance unsurpassed until the last century. Am Sci 1978;66:542–50.
[13] Wolfe M and Mark R. Gothic cathedral buttressing: the experiment at Bourges and its influence. J Soc Archit Hist 1974;33:17–26. [14] McGinnis J. Scientific methodologies in medieval Islam. J Hist Philos 2003;41:307–27. [15] Collier R. Legumes, lemons and streptomycin: a short history of the clinical trial. Can Med Assoc J 2009;180:23–4. [16] Cooper M. Trial by accident: tort law, industrial risks and the history of medical experiment. J Cult Econ 2011;4:81–96. [17] Doll R. Controlled trials: the 1948 watershed. BMJ 1998;317:1217. [18] Medical Research Council. Streptomycin in Tuberculosis Trials Committee. Streptomycin treatment for pulmonary tuberculosis. Brit Med J 1948;ii:769–82. [19] Hanson N. Patterns of discovery: an inquiry into the conceptual foundations of science. Cambridge: Cambridge University Press, 1958. [20] Dietl J. Der Aderlass in der Lungenentzündung. Wien: Kaulfuss Witwe, Prandel & Comp. 1849. [21] Kucharz E. The life and achievements of Joseph Dietl. Clio Medica 1981;16:25–35. [22] Williams DDR and Garner J. The case against “the evidence”: a different perspective on evidence-based medicine. Brit J Psychiat 2002;180:8–12.
