London Journal of Primary Care 2014;6:95–7
# 2014 Royal College of General Practitioners
Theories of Knowledge
What is evidence? Peter Toon Retired General Practitioner
Few would argue with David Sackett, the pioneer of evidence-based medicine (EBM), that good medical practice should be based on ‘the conscientious, explicit, and judicious use of current best evidence’.1 This is often thought to involve four steps: . . . .
doing a computer search for research on a topic; assessing the studies found against strict critical appraisal criteria; using those which pass to produce a systematic review; summarising the results of systematic reviews on a topic into a guideline, which clinicians must then follow.
This is, however, some way from Sackett’s vision of rational, evidence-based practice. He was clear that the best evidence needs to be integrated with clinical judgement and patient values.2
Limits to systematic reviews Computers have enormously enhanced our ability to find relevant research; searches on paper that took weeks can be completed more thoroughly in minutes. But our knowledge is often still incomplete. ‘Grey’ literature, including studies not published in English and older research, may be missed. There is a bias towards publication of the new and the unexpected, and financial incentives to fund research into new patentable medicines rather than older or non-drug treatments and negative findings may also distort the evidence base. Critical appraisal is laudable, but very strict criteria may result in most studies being discarded. A specialist has been defined as someone who learns more and more about less and less until finally he knows everything about nothing. In the same way, systematic reviews so rigorous in their pursuit of certainty that 98% of studies are discarded may tell us very little with great confidence. No one sets out to conduct poor research, but ‘ars longa vita brevis’ so most research ends up being flawed, and uncertainty is inherent in the universe from the sub-atomic level upwards.
Errors in selecting and applying evidence Many systematic reviews only include randomised controlled trials (RCTs), the ‘gold standard’ of EBM, as described in a partner paper in this issue of LJPC.3 It is a small, but logically dubious jump from seeing controlled trials as the best evidence on which to base clinical actions, to the view that no other evidence is valid. This logically false jump is widespread both within medicine and in society at large. When (as is often the case) RCT research is just not there, we still have to act – for not treating is also an action. Absence of evidence is not evidence of absence, and treatments which seem reasonable, particularly if the risk of harm and the cost are low, may be rational if the evidence is lacking rather than against them. Silver, bronze and stainless steel have their uses as much as gold, and other research designs make their particular contribution to the evidence base of practice too. Also, different research approaches reveal different kinds of insight into the whole picture.4 Good qualitative research gives insight into ‘why’ and ‘how’ questions which no RCT, no matter how well designed, can answer. Guidelines based on systematic reviews were designed as tools, not rules. In Sackett’s words (often omitted when his definition of EBM is quoted), they have to be ‘be integrated with clinical judgement and patient values’.
What is clinical judgement? All clinical situations have three aspects: . . .
general facts relevant to situations of this type; particular facts specific to this unique instance; and values of the patient and doctor.
To make a good clinical judgement we must know the general facts. What are the therapeutic options? What do we know about their effectiveness? Here, EBM makes an important contribution. But, we also need
96
P Toon
to know the particular facts of individual cases, and link them with the general facts and with conscientiously and explicitly reasoned values in order to agree with the patient a plan that is rational from all three perspectives. Numerous particular facts may mean that a guideline does not apply in a case. The patient may differ from the population on which the studies are based, be older or younger, of a different gender or genetic makeup than trial participants. Clinical trials often exclude co-morbidities to obtain a clear result, but they are common and often relevant in practice. Even if the patient is typical of research participants, values may indicate a different plan from that which the guideline advises. The patient’s values may rationally or irrationally make them averse to the evidencebased recommendation. The patient may value a shorter but merrier life than the evidence-based recommendations on, for example, alcohol consumption might allow. The patient may have a fear of tablets or be unwilling to accept the ‘sick role‘, which she or he sees as implicit in accepting treatment as a diabetic or hypertensive. Prof Sir Michael Rawlins, formerly chairman of the National Institute for Health and Clinical Excellence (NICE), estimated that a good guideline would apply in 80% of cases.5 Unfortunately, not all clinicians, and certainly not all judges, health service managers and accountants understand these limitations, and believe that not ‘complying’ with guidelines is automatically poor practice. Many medical concepts, such as heart failure or appendicitis, are both factual categories which evidence tells us can be changed by actions like drugs or surgery and value judgements, in that doctors ought to act on these facts (or at least have a compelling argument for why they did not). As a result, facts and values are often confused in medicine, leading people to try to use factual evidence to support evaluative judgements. Data can tell us the effects of different possible actions to change a condition, but it cannot help us to decide whether this change or this course of action is preferable; this is a question of values. Rational analysis of decisions about values is helped by intellectual tools from moral philosophy and concepts like justice, the good and the flourishing life, not tools of empirical science. Particular care is needed in ‘evidence-based ethics’ – using empirical research on ethical issues. Too often philosophers have made empirical judgements about the nature of the world from their armchairs on the basis of common sense, and rigorously collected data on moral beliefs, motivations and the consequences of actions can help us make good moral decisions, but it cannot replace a philosophically argued position in establishing values and preferences. Attempts to do so
make the illogical, unnoticed slide from ‘is’ to ‘ought’ that Hume famously observed.6 It is often assumed that moral judgements are subjective and personal, as opposed to facts, which are ‘objective’ and universal, and that, therefore, we cannot validly make judgements about moral values in cultures other than our own (cultural relativism) or for anyone else (subjectivism), or that moral judgements are merely matters of feeling (emotivism). Midgley clearly refutes these views.7 To think that believing that killing innocent people is wrong is just as much a matter of personal preference as a taste for whisky rather than gin is to confuse tolerance with negligence. She suggests that we confuse legitimate and unacceptable moral judgement because historically in our society too many unjustified judgements are made. She points out that all moral positions, including relativism, rely on some form of moral judgement. Saying that one ought not to make moral judgement about other cultures is itself a moral judgement, as self-contradictory as the paradox ‘All absolute statements, including this one, are false’. It is also illogical to suggest that values can absolutely take one form in one culture and another in another. Certainly, one set of relationships within a group or network can have idiosyncratic cultural norms, but cultures are not hermetically sealed entities without any contact between each other; instead they are a network of individuals each of whom has a slightly different set of values. How do we define the boundaries of a cultural group? Although there are real differences between what is seen as right and wrong in different cultures, often these do not reflect differences in intention, but in perceptions of how to achieve the same thing. Midgley gives the example of how one should treat the dead.6 In one culture it is considered quite wrong to burn one’s parent’s bodies, since one should eat them; another views eating them with horror, and would be appalled at anything other than cremation. These conflicting attitudes reflect a shared concern to show appropriate respect to the dead, a common moral value based on a value of human relationships.
Conclusion It is important that practice should be evidence based, but we need a more sophisticated understanding of what we mean by evidence. As Thomas says in a partner paper in this issue of LJPC, we need to develop a research approach that is adequate to the complex and dynamic nature of primary care.8 And we need a more nuanced approach to the values underlying healthcare if we are to practise in a truly evidencebased way.
What is evidence?
REFERENCES 1 Sackett DL, Rosenberg WM, Gray JA et al (1996) Evidence based medicine: what it is and what it isn’t. British Medical Journal 312(7023):71–2. www.ncbi.nlm. nih.gov/ pmc/articles/PMC2349778/pdf/bmj00524–0009 2 Sackett D et al (2000) Evidence-Based Medicine: How to practice and teach EBM (2e). Churchill Livingstone: Edinburgh, p. 1. See more at http://www.asha.org/ members/ebp/intro/#sthash.9dnVgjSi.dpuf 3 Baeza JI, Fraser A and Boaz A (2014) Evidence-based practice: reflections from five European case studies. London Journal of Primary Care 6:98–102. 4 Thomas P (2006) General medical practitioners need to be aware of the theories on which our work depends. Annals of Family Medicine 4:450–4. 5 Rawlings M (2011) Address to the RCGP annual conference, Liverpool.
97
6 Hume D (1984) A Treatise of Human Nature, Selby-Bigge LA (ed.). Clarendon Press: Oxford (original published 1739/40). 7 Midgley M (1989) Can’t We Make Moral Judgements? Bristol Press: Bristol. 8 Thomas P (2014) Understanding context in healthcare research and development. London Journal of Primary Care 6:103–5.
ADDRESS FOR CORRESPONDENCE
Peter Toon E-mail: [email protected] Received July 2014, revised August 2014, accepted August 2014
# 2014 Royal College of General Practitioners
Theories of Knowledge
What is evidence? Peter Toon Retired General Practitioner
Few would argue with David Sackett, the pioneer of evidence-based medicine (EBM), that good medical practice should be based on ‘the conscientious, explicit, and judicious use of current best evidence’.1 This is often thought to involve four steps: . . . .
doing a computer search for research on a topic; assessing the studies found against strict critical appraisal criteria; using those which pass to produce a systematic review; summarising the results of systematic reviews on a topic into a guideline, which clinicians must then follow.
This is, however, some way from Sackett’s vision of rational, evidence-based practice. He was clear that the best evidence needs to be integrated with clinical judgement and patient values.2
Limits to systematic reviews Computers have enormously enhanced our ability to find relevant research; searches on paper that took weeks can be completed more thoroughly in minutes. But our knowledge is often still incomplete. ‘Grey’ literature, including studies not published in English and older research, may be missed. There is a bias towards publication of the new and the unexpected, and financial incentives to fund research into new patentable medicines rather than older or non-drug treatments and negative findings may also distort the evidence base. Critical appraisal is laudable, but very strict criteria may result in most studies being discarded. A specialist has been defined as someone who learns more and more about less and less until finally he knows everything about nothing. In the same way, systematic reviews so rigorous in their pursuit of certainty that 98% of studies are discarded may tell us very little with great confidence. No one sets out to conduct poor research, but ‘ars longa vita brevis’ so most research ends up being flawed, and uncertainty is inherent in the universe from the sub-atomic level upwards.
Errors in selecting and applying evidence Many systematic reviews only include randomised controlled trials (RCTs), the ‘gold standard’ of EBM, as described in a partner paper in this issue of LJPC.3 It is a small, but logically dubious jump from seeing controlled trials as the best evidence on which to base clinical actions, to the view that no other evidence is valid. This logically false jump is widespread both within medicine and in society at large. When (as is often the case) RCT research is just not there, we still have to act – for not treating is also an action. Absence of evidence is not evidence of absence, and treatments which seem reasonable, particularly if the risk of harm and the cost are low, may be rational if the evidence is lacking rather than against them. Silver, bronze and stainless steel have their uses as much as gold, and other research designs make their particular contribution to the evidence base of practice too. Also, different research approaches reveal different kinds of insight into the whole picture.4 Good qualitative research gives insight into ‘why’ and ‘how’ questions which no RCT, no matter how well designed, can answer. Guidelines based on systematic reviews were designed as tools, not rules. In Sackett’s words (often omitted when his definition of EBM is quoted), they have to be ‘be integrated with clinical judgement and patient values’.
What is clinical judgement? All clinical situations have three aspects: . . .
general facts relevant to situations of this type; particular facts specific to this unique instance; and values of the patient and doctor.
To make a good clinical judgement we must know the general facts. What are the therapeutic options? What do we know about their effectiveness? Here, EBM makes an important contribution. But, we also need
96
P Toon
to know the particular facts of individual cases, and link them with the general facts and with conscientiously and explicitly reasoned values in order to agree with the patient a plan that is rational from all three perspectives. Numerous particular facts may mean that a guideline does not apply in a case. The patient may differ from the population on which the studies are based, be older or younger, of a different gender or genetic makeup than trial participants. Clinical trials often exclude co-morbidities to obtain a clear result, but they are common and often relevant in practice. Even if the patient is typical of research participants, values may indicate a different plan from that which the guideline advises. The patient’s values may rationally or irrationally make them averse to the evidencebased recommendation. The patient may value a shorter but merrier life than the evidence-based recommendations on, for example, alcohol consumption might allow. The patient may have a fear of tablets or be unwilling to accept the ‘sick role‘, which she or he sees as implicit in accepting treatment as a diabetic or hypertensive. Prof Sir Michael Rawlins, formerly chairman of the National Institute for Health and Clinical Excellence (NICE), estimated that a good guideline would apply in 80% of cases.5 Unfortunately, not all clinicians, and certainly not all judges, health service managers and accountants understand these limitations, and believe that not ‘complying’ with guidelines is automatically poor practice. Many medical concepts, such as heart failure or appendicitis, are both factual categories which evidence tells us can be changed by actions like drugs or surgery and value judgements, in that doctors ought to act on these facts (or at least have a compelling argument for why they did not). As a result, facts and values are often confused in medicine, leading people to try to use factual evidence to support evaluative judgements. Data can tell us the effects of different possible actions to change a condition, but it cannot help us to decide whether this change or this course of action is preferable; this is a question of values. Rational analysis of decisions about values is helped by intellectual tools from moral philosophy and concepts like justice, the good and the flourishing life, not tools of empirical science. Particular care is needed in ‘evidence-based ethics’ – using empirical research on ethical issues. Too often philosophers have made empirical judgements about the nature of the world from their armchairs on the basis of common sense, and rigorously collected data on moral beliefs, motivations and the consequences of actions can help us make good moral decisions, but it cannot replace a philosophically argued position in establishing values and preferences. Attempts to do so
make the illogical, unnoticed slide from ‘is’ to ‘ought’ that Hume famously observed.6 It is often assumed that moral judgements are subjective and personal, as opposed to facts, which are ‘objective’ and universal, and that, therefore, we cannot validly make judgements about moral values in cultures other than our own (cultural relativism) or for anyone else (subjectivism), or that moral judgements are merely matters of feeling (emotivism). Midgley clearly refutes these views.7 To think that believing that killing innocent people is wrong is just as much a matter of personal preference as a taste for whisky rather than gin is to confuse tolerance with negligence. She suggests that we confuse legitimate and unacceptable moral judgement because historically in our society too many unjustified judgements are made. She points out that all moral positions, including relativism, rely on some form of moral judgement. Saying that one ought not to make moral judgement about other cultures is itself a moral judgement, as self-contradictory as the paradox ‘All absolute statements, including this one, are false’. It is also illogical to suggest that values can absolutely take one form in one culture and another in another. Certainly, one set of relationships within a group or network can have idiosyncratic cultural norms, but cultures are not hermetically sealed entities without any contact between each other; instead they are a network of individuals each of whom has a slightly different set of values. How do we define the boundaries of a cultural group? Although there are real differences between what is seen as right and wrong in different cultures, often these do not reflect differences in intention, but in perceptions of how to achieve the same thing. Midgley gives the example of how one should treat the dead.6 In one culture it is considered quite wrong to burn one’s parent’s bodies, since one should eat them; another views eating them with horror, and would be appalled at anything other than cremation. These conflicting attitudes reflect a shared concern to show appropriate respect to the dead, a common moral value based on a value of human relationships.
Conclusion It is important that practice should be evidence based, but we need a more sophisticated understanding of what we mean by evidence. As Thomas says in a partner paper in this issue of LJPC, we need to develop a research approach that is adequate to the complex and dynamic nature of primary care.8 And we need a more nuanced approach to the values underlying healthcare if we are to practise in a truly evidencebased way.
What is evidence?
REFERENCES 1 Sackett DL, Rosenberg WM, Gray JA et al (1996) Evidence based medicine: what it is and what it isn’t. British Medical Journal 312(7023):71–2. www.ncbi.nlm. nih.gov/ pmc/articles/PMC2349778/pdf/bmj00524–0009 2 Sackett D et al (2000) Evidence-Based Medicine: How to practice and teach EBM (2e). Churchill Livingstone: Edinburgh, p. 1. See more at http://www.asha.org/ members/ebp/intro/#sthash.9dnVgjSi.dpuf 3 Baeza JI, Fraser A and Boaz A (2014) Evidence-based practice: reflections from five European case studies. London Journal of Primary Care 6:98–102. 4 Thomas P (2006) General medical practitioners need to be aware of the theories on which our work depends. Annals of Family Medicine 4:450–4. 5 Rawlings M (2011) Address to the RCGP annual conference, Liverpool.
97
6 Hume D (1984) A Treatise of Human Nature, Selby-Bigge LA (ed.). Clarendon Press: Oxford (original published 1739/40). 7 Midgley M (1989) Can’t We Make Moral Judgements? Bristol Press: Bristol. 8 Thomas P (2014) Understanding context in healthcare research and development. London Journal of Primary Care 6:103–5.
ADDRESS FOR CORRESPONDENCE
Peter Toon E-mail: [email protected] Received July 2014, revised August 2014, accepted August 2014
