Ann Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
Published online: October 2, 2014
Clinical Practice Guidelines: Based on Eminence or Evidence? Hania Szajewska Department of Paediatrics, Medical University of Warsaw, Warsaw, Poland
Key Words Evidence-based medicine · Guideline development · Recommendations · Standards
Abstract Background: Too often, clinical practice guidelines, or similar documents, are of poor quality or are eminence based. Consequently, health care decisions might be based on biased or erroneous information. Here, issues related to standards for clinical practice guidelines that ensure the inclusion of objective, transparent, and scientifically valid information will be discussed. Key Messages: To ensure the quality of guidelines, standards for the development of evidence-based clinical practice guidelines have been in development. While differences among them exist, overall, these documents emphasize the need for the establishment of transparency, management of conflicts of interest, use of a multidisciplinary development group, utilization of a systematic literature review, use of grading systems to rate the strength of evidence recommendations, articulation of recommendations, performance of an external review, and regular updating. Conclusions: No clinical practice guidelines are perfect. They vary significantly in quality and, therefore, in the trustworthiness of the yielded recommendations. Hence, it is important that one can distinguish evidence-
© 2014 S. Karger AG, Basel 0250–6807/14/0644–0325$39.50/0 E-Mail [email protected] www.karger.com/anm
based clinical practice guidelines from guidelines that are not based on evidence. Standards for the development of evidence-based guidelines have been developed. If strictly adhered to, this should lead to more valid and trustworthy guidelines. © 2014 S. Karger AG, Basel
Eminence based medicine. The more senior the colleague, the less importance he or she placed on the need for anything as mundane as evidence. Experience, it seems, is worth any amount of evidence. D. Isaacs and D. Fitzgerald
Introduction
A number of terms such as ‘clinical practice guidelines’, ‘recommendations’, ‘regulations’, ‘standards’, ‘position papers’, and ‘opinions’ exist. There is a distinction between these terms; however, they are commonly used interchangeably. Moreover, some of them have the power of law in some, albeit not all, countries. For example, the standard definition of clinical practice guidelines is ‘statements that include recommendations, intended to optimize patient care, that are informed by a systematic review of evidence and an assessment of the benefits and Prof. Hania Szajewska, MD Department of Paediatrics Medical University of Warsaw Dzialdowska 1, PL–01-184 Warsaw (Poland) E-Mail hania @ ipgate.pl
harms of alternative care options’ [1] and they are usually binding for the health care professional. In contrast, a recommendation is a statement of practice that is commonly developed by a scientific organization or a group of individuals; however, usually, it is not binding. Regardless of the terminology, any of the documents mentioned above might be ‘eminence-based’ or ‘evidence-based’ documents. Eminence-based means relying on the opinion of a group of experts in a nonstandardized, prone-to-bias way rather than relying on a critical appraisal of relevant evidence. In contrast, evidence-based implies use of the current best evidence in making decisions about the care of individual patients [2]. Too often, clinical practice guidelines, or similar documents, are of poor quality or are eminence based. Consequently, health care decisions might be based on biased or erroneous information.
Standards for the Development of Evidence-Based Guidelines
To ensure their quality, standards for the development of evidence-based guidelines have been in development. One such example is the document published in 2011 by the Institute of Medicine (IOM) of the US National Academies [1]. Briefly, it includes 8 standards and recommendations for developing trustworthy clinical practice guidelines (table 1). The first standard is the need for the use of transparency. The processes by which a clinical practice guideline is developed and funded should be described transparently to minimize bias, distortion, and conflicts of interest. The second standard relates to the management of conflicts of interest. Potential guideline development group members should declare any conflicts of interest. None, or at most only a small minority, should have such conflicts, including services from which a clinician derives a substantial proportion of his or her income. The chair and cochair should not have conflicts of interest. The guideline development group should eliminate financial ties that create such conflicts. The third standard relates to the composition of the guideline development groups. The groups should be multidisciplinary and balanced. Each group should be composed of methods experts, clinicians, representatives of stakeholders, and populations (patients) expected to be affected by the guideline.
326
Ann Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
Table 1. IOM standards for developing clinical practice guidelines
[1] 1 2 3 4 5 6 7 8
Establishment of transparency Management of conflict of interest Guideline development group composition Clinical practice guideline-systematic review intersection Establishment of evidence foundations for and rating the strength of recommendations Articulation of recommendations External review Updating
The fourth standard relates to the clinical practice guideline-systematic review intersection. Systematic reviews are considered essential to the process of guideline development. Moreover, these systematic reviews must meet the methodological standards developed by the IOM in another document [3]. The fifth standard relates to the importance of establishing evidence foundations for, as well as rating the strength of, recommendations. The guideline authors should explain the reasoning behind each recommendation, summarize the evidence for potential benefits and harms, and describe the quality and quantity of relevant evidence and the role of subjective judgments. A rating of the level of evidence and the strength of the recommendation should be provided. If differences of opinion exist, a description of these differences, together with an explanation, needs to be provided. The sixth standard relates to articulation of the recommendations. This standard includes a description of the action recommended by the guideline and when it should be used. Wording should facilitate measurement of adherence. The seventh standard relates to the need for an external review. This external review should include a full spectrum of stakeholders and reviewers, although, clearly, they should not be identified by name. In case changes are made in response to the reviewers’ comments, these should be fully explained. Finally, a draft of the guideline should be posted for public comment. The eighth standard relates to the need for updating guidelines. Here, the IOM recommends documentation of the developmental dates of the guideline, a systematic review, and a planned update. Moreover, it recommends monitoring of the literature and updating of the guideline when new evidence suggests the need for a change. While different opinions exist, most authors suggest that an update is generally required after 3–5 years [4]. Szajewska
Other examples of standards for the development of evidence-based guidelines include the documents developed by large-scale, guideline production organizations such as the Scottish Intercollegiate Guidelines Network (SIGN) [5] or, in the UK, the National Institute for Clinical Excellence (NICE) [6]. Taken together, these standards strengthen the credibility of clinical practice guidelines. Consequently, they are now being adapted for use by an increasing number of organizations and scientific societies.
Conflict of Interest
‘There’s no such thing as a free lunch.’ This famous quote by Milton Friedman, an American economist, reflects one of the major issues related to guideline development, i.e. conflict of interest. The documents developed by the IOM and others emphasize the importance of the process for managing conflicts of interest. Ideally, the guideline authors should have no conflicts of interest. In the real world, however, this is not always the case. Individuals, particularly academics, with recognized expertise in a given area are more likely to be invited to contribute to guideline development. At the same time, they are also more likely to be involved in industry-supported activities. As a consequence, financial conflict among the authors of guidelines is common. One recent study showed that 71% of chairs of clinical policy committees and 90.5% of cochairs had financial conflicts of interest [7]. It remains unclear whether real change is feasible. In many settings, research collaboration between academic investigators and industry is encouraged by universities, public funding bodies, and governmental organizations. Horizon 2020, the biggest EU Research and Innovation program, promotes collaboration with the ultimate goal of making it easier to deliver innovations. However, it is in the interest of all parties involved that such collaboration yields reliable results. For this reason, some organizations, such as the European Society for Paediatric Gastroenterology, Hepatology and Nutrition and the Early Nutrition Academy, developed recommendations for the conduct of public-private research collaborations [8]. In medicine, conflict of interest generally implies financial ties of the authors with industry, i.e. medical product manufacturers. However, conflict of interest is a much broader matter. In addition to a financial conflict of interest, a nonfinancial conflict of interest may exist. Examples of nonfinancial conflicts of interest include strongly held personal beliefs related to the topic of the Clinical Practice Guidelines
guidelines, personal relationships, institutional relationships (when the institution could be benefited or harmed, depending on the direction of the recommendation), or a desire for career development [9]. Similar to financial conflicts of interest, nonfinancial conflicts of interest may lead one to question the reliability of the guidelines. Clearly, ignoring conflicts of interest, whether financial or nonfinancial, may jeopardize the trustworthiness of the produced guidelines. On the other hand, it remains uncertain whether the exclusion of all individuals with any financial or nonfinancial conflict of interest (if feasible) would indeed ensure the integrity of the clinical practice guidelines. This is because evidence of the superiority of the latter circumstance is very limited [10]. Moreover, the tolerable proportion of subjects with a conflict of interest remains unclear. Finally, current discussions on conflicts of interest mainly focus on the authors and/or sponsors of the guidelines. However, other players might also not be free of conflicts of interest, either financial or nonfinancial. For example, lobbyists, patients’ associations, researchers, lawyers, medical writers, advertising and social networking specialists, physicians, administrators, and policy makers may also be industry supported [11]. Taken together, while eliminating all conflicts of interest of all parts involved might be an unrealistic goal, moving toward greater transparency and disclosure of potential or factual conflicts of interest should be a priority.
Role of Systematic Reviews in Guideline Development
Generally, guideline development groups emphasize the importance of a rigorous systematic review of the evidence. This is a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research and to collect and analyze data from studies that are included in the review [12]. Systematic reviews with or without a meta-analysis (the latter means that statistical methods have been used to analyze and summarize the results of the included trials) are now a well-established means of reviewing existing evidence and integrating findings from various studies. There is no single model of cooperation between the teams developing systematic review(s) and clinical practice guidelines. In practice, it ranges from totally independent groups to groups in close interaction throughout the guideline development process [13]. One major issue with systematic reviews is Ann Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
327
Associated modules
Setup phase
Prepare for ADAPTE process
Preparation
Define health questions
Scope and purpose
Search and screen guidelines
Search and screen
Assess guidelines
Assessment
Decide and select
Decision and selection
Draft guideline report
Customization
External review
External review
Plan for future review and update
Aftercare planning
Produce final guideline
Final production
Finalization phase
Fig. 1. Guideline adaptation. Summary of
Color version available online
Tasks
Adaptation phase
Phases
the ADAPTE process [15, permission obtained].
that their development is time-consuming (estimates vary from 216 to 2,518 h, with a mean of 1,139 h) and, consequently, costly [14].
Guideline Adaptation
The development of high-quality, evidence-based guidelines is a time-consuming process that requires substantial resources and expertise. Considering the above, guideline adaptation is an option to be considered by organizations or countries with financial constraints and/or a lack of knowledge and experience in developing guidelines. Even if money is no object, this prevents the waste of resources and avoids duplication of efforts. To address these problems, recently, a systematic approach to adapting guidelines produced in one setting for use in a different cultural or organization setting was proposed. This 328
Ann Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
process, known as the ADAPTE process, was developed by a group of researchers, guideline developers, and guideline implementers. The proposed process includes 3 main phases, each with a set of modules (fig. 1). The detailed ADAPTE methodology (and accompanying tool kits) is available on a website [15].
Disagreement among Guidelines
Guidelines that address the same topic but were developed by different expert groups sometimes disagree. In the field of infant nutrition, one such example is recommendations on iron intake. Recently, the American Academy of Pediatrics recommended that iron supplements (1 mg/kg/ day) should be given from 4 months of age onwards to all exclusively breast-fed infants and to partially breast-fed infants who receive more than half of their daily feeds as huSzajewska
Evidence
Clinical outcome Aware
Accepted
Fig. 2. The paths from research to im-
proved health outcomes. Adapted from Glasziou and Haynes [31].
man milk [16]. These recommendations have been questioned by some experts in the field [17, 18]. Also, the Committee on Nutrition of the European Society for Paediatric Gastroenterology, Hepatology and Nutrition does not recommend general iron supplementation of breast-fed European infants after the age of 4–6 months [19]. Exceptions include infants from high-risk groups (low socioeconomic status or living in areas with a high prevalence of iron deficiency anemia) if the infant has a low intake of iron-rich complementary foods. Such a disagreement among recommendations often reflects an unavailability of highquality evidence. However, the differences are usually only minor and relate to specific issues only. Some guidelines have generated lively discussion. In the field of pediatric nutrition, one example is the discussion on the optimal duration of exclusive breast-feeding and the age of introduction of complementary foods [20]. There is consistency across guidelines that exclusive breast-feeding for about 6 months is a desirable goal [21]. Partial breast-feeding, as well as breast-feeding for shorter periods of time, is also valuable. However, the optimal age for introduction of complementary foods (i.e. between 4 and 6 months vs. after 6 months) in exclusively breast-fed infants remains controversial [20].
Concerns about Published Research
Everyone agrees that guidelines should be based upon the best available research evidence. However, there are some concerns that a substantial portion of published research might be wrong. Ioannidis [22] discussed these concerns in his paper ‘Why most published research findClinical Practice Guidelines
Applicable
Able
Physician
Acted on
Agreed
Adhered to
Patient
ings are false’. In his opinion, whether or not a research claim is true may depend on factors such as ‘the study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field’. He also provided the following set of circumstances in which a research finding is less likely to be true: ‘when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance’. Worldwide, efforts are being made to improve the quality of scientific research. Examples include a number of standards for reporting clinical research developed jointly by journal editors, together with other stakeholders such as methodologists, researchers, clinicians, and members of professional organizations, to ensure that all details of design, conduct, and analysis are included in the manuscript. Editors now require that authors follow these standards when submitting a manuscript for publication. Among others, these standards include the following: CONSORT (Consolidated Standards of Reporting Trials) – a checklist and a flowchart for randomized controlled trials (RCT); the CONSORT statement has several extension statements (e.g. for structured abstracts, cluster RCT, pragmatic trials, and noninferiority and equivalence RCT) [23–27]; PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) – a checklist and a flowchart for a systematic review or metaanalysis of RCT [28]; STARD (Standards for the ReportAnn Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
329
ing of Diagnostic Accuracy Studies) – a checklist and a flowchart for assessing diagnostic accuracy [29], and MOOSE (Meta-analysis of Observational Studies in Epidemiology) – a checklist for a meta-analysis of observational trials [30]. If rigorously followed, the statement that a substantial portion of published research might be wrong will be substantially minimized.
‘Red Flags’ for Clinical Practice Guideline Panels
Any guidelines should be critically assessed. Recently, one expert working group summarized red flags regarding the constitution of guideline panels that should raise skepticism among guideline readers as well as the editors of medical journals [11]. Briefly, caution is needed if: ‘(1) sponsor(s) is a professional society that receives substantial industry funding; (2) sponsor is a proprietary company, or is undeclared or hidden; (3) committee chair(s) have any financial conflict; (4) multiple panel members have any financial conflict; (5) any suggestion of committee stacking that would pre-ordain a recommendation regarding a controversial topic; (6) no or limited involvement of an expert in methodology in the evaluation of evidence; (7) no external review, and (8) no inclusion of non-physician/patient representative/community stakeholders.’ Of note, these red flags have not been validated.
From Guidelines to Practice
tion. Figure 2 summarizes barriers between guidelines (or other research) and health outcomes that extend from awareness to adherence [31].
Conclusions
No clinical practice guidelines are perfect. They vary significantly in quality and, therefore, in the trustworthiness of the yielded recommendations. Hence, it is important that one can distinguish evidence-based clinical practice guidelines from guidelines that are not based on evidence. Standards for the development of evidencebased guidelines have been developed. If strictly adhered to, this should lead to more valid and trustworthy guidelines.
Acknowledgements This article reports an extended version of the presentation made at the conference ‘The Power of Programming 2014. International Conference on Developmental Origins of Adiposity and Long-Term Health’, held in Munich on March 13–15, 2014. To participate in the conference, the author received funding from the European Union’s 7th Framework Programme (FP7/2007–2013), EarlyNutrition project, under grant agreement No. 289346, and from the Ministry of Science and Higher Education (No. 2464/ 7.PR/2012/2).
Disclosure Statement
Even if the best evidence-based guidelines are available, they do not automatically lead to improved health outcomes. Hence, there is interest in knowledge transla-
The author has no conflict of interest to declare.
References 1 Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E (eds): Clinical Practice Guidelines We Can Trust. Washington, National Academies Press, 2011. 2 Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn’t. BMJ 1996;312: 71–72.
330
3 National Research Council: Finding What Works in Health Care: Standards for Systematic Reviews. Washington, The National Academies Press, 2011. 4 Vernooij RW, Sanabria AJ, Solà I, AlonsoCoello P, Martínez García L: Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks. Implement Sci 2014;9:3. 5 Scottish Intercollegiate Guidelines Network: SIGN 50: a guideline developer’s handbook. 2011. http://www.sign.ac.uk/pdf/sign50.pdf.
Ann Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
6 National Institute for Health and Clinical Excellence: The Guidelines Manual. London, National Institute for Health and Clinical Excellence, 2012. 7 Kung J, Miller RR, Mackowiak PA: Failure of clinical practice guidelines to meet institute of medicine standards: two more decades of little, if any, progress. Arch Intern Med 2012; 172:1628–1633.
Szajewska
8 Koletzko B, Benninga MA, Godfrey KM, Hornnes PJ, Kolaček S, Koletzko S, Lentze MJ, Mader S, McAuliffe FM, Oepkes D, Oddy WH, Phillips A, Rzehak P, Socha P, Szajewska H, Symonds ME, Taminiau J, Thapar N, Troncone R, Vandenplas Y, Veereman G: Public-private collaboration in clinical research during pregnancy, lactation, and childhood: joint position statement of the early nutrition academy and the European Society for Pediatric Gastroenterology, Hepatology, and Nutrition. J Pediatr Gastroenterol Nutr 2014; 58:525–530. 9 Viswanathan M, Carey TS, Belinson SE, Berliner E, Chang S, Graham E, Guise JM, Ip SS, Maglione MA, McCrory D, McPheeters M, Newberry SJ, Sista P, White CM: Identifying and Managing Nonfinancial Conflicts of Interest for Systematic Reviews. Rockville, Agency for Healthcare Research and Quality, 2013. http://www.ncbi.nlm.nih.gov/books/ NBK148586/. 10 George JN, Vesely SK, Woolf SH: Conflicts of interest and clinical recommendations: comparison of two concurrent clinical practice guidelines for primary immune thrombocytopenia developed by different methods. Am J Med Qual 2014;29:53–60. 11 Lenzer J, Hoffman JR, Furberg CD, Ioannidis JP; Guideline Panel Review Working Group: Ensuring the integrity of clinical practice guidelines: a tool for protecting patients. BMJ 2013;347:f5535. 12 Higgins JPT, Green S (eds): Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.2. 2009. www.cochrane-handbook.org (accessed March 19, 2014). 13 National Research Council: Finding What Works in Health Care: Standards for Systematic Reviews. Washington, National Academies Press, 2011. http://www.nap.edu/catalog.php?record_id=13059.
Clinical Practice Guidelines
14 Petticrew M, Roberts H: Systematic Reviews in the Social Sciences: a Practical Guide. Oxford, Blackwell, 2006. 15 ADAPTE Collaboration: ADAPTE Process: Resource Toolkit for Guideline Adaptation Version 2.0. 2009. http://www.g-i-n.net. 16 Baker RD, Greer FR; American Academy of Pediatrics, Committee on Nutrition: Diagnosis and prevention of iron deficiency and iron-deficiency anemia in infants and young children (0–3 years of age). Pediatrics 2010; 126:1040–1050. 17 Furman LM: Exclusively breastfed infants: iron recommendations are premature. Pediatrics 2011; 127:e1098–e1099, author reply e1101–e1104. 18 Hernell O, Lönnerdal B. Recommendations on iron questioned. Pediatrics 2011; 127:e1099–e1101, author reply e1101–e1104. 19 Domellöf M, Braegger C, Campoy C, Colomb V, Decsi T, Fewtrell M, Hojsak I, Mihatsch W, Molgaard C, Shamir R, Turck D, van Goudoever J; ESPGHAN Committee on Nutrition: Iron requirements of infants and toddlers. J Pediatr Gastroenterol Nutr 2014; 58: 119–129. 20 Fewtrell M, Wilson DC, Booth I, Lucas A: Six months of exclusive breast feeding: how good is the evidence? BMJ 2010;342:c5955. 21 World Health Organization: 55th World Health Assembly: Infant and Young Child Nutrition (WHA55.25). Geneva, WHO, 2002. http://apps.who.int/gb/archive/pdf_files/ WHA55/ewha5525.pdf. 22 Ioannidis JPA: Why most published research findings are false. PLoS Med 2005;2:e124. 23 Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG: CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c869. 24 Campbell MK, Piaggio G, Elbourne DR, Altman DG; CONSORT Group: Consort 2010 statement: extension to cluster randomised trials. BMJ 2012;345:e5661.
25 Hopewell S, Clarke M, Moher D, Wager E, Middleton P, Altman DG, Schulz KF; CONSORT Group: CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLoS Med 2008;5:e20. 26 Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D; CONSORT group; Pragmatic Trials in Healthcare (Practihc) group: Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ 2008; 337:a2390. 27 Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DG; CONSORT Group: Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA 2012;308:2594–2604. 28 Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D: The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009;339:b2700. 29 Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Lijmer JG, Moher D, Rennie D, de Vet HC: Standards for reporting of diagnostic accuracy: towards complete and accurate reporting of studies of diagnostic accuracy – the STARD initiative. BMJ 2003;326:41–44. 30 Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB: Meta-analysis of observational studies in epidemiology: a proposal for reporting – Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000;283:2008–2012. 31 Glasziou P, Haynes B: The path from research to improve health outcomes. Evid Based Med 2005;10:4–7.
Ann Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
331
Copyright: S. Karger AG, Basel 2014. Reproduced with the permission of S. Karger AG, Basel. Further reproduction or distribution (electronic or otherwise) is prohibited without permission from the copyright holder.
Published online: October 2, 2014
Clinical Practice Guidelines: Based on Eminence or Evidence? Hania Szajewska Department of Paediatrics, Medical University of Warsaw, Warsaw, Poland
Key Words Evidence-based medicine · Guideline development · Recommendations · Standards
Abstract Background: Too often, clinical practice guidelines, or similar documents, are of poor quality or are eminence based. Consequently, health care decisions might be based on biased or erroneous information. Here, issues related to standards for clinical practice guidelines that ensure the inclusion of objective, transparent, and scientifically valid information will be discussed. Key Messages: To ensure the quality of guidelines, standards for the development of evidence-based clinical practice guidelines have been in development. While differences among them exist, overall, these documents emphasize the need for the establishment of transparency, management of conflicts of interest, use of a multidisciplinary development group, utilization of a systematic literature review, use of grading systems to rate the strength of evidence recommendations, articulation of recommendations, performance of an external review, and regular updating. Conclusions: No clinical practice guidelines are perfect. They vary significantly in quality and, therefore, in the trustworthiness of the yielded recommendations. Hence, it is important that one can distinguish evidence-
© 2014 S. Karger AG, Basel 0250–6807/14/0644–0325$39.50/0 E-Mail [email protected] www.karger.com/anm
based clinical practice guidelines from guidelines that are not based on evidence. Standards for the development of evidence-based guidelines have been developed. If strictly adhered to, this should lead to more valid and trustworthy guidelines. © 2014 S. Karger AG, Basel
Eminence based medicine. The more senior the colleague, the less importance he or she placed on the need for anything as mundane as evidence. Experience, it seems, is worth any amount of evidence. D. Isaacs and D. Fitzgerald
Introduction
A number of terms such as ‘clinical practice guidelines’, ‘recommendations’, ‘regulations’, ‘standards’, ‘position papers’, and ‘opinions’ exist. There is a distinction between these terms; however, they are commonly used interchangeably. Moreover, some of them have the power of law in some, albeit not all, countries. For example, the standard definition of clinical practice guidelines is ‘statements that include recommendations, intended to optimize patient care, that are informed by a systematic review of evidence and an assessment of the benefits and Prof. Hania Szajewska, MD Department of Paediatrics Medical University of Warsaw Dzialdowska 1, PL–01-184 Warsaw (Poland) E-Mail hania @ ipgate.pl
harms of alternative care options’ [1] and they are usually binding for the health care professional. In contrast, a recommendation is a statement of practice that is commonly developed by a scientific organization or a group of individuals; however, usually, it is not binding. Regardless of the terminology, any of the documents mentioned above might be ‘eminence-based’ or ‘evidence-based’ documents. Eminence-based means relying on the opinion of a group of experts in a nonstandardized, prone-to-bias way rather than relying on a critical appraisal of relevant evidence. In contrast, evidence-based implies use of the current best evidence in making decisions about the care of individual patients [2]. Too often, clinical practice guidelines, or similar documents, are of poor quality or are eminence based. Consequently, health care decisions might be based on biased or erroneous information.
Standards for the Development of Evidence-Based Guidelines
To ensure their quality, standards for the development of evidence-based guidelines have been in development. One such example is the document published in 2011 by the Institute of Medicine (IOM) of the US National Academies [1]. Briefly, it includes 8 standards and recommendations for developing trustworthy clinical practice guidelines (table 1). The first standard is the need for the use of transparency. The processes by which a clinical practice guideline is developed and funded should be described transparently to minimize bias, distortion, and conflicts of interest. The second standard relates to the management of conflicts of interest. Potential guideline development group members should declare any conflicts of interest. None, or at most only a small minority, should have such conflicts, including services from which a clinician derives a substantial proportion of his or her income. The chair and cochair should not have conflicts of interest. The guideline development group should eliminate financial ties that create such conflicts. The third standard relates to the composition of the guideline development groups. The groups should be multidisciplinary and balanced. Each group should be composed of methods experts, clinicians, representatives of stakeholders, and populations (patients) expected to be affected by the guideline.
326
Ann Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
Table 1. IOM standards for developing clinical practice guidelines
[1] 1 2 3 4 5 6 7 8
Establishment of transparency Management of conflict of interest Guideline development group composition Clinical practice guideline-systematic review intersection Establishment of evidence foundations for and rating the strength of recommendations Articulation of recommendations External review Updating
The fourth standard relates to the clinical practice guideline-systematic review intersection. Systematic reviews are considered essential to the process of guideline development. Moreover, these systematic reviews must meet the methodological standards developed by the IOM in another document [3]. The fifth standard relates to the importance of establishing evidence foundations for, as well as rating the strength of, recommendations. The guideline authors should explain the reasoning behind each recommendation, summarize the evidence for potential benefits and harms, and describe the quality and quantity of relevant evidence and the role of subjective judgments. A rating of the level of evidence and the strength of the recommendation should be provided. If differences of opinion exist, a description of these differences, together with an explanation, needs to be provided. The sixth standard relates to articulation of the recommendations. This standard includes a description of the action recommended by the guideline and when it should be used. Wording should facilitate measurement of adherence. The seventh standard relates to the need for an external review. This external review should include a full spectrum of stakeholders and reviewers, although, clearly, they should not be identified by name. In case changes are made in response to the reviewers’ comments, these should be fully explained. Finally, a draft of the guideline should be posted for public comment. The eighth standard relates to the need for updating guidelines. Here, the IOM recommends documentation of the developmental dates of the guideline, a systematic review, and a planned update. Moreover, it recommends monitoring of the literature and updating of the guideline when new evidence suggests the need for a change. While different opinions exist, most authors suggest that an update is generally required after 3–5 years [4]. Szajewska
Other examples of standards for the development of evidence-based guidelines include the documents developed by large-scale, guideline production organizations such as the Scottish Intercollegiate Guidelines Network (SIGN) [5] or, in the UK, the National Institute for Clinical Excellence (NICE) [6]. Taken together, these standards strengthen the credibility of clinical practice guidelines. Consequently, they are now being adapted for use by an increasing number of organizations and scientific societies.
Conflict of Interest
‘There’s no such thing as a free lunch.’ This famous quote by Milton Friedman, an American economist, reflects one of the major issues related to guideline development, i.e. conflict of interest. The documents developed by the IOM and others emphasize the importance of the process for managing conflicts of interest. Ideally, the guideline authors should have no conflicts of interest. In the real world, however, this is not always the case. Individuals, particularly academics, with recognized expertise in a given area are more likely to be invited to contribute to guideline development. At the same time, they are also more likely to be involved in industry-supported activities. As a consequence, financial conflict among the authors of guidelines is common. One recent study showed that 71% of chairs of clinical policy committees and 90.5% of cochairs had financial conflicts of interest [7]. It remains unclear whether real change is feasible. In many settings, research collaboration between academic investigators and industry is encouraged by universities, public funding bodies, and governmental organizations. Horizon 2020, the biggest EU Research and Innovation program, promotes collaboration with the ultimate goal of making it easier to deliver innovations. However, it is in the interest of all parties involved that such collaboration yields reliable results. For this reason, some organizations, such as the European Society for Paediatric Gastroenterology, Hepatology and Nutrition and the Early Nutrition Academy, developed recommendations for the conduct of public-private research collaborations [8]. In medicine, conflict of interest generally implies financial ties of the authors with industry, i.e. medical product manufacturers. However, conflict of interest is a much broader matter. In addition to a financial conflict of interest, a nonfinancial conflict of interest may exist. Examples of nonfinancial conflicts of interest include strongly held personal beliefs related to the topic of the Clinical Practice Guidelines
guidelines, personal relationships, institutional relationships (when the institution could be benefited or harmed, depending on the direction of the recommendation), or a desire for career development [9]. Similar to financial conflicts of interest, nonfinancial conflicts of interest may lead one to question the reliability of the guidelines. Clearly, ignoring conflicts of interest, whether financial or nonfinancial, may jeopardize the trustworthiness of the produced guidelines. On the other hand, it remains uncertain whether the exclusion of all individuals with any financial or nonfinancial conflict of interest (if feasible) would indeed ensure the integrity of the clinical practice guidelines. This is because evidence of the superiority of the latter circumstance is very limited [10]. Moreover, the tolerable proportion of subjects with a conflict of interest remains unclear. Finally, current discussions on conflicts of interest mainly focus on the authors and/or sponsors of the guidelines. However, other players might also not be free of conflicts of interest, either financial or nonfinancial. For example, lobbyists, patients’ associations, researchers, lawyers, medical writers, advertising and social networking specialists, physicians, administrators, and policy makers may also be industry supported [11]. Taken together, while eliminating all conflicts of interest of all parts involved might be an unrealistic goal, moving toward greater transparency and disclosure of potential or factual conflicts of interest should be a priority.
Role of Systematic Reviews in Guideline Development
Generally, guideline development groups emphasize the importance of a rigorous systematic review of the evidence. This is a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research and to collect and analyze data from studies that are included in the review [12]. Systematic reviews with or without a meta-analysis (the latter means that statistical methods have been used to analyze and summarize the results of the included trials) are now a well-established means of reviewing existing evidence and integrating findings from various studies. There is no single model of cooperation between the teams developing systematic review(s) and clinical practice guidelines. In practice, it ranges from totally independent groups to groups in close interaction throughout the guideline development process [13]. One major issue with systematic reviews is Ann Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
327
Associated modules
Setup phase
Prepare for ADAPTE process
Preparation
Define health questions
Scope and purpose
Search and screen guidelines
Search and screen
Assess guidelines
Assessment
Decide and select
Decision and selection
Draft guideline report
Customization
External review
External review
Plan for future review and update
Aftercare planning
Produce final guideline
Final production
Finalization phase
Fig. 1. Guideline adaptation. Summary of
Color version available online
Tasks
Adaptation phase
Phases
the ADAPTE process [15, permission obtained].
that their development is time-consuming (estimates vary from 216 to 2,518 h, with a mean of 1,139 h) and, consequently, costly [14].
Guideline Adaptation
The development of high-quality, evidence-based guidelines is a time-consuming process that requires substantial resources and expertise. Considering the above, guideline adaptation is an option to be considered by organizations or countries with financial constraints and/or a lack of knowledge and experience in developing guidelines. Even if money is no object, this prevents the waste of resources and avoids duplication of efforts. To address these problems, recently, a systematic approach to adapting guidelines produced in one setting for use in a different cultural or organization setting was proposed. This 328
Ann Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
process, known as the ADAPTE process, was developed by a group of researchers, guideline developers, and guideline implementers. The proposed process includes 3 main phases, each with a set of modules (fig. 1). The detailed ADAPTE methodology (and accompanying tool kits) is available on a website [15].
Disagreement among Guidelines
Guidelines that address the same topic but were developed by different expert groups sometimes disagree. In the field of infant nutrition, one such example is recommendations on iron intake. Recently, the American Academy of Pediatrics recommended that iron supplements (1 mg/kg/ day) should be given from 4 months of age onwards to all exclusively breast-fed infants and to partially breast-fed infants who receive more than half of their daily feeds as huSzajewska
Evidence
Clinical outcome Aware
Accepted
Fig. 2. The paths from research to im-
proved health outcomes. Adapted from Glasziou and Haynes [31].
man milk [16]. These recommendations have been questioned by some experts in the field [17, 18]. Also, the Committee on Nutrition of the European Society for Paediatric Gastroenterology, Hepatology and Nutrition does not recommend general iron supplementation of breast-fed European infants after the age of 4–6 months [19]. Exceptions include infants from high-risk groups (low socioeconomic status or living in areas with a high prevalence of iron deficiency anemia) if the infant has a low intake of iron-rich complementary foods. Such a disagreement among recommendations often reflects an unavailability of highquality evidence. However, the differences are usually only minor and relate to specific issues only. Some guidelines have generated lively discussion. In the field of pediatric nutrition, one example is the discussion on the optimal duration of exclusive breast-feeding and the age of introduction of complementary foods [20]. There is consistency across guidelines that exclusive breast-feeding for about 6 months is a desirable goal [21]. Partial breast-feeding, as well as breast-feeding for shorter periods of time, is also valuable. However, the optimal age for introduction of complementary foods (i.e. between 4 and 6 months vs. after 6 months) in exclusively breast-fed infants remains controversial [20].
Concerns about Published Research
Everyone agrees that guidelines should be based upon the best available research evidence. However, there are some concerns that a substantial portion of published research might be wrong. Ioannidis [22] discussed these concerns in his paper ‘Why most published research findClinical Practice Guidelines
Applicable
Able
Physician
Acted on
Agreed
Adhered to
Patient
ings are false’. In his opinion, whether or not a research claim is true may depend on factors such as ‘the study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field’. He also provided the following set of circumstances in which a research finding is less likely to be true: ‘when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance’. Worldwide, efforts are being made to improve the quality of scientific research. Examples include a number of standards for reporting clinical research developed jointly by journal editors, together with other stakeholders such as methodologists, researchers, clinicians, and members of professional organizations, to ensure that all details of design, conduct, and analysis are included in the manuscript. Editors now require that authors follow these standards when submitting a manuscript for publication. Among others, these standards include the following: CONSORT (Consolidated Standards of Reporting Trials) – a checklist and a flowchart for randomized controlled trials (RCT); the CONSORT statement has several extension statements (e.g. for structured abstracts, cluster RCT, pragmatic trials, and noninferiority and equivalence RCT) [23–27]; PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) – a checklist and a flowchart for a systematic review or metaanalysis of RCT [28]; STARD (Standards for the ReportAnn Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
329
ing of Diagnostic Accuracy Studies) – a checklist and a flowchart for assessing diagnostic accuracy [29], and MOOSE (Meta-analysis of Observational Studies in Epidemiology) – a checklist for a meta-analysis of observational trials [30]. If rigorously followed, the statement that a substantial portion of published research might be wrong will be substantially minimized.
‘Red Flags’ for Clinical Practice Guideline Panels
Any guidelines should be critically assessed. Recently, one expert working group summarized red flags regarding the constitution of guideline panels that should raise skepticism among guideline readers as well as the editors of medical journals [11]. Briefly, caution is needed if: ‘(1) sponsor(s) is a professional society that receives substantial industry funding; (2) sponsor is a proprietary company, or is undeclared or hidden; (3) committee chair(s) have any financial conflict; (4) multiple panel members have any financial conflict; (5) any suggestion of committee stacking that would pre-ordain a recommendation regarding a controversial topic; (6) no or limited involvement of an expert in methodology in the evaluation of evidence; (7) no external review, and (8) no inclusion of non-physician/patient representative/community stakeholders.’ Of note, these red flags have not been validated.
From Guidelines to Practice
tion. Figure 2 summarizes barriers between guidelines (or other research) and health outcomes that extend from awareness to adherence [31].
Conclusions
No clinical practice guidelines are perfect. They vary significantly in quality and, therefore, in the trustworthiness of the yielded recommendations. Hence, it is important that one can distinguish evidence-based clinical practice guidelines from guidelines that are not based on evidence. Standards for the development of evidencebased guidelines have been developed. If strictly adhered to, this should lead to more valid and trustworthy guidelines.
Acknowledgements This article reports an extended version of the presentation made at the conference ‘The Power of Programming 2014. International Conference on Developmental Origins of Adiposity and Long-Term Health’, held in Munich on March 13–15, 2014. To participate in the conference, the author received funding from the European Union’s 7th Framework Programme (FP7/2007–2013), EarlyNutrition project, under grant agreement No. 289346, and from the Ministry of Science and Higher Education (No. 2464/ 7.PR/2012/2).
Disclosure Statement
Even if the best evidence-based guidelines are available, they do not automatically lead to improved health outcomes. Hence, there is interest in knowledge transla-
The author has no conflict of interest to declare.
References 1 Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E (eds): Clinical Practice Guidelines We Can Trust. Washington, National Academies Press, 2011. 2 Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn’t. BMJ 1996;312: 71–72.
330
3 National Research Council: Finding What Works in Health Care: Standards for Systematic Reviews. Washington, The National Academies Press, 2011. 4 Vernooij RW, Sanabria AJ, Solà I, AlonsoCoello P, Martínez García L: Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks. Implement Sci 2014;9:3. 5 Scottish Intercollegiate Guidelines Network: SIGN 50: a guideline developer’s handbook. 2011. http://www.sign.ac.uk/pdf/sign50.pdf.
Ann Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
6 National Institute for Health and Clinical Excellence: The Guidelines Manual. London, National Institute for Health and Clinical Excellence, 2012. 7 Kung J, Miller RR, Mackowiak PA: Failure of clinical practice guidelines to meet institute of medicine standards: two more decades of little, if any, progress. Arch Intern Med 2012; 172:1628–1633.
Szajewska
8 Koletzko B, Benninga MA, Godfrey KM, Hornnes PJ, Kolaček S, Koletzko S, Lentze MJ, Mader S, McAuliffe FM, Oepkes D, Oddy WH, Phillips A, Rzehak P, Socha P, Szajewska H, Symonds ME, Taminiau J, Thapar N, Troncone R, Vandenplas Y, Veereman G: Public-private collaboration in clinical research during pregnancy, lactation, and childhood: joint position statement of the early nutrition academy and the European Society for Pediatric Gastroenterology, Hepatology, and Nutrition. J Pediatr Gastroenterol Nutr 2014; 58:525–530. 9 Viswanathan M, Carey TS, Belinson SE, Berliner E, Chang S, Graham E, Guise JM, Ip SS, Maglione MA, McCrory D, McPheeters M, Newberry SJ, Sista P, White CM: Identifying and Managing Nonfinancial Conflicts of Interest for Systematic Reviews. Rockville, Agency for Healthcare Research and Quality, 2013. http://www.ncbi.nlm.nih.gov/books/ NBK148586/. 10 George JN, Vesely SK, Woolf SH: Conflicts of interest and clinical recommendations: comparison of two concurrent clinical practice guidelines for primary immune thrombocytopenia developed by different methods. Am J Med Qual 2014;29:53–60. 11 Lenzer J, Hoffman JR, Furberg CD, Ioannidis JP; Guideline Panel Review Working Group: Ensuring the integrity of clinical practice guidelines: a tool for protecting patients. BMJ 2013;347:f5535. 12 Higgins JPT, Green S (eds): Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.2. 2009. www.cochrane-handbook.org (accessed March 19, 2014). 13 National Research Council: Finding What Works in Health Care: Standards for Systematic Reviews. Washington, National Academies Press, 2011. http://www.nap.edu/catalog.php?record_id=13059.
Clinical Practice Guidelines
14 Petticrew M, Roberts H: Systematic Reviews in the Social Sciences: a Practical Guide. Oxford, Blackwell, 2006. 15 ADAPTE Collaboration: ADAPTE Process: Resource Toolkit for Guideline Adaptation Version 2.0. 2009. http://www.g-i-n.net. 16 Baker RD, Greer FR; American Academy of Pediatrics, Committee on Nutrition: Diagnosis and prevention of iron deficiency and iron-deficiency anemia in infants and young children (0–3 years of age). Pediatrics 2010; 126:1040–1050. 17 Furman LM: Exclusively breastfed infants: iron recommendations are premature. Pediatrics 2011; 127:e1098–e1099, author reply e1101–e1104. 18 Hernell O, Lönnerdal B. Recommendations on iron questioned. Pediatrics 2011; 127:e1099–e1101, author reply e1101–e1104. 19 Domellöf M, Braegger C, Campoy C, Colomb V, Decsi T, Fewtrell M, Hojsak I, Mihatsch W, Molgaard C, Shamir R, Turck D, van Goudoever J; ESPGHAN Committee on Nutrition: Iron requirements of infants and toddlers. J Pediatr Gastroenterol Nutr 2014; 58: 119–129. 20 Fewtrell M, Wilson DC, Booth I, Lucas A: Six months of exclusive breast feeding: how good is the evidence? BMJ 2010;342:c5955. 21 World Health Organization: 55th World Health Assembly: Infant and Young Child Nutrition (WHA55.25). Geneva, WHO, 2002. http://apps.who.int/gb/archive/pdf_files/ WHA55/ewha5525.pdf. 22 Ioannidis JPA: Why most published research findings are false. PLoS Med 2005;2:e124. 23 Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG: CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c869. 24 Campbell MK, Piaggio G, Elbourne DR, Altman DG; CONSORT Group: Consort 2010 statement: extension to cluster randomised trials. BMJ 2012;345:e5661.
25 Hopewell S, Clarke M, Moher D, Wager E, Middleton P, Altman DG, Schulz KF; CONSORT Group: CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLoS Med 2008;5:e20. 26 Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D; CONSORT group; Pragmatic Trials in Healthcare (Practihc) group: Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ 2008; 337:a2390. 27 Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DG; CONSORT Group: Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA 2012;308:2594–2604. 28 Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D: The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009;339:b2700. 29 Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Lijmer JG, Moher D, Rennie D, de Vet HC: Standards for reporting of diagnostic accuracy: towards complete and accurate reporting of studies of diagnostic accuracy – the STARD initiative. BMJ 2003;326:41–44. 30 Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB: Meta-analysis of observational studies in epidemiology: a proposal for reporting – Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000;283:2008–2012. 31 Glasziou P, Haynes B: The path from research to improve health outcomes. Evid Based Med 2005;10:4–7.
Ann Nutr Metab 2014;64:325–331 DOI: 10.1159/000365041
331
Copyright: S. Karger AG, Basel 2014. Reproduced with the permission of S. Karger AG, Basel. Further reproduction or distribution (electronic or otherwise) is prohibited without permission from the copyright holder.
