BASIC/CLINICAL SCIENCE
Quality Appraisal of Acne Clinical Practice Guidelines, 2 0 0 8 -2 0 1 3 Christopher Kawala, Dilan Fernando, and ]erry K.L. Tan
Background: A cne v u lg a ris is a c o m m o n c h ro n ic disease, an d evidence-based c lin ic a l p ra ctice g u id e lin e s (CPGs) can p ro v id e c re d ib le tre a tm e n t in fo rm a tio n .
M ethod: A lite ra tu re search fo r acne CPGs p u b lis h e d b e tw e e n J a n u a ry 2008 and S e p te m b e r 2013 w a s co n d u cte d . T w o re vie w e rs in d e p e n d e n tly a p p lie d th e A p p ra isa l o f G u id e lin e s fo r Research and E v a lu a tio n (AGREE II) in s tru m e n t. M e th o d o lo g ic a l q u a lity w a s e va lu a te d b y ra n k in g in AGREE II d o m a in s and th e h ig h e s t n u m b e r o f ite m s s co rin g ab o ve th e n e u tra l th re s h o ld score.
Results: Four CPGs fu lfille d th e se le c tio n c rite ria , and th e h ig h e s t ran ked w e re th e E uropean and M a laysia n. H ig h e st scores ach ie ve d b y th e fo rm e r w e re fo r s c o p e /p u rp o s e , sta k e h o ld e r in v o lv e m e n t, and rig o r o f d e v e lo p m e n t an d by th e la tte r w e re fo r sco p e / p u rp o se , c la rity o f p re s e n ta tio n , an d a p p lic a b ility . A p p lic a b ility w a s th e lo w e s t s co rin g o f all d o m a in s fo r all CPGs.
Conclusion: E uro pe an an d M a la ysia n acne CPGs w e re ran ked h ig h e s t fo r m e th o d o lo g ic a l q u a lity an d m a y serve to in fo rm c lin ic a l p ra c tic e an d g u id e lin e a d a p ta tio n .
Contexte: L'acne v u lg a ire e st un e m a la d ie c h ro n iq u e fre q u e n te , e t les g u id e s de p ra tiq u e c lin iq u e (GPC) fo n d e s s u r des d o nn ee s p ro b a n te s p e u v e n t e tre un e s o u rce d 'in fo rm a tio n fia b le s u r le tra ite m e n t.
M ethode: N o u s avo n s e n tre p ris un e xa m en de la d o c u m e n ta tio n a la rech erch e de GPC su r l'acne, p u b lie s de ja n v ie r 2008 a se p te m b re 2013. Deux e x a m in a te u rs o n t pro ced e, cha cun de le u r cote , a re v a lu a tio n des g u id e s a I'a id e de I'in s tru m e n t AGREE II
(Appraisal o f G uidelines fo r Research and Evaluation ). La q u a lite m e th o d o lo g iq u e a ete evaluee en fo n c tio n des d o m a in e s AGREE II et du n o m b re le p lu s eleve d 'e le m e n ts a y a n t o b te n u un e n o te au-dessus du seu il de n e u tra lity .
Resultats: Q u a tre GPC re s p e c ta ie n t les crite re s de se le ctio n , e t les g u id e s e u rop een e t m a la isie n o n t o b te n u les re s u lta ts les p lu s sieves: le p re m ie r en ce q u i co n ce rn e le ch a m p e t les o b je c tifs , la p a rtic ip a tio n des g ro u p e s concernes, e t la rig u e u r d 'e la b o ra tio n ; le seco nd en ce q u i con cern e le ch a m p e t les o b je c tifs , la cla rte de la p re s e n ta tio n , e t I'a p p lic a b ilite . D 'a ille u rs, il e st a s o u lig n e r que I'a p p lic a b ilite est le d o m a in e q u i a o b te n u le score le p lu s fa ib le dans to u s les GPC.
Conclusion: Les GPC e u rop een e t m a la isie n su r l'a cn e o n t o b te n u les m e ille u re s co te s au reg ard de la q u a lite m e th o d o lo g iq u e , et its p e u v e n t s 'a ve re r des sources d 'in fo rm a tio n s u r la p ra tiq u e c lin iq u e e t I'a d a p ta tio n des lig n e s d ire ctrice s.
CNE VULGARIS ranks as one of the most burden some skin diseases globally as rated by disabilityadjusted life-years.1 It is a chronic inflammatory disease affecting 85% of people age 12 to 24 years2 and can persist into adulthood.3 Acne is associated with reduced quality of life and increased rates of anxiety, depression, and suicidal ideation.4 Estimated cost of acne treatment (direct and
A
From the Department o f Medicine, Schulich School o f Medicine and Dentistry, University o f Western Ontario, London, ON. Address reprint requests to: Jerry K.L. Tan, MD, FRCPC, Department of Medicine, Schulich School o f Medicine and Dentistry, University of Western Ontario, 2224 Walker Road, Suite 300, Windsor, O N N 8 W 5L7; email: [email protected]. D O I 1 0 .2 3 1 0/77 50 .2 01 4.131 90
€) 2014 Canadian Dermatology Association
indirect) and loss of productivity were estimated at US$3 billion annually a decade ago in the United States.5 Potential treatments for acne are myriad and include nonprescription products, prescription medications, device-based options, and complementary and alternative treatments. Additionally, the introduction of novel treat ment options over the past decade with varying levels of supportive evidence can further lead to provider and patient uncertainty, added cost, and inadequate outcomes. Quality health care is effective, efficient, accessible, equitable, safe, acceptable, and patient centered.6 Although various factors can lead to disparities in quality health care, one major issue has been the discrepancy between medical evidence and clinical practice. Key within the framework of knowledge translation/implementation is the development of high-quality evidence-based knowledge tools that are
DECKER^ jjfy Canadian Dermatology Association I Journal of Cutaneous Medicine and Surgery, Vol 18, No 6 (November/December), 2014: pp 385-391
385
Kawala et at
relevant, accessible, and comprehensible to stakeholders, including health care providers, patients, and their supporting personnel.' However, the availability of multi ple clinical practice guidelines (CPGs) of variable quality can lead to further uncertainty, confusion, and potential misapplication. Accordingly, an imperative step in evalua tion of CPGs is evaluation of their quality. In particular, this process can identify those worthy of adoption for clinical practice, adaptation for different regions/countries, or translation into patient decision aids to assist in informed shared decision making. The objective of the current study is to evaluate the quality of recent acne CPGs using a validated instrument, the Appraisal of Guidelines for Research and Evaluation (AGREE II) tool.8
Methods A systematic literature search was performed to identify recently published evidence-based acne CPGs. Four computerized databases (ie, Guidelines International Network, PubMed, Sumsearch, and Tripdatabase) were queried with the key words “acne” and “guidelines” and search dates delimited from January 2008 to September 11,
2013. There was no language restriction for inclusion. Excluded were duplications, position papers, editorials, non-evidence-based recommendations, systematic reviews, primary research, and publications focused on single treatments (Figure 1). Further methodological details were requested of CPG developers when information within the documents was considered insufficient or unclear. Selected CPGs were evaluated using the AGREE II instrument. AGREE II comprises 23 items in 6 domains: 3 for scope and purpose, 3 for stakeholder involvement, 8 for rigor of development, 3 for clarity of presentation, 4 for applicability, and 2 for editorial independence. Each item was rated on a scale of 1 to 7 (with 1 representing “strongly disagree” and 7 representing “strongly agree”), with higher scores indicating higher quality. Item scores > 4 were reflective of greater quality.8 Two reviewers (D.F., C.K.) trained in the use of AGREE II by means of an online tutorial9,10 independently evaluated each CPG. This was followed by deliberation of disparate scores to achieve consensus. Standardized domain scores were calculated by summing scores of the individual items in each domain and standardizing the total as a percentage of the maximum score for that domain, as per AGREE II scoring instructions. Overall
Figure 1. Flow diagram for literature selection of acne clinical practice guide lines (CPGs), 2008-2013. This figure illustrates the search strategy and iden tification of evidence-based acne CPGs.
\7
386
Canadian Dermatology Association I Journal of Cutaneous Medicine and Surgery, Vol 18, No 6 (November/December), 2014: pp 385-391
Quality Appraisal o f Acne Clinical Practice Guidelines
quality rating of each CPG was based on comparison across this cohort. Specific recommendations for CPG use were not proffered as the raters were unable to adequately assess this item.
Results Four acne CPGs fulfilled the selection criteria: Malaysian,11 United Kingdom (UK),12 American Acne and Rosacea Society (AARS),13 and European S3 (Euro)14 (Table 1). Of these, three addressed adolescent and adult populations, whereas the AARS guidelines solely addressed pediatric patients. These four CPGs uniformly excluded acne variants such as acne conglobata, acne fulminans, cosmetic acne, drug-induced acne, and chloracne. Furthermore, management of acne scars, postinflammatory hyperpig mentation, rosacea, and folliculitis were excluded. All guidelines included topical, systemic, hormonal, combina tion, and maintenance therapies. With the exception of the Euro CPG, they also included dietary factors and complementary and alternative therapies as management options (Table 2). The Malaysian, Euro, and UK guidelines used the original version of the AGREE instrument as a guide in their development process. These three trended to higher scores in scope and purpose, rigor of development, and clarity of presentation than the AARS guidelines (Table 3). The Euro and Malaysian CPGs each had three highest and two second highest domain scores within this cohort. For the remainder, the UK and AARS guidelines each had one highest score and three second highest scores. The Euro guideline ranked first for scope/purpose, stakeholder involvement, and rigor of development and second for applicability and editorial independence. The Malaysian guideline ranked first for scope/purpose, clarity of pre sentation, and applicability and second for rigor of
development and editorial independence. Another mea sure of CPG quality is the total number of item scores beyond neutral (score > 4). The greatest number was observed for the Euro CPG (19), whereas the remainder had lower numbers (AARS, 17; Malaysian, 16; UK, 16). The Euro, Malaysian, and UK guidelines each achieved full scores in scope and purpose by describing overall objectives, identifying specific health questions, and addres sing the target population. For stakeholder involvement, the Euro CPG received a full score as it addressed all items within the domain. Specifically, it explicitly listed members of the group contributing to each section. Additionally, patient preferences were included in devel opment by explicitly describing the process incorporating such preferences into recommendations. The Euro CPG included a thorough explanation of the intended audience and how the guideline may be used by the latter. The highest scoring CPG for rigor of development was the Euro guideline as information on search criteria was provided in adequate detail to facilitate replication. Furthermore, the Euro guideline solely described the formal consensus technique used in the formulation of recommendations, thus directly addressing methods for formulating recommendations described. The Malaysian CPG ranked first in clarity of presentation by clearly stating the recommended action for each patient group, along with reasons for treatment selection, other treat ment options available, and relevant clinical scenarios. Treatment options were easily identifiable and presented in a single-page flowchart. Applicability was the lowest scoring domain for all CPGs, with the Malaysian guideline achieving a high score of 50% but nevertheless scoring inadequately in monitoring and auditing and in advice/tools for recommendation implementation pro vided. In editorial independence, the AARS guideline scored highest as it explicitly included a statement that
Table 1. Criteria for Guideline Selection Criteria Inclusion (for selection, must be positive) Addressed acne vulgaris Guideline documents Evidence-based recommendations Exclusion (for selection, must be negative) Did not address acne vulgaris or addressed sequelae of acne (scarring, dyspigmentation) Duplications Not guidelines (primary research, position papers, editorials, single interventions) No evidence-based recommendations AARS — American Acne and Rosacea Society; Euro
Malaysian
UK
AARS
Euro
+ + +
+ + +
+ + +
+ + +
-
-
-
-
-
-
-
-
-
-
= European S3; UK = U nited Kingdom .
-----------------------------------------------------------------------------------------------------------------Canadian Dermatology Association I Journal of Cutaneous Medicine and Surgery, Vol 18, No 6 (November/December), 2014: pp 385-391
387
Kawala et al
Table 2. Evidence-Based Acne Clinical Practice Guidelines, 2008-2013 Group
Year
Malaysian M inistry of H ealth11 (Malaysian)
2012
Title “M anagem ent of Acne”
Scope Topical therapy Systemic therapy H orm onal therapy M aintenance therapy Com bination therapy Dietary factors Corticosteroid injection Physical therapy
Sowerby Centre for Health
2009
“Acne Vulgaris (Prodigy)”
Informatics at Newcastle12 (UK)
Complementary/alternative medicines Topical therapy Systemic therapy H orm onal therapy C om bination therapy Dietary factors M aintenance therapy Physical therapy
European evidence-based (S3)
2012
“European Evidence-Based (S3) Guidelines for the Treatm ent of Acne”
guidelines14 (Euro)
Complementary/alternative medicines Topical therapy Systemic therapy Physical therapy Com bination therapy H orm onal therapy Maintenance therapy
American Acne and Rosacea Society13 (AARS)
2013
“Evidence-Based Recommendations for the Diagnosis and Treatm ent o f Pediatric Acne”
Topical therapy Systemic therapy Com bination therapy H orm onal therapy Dietary factors O ver-the-counter medications
the funding body did not influence the content of the guideline.
Discussion CPGs are the end-product of translational research and play a critical role in the practical application of evidence-based medicine. The quality of CPGs can vary greatly; thus, indicators to evaluate methodological quality of guidelines are required. This process can facilitate discernment guide lines of higher methodological quality and therefore greater credibility. The AGREE instrument is a tool that has been developed to evaluate the methodological quality of CPGs. The AGREE Next Steps Consortium (2009) further refined the original instrument to improve its measurement ability, ease of use for reviewers, and value for target populations by developing the AGREE II instrument.15 Three of the four CPGs (Malaysian, UK, and Euro) made specific reference to the original AGREE tool in
development and, consequently, had similar scores in many of the domains. It is noteworthy that the AARS guideline scored similarly to this group despite not citing AGREE in its development. All CPGs had strong domain scores for scope and purpose, with the three AGREE-guided CPGs (Euro, Malaysian, and UK) each scoring 100%. The UK domain score for stakeholder involvement was significantly weaker than the other CPGs due to a lack of target population involvement. Target population inclusion in the develop ment process is important as CPGs are typically produced with health care providers as the target audience. However, a focus on the latter may inadequately address patient concerns, preferences, and values to assist in the informed treatment decision-making process.16,12 Euro, Malaysian, and UK domain scores for rigor of development were strong in contrast to the overall neutral domain score for the AARS guideline. This relatively weaker score is largely due to inadequate methodological detail in the development
Canadian Dermatology Association I Journal o f Cutaneous Medicine and Surgery, Vol IS, No 6 (November/December), 2014: pp 385-391
Quality Appraisal of Acne Clinical Practice Guidelines
Table 3. AGREE II Scores of Acne Clinical Practice Guidelines, 2008-2013 Domain Domain 1: scope and purpose Overall objectives described Health questions described Patients described Domain score (%) Domain 2: stakeholder involvement Relevant professional groups included Patient views sought Target users defined Domain score (%) Domain 3: rigor of development Systematic literature search Selection criteria described Strengths and limits of evidence described Methods for formulating recommendations described Benefits, side effects, and risks considered in recommendations Link between recommendations and evidence External review before publication Procedure for updating guideline provided Domain score (%) Domain 4: clarity of presentation Recommendations are specific and unambiguous Different management options presented Recommendations are easily identifiable Domain score (%) Domain 5: applicability Facilitators and barriers to application described Advice/tools for recommendation implementation provided Resource implications considered Monitoring and auditing criteria provided Domain score (%) Domain 6: editorial independence Views of funding body have not influenced guideline content Competing interests recorded and addressed Domain score (%)
Euro
Malaysian
UK
AARS
7 7 7 100
7 7 7 100
7 7 7 100
7 5 7 89
7 7 7 100
6 2 7 67
2 2 6 39
6 6 5 78
7 7 7 7 6 7 5 7 94
7 6 6 3 (4)* 7 7 3 5 77
7 5 6 1 7 7 5 7 77
1 6 6 3 7 5 2 1 48
6 6 7 89
7 7 7 100
7 7 6 94
6 7 7 94
4 2 4 2 33
4 5 4 3 50
3 2 6 1 33
1 5 5 1 33
6 5 75
4 7 75
3 5 50
7 5 83
AGREE = Appraisal of Guidelines for Research and Evaluation; AARS = American Acne and Rosacea Society; Euro = European S3; UK = United Kingdom. *Score modified after receiving additional information from guideline author(s).
process and in describing the means by which specific recommendations were developed. All CPGs scored well in clarity of presentation. All CPGs had low scores for applicability, and this may reflect the downstream aspect of this domain, which can be addressed only after completion of the guidelines. 18 This consistent shortfall is apparent in CPGs in medicine and is not unique to acne—and has led to the development of a specific working group within guideline development networks to address this issue. One such initiative, the
Guidelines Implementability Research and Application Network (GIRAnet), is endeavoring to evaluate and apply tools for implementation to assist future guideline developments. 19 The low scores for the UK, Euro, and Malaysian guidelines in this domain were unexpected as they cited guidance by the original AGREE tool in their development process. Addressing applicability for a nascent guideline to encourage implementation should include a template of future initiatives to enhance recognition, uptake, and use by relevant stakeholders.
C anadian D erm atology A ssociation I Journal o f Cutaneous Medicine and Surgery, Vol 18, No 6 (November/December), 2014: pp 385-391
389
Kawala et al
Measuring the effect of these efforts in enhancing outcomes for specific target populations and in overall health use should also be considered. Domain scores for editorial independence were strong for all except the UK guideline, which had a neutral score. This weaker score was due to shortcomings in reporting funding body influence on guideline content. Considerable cost is associated with the development of CPGs, and in many countries, funding from governmental agencies is unavailable. In these circumstances, their development necessitates funding from nongovernmental and/or indus try sources. However, funding from the latter may be perceived as biasing CPG content. However, governmental funding for CPGs must also be considered within the potential agenda of reducing health care expenditures. In either case, the explicit exclusion of funding bodies from the guideline development process is imperative to minimize the risk of bias. One means to address these issues is to segregate the expert panel of the CPG from those soliciting funding support. Furthermore, the expert panel should not be made aware of the identity of the funding sources until after the CPG is ready for submission. Furthermore, to minimize the risk of bias, funding solicitors should not be involved in treatment recommendations or the voting process to generate consensus. These practical measures may facilitate the development of high-quality CPGs with adequate re sources while minimizing the risk of bias. We recognize the following limitations in this study. Subjectivity is inherent in scoring with AGREE II as differences can result based on reviewer interpretation and valuation of components addressing particular items. Nevertheless, the use of multiple raters can reduce scoring variation to facilitate the tool’s ability to discern highfrom low-quality guidelines.20 Evaluation of CPGs with the AGREE II tool is also limited by the wording of item criteria. There were several items where the fulfillment of a particular criterion required an explicit statement, provid ing no flexibility for interpretation or inference. Therefore, it was possible that low scores represented failure to present necessary information rather than shortcomings in methodological quality. To ensure that scoring was accurate, we queried guideline developers about these items as well as those scoring below the neutral threshold. The AGREE II instrument does not allow one to assess the content or the quality of recommendations found within a CPG.8 A recent publication evaluated acne CPGs with the AGREE II instrument over a 10-year period (2002-2012) and, similar to the present study, identified the Euro and Malaysian CPGs as the most highly rated.21 We sought a 390
more circumscribed timeframe of 5 years and extended the timeframe to include the current year (2013) to enhance the timeliness of treatment recommendations for adapta tion and transformation to different applications.
Conclusion Acne is a highly burdensome global condition for which evidence-based treatment paradigms from high-quality CPGs are available and may be positively impactful. We identified high-quality acne CPGs that may serve to inform daily clinical care and provide a template on which to develop additional tools for health care providers, institu tional stakeholders, patients, and their support groups. Improvements in applicability would facilitate the transla tion of these efforts to enhance patient outcomes.
Acknowledgment Financial disclosure of authors: Jerry K.L. Tan has served as an advisor, speaker, and/or trialist for Allergan, Bayer, Cipher, Dermira, Galderma, Roche, Stiefel/GSK, and Valeant. Christopher Kawala and Dilan Fernando have no conflicts to disclose.
References 1. Institute for Health Metrics and Evaluation. The global burden of disease: generating evidence, guiding policy. Seattle (WA): Institute for Health Metrics and Evaluation; 2013. Available at: http:// www.healthmetricsandevaluation.org/sites/default/ffles/policy_report/ 201 l/GBD_Generating%20Evidence_Guiding%20Policy%20FINAL. pdf (accessed May 30, 2013). 2. White GM. Recent findings in the epidemiologic evidence, classification, and subtypes of acne vulgaris. J Am Acad Dermatol 1998;39(2 Pt 3):S34-7, doi:10.1016/S0190-9622(98)70442-6. 3. Schafer T, Nienhaus A, Vieluf D, et al. Epidemiology of acne in the general population: the risk of smoking. Br J Dermatol 2001;145: 100^, doi:10.1046/j. 1365-2133.2001.04290.x. 4. Dunn LK, O’Neill JL, Feldman SR. Acne in adolescents: quality of life, self-esteem, mood, and psychological disorders. Dermatol Online J 2011;17( 1). Available at: http://www.ncbi.nlm.nih.gov/ pubmed/21272492 (accessed March 8, 2013). 5. Bickers DR, Lim HW, Margolis D, et al. The burden of skin diseases: 2004 a joint project of the American Academy of Dermatology Association and the Society for Investigative Dermatology. J Am Acad Dermatol 2006;55:490-500, doi:10.1016/ j.jaad.2006.05.048. 6. Bengoa R, Kawar R, Key P, et al. Quality of care: a process for making strategic choices in health systems. Geneva: World Health Organization; 2006. 7. Davis D, Goldman J, Palda VA. Handbook on clinical practice guidelines. Ottawa: Canadian Medical Association; 2007.
Canadian Dermatology Association I Journal o f Cutaneous Medicine and Surgery, Vol 18, No 6 (November/December), 2014: pp 385-391
Quality Appraisal of Acne Clinical Practice Guidelines
8. AGREE Next Steps Consortium. The AGREE II instrument. Available at: http://www.agreetrust.org 2009 (accessed January 28, 2013). 9. AGREE Next Steps Consortium. The AGREE II instrument overview tutorial. Available at: http://agree.machealth.ca/players/ open/index.html. 2009 (accessed January 28, 2013). 10. AGREE Next Steps Consortium. The AGREE II instrument tutorial and practice exercise. Available at: http://agree.machealth.ca/ openinstrumentfeedback.aspx?id=918e38cl-a84d-45aa-8343-145c 06eea243. 2009 (accessed January 28, 2013). 11. CPG Secretariat, Health Technology Assessment Section, Medical Development Division, Ministry of Health, Malaysia. Clinical practice guidelines. Management of acne. January 2012. Available at: http://www.moh.gov.my (accessed November 2012). 12. Sowerby Centre for Health Informatics at Newcastle. Acne vulgaris (prodigy). Available at: http://prodigy.clarity.co.uk/acne_vulgaris (accessed February 13, 2013). 13. Eichenfield LF, Krakowski AC, Piggott C, et al. Evidence-based recommendations for the diagnosis and treatment of pediatric acne. Pediatrics 2013;131 Suppl 3:S163-86, doi:10,1542/peds.20130490B. 14. Nast A, Dreno B, Bettoli V, et al. European evidence-based (S3) guidelines for the treatment of acne. J Eur Acad Dermatol Venereol 2012;26 Suppl 1:1-29, doi:10.1111/j.l468-3083.2011.04374.x.
15. Brouwers M, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in healthcare. CMAJ 2010;182:E839M2, doi:10.1503/cmai.090449. 16. MacLean S, Mulla S, Akl EA, et al. Patient values and preferences in decision making for antithrombotic therapy: a systematic review: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest 2012;141 Suppl 2: elS-23S. 17. Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 1: performance, usefulness and areas for impro vement. CMAJ 2010;182:1045-52, doi:10.1503/cmaj.091714. 18. Field MJ, Lohr KN. Clinical practice guidelines: directions for a new program. Washington (DC): National Academy Press; 1990. 19. Guidelines International Network. Introducing GIRAnet. Available at: http://www.g-i-n.net/activities/implementation/giranet (accessed July 9, 2013). 20. Vlayen J, Aertgeerts B, Hannes K, et al. A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit. Int J Qual Health Care 2005;17:235—42, doi :10.1093/intqhc/mzi027. 21. Sanclemente G, Acosta J-L, Tamayo M-E, et al. Clinical practice guidelines for treatment of acne vulgaris: a critical appraisal using the AGREE II instrument. Arch Dermatol Res 2014;306:269-77, doi:10.1007/s00403-013-1394-x.
m C anadian D erm atology A ssociation I Journal o f Cutaneous Medicine and Surgery, Vol 18, No 6 (November/December), 2014: pp 385-391
391
Copyright of Journal of Cutaneous Medicine & Surgery is the property of Decker Publishing and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Quality Appraisal of Acne Clinical Practice Guidelines, 2 0 0 8 -2 0 1 3 Christopher Kawala, Dilan Fernando, and ]erry K.L. Tan
Background: A cne v u lg a ris is a c o m m o n c h ro n ic disease, an d evidence-based c lin ic a l p ra ctice g u id e lin e s (CPGs) can p ro v id e c re d ib le tre a tm e n t in fo rm a tio n .
M ethod: A lite ra tu re search fo r acne CPGs p u b lis h e d b e tw e e n J a n u a ry 2008 and S e p te m b e r 2013 w a s co n d u cte d . T w o re vie w e rs in d e p e n d e n tly a p p lie d th e A p p ra isa l o f G u id e lin e s fo r Research and E v a lu a tio n (AGREE II) in s tru m e n t. M e th o d o lo g ic a l q u a lity w a s e va lu a te d b y ra n k in g in AGREE II d o m a in s and th e h ig h e s t n u m b e r o f ite m s s co rin g ab o ve th e n e u tra l th re s h o ld score.
Results: Four CPGs fu lfille d th e se le c tio n c rite ria , and th e h ig h e s t ran ked w e re th e E uropean and M a laysia n. H ig h e st scores ach ie ve d b y th e fo rm e r w e re fo r s c o p e /p u rp o s e , sta k e h o ld e r in v o lv e m e n t, and rig o r o f d e v e lo p m e n t an d by th e la tte r w e re fo r sco p e / p u rp o se , c la rity o f p re s e n ta tio n , an d a p p lic a b ility . A p p lic a b ility w a s th e lo w e s t s co rin g o f all d o m a in s fo r all CPGs.
Conclusion: E uro pe an an d M a la ysia n acne CPGs w e re ran ked h ig h e s t fo r m e th o d o lo g ic a l q u a lity an d m a y serve to in fo rm c lin ic a l p ra c tic e an d g u id e lin e a d a p ta tio n .
Contexte: L'acne v u lg a ire e st un e m a la d ie c h ro n iq u e fre q u e n te , e t les g u id e s de p ra tiq u e c lin iq u e (GPC) fo n d e s s u r des d o nn ee s p ro b a n te s p e u v e n t e tre un e s o u rce d 'in fo rm a tio n fia b le s u r le tra ite m e n t.
M ethode: N o u s avo n s e n tre p ris un e xa m en de la d o c u m e n ta tio n a la rech erch e de GPC su r l'acne, p u b lie s de ja n v ie r 2008 a se p te m b re 2013. Deux e x a m in a te u rs o n t pro ced e, cha cun de le u r cote , a re v a lu a tio n des g u id e s a I'a id e de I'in s tru m e n t AGREE II
(Appraisal o f G uidelines fo r Research and Evaluation ). La q u a lite m e th o d o lo g iq u e a ete evaluee en fo n c tio n des d o m a in e s AGREE II et du n o m b re le p lu s eleve d 'e le m e n ts a y a n t o b te n u un e n o te au-dessus du seu il de n e u tra lity .
Resultats: Q u a tre GPC re s p e c ta ie n t les crite re s de se le ctio n , e t les g u id e s e u rop een e t m a la isie n o n t o b te n u les re s u lta ts les p lu s sieves: le p re m ie r en ce q u i co n ce rn e le ch a m p e t les o b je c tifs , la p a rtic ip a tio n des g ro u p e s concernes, e t la rig u e u r d 'e la b o ra tio n ; le seco nd en ce q u i con cern e le ch a m p e t les o b je c tifs , la cla rte de la p re s e n ta tio n , e t I'a p p lic a b ilite . D 'a ille u rs, il e st a s o u lig n e r que I'a p p lic a b ilite est le d o m a in e q u i a o b te n u le score le p lu s fa ib le dans to u s les GPC.
Conclusion: Les GPC e u rop een e t m a la isie n su r l'a cn e o n t o b te n u les m e ille u re s co te s au reg ard de la q u a lite m e th o d o lo g iq u e , et its p e u v e n t s 'a ve re r des sources d 'in fo rm a tio n s u r la p ra tiq u e c lin iq u e e t I'a d a p ta tio n des lig n e s d ire ctrice s.
CNE VULGARIS ranks as one of the most burden some skin diseases globally as rated by disabilityadjusted life-years.1 It is a chronic inflammatory disease affecting 85% of people age 12 to 24 years2 and can persist into adulthood.3 Acne is associated with reduced quality of life and increased rates of anxiety, depression, and suicidal ideation.4 Estimated cost of acne treatment (direct and
A
From the Department o f Medicine, Schulich School o f Medicine and Dentistry, University o f Western Ontario, London, ON. Address reprint requests to: Jerry K.L. Tan, MD, FRCPC, Department of Medicine, Schulich School o f Medicine and Dentistry, University of Western Ontario, 2224 Walker Road, Suite 300, Windsor, O N N 8 W 5L7; email: [email protected]. D O I 1 0 .2 3 1 0/77 50 .2 01 4.131 90
€) 2014 Canadian Dermatology Association
indirect) and loss of productivity were estimated at US$3 billion annually a decade ago in the United States.5 Potential treatments for acne are myriad and include nonprescription products, prescription medications, device-based options, and complementary and alternative treatments. Additionally, the introduction of novel treat ment options over the past decade with varying levels of supportive evidence can further lead to provider and patient uncertainty, added cost, and inadequate outcomes. Quality health care is effective, efficient, accessible, equitable, safe, acceptable, and patient centered.6 Although various factors can lead to disparities in quality health care, one major issue has been the discrepancy between medical evidence and clinical practice. Key within the framework of knowledge translation/implementation is the development of high-quality evidence-based knowledge tools that are
DECKER^ jjfy Canadian Dermatology Association I Journal of Cutaneous Medicine and Surgery, Vol 18, No 6 (November/December), 2014: pp 385-391
385
Kawala et at
relevant, accessible, and comprehensible to stakeholders, including health care providers, patients, and their supporting personnel.' However, the availability of multi ple clinical practice guidelines (CPGs) of variable quality can lead to further uncertainty, confusion, and potential misapplication. Accordingly, an imperative step in evalua tion of CPGs is evaluation of their quality. In particular, this process can identify those worthy of adoption for clinical practice, adaptation for different regions/countries, or translation into patient decision aids to assist in informed shared decision making. The objective of the current study is to evaluate the quality of recent acne CPGs using a validated instrument, the Appraisal of Guidelines for Research and Evaluation (AGREE II) tool.8
Methods A systematic literature search was performed to identify recently published evidence-based acne CPGs. Four computerized databases (ie, Guidelines International Network, PubMed, Sumsearch, and Tripdatabase) were queried with the key words “acne” and “guidelines” and search dates delimited from January 2008 to September 11,
2013. There was no language restriction for inclusion. Excluded were duplications, position papers, editorials, non-evidence-based recommendations, systematic reviews, primary research, and publications focused on single treatments (Figure 1). Further methodological details were requested of CPG developers when information within the documents was considered insufficient or unclear. Selected CPGs were evaluated using the AGREE II instrument. AGREE II comprises 23 items in 6 domains: 3 for scope and purpose, 3 for stakeholder involvement, 8 for rigor of development, 3 for clarity of presentation, 4 for applicability, and 2 for editorial independence. Each item was rated on a scale of 1 to 7 (with 1 representing “strongly disagree” and 7 representing “strongly agree”), with higher scores indicating higher quality. Item scores > 4 were reflective of greater quality.8 Two reviewers (D.F., C.K.) trained in the use of AGREE II by means of an online tutorial9,10 independently evaluated each CPG. This was followed by deliberation of disparate scores to achieve consensus. Standardized domain scores were calculated by summing scores of the individual items in each domain and standardizing the total as a percentage of the maximum score for that domain, as per AGREE II scoring instructions. Overall
Figure 1. Flow diagram for literature selection of acne clinical practice guide lines (CPGs), 2008-2013. This figure illustrates the search strategy and iden tification of evidence-based acne CPGs.
\7
386
Canadian Dermatology Association I Journal of Cutaneous Medicine and Surgery, Vol 18, No 6 (November/December), 2014: pp 385-391
Quality Appraisal o f Acne Clinical Practice Guidelines
quality rating of each CPG was based on comparison across this cohort. Specific recommendations for CPG use were not proffered as the raters were unable to adequately assess this item.
Results Four acne CPGs fulfilled the selection criteria: Malaysian,11 United Kingdom (UK),12 American Acne and Rosacea Society (AARS),13 and European S3 (Euro)14 (Table 1). Of these, three addressed adolescent and adult populations, whereas the AARS guidelines solely addressed pediatric patients. These four CPGs uniformly excluded acne variants such as acne conglobata, acne fulminans, cosmetic acne, drug-induced acne, and chloracne. Furthermore, management of acne scars, postinflammatory hyperpig mentation, rosacea, and folliculitis were excluded. All guidelines included topical, systemic, hormonal, combina tion, and maintenance therapies. With the exception of the Euro CPG, they also included dietary factors and complementary and alternative therapies as management options (Table 2). The Malaysian, Euro, and UK guidelines used the original version of the AGREE instrument as a guide in their development process. These three trended to higher scores in scope and purpose, rigor of development, and clarity of presentation than the AARS guidelines (Table 3). The Euro and Malaysian CPGs each had three highest and two second highest domain scores within this cohort. For the remainder, the UK and AARS guidelines each had one highest score and three second highest scores. The Euro guideline ranked first for scope/purpose, stakeholder involvement, and rigor of development and second for applicability and editorial independence. The Malaysian guideline ranked first for scope/purpose, clarity of pre sentation, and applicability and second for rigor of
development and editorial independence. Another mea sure of CPG quality is the total number of item scores beyond neutral (score > 4). The greatest number was observed for the Euro CPG (19), whereas the remainder had lower numbers (AARS, 17; Malaysian, 16; UK, 16). The Euro, Malaysian, and UK guidelines each achieved full scores in scope and purpose by describing overall objectives, identifying specific health questions, and addres sing the target population. For stakeholder involvement, the Euro CPG received a full score as it addressed all items within the domain. Specifically, it explicitly listed members of the group contributing to each section. Additionally, patient preferences were included in devel opment by explicitly describing the process incorporating such preferences into recommendations. The Euro CPG included a thorough explanation of the intended audience and how the guideline may be used by the latter. The highest scoring CPG for rigor of development was the Euro guideline as information on search criteria was provided in adequate detail to facilitate replication. Furthermore, the Euro guideline solely described the formal consensus technique used in the formulation of recommendations, thus directly addressing methods for formulating recommendations described. The Malaysian CPG ranked first in clarity of presentation by clearly stating the recommended action for each patient group, along with reasons for treatment selection, other treat ment options available, and relevant clinical scenarios. Treatment options were easily identifiable and presented in a single-page flowchart. Applicability was the lowest scoring domain for all CPGs, with the Malaysian guideline achieving a high score of 50% but nevertheless scoring inadequately in monitoring and auditing and in advice/tools for recommendation implementation pro vided. In editorial independence, the AARS guideline scored highest as it explicitly included a statement that
Table 1. Criteria for Guideline Selection Criteria Inclusion (for selection, must be positive) Addressed acne vulgaris Guideline documents Evidence-based recommendations Exclusion (for selection, must be negative) Did not address acne vulgaris or addressed sequelae of acne (scarring, dyspigmentation) Duplications Not guidelines (primary research, position papers, editorials, single interventions) No evidence-based recommendations AARS — American Acne and Rosacea Society; Euro
Malaysian
UK
AARS
Euro
+ + +
+ + +
+ + +
+ + +
-
-
-
-
-
-
-
-
-
-
= European S3; UK = U nited Kingdom .
-----------------------------------------------------------------------------------------------------------------Canadian Dermatology Association I Journal of Cutaneous Medicine and Surgery, Vol 18, No 6 (November/December), 2014: pp 385-391
387
Kawala et al
Table 2. Evidence-Based Acne Clinical Practice Guidelines, 2008-2013 Group
Year
Malaysian M inistry of H ealth11 (Malaysian)
2012
Title “M anagem ent of Acne”
Scope Topical therapy Systemic therapy H orm onal therapy M aintenance therapy Com bination therapy Dietary factors Corticosteroid injection Physical therapy
Sowerby Centre for Health
2009
“Acne Vulgaris (Prodigy)”
Informatics at Newcastle12 (UK)
Complementary/alternative medicines Topical therapy Systemic therapy H orm onal therapy C om bination therapy Dietary factors M aintenance therapy Physical therapy
European evidence-based (S3)
2012
“European Evidence-Based (S3) Guidelines for the Treatm ent of Acne”
guidelines14 (Euro)
Complementary/alternative medicines Topical therapy Systemic therapy Physical therapy Com bination therapy H orm onal therapy Maintenance therapy
American Acne and Rosacea Society13 (AARS)
2013
“Evidence-Based Recommendations for the Diagnosis and Treatm ent o f Pediatric Acne”
Topical therapy Systemic therapy Com bination therapy H orm onal therapy Dietary factors O ver-the-counter medications
the funding body did not influence the content of the guideline.
Discussion CPGs are the end-product of translational research and play a critical role in the practical application of evidence-based medicine. The quality of CPGs can vary greatly; thus, indicators to evaluate methodological quality of guidelines are required. This process can facilitate discernment guide lines of higher methodological quality and therefore greater credibility. The AGREE instrument is a tool that has been developed to evaluate the methodological quality of CPGs. The AGREE Next Steps Consortium (2009) further refined the original instrument to improve its measurement ability, ease of use for reviewers, and value for target populations by developing the AGREE II instrument.15 Three of the four CPGs (Malaysian, UK, and Euro) made specific reference to the original AGREE tool in
development and, consequently, had similar scores in many of the domains. It is noteworthy that the AARS guideline scored similarly to this group despite not citing AGREE in its development. All CPGs had strong domain scores for scope and purpose, with the three AGREE-guided CPGs (Euro, Malaysian, and UK) each scoring 100%. The UK domain score for stakeholder involvement was significantly weaker than the other CPGs due to a lack of target population involvement. Target population inclusion in the develop ment process is important as CPGs are typically produced with health care providers as the target audience. However, a focus on the latter may inadequately address patient concerns, preferences, and values to assist in the informed treatment decision-making process.16,12 Euro, Malaysian, and UK domain scores for rigor of development were strong in contrast to the overall neutral domain score for the AARS guideline. This relatively weaker score is largely due to inadequate methodological detail in the development
Canadian Dermatology Association I Journal o f Cutaneous Medicine and Surgery, Vol IS, No 6 (November/December), 2014: pp 385-391
Quality Appraisal of Acne Clinical Practice Guidelines
Table 3. AGREE II Scores of Acne Clinical Practice Guidelines, 2008-2013 Domain Domain 1: scope and purpose Overall objectives described Health questions described Patients described Domain score (%) Domain 2: stakeholder involvement Relevant professional groups included Patient views sought Target users defined Domain score (%) Domain 3: rigor of development Systematic literature search Selection criteria described Strengths and limits of evidence described Methods for formulating recommendations described Benefits, side effects, and risks considered in recommendations Link between recommendations and evidence External review before publication Procedure for updating guideline provided Domain score (%) Domain 4: clarity of presentation Recommendations are specific and unambiguous Different management options presented Recommendations are easily identifiable Domain score (%) Domain 5: applicability Facilitators and barriers to application described Advice/tools for recommendation implementation provided Resource implications considered Monitoring and auditing criteria provided Domain score (%) Domain 6: editorial independence Views of funding body have not influenced guideline content Competing interests recorded and addressed Domain score (%)
Euro
Malaysian
UK
AARS
7 7 7 100
7 7 7 100
7 7 7 100
7 5 7 89
7 7 7 100
6 2 7 67
2 2 6 39
6 6 5 78
7 7 7 7 6 7 5 7 94
7 6 6 3 (4)* 7 7 3 5 77
7 5 6 1 7 7 5 7 77
1 6 6 3 7 5 2 1 48
6 6 7 89
7 7 7 100
7 7 6 94
6 7 7 94
4 2 4 2 33
4 5 4 3 50
3 2 6 1 33
1 5 5 1 33
6 5 75
4 7 75
3 5 50
7 5 83
AGREE = Appraisal of Guidelines for Research and Evaluation; AARS = American Acne and Rosacea Society; Euro = European S3; UK = United Kingdom. *Score modified after receiving additional information from guideline author(s).
process and in describing the means by which specific recommendations were developed. All CPGs scored well in clarity of presentation. All CPGs had low scores for applicability, and this may reflect the downstream aspect of this domain, which can be addressed only after completion of the guidelines. 18 This consistent shortfall is apparent in CPGs in medicine and is not unique to acne—and has led to the development of a specific working group within guideline development networks to address this issue. One such initiative, the
Guidelines Implementability Research and Application Network (GIRAnet), is endeavoring to evaluate and apply tools for implementation to assist future guideline developments. 19 The low scores for the UK, Euro, and Malaysian guidelines in this domain were unexpected as they cited guidance by the original AGREE tool in their development process. Addressing applicability for a nascent guideline to encourage implementation should include a template of future initiatives to enhance recognition, uptake, and use by relevant stakeholders.
C anadian D erm atology A ssociation I Journal o f Cutaneous Medicine and Surgery, Vol 18, No 6 (November/December), 2014: pp 385-391
389
Kawala et al
Measuring the effect of these efforts in enhancing outcomes for specific target populations and in overall health use should also be considered. Domain scores for editorial independence were strong for all except the UK guideline, which had a neutral score. This weaker score was due to shortcomings in reporting funding body influence on guideline content. Considerable cost is associated with the development of CPGs, and in many countries, funding from governmental agencies is unavailable. In these circumstances, their development necessitates funding from nongovernmental and/or indus try sources. However, funding from the latter may be perceived as biasing CPG content. However, governmental funding for CPGs must also be considered within the potential agenda of reducing health care expenditures. In either case, the explicit exclusion of funding bodies from the guideline development process is imperative to minimize the risk of bias. One means to address these issues is to segregate the expert panel of the CPG from those soliciting funding support. Furthermore, the expert panel should not be made aware of the identity of the funding sources until after the CPG is ready for submission. Furthermore, to minimize the risk of bias, funding solicitors should not be involved in treatment recommendations or the voting process to generate consensus. These practical measures may facilitate the development of high-quality CPGs with adequate re sources while minimizing the risk of bias. We recognize the following limitations in this study. Subjectivity is inherent in scoring with AGREE II as differences can result based on reviewer interpretation and valuation of components addressing particular items. Nevertheless, the use of multiple raters can reduce scoring variation to facilitate the tool’s ability to discern highfrom low-quality guidelines.20 Evaluation of CPGs with the AGREE II tool is also limited by the wording of item criteria. There were several items where the fulfillment of a particular criterion required an explicit statement, provid ing no flexibility for interpretation or inference. Therefore, it was possible that low scores represented failure to present necessary information rather than shortcomings in methodological quality. To ensure that scoring was accurate, we queried guideline developers about these items as well as those scoring below the neutral threshold. The AGREE II instrument does not allow one to assess the content or the quality of recommendations found within a CPG.8 A recent publication evaluated acne CPGs with the AGREE II instrument over a 10-year period (2002-2012) and, similar to the present study, identified the Euro and Malaysian CPGs as the most highly rated.21 We sought a 390
more circumscribed timeframe of 5 years and extended the timeframe to include the current year (2013) to enhance the timeliness of treatment recommendations for adapta tion and transformation to different applications.
Conclusion Acne is a highly burdensome global condition for which evidence-based treatment paradigms from high-quality CPGs are available and may be positively impactful. We identified high-quality acne CPGs that may serve to inform daily clinical care and provide a template on which to develop additional tools for health care providers, institu tional stakeholders, patients, and their support groups. Improvements in applicability would facilitate the transla tion of these efforts to enhance patient outcomes.
Acknowledgment Financial disclosure of authors: Jerry K.L. Tan has served as an advisor, speaker, and/or trialist for Allergan, Bayer, Cipher, Dermira, Galderma, Roche, Stiefel/GSK, and Valeant. Christopher Kawala and Dilan Fernando have no conflicts to disclose.
References 1. Institute for Health Metrics and Evaluation. The global burden of disease: generating evidence, guiding policy. Seattle (WA): Institute for Health Metrics and Evaluation; 2013. Available at: http:// www.healthmetricsandevaluation.org/sites/default/ffles/policy_report/ 201 l/GBD_Generating%20Evidence_Guiding%20Policy%20FINAL. pdf (accessed May 30, 2013). 2. White GM. Recent findings in the epidemiologic evidence, classification, and subtypes of acne vulgaris. J Am Acad Dermatol 1998;39(2 Pt 3):S34-7, doi:10.1016/S0190-9622(98)70442-6. 3. Schafer T, Nienhaus A, Vieluf D, et al. Epidemiology of acne in the general population: the risk of smoking. Br J Dermatol 2001;145: 100^, doi:10.1046/j. 1365-2133.2001.04290.x. 4. Dunn LK, O’Neill JL, Feldman SR. Acne in adolescents: quality of life, self-esteem, mood, and psychological disorders. Dermatol Online J 2011;17( 1). Available at: http://www.ncbi.nlm.nih.gov/ pubmed/21272492 (accessed March 8, 2013). 5. Bickers DR, Lim HW, Margolis D, et al. The burden of skin diseases: 2004 a joint project of the American Academy of Dermatology Association and the Society for Investigative Dermatology. J Am Acad Dermatol 2006;55:490-500, doi:10.1016/ j.jaad.2006.05.048. 6. Bengoa R, Kawar R, Key P, et al. Quality of care: a process for making strategic choices in health systems. Geneva: World Health Organization; 2006. 7. Davis D, Goldman J, Palda VA. Handbook on clinical practice guidelines. Ottawa: Canadian Medical Association; 2007.
Canadian Dermatology Association I Journal o f Cutaneous Medicine and Surgery, Vol 18, No 6 (November/December), 2014: pp 385-391
Quality Appraisal of Acne Clinical Practice Guidelines
8. AGREE Next Steps Consortium. The AGREE II instrument. Available at: http://www.agreetrust.org 2009 (accessed January 28, 2013). 9. AGREE Next Steps Consortium. The AGREE II instrument overview tutorial. Available at: http://agree.machealth.ca/players/ open/index.html. 2009 (accessed January 28, 2013). 10. AGREE Next Steps Consortium. The AGREE II instrument tutorial and practice exercise. Available at: http://agree.machealth.ca/ openinstrumentfeedback.aspx?id=918e38cl-a84d-45aa-8343-145c 06eea243. 2009 (accessed January 28, 2013). 11. CPG Secretariat, Health Technology Assessment Section, Medical Development Division, Ministry of Health, Malaysia. Clinical practice guidelines. Management of acne. January 2012. Available at: http://www.moh.gov.my (accessed November 2012). 12. Sowerby Centre for Health Informatics at Newcastle. Acne vulgaris (prodigy). Available at: http://prodigy.clarity.co.uk/acne_vulgaris (accessed February 13, 2013). 13. Eichenfield LF, Krakowski AC, Piggott C, et al. Evidence-based recommendations for the diagnosis and treatment of pediatric acne. Pediatrics 2013;131 Suppl 3:S163-86, doi:10,1542/peds.20130490B. 14. Nast A, Dreno B, Bettoli V, et al. European evidence-based (S3) guidelines for the treatment of acne. J Eur Acad Dermatol Venereol 2012;26 Suppl 1:1-29, doi:10.1111/j.l468-3083.2011.04374.x.
15. Brouwers M, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in healthcare. CMAJ 2010;182:E839M2, doi:10.1503/cmai.090449. 16. MacLean S, Mulla S, Akl EA, et al. Patient values and preferences in decision making for antithrombotic therapy: a systematic review: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest 2012;141 Suppl 2: elS-23S. 17. Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 1: performance, usefulness and areas for impro vement. CMAJ 2010;182:1045-52, doi:10.1503/cmaj.091714. 18. Field MJ, Lohr KN. Clinical practice guidelines: directions for a new program. Washington (DC): National Academy Press; 1990. 19. Guidelines International Network. Introducing GIRAnet. Available at: http://www.g-i-n.net/activities/implementation/giranet (accessed July 9, 2013). 20. Vlayen J, Aertgeerts B, Hannes K, et al. A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit. Int J Qual Health Care 2005;17:235—42, doi :10.1093/intqhc/mzi027. 21. Sanclemente G, Acosta J-L, Tamayo M-E, et al. Clinical practice guidelines for treatment of acne vulgaris: a critical appraisal using the AGREE II instrument. Arch Dermatol Res 2014;306:269-77, doi:10.1007/s00403-013-1394-x.
m C anadian D erm atology A ssociation I Journal o f Cutaneous Medicine and Surgery, Vol 18, No 6 (November/December), 2014: pp 385-391
391
Copyright of Journal of Cutaneous Medicine & Surgery is the property of Decker Publishing and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
