ORIGINAL ARTICLES
An Overview of Systematic Review Kathy A. Baker, PhD, RN, ACNS-BC, FAAN, Susan Mace Weeks, DNP, RN, CNS, LMFT, FAAN Systematic review is an invaluable tool for the practicing clinician. A well-designed systematic review represents the latest and most complete information available on a particular topic or intervention. This article highlights the key elements of systematic review, what it is and is not, and provides an overview of several reputable organizations supporting the methodological development and conduct of systematic review. Important aspects for evaluating the quality of a systematic review are also included. Keyword: systematic review. Ó 2014 by American Society of PeriAnesthesia Nurses IN THE RAPIDLY CHANGING CONTEXT of health care delivery, busy clinicians are dependent on easy access to the best available evidence. As knowledge has been established through focused research efforts, the need to synthesize this growing body of knowledge has resulted in a methodology known as systematic review. A systematic review is the basis for evidence-based practice.1 Systematic review involves analysis of the published evidence identified from an exhaustive review around a focused question. The goal of a systematic review is to synthesize the latest research findings into a single document that represents the most up-to-date and complete representation of the body of knowledge on a particular topic. Systematic reviews help clinicians
Kathy A. Baker, PhD, RN, ACNS-BC, FAAN, is an Associate Professor, Harris College of Nursing & Health Sciences, and Director, Division of Nursing Graduate Studies & Scholarship and Deputy Director at The Center for Evidence-Based Practice & Research: A Collaborating Center of the Joanna Briggs Institute, Texas Christian University, Fort Worth, TX; and Susan Mace Weeks, DNP, RN, CNS, LMFT, FAAN, is a Professor and Associate Dean, Harris College of Nursing & Health Sciences, and Director at The Center for Evidence-Based Practice & Research: A Collaborating Center of the Joanna Briggs Institute, Texas Christian University, Fort Worth, TX. Conflict of interest: None to report. Address correspondence to Kathy A. Baker, PhD, RN, ACNSBC, FAAN, Texas Christian University, TCU Box 298627, Fort Worth, TX, 76129; e-mail address: [email protected]. Ó 2014 by American Society of PeriAnesthesia Nurses 1089-9472/$36.00 http://dx.doi.org/10.1016/j.jopan.2014.07.002
454
identify what is known and not known on a topic, particularly an intervention; identify and understand inconsistencies among research findings; and help delineate whether findings can be applied to specific subgroups of patients.2 The clinician can be confident that the systematic review, if published using the systematic review protocol guidelines from a reputable organization, represents the latest and most complete information available on the particular topic of interest.
What Is Systematic Review A systematic review is a scientific inquiry that relies on a focused question and predetermined protocol to identify, critique, and ultimately summarize different primary research studies. As opposed to relying on the results of a single research study, a systematic review provides an analysis of the results of multiple research studies to formulate the current state of knowledge regarding the focus of study. The results can vary from authoritative statements of practice advisories to recommendations for further research. The process of a systematic review is just as it sounds: systematic. Readers of systematic reviews benefit from an understanding of the elements of a high-quality systematic review. This knowledge allows the reader to make an independent judgment of the potential value, validity, and reliability of each systematic review. The ‘‘systematic’’ component of a systematic review is the defining feature of the review itself and results in the systematic
Journal of PeriAnesthesia Nursing, Vol 29, No 6 (December), 2014: pp 454-458
SYSTEMATIC REVIEW
review being considered the highest level of evidence available to guide and inform practice. As a scientific overview of primary research on a particular research question, a systematic review tries to identify, select, synthesize, and appraise all high-quality research evidence relevant to a well-honed question. Systematic reviews seek to collate all evidence that fits prespecified eligibility criteria to address a specific research question. The eligibility criteria is composed of both inclusion and exclusion criteria that delineate what types of studies will and will not be included in the systematic review. ‘‘Systematic reviews aim to minimize bias by using explicit, transparent, and systematic methods’’ (http://www.cochrane.org/ about-us/evidence-based-health-care/).
455
the studies identified in the first two studies are then searched. The list of databases to be searched should allow access to both published peerreviewed literature and unpublished literature that is sometimes referred to as ‘‘grey literature.’’ For data extraction and analysis, a systematic critical appraisal process unique to each type of study design is conducted by two appraisers working independently. Finally, the validation of study selection for analysis should occur by at least two independent reviewers. Throughout the process, two appraisers work independently through each step and then engaging in dialogue until they come to agreement.3 Documentation of the entire process should be maintained.
Systematic Review Organizations It is important to point out that historically, systematic reviews have been focused solely on randomized controlled trials (trials of cause and effect). This has changed, however, in recent years to include systematic reviews of other quantitative designs, as well as qualitative and economic study designs. As is well known in the discipline of nursing, many questions of great importance cannot be answered through randomized controlled trials (examples include quality of life studies, death and end-of life decisions, and experiential studies). Through innovative and novel research design methods, credible organizations have developed ways to synthesize qualitative studies that have similar methodological qualities into concise themes. These consolidated themes, in turn, guide practice and future research.
What Is Not Systematic Review Even in highly regarded peer-reviewed journals, authors will title their work a systematic review when it is actually another type of review (ie, critical, integrated, or literature). A systematic review should report explicit, rigorous, and reproducible methods to minimize bias. These methods include a well-defined research statement/question; specific inclusion and exclusion criteria for studies to be included in the analysis; and exhaustive comprehensive searching using three steps: (1) predefined databases are searched using specified terms in the keywords, title, and abstract fields, (2) keywords and index terms from the literature obtained in the first step are searched across all appropriate databases, (3) reference lists from
There are several international organizations that devote their expertise to developing the methodologies behind systematic review so that the synthesis of the best evidence is valid and reliable. These include the Campbell Collaboration, Centre for Reviews and Dissemination, Cochrane Collaboration, Institute of Medicine, and Joanna Briggs Institute. These organizations provide guidance, training, and software to support the synthesis of varied types of evidence including quantitative, qualitative, and economic evidences. Although independent and distinct, the organizations work collaboratively to promote synthesis methodology and the importance of evidence synthesized in the form of a systematic review. Each of these specialized organizations have devoted enormous resources and expertise to furthering the rigor and effectiveness of systematic review for changing health care and social practices based on best evidence. Campbell Collaboration The Campbell Collaboration (C2) (http://www. campbellcollaboration.org/artman2/uploads/1/ Campbell_mal_Campbell_Collaboration_Improving_ the_evidence_base_for_social_policy_and_practice. pdf) based in Oslo, Norway, focuses on systematic reviews in education, criminal justice, social policy, and programs synthesizing the effect of social intervention. Their goal is to help people make wellinformed decisions by preparing, maintaining, and disseminating systematic reviews.
456
Centre for Reviews and Dissemination The Centre for Reviews and Dissemination (http:// www.york.ac.uk/inst/crd/about_us.htm) is part of the National Institute for Health Research and a department of the University of York, United Kingdom. The group generates research-based information about the effects of health and social care interventions and undertakes systematic reviews evaluating the research evidence on health and public health questions of national and international importance. Cochrane Collaboration The Cochrane Collaboration (http://www.cochrane. org/about-us), also based in the United Kingdom, is an international network of more than 28,000 people from more than 120 countries. The organization’s mission is to ‘‘promote evidence-informed health decision making by producing high-quality, relevant, accessible systematic reviews, and other synthesized research evidence’’ (http://www.cochrane.org/ about-us). This is accomplished through reviews published online in the Cochrane Database of Systematic Reviews, part of The Cochrane Library. The organization boasts the largest collection of records of randomized controlled trials in the world, called CENTRAL, published as part of The Cochrane Library. The collaboration’s work focuses primarily on synthesis of medical trials. Institute of Medicine Framework The Institute of Medicine4 at the request of the United States Congress developed four standards to guide the systematic review process. These include standards for (1) initiating a systematic review, (2) finding and assessing individual studies, (3) synthesizing the body of evidence, and (4) reporting systematic reviews. The standards are discussed in detail in the book Finding What Works in Healthcare: Standard for Systematic Review available online free of charge at https://download. nap.edu/login.php?record_id513059&page5% 2Fdownload.php%3Frecord_id%3D13059. Joanna Briggs Institute The Joanna Briggs Institute (JBI; http://www.joannabriggs.edu.au/) is to nursing and allied health what the Cochrane Collaboration is to medicine.
BAKER AND WEEKS
The JBI, housed in Adelaide, South Australia, is an international not-for-profit, membership-based, research, and development organization with more than 70 entities across the world (http:// joannabriggs.org/about.html). The group promotes and supports the synthesis, transfer, and utilization of evidence (systematic review) through identifying feasible, appropriate, meaningful, and effective health care practices to assist in the improvement of health care outcomes globally. Their mission is to provide evidence to inform clinical decision making at the point of care. Although the roots of the JBI are in nursing, their work has expanded to include all health disciplines including health policy and health journalists. Systematic reviews conducted through the JBI are based on the JBI Model of Evidence-Based Healthcare.5
Critical Appraisal of Systematic Reviews Critical appraisal of any source of evidence is an important step in scientific inquiry. Regardless of the focus of the appraisal, primary research study, or systematic review, the investigator must understand the elements that compose a quality source of evidence and evaluate the source being considered against those quality standards. Another critical aspect of systematic review is transparency of the review process. ‘‘A properly conducted systematic review faithfully summarizes the evidence from all relevant studies on the topic of interest, and it does so concisely and transparently (p. 377).’’2 As with all reputable science, replication should be possible. In systematic review, the researchers will document every step of the process so that their work can be replicated and validated.6 Transparency also contributes to the reader’s ability to validate that rigor was upheld in the systematic review process. A highquality systematic review will be explicit and transparent about the methods used to conduct the review. This transparency is important to allow critical appraisal of the actual systematic review. Conclusions drawn from systematic reviews that are vague about their methods or that seem to skip reporting of specific steps should be viewed with suspicion. Standards for initiating a systematic review provide guidance in establishing a team with
SYSTEMATIC REVIEW
appropriate expertise. It is acknowledged that a single investigator is not sufficient to produce a high-quality systematic review. A minimum of two reviewers is needed to minimize bias and promote objectivity. Potential conflicts of interest should be identified and disclosed by the reviewers. Although the review team makes final decisions about what is and is not included in a systematic review, stakeholder input should be solicited as needed. The need for a systematic review, as well as the process that will be used to conduct the review, should be clearly articulated before the actual review begins. A solid systematic review is guided by a defined protocol developed before the start of the review.6 The protocol should include a justification for the review, a literature search strategy, criteria for including or excluding individual studies, methods of study appraisal, outcome measures to be evaluated, methods to be used to extract data from studies, and processes to be used when reviewers disagree. A well-designed systematic review protocol should be subjected to peer review and made available for public comment to allow refinement and improvement of the planned systematic review. A systematic review should include collaboration with a health sciences librarian who is able to help refine the search strategy and ensure a comprehensive search for relevant evidence. Exhaustive search strategies may include contacting study authors to clarify study information, hand searching of journals and conference abstracts, and searching for grey literature and unpublished studies. Once studies have been retrieved, the reviewers should independently review the retrieved studies for study inclusion or exclusion using the predefined criteria from the systematic review protocol. This blinded work increases the likelihood of including studies of high quality and relevance while excluding studies of lower quality. This independent blinded approach should be repeated during critical appraisal of selected studies and the data extraction phase. The methods that will be used to synthesize the evidence being evaluated should be described in the systematic review protocol.6 These methods should be explicit and should recognize the type of research methodology being included in the system-
457
atic review. Specifically, evidence from qualitative studies should be evaluated and pooled in a manner that is distinct from evidence drawn from quantitative studies. Although quantitative studies may allow the pooling of data through a unique quantitative analysis known as a meta-analysis, evidence drawn from qualitative studies may pooled in a unique qualitative analysis known as meta-synthesis. A comprehensive systematic review may include evidence from a variety of study methodologies including quantitative, qualitative, and economic evidences. An emerging area of synthesis is the pooling of multiple systematic reviews, known as an umbrella review. An umbrella review is essentially a systematic review of systematic reviews.7 The final stage of systematic review production is the preparation of the final report. A key element of this stage is external peer review by an independent party to ensure the scientific quality of the systematic review. A well-written systematic review report will include the study rationale, reporting of systematic review methods, results section, and discussion section. The external peer review process used to evaluate both the protocol and the final report should be explicitly reported. The final systematic review should be available in a manner that ensures access by clinicians who will use the review to guide their practice. This may be accomplished by publication in libraries of the organizations known for systematic review production (eg, the Joanna Briggs Institute Library of Systematic Reviews, the Cochrane Library, or the Campbell Library) and/or other professional peer-reviewed publications. One excellent resource designed for authors and journal reviewers, but an excellent tool for critiquing systematic reviews as a research consumer is the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist (http:// www.prisma-statement.org/). The checklist provides a comprehensive overview of key components of a quality systematic review.
Summary Direct care providers rely on evidence to guide and inform practice. Often, the evidence on which practice is based is outdated, inadequately synthesized, misinformed, and unsuccessfully disseminated. This poorly constructed synthesis of
BAKER AND WEEKS
458
evidence can ultimately lead to misinformed health care providers and a poor quality of care for our patients. Although clinical reasoning can never be replaced, quality systematic reviews offer direct care providers timely evidence to assist with clinical decision making. Systematic reviews are an ‘‘improved reflection on reality’’ that offer efficient integration
of information, provide data for rational decision making, establish the generalizability of a body of knowledge, assess consistency of relationships, and increase the power of aggregated studies.8 Clinicians dedicated to application of best evidence in their practice will benefit from the knowledge of appraising and using systematic reviews.
References 1. Holly C, Salmond C, Saimbert MK. Comprehensive Systematic Review for Advanced Nursing Practice. New York, NY: Springer; 2012. 2. Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: Synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126:376-380. 3. Joanna Briggs Institute (JBI). Joanna Briggs Institute Comprehensive Review Training Program: The Systematic Review of Evidence Generated through Quantitative Review. Adelaide, South Australia: Joanna Briggs Institute; 2013:9. 4. Institute of Medicine (IOM). Finding What Works in Health Care: Standards for Systematic Re-
views. Washington, DC: The National Academies Press; 2011. 5. Pearson A, Weeks S, Stern C. Translation Science and the JBI Model of Evidence-based Healthcare. Philadelphia, PA: Lippincott; 2011. 6. Evans D. Systematic reviews of nursing research. Intensive Crit Care Nurs. 2001;17:51-57. 7. Joanna Briggs Institute (JBI). Joanna Briggs Institute Reviewers’ Manual: 2011 Edition. Adelaide, South Australia: Joanna Briggs Institute; 2011. 8. Mulrow CD. Systematic reviews: Rationale for systematic reviews. BMJ. 1994;309:597-599.
An Overview of Systematic Review Kathy A. Baker, PhD, RN, ACNS-BC, FAAN, Susan Mace Weeks, DNP, RN, CNS, LMFT, FAAN Systematic review is an invaluable tool for the practicing clinician. A well-designed systematic review represents the latest and most complete information available on a particular topic or intervention. This article highlights the key elements of systematic review, what it is and is not, and provides an overview of several reputable organizations supporting the methodological development and conduct of systematic review. Important aspects for evaluating the quality of a systematic review are also included. Keyword: systematic review. Ó 2014 by American Society of PeriAnesthesia Nurses IN THE RAPIDLY CHANGING CONTEXT of health care delivery, busy clinicians are dependent on easy access to the best available evidence. As knowledge has been established through focused research efforts, the need to synthesize this growing body of knowledge has resulted in a methodology known as systematic review. A systematic review is the basis for evidence-based practice.1 Systematic review involves analysis of the published evidence identified from an exhaustive review around a focused question. The goal of a systematic review is to synthesize the latest research findings into a single document that represents the most up-to-date and complete representation of the body of knowledge on a particular topic. Systematic reviews help clinicians
Kathy A. Baker, PhD, RN, ACNS-BC, FAAN, is an Associate Professor, Harris College of Nursing & Health Sciences, and Director, Division of Nursing Graduate Studies & Scholarship and Deputy Director at The Center for Evidence-Based Practice & Research: A Collaborating Center of the Joanna Briggs Institute, Texas Christian University, Fort Worth, TX; and Susan Mace Weeks, DNP, RN, CNS, LMFT, FAAN, is a Professor and Associate Dean, Harris College of Nursing & Health Sciences, and Director at The Center for Evidence-Based Practice & Research: A Collaborating Center of the Joanna Briggs Institute, Texas Christian University, Fort Worth, TX. Conflict of interest: None to report. Address correspondence to Kathy A. Baker, PhD, RN, ACNSBC, FAAN, Texas Christian University, TCU Box 298627, Fort Worth, TX, 76129; e-mail address: [email protected]. Ó 2014 by American Society of PeriAnesthesia Nurses 1089-9472/$36.00 http://dx.doi.org/10.1016/j.jopan.2014.07.002
454
identify what is known and not known on a topic, particularly an intervention; identify and understand inconsistencies among research findings; and help delineate whether findings can be applied to specific subgroups of patients.2 The clinician can be confident that the systematic review, if published using the systematic review protocol guidelines from a reputable organization, represents the latest and most complete information available on the particular topic of interest.
What Is Systematic Review A systematic review is a scientific inquiry that relies on a focused question and predetermined protocol to identify, critique, and ultimately summarize different primary research studies. As opposed to relying on the results of a single research study, a systematic review provides an analysis of the results of multiple research studies to formulate the current state of knowledge regarding the focus of study. The results can vary from authoritative statements of practice advisories to recommendations for further research. The process of a systematic review is just as it sounds: systematic. Readers of systematic reviews benefit from an understanding of the elements of a high-quality systematic review. This knowledge allows the reader to make an independent judgment of the potential value, validity, and reliability of each systematic review. The ‘‘systematic’’ component of a systematic review is the defining feature of the review itself and results in the systematic
Journal of PeriAnesthesia Nursing, Vol 29, No 6 (December), 2014: pp 454-458
SYSTEMATIC REVIEW
review being considered the highest level of evidence available to guide and inform practice. As a scientific overview of primary research on a particular research question, a systematic review tries to identify, select, synthesize, and appraise all high-quality research evidence relevant to a well-honed question. Systematic reviews seek to collate all evidence that fits prespecified eligibility criteria to address a specific research question. The eligibility criteria is composed of both inclusion and exclusion criteria that delineate what types of studies will and will not be included in the systematic review. ‘‘Systematic reviews aim to minimize bias by using explicit, transparent, and systematic methods’’ (http://www.cochrane.org/ about-us/evidence-based-health-care/).
455
the studies identified in the first two studies are then searched. The list of databases to be searched should allow access to both published peerreviewed literature and unpublished literature that is sometimes referred to as ‘‘grey literature.’’ For data extraction and analysis, a systematic critical appraisal process unique to each type of study design is conducted by two appraisers working independently. Finally, the validation of study selection for analysis should occur by at least two independent reviewers. Throughout the process, two appraisers work independently through each step and then engaging in dialogue until they come to agreement.3 Documentation of the entire process should be maintained.
Systematic Review Organizations It is important to point out that historically, systematic reviews have been focused solely on randomized controlled trials (trials of cause and effect). This has changed, however, in recent years to include systematic reviews of other quantitative designs, as well as qualitative and economic study designs. As is well known in the discipline of nursing, many questions of great importance cannot be answered through randomized controlled trials (examples include quality of life studies, death and end-of life decisions, and experiential studies). Through innovative and novel research design methods, credible organizations have developed ways to synthesize qualitative studies that have similar methodological qualities into concise themes. These consolidated themes, in turn, guide practice and future research.
What Is Not Systematic Review Even in highly regarded peer-reviewed journals, authors will title their work a systematic review when it is actually another type of review (ie, critical, integrated, or literature). A systematic review should report explicit, rigorous, and reproducible methods to minimize bias. These methods include a well-defined research statement/question; specific inclusion and exclusion criteria for studies to be included in the analysis; and exhaustive comprehensive searching using three steps: (1) predefined databases are searched using specified terms in the keywords, title, and abstract fields, (2) keywords and index terms from the literature obtained in the first step are searched across all appropriate databases, (3) reference lists from
There are several international organizations that devote their expertise to developing the methodologies behind systematic review so that the synthesis of the best evidence is valid and reliable. These include the Campbell Collaboration, Centre for Reviews and Dissemination, Cochrane Collaboration, Institute of Medicine, and Joanna Briggs Institute. These organizations provide guidance, training, and software to support the synthesis of varied types of evidence including quantitative, qualitative, and economic evidences. Although independent and distinct, the organizations work collaboratively to promote synthesis methodology and the importance of evidence synthesized in the form of a systematic review. Each of these specialized organizations have devoted enormous resources and expertise to furthering the rigor and effectiveness of systematic review for changing health care and social practices based on best evidence. Campbell Collaboration The Campbell Collaboration (C2) (http://www. campbellcollaboration.org/artman2/uploads/1/ Campbell_mal_Campbell_Collaboration_Improving_ the_evidence_base_for_social_policy_and_practice. pdf) based in Oslo, Norway, focuses on systematic reviews in education, criminal justice, social policy, and programs synthesizing the effect of social intervention. Their goal is to help people make wellinformed decisions by preparing, maintaining, and disseminating systematic reviews.
456
Centre for Reviews and Dissemination The Centre for Reviews and Dissemination (http:// www.york.ac.uk/inst/crd/about_us.htm) is part of the National Institute for Health Research and a department of the University of York, United Kingdom. The group generates research-based information about the effects of health and social care interventions and undertakes systematic reviews evaluating the research evidence on health and public health questions of national and international importance. Cochrane Collaboration The Cochrane Collaboration (http://www.cochrane. org/about-us), also based in the United Kingdom, is an international network of more than 28,000 people from more than 120 countries. The organization’s mission is to ‘‘promote evidence-informed health decision making by producing high-quality, relevant, accessible systematic reviews, and other synthesized research evidence’’ (http://www.cochrane.org/ about-us). This is accomplished through reviews published online in the Cochrane Database of Systematic Reviews, part of The Cochrane Library. The organization boasts the largest collection of records of randomized controlled trials in the world, called CENTRAL, published as part of The Cochrane Library. The collaboration’s work focuses primarily on synthesis of medical trials. Institute of Medicine Framework The Institute of Medicine4 at the request of the United States Congress developed four standards to guide the systematic review process. These include standards for (1) initiating a systematic review, (2) finding and assessing individual studies, (3) synthesizing the body of evidence, and (4) reporting systematic reviews. The standards are discussed in detail in the book Finding What Works in Healthcare: Standard for Systematic Review available online free of charge at https://download. nap.edu/login.php?record_id513059&page5% 2Fdownload.php%3Frecord_id%3D13059. Joanna Briggs Institute The Joanna Briggs Institute (JBI; http://www.joannabriggs.edu.au/) is to nursing and allied health what the Cochrane Collaboration is to medicine.
BAKER AND WEEKS
The JBI, housed in Adelaide, South Australia, is an international not-for-profit, membership-based, research, and development organization with more than 70 entities across the world (http:// joannabriggs.org/about.html). The group promotes and supports the synthesis, transfer, and utilization of evidence (systematic review) through identifying feasible, appropriate, meaningful, and effective health care practices to assist in the improvement of health care outcomes globally. Their mission is to provide evidence to inform clinical decision making at the point of care. Although the roots of the JBI are in nursing, their work has expanded to include all health disciplines including health policy and health journalists. Systematic reviews conducted through the JBI are based on the JBI Model of Evidence-Based Healthcare.5
Critical Appraisal of Systematic Reviews Critical appraisal of any source of evidence is an important step in scientific inquiry. Regardless of the focus of the appraisal, primary research study, or systematic review, the investigator must understand the elements that compose a quality source of evidence and evaluate the source being considered against those quality standards. Another critical aspect of systematic review is transparency of the review process. ‘‘A properly conducted systematic review faithfully summarizes the evidence from all relevant studies on the topic of interest, and it does so concisely and transparently (p. 377).’’2 As with all reputable science, replication should be possible. In systematic review, the researchers will document every step of the process so that their work can be replicated and validated.6 Transparency also contributes to the reader’s ability to validate that rigor was upheld in the systematic review process. A highquality systematic review will be explicit and transparent about the methods used to conduct the review. This transparency is important to allow critical appraisal of the actual systematic review. Conclusions drawn from systematic reviews that are vague about their methods or that seem to skip reporting of specific steps should be viewed with suspicion. Standards for initiating a systematic review provide guidance in establishing a team with
SYSTEMATIC REVIEW
appropriate expertise. It is acknowledged that a single investigator is not sufficient to produce a high-quality systematic review. A minimum of two reviewers is needed to minimize bias and promote objectivity. Potential conflicts of interest should be identified and disclosed by the reviewers. Although the review team makes final decisions about what is and is not included in a systematic review, stakeholder input should be solicited as needed. The need for a systematic review, as well as the process that will be used to conduct the review, should be clearly articulated before the actual review begins. A solid systematic review is guided by a defined protocol developed before the start of the review.6 The protocol should include a justification for the review, a literature search strategy, criteria for including or excluding individual studies, methods of study appraisal, outcome measures to be evaluated, methods to be used to extract data from studies, and processes to be used when reviewers disagree. A well-designed systematic review protocol should be subjected to peer review and made available for public comment to allow refinement and improvement of the planned systematic review. A systematic review should include collaboration with a health sciences librarian who is able to help refine the search strategy and ensure a comprehensive search for relevant evidence. Exhaustive search strategies may include contacting study authors to clarify study information, hand searching of journals and conference abstracts, and searching for grey literature and unpublished studies. Once studies have been retrieved, the reviewers should independently review the retrieved studies for study inclusion or exclusion using the predefined criteria from the systematic review protocol. This blinded work increases the likelihood of including studies of high quality and relevance while excluding studies of lower quality. This independent blinded approach should be repeated during critical appraisal of selected studies and the data extraction phase. The methods that will be used to synthesize the evidence being evaluated should be described in the systematic review protocol.6 These methods should be explicit and should recognize the type of research methodology being included in the system-
457
atic review. Specifically, evidence from qualitative studies should be evaluated and pooled in a manner that is distinct from evidence drawn from quantitative studies. Although quantitative studies may allow the pooling of data through a unique quantitative analysis known as a meta-analysis, evidence drawn from qualitative studies may pooled in a unique qualitative analysis known as meta-synthesis. A comprehensive systematic review may include evidence from a variety of study methodologies including quantitative, qualitative, and economic evidences. An emerging area of synthesis is the pooling of multiple systematic reviews, known as an umbrella review. An umbrella review is essentially a systematic review of systematic reviews.7 The final stage of systematic review production is the preparation of the final report. A key element of this stage is external peer review by an independent party to ensure the scientific quality of the systematic review. A well-written systematic review report will include the study rationale, reporting of systematic review methods, results section, and discussion section. The external peer review process used to evaluate both the protocol and the final report should be explicitly reported. The final systematic review should be available in a manner that ensures access by clinicians who will use the review to guide their practice. This may be accomplished by publication in libraries of the organizations known for systematic review production (eg, the Joanna Briggs Institute Library of Systematic Reviews, the Cochrane Library, or the Campbell Library) and/or other professional peer-reviewed publications. One excellent resource designed for authors and journal reviewers, but an excellent tool for critiquing systematic reviews as a research consumer is the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist (http:// www.prisma-statement.org/). The checklist provides a comprehensive overview of key components of a quality systematic review.
Summary Direct care providers rely on evidence to guide and inform practice. Often, the evidence on which practice is based is outdated, inadequately synthesized, misinformed, and unsuccessfully disseminated. This poorly constructed synthesis of
BAKER AND WEEKS
458
evidence can ultimately lead to misinformed health care providers and a poor quality of care for our patients. Although clinical reasoning can never be replaced, quality systematic reviews offer direct care providers timely evidence to assist with clinical decision making. Systematic reviews are an ‘‘improved reflection on reality’’ that offer efficient integration
of information, provide data for rational decision making, establish the generalizability of a body of knowledge, assess consistency of relationships, and increase the power of aggregated studies.8 Clinicians dedicated to application of best evidence in their practice will benefit from the knowledge of appraising and using systematic reviews.
References 1. Holly C, Salmond C, Saimbert MK. Comprehensive Systematic Review for Advanced Nursing Practice. New York, NY: Springer; 2012. 2. Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: Synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126:376-380. 3. Joanna Briggs Institute (JBI). Joanna Briggs Institute Comprehensive Review Training Program: The Systematic Review of Evidence Generated through Quantitative Review. Adelaide, South Australia: Joanna Briggs Institute; 2013:9. 4. Institute of Medicine (IOM). Finding What Works in Health Care: Standards for Systematic Re-
views. Washington, DC: The National Academies Press; 2011. 5. Pearson A, Weeks S, Stern C. Translation Science and the JBI Model of Evidence-based Healthcare. Philadelphia, PA: Lippincott; 2011. 6. Evans D. Systematic reviews of nursing research. Intensive Crit Care Nurs. 2001;17:51-57. 7. Joanna Briggs Institute (JBI). Joanna Briggs Institute Reviewers’ Manual: 2011 Edition. Adelaide, South Australia: Joanna Briggs Institute; 2011. 8. Mulrow CD. Systematic reviews: Rationale for systematic reviews. BMJ. 1994;309:597-599.
