Evidence-based health care management: what is the research evidence available for health care managers?

Health Services Research and Practice

Evidence-Based Health Care Management: What Is the Research Evidence Available for Health Care Managers?

Evaluation & the Health Professions 2014, Vol. 37(3) 314-334 ª The Author(s) 2013 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/0163278713511325 ehp.sagepub.com

Mirou Jaana1,2, Smruti Vartak3, and Marcia M. Ward3

Abstract In light of increasing interest in evidence-based management, we conducted a scoping review of systematic reviews (SRs) and meta-analyses (MAs) to determine the availability and accessibility of evidence for health care managers; 14 MAs and 61 SRs met the inclusion criteria. Most reviews appeared in medical journals (53%), originated in the United States (29%) or United Kingdom (22%), were hospital-based (55%), and targeted clinical providers (55%). Topics included health services organization (34%), quality/patient safety (17%), information technology (15%), organization/workplace management

1

Telfer School of Management, University of Ottawa, Ottawa, Ontario, Canada School of Business, Lebanese American University, Beirut, Lebanon 3 Department of Health Management and Policy, University of Iowa, Iowa City, IA, USA 2

Corresponding Author: Mirou Jaana, Telfer School of Management, University of Ottawa, 55 Laurier Ave. East, Ottawa, Ontario, Canada K1N6N5. Email: [email protected]

Downloaded from ehp.sagepub.com at NORTHWESTERN UNIV LIBRARY on March 24, 2015

Jaana et al.

315

(13%), and health care workforce (12%). Most reviews addressed clinical topics of relevance to managers; management-related interventions were rare. The management issues were mostly classified as operational (65%). Surprisingly, 96.5% of search results were not on target. A better classification within PubMed is needed to increase the accessibility of meaningful resources and facilitate evidence retrieval. Health care journals should take initiatives encouraging the publication of reviews in relevant management areas. Keywords evidence-based management, health care managers, health care management, scoping review, systematic review, meta-analysis, health care organizations

Introduction The concepts of evidence-based medicine and evidence-based practice have been used for decades to guide clinical practice and reduce potential variation in the quality of care. They refer to the systematic gathering and critical appraisal of research evidence that support clinical decision making and optimal provision of patient care (Rosenberg & Donald, 1995). In a clinical environment dominated by the notion of ‘‘evidence base,’’ the practice of evidence-based health care management is likewise expected. However, the concept of evidence-based management (EBMgt) remains new, and it is unclear whether scientific research evidence is actually available in the form of scientific publications and readily accessible for health care managers. Scholars have been divided in opinions regarding the benefits and challenges associated with the use of EBMgt (Alexander, Hearld, Jiang, & Fraser, 2007; Arndt & Bigelow, 2009; Kovner & Rundall, 2006; Walshe & Rundall, 2001; Young, 2002). Yet, even researchers who cautioned against its use recognized the value of health care managers incorporating evidence from research when planning and implementing decisions (Arndt & Bigelow, 2009). In fact, managers are expected to be challenged increasingly, both internally and externally, about the decisions they make (Alexander et al., 2007; Neuhauser, 2000; Walshe & Rundall, 2001). A review of the literature on EBMgt revealed a debate in its early development over the meaning of the concept (Young, 2002). One perspective perceived it as managers encouraging and facilitating the application of evidence-based medicine and practice (Caine & Kenrick, 1997; Luker, 1997). Another perspective viewed EBMgt as an individual activity leading


316

Evaluation & the Health Professions 37(3)

to accountable management decisions (Hewison, 1997; McClarey, 1998). Over time, research led to refinement of the concept. This evolution led Kovner and Rundall (2006, p. 6) to define evidence-based health services management as ‘‘the systematic application of the best available evidence to the evaluation of managerial strategies for improving the performance of health services organizations.’’ More recently, Briner, Denyer, and Rousseau (2009) described EBMgt as a way of thinking about how to make decisions using different types of information and employing various kinds of research evidence to support the process and outcomes of practitioners’ decision making. Among the most efficient tools for examining the scientific research evidence in a certain field or discipline are systematic reviews (SRs) and meta-analyses (MAs). SRs are considered essential tools for identifying and critically appraising relevant research from studies in a specific area (Greenhalgh, 1997). Recently, SRs have become fundamental components for evidence-based practice, given their unique methodology for identifying, appraising, and synthesizing the best evidence in a certain domain (Briner, Denyer, & Rousseau, 2009). MAs are similar to SRs in their goal, but further quantify the findings and employ statistical analyses to summarize findings and examine relationships between variables of interest (Greenhalgh, 1997). The growth of MAs and SRs has prompted development of guidelines (e.g., PRISMA—Preferred Reporting Items for Systematic Reviews and Meta-Analyses and Quality of Reporting of MetaAnalyses) that lay out best practices in conducting and publishing these important tools (Moher et al., 1994; Moher, Liberati, Tetziaff, & Altman, 2009). In the area of management in general, there has been discussion regarding the existence of evidence in the literature about EBMgt and its effectiveness (Briner et al., 2009; Reay, Berta, & Kohn, 2009). Yet, this discussion has focused on EBMgt in general and in the business field. At present, little is known about the evidence available for health care managers, in specific, to which they may refer in their decision making and practices. This article contributes to this area by examining the extent to which MAs and SRs are available (i.e., published in the scientific literature) and readily accessible (i.e., easy to find upon searching the scientific literature) for health care managers to support their decision making. Specifically, we present the evolution of SRs and MAs available for health care managers between 2000 and 2010, and conduct a scoping review over a 2-year period (2009–2010) to determine the size and nature of scientific evidence to which health care managers may refer for EBMgt.


Jaana et al.

317

A scoping review is usually developed with the goal of providing an initial overview of the size and nature of available research evidence on a particular topic that has not yet been fully explored (Anderson, Allen, Peckham, & Goodwin, 2008; Arksey & O’Malley, 2005; Levac, Colquhoun, & O’Brien, 2010). Therefore, it is usually conducted with a mapping approach to assess the breadth and size of research activities in a certain area, which is informative to managers and researchers, and sets the stage for future SRs (Arksey & O’Malley, 2005). It is important to note that scoping reviews differ from SRs in that they are usually performed over a short period of time, are exploratory in nature, and do not present a comprehensive research synthesis or quality appraisal (Cruzes & Dyba, 2011). In this article, we conducted an exhaustive search to find SRs and MAs on EBMgt in the health care field. We critically appraised them to identify those that met the PRISMA guidelines. We then adopted a scoping approach and examined the extent, range, and nature of these reviews. In doing so, we mapped their content onto publication characteristics, management areas, manager roles, target audiences, organization type, focus, and review implications to characterize the evidence available for health care managers.

Method A literature search was conducted on PubMed using the term ‘‘evidence based management’’ to identify all potential relevant articles as a starting point for this review. This initial search produced 41,549 articles, which were reduced to 32,569 articles after limiting the search to the English language and the time period 2000–2010. PubMed includes a ‘‘limit’’ parameter for MAs, which was used; yet, no parallel limit exists for SRs. Hence, we explored several strategies and decided to limit the search to ‘‘review,’’ and further to the subset ‘‘SRs’’ within reviews. As a result, we identified 687 MAs and 7,290 SRs. The search was further refined to include those reviews that had abstracts, yielding 668 MAs and 6,543 SRs. The distribution of these articles over the 10-year period revealed an increasing trend in the number of published reviews with a steep increase over the last few years. Given this pattern, in-depth analysis of content was limited to articles published in the most recent 2 years (2009 and 2010) of the search, yielding 267 MAs and 2,009 SRs. A flowchart of the review steps is shown in Figure 1. At each step, two reviewers examined the criteria for inclusion of the MAs and SRs and


318


PubMed search for meta-analyses N = 267

PubMed search for systematic reviews N = 2009

224 articles excluded based on title

1623 articles excluded based on title 43 articles retrieved for abstract examination

386 articles retrieved for abstract examination

22 articles excluded based on abstract examination

3 articles excluded for not meeting the criteria of a meta-analysis

2 articles excluded for non-relevance to managers

246 articles excluded based on abstract examination 19 articles retained for full text examination

142 articles retained for full text examination

14 retained and examined in this review

61 retained and examined in this review

73 articles excluded for not meeting the criteria of a systematic review 8 articles excluded for non-relevance to managers

Figure 1. Flowchart of the search and selection process.

independently made determinations. A third reviewer served as a tiebreaker for all disagreements. Where questions remained, the three reviewers discussed the criteria and reached consensus. Because of the large number of articles meeting initial criteria, the first review step examined the titles of the articles. This step was very informative in that over 80% (224 MAs and 1,623 SRs) were excluded because the subject matter was clearly not related to health services management. Instead, nearly all of these excluded articles were on the topic of management of diseases or specific clinical conditions, making them relevant to clinicians, but not to health care managers. As a second review step, this process was repeated by examining the abstracts of the remaining articles. Upon further review, 22 MAs and 246 SRs were excluded at this step for the same reason. For the third review step, the full articles of the 19 MAs and 142 SRs full articles were examined in depth. Using the same criteria as above, an additional two MAs and eight SRs were excluded for nonrelevance to managers. At this


Jaana et al.

319

step, 3 MAs and 73 SRs were also eliminated for not meeting the key PRISMA criteria related to processes that constitute an SR or an MA (i.e., clarity in question formulation and methods for identifying, selecting, and appraising studies). As a final step, the remaining 14 MAs and 61 SRs were examined in detail for content in relation to EBMgt. The complete list of MAs and SRs included in this study appears in the Appendix.

Coding for Content Mapping A coding sheet provided definitions and criteria for content review; a copy is available upon request. Coding of general information included the number of studies/articles in each review, the country of origin, and the predominant focus of the journal. Data on the type of empirical articles reviewed included information related to whether a specific intervention was studied (e.g., clinical, managerial), the population targeted (e.g., providers, patients), the study setting (e.g., clinics, hospitals, community), the outcome measures, and the unit of analysis. Coding of management content included the focus of the review (e.g., economic evaluations, health information technology, patient safety, and quality improvement), the intended target audience (e.g., health care providers, researchers, administrators), the focus of implications (e.g., health care organizations, policy makers), and the type of management issues addressed (e.g., operational management, theoretical/conceptual issues, strategic management). In addition, the coders used a 7-point Likert-type scale to code their judgment of the usefulness of the topic to a wide range of health care managers, and the strength of the evidence in the review. For the usefulness dimension, the coders based their ratings on whether the reviews would attract/be read by a wide spectrum of health care managers regardless of the setting and type of health services organization and the extent to which managers could put the information in the review into practice. For the strength of evidence, the coders rated this dimension taking into consideration the number and quality of the original studies included in the reviews as well as the quality of the review itself (i.e., rigor in identifying, appraising, and synthesizing evidence). Coding categories were developed through review of the content of the first 50 articles and refined through discussion by the three authors to combine infrequent codes and eliminate unclear or redundant categories.


320


Findings General Overview The content of the MAs and SRs did not differ substantially in relation to the categories included in our coding scheme; so the findings described below reflect the combined results. Publication Characteristics. EBMgt MAs and SRs were published in journals classified as medical (53%), nursing (8%), health services research (8%), medical informatics (7%), management (3%), and others (12%). In addition to journals, 9% of the articles were published by the Cochrane Collaborative, a collaboration of over 100 countries focused on evidence-based practice that conducts numerous MAs and SRs. The search was limited to articles published in English, and thus it is not surprising that the most frequent countries of origin were the United States (28%), United Kingdom (21%), Australia (16%), Canada (15%), European countries (13%), and others (7%). The average number of empirical studies included in the MAs and SRs were 23 studies (range ¼ 6–38) and 37 studies (range ¼ 2–219), respectively. Combining review types, the number of studies included in the reviews was fairly evenly distributed, with a significant number of reviews (around 40%) incorporating more than 30 articles: 0–10 studies (19%); 11–20 studies (25%); 21–30 studies (17%); 31–40 studies (15%); and >40 studies (24%). Target Audience. The intended target audience for these EBMgt reviews was primarily identified as clinical providers (55%) and/or administrators (42%). Articles could have multiple target audiences and others included researchers (26%), quality managers (11%), international stakeholders (3%), workplace education entities (2%), and combinations or crossfunctional audiences such as multidisciplinary teams, staff, and family (7%). The reviews were viewed as having the greatest implications for health care organizations (88%) or policy makers (12%). Only a few articles were viewed as having implications for academic institutions (3%) or health care financing organizations (3%); five reviews presented implications at more than one level. Review Focus and Management Issues. The breakdown of categories of management content was captured in two ways. First, the general type of management area captured in the focus of the review was coded as organization of


Jaana et al.

321

health services (35%); patient safety and quality improvement (17%); health information technology (16%); organization/workplace management (13%); health care workforce (12%); economic evaluation (1%); and other topics (8%). Of the 75 reviews, 2 reviews focused on multiple management areas. Second, coders categorized the primary management issues, using a classification scheme adopted from Kovner and Rundall (2006, pp. 7–9) in which management issues were categorized as core business transactions, operational management, or strategic management. In doing so, the primary management issue of the reviews was most often operational management (64%) followed by strategic management (16%), with none of these reviews focused on core business transactions. However, a number of EBMgt reviews did not fit this scheme and could be best categorized as theoretical/conceptual (12%) or international (4%), and others focused on instrument development and validation. Focus of Interventions Studied. In terms of EBMgt content, 32% of the reviews focused on interventions. Of these, nearly 75% covered clinical issues of interest to managers (e.g., telemedicine, effectiveness of primary care nursing, crisis interventions), while 25% focused on management-related interventions (e.g., team effectiveness, pay for performance, framework for patient safety training). The reviewed intervention studies were mostly based in hospitals (55%), clinics (13%), community settings (13%), palliative care/hospice (5%), or mental health settings (5%). The targeted population was most commonly providers (56%; e.g., volume–outcome relationships, barriers/facilitators for continuous quality improvement), or patients (28%; e.g., community and telehealth interventions). The focus of analysis was generally individual (61%), followed by institutional (15%), organizational (14%), and system (5%). Usefulness and Strength of Evidence. The coders rated each review based on the usefulness of the topic to a wide range of managers. As shown in Figure 2, the most common rating was 5 on a 7-point Likert-type scale with higher scores indicating greater usefulness. Likewise, the coders rated each MA and SR on the strength of the evidence presented in the review. As can be seen in Figure 2, the most common rating was 6 on a 7-point Likert-type scale, indicating quite strong evidence in the reviews. Content by Management Issues and Review Foci. To better understand the content of the available EBMgt, we examined the types of management issues


322


Figure 2. Ratings of review usefulness and strength of evidence.

across the various health care management areas (Table 1). This pointed to considerable consistency in that operational issues were dominant across all foci.

Discussion and Implications EBMgt has gained increased attention in recent years. However, when asked if they referred to or used health care management research, ‘‘many executives say the articles are often inaccessible’’ (Alexander et al., 2007). There is also a perception among managers that health care research is often too focused on research implications at the expense of looking into what managers need to know (Alexander et al., 2007; Kovner & Rundall, 2006). This review is exploratory in nature and was conducted with a scoping approach that aims to uncover the nature, extent, and accessibility of scientific evidence for health care managers.

Current Situation: Evidence Availability and Accessibility PubMed searches of EBMgt over a 2-year period yielded 2,276 MAs and SRs. Surprisingly, 96.5% of these were not on target in terms of relevance to managers and meeting guidelines for an SR, which echoes the concerns voiced by health care managers over the accessibility and empirical focus of published evidence (Alexander et al., 2007; Kovner, Elton, & Billings, 2000; Pfeffer & Sutton, 2006). Relatively few MAs and SRs on EBMgt


323


52 20 12 8 8

58 33 8 8 — 92 — — 8 — 100 — — — — 56 — 11 11 22

Organization of Health Services (N ¼ 26) Operational Strategic Theoretical/conceptual International Other

Patient safety and quality improvement (N ¼ 13) Operational Strategic Theoretical/conceptual International Other

Health information technology (N ¼ 12) Operational Strategic Theoretical/conceptual International Other

Organization/workplace (N ¼ 10) Operational Strategic Theoretical/conceptual International Other

Health care workforce (N ¼ 9) Operational Strategic Theoretical/conceptual International Other

Content was mostly related to: 1. Nursing workforce (e.g., recruitment and retention, medication administration, and disease management); 2. Various topics including employee assistance programs, teams practice, and health professional’s training

Reviews mostly focused on clinical processes including: 1. Clinical handover and nursing hand offs; 2. Collaboration and teamwork (e.g., impacts of interprofessional collaboration, improving team effectiveness, and role of teamwork/communication)

Topics were mostly divided into three subareas: 1. Telehealth (e.g., teledermatology, telerehabilitation); 2. Adoption of HIT (e.g., electronic health record system, new nonpharmaceutical technologies); 3. Impacts of HIT (e.g., impacts on health care quality, impacts of computerized pharmacy clinical decision support, and role of HIT in chronic disease management)

Reviews varied significantly in content: 1. One group discussed aspects related to quality improvement (QI) in general (e.g., trauma QI, barriers and facilitators for continuous QI, and QI tools); 2. Other reviews varied in scope including patient safety training, risk factors and prevention of fall injuries, factors related to prescribing errors, and pay for performance

Reviews mostly covered: 1. Specific topics related to management and interventions for specific conditions and diseases (e.g., asthma, mental illness, maternal/neonatal care, etc.); 2. Topics related to team and multidisciplinary patient care approaches (e.g., rapid response and rehabilitation); 3. General topics at the macro level (e.g., effects of gatekeeping, transitional care, health systems integration, continuity of information)

Observations

a Percentage distribution of reviews within each management area (e.g., organization of health services, patient safety, QI, etc.) that focused on different types of management issues (e.g., operational, strategic, etc.). HIT ¼ Health Information Technology.

%a

Reviews across major management areas by types of management issues

Table 1. Overview of Reviews Content Across Major Management Areas by Types of Management Issues.

324


topics are available in the current literature, which is a significant limitation as these could be powerful information sources for health care managers to guide their decision making and practices. Briner et al. (2009) described the difficulty of practicing EBMgt without having accessible SRs of existing evidence in relevant management areas. This is especially important in critical areas demanding attention, such as quality improvement and health information technology where only 17% and 15% of the reviews were related to these areas, respectively. With recent calls for increasing use of EBMgt, it is critical to make relevant evidence in these areas readily accessible to health care managers. To further this goal, journals publishing articles with a health care management focus should encourage the publication of MAs and SRs that summarize and critically appraise research in relevant management areas. More importantly, finding the limited number of MAs and SRs of interest to health care managers can be an exhausting exercise. Searching popular research repositories, such as PubMed, relies on Medical Subject Headings (MeSH) terms that are standardized key words for categorizing topics. The search term ‘‘evidence-based management’’ is a MeSH term, but it has been applied far too broadly in PubMed, yielding search results that are overwhelmingly irrelevant for health care managers. Thus, a better classification within PubMed is needed to increase the accessibility of meaningful resources for managers. Without refinement of the MeSH terms, the available literature on EBMgt that could help health care managers will remain challenging to retrieve. None of the EBMgt reviews included in this scoping review were on similar topics, which reflects the diverse environment and decision making that health care managers face. However, beyond the varied topics, there were some commonalities across the included MAs and SRs. The type of health management areas in these reviews focused most commonly on organization of health services (35%), patient safety and quality improvement (17%), health information technology (16%), organization/workplace management (13%), and health care workforce (12%). Although 42% of the reviews had administrators as a target audience, most were published in medical outlets (53%) as compared to management journals (3%), thus diminishing their visibility and accessibility for their audience. Using the Kovner and Rundall (2006) scheme, two thirds of the reviews addressed operational management issues. Thus, the available MAs and SRs, if located, could actually help health care managers quickly access relevant studies that shed light on issues commonly faced in health care settings, namely hospitals, clinics, and community organizations.


Jaana et al.

325

Limitations We may have missed reviews that would be useful for health care managers but were not caught in the search process given the current classification. This is especially noticeable in the area of health information technology in which review articles on telemedicine and telemonitoring, for example, have become abundant in recent years (e.g., Ekeland, Bowes, & Flottorp, 2010; Jaana, Pare´, & Sicotte, 2009). These review articles presenting evidence on the impacts, effectiveness, and economic evaluations of telehealth models would be useful for health care managers in light of the pressure for addressing the challenges of aging populations and increasing prevalence of chronic diseases. It is also important to note that, given the large volume of reviews that resulted from the search as a consequence of the current classification, it was challenging to conduct a scoping review over an expanded period of time. Since there was a steep increase in the number of published reviews over the last few years, it seemed logical and reasonable to focus this scoping review on the last 2 years of the search.

Implications and Future Directions Reviews on direct management-related interventions (e.g., improving transitional care between institutions, communication interventions, design aspects at the system level) were rare. Similarly, reviews on interventions outside the hospital setting were also limited, indicating minimal evidence available on interventions in other areas such as clinics, community settings, long-term care, and palliative/hospice care settings. These areas necessitate attention by researchers to generate useful research evidence to guide management practice. In addition, reviews on strategic management issues (e.g., comparing care delivery across settings, regional collaborations, policy interventions, system-level initiatives) were few (16%) despite the weight and critical nature of decisions made at this level. These types of issues often involve significant resources and changes to organizations and health care systems. Thus, providing relevant research evidence to support managers’ decisions at this level is valuable. A quick search on Pubmed of these types of issues revealed that empirical studies exist on such topics as leadership and the health profession, partnerships and mergers, health systems, innovations, and so on. Therefore, future research should systematically


326


review empirical studies in these areas that could inform managers’ decisions at the strategic level. Given that in 55% of the reviews, the intended target audience was clinical providers who are already accustomed to the concept of evidence-based practice, these stakeholders may play an important role in informing managers about existing evidence when found. This could be done through creating ‘‘research cafe´s’’ that would bring together providers and administrators to exchange information about important evidence/reviews relevant to the organization’s goals and challenges. It may also be achieved through the creation of a ‘‘knowledge database,’’ consisting of a repository shared by clinicians and managers, which provides a platform for sharing reviews believed to be relevant to initiatives, projects, challenges, or current practices in their organization. In addition, the creation of an executive track in peer-reviewed journals, especially health care management journals, would facilitate the accessibility of these reviews to health care managers in a format appropriate for a practitioners’ audience. This practice has been observed in the Management Information Systems (MIS) field whereby MIS Quarterly Executive, which is the executive branch of a renowned journal in the field, is widely read by executives given its focus on research directly relevant and useful for their practice. Furthermore, an MISQ Review branch was created as an outlet for sharing knowledge and synthesizing evidence in the MIS field. Similar initiatives may benefit the health care management field and encourage scholars to conduct reviews that synthesize evidence on relevant contemporary issues of importance to health care management practices. Finally, in order to facilitate the uptake of research evidence by practitioners, it is important that authors of SRs and MAs incorporate a section clearly outlining the relevance of their findings to health care managers and organizations. In our coding and analysis, we noticed that the intended target audience was not often evident in the reviews. This issue has been previously voiced by managers who complain that health care research is often too focused on research implications at the expense of what managers need to know (Alexander et al., 2007; Kovner & Rundall, 2006; Sheldon, 2005). In conclusion, existing scientific evidence may be used by health care managers if access to it is facilitated by easier classification. Finding ways to facilitate the uptake of scientific research evidence would involve input from health care journals and researchers to enhance the visibility and readability of these reviews.


Jaana et al.

327

Appendix References for 75 Articles (Systematic Reviews and Meta-Analyses) Included in this Review Alexander, G., & Staggers, N. (2009). A systematic review of the designs of clinical technology findings and recommendations for future research. Advances in Nursing Science, 32, 252–279. Archampong, D., Borowski, D. W., & Dickinson, H. O. (2010). Impact of surgeon volume on outcomes of rectal cancer surgery: A systematic review and metaanalysis. Surgery– Journal of the Royal College of Surgeons of Edinburgh, 8, 341–352. Barlow, K. M., & Zangaro, G. A. (2010). Meta-analysis of the reliability and validity of the anticipated turnover scale across studies of registered nurses in the United States. Journal of Nursing Management, 18, 862–873. Bauermeister, J. A., Tross, S., & Ehrhardt, A. A. (2009). A review of HIV/AIDS system-level interventions. AIDS and Behavior, 13, 430–448. Berta, W., Barnsley, J., Bloom, J., Cockerill, R., Davis, D., Jaakkimainen, L., . . . Vayda, E. (2009). Enhancing continuity of information: Essential components of consultation reports. Canadian Family Physician. Medecin de Famille Canadien, 55, 624–625 e1–5. Bost, N., Crilly, J., Wallis, M., Patterson, E., & Chaboyer, W. (2010). Clinical handover of patients arriving by ambulance to the emergency department: A literature review. International Emergency Nursing, 18, 210–220. Buljac-Samardzic, M., Dekker-van Doorn, C. M., Van Wijngaarden, J. D. H., & Van Wijk, K. P. (2010). Interventions to improve team effectiveness: A systematic review. Health Policy, 94, 183–195. Candy, B., Holman, A., Leurent, B., Davis, S., & Jones, L. (2011). Hospice care delivered at home, in nursing homes and in dedicated hospice facilities: A systematic review of quantitative and qualitative evidence. International Journal of Nursing Studies, 48, 121–133. (Epub September 16, 2010.) Case, R., Haynes, D., Holaday, B., & Parker, V. G. (2010). Evidence-based nursing: The role of the advanced practice registered nurse in the management of heart failure patients in the outpatient setting. Dimensions of Critical Care Nursing: DCCN, 29, 57–62. Castillo, V. H., Martinez-Garcia, A. I., & Pulido, J. R. (2010). A knowledge-based taxonomy of critical factors for adopting electronic health record systems by physicians: A systematic literature review. BMC Medical Informatics and Decision Making, 10, 60. Chan, P. S., Jain, R., Nallmothu, B. K., Berg, R. A., & Sasson C. (2010). Rapid response teams: A systematic review and meta-analysis. Archives of Internal Medicine, 170, 18–26. Chenoweth, L., Jeon, Y. H., Merlyn, T., & Brodaty, H. (2010). A systematic review of what factors attract and retain nurses in aged and dementia care. Journal of Clinical Nursing, 19, 156–167. Coleman, K., Mattke, S., Perrault, P. J., & Wagner, E. H. (2009). Untangling practice redesign from disease management: How do we best care for the chronically ill? Annual Review of Public Health, 30, 385–408. (continued) Downloaded from ehp.sagepub.com at NORTHWESTERN UNIV LIBRARY on March 24, 2015

328


Appendix (continued) Dieterich, M., Irving, C. B., Park, B., & Marshall, M. (2010). Intensive case management for severe mental illness. Cochrane Database of Systematic Reviews, CD007906. Dogherty, E. J., Harrison, M. B., & Graham, I. D. (2010). Facilitation as a role and process in achieving evidence-based practice in nursing: A focused review of concept and meaning. Worldviews On Evidence-Based Nursing, 7, 76–89. Doug, M, Adi, Y., Williams, J., Paul, M., Kelly, D., Petchey, R., & Carter, Y. H. (2011). Transition to adult services for children and young people with palliative care needs: A systematic review. Archives of Disease in Childhood, 96, 78–84. (Epub November 30, 2009). Findlay, B., Smith, K., Crawford, C. C., Coulter, I., Khorsan, R., & Jonas, W. B. (2010). Methodological complexities associated with systematic review of healing relationships. Alternative Therapies in Health and Medicine, 16, 40–46. Fung-Kee-Fung, M, Watters, J, Crossley, C, Goubanova, E, Abdulla, A, Stern, H, & Oliver, T. K. (2009). Regional collaborations as a tool for quality improvements in surgery: A systematic review of the literature. Annals of Surgery, 249, 565–572. Gaikwad, R., & Warren, J. (2009). The role of home-based information and communications technology interventions in chronic disease management: A systematic literature review. Health Informatics Journal, 15, 122–146. Garcia-Lizana, F., & Munoz-Mayorga, I. (2010). Telemedicine for depression: A systematic review. Perspectives in Psychiatric Care, 46, 119–126. Gurses, A. P., Marsteller, J. A., Ozok, A. A., Xiao, Y., Owens, S., & Pronovost, P. J. (2010). Using an interdisciplinary approach to identify factors that affect clinicians’ compliance with evidence-based guidelines. Critical Care Medicine, 38, S282–S291. Haesler, E., Bauer, M., & Nay, R. (2010). Recent evidence on the development and maintenance of constructive staff-family relationships in the care of older people: A report on a systematic review update. International Journal of Evidence-based Healthcare, 8, 45–74. Hamm, M. P., Osmond, M., Curran, J., Scott, S., Ali, S., Hartling, L., . . . Newton, A. S. (2010). A systematic review of crisis interventions used in the emergency department recommendations for pediatric care and research. Pediatric Emergency Care, 26, 952–962. Handoll, H. H., Cameron, I. D., Mak, J. C., & Finnegan, T. P. (2009). Multidisciplinary rehabilitation for older people with hip fractures. Cochrane Database of Systematic Reviews, CD007125. Hewson-Conroy, K. M., Elliott, D., & Burrell, A. R. (2010). Quality and safety in intensive care: A means to an end is critical. Australian Critical Care, 23, 109–129. Horstick, O., Runge-Ranzinger, S., Nathan, M. B., & Kroeger, A. (2010, June). Dengue vector-control services: How do they work? A systematic literature review and country case studies. Transactions of the Royal Society of Tropical Medicine & Hygiene, 104, 379–386. Hudson, P. L., Trauer, T., Graham, S., Grande, G., Ewing, G., Payne, S., . . . Thomas, K. (2010). A systematic review of instruments related to family caregivers of palliative care patients. Palliative Medicine, 24, 656–668. (continued)


Jaana et al.

329

Appendix (continued) Ilott, I., Booth, A., Rick, J., & Patterson, M. (2010, June). How do nurses, midwives and health visitors contribute to protocol-based care? A synthesis of the UK literature. International Journal of Nursing Studies, 47, 770–780. Inglis, S. C., Clark, R. A., McAlister, F. A., Ball, J., Lewinter, C., Cullington, D., . . . Cleland, J. G. (2010). Structured telephone support or telemonitoring programmes for patients with chronic heart failure. Cochrane Database of Systematic Reviews, CD007228. Jamal, A., McKenzie, K., & Clark, M. (2009).The impact of health information technology on the quality of medical and health care: A systematic review. The HIM journal, 38, 26–37. Johansson, T., & Wild, C. (2011). Telerehabilitation in stroke care: A systematic review. Journal Of Telemedicine And Telecare, 17, 1–6. (Epub November 19, 2010). Juillard, C. J., Mock, C., Goosen, J., Joshipura, M., & Civil, I. (2009). Establishing the evidence base for trauma quality improvement: A collaborative WHO-IATSIC review. World Journal of Surgery, 33, 1075–1086. Keleher, H., Parker, R., Abdulwadud, O., & Francis, K. (2009). Systematic review of the effectiveness of primary care nursing. International Journal of Nursing Practice, 15, 16–24. Kilner, E., & Sheppard, L. A. (2010). The role of teamwork and communication in the emergency department: A systematic review. International Emergency Nursing, 18, 127–137. Kolehmainen, N., Francis, J., Duncan, E., & Fraser, C. (2010). Community professionals’ management of client care: A mixed-methods systematic review. Journal of Health Services Research & Policy, 15, 47–55. Kongnyuy, E. J., & Uthman, O. A. (2009). Use of criterion-based clinical audit to improve the quality of obstetric care: A systematic review. Acta Obstetricia et Gynecologica Scandinavica, 88, 873–881. LaMantia, M. A., Scheunemann, L. P., Viera, A. J., Busby-Whitehead, J., & Hanson, L. C. (2010). Interventions to improve transitional care between nursing homes and hospitals: A systematic review. Journal of the American Geriatrics Society, 58, 777– 782. Lassi, Z. S., Haider, B. A., & Bhutta, Z. A. (2010). Community-based intervention packages for reducing maternal and neonatal morbidity and mortality and improving neonatal outcomes. Cochrane Database of Systematic Reviews, CD7754. Lemmens, K. M. M., Nieboer, A. P., & Huijsman, R. (2009). A systematic review of integrated use of disease-management interventions in asthma and COPD. Respiratory Medicine, 103, 670–691. Lloyd-Evans, B., Slade, M., Jagielska, D., & Johnson, S. (2009). Residential alternatives to acute psychiatric hospital admission: Systematic review. British Journal Of Psychiatry, 195, 109–117. Macy, R. J., & Johns, N. (2011). Aftercare services for international sex trafficking survivors: Informing U.S. service and program development in an emerging practice area. Trauma, Violence, & Abuse, 12, 87–98. (Epub December 31, 2010). (continued)


330


Appendix (continued) Martin, J. S., Ummenhofer, W., Manser, T., & Spirig, R. (2010). Interprofessional collaboration among nurses and physicians: Making a difference in patient outcome. Swiss Medical Weekly, 140, 33–39. Mayer, E. K., Purkayastha, S., Athanasiou, T., Darzi, A., & Vale, J. A. (2009). Assessing the quality of the volume-outcome relationship in uro-oncology. BJU International, 103, 341–349. McColl, M. A., Shortt, S., Godwin, M., Smith, K., Rowe, K., O’Brien, P., & Donnelly, C. (2009). Models for integrating rehabilitation and primary care: A scoping study. Archives of Physical Medicine and Rehabilitation, 90, 1523–1531. McGilton, K. S., Boscart, V., Fox, M., Sidani, S., Rochon, E., & Sorin-Peters, R. (2009). A systematic review of the effectiveness of communication interventions for health care providers caring for patients in residential care settings. Worldviews On Evidence-Based Nursing, 6, 149–159. McGowan, J. L., Grad, R., Pluye, P., Hannes, K., Deane, K., Labrecque, M., . . . Tugwell P. (2009). Electronic retrieval of health information by healthcare providers to improve practice and patient care. Cochrane Database of Systematic Reviews, CD004749. Miller, A., & Archer, J. (2010). Impact of workplace based assessment on doctors’ education and performance: A systematic review. British Medical Journal, 341, c5064. Murray, M. A., Fiset, V., Young, S., & Kryworuchko, J. (2009). Where the dying live: A systematic review of determinants of place of end-of-life cancer care. Oncology Nursing Forum, 36, 69–77. Neubeck, L., Redfern, J., Fernandez, R., Briffa, T., Bauman, A., & Freedman, S. B. (2009). Telehealth interventions for the secondary prevention of coronary heart disease: A systematic review. European Journal of Cardiovascular Prevention and Rehabilitation: official journal of the European Society of Cardiology, Working Groups on Epidemiology & Prevention and Cardiac Rehabilitation and Exercise Physiology, 16, 281– 289. Nosbusch, J. M., Weiss, M. E., & Bobay, K. L. (2011). An integrated review of the literature on challenges confronting the acute care staff nurse in discharge planning. Journal of Clinical Nursing, 20, 754–774. (Epub October 19, 2010). Opiyo, N., & English, M. (2010). In-service training for health professionals to improve care of the seriously ill newborn or child in low and middle-income countries (review). Cochrane Database of Systematic Reviews, CD007071. Perez, A., Doloresco, F., Hoffman J.M., Meek, P. D., Touchette, D. R., Vermeulen, L. C., Schumock, G. T. (2009). ACCP: Economic evaluations of clinical pharmacy services: 2001-2005. Pharmacotherapy, 29, 128. Pollack, K. M., Austin, W., & Grisso, J. A. (2010). Employee assistance programs: A workplace resource to address intimate partner violence. Journal of Womens Health (Larchmt), 19, 729–733. Riesenberg, L. A., Leisch, J., & Cunningham, J. M. (2010). Nursing handoffs: A systematic review of the literature. The American Journal of Nursing, 110, 24–34. (continued)


Jaana et al.

331

Appendix (continued) Robert, G., Greenhalgh, T., MacFarlane, F., & Peacock, R. (2010). Adopting and assimilating new non-pharmaceutical technologies into health care: A systematic review. Journal Of Health Services Research & Policy, 15, 243–250. Robertson, J., Walkom, E., Pearson, S. A., Hains, I., Williamsone, M., & Newby, D. (2010). The impact of pharmacy computerised clinical decision support on prescribing, clinical and patient outcomes: A systematic review of the literature. The International Journal of Pharmacy Practice, 18, 69–87. Sammer, C. E., Lykens, K., Singh, K. P., Mains, D. A., & Lackan, N. A. (2010). What is patient safety culture? A review of the literature. Journal of Nursing Scholarship, 42, 156–165. Schedlbauer, A., Prasad, V., Mulvaney, C., Phansalkar, S., Stanton, W., Bates, D. W., & Avery, A. J. (2009, July-August). What evidence supports the use of computerized alerts and prompts to improve clinicians’ prescribing behavior? Journal of American Medical Informatics Association, 16, 531–538. Shaw, C., McNamara, R., Abrams, K., Cannings-John, R., Hood, K., Longo, M., . . . Williams, K. (2009). Systematic review of respite care in the frail elderly. Health Technology Assessment, 13, 1–224. Sloman, M., & Williamson, G. R. (2009). Thrombolysis administration by nurses: An evolving UK evidence base? International Emergency Nursing, 17, 193–202. Solomons, N. M., & Spross, J. A. (2011). Evidence-based practice barriers and facilitators from a continuous quality improvement perspective: An integrative review. Journal of Nursing Management, 19, 109–120. (Epub December 13, 2010.) Suter, E., Oelke, N. D., Adair, C. E., & Armitage, G. D. (2009). Ten key principles for successful health systems integration. Healthcare Quarterly, 13, 16–23. Taylor, J.L., & Rew, L. (2011). A systematic review of the literature: Workplace violence in the emergency department. J Clin Nurs., 20(7-8), 1072-1085. (Epub 2010 Sep 15) Taylor, T. L., Killaspy, H., Wright, C., Turton, P., White, S., Kallert, T. W., . . . King, M. B. (2009). A systematic review of the international published literature relating to quality of institutional care for people with longer term mental health problems. BMC Psychiatry, 9, 55. Tully, M. P., Ashcroft, D. M., Dornan, T., Lewis, P. J., Taylor, D., & Wass, V. (2009). The causes of and factors associated with prescribing errors in hospital inpatients: A systematic review. Drug Safety: An international Journal of Medical Toxicology and Drug Experience, 32, 819–836. Van Herck, P., De Smedt, D., Annemans, L., Remmen, R., Rosenthal, M. B., & Sermeus, W. (2010). Systematic review: Effects, design choices, and context of pay-for-performance in health care. BMC Health Services Research, 10, 247–260. Velasco Garrido, M., Zentner, A., & Busse, R. (2011). The effects of gatekeeping: A systematic review of the literature. Scandinavian Journal of Primary Health Care, 29, 28–38. (Epub December 30, 2010.) Warshaw, E. M., Hillman, Y. J., Greer, N. L., Hagel, E. M, MacDonald, R., Rutks, I. R., Wilt, T. J. (2010). Teledermatology for diagnosis and management of skin conditions: A systematic review. Journal of the American Academy of Dermatology, 64, 759–772. (continued) Downloaded from ehp.sagepub.com at NORTHWESTERN UNIV LIBRARY on March 24, 2015

332


Appendix (continued) Weaver, S. J., Lyons, R., DiazGranados, D., Rosen, M. A., Salas, E., Oglesby, J., . . . King, H. B. (2010). The anatomy of health care team training and the state of practice: A critical review. Academic Medicine: Journal of the Association of American Medical Colleges, 85, 1746–1760. Welch, G., Garb, J., Zagarins, S., Lendel, I., & Gabbay, R. A. (2010). Nurse diabetes case management interventions and blood glucose control: Results of a metaanalysis. Diabetes Research and Clinical Practice, 88, 1–6. West, E., Mays, N., Rafferty, A. M., Rowan, K., & Sanderson, C. (2009). Nursing resources and patient outcomes in intensive care: A systematic review of the literature. International Journal of Nursing Studies, 46, 993–1011. Willgoss, T. G., Yohannes, A. M., & Mitchell, D. (2010). Review of risk factors and preventative strategies for fall-related injuries in people with intellectual disabilities. Journal of Clinical Nursing, 19, 2100–2109. Young, T., & Busgeeth, K. (2010). Home-based care for reducing morbidity and mortality in people infected with HIV/AIDS. Cochrane Database of Systematic Reviews, CD005417. Zaugg, B., & Wangler, M. (2009). A model framework for patient safety training in chiropractic: A literature synthesis. Journal of Manipulative and Physiological Therapeutics, 32, 493–499. Zwarenstein, M., Goldman, J., & Reeves, S. (2009). Interprofessional collaboration: Effects of practice-based inventions on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, CD000072.

Authors’ Note The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ. A preliminary version of the results was presented (poster) at the Academy Health Annual Research Meeting in Florida (June 2012).

Acknowledgment The authors thank Mark Willis, MHA, for assistance in the project.

Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by grant number R18HS018396 from the Agency for Healthcare Research and Quality (AHRQ).


Jaana et al.

333

References Alexander, J. A., Hearld, L. R., Jiang, H. J., & Fraser, I. (2007). Increasing the relevance of research to health care managers: Hospital CEO imperatives for improving quality and lowering costs. Health Care Management Review, 32, 150–159. Anderson, S., Allen, P., Peckham, S., & Goodwin, N. (2008). Asking the right questions: Scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems, 6, 7. Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8, 19–32. Arndt, M., & Bigelow, B. (2009). Evidence-based management in health care organizations: A cautionary note. Health Care Management Review, 34, 206–213. Briner, R. B., Denyer, D., & Rousseau, D. M. (2009). Evidence-based management: Concept cleanup time? Academy of Management Perspectives, 23, 19–32. Caine, C., & Kenrick, M. (1997). The role of clinical directorate managers in facilitating evidence-based practice: A report of an exploratory study. Journal of Nursing Management, 5, 157–165. Cruzes, D. S., & Dyba, T. (2011). Research synthesis in software engineering: A tertiary study. Information and Software Technology, 53, 440–455. Ekeland, A. G., Bowes, A., & Flottorp, S. (2010). Effectiveness of telemedicine: A systematic review of reviews. International Journal of Medical Informatics, 79, 736–771. doi:10.1016/j.ijmedinf.2010.08.006 Greenhalgh, T. (1997). Papers that summarize other papers (systematic reviews and meta-analyses). British Medical Journal, 315, 672–675. Hewison, A. (1997). Evidence-based medicine: What about evidence-based management? Journal of Nursing Management, 5, 195–198. Jaana, M., Pare´, G., & Sicotte, C. (2009). Home telemonitoring for respiratory conditions: A systematic review. The American Journal of Managed Care, 15, 313–320. Kovner, A. R., Elton, J. J., & Billings, J. (2000). Transforming health management: An evidence-based approach. Frontiers of Health Services Management, 16, 3–25. Kovner, A. R., & Rundall, T. G. (2006). Evidence-based management reconsidered. Frontiers of Health Services Management, 22, 3–22. Levac, D., Colquhoun, H., & O’Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science, 5, 69. Luker, K. (1997). Research and the configuration of nursing services. Journal of Clinical Nursing, 6, 259–267. McClarey, M. (1998). Implementing clinical effectiveness. Nursing Management, 5, 16–19.


334


Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1994). Improving the quality of reporting of meta-analysis of randomized controlled trials. The QUOROM statement. Lancet, 354(9193), 1896–1900. Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G., & the PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med, 6(6), e1000097. doi:10. 1371/journal.pmed.1000097 Neuhauser, D. (2000). The challenge of evidence-based management. Frontiers of Health Services Management, 16, 39–44. Pfeffer, J., & Sutton, R. (2006). Evidence-based management. Harvard Business Review, 84, 62–74. Reay, T., Berta, W., & Kohn, M. K. (2009). What’s the evidence on evidence-based management? Academy of Management Perspective, 23, 5–18. Rosenberg, W., & Donald, A. (1995). Evidence based medicine: An approach to clinical problem-solving. British Medical Journal, 310, 1122–1126. Sheldon, T. (2005). Making evidence synthesis more useful for management and policy making. Journal of Health Services Research and Policy, 10, 1–4. Walshe, K., & Rundall, T. (2001). Evidence-based management: From theory to practice in health care. The Milbank Quarterly, 79, 429–447. Young, S. K. (2002). Evidence-based management: A literature review. Journal of Nursing Management, 10, 145–151.


What is health for?--Human priorities in health care.

The health care transition research consortium health care transition model: a framework for research and practice.

Co-payments for health care: what is their real cost?

Foundation of evidence-based decision making for health care managers, part 1: systematic review.

Indian health care: what the dispute is all about.

What health care is learning from the aviation industry.

An Inside Look at Managed Care Executive Conversations within the Health Payer Council: What Is Evidence?

Impact of community participation in primary health care: what is the evidence?

Foundation of evidence-based decision making for health care managers-part II: meta-analysis and applying the evidence.

Public Health Laboratories and the Affordable Care Act: What the New Health-Care System Means for Public Health Preparedness.

Emergency planning and management in health care: priority research topics.

Telephonic advance care planning facilitated by health plan case managers.

What is asthma? Pathophysiology, demographics, and health care costs.

A culture of health and healthiest nation initiatives: the relevance of health care management research.

Outcomes and the management of health care. Health Services Research Group.

Outcome measures for primary health care: what are the research priorities?

Health Services Research: Cardiology in the Changing Health Care Landscape.

Health insurance is not health care.

Is American Health Care Uniquely Inefficient? Evidence from Prescription Drugs.

Syria: What should health care professionals do?

Shared decision making in preventive health care: What it is; what it is not.

Advance care planning in palliative care: a national survey of health professionals and service managers.

Willingness to participate in accountable care organizations: health care managers' perspective.

Public opinions about health care. Health Services Research Group.