SUPPLEMENT ARTICLE
The Importance of Context in Implementation Research Nancy Edwards, RN, PhD* and Pierre M. Barker, MD, MB, ChB†‡
Abstract: This article describes the pertinence of context in HIV/ AIDS implementation research. Without attending to context and how it interacts with interventions, national protocols for HIV/AIDS interventions are likely to fail or underperform. With its focus on what works, for whom, under what contextual circumstances, and whether interventions are scalable, implementation research yields context-sensitive designs and enhances the likelihood of scale-up for equitable outcomes. A framework for implementation science is presented alongside a review of published HIV/AIDS protocols for complex interventions. A case study of the South African Prevention of Mother-to-Child Transmission of HIV program highlights the application of complex system improvement principles in developing adaptive and context-sensitive scale-up designs. Preliminary recommendations are provided that can be used to characterize context when reporting interventions and describing how context can be accounted for in implementation strategies. Key Words: implementation science, context-sensitive research, HIV/AIDS (J Acquir Immune Defic Syndr 2014;67:S157–S162)
INTRODUCTION Although due attention is given to rigorous cause-andeffect fixed-protocol designs for efficacy and effectiveness research, the usefulness of these studies to health system planners and implementers is limited in the more complex settings and systems that are encountered outside of controlled study environments. Like much of the efficacy and effectiveness literature, many studies of HIV interventions have incomplete descriptions of the contextual environment in which the studies are conducted, in part due to the lack of a common typology to elucidate its features.1 Consequently, context is rarely and inconsistently mentioned in systematic reviews, creating difficulties for decision makers and program planners who play a key role in contextualizing interventions.2 The purpose of this article is to describe the pertinence of context in implementation research, propose some preliminary recommendations that can be used to characterize From the *School of Nursing, University of Ottawa, Ottawa, CA; †Institute for Health Care Improvement (IHI), Cambridge, MA; and ‡Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC. The authors have no funding or conflicts of interest to disclose. Correspondence to: Nancy Edwards, RN, PhD, School of Nursing, University of Ottawa, 1 Stewart Street, Suite 124, Ottawa, ON K1N 6N5, Canada (e-mail: [email protected]). Copyright © 2014 by Lippincott Williams & Wilkins. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
context when reporting interventions, and describe how context can be accounted for in implementation strategies.
RATIONALE AND KEY CONSIDERATIONS Implementation science examines what works, for whom, under what contextual circumstances, and whether interventions are scalable in equitable ways.3 The intervention includes the “what”—typically, a biomedical activity (eg, a drug given in a specific format to a defined population) or behavioral activity (eg, counseling using a theory-informed approach)—as well as the “how”—the implementation activities that are required to achieve full (equitable) coverage of the biomedical or behavioral intervention. The schemata in Figure 1 show the progression of knowledge from efficacy to full-scale implementation on 3 dimensions: determinants and their pathways, framing the research question, and design of the intervention. As described below, each of these dimensions is represented by a continuum. The first axis of implementation has proximal and sociostructural determinants as its end points. Proximal determinants are individual characteristics such as socioeconomic status, household income, or education level. Sociostructural determinants reflect embedded social conditions such as class, stigma, or discrimination, and power relationships. Even when officially repealed or abolished (eg, Apartheid in South Africa and residential schools for aboriginal children in Canada), these determinants may continue to drive inequities. The second axis concerns the framing of research questions, with causal questions at 1 end of the continuum (eg, Does a particular intervention produce a specific outcome in heterogeneous implementation contexts?), and adaptation questions at the other (eg, What adaptations and improvements to the intervention are required so that the intended outcomes are achieved in heterogeneous implementation contexts?). These adaptation processes may produce new emergent properties of the intervention, as well as the system and related subsystems into which they are introduced.4 The third axis focuses on the intervention, contrasting contextualized versus standardized interventions. Efficacy studies are typically conducted with fixed protocols applied in controlled environments; fidelity to the protocol is key. Adherence to the underlying theory (eg, behavior change theory) underpinning the intervention is considered essential, but that adherence is challenged the moment the intervention strays into environments that are different from those of the original studies.5 Various approaches (eg, protocols, manuals) are used by researchers and implementers to try to ensure interventions are delivered as planned. Effectiveness studies often examine the causes of deviation from fidelity to
J Acquir Immune Defic Syndr Volume 67, Supplement 2, November 1, 2014
www.jaids.com |
S157
Edwards and Barker
J Acquir Immune Defic Syndr Volume 67, Supplement 2, November 1, 2014
FIGURE 1. Progression of knowledge from efficacy to full-scale implementation.
the protocol (eg, provider/patient adherence). In contrast, adaptive interventions adjust, purposefully or otherwise, through contextual interactions with interventions and systems feedback processes to replicate the desired outcomes of the intervention in a more heterogeneous set of external influences.4 Contextualized interventions are guided by a theory of change, with an emphasis on understanding how the intervention and context interact as interventions are implemented and scaled up, aiming to achieve high degrees of effectiveness in heterogeneous complex environments. These 3 axes highlight the pertinence of context to implementation science and the different ways in which context is understood, examined, and addressed across efficacy, effectiveness, and adaptive implementation studies. With respect to efficacy and effectiveness studies, context is considered a nuisance factor, a cointervention or confounder that must be controlled or adjusted.6 In contrast, implementation science questions can only be answered when the intervention is tested in a heterogeneous mix of contextual settings, since the adaptability of the intervention to different settings will test the limits of its scalability. Both proximal and more distal elements of context are considered relevant. Furthermore, what is relevant is determined by what contextual elements (both static and dynamic) interact with the intervention, shaping and modifying it, rather than how context is defined a priori by the research team. Although efficacy studies aim to reduce the signal-tonoise ratio by controlling for context, implementation science seeks to understand and examine context, how it shapes and interacts with interventions, and how interventions are modified/adapted by patients, providers, organizations, and communities in response to shifting contextual circumstances. It follows that key features of implementation science are the purposeful selection of heterogeneous contexts (systems, organizations, and populations), understanding the
S158
| www.jaids.com
interaction between interventions and context through an adaptation lens, and the transposition of findings7 to other contexts and systems.
SCALABILITY AND REPLICABILITY OF INTERVENTIONS Although this review focuses on the importance of context in the scalability of an intervention, we must differentiate between scalability and replicability. Scalability concerns the equitable reach of the intervention. The rate, pace, and reach of scalability is determined by many political, infrastructural, and personnel factors as well as the characteristics of the populations themselves.8,9 The field of HIV implementation is notable for the profusion of effectiveness studies. Many are not scalable, however, either because they require a level of resources and infrastructural elements that do not currently exist in the broader health system, or they lack the methods and capability to adapt the intervention into complex environments that may be very different from the original study environments. It is problematic to simply replicate most efficacy and effectiveness studies. Replicability implies disseminating the intervention without further adaptation and is unlikely to succeed unless the new environment is very similar to the test environment. Thus, rapid spread can only occur once testing and adaptation of the delivery process has occurred in multiple contexts likely to be encountered in a complex scale-up environment. Only when adapted to work across multiple contexts can an intervention be replicated with little further adaptation. This approach was used during the latter stages of the scale-up of the South African Prevention of Mother-to-Child Transmission (PMTCT) program.10–12 At the same time, key attributes of the intervention itself (eg, relative 2014 Lippincott Williams & Wilkins
J Acquir Immune Defic Syndr Volume 67, Supplement 2, November 1, 2014
advantage, compatibility, simplicity) can increase the likelihood of dissemination.13 Similarly, strategies for preparing an environment for change and advice on promoting rapid spread of a new intervention (eg, leadership roles for change, packaging the new ideas, communication strategies, strengthening the social system of the adopter community, measurement and feedback, and knowledge management) have been well described.14 However, much less attention has been given to describing context and the accommodation of context in implementation research designs.
ASSESSMENT OF CONTEXT IN HIV/AIDS STUDY PROTOCOLS A number of authors have identified limitations in how context is reported within studies,15 leading to subsequent difficulties in operationalizing the interventions. To survey how much attention is directed toward context as a factor in evaluation of HIV implementation studies in complex environments, we reviewed articles describing study protocols related to HIV/ AIDS, identified through a search of the BioMed Central database of study protocols. We identified any articles with the key words “HIV” or “AIDS.” Nineteen articles were found. These were then screened using the following inclusion criteria: (1) protocol is for an intervention (17 articles), (2) study was to be undertaken in at least 1 low- or middle-income country (15 articles), and (3) the intervention described was deemed complex, defined as multistrategy and/or multilevel, resulting in 6 articles that were used for the final review. We (N.E. and C.A.) read each article in its entirety and extracted any text that described context (Table 1). The pertinent text was then grouped into 2 categories: contextual elements and the ways in which these contextual elements had been used or would be used in relation to the study protocol. The first author also reviewed the articles for any explicit reference of a guiding framework used to identify contextual parameters for study. With the exception of gender analysis16–18 and domains of care provision,19 none of the protocols referred to a framework that had been explicitly used or would be used to guide the identification of contextual characteristics. The types of contextual characteristics described varied considerably and tended to focus on the study settings (ie, communities or facilities where study would be undertaken rather than the health system as a whole). Thus, descriptions of health system features such as financing, governance, leadership, human resource capacity, health information systems, and universal access to health services were largely absent.
A CASE STUDY OF THE SOUTH AFRICAN PMTCT PROGRAM The South African health system has started more patients on antiretroviral treatment than any other nation and has successfully scaled up its PMTCT.11 The South African PMTCT program is a leading model where a complex intervention was implemented at full national scale across a large number of different geographic and sociocultural contexts. By 2012, the nationally reported transmission rate for Mother-to 2014 Lippincott Williams & Wilkins
Context-Sensitive Designs for Health Programs
Child Transmission (MTCT) was less than 3%,12 close to the reported transmission rates under clinical trial conditions.22 But the South African rollout of the PMTCT program underwent significant evolution, from a typical largely ineffective, contextinsensitive, cascaded-training approach to a sophisticated health systems intervention that used modern adaptive designs. Major social and political drama preceded the launch of the South African PMTCT program in 2002. The program was initiated when the epidemic had been underway for at least 10 years, by a reluctant administration that had been forced to do so by public action and court order.23 The reasons for the delay, which occurred in the early postapartheid era, were complex, and undoubtedly the government’s antipathy toward PMTCT reflected the slow start and initial poor performance of the program. Although efficacy studies showed single-dose nevirapine decreased MTCT of HIV by more than half, more than 2 years after full deployment of the PMTCT program (training of nursing staff at all clinical sites across the country, and universal availability of the nevirapine), an impact study in KwaZulu-Natal province showed the program was minimally effective.24 This prompted several demonstration projects that used more adaptive designs to improve the performance of the PMTCT program.10,25 These designs, increasingly being promoted for improving performance of health systems at a large scale, followed principles of complex system improvement adapted from manufacturing processes.26 The approach, known collectively as Quality Improvement (QI), seeks to improve performance by developing a common simplified view of the components and linkages of the system, real-time data feedback to track system performance, understanding the psychology of system change, and crucially, the testing and incorporation of ideas for performance improvement from the front-line practitioners, managers, and customers in a broader range of contexts. After successful demonstration of the effectiveness of this approach, the South African government adopted the method and tested its wider application11 before rapidly scaling up key elements of the approach across the country. This scale-up was promoted by significant political shifts resulting in crucial senior governmental leadership support for urgent solutions to the HIV epidemic for the first time. Other key determinants were more efficacious drug regimes, deployment of mobile treatment teams, and policy changes that allowed nurses to initiate antiretroviral therapy.11 As a result, within a few years, the MTCT rate across the South African public health delivery system fell to levels that were unprecedented in a public system of this size.12 There were additional lessons in the effect of context on the rate and success of efforts to implement and scale the program. One of the demonstration projects was designed as a cluster randomized trial (CRT) of 2 levels of health systems support27—a QI intervention with and without the use of a face-to-face collaborative learning system28 to promote context-sensitive improvement ideas. The project was implemented simultaneously in 3 districts, using a step wedge design, with clusters of clinics and their supervisors. The clusters included clinics from the same district but ignored natural and well-established referral and administrative linkages outside the boundaries of the cluster. www.jaids.com |
S159
J Acquir Immune Defic Syndr Volume 67, Supplement 2, November 1, 2014
Edwards and Barker
TABLE 1. Contextual Elements and Their Application in Selected HIV/AIDS Complex Intervention Protocols Articles Where Contextual Elements and Their Application Were Explicit 16
17
Warren et al ; Abramsky et al ; Andersson et al18
Harding et al19; Abramsky et al17; Hayes et al20; Andersson et al18
Harding et al19; Warren et al16; Tomlinson et al21
Warren et al16; Abramsky et al17; Tomlinson et al21; Hayes et al20; Andersson et al18
Harding et al19; Tomlinson et al21
Warren et al16; Abramsky et al17
Contextual Elements and Examples Cultural practices and gender norms Examples: gender violence, gender inequality, and related social norms, HIV stigma, sexual entitlement, and sexual coercion Characteristics of the study population and/or communities where the study was going to be undertaken Examples: prevalence of HIV and male circumcision; HIV-related morbidity and mortality rates; patterns of poverty, employment, and housing; population size and mobility; mobile phone ownership; patterns of alcohol use and social support structures Characteristics of health facilities where research was to be implemented
Examples: level of health care facility; urban, rural, or remote access to health facility; administrative and service delivery structures such as referral systems Characteristics of formal and informal health workers including traditional healers Examples: a strong or exclusive orientation to those health workers who would be engaged in the delivery of the intervention Sources of funding for the intervention
Examples: how PEPFAR program funds were distributed across types of activities (eg, treatment versus prevention) Sociopolitical influences expected to impact on intervention delivery
Application of Contextual Elements Shaped intervention design (eg, gender-focused intervention) Informed data collection approaches, selection of outcomes, and proposed intention to treat analyses Shaped intervention design
Informed eligibility of study participants or study settings and/or plans for stratified sampling or analysis
Informed eligibility criteria for health facilities, choice of comparative facilities, plans for stratified analysis, and estimates of health facility retention rates
Description of who would undertake the intervention with discussion of its relevance and feasibility
Accessibility and affordability of intervention and who was eligible to receive intervention
Anticipated problems with adherence and/or discontinuities in offering intervention and related health services
Examples: political violence and unrest, poor transportation, and temporary closure of some services, dynamic government policies related to integrated care
Leadership attributes and management functionality had a major impact on the ability to effect change. One district was led by a highly functional, energized district manager who was already using systems thinking to organize health program activities in his district, whereas the second district had weak leadership and was driven by a power struggle for control of clinic activities between municipal and district management structures. The third district had moderate strength of leadership, but a shortage of personnel led to the disbanding of the cadre of nurse supervisors that formed the basis of the cluster randomization. Within the districts, there was some urban/rural variation in sociocultural and economic characteristics, but there was ethnic and racial homogeneity and uniformly high clinic attendance (.90%). The CRT design was abandoned after 12 months because of a failure of this design to accommodate key
S160
| www.jaids.com
context-driven issues that were essential to the integrity of the study: • The randomization forced a sequence of scale-up that did not use the interest and good will of an early adopter community that could have supported the innovation phases of the scale-up; instead, we were forced to start the implementation with a spectrum of willing and unwilling participants. • The randomization unit in 1 district was destroyed when an administrative decision removed the key nurse–supervisor role around which the randomization was constructed. • The support of the participants for the project was severely tested when the study design prevented the spread of the intervention along natural lines of clinic aggregation and patient referral. 2014 Lippincott Williams & Wilkins
J Acquir Immune Defic Syndr Volume 67, Supplement 2, November 1, 2014
Context-Sensitive Designs for Health Programs
Equally important, the CRT, through its inherent design characteristics, assumes a homogeneity (or control) of the external environment and resists any change of the intervention design. As such, the CRT design is destined for failure when attempting to evaluate complex interventions being applied in ever-changing, heterogeneous environments. Like any large complex administrative unit, a health district includes multiple shifting and interlinked human networks. Any implementation study has to account for these deep linkages, the tendency for the linkages and resources that nourish them to change repeatedly over time, and the wide variation in competencies of leaders, managers, and front-line workers. This mismatch between CRT design and the contextual properties of the health systems in which interventions are being implemented and scaled up is particularly acute at the start of a scale-up process, when new contexts are being encountered and the theory of change and the intervention itself are evolving in response to these new contexts.29 Some argue CRTs can be used to evaluate the replication phase of scale-up.30 By contrast, adaptive designs are better suited to generate and study novel strategies that are required to implement and scale-up interventions in complex environments. The 3-district PMTCT project in KwaZulu Natal thrived when the randomization was abandoned and a more context-sensitive scale-up design was adopted. Each district pursued the same common objective (decrease MTCT to ,5%) but used a different implementation and scale-up strategy.27 The project resulted in numerous innovations for PMTCT that were collated and rapidly scaled across the province.31 The QI approach provided a template for a rapid national scale-up that required minimal further modification since it had already been tested in multiple contexts.10 This experience provoked questions on whether cluster randomized designs are appropriate if they disrupt naturally occurring networks or disregard the system properties of administrative structures (eg, districts) and whether they are appropriate for testing scale-up of complex health system interventions.30 There is growing consensus that adaptive designs are required so that the interventions can remain highly effective across large-scale heterogeneous implementation contexts.4–6 Rapid implementation and scale-up of complex interventions into heterogeneous environments requires a clear aim (outcome); a core set of measures that track key processes and progress toward that outcome; and a theory of change that incorporates anticipating, learning from, and adapting to local context. For PMTCT in South Africa, the aim was to reduce MTCT to less than 5% by 2013. A core set of 7 indicators tracked the continuum of care from early antenatal care through to early testing of the infant for HIV and became the basis of a quarterly report.12 The theory of change was that success would require a combination of motivated leadership, data feedback systems, and rapid-cycle testing of local ideas applied to a simplified care pathway for PMTCT that was clearly understood from leadership to the front line. After successful demonstrations, the tested change package was modified in a range of different contexts, and the national Department of Health was then able to move quickly to implement well-tested, context-sensitive changes at scale.
RECOMMENDATIONS
2014 Lippincott Williams & Wilkins
A framework for understanding and describing context, as well as an adaptive approach for implementing and scaling up interventions in a context-sensitive way, are essential to advance the field of implementation science in HIV/AIDS. Addressing context involves the development of well-informed assumptions regarding the context for delivery, not just in the setting where the study will take place but also in the wider health system that is the target for scale-up. Contextual parameters must include what health personnel (formal and informal) are in place, whether health services are accessible and affordable for those in need, how the health system is financed and governed, and how health information systems are operating. Implementation science protocols and articles describing study results should make explicit reference to the contextual framework being used. Effort is needed to identify common typologies, but in the meantime, frameworks such as the WHO health systems framework9 or the Greenhalgh framework for organizational change32 offer good starting points. Learning from the South African experience, a framework for context-sensitive implementation and scale-up should attend to the sequencing of scale-up components. These include the following: (1) provide a deep understanding of the dimensions of context to plan scale-up, develop a theory of change, and design a feedback measurement system and an adaptive strategy for implementation; (2) undertake prototype testing of the theory of change and early development of implementation ideas; (3) implement early scale-up testing in various complex environments reflecting anticipated variations in the scale-up area; and (4) follow-up with rapid scale-up, ensuring full (equitable) coverage with replication and minor further adaptation of implementation strategies that have been well tested across multiple contexts. Finally, a receptive environment to facilitate scale-up is also needed. This involves building support for change with leadership, managers, front-line practitioners, and the target population. It also includes ensuring that the environment is prepared with data systems, commodities such as supplies and essential drugs, clinical knowledge. ACKNOWLEDGMENTS The authors thank Cody Anderson for assistance with the literature review and formatting the article. REFERENCES 1. Edwards N, Di Ruggiero E. Exploring which context matters in the study of health inequities and their mitigation. Scand J Public Health. 2011;39 (6 suppl):43–49. 2. Lewin S, Bosch-Capblanch X, Oliver S, et al. Guidance for evidenceinformed policies about health systems: assessing how much confidence to place in the research evidence. PLoS Med. 2012;9:e1001187. 3. Fogarty International Center. Frequently Asked Questions About Implementation Science. 2013. Available at: http://www.fic.nih.gov/News/Events/ implementation-science/Pages/faqs.aspx. Accessed August 11, 2014. 4. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117. 5. Parry GJ, Carson-Stevens A, Luff DF, et al. Recommendations for evaluation of health care improvement initiatives. Acad Pediatr. 2013;13 (6 suppl):S23–S30.
www.jaids.com |
S161
Edwards and Barker
J Acquir Immune Defic Syndr Volume 67, Supplement 2, November 1, 2014
6. Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ. 2004;328:1561–1563. 7. Haddad S. Research approaches and their perspectives (part II): effectiveness of population health interventions, positivist approach. In: Proceedings of the International Conference to Advance a Population Health Intervention Research Agenda. Montreal, Canada; 2012:26–29. 8. Edwards N. Scaling-up health innovations and interventions in public health: a brief review of the current state-of-the-science. In: Conference to Advance the State of the Science and Practice on Scale-up and Spread of Effective Health Programs. Washington, DC; 2010. 9. World Health Organization. The WHO Health Systems Framework. Available at: http://www.wpro.who.int/health_services/health_systems_ framework/en/. Accessed July 23, 2014. 10. Mate KS, Ngubane G, Barker PM. A quality improvement model for the rapid scale-up of a program to prevent mother-to-child HIV transmission in South Africa. Int J Qual Health Care. 2013;25:373–380. 11. Barron P, Pillay Y, Doherty T, et al. Eliminating mother-to-child HIV transmission in South Africa. Bull World Health Organ. 2013;91:70–74. 12. Bhardwaj S, Barron P, Pillay Y, et al. Elimination of mother-to-child transmission of HIV in South Africa: rapid scale-up using quality improvement. S Afr Med J. 2014;104:239–243. 13. Rogers EM. Diffusion of Innovations. 4th ed. New York, NY: The Free Press; 1995. 14. Nolan K, Schall MW. Using a framework for spread: the case of patient access in the Veterans Health Administration. Jt Comm J Qual Patient Saf. 2005;31:339–347. 15. Luoto J, Shekelle PG, Maglione MA, et al. Reporting of context and implementation in studies of global health interventions: a pilot study. Implement Sci. 2014;9:57. 16. Warren CE, Mayhew SH, Vassall A, et al. Study protocol for the integra initiative to assess the benefits and costs of integrating sexual and reproductive health and HIV services in Kenya and Swaziland. BMC Public Health. 2012;12:973. 17. Abramsky T, Devries K, Kiss L, et al. A community mobilisation intervention to prevent violence against women and reduce HIV/AIDS risk in Kampala, Uganda (the SASA! Study): study protocol for a cluster randomised controlled trial. Trials. 2012;13:96. 18. Andersson N, Cockcroft A, Thabane L, et al. HIV prevention in favour of the choice-disabled in southern Africa: study protocol for a randomised controlled trial. Trials. 2013;14:274. 19. Harding R, Simms V, Penfold S, et al. Multi-centred mixed-methods PEPFAR HIV care & support public health evaluation: study protocol. BMC Public Health. 2010;10:584.
S162
| www.jaids.com
20. Hayes R, Ayles H, Beyers N, et al. HPTN 071 (PopART): rationale and design of a cluster-randomised trial of the population impact of an HIV combination prevention intervention including universal testing and treatment—a study protocol for a cluster randomised trial. Trials. 2014;15:57. 21. Tomlinson M, Doherty T, Jackson D, et al. An effectiveness study of an integrated, community-based package for maternal, newborn, child and HIV care in South Africa: study protocol for a randomized controlled trial. Trials. 2011;12:236. 22. Paintsil E, Andiman WA. Update on successes and challenges regarding mother-to-child transmission of HIV. Curr Opin Pediatr. 2009; 21:94–101. 23. Minister of Health v Treatment Action Campaign. CCT 8/02 (Constitutional Court of South Africa, 2002). 24. Rollins N, Little K, Mzolo S, et al. Surveillance of mother-to-child transmission prevention programmes at immunization clinics: the case for universal screening. AIDS. 2007;21:1341–1347. 25. Doherty T, Chopra M, Nsibande D, et al. Improving the coverage of the PMTCT programme through a participatory quality improvement intervention in South Africa. BMC Public Health. 2009;9:406. 26. Deming WE. Out of the Crisis. Cambridge, MA: MIT, Center for Advanced Educational Services; 1986. 27. Mate KS, Ngidi WH, Reddy J, et al. A case report of evaluating a largescale health systems improvement project in an uncontrolled setting: a quality improvement initiative in KwaZulu-Natal, South Africa. BMJ Qual Saf. 2013;22:891–898. 28. Institution for Healthcare Improvement. The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. Boston, MA: Institute for Healthcare Improvement; 2003. Available at: www.IHI. org. Accessed August 11, 2014. 29. Svoronos T, Mate KS. Evaluating large-scale health programmes at a district level in resource-limited countries. Bull World Health Organ. 2011; 89:831–837. 30. Shojania KG. Conventional evaluations of improvement interventions: more trials or just more tribulations? BMJ Qual Saf. 2013;22: 881–884. 31. South African National Department of Health. Models for the Scale up of HIV Prevention, Treatment and Care From South Africa and Beyond. Pretoria, South Africa: South African National Department of Health; 2010. 32. Greenhalgh T, Robert G, MacFarlane F, et al. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82:581–629.
2014 Lippincott Williams & Wilkins
The Importance of Context in Implementation Research Nancy Edwards, RN, PhD* and Pierre M. Barker, MD, MB, ChB†‡
Abstract: This article describes the pertinence of context in HIV/ AIDS implementation research. Without attending to context and how it interacts with interventions, national protocols for HIV/AIDS interventions are likely to fail or underperform. With its focus on what works, for whom, under what contextual circumstances, and whether interventions are scalable, implementation research yields context-sensitive designs and enhances the likelihood of scale-up for equitable outcomes. A framework for implementation science is presented alongside a review of published HIV/AIDS protocols for complex interventions. A case study of the South African Prevention of Mother-to-Child Transmission of HIV program highlights the application of complex system improvement principles in developing adaptive and context-sensitive scale-up designs. Preliminary recommendations are provided that can be used to characterize context when reporting interventions and describing how context can be accounted for in implementation strategies. Key Words: implementation science, context-sensitive research, HIV/AIDS (J Acquir Immune Defic Syndr 2014;67:S157–S162)
INTRODUCTION Although due attention is given to rigorous cause-andeffect fixed-protocol designs for efficacy and effectiveness research, the usefulness of these studies to health system planners and implementers is limited in the more complex settings and systems that are encountered outside of controlled study environments. Like much of the efficacy and effectiveness literature, many studies of HIV interventions have incomplete descriptions of the contextual environment in which the studies are conducted, in part due to the lack of a common typology to elucidate its features.1 Consequently, context is rarely and inconsistently mentioned in systematic reviews, creating difficulties for decision makers and program planners who play a key role in contextualizing interventions.2 The purpose of this article is to describe the pertinence of context in implementation research, propose some preliminary recommendations that can be used to characterize From the *School of Nursing, University of Ottawa, Ottawa, CA; †Institute for Health Care Improvement (IHI), Cambridge, MA; and ‡Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC. The authors have no funding or conflicts of interest to disclose. Correspondence to: Nancy Edwards, RN, PhD, School of Nursing, University of Ottawa, 1 Stewart Street, Suite 124, Ottawa, ON K1N 6N5, Canada (e-mail: [email protected]). Copyright © 2014 by Lippincott Williams & Wilkins. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
context when reporting interventions, and describe how context can be accounted for in implementation strategies.
RATIONALE AND KEY CONSIDERATIONS Implementation science examines what works, for whom, under what contextual circumstances, and whether interventions are scalable in equitable ways.3 The intervention includes the “what”—typically, a biomedical activity (eg, a drug given in a specific format to a defined population) or behavioral activity (eg, counseling using a theory-informed approach)—as well as the “how”—the implementation activities that are required to achieve full (equitable) coverage of the biomedical or behavioral intervention. The schemata in Figure 1 show the progression of knowledge from efficacy to full-scale implementation on 3 dimensions: determinants and their pathways, framing the research question, and design of the intervention. As described below, each of these dimensions is represented by a continuum. The first axis of implementation has proximal and sociostructural determinants as its end points. Proximal determinants are individual characteristics such as socioeconomic status, household income, or education level. Sociostructural determinants reflect embedded social conditions such as class, stigma, or discrimination, and power relationships. Even when officially repealed or abolished (eg, Apartheid in South Africa and residential schools for aboriginal children in Canada), these determinants may continue to drive inequities. The second axis concerns the framing of research questions, with causal questions at 1 end of the continuum (eg, Does a particular intervention produce a specific outcome in heterogeneous implementation contexts?), and adaptation questions at the other (eg, What adaptations and improvements to the intervention are required so that the intended outcomes are achieved in heterogeneous implementation contexts?). These adaptation processes may produce new emergent properties of the intervention, as well as the system and related subsystems into which they are introduced.4 The third axis focuses on the intervention, contrasting contextualized versus standardized interventions. Efficacy studies are typically conducted with fixed protocols applied in controlled environments; fidelity to the protocol is key. Adherence to the underlying theory (eg, behavior change theory) underpinning the intervention is considered essential, but that adherence is challenged the moment the intervention strays into environments that are different from those of the original studies.5 Various approaches (eg, protocols, manuals) are used by researchers and implementers to try to ensure interventions are delivered as planned. Effectiveness studies often examine the causes of deviation from fidelity to
J Acquir Immune Defic Syndr Volume 67, Supplement 2, November 1, 2014
www.jaids.com |
S157
Edwards and Barker
J Acquir Immune Defic Syndr Volume 67, Supplement 2, November 1, 2014
FIGURE 1. Progression of knowledge from efficacy to full-scale implementation.
the protocol (eg, provider/patient adherence). In contrast, adaptive interventions adjust, purposefully or otherwise, through contextual interactions with interventions and systems feedback processes to replicate the desired outcomes of the intervention in a more heterogeneous set of external influences.4 Contextualized interventions are guided by a theory of change, with an emphasis on understanding how the intervention and context interact as interventions are implemented and scaled up, aiming to achieve high degrees of effectiveness in heterogeneous complex environments. These 3 axes highlight the pertinence of context to implementation science and the different ways in which context is understood, examined, and addressed across efficacy, effectiveness, and adaptive implementation studies. With respect to efficacy and effectiveness studies, context is considered a nuisance factor, a cointervention or confounder that must be controlled or adjusted.6 In contrast, implementation science questions can only be answered when the intervention is tested in a heterogeneous mix of contextual settings, since the adaptability of the intervention to different settings will test the limits of its scalability. Both proximal and more distal elements of context are considered relevant. Furthermore, what is relevant is determined by what contextual elements (both static and dynamic) interact with the intervention, shaping and modifying it, rather than how context is defined a priori by the research team. Although efficacy studies aim to reduce the signal-tonoise ratio by controlling for context, implementation science seeks to understand and examine context, how it shapes and interacts with interventions, and how interventions are modified/adapted by patients, providers, organizations, and communities in response to shifting contextual circumstances. It follows that key features of implementation science are the purposeful selection of heterogeneous contexts (systems, organizations, and populations), understanding the
S158
| www.jaids.com
interaction between interventions and context through an adaptation lens, and the transposition of findings7 to other contexts and systems.
SCALABILITY AND REPLICABILITY OF INTERVENTIONS Although this review focuses on the importance of context in the scalability of an intervention, we must differentiate between scalability and replicability. Scalability concerns the equitable reach of the intervention. The rate, pace, and reach of scalability is determined by many political, infrastructural, and personnel factors as well as the characteristics of the populations themselves.8,9 The field of HIV implementation is notable for the profusion of effectiveness studies. Many are not scalable, however, either because they require a level of resources and infrastructural elements that do not currently exist in the broader health system, or they lack the methods and capability to adapt the intervention into complex environments that may be very different from the original study environments. It is problematic to simply replicate most efficacy and effectiveness studies. Replicability implies disseminating the intervention without further adaptation and is unlikely to succeed unless the new environment is very similar to the test environment. Thus, rapid spread can only occur once testing and adaptation of the delivery process has occurred in multiple contexts likely to be encountered in a complex scale-up environment. Only when adapted to work across multiple contexts can an intervention be replicated with little further adaptation. This approach was used during the latter stages of the scale-up of the South African Prevention of Mother-to-Child Transmission (PMTCT) program.10–12 At the same time, key attributes of the intervention itself (eg, relative 2014 Lippincott Williams & Wilkins
J Acquir Immune Defic Syndr Volume 67, Supplement 2, November 1, 2014
advantage, compatibility, simplicity) can increase the likelihood of dissemination.13 Similarly, strategies for preparing an environment for change and advice on promoting rapid spread of a new intervention (eg, leadership roles for change, packaging the new ideas, communication strategies, strengthening the social system of the adopter community, measurement and feedback, and knowledge management) have been well described.14 However, much less attention has been given to describing context and the accommodation of context in implementation research designs.
ASSESSMENT OF CONTEXT IN HIV/AIDS STUDY PROTOCOLS A number of authors have identified limitations in how context is reported within studies,15 leading to subsequent difficulties in operationalizing the interventions. To survey how much attention is directed toward context as a factor in evaluation of HIV implementation studies in complex environments, we reviewed articles describing study protocols related to HIV/ AIDS, identified through a search of the BioMed Central database of study protocols. We identified any articles with the key words “HIV” or “AIDS.” Nineteen articles were found. These were then screened using the following inclusion criteria: (1) protocol is for an intervention (17 articles), (2) study was to be undertaken in at least 1 low- or middle-income country (15 articles), and (3) the intervention described was deemed complex, defined as multistrategy and/or multilevel, resulting in 6 articles that were used for the final review. We (N.E. and C.A.) read each article in its entirety and extracted any text that described context (Table 1). The pertinent text was then grouped into 2 categories: contextual elements and the ways in which these contextual elements had been used or would be used in relation to the study protocol. The first author also reviewed the articles for any explicit reference of a guiding framework used to identify contextual parameters for study. With the exception of gender analysis16–18 and domains of care provision,19 none of the protocols referred to a framework that had been explicitly used or would be used to guide the identification of contextual characteristics. The types of contextual characteristics described varied considerably and tended to focus on the study settings (ie, communities or facilities where study would be undertaken rather than the health system as a whole). Thus, descriptions of health system features such as financing, governance, leadership, human resource capacity, health information systems, and universal access to health services were largely absent.
A CASE STUDY OF THE SOUTH AFRICAN PMTCT PROGRAM The South African health system has started more patients on antiretroviral treatment than any other nation and has successfully scaled up its PMTCT.11 The South African PMTCT program is a leading model where a complex intervention was implemented at full national scale across a large number of different geographic and sociocultural contexts. By 2012, the nationally reported transmission rate for Mother-to 2014 Lippincott Williams & Wilkins
Context-Sensitive Designs for Health Programs
Child Transmission (MTCT) was less than 3%,12 close to the reported transmission rates under clinical trial conditions.22 But the South African rollout of the PMTCT program underwent significant evolution, from a typical largely ineffective, contextinsensitive, cascaded-training approach to a sophisticated health systems intervention that used modern adaptive designs. Major social and political drama preceded the launch of the South African PMTCT program in 2002. The program was initiated when the epidemic had been underway for at least 10 years, by a reluctant administration that had been forced to do so by public action and court order.23 The reasons for the delay, which occurred in the early postapartheid era, were complex, and undoubtedly the government’s antipathy toward PMTCT reflected the slow start and initial poor performance of the program. Although efficacy studies showed single-dose nevirapine decreased MTCT of HIV by more than half, more than 2 years after full deployment of the PMTCT program (training of nursing staff at all clinical sites across the country, and universal availability of the nevirapine), an impact study in KwaZulu-Natal province showed the program was minimally effective.24 This prompted several demonstration projects that used more adaptive designs to improve the performance of the PMTCT program.10,25 These designs, increasingly being promoted for improving performance of health systems at a large scale, followed principles of complex system improvement adapted from manufacturing processes.26 The approach, known collectively as Quality Improvement (QI), seeks to improve performance by developing a common simplified view of the components and linkages of the system, real-time data feedback to track system performance, understanding the psychology of system change, and crucially, the testing and incorporation of ideas for performance improvement from the front-line practitioners, managers, and customers in a broader range of contexts. After successful demonstration of the effectiveness of this approach, the South African government adopted the method and tested its wider application11 before rapidly scaling up key elements of the approach across the country. This scale-up was promoted by significant political shifts resulting in crucial senior governmental leadership support for urgent solutions to the HIV epidemic for the first time. Other key determinants were more efficacious drug regimes, deployment of mobile treatment teams, and policy changes that allowed nurses to initiate antiretroviral therapy.11 As a result, within a few years, the MTCT rate across the South African public health delivery system fell to levels that were unprecedented in a public system of this size.12 There were additional lessons in the effect of context on the rate and success of efforts to implement and scale the program. One of the demonstration projects was designed as a cluster randomized trial (CRT) of 2 levels of health systems support27—a QI intervention with and without the use of a face-to-face collaborative learning system28 to promote context-sensitive improvement ideas. The project was implemented simultaneously in 3 districts, using a step wedge design, with clusters of clinics and their supervisors. The clusters included clinics from the same district but ignored natural and well-established referral and administrative linkages outside the boundaries of the cluster. www.jaids.com |
S159
J Acquir Immune Defic Syndr Volume 67, Supplement 2, November 1, 2014
Edwards and Barker
TABLE 1. Contextual Elements and Their Application in Selected HIV/AIDS Complex Intervention Protocols Articles Where Contextual Elements and Their Application Were Explicit 16
17
Warren et al ; Abramsky et al ; Andersson et al18
Harding et al19; Abramsky et al17; Hayes et al20; Andersson et al18
Harding et al19; Warren et al16; Tomlinson et al21
Warren et al16; Abramsky et al17; Tomlinson et al21; Hayes et al20; Andersson et al18
Harding et al19; Tomlinson et al21
Warren et al16; Abramsky et al17
Contextual Elements and Examples Cultural practices and gender norms Examples: gender violence, gender inequality, and related social norms, HIV stigma, sexual entitlement, and sexual coercion Characteristics of the study population and/or communities where the study was going to be undertaken Examples: prevalence of HIV and male circumcision; HIV-related morbidity and mortality rates; patterns of poverty, employment, and housing; population size and mobility; mobile phone ownership; patterns of alcohol use and social support structures Characteristics of health facilities where research was to be implemented
Examples: level of health care facility; urban, rural, or remote access to health facility; administrative and service delivery structures such as referral systems Characteristics of formal and informal health workers including traditional healers Examples: a strong or exclusive orientation to those health workers who would be engaged in the delivery of the intervention Sources of funding for the intervention
Examples: how PEPFAR program funds were distributed across types of activities (eg, treatment versus prevention) Sociopolitical influences expected to impact on intervention delivery
Application of Contextual Elements Shaped intervention design (eg, gender-focused intervention) Informed data collection approaches, selection of outcomes, and proposed intention to treat analyses Shaped intervention design
Informed eligibility of study participants or study settings and/or plans for stratified sampling or analysis
Informed eligibility criteria for health facilities, choice of comparative facilities, plans for stratified analysis, and estimates of health facility retention rates
Description of who would undertake the intervention with discussion of its relevance and feasibility
Accessibility and affordability of intervention and who was eligible to receive intervention
Anticipated problems with adherence and/or discontinuities in offering intervention and related health services
Examples: political violence and unrest, poor transportation, and temporary closure of some services, dynamic government policies related to integrated care
Leadership attributes and management functionality had a major impact on the ability to effect change. One district was led by a highly functional, energized district manager who was already using systems thinking to organize health program activities in his district, whereas the second district had weak leadership and was driven by a power struggle for control of clinic activities between municipal and district management structures. The third district had moderate strength of leadership, but a shortage of personnel led to the disbanding of the cadre of nurse supervisors that formed the basis of the cluster randomization. Within the districts, there was some urban/rural variation in sociocultural and economic characteristics, but there was ethnic and racial homogeneity and uniformly high clinic attendance (.90%). The CRT design was abandoned after 12 months because of a failure of this design to accommodate key
S160
| www.jaids.com
context-driven issues that were essential to the integrity of the study: • The randomization forced a sequence of scale-up that did not use the interest and good will of an early adopter community that could have supported the innovation phases of the scale-up; instead, we were forced to start the implementation with a spectrum of willing and unwilling participants. • The randomization unit in 1 district was destroyed when an administrative decision removed the key nurse–supervisor role around which the randomization was constructed. • The support of the participants for the project was severely tested when the study design prevented the spread of the intervention along natural lines of clinic aggregation and patient referral. 2014 Lippincott Williams & Wilkins
J Acquir Immune Defic Syndr Volume 67, Supplement 2, November 1, 2014
Context-Sensitive Designs for Health Programs
Equally important, the CRT, through its inherent design characteristics, assumes a homogeneity (or control) of the external environment and resists any change of the intervention design. As such, the CRT design is destined for failure when attempting to evaluate complex interventions being applied in ever-changing, heterogeneous environments. Like any large complex administrative unit, a health district includes multiple shifting and interlinked human networks. Any implementation study has to account for these deep linkages, the tendency for the linkages and resources that nourish them to change repeatedly over time, and the wide variation in competencies of leaders, managers, and front-line workers. This mismatch between CRT design and the contextual properties of the health systems in which interventions are being implemented and scaled up is particularly acute at the start of a scale-up process, when new contexts are being encountered and the theory of change and the intervention itself are evolving in response to these new contexts.29 Some argue CRTs can be used to evaluate the replication phase of scale-up.30 By contrast, adaptive designs are better suited to generate and study novel strategies that are required to implement and scale-up interventions in complex environments. The 3-district PMTCT project in KwaZulu Natal thrived when the randomization was abandoned and a more context-sensitive scale-up design was adopted. Each district pursued the same common objective (decrease MTCT to ,5%) but used a different implementation and scale-up strategy.27 The project resulted in numerous innovations for PMTCT that were collated and rapidly scaled across the province.31 The QI approach provided a template for a rapid national scale-up that required minimal further modification since it had already been tested in multiple contexts.10 This experience provoked questions on whether cluster randomized designs are appropriate if they disrupt naturally occurring networks or disregard the system properties of administrative structures (eg, districts) and whether they are appropriate for testing scale-up of complex health system interventions.30 There is growing consensus that adaptive designs are required so that the interventions can remain highly effective across large-scale heterogeneous implementation contexts.4–6 Rapid implementation and scale-up of complex interventions into heterogeneous environments requires a clear aim (outcome); a core set of measures that track key processes and progress toward that outcome; and a theory of change that incorporates anticipating, learning from, and adapting to local context. For PMTCT in South Africa, the aim was to reduce MTCT to less than 5% by 2013. A core set of 7 indicators tracked the continuum of care from early antenatal care through to early testing of the infant for HIV and became the basis of a quarterly report.12 The theory of change was that success would require a combination of motivated leadership, data feedback systems, and rapid-cycle testing of local ideas applied to a simplified care pathway for PMTCT that was clearly understood from leadership to the front line. After successful demonstrations, the tested change package was modified in a range of different contexts, and the national Department of Health was then able to move quickly to implement well-tested, context-sensitive changes at scale.
RECOMMENDATIONS
2014 Lippincott Williams & Wilkins
A framework for understanding and describing context, as well as an adaptive approach for implementing and scaling up interventions in a context-sensitive way, are essential to advance the field of implementation science in HIV/AIDS. Addressing context involves the development of well-informed assumptions regarding the context for delivery, not just in the setting where the study will take place but also in the wider health system that is the target for scale-up. Contextual parameters must include what health personnel (formal and informal) are in place, whether health services are accessible and affordable for those in need, how the health system is financed and governed, and how health information systems are operating. Implementation science protocols and articles describing study results should make explicit reference to the contextual framework being used. Effort is needed to identify common typologies, but in the meantime, frameworks such as the WHO health systems framework9 or the Greenhalgh framework for organizational change32 offer good starting points. Learning from the South African experience, a framework for context-sensitive implementation and scale-up should attend to the sequencing of scale-up components. These include the following: (1) provide a deep understanding of the dimensions of context to plan scale-up, develop a theory of change, and design a feedback measurement system and an adaptive strategy for implementation; (2) undertake prototype testing of the theory of change and early development of implementation ideas; (3) implement early scale-up testing in various complex environments reflecting anticipated variations in the scale-up area; and (4) follow-up with rapid scale-up, ensuring full (equitable) coverage with replication and minor further adaptation of implementation strategies that have been well tested across multiple contexts. Finally, a receptive environment to facilitate scale-up is also needed. This involves building support for change with leadership, managers, front-line practitioners, and the target population. It also includes ensuring that the environment is prepared with data systems, commodities such as supplies and essential drugs, clinical knowledge. ACKNOWLEDGMENTS The authors thank Cody Anderson for assistance with the literature review and formatting the article. REFERENCES 1. Edwards N, Di Ruggiero E. Exploring which context matters in the study of health inequities and their mitigation. Scand J Public Health. 2011;39 (6 suppl):43–49. 2. Lewin S, Bosch-Capblanch X, Oliver S, et al. Guidance for evidenceinformed policies about health systems: assessing how much confidence to place in the research evidence. PLoS Med. 2012;9:e1001187. 3. Fogarty International Center. Frequently Asked Questions About Implementation Science. 2013. Available at: http://www.fic.nih.gov/News/Events/ implementation-science/Pages/faqs.aspx. Accessed August 11, 2014. 4. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117. 5. Parry GJ, Carson-Stevens A, Luff DF, et al. Recommendations for evaluation of health care improvement initiatives. Acad Pediatr. 2013;13 (6 suppl):S23–S30.
www.jaids.com |
S161
Edwards and Barker
J Acquir Immune Defic Syndr Volume 67, Supplement 2, November 1, 2014
6. Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ. 2004;328:1561–1563. 7. Haddad S. Research approaches and their perspectives (part II): effectiveness of population health interventions, positivist approach. In: Proceedings of the International Conference to Advance a Population Health Intervention Research Agenda. Montreal, Canada; 2012:26–29. 8. Edwards N. Scaling-up health innovations and interventions in public health: a brief review of the current state-of-the-science. In: Conference to Advance the State of the Science and Practice on Scale-up and Spread of Effective Health Programs. Washington, DC; 2010. 9. World Health Organization. The WHO Health Systems Framework. Available at: http://www.wpro.who.int/health_services/health_systems_ framework/en/. Accessed July 23, 2014. 10. Mate KS, Ngubane G, Barker PM. A quality improvement model for the rapid scale-up of a program to prevent mother-to-child HIV transmission in South Africa. Int J Qual Health Care. 2013;25:373–380. 11. Barron P, Pillay Y, Doherty T, et al. Eliminating mother-to-child HIV transmission in South Africa. Bull World Health Organ. 2013;91:70–74. 12. Bhardwaj S, Barron P, Pillay Y, et al. Elimination of mother-to-child transmission of HIV in South Africa: rapid scale-up using quality improvement. S Afr Med J. 2014;104:239–243. 13. Rogers EM. Diffusion of Innovations. 4th ed. New York, NY: The Free Press; 1995. 14. Nolan K, Schall MW. Using a framework for spread: the case of patient access in the Veterans Health Administration. Jt Comm J Qual Patient Saf. 2005;31:339–347. 15. Luoto J, Shekelle PG, Maglione MA, et al. Reporting of context and implementation in studies of global health interventions: a pilot study. Implement Sci. 2014;9:57. 16. Warren CE, Mayhew SH, Vassall A, et al. Study protocol for the integra initiative to assess the benefits and costs of integrating sexual and reproductive health and HIV services in Kenya and Swaziland. BMC Public Health. 2012;12:973. 17. Abramsky T, Devries K, Kiss L, et al. A community mobilisation intervention to prevent violence against women and reduce HIV/AIDS risk in Kampala, Uganda (the SASA! Study): study protocol for a cluster randomised controlled trial. Trials. 2012;13:96. 18. Andersson N, Cockcroft A, Thabane L, et al. HIV prevention in favour of the choice-disabled in southern Africa: study protocol for a randomised controlled trial. Trials. 2013;14:274. 19. Harding R, Simms V, Penfold S, et al. Multi-centred mixed-methods PEPFAR HIV care & support public health evaluation: study protocol. BMC Public Health. 2010;10:584.
S162
| www.jaids.com
20. Hayes R, Ayles H, Beyers N, et al. HPTN 071 (PopART): rationale and design of a cluster-randomised trial of the population impact of an HIV combination prevention intervention including universal testing and treatment—a study protocol for a cluster randomised trial. Trials. 2014;15:57. 21. Tomlinson M, Doherty T, Jackson D, et al. An effectiveness study of an integrated, community-based package for maternal, newborn, child and HIV care in South Africa: study protocol for a randomized controlled trial. Trials. 2011;12:236. 22. Paintsil E, Andiman WA. Update on successes and challenges regarding mother-to-child transmission of HIV. Curr Opin Pediatr. 2009; 21:94–101. 23. Minister of Health v Treatment Action Campaign. CCT 8/02 (Constitutional Court of South Africa, 2002). 24. Rollins N, Little K, Mzolo S, et al. Surveillance of mother-to-child transmission prevention programmes at immunization clinics: the case for universal screening. AIDS. 2007;21:1341–1347. 25. Doherty T, Chopra M, Nsibande D, et al. Improving the coverage of the PMTCT programme through a participatory quality improvement intervention in South Africa. BMC Public Health. 2009;9:406. 26. Deming WE. Out of the Crisis. Cambridge, MA: MIT, Center for Advanced Educational Services; 1986. 27. Mate KS, Ngidi WH, Reddy J, et al. A case report of evaluating a largescale health systems improvement project in an uncontrolled setting: a quality improvement initiative in KwaZulu-Natal, South Africa. BMJ Qual Saf. 2013;22:891–898. 28. Institution for Healthcare Improvement. The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. Boston, MA: Institute for Healthcare Improvement; 2003. Available at: www.IHI. org. Accessed August 11, 2014. 29. Svoronos T, Mate KS. Evaluating large-scale health programmes at a district level in resource-limited countries. Bull World Health Organ. 2011; 89:831–837. 30. Shojania KG. Conventional evaluations of improvement interventions: more trials or just more tribulations? BMJ Qual Saf. 2013;22: 881–884. 31. South African National Department of Health. Models for the Scale up of HIV Prevention, Treatment and Care From South Africa and Beyond. Pretoria, South Africa: South African National Department of Health; 2010. 32. Greenhalgh T, Robert G, MacFarlane F, et al. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82:581–629.
2014 Lippincott Williams & Wilkins
