Evidence-Based Program Replication: Translational Activities, Experiences, and Challenges.

508888 research-article2013

JAG34510.1177/0733464813508888Journal of Applied GerontologyPrimetica et al.

Translational Science

Evidence-Based Program Replication: Translational Activities, Experiences, and Challenges

Journal of Applied Gerontology 2015, Vol. 34(5) 652–670 © The Author(s) 2013 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/0733464813508888 jag.sagepub.com

Branka Primetica,1 Heather L. Menne,1 Salli Bollin,2 Linda Teri,3 and Marc Molea4

Abstract With a growing number of evidence-based programs, it is necessary to understand the translation activities, experiences, and challenges of program replication in a community setting. This article reviews the implementation tasks necessary for agencies to implement the Reducing Disability in Alzheimer’s Disease (RDAD) intervention. It presents the importance of using original evidence-based program protocols and enhancing them to best fit service settings by reviewing the translation and implementation activities of (a) selecting and training program and supervisory staff; (b) recruiting, screening, and consenting participants to enroll in the program; and (c) developing a manual to guide community-based program replication. Furthermore, the process revealed that the replication of an evidence-based program can take place within the realities of a community setting with input from program oversight, implementation, and evaluation staff and the original researcher. Keywords evidence-based, replication, Alzheimer’s disease, exercise, caregiving Manuscript received: February 6, 2013; final revision received: June 1, 2013; accepted: August 5, 2011 1Benjamin

Rose Institute on Aging, Cleveland, OH, USA Association, Toledo, OH, USA 3University of Washington School of Nursing, Seattle, USA 4Ohio Department of Aging, Columbus, USA 2Alzheimer’s

Corresponding Author: Branka Primetica, Margaret Blenkner Research Institute, Benjamin Rose Institute on Aging, 11890 Fairhill Rd., Cleveland, OH 44120, USA. Email: [email protected]

Downloaded from jag.sagepub.com by guest on November 15, 2015

653

Primetica et al.

Introduction The replication of evidence-based programs has led to an increased need for support and dissemination through community agencies (Bass & Judge, 2010). Evidence-based programs have been deemed efficacious and feasible to implement; therefore, agencies and funding sources consider them ideal for replication as compared with programs that have not undergone rigorous empirical evaluation (Tilly, 2010). However, community agencies need to take into account broader program components such as staffing, marketing, training, and monitoring fidelity during the implementation process, all of which require financial resources. These program components guide the administration of the evidence-based program as well as serve as a roadmap for decision making and development of program administration tools. This article will discuss how the components unfold using the experiences of the state of Ohio’s replication of the Reducing Disability in Alzheimer’s Disease (RDAD) program (Teri, Gibbons, et al., 2003; Teri et al., 2012) that led to serving 630 families. Furthermore, it will describe the implementation and translation activities, which were necessary to offer an evidence-based program in a community setting, while highlighting the experiences and challenges faced by the evaluators, agencies, and interventionists. The terms replication and implementation are used throughout this article to talk about activities that took place from “replicating” an evidence-based intervention to “implementing” it within several Alzheimer’s Association Chapters. According to Fixsen, Naoom, Blase, Friedman, and Wallace (2005), “implementation” is defined as a specific set of activities that are put into practice using sufficient, detailed levels of information. The replication demonstrates how an evidence-based program can be taken from a “lab” or academic setting and, with the appropriate infrastructure, be adapted to a community setting. Understanding the “how” and “why” of translational activities is important to place program outcomes and experiences in the proper context; therefore, this article outlines the experiences and challenges of an evidencebased program and its translation activities in advance of participant outcome analyses and publications.

Background The number of caregivers experiencing burden with caring for family members is growing, especially for those caring for persons with memory problems. In Ohio alone, there are an estimated 230,000 individuals with Alzheimer’s disease and related dementias (Alzheimer’s Association, 2012) and almost 587,000 unpaid caregivers help care for these individuals. It is


654

Journal of Applied Gerontology 34(5)

important for caregivers to care for their own physical and emotional health, considering that they are the foundation keeping the caregiving situation intact. Therefore, providing caregivers with skills to help manage difficult behaviors of persons with dementia (PWDs) is critical to improving the wellbeing of the caregiver and strengthening the care provided to the PWD. At the same time, improving the physical functioning of PWDs may decrease caregiver burden. As research suggests, older adults can increase flexibility, balance, and strength with exercise and subsequently improve physical health, affect, and behavioral distress (Teri, Gibbons, et al., 2003). In addition, it is evident that older adults who engage in regular exercise can ultimately reduce the effects of chronic illness; therefore, evidence-based exercise programs that address chronic illness are becoming increasingly important (Chard & Stuart, 2012). To address the needs of dementia caregiving families in Ohio, two grants were received from the Administration on Aging (AoA) to replicate the RDAD program in community settings. RDAD was developed by Teri and colleagues at the University of Washington and is rooted in the Seattle Protocols (Teri, Gibbons, et al., 2003). The Seattle Protocols involve teaching caregivers to modify their interactions in ways that result in decreased behavior problems of PWDs and improved well-being for caregivers and PWDs (Teri et al., 2012). The RDAD intervention adds emphasis on the importance of making exercise a regular activity for the PWD (Teri, Gibbons, et al., 2003; Teri et al., 2012). The evidence for RDAD was established in a randomized controlled trial (RCT) study; for cognitively impaired RDAD participants, there was a higher level of exercise activity, improved physical activity, and decreased depressive symptoms (Teri, Gibbons, et al., 2003).

Translation and Implementation Activities The RDAD in Ohio Replication, funded by AoA through two Alzheimer’s Disease Supportive Services Program (ADSSP) grants, is a collaborative project between the Ohio Department on Aging, Alzheimer’s Association Chapters serving Ohio, and the Benjamin Rose Institute on Aging in partnership with the original RDAD researcher, Dr. Linda Teri (LT). These key collaborators coordinated and created a plan for replication and evaluation; this allowed project partners to review the original program and revise protocols to best fit a community service agency. Table 1 outlines RDAD program components that were modified to replicate RDAD in the community. As in other evidence-based program replications, “readiness” for implementation required consideration of topics such as “who are our interventionists?” or “who will be our participants?” (Fixsen et al., 2005). In addition to these typical readiness


655

Primetica et al. Table 1. Similarities and Differences in Program Implementation. Original RDAD Interventionists/ Trainers

Social workers Physical therapists

Administrators

Record the recruitment and progress of families Oversee program delivery activities Trainer supervision

Participants

Probable Alzheimer’s disease Ambulatory Community-dwelling Involved caregiver Alzheimer’s disease patient registry referrals Community referrals

Recruitment

RDAD in Ohio Social workers Nurses Related aging/social service fields One at each chapter Record the recruitment and progress of families Oversee program delivery activities Trainer supervision Monthly technical assistance calls Probable Alzheimer’s disease Other dementia Ambulatory Community-dwelling Involved caregiver Chapter referrals Community referrals

Note: RDAD = Reducing Disability in Alzheimer’s Disease.

considerations, this article describes the practical implementation tasks that can be quantified to advance the field of translational science. To date, few evidence-based program replication projects have shared details about the decisions made regarding translation activities. The purpose of this article is to outline the experiences and challenges of those translation activities in advance of outcome analyses that explain participants’ experiences.

Setting To implement an evidence-based program, there are two levels of settings to consider—the implementation agency and the location of program delivery. The identification of an implementation agency was an important step in the replication process. Considerations about the implementation agency include knowing about the community and the agency itself and, most importantly, stakeholder buy-in (Rosenheck, 2001). For the RDAD replication, the Ohio Department on Aging sought to collaborate with the seven Alzheimer’s Association Chapters that serve Ohio as implementation agencies because


656


their existing client population focuses on serving older adults with dementia and their family caregivers. In addition, their staff exhibits the knowledge, skills, and clinical expertise to work with this population and to implement an evidence-based intervention. When approaching individual Chapters, it was also important for the management and staff to recognize their own mission, role in the community, and internal operations to determine compatibility with the implementation (Whitelaw, 2010). Once a clear set of goals and a plan for ongoing communication were put into place, the Chapter/implementation agency became a driving force with creating a successful implementation. The second setting is related to where the program was delivered to families. Just as in the RDAD RCT (Teri et al., 1998), the Ohio replication program sessions took place in the convenience of the family’s home with the family caregiver and PWD. While RDAD was designed and tested as an inhome program, this model was a new method of service delivery for some Chapters. From a translational standpoint, the ability of the project partners to discuss and understand how the settings are or are not suitable is an advancement of the current knowledge about implementing evidence-based programs in the community.

Interventionists Before beginning an evidence-based program, it is important for implementation agencies to give thoughtful consideration to which staff will provide the program (Bass & Judge, 2010). Practitioner selection involves (a) outlining methods for identifying the appropriate staff; (b) detailing which staff characteristics, such as education, experience, willingness to learn, and good judgment, are ideal; and (c) reviewing who is qualified to deliver the program. The interventionists who implemented the RDAD program with families were referred to as “Trainers.” In the Ohio replication, Trainers were clinical staff identified at each Chapter based on their clinical experience and desire to deliver an evidence-based intervention. Most of the 27 Trainers across the State of Ohio were female (89%) and White (96%). The first RDAD Trainers in Ohio had an average of 18 years of experience working with PWDs and family caregivers in community agencies and health care, and possessed backgrounds in the areas of nursing, social work, and counseling. These Trainer characteristics were different from RDAD RCT, which included a physical therapist and a social worker, who both followed a research protocol to work with families on dealing with behavioral issues related to dementia and to teach the exercise component. RDAD Trainers in Ohio learned new skills for delivering the intervention by receiving formal and informal training. The first group of RDAD Trainers


657

Primetica et al.

participated in training and implementation planning, which lasted for one and a half days. This session was led by the original RDAD RCT researcher (LT) to educate the Trainers on the intervention components, and it also involved extensive time planning and discussing the best methods for recruiting, screening, and keeping track of participating families as the program began. Without the need to plan and discuss how families should be recruited, screened, and tracked, future trainings were offered in a structured 1-day session and include an introduction to the program history, education and roleplaying scenarios about the behavior management components, education and practice with the exercises, training on the cognitive and physical assessment of the PWD, and training on recruitment, screening, monitoring, and evaluation activities of the program. As part of ongoing training and supervision, Trainers participated in regular informal supervision with other RDAD program staff, group calls with the original researcher (LT), and an annual 1-day in-person “refresher” training session. In the original RDAD RCT, the training of clinical staff involved an iterative process beginning with the program development, testing by the lead researchers, and then extending to training and supervision of clinicians working in the community (Teri et al., 2012). For the replication project, aside from identifying the Trainers and offering training, the program was designed to have RDAD Trainers evaluate their training activities at the end of the session by providing them with short, anonymous evaluation forms to rate the quality of the training content (e.g., exercise review) as well as space for sharing suggestions about future training sessions. These training evaluations were especially beneficial in obtaining feedback from the staff and helped to design subsequent training activities and materials. The evaluations also helped to quantify the skills learned and remaining needs of participating staff. The training and evaluation activities addressed barriers to implementation experienced by agencies and Trainers. For example, agencies needed to accommodate the intervention practices, such as Trainer time allocation to deliver RDAD and balance existing job responsibilities. Ongoing support was necessary for Trainers to use the behavioral management and exercise techniques required by RDAD. Ultimately, it was a matter of embracing Trainers’ experiences in the area of caregiving and dementia and teaching them new methods to work with these families.

Administrators Offering an evidence-based program in multiple community settings necessitated a “go to” person in each setting who could promote the program among agency staff, oversee the program implementation, and be available


658


for weekly supervision of Trainers on-site. In a fairly large state such as Ohio, it was not possible for the lead collaborators to have their own representatives embedded within each implementation agency. In addition, the program benefited by having a staff member at each agency who could advocate for the program with his or her agency leadership, understand their organizational culture, and invest in the intervention (Bass & Judge, 2010). For the Ohio replication, this liaison between the implementation staff, agency leadership, and program evaluators was called the “Program Administrator.” In the original RDAD RCT, the Program Administrator’s role was filled by a research staff member who had the responsibility of implementing research field operations protocols (e.g., screen and track participants, oversee the program delivery); however, for the Ohio replication, it was important to have a primary Program Administrator at each Chapter (see Table 1 for Program Administrator duties). In the evidence-based program replication, the role of Program Administrator was critical to assure that the program was implemented with fidelity and that staff have an on-site person to turn to with questions. Additional information needs to be gathered to understand this type of role in evidence-based programs. Should a Program Administrator be someone in a supervisory role or someone who is more of a peer with the staff implementing the program? In addition, a Program Administrator was the ideal person to advance a program quality assurance plan that blends the needs of program fidelity with the realities of the agency’s infrastructure and culture. In the Ohio Replication of RDAD, the role of the Program Administrator was especially important because this person could observe any issues that impeded the implementation at the respective agency, report to the evaluation team, and deliver feedback to the Trainers. In addition to supervising and coordinating with other program leaders, Program Administrators tracked the recruitment and progress of families. Creation of a tracking database was an essential translation activity as the database was used to track contact information, program utilization, and completion of evaluation components for all participants. The database was developed using Microsoft ACCESS and included family member names and contact information; dates of enrollment, discharge, completion of sessions, and evaluation components (e.g., assessment, survey); and contact history. All partners were able to access the database via a secure, remote server managed by the evaluation team. With multiple Chapters replicating RDAD, extensive information technology support was needed to assure that the database was secure while also being accessible to the seven Chapters and evaluators at the Benjamin Rose Institute on Aging.


659

Primetica et al.

Participants As a community agency looks to replicate an evidence-based program, consideration needs to be given to who will participate in the program so that the agency can determine whether there is a match between the agency’s typical clientele and the target population for the evidence-based program. Community characteristics must be examined to ensure that there are an adequate number of enrollees to deliver and sustain the program. In addition, evidence-based programs must recognize competition from other programs or services as a factor that may affect enrollment numbers (Bass & Judge, 2010). For the RDAD program, there was a natural fit to have Chapters engage in the program because of their focus on dementia and caregiving. Eligibility criteria for the original RDAD intervention required that PWDs meet the National Institute of Neurological and Communicative Diseases and Stroke and the Alzheimer’s Disease and Related Disorders Association (NINCDSADRDA) criteria for probable Alzheimer’s disease (Teri et al., 1998). This criterion of a “probable Alzheimer’s disease” diagnosis was considered restrictive as Ohio made plans to replicate the RDAD program. There was concern that too few doctors were willing to give the probable Alzheimer’s disease diagnosis and Chapters also did not want to turn away potential participants who had a general diagnosis of “dementia.” After extensive discussion between the original RDAD RCT researcher (LT), the replication implementation leader Salli Bollin (SB), the replication evaluator Heather Menne (HM), and the initial group of RDAD Trainers, it was decided that persons with a dementia or related diagnosis (e.g., Alzheimer’s disease, frontotemporal dementia) would be considered for inclusion into the program. The remaining eligibility criteria from the original RDAD RCT were maintained in the Ohio replication (see Table 1). The decision to broaden the dementia diagnosis criteria allowed a greater number of participants as evidenced by approximately 35% of the participating PWDs had a non-Alzheimer’s dementia diagnosis as of September 2012. Therefore, the ability to track this type of information and quantify the results of translation tasks is imperative as evidence-based programs expand to a myriad of community agencies. In addition, the expanded diagnosis criteria will be explored in future publications on participant outcomes.

Recruitment In addition to knowing the eligibility criteria for program participants, it is important to consider the methods used to promote the program. For the


660


RDAD replication, various recruitment strategies were used, ranging from information presented at staff meetings to community educational programs. Chapter-focused recruitment included referrals from other Chapter programs, Chapter staff, or via postings on the Chapter website and in the newsletter. Additional recruitment conducted by the Chapter included reaching out and sharing promotional materials with other community agencies (e.g., Area Agencies on Aging, physician offices) or through press releases in newspapers and other media. Fliers were also distributed at community events, such as health fairs. The original RDAD program recruited participants from the Alzheimer’s Disease Patient Registry (ADPR) and community referrals (Teri et al., 1998). Due to the creation of the multisite tracking database, it was possible for the project partners to understand how participants were recruited to the program. For example, about 75% of the families who expressed interest in RDAD were referred via Chapter staff, other Chapter services, or Chapter promotional materials (e.g., newsletters, website). Knowing the sources of referrals helps implementation agencies focus their recruitment efforts where they can be most beneficial in replicating an evidence-based program.

Consent and Enrollment Procedures To ensure that the replication of an evidence-based program yields results similar to the original study, specific research processes are needed. For example, to collect data from program participants for analysis and dissemination, it is necessary to have participants meet eligibility criteria and complete an informed consent process where participants understand that their information will be kept private and only shared in the aggregate with information from other participants (Reinhardt, 2010). Prior to beginning the replication of RDAD, all evaluation consent materials were reviewed and approved by the Institutional Review Board (IRB) at the Benjamin Rose Institute on Aging. The RCT of RDAD had approval from University of Washington’s IRB. For the replication, once families expressed interest in the program, they were screened for eligibility. Chapter staff used a screening script to determine whether families were eligible to participate in the RDAD replication (see Table 1). Families who met the eligibility criteria for the replication were scheduled for Session 1, where the caregiver and the PWD signed forms related to evaluation consent and Chapter-specific program participation. A Cognitive and Functional Assessment was conducted with the PWD, which included brief tests of cognitive ability (Katzman et al., 1983), walking speed, standing balance, and functional reach (Teri et al., 1998). This assessment


661

Primetica et al.

was not conducted to exclude participants; however, the Trainer’s observations during the assessment, paired with clinical judgment and program experience, served to reassure that the PWD was cognitively and physically able to participate in the program. As with the recruitment of families, information in the multisite tracking database allowed the project partners to know how many families who expressed interest in the program were eligible to participate and how many declined to participate once they learned what was involved in the program. An often unexplored aspect of programs is the information about which families are not eligible and why they are not eligible. In the case of the RDAD in Ohio replication, most families were not eligible because the PWD was too impaired to participate. This type of information can also assist agencies as they focus their recruitment and marketing resources.

RDAD in Ohio Replication At the onset of replicating an evidence-based program, agencies in community settings need to review the evidence-based programs’ topics, protocols, and tools to assure that they are a proper fit for replication in the new setting (Tomioka, Braun, Compton, & Tanoue, 2012). For the replication of RDAD, no modifications were made to the core program offered to families as deemed efficacious in the original RCT. Sessions followed the same topic and scheduling protocol; and Trainers used the same tools, readings, and handouts. The exercise training component consisting of exercises for (a) strength (e.g., dorsiflexion, knee extension, hip flexors), (b) balance (e.g., transfer exercises, advanced walking skills), (c) flexibility, and (d) endurance activities, such as walking or other aerobic activity, were encouraged. Topics for the behavior problem management piece included (a) maximizing cognitive functioning, (b) using the ABC (Activator–Behavior–Consequence) technique and related card to problem-solve difficulties, (c) reinforcing pleasant activities, and (d) enhancing caregiver resources and skills. The original RCT and replication sessions were 12, 1-hr sessions over 3 months and monthly follow-up over a 3-month period. Participants in the Ohio replication received a 16th session 1 year after starting the program to capture evaluation data.

RDAD in Ohio Replication Manual Teri and colleagues provided their copyrighted Reducing Disability in Alzheimer’s Disease: A Manual for Therapists (Teri, McCurry, et al., 2003) intervention manual, which outlines the session objectives, tasks, and handouts. To assure that the different replication sites followed the same Downloaded from jag.sagepub.com by guest on November 15, 2015

662


procedures, it was necessary to prepare a Replication Manual that (a) articulated additional process and administration topics (e.g., screening), and (b) accompanied Teri and colleagues’ original protocols. It was also important to create the Replication Manual in a format suitable for staff in an agency setting (Whitelaw, 2010). Therefore, the RDAD Replication Manual is designed for RDAD Trainers, who offer the RDAD program to families, and Program Administrators, who administer and manage the program. The format of the manual guides each component of the Ohio Replication by including flowcharts, summary tables, sample handouts, and evaluation materials. The manual was used as a formal training and technical assistance tool during initial training and throughout the RDAD implementation. An additional step in the science of translating evidencebased programs will be to evaluate to what extent program manuals and materials are (a) used by the staff beyond initial training, (b) user-friendly during and after training, and (c) in a format that is most preferred for manuals and materials (e.g., hard copy, electronic).

Evaluation The Reach, Efficacy, Adoption, Implementation, and Maintenance (RE-AIM) model was used as a guide for coordinating implementation and evaluation activities (Glasgow, Vogt, & Boles, 1999). RE-AIM is a commonly used framework for evaluating the replication of evidence-based programs because it includes an organized manner for considering a range of areas relevant to program implementation (Bryant, Altpeter, & Whitelaw, 2006; Glasgow et al., 1999). This model allowed the project partners to understand the broad impact of the program replication on participating families, staff, and agencies. The purpose of this article is to outline the translation steps, experiences, and challenges of replicating an evidence-based program; future papers will focus on the quantified outcomes of the program impact, such as caregiver strain and PWD physical functioning. In preparing to replicate an evidence-based program, community agencies need to decide how program impact will be measured. One way to determine the impact is to use the same data collection methods, measures, and frequency that were used to evaluate the original RDAD RCT. Measures from the original RCT were used in the Ohio Replication, which included physical performance assessments, self-report through caregiver proxy, and adherence measures (Teri et al., 1998). To facilitate the replication to a community setting, slight evaluation changes were needed. The evaluation differences between the original RCT and the Ohio replication include modified data collection methods and alternate measures (Table 2). For example, in the Ohio replication, it was not cost-effective to send an Interviewer and a Trainer to the Downloaded from jag.sagepub.com by guest on November 15, 2015

663

Primetica et al.

Table 2. Similarities and Differences in Evaluation Methods, Measures, and Data Collection Frequency.

Methods Blinded interviewer conducts assessment, CG interview Trainer conducts assessment CG self-completes CG survey Trainer completes adherence measure Trainer completes trainer survey Agency leadership completes agency interview Measures PWD and CG demographic information Mini-mental status examination (Folstein, Folstein, & McHugh, 1975) Short Blessed Test (Katzman et al., 1983) 8-feet walk speed, functional balance ability, functional reach (Teri et al., 1998) Cornell and Hamilton Depression Scales (Alexopoulos, Abrams, Young, & Shamoian, 1988; Hamilton, 1960) CES-D (Radloff, 1977) Sickness Impact Profile Subscales (Bergner et al., 1976) Revised Memory and Behavior Problem Checklist (Teri et al., 1992) SF-36 Subscales (Li, Wang, & Shen, 2003) PWD activity days, falls, and exercise frequency (Teri et al., 1998) CG strain (Bass, Clark, Looman, McCarthy, & Eckert, 2003) CG unmet information needs (Bass et al., 2012) Program satisfaction Data collection frequency (assessments/surveys) Baseline 3 months 6 months 12 months 24 months

Original RDAD

RDAD in Ohio

X

X

X X X X X

X X

X

X

X X

X

X

X X

X

X

X X

X X X X X

X X X X X

X X X X

Note: RDAD = Reducing Disability in Alzheimer’s Disease. PWD = Person with Dementia; CG = Caregiver; CES-D = Center for Epidemiological Studies–Depression Scale; SF-36 = Short Form–36.

participating families; therefore, it was decided that trainers would (a) conduct the assessment of the PWD and (b) give caregivers a self-administered survey to complete on their own time and return to the evaluation team via a business reply envelope. In addition, measures were added to assess the caregiver experiences, and different cognitive impairment and depression measures were used to ease data collection burden (Table 2). Upcoming analyses and articles


664


will focus on strain and well-being outcomes of participating caregivers, the physical health and functional ability outcomes of participating PWDs, and characteristics of which participants were most likely to complete the program. Decisions about the evaluation plan were made collaboratively between the evaluation team, the original researcher, and the implementation staff. By working together, the project partners gained perspective on one another’s evaluation needs and desires to come to a consensus on a plan that would yield practical and informative results with a realistic data collection method. This balance of research and practice helps inform the translation of evidence-based programs and may be an indicator of long-term sustainability of the program in a community agency (Prohaska & Etkin, 2010).

Fidelity Monitoring Fidelity monitoring is a method for performance improvement for any program, service, or task and is most essential for the translation of evidencebased programs (Fixsen et al., 2005). Process measures, such as checklists or rating scales, serve as aides to assure fidelity as well as dosage of the original intervention design (Frank, Coviak, Healy, Belza, & Casado, 2008). For the RDAD replication, fidelity monitoring followed the methods outlined in the original RDAD RCT (i.e., Trainers completed Treatment Compliance paperwork and updated a Program Checklist for content reviewed after each session), and additional mechanisms were developed to facilitate the monitoring across the seven replication Chapters (e.g., paperwork reviewed by Program Administrators). Fidelity monitoring assures that RDAD was consistent with the core components of the program originally tested by Teri and colleagues, and when the monitoring is documented, it allows for added training and support to staff who are implementing the program. The content, order, and mechanisms for delivery were documented throughout the Ohio implementation. For each session, Trainers completed the original RDAD treatment compliance measure to document the tasks completed during each session. Two monitoring mechanisms were added to the Ohio procedures. First, paperwork was monitored on a monthly basis. This was a process consistent with how Chapters monitored their usual programs and it assured that Program Administrators were aware of how Trainers and families were progressing in the program. Second, based on the compliance measure, the evaluation team documented any implementation activities that were not consistent with the protocol (e.g., a Trainer working with a family for 2 hr when sessions are only to last for 1 hr). These two processes added extra “checks” for how the program was being implemented and allowed for correction to Trainers who


665

Primetica et al.

were not following procedures. The fidelity documentation provided the evaluators with evidence to identify which components of the program were executed and which components were not delivered consistently. Understanding the fidelity of program implementation not only supports the implementation staff but also aids in future translation activities that better enhance programs.

Discussion Collaboration and communication are critical for replicating any evidencebased program and are essential in moving agencies beyond being “ready” for actual implementation. These attributes have been demonstrated in the success of enrolling 630 families into the Ohio RDAD Replication between September 2008 and September 2012. Examples of collaboration and communication can be seen across the implementation and translation activities. From the beginning of program implementation, it was necessary to have ongoing communication among the Ohio team members (i.e., Ohio Department on Aging, the Alzheimer’s Association Chapters, and the Benjamin Rose Institute on Aging) as well as regular communication with the original RDAD RCT researcher (LT). This communication led to feedback and assistance with decision making to assure that the intervention was carried out in the manner it was intended and to advance the replication within the Chapters. Early discussions accommodated the program into the Chapters without damaging the integrity of the evidence base. For example, two accommodations to the program implementation included (a) the expansion of the dementia criteria for participant eligibility and (b) the change from using a formal Interviewer to having the Trainer conduct the assessment and the caregiver complete and return a survey. Throughout the process, decisions were made based on input from all the project partners at the state level, the Chapter staff, the evaluation team, and the original RDAD RCT principal investigator. Beyond the initial readiness considerations, practical implementation tasks were completed largely due to group communication and cooperation: (a) creating a system to track participants’ progress, (b) training staff, and (c) monitoring program fidelity. The need to track participants’ progress was complex due to the involvement of multiple Chapters and multiple data elements being tracked. As a resolution to this challenge, the Chapters and evaluation team worked together to develop and maintain the tracking database. The resulting database evolved throughout the program period and was maintained through a secure, virtual system hosted by the evaluators. Tracking systems developed for a specific program are effective mechanisms to


666


manage information and can significantly improve service delivery. The system also played an integral role in tracking fidelity monitoring activities. Staff training involved a joint approach from the original RCT researcher and the Ohio replication team using the Implementation Manual (Teri, McCurry, et al., 2003), which was enhanced with training on Ohio-specific components and the user-friendly Ohio RDAD Replication Manual. Trainers also had access to an exercise consultant when exercise-related concerns arose. Beyond the initial RDAD training, Trainers and Program Administrators participated in additional supervision, statewide calls with the original researcher, and annual in-person “refresher” sessions. Supervision and monitoring of fidelity allowed for enhancing the skills of the staff as well as guiding future improvements related to implementation.

Conclusion Agencies adopting evidence-based programs endure a process that includes (a) readiness, (b) implementation, and (c) sustainability (Gitlin, Earland, Piersol, & Shaw, 2010). A growing literature addresses issues of readiness and implementation (Bass & Judge, 2010; Bryant et al., 2006; Fixsen et al., 2005; Prohaska & Etkin, 2010), but more is needed about the sustainability of evidence-based programs in the community setting. Translational science can help advance this literature and the broader understanding of a program’s trajectory. For the replication of RDAD in Ohio, collaboration and communication were keys to serving 630 families over a period of 4 years. To advance support and further the progress of RDAD, interviews with the RDAD Chapter Leadership identified ongoing factors for long-term sustainability of RDAD: (a) community interest in the program, (b) agency willingness to buy into and maintain the program, and (c) continuous funding streams. As with any program designed to serve populations in need, potential clients or participants need to learn about the program through varied and numerous marketing efforts. Agencies cannot rely solely on program referrals from their current client base; rather, agencies need to carefully articulate a message that will promote the program. Agency leadership needs to maintain the momentum of the program with staff as well as supervise the fidelity to the original program model. Finally, continuous funding to pay for program operations must be identified. This funding could come from fees paid by participants and/or support from outside sources. With input from program oversight, implementation, and evaluation staff as well as the original researcher, the replication of an evidence-based program can take place within the realities of a community setting. The next steps for RDAD in Ohio, as for many evidence-based programs, requires


667

Primetica et al.

further streamlining of procedures, collaboration with more stakeholders, and further replication in other agencies (e.g., senior centers, home health and adult day programs) or delivery models (e.g., group vs. in-home). In addition, dissemination of results will inform how the implementation of this program in community settings impacts PWDs and their family caregivers. These necessary steps will require a focused commitment to achieve the goal of offering a program for the purposes of long-term sustainability. Authors’ Note Institutional Review Board (IRB) for this research was approved by the Benjamin Rose Institute on Aging Institutional Review Board on February 5, 2009. No specific IRB protocol number was assigned to this project. The Benjamin Rose Institute on Aging IRB’s Human Subject Assurance Number is 00002207. Grantees undertaking projects under government sponsorship are encouraged to express freely their findings and conclusions. Points of view or opinions do not, therefore, necessarily represent official Administration on Aging policy.

Declaration of Conflicting Interests The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported, in part by Grants 90AE0329 and 90AE0340, from the U.S. Administration for Community Living, Department of Health and Human Services, Washington, DC.

References Alexopoulos, G. S., Abrams, R. C., Young, R. C., & Shamoian, C. A. (1988). Cornell scale for depression in dementia. Biological Psychiatry, 23, 271-284. Alzheimer’s Association. (2012). 2012: Alzheimer’s disease facts and figures. Washington, DC: Author. Retrieved from http://www.alz.org/downloads/Facts_ Figures_2012.pdf Bass, D. M., Clark, P. A., Looman, W. J., McCarthy, C. A., & Eckert, S. (2003). The Cleveland Alzheimer’s managed care demonstration: Outcomes after 12 months of implementation. The Gerontologist, 43, 73-85. Bass, D. M., & Judge, K. S. (2010). Challenges implementing evidence-based programs. Generations, 34, 51-58. Bass, D. M., Judge, K. S., Snow, L., Wilson, N., Looman, W., McCarthy, K., . . . Kunik, M. (2012). Prevalence and predictors of depression, care-related strain, and unmet needs among caregivers of patients with dementia. The American Journal of Geriatric Psychiatry, 20, 239-247.


668


Bergner, M., Bobbitt, R. A., Kressel, S., Pollard, W. E., Gilson, B. S., & Morris, J. R. (1976). The Sickness Impact Profile: Conceptual formulation and methodology for the development of a health-status measure. International Journal of Health Service, 6, 393-415. Bryant, L. L., Altpeter, M., & Whitelaw, N. A. (2006). Evaluation of health-promotion programs for older adults: An introduction. Journal of Applied Gerontology, 25, 197-213. Chard, S. E., & Stuart, M. (2012). The ecological perspective on the community translation of exercise research for older adults. Journal of Applied Gerontology, 31, 28-51. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-Mental State: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189-198. Frank, J. C., Coviak, C. P., Healy, T. C., Belza, B., & Casado, B. L. (2008). Addressing fidelity in evidence-based health-promotion programs for older adults. Journal of Applied Gerontology, 27, 4-33. Gitlin, L. N., Earland, T. V., Piersol, C. V., & Shaw, G. (2010). Enhancing quality of life in functionally vulnerable older adults: From randomized trial to standard care. Generations, 34, 84-87. Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public-health impact of health-promotion interventions: The RE-AIM framework. American Journal of Public Health, 89, 1322-1327. Hamilton, M. (1960). A rating scale for depression. Journal of Neurology, Neurosurgery, and Psychiatry, 23, 56-62. Katzman, R., Brown, T., Fukd, P., Peck, A., Schechter, R., & Shimmel, H. (1983). Validation of a short orientation-memory-concentration test of cognitive impairment. American Journal Psychiatry, 140, 734-739. Li, L., Wang, H. M., & Shen, Y. (2003). Chinese SF-36 Health Survey: Translation, cultural adaptation, validation, and normalization. Journal of Epidemiological Community Health, 57, 259-263. Prohaska, T. R., & Etkin, C. D. (2010). External validity and translation from research to implementation. Generations, 34, 59-65. Radloff, L. S. (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385-401. Reinhardt, J. P. (2010). Research methods in evidence-based practice: Understanding the evidence. Generations, 34, 36-42. Rosenheck, R. A. (2001). Organizational process: A missing link between research and practice. Psychiatric Services, 52, 1607-1612. Teri, L., Gibbons, L. E., McCurry, S. M., Logsdon, R. G., Buchner, D. M., Barlow, W. E., . . .Larson, E. B. (2003). Exercise plus behavioral management in patients


669

Primetica et al.

with Alzheimer disease: A randomized controlled trial. Journal of the American Medical Association, 290, 2015-2022. Teri, L., McCurry, S. M., Buchner, D. M., Logsdon, R. G., LaCroix, A. Z., Kukull, W. A., . . .Larson, E. B. (1998). Exercise and activity level in Alzheimer’s disease: A potential treatment focus. Journal of Rehabilitation Research and Development, 35, 411-419. Teri, L., McCurry, S. M., Logsdon, R. G., Moore, A., Tyll, L., Ebel, S., . . .Jordan, J. (2003). Reducing disability in Alzheimer’s: A manual for therapists. Seattle: University of Washington. Teri, L., McKenzie, G., Logsdon, R. G., McCurry, S. M., Bollin, S., Mead, J., & Menne, H. (2012). Translation of two evidence-based programs for training families to improve care of persons with dementia. The Gerontologist, 52, 452-459. doi:10.1093/geront/gnr132 Teri, L., Truax, P., Logsdon, R., Uomoto, J., Zarit, S., & Vitaliano, P. P. (1992). Assessment of behavioral problems in dementia: The Revised Memory and Behavior Problems Checklist (RMBPC). Psychology and Aging, 7, 622-631. Tilly, J. (2010). The Administration on Aging’s experiences with health, prevention, and wellness. Generations, 34, 20-25. Tomioka, M., Braun, K. L., Compton, M., & Tanoue, L. (2012). Adapting Stanford’s chronic disease self-management program to Hawaii’s multicultural population. The Gerontologist, 52, 121-132. Whitelaw, N. (2010). System change and organizational capacity for evidence-based practices: Lessons from the field. Generations, 34, 43-50.

Author Biographies Branka Primetica, MSW, is a senior research analyst at the Margaret Blenkner Research Institute of the Benjamin Rose Institute on Aging. Her background includes conducting research and working with family caregivers and at-risk older adults, including those with behavioral and cognitive health impairments. She is a project manager for several initiatives that aim to improve care and services as well as the psychosocial well-being of older adults and their caregivers. Heather L. Menne, PhD, is a senior research scientist at the Margaret Blenkner Research Institute of the Benjamin Rose Institute on Aging. Her research explores the theoretical underpinnings of the stress process for individuals who have cognitive impairment, family caregivers, and formal caregivers. Her recent work includes serving as the Evaluation Leader of the evidence-based Reducing Disability in Alzheimer’s Disease program in Ohio. Salli Bollin, MSW, is the executive director of the Alzheimer’s Association, Northwest Ohio Chapter. She is responsible for strategic planning, program development, and public policy activities at local, state, and federal levels. She serves as the Implementation Leader for the statewide replication of the Reducing Disability in Alzheimer’s Disease program, overseeing program activities, including training staff and coordinating leaders at the Alzheimer’s Association Chapters.


670


Linda Teri, PhD, is a professor in the Department of Psychosocial & Community Health and adjunct professor in the Departments of Psychology and Psychiatry & Behavioral Science at the University of Washington. She is the director of the Northwest Roybal Center (NRC): Improving Healthcare for Cognitively Impaired Elders and Their Caregivers and the Northwest Research Group on Aging. Her research focuses on (a) the ongoing development, implementation, and evaluation of treatment programs for Alzheimer’s disease patients and their caregivers; (b) investigating the relationship between cognitive, affective, and behavioral function as it relates to disease progression and treatment; and (c) developing and evaluating training programs to increase independence and improve physical functioning in older adults. Marc Molea, AICP, MHA, is the chief of Strategic Partnerships at the Ohio Department of Aging. He supports the development of a comprehensive, integrated, and coordinated system of elder-friendly, person-centered programs and services. He identifies new and innovative partnerships, explores opportunities to promote policy and systems change, and engages in research and dissemination of best practices and innovations to meet the needs of older adults.


Challenges of Translational Science.

Omics challenges and unmet translational needs.

Translational strategies and challenges in regenerative medicine.

High-throughput translational medicine: challenges and solutions.

The challenges of replication.

Program synthesis: challenges and opportunities.

Upcoming translational challenges for uveal melanoma.

Upcoming translational challenges for uveal melanoma.

Biliary tract cancer stem cells - translational options and challenges.

Translational Research for Tuberculosis Elimination: Priorities, Challenges, and Actions.

Translational and regulatory challenges for exon skipping therapies.

Cell-based therapy technology classifications and translational challenges.

The perfect storm: challenges and opportunities for translational medicine.

Early intervention program (EIP)--the Indian experiences.

The experiences of everyday activities post-stroke.

2008-2011 sylvatic rabies epidemic in Italy: challenges and experiences.

ORChestrating the human DNA replication program.

Consolidating exposure scenario information for mixtures--experiences and challenges.

Introduction: Young Fatherhood: Lived experiences and policy challenges.

Maternal immunization. Clinical experiences, challenges, and opportunities in vaccine acceptance.

Bangladesh Midlife Women's Health Study (BMWHS): methods, challenges and experiences.

Global reprogramming of the cellular translational landscape facilitates cytomegalovirus replication.

Challenges to Implementation of Global Translational Collaboration Platforms.

Gene therapy for GM1 gangliosidosis: challenges of translational medicine.