This article was downloaded by: [University of Newcastle, Australia] On: 30 December 2014, At: 23:58 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of Evidence-Based Social Work Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/webs20
Improving Impulse Control: Using an Evidence-Based Practice Approach a
Kendra J. Garrett & K. Giddings
b
a
School of Social Work, St. Catherine University and University of St. Thomas , St. Paul , Minnesota , USA b
Licensed Clinical Social Worker , Minnesota , USA Published online: 09 Jan 2014.
Click for updates To cite this article: Kendra J. Garrett & K. Giddings (2014) Improving Impulse Control: Using an Evidence-Based Practice Approach, Journal of Evidence-Based Social Work, 11:1-2, 73-83, DOI: 10.1080/15433714.2013.840197 To link to this article: http://dx.doi.org/10.1080/15433714.2013.840197
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Journal of Evidence-Based Social Work, 11:73–83, 2014 Copyright © Taylor & Francis Group, LLC ISSN: 1543-3714 print/1543-3722 online DOI: 10.1080/15433714.2013.840197
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
Improving Impulse Control: Using an Evidence-Based Practice Approach Kendra J. Garrett School of Social Work, St. Catherine University and University of St. Thomas, St. Paul, Minnesota, USA
K. Giddings Licensed Clinical Social Worker, Minnesota, USA
School social workers in a suburban school district implemented a two-year collaboration to identify empirically effective interventions and apply them to their own practice. During the first year of the project, the workers and the consultant discussed the use of evidence-based practice and developed strategies and tools for monitoring and evaluating practice. In the second year, each worker monitored the progress of one student using a standardized intervention. Five social workers completed the project; collected data on outcomes showed improved self-control for all participants. Limits on measurement and adaptations to individualize interventions make it impossible to be certain that the intervention was the primary cause of these outcomes. After the project the workers were not yet completely comfortable with the use of evidence-based practice to monitor and evaluate their practice. Recommendations for improving implementation of new evidence-based practice programs include providing ongoing discussion and feedback to workers, encouraging flexibility in implementation of curricula, simplifying record keeping so data reporting is consistent, and increasing baseline and follow-up measurements to strengthen the research. Keywords: Evidence-based practice, outcome evaluation, impulse control, case study
As schools have been mandated to use teaching methods that are empirically effective, school social workers have also been encouraged to move toward the use of evidence-based interventions in their work with students (Franklin & Hopson, 2004; National Association of Social Workers [NASW], 1999, 2002; Raines, 2008). Challenges to adopting an evidence-based approach include concerns that evidence-based practice (EBP) takes too much time, that there is too little research available to guide practice adequately, and that the use of EBP is formulaic and prevents workers from using creativity and autonomy (Gibbs & Gambrill, 2002; Schaeffer et al., 2005). Yet there are many good reasons to move toward using evidence in school social work practice because it is ethically, legally, and clinically appropriate (Raines, 2008). This article will describe the process one school social work department used to begin to integrate an evidence-based approach into their work with students. Address correspondence to Kendra J. Garrett, School of Social Work, University of St. Thomas, SCB 201, 2115 Summit Ave., St. Paul, MN 55105, USA. E-mail: [email protected]
73
74
K. GARRETT AND K. GIDDINGS
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
EBP involves using the best research available to answer practice questions by using methods shown empirically to be effective, applying critical thinking in making practice decisions, monitoring interventions to be sure they are implemented as intended, and tracking outcomes (Roberts & Yeager, 2004). Far from being a “cookbook approach,” social workers must be able to identify good research, evaluate it, and determine the extent to which the research would be appropriate in a particular setting with the client in question. The social worker must also have the relationship skills to discuss the intervention with the client to determine if the proposed treatment meets the client’s wishes. This process may involve some alteration of the intervention to better fit the specific situation (Hamilton, Kendall, Gosch, Furr, & Sood, 2008; Raines, 2008).
THE PROCESS The school social workers in a suburban school district decided, with the support of the special education director, to increase their knowledge of EBP in hopes of improving the services they provided to students. They took Franklin’s (1999) advice to partner with an academic institution for this process and began meeting with the senior author with the goals of becoming more comfortable applying research to their practice decisions and improving their ability to monitor and evaluate their practice. During the first year of the project, they discussed the EBP process and developed strategies and tools for monitoring and evaluating practice. They decided, as Schaeffer et al. (2005) suggested, to begin by collectively choosing one intervention that all would use over the second year of the project. The social workers settled on impulsivity as the student problem they would focus on for the year. Impulse control is of concern because of its association with academic success (Buckner, Mezzacappa, & Beardslee, 2009; Ponitz & McClelland, 2009; Shoda, Mischel, & Peake, 1990), aggression (Mischel, Shoda, & Rodrigues, 1989), social competence (Buckner et al., 2009), and adolescent substance abuse (Wills, Ainette, Stoolmiller, Gibbons, & Shinar, 2008). The search for interventions that were empirically validated indicated that impulsivity is best treated with cognitive-behavioral therapy (CBT), to include a variety of techniques, such as operant reinforcement, self-instruction, problem solving, modeling, role-playing, rehearsal, relaxation training, imagery, and group discussion (Baer & Nietzel, 1991; Fink & McCown, 1993; Kendall & Braswell, 1981; Posavac, Sheridan, & Posavac, 1999). A search for CBT programs that could be used to treat impulsivity yielded a number of possible programs and curricula aimed at improving impulse control in students. The social workers chose Hunter and His Amazing Remote Control (Copeland, 1998). Hunter has a series of lessons with accompanying activities that use the metaphor of a television remote control to help students regulate behavior. Students can pause, change channels (tune in after becoming distracted), fast forward (to anticipate consequences), rewind (to consider alternate options when behavior goes awry), use slow motion (to talk about emotions), coach (to use self-talk), zap (to stop negative thinking), and “way to go” (to self-affirm). Hunter was designed for elementary students, and the social workers anticipated that they would need to make adaptations for some of their older students. The workers agreed to apply the curriculum to one student each during the 2008–2009 academic year.
Measurement All social workers used the teacher-rated Self-Control Rating Scale (SCRS) developed by Kendall and Wilcox (Fischer & Corcoran, 2007) as a pre- and post-test measure of impulse control. This 33-item scale, completed by teachers, asks questions about the student’s ability to follow
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
IMPROVING IMPULSE CONTROL
75
through, self-calm, work for long-range goals, and other elements associated with the self-control construct. Teacher responses are summed to obtain the rating score. Higher scores denote greater impulsivity. Some social workers supplemented this measure with parent ratings of the SCRS. Some also used a 6-point individualized rating scale in each session with the student to assess the student’s ongoing progress on specific goals. This scale indicated the goal, and asked students to rate themselves on a scale ranging from 1 D “if things got worse” to 6 D “perfection.” The social workers also had the option of using the Children’s Perceived Self-Control Rating Scale developed by Humphrey (Fischer & Corcoran, 2007) as a supplementary pre- and post-test. This 11-item scale asks students to rate each question with “usually yes” or “usually no.” The measure contains questions about distractions, patience, anticipating consequences, and other items involved in perceived self-control. All but items 1 and 8 are reverse scored on this measure, and a low score is desirable. The school district’s Director of Teaching and the University of St. Thomas Institutional Review Board approved the project. Students could receive services, including the Hunter intervention, whether or not parents chose to allow their child/children to participate in the research portion of the project. Parent permission was required for the social workers to share the results anonymously with the university consultant. The district has 14 social workers. Of these, two do not have direct contact with students. Two had students who agreed to participate but moved away during the school year. Half of the remaining social workers implemented the program with students whose parents consented to share their data for this report. The social workers reported several barriers to their involvement. Some of the challenges included philosophical objections to “proving” their effectiveness with students, difficulty getting the requisite paperwork back from parents, the extra time and effort required, concerns about losing their personal style in the use of a proscribed program, changes in teachers between pre- and post-test, and concerns that many students did not fit the program (because of complex needs, age, or cognitive deficits). In this school district, social workers are considered related-service providers in the special education department. They often have responsibilities for services on students’ individual education plans (IEPs), and some of them view this IEP goal writing and tracking as meeting part of their EBP and outcome-measurement responsibilities.
CASE SUMMARIES Five social workers participated in this project by gathering pre- and post-test data on self-control with the SCRS as they used the Hunter curriculum to address impulsivity with one student. Each had a different experience with the Hunter curriculum.
Student A The school social worker worked with students aged 18–21 years, in a program for young adults with disabilities who are transitioning to independent living. These students are all special education students with IEPs. Their curriculum focuses on work, recreation, community membership, independent living, and post-secondary options. The school social worker had to modify the Hunter curriculum from its primary-grade audience to be appropriate for young adult participants. The social worker removed any reference to the Hunter story or the remote control while staying true to all cognitive-behavioral therapy concepts. Social Worker A was able to use some of the activities in the Hunter curriculum without modification. This social worker also had interest and expertise in yoga and supplemented the Hunter sessions with yoga and mindfulness practice,
76
K. GARRETT AND K. GIDDINGS
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
especially during the “slow-motion” lessons. The SCRS scale required modification of the wording to make it age-appropriate. The social worker administered the modified curriculum in a group of four students. The participating student was a legal adult and chose to self-answer the pre-test and post-test questions rather than asking the teacher to complete the rating. The student’s SCRS rating was 90 at pre-test and was 78 at post-test. Because of the two simultaneous interventions, it is not possible to know if the modified Hunter curriculum, the yoga, or the combination of the two interventions created the improvement in the SCRS scores. The reliability of the SCRS may also have been compromised by the fact that the student self-rated this instrument (rather than a teacher rating). Student B The school social worker worked with junior-high–aged students in grades 7–9 in an alternative learning center. The worker provided the Hunter curriculum to a group of 5 students over 15 sessions. An 8th grade student was monitored. This school social worker believed that the curriculum was age appropriate, even though the curriculum was written for an elementary-aged audience. For this student participant, two teachers completed pre- and post-test ratings on the SCRS. One teacher’s rating fell from 169 to 124, showing teacher-perceived growth. The other’s ratings increased from 112 to 127 (showing a perception that the student’s behavior deteriorated over the school year). Social Worker B attributed this inconsistency to a disadvantage in using numerical rating scales to measure progress, because they are “relative to a student’s behavior in the recent past.” This worker believed that “the teacher’s expectations for the student’s behavior changed”; the teacher expected better behavior by the end of the school year. Social Worker B’s observation of the student was that the student’s behavior improved. Over the course of the school year this student was placed on medication and the parents received education and counseling about medications. Social Worker B attributed changes in the student’s behavior to a combination of the Hunter curriculum, medication, and parent intervention. Student C The school social worker in an elementary-school setting presented the curriculum to a mainstream 4th-grade classroom over 8 sessions. The student participant was a special education student being served under an autism spectrum disorder (ASD) label, who also presented with severe deficits in attention and working memory, high levels of distractibility, and slow information processing. The social worker chose to follow this student because the parents were willing to allow the research and complete the rating scales, and the teacher was supportive of the school social worker’s presenting the Hunter curriculum in her classroom, thus allowing teacher reinforcement. The social worker also saw this student individually to focus on additional social-emotional goals from the student’s IEP. The student responded well to Hunter and was attentive. The social worker did not make any major adaptations to the Hunter curriculum in this classroom setting. In addition to the teacher rating of the SCRS, both parents also completed the measure. Progress with Student C was encouraging; in addition to the teacher’s perceived growth, the mother perceived great growth, and the father perceived a slight improvement (see Table 1). Student D The school social worker in a high school worked individually with a 16-year-old student who was transitioning from special education to mainstream classes. The student saw the social worker to address lingering impulse-control issues. The social worker presented the Hunter curriculum
IMPROVING IMPULSE CONTROL
77
TABLE 1 Self-Control Rating Scale for Student C
Teacher Rating Mother’s Rating Father’s Rating
Pre-Test
Post-Test
102 137 128
83 96 124
Change 19 41 4
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
Note. Low score is desirable.
over seven individual sessions. Because the student was older than the elementary-age students for whom Hunter was designed, major adaptations to the curriculum were necessary. Social Worker D used the remote control concept and activities but presented them orally as discussion points, rather than using the student worksheets. The term changed between the beginning and end of this intervention. Because classes changed, different teachers rated the student on post-test than on pre-test. The worker was, therefore, extremely cautious about the large drop in score from 161 to 60 (indicating strong improvement). The social worker noted the student continued to skip classes and arrive late to school. Each week, Student D completed a self-rating (“focusing on work”), reviewed the previous week’s progress, and learned a new skill. Student D also completed the Perceived Self-Control Scale; both the pre- and post-test ratings were a 5, indicating no change in the student’s perception of self-control. The weekly rating scale (see Figure 1), however, indicates the student perceived weekly improvement in focusing on work over the course of treatment.
Student E The social worker in an elementary setting presented the Hunter curriculum individually to a 2ndgrade student with ASD over a 6-month period. The social worker had to adapt the curriculum by breaking it into smaller steps. Recognizing “getting stuck in rigid thinking” as an important issue for this student, the social worker interpreted, personalized, and emphasized that idea as part of the “rewind” button (to help the student’s mind get “unstuck”). Teacher ratings of self-control improved from 133 to 119. The student rated the Children’s Perceived Self-Control Scale. This
FIGURE 1 Student D self-rating of focusing on work (rating: 1 is low; 6 is high).
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
78
K. GARRETT AND K. GIDDINGS
FIGURE 2
Student E self-rating of staying focused on work (rating: 1 is low; 6 is high).
rating improved from an 8 on pre-test to a 4 on post-test. Weekly ratings of the student’s goal of staying focused on work showed the student perceived steady improvement (see Figure 2).
OUTCOMES All social workers who participated used the SCRS as a pre-test and post-test of the changes in their students’ impulse control, and most showed gains (see Table 2). There is some evidence of progress for all students, but the social workers were cautious about drawing conclusions from the results, as there were challenges in the measurement process. One social worker changed the wording of the questions to be more appropriate for an adult student. Teachers had changed for another student, so a different teacher rated the post-test than the pre-test. Still another had mixed results on measures from two different teachers. There was wide variety in the way the program was administered as well, making it difficult to isolate the Hunter curriculum as the cause of any changes. All students received supplements to the Hunter curriculum, including additional lessons, use of yoga for calming, and starting medication over the course of the intervention. The program was administered over different lengths of time, and different modalities (individually, in small groups, or in a classroom setting). The workers individualized the intervention to meet student needs. TABLE 2 Teacher Ratings of Self-Control Rating Scale Student Aa B Teacher 1 B Teacher 2 C Db E
Teacher Pre-Test
Teacher Post-Test
Change
90 112 169 102 161 131
78 127 124 83 60 119
12 C15 45 19 101 12
Note. Low score is desirable. a Self-rating done by student. b Post-test rated by different teacher than pre-test.
IMPROVING IMPULSE CONTROL
79
LESSONS LEARNED The school social workers who participated in this 2-year project and the consultant learned valuable lessons about how to move toward the use of EBP in the district. Some of the lessons became more clear when the workers were surveyed about their experience at the end of the second year of the program.
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
Motivation One lesson learned is the effect of motivation on the process. The social workers who participated did so in part because they perceived pressure from the district (4 responses) and/or the supervisor (1 response) to show outcomes of their practice. They indicated also that they were internally motivated (4 responses) to see results of their practice and were interested in the project (4 responses). School social workers are feeling the pressures of No Child Left Behind (Raines, 2008) to show that their services are producing positive outcomes that help students be successful. Making changes entails some risk, and Panzano and Roth (2006) found that agencies that were quicker to adopt new or innovative ideas (including EBP) were those that believed the risks involved were manageable. This implies that those who supervise school social work programs have an important role to provide support and encouragement to departments as they take on new practice evaluation activities, providing assurance, for example, that a poor student outcome would not be viewed as a failure or be used to sanction the worker. An administration that supports learning about evidence-based programs and monitoring student outcomes can greatly enhance social workers’ willingness to participate in more formal evaluation. Administrators can provide additional support by encouraging adoption of EBP and providing time and money for consultation, record keeping, and identification of successful programs. Feedback and Discussion The school social workers were cautiously pleased with their progress towards EBP. Those who participated rated the experience “neutral” (2 responses), “positive” (2 responses), and “very positive” (1 response). In written comments, however, they indicated that they were not certain about the usefulness of the project and wanted more feedback about how to use data they had generated. A lesson learned is to develop more opportunities to discuss the outcomes and to consider together how the information can be used to enhance practice. So in implementing EBP programs, social workers should include discussions on interpretation of data, student improvements, revising treatment plans based on the data, and explanations of the findings to students and parents. Flexibility The tension between following a manual closely and making changes as appropriate for a client can often be resolved by allowing a certain amount of flexibility within a treatment program (Hamilton et al., 2008; Raines, 2008). Workers must attend to the therapeutic alliance with their clients, involving them in the treatment process, and altering programs to meet their particular needs (e.g., by adding relevant examples; Hamilton et al., 2008) or to meet the cultural, cognitive, or developmental needs of a student (Raines, 2008). In this case, the social workers slowed lessons for some students, changed worksheets to discussions, and supplemented the curriculum with yoga. Adaptation should remain true to the basic principles of the intervention and meet the goals of the individual sessions identified in treatment manuals (Hamilton et al., 2008). While sessions
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
80
K. GARRETT AND K. GIDDINGS
may be supplemented, they should not be abridged; the length of an intervention should not be shortened (Raines, 2008). Discussions about the changes made over the course of the intervention demonstrated that the workers were using CBT principles in making adaptations and were slowing the curriculum down and/or supplementing to make certain that the children were understanding the concepts. While the social workers adapted the curriculum to meet individual needs, it was also helpful that all of the social workers used a similar curriculum and the same basic measurements for their students. This allowed for group consultation and an opportunity for the social workers to compare notes and discuss strategies for adapting the curriculum. Having one measurement for all (the SCRS completed by the classroom teacher) allowed the social workers to compile data. If more students were involved social workers could use statistical measures to compare students by disability, gender, race, age, diagnosis, or other characteristics, perhaps further adding to the body of knowledge regarding treatment of students with impulse-control disorders. A major disadvantage of the use of consistent curriculum and measurement was that two of the social workers who did not participate indicated that the reason they chose not to participate was that the curriculum did not fit the students in their case loads. Others reported great difficulty measuring behavior of their unique students. Record Keeping The social workers in this district are using many different strategies for record keeping. It would have been helpful, as part of the planning of the project, if more attention had been given to standardizing the record keeping and the way workers recorded data. Templates that made record keeping easier for the workers could have simplified documentation, both on an ongoing basis and for summary reporting. For example, each student participant could have a coversheet that included space for demographic information (age, grade, gender, and disability status), intervention method (individual, group, and classroom), supplementary services provided (yoga, parent support, teacher consult, and classroom interventions), pre- and post-test ratings, and weekly notes, including a place for self- and teacher-ratings of progress. Once good record-keeping formats have been developed, it would be helpful to consult with the information technology department and develop a record-keeping strategy that would allow social workers to enter progress notes into an electronic record linked to an annual report, as Kossman, Lamb, O’Brien, Predmor, and Prescher (2005) suggest. This would have the advantage of saving time and simplifying record keeping. Reports could then be used to share information regarding social work services provided as well as outcomes of those services. Research The purpose of this project was to strengthen the use of evidence-based practice in one school social work department. The emphasis was on monitoring and evaluating practice outcomes in a real-world setting rather than on conducting research. Nevertheless, greater rigor in design would have enhanced monitoring of practice outcomes. This project used a pre-experimental design in which a pre-test was conducted before the intervention and a post-test after. This does not allow the workers to know for sure if gains were the result of the intervention or if other causes (e.g., changes in parenting practices or medications) contributed to the outcomes. This could have been improved through the use of more baseline measures for each student before beginning the intervention to see if their impulse-control scores were stable. A stable baseline would have given more assurance that subsequent gains were the result of the intervention (Monette, Sullivan & DeJong, 2008). Asking busy teachers to complete multiple baseline measures, however, may be problematic, so shorter measures might need to be developed.
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
IMPROVING IMPULSE CONTROL
81
The use of a control group that did not receive the intervention as a comparison would have lent even more strength to the project (Monette, Sullivan & DeJong, 2008). Randomized controlled trials have limitations in a school setting, and there are ethical concerns about withholding treatment. DeAnda (2007) suggests the use of quasi-experimental designs for groups of children that fit into the school calendar. A number of students could be pre-tested on the measure, and then groups receive services on a schedule that fits with the school calendar. When the first group of students has finished receiving services, students are again tested on the measure. The group that does not receive the intervention serves as a control/comparison group for the first group. Then the second group of students receives the intervention and serves as control/comparison for the third group, and so on. Over the course of a year several groups of students may receive the intervention and serve as comparisons for other groups. This method, however, may be better suited for classroom-wide interventions than for individual interventions. Follow-Up Compilation of the findings show that the student participants made positive gains, although the social workers expressed doubt about the measurement process and the permanence of the gains. This suggests that a 6-month follow-up would provide more information about the long-term benefits of the interventions to improve impulse control during the next academic year. Change Takes Time One year of instituting EBP in a school setting is a good start. Change to a new way of thinking about practice does not happen quickly. The feedback questionnaires indicated only a slight change in plans to use practice-monitoring strategies in the future. Schaeffer et al. (2005) recommend a grass-roots, democratic approach to choosing curriculum for EBP projects and voluntary participation, at least at first. All of the social workers have not yet determined that EBP fits the needs of their practice. While the social workers who participated felt that they learned valuable lessons, they did not feel confident that they could implement an EBP project independently.
CONCLUSION Taking the first step in using EBP in a school social work setting illustrates some of the challenges school social workers face in using evidence-based interventions. Students are complicated, and issues arose around the appropriateness of the curriculum, given the age range of students (elementary-aged to 21 years), their unique characteristics, different needs, and IEP requirements for providing services in individual, group or classroom settings. In some cases the curriculum needed considerable modification to meet the needs of the students. The curriculum was presented in formats ranging from individual to classroom. In one case, the student participated in a classroom session on the Hunter curriculum and received supplementary individual sessions that focused on additional goals. In another, the social worker used a combination of a modified Hunter curriculum and yoga. Given the adaptations that took place to individualize Hunter to meet the needs of the students in this project, it is difficult to sort out what led to gains in impulse control. Measurements also needed some adaptation for some of the students, causing concerns about reliability and validity. In one case, the teacher who rated the pre-test was no longer the student’s teacher at time of post-test. In another, the social worker modified the measurement to fit with
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
82
K. GARRETT AND K. GIDDINGS
the needs of an adult student and the student self-rated. It is far from certain that the gains noted in the changed ratings were actual changes in behavior. It is even less certain if the gains will remain over time. Use of additional measures might confirm that gains show up in more than one way. And these measurements need to be repeated after some months to see if gains are stable. The authors anticipated that implementation of the Hunter curriculum would provide group data that could be aggregated. Instead, because of individualization of the curriculum and different measurement strategies, the outcomes are better considered as a series of case studies. Nevertheless, the program showed considerable promise, as all self-control ratings but one indicated growth over the time that the student used the Hunter curriculum. The purpose of this project was not, strictly speaking, to evaluate the Hunter curriculum. What these social workers most hoped to accomplish was becoming more comfortable with the use of scientific research to inform their practice decisions and improving their ability to monitor and evaluate their practice. These goals were partially met. These school social workers also hoped to help students improve their impulse-control behavior at school, and data showed some progress for these students. An additional goal of the project was to move toward a more evidence-based approach to school social work practice. This goal was also partially met. The workers discovered that use of EBP strategies allows for a flexible, individualized approach to practice while increasing their confidence that they were choosing methods known to be empirically effective. They were less certain of their ability to apply data they gathered to modify their interventions as they progressed in their work with students. Perhaps implementation of EBP programs will require ongoing group support and feedback until social workers become more comfortable using data to monitor their individual modifications to the evidence-based curricula they have chosen. Most of the participating social workers expressed pleasure in using a CBT-based curriculum that had empirical backing. They were sometimes frustrated at the limits of the particular curriculum meant for elementary school students to a more diverse population. This is, perhaps, one of the challenges to the profession in implementing empirically based treatments. As social workers individualize, they will need to balance fidelity to the original protocols with changes needed for their unique clients. As social workers move toward the use of EBP, more guidelines on making these kinds of modifications are needed. As social work moves to overcome barriers to the use of EBP, the profession must consider the role of education. It may be that social work students are not gaining sufficient competence in analysis and application of empirical practice data. As the profession moves toward EBP, perhaps it should rethink the way evaluation is being taught in both practice and research courses to assure that future social workers have a comprehensive understanding of how to use data to modify and improve practice. And professional organizations should reach out to those who are already in practice to offer continuing education in the use of EBP.
REFERENCES Baer, R. A., & Nietzel, M. T. (1991). Cognitive and behavioral treatment of impulsivity in children: A meta-analytic review of the outcome literature. Journal of Clinical Child Psychology, 20, 400–412. Buckner, J. C., Mezzacappa, E., & Beardslee, W. R. (2009). Self-regulation and its relations to adaptive functioning in low-income youths. American Journal of Orthopsychiatry, 79, 19–30. Copeland, L. (1998). Hunter and his amazing remote control. Chapin, SC: Youthlight. deAnda, D. (2007). Intervention research and program evaluation in the school setting: Issues and alternative research and alternative research design. Children & Schools, 29, 87–94. Fink, A. D., & McCown, W. G. (1993). Impulsivity in children and adolescents: Measurement, causes, and treatment. In W. G. McCown, J. L. Johnson, & M. B. Sure (Eds.), The impulsive client: Theory, research, and treatment (pp. 279–308). Washington DC: American Psychological Association.
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
IMPROVING IMPULSE CONTROL
83
Fischer, J., & Corcoran, K. (2007). Measures for clinical practice and research (4th ed.) New York, NY: Oxford University Press. Franklin, C. G. (1999). Research on practice: Better than you think? Social Work in Education, 21, 3–9. Franklin, C., & Hopson, L. (2004). Into the schools with evidence-based practice, Children & Schools, 26(2), 67–70. Gibbs, L., & Gambrill, E. (2002). Evidence-based practice: Counterarguments to objections. Research on Social Work Practice, 12, 452–476. Hamilton, J. D., Kendall, P. C., Gosch, E., Furr, J. M., & Sood, E. (2008). Flexibility within fidelity. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 987–993. Kendall, P. C., & Braswell, L. (1981). Cognitive-behavioral self-control therapy for children: A components analysis. Journal of Consulting and Clinical Psychology, 50, 672–689. Kossman, H. D., Lamb, J. M., O’Brien, M. W., Predmor, S. M., & Prescher, M. J. (2005). Measuring productivity in medical social work. Social Work in Health Care, 42(1), 1–16. Mischel, W., Shoda, Y., & Rodrigues, M. L. (1989). Delay of gratification in children. Science, 244(4907), 933–938. Monette, D. R., Sullivan, T. J., & DeJong, C. R. (2008). Applied social research: Tool for the human services (7th ed.). Belmont, CA: Brooks/Cole. National Association of Social Workers. (1999). Code of ethics of the National Association of Social Workers. Washington, DC: NASW. National Association of Social Workers. (2002). NASW standards for school social work services. Washington, DC: NASW. Panzano, P. C., & Roth, D. (2006). The decision to adopt evidence-based and other innovative mental health practices; risky business? Psychiatric Services, 57, 1153–1161. Ponitz, C. C., & McClelland, M. M. (2009). A structured observation of behavioral self-regulation and its contribution to kindergarten outcomes. Developmental Psychology, 45, 605–619. Posavac, H. D., Sheridan, S. M., & Posavac, S. S. (1999). A cueing procedure to control impulsivity in children with attention deficit hyperactivity disorder. Behavior Modification, 23, 234–243. Raines, J. C. (2008). Evidence-based practice in school mental health. New York, NY: Oxford University Press. Roberts, A. R., & Yeager, K. (2004). Systematic reviews of evidence-based studies and practice-based research: How to search for, develop, and use them. In A. R. Roberts & K. R. Yeager (Eds.), Evidence-based practice manual: Research and outcome measures in health and human service (pp. 3–14). New York, NY: Oxford University Press. Schaeffer, C. M., Bruns, W., Weist, M., Stephan, S. H., Goldstein, J., & Simpson, Y. (2005). Overcoming challenges to using evidence-based interventions in schools. Journal of Youth and Adolescence, 34, 15–22. Shoda, Y., Mischel, W., & Peake, P. K. (1990). Predicting adolescent cognitive and self-regulatory competencies from preschool delay of gratification: Identifying diagnostic conditions. Developmental Psychology, 26, 978–986. Wills, T. A., Ainette, M. G., Stoolmiller, M., Gibbons, F. X., & Shinar, O. (2008). Good self-control as a buffering agent for adolescent substance use: An investigation in early adolescence with time-varying covariates. Psychology of Addictive Behaviors, 22, 459–471.
Journal of Evidence-Based Social Work Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/webs20
Improving Impulse Control: Using an Evidence-Based Practice Approach a
Kendra J. Garrett & K. Giddings
b
a
School of Social Work, St. Catherine University and University of St. Thomas , St. Paul , Minnesota , USA b
Licensed Clinical Social Worker , Minnesota , USA Published online: 09 Jan 2014.
Click for updates To cite this article: Kendra J. Garrett & K. Giddings (2014) Improving Impulse Control: Using an Evidence-Based Practice Approach, Journal of Evidence-Based Social Work, 11:1-2, 73-83, DOI: 10.1080/15433714.2013.840197 To link to this article: http://dx.doi.org/10.1080/15433714.2013.840197
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Journal of Evidence-Based Social Work, 11:73–83, 2014 Copyright © Taylor & Francis Group, LLC ISSN: 1543-3714 print/1543-3722 online DOI: 10.1080/15433714.2013.840197
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
Improving Impulse Control: Using an Evidence-Based Practice Approach Kendra J. Garrett School of Social Work, St. Catherine University and University of St. Thomas, St. Paul, Minnesota, USA
K. Giddings Licensed Clinical Social Worker, Minnesota, USA
School social workers in a suburban school district implemented a two-year collaboration to identify empirically effective interventions and apply them to their own practice. During the first year of the project, the workers and the consultant discussed the use of evidence-based practice and developed strategies and tools for monitoring and evaluating practice. In the second year, each worker monitored the progress of one student using a standardized intervention. Five social workers completed the project; collected data on outcomes showed improved self-control for all participants. Limits on measurement and adaptations to individualize interventions make it impossible to be certain that the intervention was the primary cause of these outcomes. After the project the workers were not yet completely comfortable with the use of evidence-based practice to monitor and evaluate their practice. Recommendations for improving implementation of new evidence-based practice programs include providing ongoing discussion and feedback to workers, encouraging flexibility in implementation of curricula, simplifying record keeping so data reporting is consistent, and increasing baseline and follow-up measurements to strengthen the research. Keywords: Evidence-based practice, outcome evaluation, impulse control, case study
As schools have been mandated to use teaching methods that are empirically effective, school social workers have also been encouraged to move toward the use of evidence-based interventions in their work with students (Franklin & Hopson, 2004; National Association of Social Workers [NASW], 1999, 2002; Raines, 2008). Challenges to adopting an evidence-based approach include concerns that evidence-based practice (EBP) takes too much time, that there is too little research available to guide practice adequately, and that the use of EBP is formulaic and prevents workers from using creativity and autonomy (Gibbs & Gambrill, 2002; Schaeffer et al., 2005). Yet there are many good reasons to move toward using evidence in school social work practice because it is ethically, legally, and clinically appropriate (Raines, 2008). This article will describe the process one school social work department used to begin to integrate an evidence-based approach into their work with students. Address correspondence to Kendra J. Garrett, School of Social Work, University of St. Thomas, SCB 201, 2115 Summit Ave., St. Paul, MN 55105, USA. E-mail: [email protected]
73
74
K. GARRETT AND K. GIDDINGS
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
EBP involves using the best research available to answer practice questions by using methods shown empirically to be effective, applying critical thinking in making practice decisions, monitoring interventions to be sure they are implemented as intended, and tracking outcomes (Roberts & Yeager, 2004). Far from being a “cookbook approach,” social workers must be able to identify good research, evaluate it, and determine the extent to which the research would be appropriate in a particular setting with the client in question. The social worker must also have the relationship skills to discuss the intervention with the client to determine if the proposed treatment meets the client’s wishes. This process may involve some alteration of the intervention to better fit the specific situation (Hamilton, Kendall, Gosch, Furr, & Sood, 2008; Raines, 2008).
THE PROCESS The school social workers in a suburban school district decided, with the support of the special education director, to increase their knowledge of EBP in hopes of improving the services they provided to students. They took Franklin’s (1999) advice to partner with an academic institution for this process and began meeting with the senior author with the goals of becoming more comfortable applying research to their practice decisions and improving their ability to monitor and evaluate their practice. During the first year of the project, they discussed the EBP process and developed strategies and tools for monitoring and evaluating practice. They decided, as Schaeffer et al. (2005) suggested, to begin by collectively choosing one intervention that all would use over the second year of the project. The social workers settled on impulsivity as the student problem they would focus on for the year. Impulse control is of concern because of its association with academic success (Buckner, Mezzacappa, & Beardslee, 2009; Ponitz & McClelland, 2009; Shoda, Mischel, & Peake, 1990), aggression (Mischel, Shoda, & Rodrigues, 1989), social competence (Buckner et al., 2009), and adolescent substance abuse (Wills, Ainette, Stoolmiller, Gibbons, & Shinar, 2008). The search for interventions that were empirically validated indicated that impulsivity is best treated with cognitive-behavioral therapy (CBT), to include a variety of techniques, such as operant reinforcement, self-instruction, problem solving, modeling, role-playing, rehearsal, relaxation training, imagery, and group discussion (Baer & Nietzel, 1991; Fink & McCown, 1993; Kendall & Braswell, 1981; Posavac, Sheridan, & Posavac, 1999). A search for CBT programs that could be used to treat impulsivity yielded a number of possible programs and curricula aimed at improving impulse control in students. The social workers chose Hunter and His Amazing Remote Control (Copeland, 1998). Hunter has a series of lessons with accompanying activities that use the metaphor of a television remote control to help students regulate behavior. Students can pause, change channels (tune in after becoming distracted), fast forward (to anticipate consequences), rewind (to consider alternate options when behavior goes awry), use slow motion (to talk about emotions), coach (to use self-talk), zap (to stop negative thinking), and “way to go” (to self-affirm). Hunter was designed for elementary students, and the social workers anticipated that they would need to make adaptations for some of their older students. The workers agreed to apply the curriculum to one student each during the 2008–2009 academic year.
Measurement All social workers used the teacher-rated Self-Control Rating Scale (SCRS) developed by Kendall and Wilcox (Fischer & Corcoran, 2007) as a pre- and post-test measure of impulse control. This 33-item scale, completed by teachers, asks questions about the student’s ability to follow
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
IMPROVING IMPULSE CONTROL
75
through, self-calm, work for long-range goals, and other elements associated with the self-control construct. Teacher responses are summed to obtain the rating score. Higher scores denote greater impulsivity. Some social workers supplemented this measure with parent ratings of the SCRS. Some also used a 6-point individualized rating scale in each session with the student to assess the student’s ongoing progress on specific goals. This scale indicated the goal, and asked students to rate themselves on a scale ranging from 1 D “if things got worse” to 6 D “perfection.” The social workers also had the option of using the Children’s Perceived Self-Control Rating Scale developed by Humphrey (Fischer & Corcoran, 2007) as a supplementary pre- and post-test. This 11-item scale asks students to rate each question with “usually yes” or “usually no.” The measure contains questions about distractions, patience, anticipating consequences, and other items involved in perceived self-control. All but items 1 and 8 are reverse scored on this measure, and a low score is desirable. The school district’s Director of Teaching and the University of St. Thomas Institutional Review Board approved the project. Students could receive services, including the Hunter intervention, whether or not parents chose to allow their child/children to participate in the research portion of the project. Parent permission was required for the social workers to share the results anonymously with the university consultant. The district has 14 social workers. Of these, two do not have direct contact with students. Two had students who agreed to participate but moved away during the school year. Half of the remaining social workers implemented the program with students whose parents consented to share their data for this report. The social workers reported several barriers to their involvement. Some of the challenges included philosophical objections to “proving” their effectiveness with students, difficulty getting the requisite paperwork back from parents, the extra time and effort required, concerns about losing their personal style in the use of a proscribed program, changes in teachers between pre- and post-test, and concerns that many students did not fit the program (because of complex needs, age, or cognitive deficits). In this school district, social workers are considered related-service providers in the special education department. They often have responsibilities for services on students’ individual education plans (IEPs), and some of them view this IEP goal writing and tracking as meeting part of their EBP and outcome-measurement responsibilities.
CASE SUMMARIES Five social workers participated in this project by gathering pre- and post-test data on self-control with the SCRS as they used the Hunter curriculum to address impulsivity with one student. Each had a different experience with the Hunter curriculum.
Student A The school social worker worked with students aged 18–21 years, in a program for young adults with disabilities who are transitioning to independent living. These students are all special education students with IEPs. Their curriculum focuses on work, recreation, community membership, independent living, and post-secondary options. The school social worker had to modify the Hunter curriculum from its primary-grade audience to be appropriate for young adult participants. The social worker removed any reference to the Hunter story or the remote control while staying true to all cognitive-behavioral therapy concepts. Social Worker A was able to use some of the activities in the Hunter curriculum without modification. This social worker also had interest and expertise in yoga and supplemented the Hunter sessions with yoga and mindfulness practice,
76
K. GARRETT AND K. GIDDINGS
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
especially during the “slow-motion” lessons. The SCRS scale required modification of the wording to make it age-appropriate. The social worker administered the modified curriculum in a group of four students. The participating student was a legal adult and chose to self-answer the pre-test and post-test questions rather than asking the teacher to complete the rating. The student’s SCRS rating was 90 at pre-test and was 78 at post-test. Because of the two simultaneous interventions, it is not possible to know if the modified Hunter curriculum, the yoga, or the combination of the two interventions created the improvement in the SCRS scores. The reliability of the SCRS may also have been compromised by the fact that the student self-rated this instrument (rather than a teacher rating). Student B The school social worker worked with junior-high–aged students in grades 7–9 in an alternative learning center. The worker provided the Hunter curriculum to a group of 5 students over 15 sessions. An 8th grade student was monitored. This school social worker believed that the curriculum was age appropriate, even though the curriculum was written for an elementary-aged audience. For this student participant, two teachers completed pre- and post-test ratings on the SCRS. One teacher’s rating fell from 169 to 124, showing teacher-perceived growth. The other’s ratings increased from 112 to 127 (showing a perception that the student’s behavior deteriorated over the school year). Social Worker B attributed this inconsistency to a disadvantage in using numerical rating scales to measure progress, because they are “relative to a student’s behavior in the recent past.” This worker believed that “the teacher’s expectations for the student’s behavior changed”; the teacher expected better behavior by the end of the school year. Social Worker B’s observation of the student was that the student’s behavior improved. Over the course of the school year this student was placed on medication and the parents received education and counseling about medications. Social Worker B attributed changes in the student’s behavior to a combination of the Hunter curriculum, medication, and parent intervention. Student C The school social worker in an elementary-school setting presented the curriculum to a mainstream 4th-grade classroom over 8 sessions. The student participant was a special education student being served under an autism spectrum disorder (ASD) label, who also presented with severe deficits in attention and working memory, high levels of distractibility, and slow information processing. The social worker chose to follow this student because the parents were willing to allow the research and complete the rating scales, and the teacher was supportive of the school social worker’s presenting the Hunter curriculum in her classroom, thus allowing teacher reinforcement. The social worker also saw this student individually to focus on additional social-emotional goals from the student’s IEP. The student responded well to Hunter and was attentive. The social worker did not make any major adaptations to the Hunter curriculum in this classroom setting. In addition to the teacher rating of the SCRS, both parents also completed the measure. Progress with Student C was encouraging; in addition to the teacher’s perceived growth, the mother perceived great growth, and the father perceived a slight improvement (see Table 1). Student D The school social worker in a high school worked individually with a 16-year-old student who was transitioning from special education to mainstream classes. The student saw the social worker to address lingering impulse-control issues. The social worker presented the Hunter curriculum
IMPROVING IMPULSE CONTROL
77
TABLE 1 Self-Control Rating Scale for Student C
Teacher Rating Mother’s Rating Father’s Rating
Pre-Test
Post-Test
102 137 128
83 96 124
Change 19 41 4
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
Note. Low score is desirable.
over seven individual sessions. Because the student was older than the elementary-age students for whom Hunter was designed, major adaptations to the curriculum were necessary. Social Worker D used the remote control concept and activities but presented them orally as discussion points, rather than using the student worksheets. The term changed between the beginning and end of this intervention. Because classes changed, different teachers rated the student on post-test than on pre-test. The worker was, therefore, extremely cautious about the large drop in score from 161 to 60 (indicating strong improvement). The social worker noted the student continued to skip classes and arrive late to school. Each week, Student D completed a self-rating (“focusing on work”), reviewed the previous week’s progress, and learned a new skill. Student D also completed the Perceived Self-Control Scale; both the pre- and post-test ratings were a 5, indicating no change in the student’s perception of self-control. The weekly rating scale (see Figure 1), however, indicates the student perceived weekly improvement in focusing on work over the course of treatment.
Student E The social worker in an elementary setting presented the Hunter curriculum individually to a 2ndgrade student with ASD over a 6-month period. The social worker had to adapt the curriculum by breaking it into smaller steps. Recognizing “getting stuck in rigid thinking” as an important issue for this student, the social worker interpreted, personalized, and emphasized that idea as part of the “rewind” button (to help the student’s mind get “unstuck”). Teacher ratings of self-control improved from 133 to 119. The student rated the Children’s Perceived Self-Control Scale. This
FIGURE 1 Student D self-rating of focusing on work (rating: 1 is low; 6 is high).
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
78
K. GARRETT AND K. GIDDINGS
FIGURE 2
Student E self-rating of staying focused on work (rating: 1 is low; 6 is high).
rating improved from an 8 on pre-test to a 4 on post-test. Weekly ratings of the student’s goal of staying focused on work showed the student perceived steady improvement (see Figure 2).
OUTCOMES All social workers who participated used the SCRS as a pre-test and post-test of the changes in their students’ impulse control, and most showed gains (see Table 2). There is some evidence of progress for all students, but the social workers were cautious about drawing conclusions from the results, as there were challenges in the measurement process. One social worker changed the wording of the questions to be more appropriate for an adult student. Teachers had changed for another student, so a different teacher rated the post-test than the pre-test. Still another had mixed results on measures from two different teachers. There was wide variety in the way the program was administered as well, making it difficult to isolate the Hunter curriculum as the cause of any changes. All students received supplements to the Hunter curriculum, including additional lessons, use of yoga for calming, and starting medication over the course of the intervention. The program was administered over different lengths of time, and different modalities (individually, in small groups, or in a classroom setting). The workers individualized the intervention to meet student needs. TABLE 2 Teacher Ratings of Self-Control Rating Scale Student Aa B Teacher 1 B Teacher 2 C Db E
Teacher Pre-Test
Teacher Post-Test
Change
90 112 169 102 161 131
78 127 124 83 60 119
12 C15 45 19 101 12
Note. Low score is desirable. a Self-rating done by student. b Post-test rated by different teacher than pre-test.
IMPROVING IMPULSE CONTROL
79
LESSONS LEARNED The school social workers who participated in this 2-year project and the consultant learned valuable lessons about how to move toward the use of EBP in the district. Some of the lessons became more clear when the workers were surveyed about their experience at the end of the second year of the program.
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
Motivation One lesson learned is the effect of motivation on the process. The social workers who participated did so in part because they perceived pressure from the district (4 responses) and/or the supervisor (1 response) to show outcomes of their practice. They indicated also that they were internally motivated (4 responses) to see results of their practice and were interested in the project (4 responses). School social workers are feeling the pressures of No Child Left Behind (Raines, 2008) to show that their services are producing positive outcomes that help students be successful. Making changes entails some risk, and Panzano and Roth (2006) found that agencies that were quicker to adopt new or innovative ideas (including EBP) were those that believed the risks involved were manageable. This implies that those who supervise school social work programs have an important role to provide support and encouragement to departments as they take on new practice evaluation activities, providing assurance, for example, that a poor student outcome would not be viewed as a failure or be used to sanction the worker. An administration that supports learning about evidence-based programs and monitoring student outcomes can greatly enhance social workers’ willingness to participate in more formal evaluation. Administrators can provide additional support by encouraging adoption of EBP and providing time and money for consultation, record keeping, and identification of successful programs. Feedback and Discussion The school social workers were cautiously pleased with their progress towards EBP. Those who participated rated the experience “neutral” (2 responses), “positive” (2 responses), and “very positive” (1 response). In written comments, however, they indicated that they were not certain about the usefulness of the project and wanted more feedback about how to use data they had generated. A lesson learned is to develop more opportunities to discuss the outcomes and to consider together how the information can be used to enhance practice. So in implementing EBP programs, social workers should include discussions on interpretation of data, student improvements, revising treatment plans based on the data, and explanations of the findings to students and parents. Flexibility The tension between following a manual closely and making changes as appropriate for a client can often be resolved by allowing a certain amount of flexibility within a treatment program (Hamilton et al., 2008; Raines, 2008). Workers must attend to the therapeutic alliance with their clients, involving them in the treatment process, and altering programs to meet their particular needs (e.g., by adding relevant examples; Hamilton et al., 2008) or to meet the cultural, cognitive, or developmental needs of a student (Raines, 2008). In this case, the social workers slowed lessons for some students, changed worksheets to discussions, and supplemented the curriculum with yoga. Adaptation should remain true to the basic principles of the intervention and meet the goals of the individual sessions identified in treatment manuals (Hamilton et al., 2008). While sessions
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
80
K. GARRETT AND K. GIDDINGS
may be supplemented, they should not be abridged; the length of an intervention should not be shortened (Raines, 2008). Discussions about the changes made over the course of the intervention demonstrated that the workers were using CBT principles in making adaptations and were slowing the curriculum down and/or supplementing to make certain that the children were understanding the concepts. While the social workers adapted the curriculum to meet individual needs, it was also helpful that all of the social workers used a similar curriculum and the same basic measurements for their students. This allowed for group consultation and an opportunity for the social workers to compare notes and discuss strategies for adapting the curriculum. Having one measurement for all (the SCRS completed by the classroom teacher) allowed the social workers to compile data. If more students were involved social workers could use statistical measures to compare students by disability, gender, race, age, diagnosis, or other characteristics, perhaps further adding to the body of knowledge regarding treatment of students with impulse-control disorders. A major disadvantage of the use of consistent curriculum and measurement was that two of the social workers who did not participate indicated that the reason they chose not to participate was that the curriculum did not fit the students in their case loads. Others reported great difficulty measuring behavior of their unique students. Record Keeping The social workers in this district are using many different strategies for record keeping. It would have been helpful, as part of the planning of the project, if more attention had been given to standardizing the record keeping and the way workers recorded data. Templates that made record keeping easier for the workers could have simplified documentation, both on an ongoing basis and for summary reporting. For example, each student participant could have a coversheet that included space for demographic information (age, grade, gender, and disability status), intervention method (individual, group, and classroom), supplementary services provided (yoga, parent support, teacher consult, and classroom interventions), pre- and post-test ratings, and weekly notes, including a place for self- and teacher-ratings of progress. Once good record-keeping formats have been developed, it would be helpful to consult with the information technology department and develop a record-keeping strategy that would allow social workers to enter progress notes into an electronic record linked to an annual report, as Kossman, Lamb, O’Brien, Predmor, and Prescher (2005) suggest. This would have the advantage of saving time and simplifying record keeping. Reports could then be used to share information regarding social work services provided as well as outcomes of those services. Research The purpose of this project was to strengthen the use of evidence-based practice in one school social work department. The emphasis was on monitoring and evaluating practice outcomes in a real-world setting rather than on conducting research. Nevertheless, greater rigor in design would have enhanced monitoring of practice outcomes. This project used a pre-experimental design in which a pre-test was conducted before the intervention and a post-test after. This does not allow the workers to know for sure if gains were the result of the intervention or if other causes (e.g., changes in parenting practices or medications) contributed to the outcomes. This could have been improved through the use of more baseline measures for each student before beginning the intervention to see if their impulse-control scores were stable. A stable baseline would have given more assurance that subsequent gains were the result of the intervention (Monette, Sullivan & DeJong, 2008). Asking busy teachers to complete multiple baseline measures, however, may be problematic, so shorter measures might need to be developed.
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
IMPROVING IMPULSE CONTROL
81
The use of a control group that did not receive the intervention as a comparison would have lent even more strength to the project (Monette, Sullivan & DeJong, 2008). Randomized controlled trials have limitations in a school setting, and there are ethical concerns about withholding treatment. DeAnda (2007) suggests the use of quasi-experimental designs for groups of children that fit into the school calendar. A number of students could be pre-tested on the measure, and then groups receive services on a schedule that fits with the school calendar. When the first group of students has finished receiving services, students are again tested on the measure. The group that does not receive the intervention serves as a control/comparison group for the first group. Then the second group of students receives the intervention and serves as control/comparison for the third group, and so on. Over the course of a year several groups of students may receive the intervention and serve as comparisons for other groups. This method, however, may be better suited for classroom-wide interventions than for individual interventions. Follow-Up Compilation of the findings show that the student participants made positive gains, although the social workers expressed doubt about the measurement process and the permanence of the gains. This suggests that a 6-month follow-up would provide more information about the long-term benefits of the interventions to improve impulse control during the next academic year. Change Takes Time One year of instituting EBP in a school setting is a good start. Change to a new way of thinking about practice does not happen quickly. The feedback questionnaires indicated only a slight change in plans to use practice-monitoring strategies in the future. Schaeffer et al. (2005) recommend a grass-roots, democratic approach to choosing curriculum for EBP projects and voluntary participation, at least at first. All of the social workers have not yet determined that EBP fits the needs of their practice. While the social workers who participated felt that they learned valuable lessons, they did not feel confident that they could implement an EBP project independently.
CONCLUSION Taking the first step in using EBP in a school social work setting illustrates some of the challenges school social workers face in using evidence-based interventions. Students are complicated, and issues arose around the appropriateness of the curriculum, given the age range of students (elementary-aged to 21 years), their unique characteristics, different needs, and IEP requirements for providing services in individual, group or classroom settings. In some cases the curriculum needed considerable modification to meet the needs of the students. The curriculum was presented in formats ranging from individual to classroom. In one case, the student participated in a classroom session on the Hunter curriculum and received supplementary individual sessions that focused on additional goals. In another, the social worker used a combination of a modified Hunter curriculum and yoga. Given the adaptations that took place to individualize Hunter to meet the needs of the students in this project, it is difficult to sort out what led to gains in impulse control. Measurements also needed some adaptation for some of the students, causing concerns about reliability and validity. In one case, the teacher who rated the pre-test was no longer the student’s teacher at time of post-test. In another, the social worker modified the measurement to fit with
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
82
K. GARRETT AND K. GIDDINGS
the needs of an adult student and the student self-rated. It is far from certain that the gains noted in the changed ratings were actual changes in behavior. It is even less certain if the gains will remain over time. Use of additional measures might confirm that gains show up in more than one way. And these measurements need to be repeated after some months to see if gains are stable. The authors anticipated that implementation of the Hunter curriculum would provide group data that could be aggregated. Instead, because of individualization of the curriculum and different measurement strategies, the outcomes are better considered as a series of case studies. Nevertheless, the program showed considerable promise, as all self-control ratings but one indicated growth over the time that the student used the Hunter curriculum. The purpose of this project was not, strictly speaking, to evaluate the Hunter curriculum. What these social workers most hoped to accomplish was becoming more comfortable with the use of scientific research to inform their practice decisions and improving their ability to monitor and evaluate their practice. These goals were partially met. These school social workers also hoped to help students improve their impulse-control behavior at school, and data showed some progress for these students. An additional goal of the project was to move toward a more evidence-based approach to school social work practice. This goal was also partially met. The workers discovered that use of EBP strategies allows for a flexible, individualized approach to practice while increasing their confidence that they were choosing methods known to be empirically effective. They were less certain of their ability to apply data they gathered to modify their interventions as they progressed in their work with students. Perhaps implementation of EBP programs will require ongoing group support and feedback until social workers become more comfortable using data to monitor their individual modifications to the evidence-based curricula they have chosen. Most of the participating social workers expressed pleasure in using a CBT-based curriculum that had empirical backing. They were sometimes frustrated at the limits of the particular curriculum meant for elementary school students to a more diverse population. This is, perhaps, one of the challenges to the profession in implementing empirically based treatments. As social workers individualize, they will need to balance fidelity to the original protocols with changes needed for their unique clients. As social workers move toward the use of EBP, more guidelines on making these kinds of modifications are needed. As social work moves to overcome barriers to the use of EBP, the profession must consider the role of education. It may be that social work students are not gaining sufficient competence in analysis and application of empirical practice data. As the profession moves toward EBP, perhaps it should rethink the way evaluation is being taught in both practice and research courses to assure that future social workers have a comprehensive understanding of how to use data to modify and improve practice. And professional organizations should reach out to those who are already in practice to offer continuing education in the use of EBP.
REFERENCES Baer, R. A., & Nietzel, M. T. (1991). Cognitive and behavioral treatment of impulsivity in children: A meta-analytic review of the outcome literature. Journal of Clinical Child Psychology, 20, 400–412. Buckner, J. C., Mezzacappa, E., & Beardslee, W. R. (2009). Self-regulation and its relations to adaptive functioning in low-income youths. American Journal of Orthopsychiatry, 79, 19–30. Copeland, L. (1998). Hunter and his amazing remote control. Chapin, SC: Youthlight. deAnda, D. (2007). Intervention research and program evaluation in the school setting: Issues and alternative research and alternative research design. Children & Schools, 29, 87–94. Fink, A. D., & McCown, W. G. (1993). Impulsivity in children and adolescents: Measurement, causes, and treatment. In W. G. McCown, J. L. Johnson, & M. B. Sure (Eds.), The impulsive client: Theory, research, and treatment (pp. 279–308). Washington DC: American Psychological Association.
Downloaded by [University of Newcastle, Australia] at 23:58 30 December 2014
IMPROVING IMPULSE CONTROL
83
Fischer, J., & Corcoran, K. (2007). Measures for clinical practice and research (4th ed.) New York, NY: Oxford University Press. Franklin, C. G. (1999). Research on practice: Better than you think? Social Work in Education, 21, 3–9. Franklin, C., & Hopson, L. (2004). Into the schools with evidence-based practice, Children & Schools, 26(2), 67–70. Gibbs, L., & Gambrill, E. (2002). Evidence-based practice: Counterarguments to objections. Research on Social Work Practice, 12, 452–476. Hamilton, J. D., Kendall, P. C., Gosch, E., Furr, J. M., & Sood, E. (2008). Flexibility within fidelity. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 987–993. Kendall, P. C., & Braswell, L. (1981). Cognitive-behavioral self-control therapy for children: A components analysis. Journal of Consulting and Clinical Psychology, 50, 672–689. Kossman, H. D., Lamb, J. M., O’Brien, M. W., Predmor, S. M., & Prescher, M. J. (2005). Measuring productivity in medical social work. Social Work in Health Care, 42(1), 1–16. Mischel, W., Shoda, Y., & Rodrigues, M. L. (1989). Delay of gratification in children. Science, 244(4907), 933–938. Monette, D. R., Sullivan, T. J., & DeJong, C. R. (2008). Applied social research: Tool for the human services (7th ed.). Belmont, CA: Brooks/Cole. National Association of Social Workers. (1999). Code of ethics of the National Association of Social Workers. Washington, DC: NASW. National Association of Social Workers. (2002). NASW standards for school social work services. Washington, DC: NASW. Panzano, P. C., & Roth, D. (2006). The decision to adopt evidence-based and other innovative mental health practices; risky business? Psychiatric Services, 57, 1153–1161. Ponitz, C. C., & McClelland, M. M. (2009). A structured observation of behavioral self-regulation and its contribution to kindergarten outcomes. Developmental Psychology, 45, 605–619. Posavac, H. D., Sheridan, S. M., & Posavac, S. S. (1999). A cueing procedure to control impulsivity in children with attention deficit hyperactivity disorder. Behavior Modification, 23, 234–243. Raines, J. C. (2008). Evidence-based practice in school mental health. New York, NY: Oxford University Press. Roberts, A. R., & Yeager, K. (2004). Systematic reviews of evidence-based studies and practice-based research: How to search for, develop, and use them. In A. R. Roberts & K. R. Yeager (Eds.), Evidence-based practice manual: Research and outcome measures in health and human service (pp. 3–14). New York, NY: Oxford University Press. Schaeffer, C. M., Bruns, W., Weist, M., Stephan, S. H., Goldstein, J., & Simpson, Y. (2005). Overcoming challenges to using evidence-based interventions in schools. Journal of Youth and Adolescence, 34, 15–22. Shoda, Y., Mischel, W., & Peake, P. K. (1990). Predicting adolescent cognitive and self-regulatory competencies from preschool delay of gratification: Identifying diagnostic conditions. Developmental Psychology, 26, 978–986. Wills, T. A., Ainette, M. G., Stoolmiller, M., Gibbons, F. X., & Shinar, O. (2008). Good self-control as a buffering agent for adolescent substance use: An investigation in early adolescence with time-varying covariates. Psychology of Addictive Behaviors, 22, 459–471.
