J. DRUG EDUCATION, Vol. 43(3) 255-275, 2013

THE IMPACT OF SCIENCE EDUCATION GAMES ON PRESCRIPTION DRUG ABUSE ATTITUDES AMONG TEENS: A CASE STUDY*

YVONNE KLISCH KRISTI G. BOWLING LESLIE M. MILLER Rice University Center for Technology in Teaching and Learning, Houston, Texas MIGUEL A. RAMOS University of Houston, Texas

ABSTRACT

Two online science education games, in which players learn about the risks of prescription drug abuse in the context of investigating crimes, were evaluated to determine shifts of prescription drug abuse attitudes attributable to game exposure. High school students from grades 11 and 12 (n = 179) were assigned to one of the games and participated in a pretest, two game-play sessions, and a delayed posttest. Students in both groups demonstrated more negative attitudes toward prescription drug abuse after playing the game, driven by changes of students’ normative beliefs and their ability to make the connection between prescription drug abuse and illicit drugs. A secondary

*This study was supported by Grant Number R25DA031110 from the National Institute on Drug Abuse, National Institutes of Health. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Institute on Drug Abuse or the National Institutes of Health. 255 Ó 2013, Baywood Publishing Co., Inc. doi: http://dx.doi.org/10.2190/DE.43.3.d http://baywood.com

256 / KLISCH ET AL.

aim was to assess gains in science knowledge; however, due to low internal consistency reliabilities of content measures, students’ knowledge acquisition could not be determined.

INTRODUCTION Non-medical use of prescription-type pain relievers, tranquilizers, stimulants, and sedatives has become highly prevalent among youth in the United States, second only to abuse of marijuana (Hernandez & Nelson, 2010). According to the 2012 National Survey on Drug Use and Health, approximately 2.6% of U.S. youth aged 12 to 17 reported current (past month) use of a prescription drug for non-medical purposes (Substance Abuse and Mental Health Services Administration [SAMHSA], 2013). After an initial decline from 4.0% in 2002 to 2.8% in 2008, no significant change has occurred since. The prevalence is driven by an increased environmental availability of the drugs. Between 1991 and 2010, prescriptions for opioid analgesics, for example, increased from about 75.5 million to 209.5 million (National Institute on Drug Abuse, 2011). For the most part, the drugs are easy to obtain. In fact, during 2011-2012, 54.0% of people ages 12 or older who used pain relievers non-medically in the past year acquired the drugs from a friend or relative for free, while 10.9% bought them from a friend or relative, and 4.0% took them from a friend or relative without asking (SAMHSA, 2013). Because doctors prescribe these medications, the misconception exists that they are safe to take under any circumstances. This is reflected in the decline of perceived harmfulness of non-medical prescription drug use among youth found in the 2012 Monitoring the Future survey (Johnston, O’Malley, Bachman, & Schulenberg, 2013). For example, the percentage of 12th graders who believed that people risk harming themselves if they try narcotics such as OxyContin or Vicodin once or twice decreased from 40.4% in 2010 to 38.4% in 2012. These combined data indicate the ongoing need to educate adolescents about the biological effects of prescription drug abuse. The introduction of this health content into the school curriculum, however, becomes more difficult given the demise of school-based health and drug education programs in many states (Kann, Telljohann, & Wooley, 2007). The infusion of science curricula with health-related topics is a potential solution. Holtz and Twombly (2007) provide the theoretical basis for a science education approach to school-based drug prevention efforts. As they summarize: Science education differs from prevention in a number of ways. First, science education presents information about effects of drugs on the brain and the body without overt injunctions to avoid use. In a science education curriculum, the content is unvalenced, and drugs are never described as bad or dangerous. Second, science education curricula are matched to state and national standards of learning so they can be implemented in the regular

GAMES TO CHANGE PRESCRIPTION DRUG ABUSE ATTITUDES /

257

classroom as part of the standard science curriculum.[…] In this manner, science education can be a type of “stealth” prevention in which students are presented with facts that are likely to change their attitudes and behaviors regarding drug use, but not direction to do so.” (pp. 318, 319)

The purpose of this study was to examine whether exposure to two interactive forensic science games, CSI Web Adventures: Bitter Pill (Case 4) and Fatal Interactions (Case 5) could impact high school students’ attitudes toward prescription drug abuse while simultaneously teaching the related science content. Serious games are a subset of science curricula that many schools integrate into their existing course of study. The authors’ own research and development of serious games that address drug and alcohol abuse, while incorporating required science standards, has indicated that science knowledge is improved (Klisch, Miller, Beier, & Wang, 2012; Miller, Moreno, Willcockson, Smith, & Mayes, 2006; Miller, Schweingruber, Oliver, Mayes, & Smith, 2002) and attitudes toward drugs are modified as a result (Klisch, Miller, Wang, & Epstein, 2012). For example, a game framed around environmental issues, which described the consequences of inhalant abuse in terms of brain pollutants, resulted in increased science knowledge and more negative attitudes toward inhalants among middle school students (Klisch, Miller, Wang, et al., 2012). With funding from the National Institute on Drug Abuse, Bitter Pill and Fatal Interactions were designed to cover different aspects of the biological consequences of prescription drug abuse while using a forensic science narrative in which the player assumes the role of a crime scene investigator. Because there is a negative correlation between the perceived risk of using a drug and the actual drug use (Johnston et al., 2013), informing students about the risks of the non-medical use of prescription drugs may influence their attitudes toward this behavior, as suggested by the Theory of Reasoned Action (Ajzen, 1991). Thus, there was an expectation that exposure to accurate, scientific knowledge presented in the games would result in more negative attitudes toward prescription drug abuse. GAME DEVELOPMENT AND INSTRUCTIONAL DESIGN Computer games developed for the purpose of teaching have become popular in U.S. classrooms (Grunwald Associates, 2009; Millstone, 2012). These educational games reflect the learning styles of a more digitally savvy generation and represent trends in both formal and informal educational settings toward technology use (Gee, 2003; Ito, Horst, Bittanti, Boyd, Herr-Stephenson, Lange, et al., 2008; Project Tomorrow, 2012; Rideout, Foehr, & Roberts, 2010). Teachers value the power of games to sustain students’ focus on specific tasks and to increase motivation and engagement, especially among lower performing students (Millstone, 2012). Previous evaluations conducted by the authors (Bowling, Klisch, & Beier, 2013; Klisch, Miller, Beier, et al., 2012; Klisch, Miller,

258 / KLISCH ET AL.

Wang, et al., 2012; Miller, Chang, Wang, Beier, & Klisch, 2011; Miller, Moreno, Estrera, & Lane, 2004; Miller et al., 2002, 2006) and research by others (for example, Halpern, Millis, Graesser, Butler, Forsyth, & Cai, 2012; Meluso, Zheng, Spires, & Lester, 2012; Shegog, Lazarus, Murray, Diamond, Sessions, & Zsigmond, 2012; Wu & Pedersen, 2011) have indicated that games are effective tools to promote learning of scientific content in secondary and higher education. CSI Web Adventures Bitter Pill and Fatal Interactions were designed with the participation and input of content experts, educators, and students (Figure 1). The creation of the original game series (http://forensics.rice.edu) consisting of three cases, which was developed with funding from the National Science Foundation and in cooperation with CBS Consumer Products, is described elsewhere (Miller et al., 2011). The games are available using any web browser and are accessible free of charge. Each of the cases takes approximately 1 hour to complete and a player’s progress can be saved so the mystery need not be completed in one continuous time period. While the popular, original three games focused solely on forensic science techniques and processes, the two new cases were created to teach about prescription drug abuse within the context of the forensic investigation of drug-related crimes. The negative consequences of non-medical prescription drug use are highlighted in both games, but an effort was made to include information on the benefits of the medications when used as prescribed to avoid any stigma of the drugs’ legitimate purposes. The specific prescription drug abuse content covered in Bitter Pill and Fatal Interactions is summarized in Table 1. Opioid pain reliever abuse was chosen as the topic for Bitter Pill. These drugs continue to have high rates of non-medical use in grades 8, 10, and 12 (Johnston et al., 2013). According to the Centers for Disease Control and Prevention (2012), the number of deaths resulting from respiratory depression after unintentional opioid overdoses is increasing, surpassing death rates of heroin and cocaine overdoses combined. A number of overdose deaths have resulted particularly with the drug OxyContin, which was designed to be a slow-release formulation of its active ingredient oxycodone (Cone, Fant, Rohay, Caplan, Ballina, Reder, et al., 2003). At the beginning of Bitter Pill, players are asked to investigate a house fire and the death of a young man found at the scene. Players gather evidence and use gas chromatography to reveal that the fire was started by the ignition of gasoline, confirming that it was arson. An autopsy and a subsequent spectrophotometry analysis on the victim’s blood indicate that the victim did not die from carbon monoxide poisoning and that he was already dead when the fire started. Instead, an ELISA test and gas chromatography/mass spectrometry confirm that the cause of death was an oxycodone overdose. The investigation now shifts to the victim’s abuse of oxycodone, which he acquired by

GAMES TO CHANGE PRESCRIPTION DRUG ABUSE ATTITUDES /

259

Figure 1. Screen capture of the CSI Web Adventures homepage (http://forensics.rice.edu). Case 4: Bitter Pill and Case 5: Fatal Interactions are on the right, highlighted with a white dashed line.

stealing a prescription. Players consult a physician to gather information about oxycodone and other opioids and the biological effects of medical versus non-medical use. The drug education subtext in Fatal Interactions focuses on adverse drug interactions and what doctors, pharmacists, and patients can do to avoid them. Adverse reactions as a result of taking two or more drugs at the same time is especially problematic in the context of opioid pain reliever abuse, whose effects on the respiratory system can be exaggerated and prolonged by other drugs, particularly central nervous system depressants (Cone, Fant, Rohay, Caplan, Ballina, Reder, et al., 2004). In Fatal Interactions, players are called to a pain clinic to investigate a potential suicide of one of the doctors. However, bullet comparisons and DNA analyses suggest that the doctor has been murdered. Players find threatening e-mails written by an anonymous sender who claimed the doctor was responsible for turning his or her late daughter into a drug addict. After revealing this person’s identity, players find out that the victim had given the daughter increasing amounts of opioid painkillers and muscle relaxants, which were paid for in cash. Players

260 / KLISCH ET AL.

Table 1. Prescription Drug Abuse Related Content in CSI Web Adventures: Bitter Pill and Fatal Interactions Bitter Pill • Oxycodone is an opioid, the same group that also contains morphine and its synthetic derivative, heroin. • Oxycodone is very similar to the body’s own opioids, which are produced in the brain to fight pain. • Both oxycodone and body-produced painkillers act on the brain’s reward system to decrease the perception of pain. They are also involved in producing feelings of pleasure. • If a person is in pain, it is safe to take oxycodone because it acts together with the body’s own painkillers. If the pain is gone, the person can stop taking oxycodone. • Taking oxycodone without being in pain is not safe. The excess of unneeded oxycodone can lead to brain changes and symptoms such as tolerance, withdrawal, and craving, which are signs of addiction. • Taking too much oxycodone can be fatal due to the drug’s ability to slow down or stop breathing. Fatal Interactions • Drug interactions (unwanted effects from combining two or more drugs) can prolong or intensify one or both of the drugs’ effects. • To avoid drug interactions, patients should always indicate which medications they are already taking when visiting a doctor. • Prescribing drugs without medical indication is illegal. • Pharmacists are legally required to make sure the prescriptions he/she fills are for medical treatment, and to warn patients about potential drug interactions. • There are certain classes of common prescription drugs that can cause severe, even fatal drug interactions (CNS depressants, opioids, stimulants, muscle relaxants, and certain over-the-counter cough medications).

also find a subpoena ordering the victim to give testimony about the clinic’s involvement in violations against the Controlled Substances Act. Players are asked to determine the drug interactions the daughter could have been experiencing based on the different drugs she was taking, and to investigate how the doctor and a pharmacist were involved in illegally providing the drugs.

GAMES TO CHANGE PRESCRIPTION DRUG ABUSE ATTITUDES /

261

Embedding content related to prescription drug abuse into the narrative of a forensic investigation is complementary to the notion that messages about harm can best be reinforced through subtle means rather than head-on confrontation. Learning about somebody else’s drug use has been shown to be more persuasive than messages that are directly aimed at the target of influence (Crano, Siegel, Alvaro, & Patel, 2007; Frey & Eagly, 1993; Holtz & Twombly, 2007). Other research suggests that resistance to anti-drug messages is enhanced when the persuasive intent of a message is clear and threatens a valued belief, poses self-image threats such as appearing gullible, or when the target audience is pre-exposed to weak forms of counter-attitudinal messages (McGuire, 1964; Wood & Quinn, 2003). In Bitter Pill and Fatal Interactions, adolescents actively gather scientific facts about prescription drug abuse during a crime investigation, which provides the basis for indirect, non-threatening anti-drug messages. In addition, the forensic storyline sets a motivational context, provides structure to the scientific content, and allows students to inhabit an active role in gathering the information in order to solve the case. These specific elements have been shown to increase the effectiveness of educational games (Barab, Scott, Siyahhan, Goldstone, Ingram-Goble, Zuiker, et al., 2009; Brown, Collins, & Duguid, 1989; Shelton & Scoresby, 2011; Squire, 2006). STUDY DESIGN AND EVALUATION METHODS Participants Approval of the study design and protection of human subjects was obtained through the university Institutional Review Board. High school forensic science teachers were recruited through e-mails sent to: (a) potential volunteers whose addresses had been collected at professional development meetings, and (b) teachers on the mailing list of the local Department of Education. The requirements for inclusion were approval from the school principal, availability of at least two elective forensic science classes of equivalent academic achievement and grade level comparison (11th grade or 12th grade or mixed grades), guarantee of adequate computer and Internet access, and assurance that students had not previously played any of the CSI Web Adventures. A $200 stipend was offered to teachers for their adherence to the testing protocol and for returning both parental consent and student assent of those who elected to participate. To protect students from coercion, it was made clear that study participation was entirely voluntary and that it would not impact students’ grades or teacher stipends. Five teachers from public high schools in Texas participated in the study. The final sample consisted of 179 students (63.1% female) with 34 students (19%) in 11th grade and 145 (81%) in 12th grade. All classes were mixed-grade elective forensic science classes. A detailed breakdown of demographics in each study group is provided in Table 2. To validate the representation of

262 / KLISCH ET AL.

Table 2. Demographics of Study Groups Sample characteristic

Bitter Pill players

Fatal Interactions players

n

95

84

Grade

18 11th (18.9%) 77 12th (81.1%)

16 11th (19.0%) 68 12th (81.0%)

Gender

59 Female (62.1%) 36 Male (37.9%)

54 Female (64.3%) 30 Male (35.7%)

Ethnicity

39 Caucasian (41.05%) 39 Hispanic (41.05%) 12 African American (12.6%) 4 Other (4.2%) 1 Native American (1.1%)

44 Caucasian (52.4%) 23 Hispanic (27.4%) 11 African American (13.1%) 4 Other (4.7%) 2 Asian/Pacific Islander (2.4%)

socio-economic diversity, schools in the sample were assigned to one of three economic groups as operationalized by free/reduced lunch percentages of the total school population, ranging from low disadvantage (less than 29% of students receive free-reduced lunch), to middle disadvantage (between 29-59% of students receive free-reduced lunch), and high disadvantage (more than 59% of students receive free-reduced lunch). The final sample contained 73 (40.8%) students attending low disadvantaged schools, 69 (38.5%) from middle disadvantage schools, and 37 (20.7%) from high disadvantage schools. Instruments Attitudes toward Prescription Drug Abuse

Attitudes toward prescription drug abuse were measured with a 10-item scale based on a previously developed measure used to assess attitudes toward inhalant abuse (Klisch, Miller, Wang, et al., 2012). For example, one statement was, “It is okay for people to take prescription drugs even if they are not sick, because they are safe” (see the Appendix for the complete set of items). Students rated their level of agreement to the different statements on a 5-point Likert response format ranging from 1 = “strongly disagree” to 5 = “strongly agree.” The scale was coded so that higher scores reflect more positive attitudes toward prescription drug abuse. Thus, in this study the desired outcome is to shift attitudes in a more negative direction, indicating less approval or tolerance for prescription drug abuse. Internal consistency reliability estimates (Cronbach’s alpha; Cronbach,

GAMES TO CHANGE PRESCRIPTION DRUG ABUSE ATTITUDES /

263

1951) were .816 (pretest) and .847 (posttest) in the Bitter Pill group, and .722 (pretest) and .848 (posttest) in the Fatal Interactions group. Considering that the measure is a new approach to address prescription drug abuse attitudes and the scale only consisted of 10 items, reliability estimates of .70 or higher were considered satisfactory (Nunnally, 1978). Content Learning

Content knowledge acquisition was measured with 28 multiple-choice items with 4 response options. The test items were developed based on the results of a pilot test performed with 78 students from mixed grade (11th and 12th) forensic science classes different from the participants of this field test. Acquisition of knowledge related to prescription drug abuse (PDA) was assessed with 10 questions, 4 pertaining to Bitter Pill and 6 pertaining to Fatal Interactions. An example is, “Overdosing painkillers such as oxycodone can cause a person to . . .” (correct answer: stop breathing). While the assessment of forensic science (FS) content learning was not the focus of this study, 18 questions, 9 pertaining to Bitter Pill, and another 9 to Fatal Interactions, were included to ensure that the games are efficacious in the context of forensic science classes. For example, a question related to Bitter Pill asked, “What technique would a forensic scientist use to confirm if an ignitable liquid was used to start a fire?” (correct answer: gas chromatography). Internal consistency reliability estimates (Cronbach’s alpha) were unsatisfactory (Bitter Pill: PDA pretest –.028, PDA posttest: .379, FS pretest .136, FS posttest .426; Fatal Interactions: PDA pretest .493, PDA posttest .552, FS pretest –.072, FS posttest .579). Ideally, knowledge items in each content area would constitute uniform scales rather than measure independent concepts. However, the low reliability values affirm that the content knowledge items used in this study measured distinct aspects of the prescription drug abuse or forensic science content area. Therefore, the study will focus on affect measures which were much more reliable while discussing possible reasons for the content exam reliability issues. Posttest-Only Measures

Posttest-only measures included satisfaction with the game, game usability, and role-play experience. All three scales have been previously used and validated (Bowling et al., 2013; Klisch, Miller, Beier, et al., 2012; Klisch, Miller, Wang, et al., 2012; Miller et al., 2011). Students’ level of engagement was assessed with four items developed based on a scale measuring behavioral, emotional, and cognitive aspects of school engagement (Fredericks, Blumenfeld, Friedel, & Paris, 2005). Internal consistency reliability estimates (Cronbach’s alpha) for the Bitter Pill group were .769 for satisfaction, .691 for usability, .616 for role-play, and .722 for engagement. For the Fatal Interactions group, reliability estimates were .849 for satisfaction, .729 for usability, .712 for role-play, and .850 for

264 / KLISCH ET AL.

engagement. Because of the low reliability estimates of the usability and role-play measures in the Bitter Pill group, those data were excluded from analysis. In addition, two open-ended questions were included that asked students to indicate what they liked most and least about the game, respectively. Answering these open-ended questions was optional. Procedure The selected quasi-experimental study design involved two matched classes from each teacher. One class was randomly assigned Bitter Pill, and the other comparable class was assigned Fatal Interactions. A study website was provided that contained links to the pretest, assigned game, and posttest. Students participated in four 45-minute sessions, working independently on school computers. In Session 1, students completed the pretest, which began with a brief assent form explaining the research and purpose of the study. If students chose to decline participation, they were automatically taken to the end of the pretest and no data were collected. Upon assenting participation, students were asked for an identification number that had been assigned to them by their teachers. Subsequent screens included an assessment of attitudes toward prescription drug abuse (PDA) and the pretest of content knowledge pertaining to content of both games, followed by basic demographic information (e.g., grade, gender, ethnicity). In Sessions 2 and 3, students played their assigned game. Requirements for these sessions were that Session 2 was scheduled with a minimum delay of 3 days after Session 1 (the longest actual delay was 7 days), and Sessions 2 and 3 were administered over the course of a maximum of 1 week (the longest actual spread was 4 days). To test the persistent effects of the game beyond the immediacy of the intervention, Session 4 was scheduled with a minimum delay of 3 days after finishing Session 3 (the longest actual delay was 5 days). This session consisted of the posttest with additional assessments of satisfaction, role-play experience, engagement, and game usability. Teachers were instructed to avoid pre-teaching the game content and to refrain from providing recap of content until after the conclusion of the field test. This approach allowed the assessment of the game exposure relatively independent of teacher instruction. RESULTS Of the 246 students who completed the posttest and assented their participation (125 in the Bitter Pill group and 121 in the Fatal Interactions group), 67 were excluded for the following reasons: data could not be matched to the pretest (33 students), data were duplicated because students took the test twice (three students), students were absent in one or both of the gameplay sessions (28

GAMES TO CHANGE PRESCRIPTION DRUG ABUSE ATTITUDES /

265

students), or students did not provide a signed consent form (three students). The final sample included 179 students: 95 who played Bitter Pill and 84 who played Fatal Interactions. Changes in Attitudes toward Prescription Drug Abuse Comparisons of pretest and posttest mean scores on the prescription drug abuse attitude scale using paired-sample t-tests are shown in Table 3. In both study groups, students’ attitudes toward prescription drug abuse shifted significantly toward more negative attitudes, with a moderate effect size (e.g., d-score between .20 and .80; Cohen, 1988) in the Bitter Pill group (pretest mean: 1.91, SD: 0.49; posttest mean: 1.63, SD: 0.61; t = 5.105, p = .01, d = .74) and with a large effect size in the Fatal Interactions group (pretest mean: 1.87, SD: 0.39; posttest mean: 1.53, SD: 0.57, t = 5.582, p = .01, d = .90). Examining the frequencies of answers to the items included in the prescription drug abuse attitudes scale, three statements emerged with highest shifts in both students who played Bitter Pill and those who played Fatal Interactions: 1. “Abusing prescription drugs is just typical teenage behavior” (percentage of students who answered “completely disagree” changed from 30.5% in the pretest to 43.2% in the posttest in the Bitter Pill group and from 33.3% to 50.0% in the Fatal Interactions group); 2. “If my friends were abusing prescription drugs, I probably wouldn’t mind” (percentage of students who answered “completely disagree” changed from 45.3% to 54.7% from pre- to posttest in the Bitter Pill group and from 53.6% to 65.5% in the Fatal Interactions group); and 3. “Abusing prescription drugs is just as bad as abusing other drugs” (percentage of students who answered “completely agree” changed from 53.7% to 63.2% from pre- to posttest in the Bitter Pill group and from 50.0% to 70.2% in the Fatal Interactions group). Table 3. Descriptive Statistics, Paired-Sample t-Test Scores, and Effect Sizes for Comparing Pre- and Posttest Attitudes toward Prescription Drug Abuse Study group

df

t

Pretest mean (SD)

Posttest mean (SD)

d

Bitter Pill playersa

94

5.105**

1.91 (0.49)

1.63 (0.61)

.74

Fatal Interactions playersb

83

5.582**

1.87 (0.39)

1.53 (0.57)

.90

Note: Effect sizes (d-scores) adjusted for pre- and posttest correlations (i.e., test of dependent means). Pre- and posttest correlation Bitter Pill players: .528, Fatal Interactions players: .387. an = 95; bn = 84; **p £ 0.01.

266 / KLISCH ET AL.

Given the gender differences reported for prescription drug abuse (Huang, Dawson, Stinson, Hasin, Ruan, Saha, et al., 2006; Johnston et al., 2013; SAMHSA, 2013), a remaining question was whether the interventions were equally effective with regard to a negative shift in attitudes toward prescription drug abuse among both males, who are more likely to abuse prescription drugs, and females. To this end, a series of repeated measures ANOVAs using the GLM model were performed (Figure 2). A repeated-measures ANOVA design is used to reveal changes in test scores over several iterations of the dependent measure relative to the independent units (e.g., students; Glass & Hopkins, 1996). In this study, the design calls for a repeated-measures ANOVA with a betweensubjects variable, which compares performance between groups of students (for example genders) in addition to a within-subjects factor (e.g., test administration). Although both pretest and posttest attitudes toward prescription drug abuse were slightly higher among male students in both groups, there was no evidence of interaction between gender and change of attitudes toward prescription drug abuse (Bitter Pill group: F(1, 93) = 2.538, p = .115; Fatal Interactions group: F(1, 82) = 1.327, p = .253). All students demonstrated more negative attitudes toward prescription drug abuse after exposure to the games as seen in the downward trending lines in Figure 2. Posttest-Only Measures To compare mean scores on posttest-only measures (e.g., satisfaction and engagement) between the study groups, independent sample t-test were performed (Table 4). Students who played Fatal Interactions rated their satisfaction with the game, as well as their level of engagement while playing the game, significantly higher than students who played Bitter Pill (satisfaction: Bitter Pill mean: 3.68, SD: 0.84; Fatal Interactions mean: 4.05, SD: 0.77; t = –3.039, p = .01; engagement: Bitter Pill mean: 3.45, SD: 0.78; Fatal Interactions mean: 3.74, SD: 0.85; t = –2.363, p = .05). In the Fatal Interactions group, usability (mean: 3.38, SD: 0.86) and role-play experience (mean: 3.77, SD: 0.74) were rated above average. Due to the low reliabilities of the same measures in the Bitter Pill group, means related to this game were not determined. Although recent statistics show that gender gaps regarding the use of digital games are diminishing (Lenhart, Kahne, Middaugh, Macgill, Evans, & Vitak, 2008), it is important for developers of game-based curricula to examine potential gender differences in game-related ratings. Independent sample t-tests revealed that among students who played Bitter Pill, female students showed significantly higher ratings of satisfaction (female mean: 3.90, SD: 0.87; male mean: 3.32, SD: 0.65; t(93) = 3.455, p = .01) and engagement (female mean: 3.57, SD: 0.83; male mean: 3.25, SD: 0.66; t(93) = 1.984, p = .05) compared to male students. No significant gender differences were found in the Fatal Interactions group (satisfaction: female mean: 4.12, SD: 0.80; male mean: 3.92, SD: 0.70;

GAMES TO CHANGE PRESCRIPTION DRUG ABUSE ATTITUDES /

267

Figure 2. Repeated measures ANOVA analysis of gender differences in prescription drug abuse attitudes. Top: Bitter Pill group. Means: females pretest: 1.85 (SD 0.57), posttest: 1.49 (SD 0.52); males pretest: 2.02 (SD 0.57), posttest: 1.84 (SD 0.69); total sample pretest: 1.91 (SD 0.49), posttest: 1.63 (SD 0.61). Bottom: Fatal Interactions group. Means: females pretest: 1.80 (SD 0.36), posttest: 1.51 (SD 0.59); males pretest: 2.00 (SD 0.42), posttest: 1.57 (0.45); total sample pretest: 1.87 (SD 0.39), posttest 1.53 (SD 0.57). *PDA = prescription drug abuse.

268 / KLISCH ET AL.

Table 4. Descriptive Statistics and Independent Sample t-Test Scores for Comparing Posttest-Only Measures between Students Who Played Bitter Pill and Students Who Played Fatal Interactions Measure

Bitter Pill Mean (SD)a

Fatal Interactions Mean (SD)b

t

df

Satisfaction

3.68 (0.84)

4.05 (0.77)

–3.039**

177

Usability

n.d.

3.38 (0.86)

n.d.

n.d.

Role-play

n.d.

3.77 (0.74)

n.d.

n.d.

3.45 (0.78)

3.74 (0.85)

–2.363*

177

Engagement

an = 95; bn = 84; *p £ 0.05; **p £ 0.01. n.d. = not determined.

t(67.434) = 1.197, p = .235; engagement: female mean: 3.80, SD: 0.89; male mean: 3.63, SD: 0.79, t(66.073) = .867, p = .389). DISCUSSION The goal of this evaluation was to examine whether two forensic science online games, in which prescription drug abuse content is included as a subtext to the investigation of crimes, can be successful in increasing students’ knowledge about and changing their attitudes toward prescription drug abuse. First, and possibly most critical from a prevention perspective, is the change of drug-related attitudes. The demonstrated shifts toward more negative prescription drug abuse attitudes observed in both study groups, resulting from students interacting with their assigned game in two class periods, reflected a moderate effect size for Bitter Pill and a large effect size for Fatal Interactions. While students’ attitudes toward prescription drug abuse were fairly negative at baseline (means of 1.91 and 1.87, respectively), there was nevertheless an overall shift to even more negative attitudes (posttest means of 1.63 and 1.53, respectively). The finding that female students in both study groups had lower, more negative baseline prescription drug abuse attitudes than males is reflective of the higher prevalence of prescription drug abuse (and drug abuse in general) among males (Huang et al., 2006; Johnston et al., 2013; SAMHSA, 2013). In 2012, 15.5% of male and 13.7% of female 12th grade students reported use of any prescription drug without medical supervision in the past year (Johnston et al., 2013). Although male students’ posttest attitudes did not reach the same low levels as observed in the female study population (female posttest means: 1.49 and 1.51, respectively; male posttest means: 1.84 and 1.57, respectively),

GAMES TO CHANGE PRESCRIPTION DRUG ABUSE ATTITUDES /

269

there was no difference in the magnitude of the attitudinal shift, suggesting that the games were equally effective with regard to prescription drug abuse attitude change among both genders. Obviously, the question remains as to whether the negative attitudes transfer to behavior. Under the Theory of Reasoned Action, one might expect to see avoidance of these drugs unless prescribed by a doctor. The storyline content made clear the distinction between appropriate and inappropriate use, thus a well-informed citizen (student) would be capable of making these distinctions. A factor that limits the generalizability of the prescription drug abuse attitude change is the lack of a control group that would have allowed a comparison of students who played one of the CSI games to those who did not receive an intervention at all. Interestingly, two of the statements in the attitudes toward prescription drug abuse scale that showed the largest negative shift in both study groups were related to students’ normative beliefs of prescription drug abuse. Normative beliefs, which refer to perceptions that substance use among peers is both prevalent and acceptable (Hansen, 1993), have been found to be positively related to and predictive of substance use (Johnston et al., 2013; Stephens, Sloboda, Stephens, Teasdale, Grey, Hawthorne, et al., 2009). The games’ potential to dispel the notion that abusing prescription drugs is the norm among their peers and to decrease their willingness to condone friends’ prescription drug abuse is a promising first step in prevention efforts. The third statement among the three with the largest shift related to the idea that abusing prescription drugs is equally harmful as abusing illicit drugs. The increase in percentage of students who agreed with this statement is an important finding as it has been previously reported that adolescents view prescription drugs differently than illicit drugs, and that this perception could be the reason that traditional drug prevention measures such as life skills training fail in preventing prescription drug abuse (Boyd, McCabe, Cranford, & Young, 2006; Twombly, Holtz, & Agnew, 2011). Helping students make the connection between non-medical prescription drug use and abuse of illicit drugs might assist in eliminating the misconception that taking prescription drugs without a medical reason is safe. The second outcome of the study concerns game-related measures, which affirmed that students find these game scenarios appealing and motivating. Overall, students in both groups rated their satisfaction with the games and their level of engagement while playing the games above average. Thus, carefully crafting the learning objectives of a game and then constructing a narrative to elicit player participation were successful strategies that kept the students engaged in “solving the crime.” A comparison of the two games revealed that Fatal Interactions scored higher than Bitter Pill in both

270 / KLISCH ET AL.

satisfaction and engagement, driven by the comparatively low satisfaction and engagement ratings within the male study population in the Bitter Pill group. To gain additional insight in factors that might have influenced these game-related ratings, answers to the open-ended question “What did you like LEAST about the game?” were analyzed. One key difference was that 16.3% of the responses given by students who played Bitter Pill suggested that the game was too hard, too confusing, or too long, whereas only 9.1% of responses fell into this category in the Fatal Interactions group. Increased beta testing and a more detailed assessment of factors that influence students’ level of satisfaction and engagement would help to better understand the differences in game-related posttestonly measures. A limitation of this study is the lack of data that supports content learning. Due to the low internal consistency reliabilities of the content measures created for this study, the question whether the games could deliver both science- and health-related content effectively remains unanswered. Teachers’ primary goals are typically to convey the subject matter of their specific course. It would have been important to examine if the weaving together of forensic science and drug education did support content learning in both areas, which would be in line with results from previous evaluations of similar educational games (Bowling et al., 2013; Klisch, Miller, Beier, et al., 2012; Klisch, Miller, Wang, et al., 2012; Miller et al., 2002, 2004, 2006, 2011). In addition, more reliable content scales would have provided insight into the correlation between prescription drug attitude changes and related content learning. Previous research of a similar game showed that posttest performance on knowledge tests related to inhalants predicted a negative shift of attitudes toward inhalant abuse (Klisch, Miller, Wang, et al., 2012). Future studies will be needed to examine whether a similar direct interaction between learning about and attitudes toward prescription drug abuse can be observed. Strategies to improve the reliability of the content measures for future research could involve focusing on fewer topics that are each covered by multiple questions or probing with focus groups their interpretations of the questions. CONCLUSIONS The results presented in this study regarding the evaluation of CSI Web Adventures: Bitter Pill and Fatal Interactions suggest that including scienceand health-related information about prescription drug abuse as a subtext to forensic science can be effective in promoting healthier attitudes toward non-medical use of prescription drugs. Thus, it is possible with science-based games to achieve a drug education driven agenda. Further investigations of games as stealth vehicles for drug education, perhaps in combination with or in contrast to traditional drug prevention programs, are needed to confirm the efficacy of the approach.

GAMES TO CHANGE PRESCRIPTION DRUG ABUSE ATTITUDES /

271

APPENDIX Items for Attitude Assessment across Both Cases Attitudes toward Prescription Drug Abuse 1. Taking prescription drugs that were prescribed for somebody else is risky for your health. (reverse coded) 2. Nobody really dies from abusing prescription drugs. 3. Abusing prescription drugs is just typical teenage behavior. 4. It is okay to take prescription drugs to get high since they are legal. 5. Abusing prescription drugs is just as bad as abusing other drugs. (reverse coded) 6. Taking prescription drugs to get high is a pretty dumb idea. (reverse coded) 7. You can't really get addicted to prescription drugs. 8. There is really nothing wrong with abusing prescription drugs. 9. It is okay for people to take prescription drugs even if they are not sick, because they are safe. 10. If my friends were abusing prescription drugs, I probably wouldn't mind. Posttest-Only Items Role-play

1. My role as a CSI agent in the game seemed realistic. 2. I did NOT feel like I was part of the crime solving team in the game. (reverse coded) 3. Role-playing games about other science careers would be interesting to me. Satisfaction

1. I liked playing the game. 2. I would recommend the game to my friends. 3. The game was boring. (reverse coded) Engagement

1. 2. 3. 4.

I wanted to solve the case, no matter how hard it was. The game really grabbed my attention. The game challenged me. After a while, I stopped paying attention. (reverse coded)

Usability

1. I felt like I needed more hints on how to play the game. (reverse coded) 2. The instructions given in the game were clear to me. 3. Sometimes I was stuck and didn't know what to do next. (reverse coded)

272 / KLISCH ET AL.

ACKNOWLEDGMENTS We would like to acknowledge Dr. Cathrine Sasek at the National Institute on Drug Abuse for her support of our research. We thank the consultants who were involved in developing the games, as well as the teachers and students who participated in the field test. REFERENCES Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179-211. Barab, S. A., Scott, B., Siyahhan, S., Goldstone, R., Ingram-Goble, A., Zuiker, S. J., et al. (2009). Transformational play as a curricular scaffold: Using videogames to support science education. Journal of Science Education and Technology, 18(4), 305-320. Bowling, K. G., Klisch, Y., & Beier, M. (2013). Examining an online microbiology game as an effective tool for teaching the scientific process. Journal of Microbiology & Biology Education, 14(1), 58-65. Boyd, C. J., McCabe, S. E., Cranford, J. A., & Young, A. (2006). Adolescents’ motivations to abuse prescription medications. Pediatrics, 118(6), 2472-2480. Brown, J. S., Collins, A., & Duguid, S. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-42. Centers for Disease Control and Prevention (CDC). (2012). CDC Grand Rounds: Prescription drug overdoses—A U.S. epidemic. Morbidity and Mortality Weekly Report (MMWR). Retrieved from http://www.cdc.gov/mmwr/preview/mmwrhtml/ mm6101a3.htm?s_cid=mm6101a3_w Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Cone, E. J., Fant, R. V., Rohay, J. M., Caplan, Y. H., Ballina, M., Reder, R. F., et al. (2003). Oxycodone involvement in drug abuse deaths: A DAWN-based classification scheme applied to an oxycodone postmortem database containing over 1000 cases. Journal of Analytical Toxicology, 27(2), 57-67. Cone, E. J., Fant, R. V., Rohay, J. M., Caplan, Y. H., Ballina, M., Reder, R. F., et al. (2004). Oxycodone involvement in drug abuse deaths. II. Evidence for toxic multiple drug-drug interactions. Journal of Analytical Toxicology, 28(7), 616-624. Crano, W. D., Siegel, J. T., Alvaro, E. M., & Patel, N. M. (2007). Overcoming adolescents’ resistance to anti-inhalant appeals. Psychology of Addictive Behaviors, 21(4), 516-524. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334. Fredericks, J. A., Blumenfeld, P., Friedel, J., & Paris, A. (2005). School engagement. In K. A. Moore & L. Lippman (Eds.), What do children need to flourish? Conceptualizing and measuring indicators of positive development. New York, NY: Springer Science and Business Media. Retrieved from http://www.performwell.org/index.php/ find-surveyassessments/list-alpha/s Frey, K. P., & Eagly, A. H. (1993). Vividness can undermine the persuasiveness of messages. Journal of Personality & Social Psychology, 65(1), 32-44.

GAMES TO CHANGE PRESCRIPTION DRUG ABUSE ATTITUDES /

273

Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York, NY: Palgrave Macmillan. Glass, G. V., & Hopkins, K. D. (1996). Statistical methods in education and psychology. Boston, MA: Allyn and Bacon. Grunwald Associates, LLC. (2009). Digitally inclined. Retrieved from http://www.pbs.org/ teachers/_files/pdf/annual-pbs-survey-report.pdf Halpern, D. F., Millis, K., Graesser, A. C., Butler, H., Forsyth, C., & Cai, Z. (2012). Operation ARA: A computerized learning game that teaches critical thinking and scientific reasoning. Thinking Skills and Creativity, 7(2), 93-100. Hansen, W. B. (1993). School-based alcohol prevention programs. Alcohol Health & Research World, 17(1), 54-60. Hernandez, S. H., & Nelson, L. S. (2010). Prescription drug abuse: Insight into the epidemic. Clinical Pharmacology and Therapeutics, 88(3), 307-317. Holtz, K. D., & Twombly, E. C. (2007). A preliminary evaluation of the effects of a science education curriculum on changes in knowledge of drugs in youth. Journal of Drug Education, 37(3), 317-333. Huang, B., Dawson, D. A., Stinson, F. S., Hasin, D. S., Ruan, W. J., Saha, T. D., et al. (2006). Prevalence, correlates, and comorbidity of non-medical prescription drug use and drug use disorders in the United States: Results of the national epidemiologic survey on alcohol and related conditions. Journal of Clinical Psychiatry, 97(7), 1062-1073. Ito, M., Horst, H., Bittanti, M., Boyd, D., Herr-Stephenson, B., Lange, P. G., et al. (2008). Living and learning with new media: Summary of findings from the digital youth project. Retrieved from http://www.macfound.org/press/publications/living-andlearning-with-new-media-summary-of-findings-from-the-digital-youth-project/ Johnston, L. D., O’Malley, P. M., Bachman, J. G., & Schulenberg, J. E. (2013). Monitoring the Future national survey results on drug use, 1975-2012: Volume I, Secondary school students. Ann Arbor. MI: Institute for Social Research, The University of Michigan. Retrieved from http://www.monitoringthefuture.org/ Kann, L., Telljohann, S. K., & Wooley, S. F. (2007). Health education: Results from the school health policies and programs study 2006. Journal of School Health, 77(8), 408-434. Klisch, Y., Miller, L., Beier, M., & Wang, S. (2012). Teaching the biological consequences of alcohol abuse through an online game: Impacts among secondary students. CBE Life Sciences Education, 11(1), 94-102. Klisch, Y., Miller, L., Wang, S., & Epstein, J. (2012). The impact of a science education game on students’ learning and perception of inhalants as body pollutants. Journal of Science Education and Technology, 21(2), 295-303. Lenhart, A., Kahne, J., Middaugh, E., Macgill, A., Evans, C., & Vitak, J. (2008). Teens, video games and civics. Retrieved from http://www.pewinternet.org/Reports/2008/ Teens-Video-Games-and-Civics.aspx McGuire, W. J. (1964). Inducing resistance to persuasion. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 1, pp. 191-229). New York, NY: Academic Press. Meluso, A., Zheng, M., Spires, H. A., & Lester, J. (2012). Enhancing 5th graders’ science content knowledge and self-efficacy through game-based learning. Computers & Education, 59(2), 497-504.

274 / KLISCH ET AL.

Miller, L. M., Chang, C.-I., Wang, S., Beier, M., & Klisch, Y. (2011). Learning and motivational impacts of a multimedia science game. Computers & Education, 57(1), 1425-1433. Miller, L. M., Moreno, J., Estrera, V., & Lane, D. (2004). Efficacy of MedMyst: An internet teaching tool for middle school microbiology. Journal of Microbiology & Biology Education, 5, 13-20. Miller, L. M., Moreno, J., Willcockson, I., Smith, D., & Mayes, J. (2006). An online, interactive approach to teaching neuroscience to adolescents. CBE Life Sciences Education, 5(2), 137-143. Miller, L. M., Schweingruber, H., Oliver, R., Mayes, J., & Smith, D. (2002). Teaching neuroscience through web adventures: Adolescents reconstruct the history and science of opioids. The Neuroscientist, 8(1), 16-21. Millstone, J., The Joan Ganz Cooney Center at Sesame Workshop in collaboration with BrainPOP. (2012). Teacher attitudes about digital games in the classroom. Retrieved from http://www.joanganzcooneycenter.org/publications/ National Institute on Drug Abuse. (2011). Topics in brief: Prescription drug abuse. Retrieved from http://www.drugabuse.gov/publications/topics-in-brief/prescriptiondrug-abuse Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, NY: McGraw-Hill. Project Tomorrow. (2012). Mapping a personalized learning journey—K-12 students and parents connect the dots with digital learning. Speak Up 2011. National findings, K-12 students & parents. Retrieved from http://www.tomorrow.org/speakup/2012_ PersonalizedLearning.html Rideout, V. G., Foehr, U. G., & Roberts, D. F. (2010). Generation M2: Media in the lives of 8-18-year olds. Retrieved from http://www.kff.org/entmedia/8010.cfm Shegog, R., Lazarus, M. M., Murray, N. G., Diamond, P. M., Sessions, N., & Zsigmond, E. (2012). Virtual transgenics: Using a molecular biology simulation to impact student academic achievement and attitudes. Research in Science Education, 42(5), 875-890. Shelton, B. E., & Scoresby, J. (2011). Aligning game activity with educational goals: Following a constrained design approach to instructional computer games. Educational Technology Research and Development, 59(1), 113-138. Squire, K. D. (2006). From content to context: Videogames as designed experiences. Educational Researcher, 35(8), 19-29. Stephens, P. C., Sloboda, Z., Stephens, R. C., Teasdale, B., Grey, S. F., Hawthorne, R. D., et al. (2009). Universal school-based substance abuse prevention programs: Modeling targeted mediators and outcomes for adolescent cigarette, alcohol, and marijuana use. Drug and Alcohol Dependence, 102(1-3), 19-29. Substance Abuse and Mental Health Services Administration (SAMHSA). (2013). Results from the 2012 national survey on drug use and health: Detailed tables. Retrieved on September 16, 2013, from http://www.samhsa.gov/data/NSDUH.aspx Twombly, E. C., Holtz, K. D., & Agnew, C. B. (2011). Resonant messages to prevent prescription drug misuse by teens. Journal of Alcohol & Drug Education, 55(1), 38-52. Wood, W., & Quinn, J. M. (2003). Forewarned and forearmed? Two meta-analytic syntheses of forewarnings of influence appeals. Psychological Bulletin, 129(1), 119-138.

GAMES TO CHANGE PRESCRIPTION DRUG ABUSE ATTITUDES /

275

Wu, H.-L., & Pedersen, S. (2011). Integrating computer- and teacher-based scaffolds in science inquiry. Computers & Education, 57(4), 2352-2363.

Direct reprint requests to: Yvonne Klisch, Ph.D. Rice University Center for Technology in Teaching and Learning 6100 Main Street, MS 120 Houston, TX 77005 e-mail: [email protected]

Copyright of Journal of Drug Education is the property of Baywood Publishing Company, Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.