Research Brief Intention to Change Nutrition-Related Behaviors in Adult Participants of a Supplemental Nutrition Assistance Program–Education Mateja R. Savoie, MPH, RD1,2; Monica Mispireta, MHS, MD1; Linda L. Rankin, PhD, RD1; Karen Neill, PhD, RN, SANE-A3; Heidi LeBlanc, MS, CFCS2; Debra Christofferson, MDA, RD2 ABSTRACT Objective: To determine whether participation in selected Supplemental Nutrition Assistance Program– Education (SNAP-Ed) lessons had an impact on the intent to improve nutrition-related behaviors of participants. Methods: A quantitative study using a retrospective post-then-pre design to measure SNAP-Ed outcomes of 203 adult participants after selected nutrition lessons in 14 counties across the state of Utah. After the intervention participants completed a retrospective post-then-pre survey evaluating intent to improve nutrition behaviors related to the SNAP-Ed lessons. Wilcoxon signed rank test with Bonferroni correction and paired t test were used. Results: Participants reported sometimes engaging in nutrition related behaviors before attending SNAPEd lessons and intent to usually engage in these behaviors after attending SNAP-Ed lessons. Conclusion and Implications: This study demonstrated that participation in selected SNAP-Ed lessons was positively related to the intent of participants to improve nutrition-related behaviors. Key Words: Supplemental Nutrition Assistance Program–Education, nutrition education, program evaluation, Theory of Planned Behavior, fruit, vegetable, grains, activity (J Nutr Educ Behav. 2015;47:81-85.) Accepted August 11, 2014. Published online September 27, 2014.
INTRODUCTION The Supplemental Nutrition Assistance Program-Education (SNAP–Ed), a primary component of the Supplemental Nutrition Assistance Program (SNAP), is the largest nutrition education program in the US.1 In 2010, the budget for SNAP-Ed was $372 million, with the program reaching over 41 million people.2,3 The goal of SNAPEd is to improve nutritional intake among SNAP participants and other low-income individuals by promoting healthy habits consistent with current Dietary Guidelines for Americans (DGA).2 In the state of Utah, SNAPEd provides evidence-based outcomedriven nutrition education for
participants in a large group setting. Lessons are developed by registered dietitians and delivered by certified paraprofessional nutrition education assistants (NEAs). Nutrition education assistants are prepared to deliver this type of education through training in the National Nutrition Paraprofessional Certification Program.4 Nutrition education is considered an important component of food assistance programs but it has been widely recognized that evaluation of their impact needs improvement.5 Although SNAP-Ed is the largest federally funded nutrition education program in the US, few studies have been published on program outcomes. Important to quality improve-
1
Kasiska School of Health Professions, Pocatello, ID Department of Nutrition, Dietetics, and Food Sciences, Utah State University, Logan, UT 3 School of Nursing, Idaho State University, Pocatello, ID Address for correspondence: Mateja R. Savoie, MPH, RD, Department of Nutrition, Dietetics, and Food Sciences, Utah State University, 8749 Old Main Hill Rd, Logan, UT 84322; Phone: (218) 766-1496; Fax: (435) 797-0897; E-mail: [email protected] Ó2015 SOCIETY FOR NUTRITION EDUCATION AND BEHAVIOR http://dx.doi.org/10.1016/j.jneb.2014.08.009 2
Journal of Nutrition Education and Behavior Volume 47, Number 1, 2015
ment and effective continuous program implementation, evaluation of outcomes can facilitate successful replication of SNAP-Ed programs across the nation. Demonstrating program effectiveness is important to continued budget allocation, program growth, and expanded implementation. The purpose of this study was to use the Theory of Planned Behavior to assess whether participation in selected SNAP-Ed lessons had an impact on the intent of participants to improve nutrition-related behaviors.
METHODS Theoretical Application The use of theoretical models to evaluate behavior change has been suggested as a strategy to improve the evaluation process of SNAP-Ed programs.6 This study was designed using the Theory of Planned Behavior. In this theory, 3 constructs explain the process of behavior change: perceived behavioral control, intention to change, and subjective norm.7 This study incorporated the first 2
81
82 Savoie et al constructs, which are considered to be the most proximal determinants of behavior change.8 The intent to change construct was used as the foundation for the evaluation process. Because intentions can be expected to engender behaviors only if an individual has control over the behavior, a wide variety of optional classes were made available to SNAP participants using a flexible schedule. This allowed participants to have control over the type and number of classes attended, and when they attended (perceived behavioral control).
Study Design The researchers used a retrospective post-then-pre design to assess whether participation in SNAP-Ed had an impact on the intent to change nutrition-related behaviors. This approach used the determination of intention to change behaviors as a predictor of future behavior change. After each lesson was completed, retrospective post-then-pre questionnaires were administered sequentially to each participant. This design is different from the traditional pretest– posttest design in which participants complete a pretest before the intervention and a posttest after its completion.9 Although the pretest–posttest design is commonly used, estimates obtained can be biased because of to pretest overestimation, which can threaten internal validity.10,11 A retrospective post-then-pre design minimizes this type of bias and reduces incomplete data because retrospective post-then pre questionnaires are administered sequentially.10,12
Participants and Recruitment The protocol for this study was approved by the Institutional Review Board of Utah State University, and the Human Subjects Committee at Idaho State University. A convenience sample of 203 participants aged $ 18 years who attended selected SNAP-Ed classes across the state of Utah in 2012 was obtained for the study. Participants were recruited through the placement of flyers detailing the lessons in SNAPEd classes, public health departments, extension offices, food pantries, and other locations in which services are provided to low-income individuals.
Journal of Nutrition Education and Behavior Volume 47, Number 1, 2015
Description of Intervention This study focused on Utah's SNAP-Ed Cooking Basics curriculum. Specifically selected lessons include Menu Planning and Shopping and the MyPlate lessons, which are the most representative of the current DGA. The Menu Planning and Shopping lesson emphasized planning for shopping and cooking, and patterns of meal intake. The MyPlate lesson emphasized the amount and frequency of consumption of fruit, vegetables, whole grains, dairy, protein, fat, and salt. Lessons were delivered using a variety of educational methods including a lecture component, a cooking demonstration, and a sample tasting of the food prepared in class. In addition, a handout was provided with recipes and facts from the lesson to allow for reflection on learning outcomes. Recipes taught in the lessons were developed to use minimal resources and short cooking times, which can foster translation into healthy nutritional habits and behaviors outside the classroom.
Instrument The questionnaire used in this study was developed by faculty at Utah State University. The questionnaire included demographic, SNAP eligibility, SNAP-Ed participation, and intention to change questions. Questions were designed and developed to measure the intention to change nutrition-related behaviors. Questions developed for the pretest emphasized current nutrition-related behaviors of participants. Questions developed for the posttest focused on intent to change the same behaviors as a result of what was learned in lessons attended by participants. There were 7 questions related to the Menu Planning and Shopping lesson and 8 related to the MyPlate lesson. Possible responses to the questions were formatted on a 5-point Likert-type scale consisting of never, seldom, sometimes, usually, and always. The questionnaire took approximately 5 minutes for participants to complete. To assess the content validity of the questions, 3 NEAs were asked to independently review the material covered in each lesson and the questions developed for the questionnaire
to measure outcomes of each lesson. The NEAs consistently reported that the lesson content for the Menu Planning and Shopping lesson and the MyPlate lesson matched the designated questions, supporting content validity.4 Participants completed the questionnaire after attending either the Menu Planning and Shopping or the MyPlate lesson. Participants were surveyed through sequential completion of the retrospective post-thenpre questionnaire, which included all questions related to the content covered in each lesson as well as additional questions as described above. The NEAs provided participants with detailed instructions on questionnaire completion and assisted participants with low literacy levels.
Data Analysis In accordance with the perceived behavioral control construct, participants were allowed to determine what SNAP-Ed class they attended. A total of 100 participants chose to attend the Menu Planning and Shopping lesson and 103 chose to attend the MyPlate lesson. Participants who answered < 5 questions per lesson were not included in the creation of the lesson scores. Researchers determined that a response rate < 5 would decrease the validity of the questionnaire results for that participant. One participant attending the Menu Planning and Shopping lesson and 16 participants attending the MyPlate lesson were excluded from the calculation of lesson scores owing to their limited responses to the questionnaire. The analytical sample for this analysis was 186: 99 for the Menu Planning and Shopping lesson and 87 for the MyPlate lesson. The authors estimated the sample size for this study using Power Analysis and Sample Size (PASS) software (version 8.0, NCSS LLC, Kaysville, UT, 2008). It was estimated that with an overall sample of 83 participants per lesson, the study had 80% power at a .05 significance level to detect an effect size $ 0.31 between the mean posttest and pretest lesson scores for each individual lesson. Responses from the retrospective post-then-pre questionnaires were compared for each lesson to assess the
Journal of Nutrition Education and Behavior Volume 47, Number 1, 2015 impact of participation in SNAP-Ed lessons on the intent of participants to change nutrition-related behaviors. The researchers compared responses from the pretest and posttest for each individual question using the Wilcoxon signed-rank test. To determine the impact of each lesson as a whole, pretest and posttest lesson scores were created by calculating the arithmetic mean of all available responses for the pretest and posttest questions for each of the 2 lessons assessed in this study, and then compared. Internal consistency of questions within each lesson was assessed using Cronbach alpha. Cronbach alpha coefficients for the retrospective post-then-pre questionnaires for both lessons were all > .7, which suggested moderate internal consistency and therefore validated the creation of the lesson scores. The Likert-type scale was measured from 1 to 5, with 1 ¼ never to 5 ¼ always. Wilcoxon signed-rank test with Bonferroni correction was used to assess the difference in median responses of the pretest and posttest results for each individual question. The researchers used paired t test to assess the difference in mean responses of the pretest and posttest lesson score for each of the 2 lessons. Data were analyzed using SPSS 21.0 (SPSS, Inc, Chicago, IL, 2012).
RESULTS Table 1 shows selected characteristics of study participants. The majority of participants were non-Hispanic, Caucasian women. Age of participants ranged from 18 to over 60 years with the largest proportion being between 18 and 34 years. More than 50% reported having $ 1 children under age 18 years in the household and qualifying or receiving federal assistance. A total of 65% of participants reported this was the first time they had taken a SNAP-Ed class. Table 2 compares the pre and post mean, median, SDs, and interquartile ranges for each of the 7 individual questions and the lesson score related to the Menu Planning and Shopping lesson. To attempt to determine whether this lesson increased the intention to change the specific nutrition-related behaviors taught, pre and posttest responses were compared. The mean responses of
Table 1. Sociodemographic Characteristics of Adult Supplemental Nutrition Assistance Program–Education Participants in Utah (n ¼ 203) Characteristics Female gender Age, y 18–34 35–59 $ 60
Total (%) 78.6 41.2 33.7 25.1
Race American Indian/ Alaska Native Asian Black/African American White Other Multiracial Hispanic ethnicity
79.4 8.9 8.0 27.7
Children age # 18 y in household None 1–2 3–4 5–6 $7
41.3 28.5 25.7 3.9 0.6
SNAP benefits Receiving SNAP benefits Qualify for SNAP benefits Receive other assistance Lesson MyPlate Menu Planning and Shopping
2.6 1.0 0.1
46.7 54.4 52.2
50.7 49.3
SNAP indicates Supplemental Nutrition Assistance Program. the individual questions from pretest to posttest increased significantly (P < .001), as did the mean of the lesson score. In general, participants reported before attending the Menu Planning and Shopping lesson that they sometimes engaged in the specific behaviors evaluated; after attending the lesson, participants reported they would usually engage in those behaviors. Table 3 compares the pre and post mean, median, SD, and interquartile ranges for each of the 8 individual questions and the lesson score related to the MyPlate lesson. To determine whether this lesson increased the
Savoie et al 83 intention to change the specific nutrition-related behaviors taught, the researchers conducted this analysis and compared pre- and posttest responses. Mean responses to individual questions from pretest to posttest increased significantly (P < .001), as did the mean of the lesson score. Results indicated that before attending the MyPlate lesson, participants sometimes engaged in the specific behaviors evaluated; after attending the lesson, they reported that they would usually engage in those behaviors.
DISCUSSION Results demonstrate that participation in either of 2 selected SNAP-Ed lessons had a positive impact on the reported intent of participants to improve nutrition-related behaviors consistent with the 2010 DGA. Results of this study are consistent with previous findings. Previous studies indicated a positive impact of nutrition education on fruit and vegetable consumption among low-income individuals, and further expanded these findings to other dietary habits in a sample of SNAP-Ed participants.13,14 Findings indicated that participation in federally funded nutrition education programs can change the intentions to improve nutrition-related behaviors and increase the consumption of healthy foods including fruits, vegetables, and whole grains.14 The current study builds on the recent literature that demonstrated the impact of federally funded nutrition education on changing behavioral intentions and actual consumption of fruits and vegetables in low-income individuals.13,14 These results provide evidence that nutrition education for federally funded programs targeted to lowincome individuals may be effective in improving reported intent related to healthy nutrition-related behaviors and potentially changing behaviors. Despite the strengths of this study, some limitations need to be mentioned. This study investigated intention to change behaviors instead of actual behavior change. According to the Theory of Planned Behavior, intention is considered to be the largest motivating factor for behavior change.6 Numerous observational and experimental studies have shown a
84 Savoie et al
Journal of Nutrition Education and Behavior Volume 47, Number 1, 2015
Table 2. Reported Intention to Change Nutrition-Related Behaviors in Adult Participants of a Menu Planning and Shopping Supplemental Nutrition Assistance Program–Education Lesson Pretest
Menu Planning and Shopping Lesson
Mean ± SD
Posttest
Median (interquartile range)
Mean ± SD
P*
Median (interquartile range)
a
Questions Have enough food to last through end of month Plan menu before shopping Shop with grocery list Make food purchases based on nutrition facts label Prepare meals at home at least 3 times/wk Eat meals together as family at least 3 times/wk Eat breakfast within 2 h of waking Lesson scoreb
3.5 1.2 2.7 1.2 3.4 1.2 2.8 1.2 3.9 1.2 3.8 1.2 3.3 1.3 3.3 0.8
3 (3–5) 3 (2–5) 3 (3–4) 3 (2–4) 4 (3–5) 4 (3–5) 3.5 (3–5)
4.1 1.0 3.9 0.9 4.1 0.9 3.7 1.0 4.3 1.0 4.2 1.1 4.0 1.1 4.0 0.8
4 (4–5) 4 (3–5) 4 (4–5) 4 (3–5) 5 (4–5) 4 (4–5) 4 (4–5)
< .001 < .001 < .001 < .001 < .001 < .001 < .001 < .001
*P < .05 is considered significant. a Comparisons were performed using Wilcoxon signed-rank test; bComparisons were performed using paired t test. Note: Values are from a Likert scale (1 ¼ never, 2 ¼ seldom, 3 ¼ sometimes, 4 ¼ usually, 5 ¼ always). positive association between changes in behavioral intentions and subsequent behavior change, which supports this proposition.15,16 However, it has also been argued that estimates of intention to change can overestimate actual behavior changes. A recent meta-analysis of randomized controlled trials examining this association reported that on average, medium to large changes in intention generate small to medium changes in behavior.17 Thus, the effect sizes presented in this study might be overestimating the impact of the intervention
on actual behavior change. The post-then-pre questionnaire used in this study is a self-reported instrument. Self-reported instruments are subjective measurements that estimate participant report of intention to change behavior, which increases the risk for bias and may cause overestimation of behavioral intention.11 The researchers did not use a control group, which limits the causal inference of the results. Improvements in future behaviors reported by participants might be a consequence of their awareness of being studied
(Hawthorne effect).18 Owing to the unique design of the study, participants may not have fully understood how to fill out the questionnaires, which could also contribute to measurement error. This study focused on the Menu Planning and Shopping and the MyPlate lessons only; therefore, the impact of the greater nutrition education curricula of the SNAP-Ed program was not evaluated. Data were collected from participants during 1 data collection period. Therefore, no follow-up data were available to demonstrate the
Table 3. Reported Intention to Change Nutrition-Related Behaviors in Adult Participants of a MyPlate Supplemental Nutrition Assistance Program–Education Lesson in Utah Pretest
MyPlate Lesson Questionsa Eat at least 2.5 cups/d of vegetables Eat at least 2 cups/d of fruit Eat more whole grains than refined grains Eat at least 3 cups/d or equivalent of dairy Choose at least 2 servings of lean protein Replace saturated and trans fats with heart-healthy fats Choose to be physically active for at least 30 min, 5 d/wk Prepare foods without adding salt Lesson scoreb
Posttest
P
Median Median (interquartile (interquartile range) range) Mean ± SD Mean ± SD 3.4 1.1 3.4 1.0 3.5 1.1 3.3 1.2 3.4 1.1 3.1 1.2 3.4 1.2 3.3 1.1 3.3 0.8
4 (3–4) 3 (3–4) 4 (3–4) 3 (3–4) 3 (3–4) 3 (2–4) 4 (3–4) 3 (2–4)
4.3 0.9 4.1 0.9 4.2 0.9 4.0 1.0 4.2 0.9 4.1 1.0 4.2 1.0 4.0 1.1 4.0 0.8
4 (4–5) 4 (4–5) 4 (4–5) 4 (4–5) 4 (4–5) 4 (4–5) 4 (4–5) 4 (3–5)
< .001 < .001 < .001 < .001 < .001 < .001 < .001 < .001 < .001
Comparisons were performed using Wilcoxon signed-rank test; bComparisons were performed using paired t test; P < .05 is considered significant. Note: Values are from a Likert scale (1 ¼ never, 2 ¼ seldom, 3 ¼ sometimes, 4 ¼ usually, 5 ¼ always). a
Journal of Nutrition Education and Behavior Volume 47, Number 1, 2015 frequency of SNAP-Ed participation after attending either of the 2 selected lessons. Data on continued participation would have strengthened the results of this study by further concluding intention to change nutrition-related behaviors. Finally, although over half of participants reported this was their first SNAP-Ed lesson, there was diversity in the number of lessons participants had taken previously. It is possible that participants who had attended multiple lessons had greater intentions to change compared with new participants.
IMPLICATIONS FOR RESEARCH AND PRACTICE Findings from this study have direct implications for continued implementation of these SNAP-Ed lessons. Supplemental Nutrition Assistance Program–Education programs may consider replicating the retrospective post-then-pre design as an initial measurement to determine behavioral intentions of SNAP-Ed participants given the readability, convenience, and cost effectiveness of this design.19 The positive association between changes in behavioral intentions and subsequent behavior change makes this design a feasible and valuable evaluation method for SNAP-Ed programs. Future research should investigate the long-term indication of behavior change. Programs could enhance research by collecting data on actual behavior change to investigate how intentions to change nutrition-related behaviors translate into behavior change in this population. Future studies may incorporate telephone call surveys several weeks after completion of the SNAP-Ed class for follow-up data. Participants could also send in grocery receipts to determine purchasing behavior before and after SNAP-Ed education. This study provides validated evaluation materials that can be adopted by SNAP-Ed programs to assess similar interventions. Results from this study, together with those previously demonstrating a positive impact of nutrition education on nutrition-related behaviors, highlight the importance of continued research of federally funded nutrition education programs.
Demonstrating program impact through research and program evaluation supports future research opportunities for the SNAP-Ed program. Further studies should focus on assessing the long-term effect of nutrition education on nutrition-related behavior change among SNAP-Ed participants and their families and evaluate the impact of individual components of nutrition education on actual behavior change.
ACKNOWLEDGMENTS The authors greatly appreciate Teri Peterson for assistance with statistical analysis. The authors also thank Utah State University for allowing researchers to use the Utah SNAP-Ed data for this study, and SNAP-Ed participants in Utah for willingness to complete this survey. This study was completed as a thesis by Mateja R. Savoie and faculty at Idaho State University.
REFERENCES 1. US Department of Agriculture, Food, and Nutrition Service. Supplemental nutrition assistance program education guidance: nutrition education and obesity prevention grant program. https://www .extension.org/sites/default/files/FY13 Guidance.pdf. Accessed July 12, 2014. 2. US Department of Agriculture, Food, and Nutrition Service. Supplemental nutrition assistance program: number of persons participating. http://www .fns.usda.gov/pd/29SNAPcurrPP.htm. Accessed July 12, 2014. 3. US Department of Agriculture, Food, and Nutrition Service. EARS overview. http://snap.nal.usda.gov/national-snaped/education-and-administrative-repor ting-system/ears-overview. Accessed July 12, 2014. 4. Christofferson D, Christensen N, LeBlanc H, Bunch M. Developing an online certification program for nutrition education assistants. J Nutr Educ Behav. 2012;44:407-414. 5. US Department of General Accounting Office. Nutrition education: USDA provides services through multiple programs but stronger linkage among efforts are needed. http://www.gao.gov/assets/250/ 242135.pdf. Accessed July 12, 2014. 6. Guthrie JF, Stommes E, Voichick J. Evaluating food stamp nutrition
Savoie et al 85
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
education: issues and opportunities. J Nutr Educ Behav. 2006;38:6-11. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50:179-211. Kothe EJ, Mullan BA, Butow P. Promoting fruit and vegetable consumption: testing an intervention based on the theory of planned behaviour. Appetite. 2012;58:997-1004. Howard G, Dailey P. Response-shift bias: a source of contamination of selfreport measures. J Appl Psychol. 1979; 64:144-150. Howard G, Ralph K, Gulanick N, Maxwell S, Nance D, Gerber S. Internal invalidity in pretest-posttest selfreport evaluations and a re-evaluation of retrospective pretests. Appl Psychol Meas. 1979;3:1-23. Pratt CC, McGuigan WM, Katzev AR. Measuring program outcomes: using retrospective pretest methodology. Am J Eval. 2000;21:341-349. Raidl M, Johnson S, Gardiner K, et al. Use retrospective surveys to obtain complete data sets and measure impact in extension programs. J Extension. 2004;42(2). Heneman K, Block-Joy A, ZidenbergCherr S, et al. A ‘‘Contract for Change’’ increases produce consumption in lowincome women: a pilot study. A Am Diet Assoc. 2005;105:1793-1796. Ritchie LD, Whaley SE, Spector P, Gomez J, Crawford PB. Favorable impact of nutrition education on California WIC families. J Nutr Educ Behav. 2010;42:S2-S10. Kim KW, Reicks M, Sjoberg S. Applying the theory of planned behavior to predict dairy product consumption by older adults. J Nutr Educ Behav. 2003;35:294-301. Armitage CJ, Conner M. Efficacy of the theory of planned behaviour: a meta-analytic review. Br J Soc Psychol. 2001;40:471-499. Webb TL, Sheeran P. Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychol Bull. 2006;132:249-268. Campbell MK, Honess-Morreale L, Farrell D, Carbone E, Brasure M. A tailored multimedia nutrition education pilot program for low-income women receiving food assistance. Health Educ Res. 1999;14:257-267. Davis G. Using a retrospective pre-post questionnaire to determine program impact. J Extension. 2003;41(4).
INTRODUCTION The Supplemental Nutrition Assistance Program-Education (SNAP–Ed), a primary component of the Supplemental Nutrition Assistance Program (SNAP), is the largest nutrition education program in the US.1 In 2010, the budget for SNAP-Ed was $372 million, with the program reaching over 41 million people.2,3 The goal of SNAPEd is to improve nutritional intake among SNAP participants and other low-income individuals by promoting healthy habits consistent with current Dietary Guidelines for Americans (DGA).2 In the state of Utah, SNAPEd provides evidence-based outcomedriven nutrition education for
participants in a large group setting. Lessons are developed by registered dietitians and delivered by certified paraprofessional nutrition education assistants (NEAs). Nutrition education assistants are prepared to deliver this type of education through training in the National Nutrition Paraprofessional Certification Program.4 Nutrition education is considered an important component of food assistance programs but it has been widely recognized that evaluation of their impact needs improvement.5 Although SNAP-Ed is the largest federally funded nutrition education program in the US, few studies have been published on program outcomes. Important to quality improve-
1
Kasiska School of Health Professions, Pocatello, ID Department of Nutrition, Dietetics, and Food Sciences, Utah State University, Logan, UT 3 School of Nursing, Idaho State University, Pocatello, ID Address for correspondence: Mateja R. Savoie, MPH, RD, Department of Nutrition, Dietetics, and Food Sciences, Utah State University, 8749 Old Main Hill Rd, Logan, UT 84322; Phone: (218) 766-1496; Fax: (435) 797-0897; E-mail: [email protected] Ó2015 SOCIETY FOR NUTRITION EDUCATION AND BEHAVIOR http://dx.doi.org/10.1016/j.jneb.2014.08.009 2
Journal of Nutrition Education and Behavior Volume 47, Number 1, 2015
ment and effective continuous program implementation, evaluation of outcomes can facilitate successful replication of SNAP-Ed programs across the nation. Demonstrating program effectiveness is important to continued budget allocation, program growth, and expanded implementation. The purpose of this study was to use the Theory of Planned Behavior to assess whether participation in selected SNAP-Ed lessons had an impact on the intent of participants to improve nutrition-related behaviors.
METHODS Theoretical Application The use of theoretical models to evaluate behavior change has been suggested as a strategy to improve the evaluation process of SNAP-Ed programs.6 This study was designed using the Theory of Planned Behavior. In this theory, 3 constructs explain the process of behavior change: perceived behavioral control, intention to change, and subjective norm.7 This study incorporated the first 2
81
82 Savoie et al constructs, which are considered to be the most proximal determinants of behavior change.8 The intent to change construct was used as the foundation for the evaluation process. Because intentions can be expected to engender behaviors only if an individual has control over the behavior, a wide variety of optional classes were made available to SNAP participants using a flexible schedule. This allowed participants to have control over the type and number of classes attended, and when they attended (perceived behavioral control).
Study Design The researchers used a retrospective post-then-pre design to assess whether participation in SNAP-Ed had an impact on the intent to change nutrition-related behaviors. This approach used the determination of intention to change behaviors as a predictor of future behavior change. After each lesson was completed, retrospective post-then-pre questionnaires were administered sequentially to each participant. This design is different from the traditional pretest– posttest design in which participants complete a pretest before the intervention and a posttest after its completion.9 Although the pretest–posttest design is commonly used, estimates obtained can be biased because of to pretest overestimation, which can threaten internal validity.10,11 A retrospective post-then-pre design minimizes this type of bias and reduces incomplete data because retrospective post-then pre questionnaires are administered sequentially.10,12
Participants and Recruitment The protocol for this study was approved by the Institutional Review Board of Utah State University, and the Human Subjects Committee at Idaho State University. A convenience sample of 203 participants aged $ 18 years who attended selected SNAP-Ed classes across the state of Utah in 2012 was obtained for the study. Participants were recruited through the placement of flyers detailing the lessons in SNAPEd classes, public health departments, extension offices, food pantries, and other locations in which services are provided to low-income individuals.
Journal of Nutrition Education and Behavior Volume 47, Number 1, 2015
Description of Intervention This study focused on Utah's SNAP-Ed Cooking Basics curriculum. Specifically selected lessons include Menu Planning and Shopping and the MyPlate lessons, which are the most representative of the current DGA. The Menu Planning and Shopping lesson emphasized planning for shopping and cooking, and patterns of meal intake. The MyPlate lesson emphasized the amount and frequency of consumption of fruit, vegetables, whole grains, dairy, protein, fat, and salt. Lessons were delivered using a variety of educational methods including a lecture component, a cooking demonstration, and a sample tasting of the food prepared in class. In addition, a handout was provided with recipes and facts from the lesson to allow for reflection on learning outcomes. Recipes taught in the lessons were developed to use minimal resources and short cooking times, which can foster translation into healthy nutritional habits and behaviors outside the classroom.
Instrument The questionnaire used in this study was developed by faculty at Utah State University. The questionnaire included demographic, SNAP eligibility, SNAP-Ed participation, and intention to change questions. Questions were designed and developed to measure the intention to change nutrition-related behaviors. Questions developed for the pretest emphasized current nutrition-related behaviors of participants. Questions developed for the posttest focused on intent to change the same behaviors as a result of what was learned in lessons attended by participants. There were 7 questions related to the Menu Planning and Shopping lesson and 8 related to the MyPlate lesson. Possible responses to the questions were formatted on a 5-point Likert-type scale consisting of never, seldom, sometimes, usually, and always. The questionnaire took approximately 5 minutes for participants to complete. To assess the content validity of the questions, 3 NEAs were asked to independently review the material covered in each lesson and the questions developed for the questionnaire
to measure outcomes of each lesson. The NEAs consistently reported that the lesson content for the Menu Planning and Shopping lesson and the MyPlate lesson matched the designated questions, supporting content validity.4 Participants completed the questionnaire after attending either the Menu Planning and Shopping or the MyPlate lesson. Participants were surveyed through sequential completion of the retrospective post-thenpre questionnaire, which included all questions related to the content covered in each lesson as well as additional questions as described above. The NEAs provided participants with detailed instructions on questionnaire completion and assisted participants with low literacy levels.
Data Analysis In accordance with the perceived behavioral control construct, participants were allowed to determine what SNAP-Ed class they attended. A total of 100 participants chose to attend the Menu Planning and Shopping lesson and 103 chose to attend the MyPlate lesson. Participants who answered < 5 questions per lesson were not included in the creation of the lesson scores. Researchers determined that a response rate < 5 would decrease the validity of the questionnaire results for that participant. One participant attending the Menu Planning and Shopping lesson and 16 participants attending the MyPlate lesson were excluded from the calculation of lesson scores owing to their limited responses to the questionnaire. The analytical sample for this analysis was 186: 99 for the Menu Planning and Shopping lesson and 87 for the MyPlate lesson. The authors estimated the sample size for this study using Power Analysis and Sample Size (PASS) software (version 8.0, NCSS LLC, Kaysville, UT, 2008). It was estimated that with an overall sample of 83 participants per lesson, the study had 80% power at a .05 significance level to detect an effect size $ 0.31 between the mean posttest and pretest lesson scores for each individual lesson. Responses from the retrospective post-then-pre questionnaires were compared for each lesson to assess the
Journal of Nutrition Education and Behavior Volume 47, Number 1, 2015 impact of participation in SNAP-Ed lessons on the intent of participants to change nutrition-related behaviors. The researchers compared responses from the pretest and posttest for each individual question using the Wilcoxon signed-rank test. To determine the impact of each lesson as a whole, pretest and posttest lesson scores were created by calculating the arithmetic mean of all available responses for the pretest and posttest questions for each of the 2 lessons assessed in this study, and then compared. Internal consistency of questions within each lesson was assessed using Cronbach alpha. Cronbach alpha coefficients for the retrospective post-then-pre questionnaires for both lessons were all > .7, which suggested moderate internal consistency and therefore validated the creation of the lesson scores. The Likert-type scale was measured from 1 to 5, with 1 ¼ never to 5 ¼ always. Wilcoxon signed-rank test with Bonferroni correction was used to assess the difference in median responses of the pretest and posttest results for each individual question. The researchers used paired t test to assess the difference in mean responses of the pretest and posttest lesson score for each of the 2 lessons. Data were analyzed using SPSS 21.0 (SPSS, Inc, Chicago, IL, 2012).
RESULTS Table 1 shows selected characteristics of study participants. The majority of participants were non-Hispanic, Caucasian women. Age of participants ranged from 18 to over 60 years with the largest proportion being between 18 and 34 years. More than 50% reported having $ 1 children under age 18 years in the household and qualifying or receiving federal assistance. A total of 65% of participants reported this was the first time they had taken a SNAP-Ed class. Table 2 compares the pre and post mean, median, SDs, and interquartile ranges for each of the 7 individual questions and the lesson score related to the Menu Planning and Shopping lesson. To attempt to determine whether this lesson increased the intention to change the specific nutrition-related behaviors taught, pre and posttest responses were compared. The mean responses of
Table 1. Sociodemographic Characteristics of Adult Supplemental Nutrition Assistance Program–Education Participants in Utah (n ¼ 203) Characteristics Female gender Age, y 18–34 35–59 $ 60
Total (%) 78.6 41.2 33.7 25.1
Race American Indian/ Alaska Native Asian Black/African American White Other Multiracial Hispanic ethnicity
79.4 8.9 8.0 27.7
Children age # 18 y in household None 1–2 3–4 5–6 $7
41.3 28.5 25.7 3.9 0.6
SNAP benefits Receiving SNAP benefits Qualify for SNAP benefits Receive other assistance Lesson MyPlate Menu Planning and Shopping
2.6 1.0 0.1
46.7 54.4 52.2
50.7 49.3
SNAP indicates Supplemental Nutrition Assistance Program. the individual questions from pretest to posttest increased significantly (P < .001), as did the mean of the lesson score. In general, participants reported before attending the Menu Planning and Shopping lesson that they sometimes engaged in the specific behaviors evaluated; after attending the lesson, participants reported they would usually engage in those behaviors. Table 3 compares the pre and post mean, median, SD, and interquartile ranges for each of the 8 individual questions and the lesson score related to the MyPlate lesson. To determine whether this lesson increased the
Savoie et al 83 intention to change the specific nutrition-related behaviors taught, the researchers conducted this analysis and compared pre- and posttest responses. Mean responses to individual questions from pretest to posttest increased significantly (P < .001), as did the mean of the lesson score. Results indicated that before attending the MyPlate lesson, participants sometimes engaged in the specific behaviors evaluated; after attending the lesson, they reported that they would usually engage in those behaviors.
DISCUSSION Results demonstrate that participation in either of 2 selected SNAP-Ed lessons had a positive impact on the reported intent of participants to improve nutrition-related behaviors consistent with the 2010 DGA. Results of this study are consistent with previous findings. Previous studies indicated a positive impact of nutrition education on fruit and vegetable consumption among low-income individuals, and further expanded these findings to other dietary habits in a sample of SNAP-Ed participants.13,14 Findings indicated that participation in federally funded nutrition education programs can change the intentions to improve nutrition-related behaviors and increase the consumption of healthy foods including fruits, vegetables, and whole grains.14 The current study builds on the recent literature that demonstrated the impact of federally funded nutrition education on changing behavioral intentions and actual consumption of fruits and vegetables in low-income individuals.13,14 These results provide evidence that nutrition education for federally funded programs targeted to lowincome individuals may be effective in improving reported intent related to healthy nutrition-related behaviors and potentially changing behaviors. Despite the strengths of this study, some limitations need to be mentioned. This study investigated intention to change behaviors instead of actual behavior change. According to the Theory of Planned Behavior, intention is considered to be the largest motivating factor for behavior change.6 Numerous observational and experimental studies have shown a
84 Savoie et al
Journal of Nutrition Education and Behavior Volume 47, Number 1, 2015
Table 2. Reported Intention to Change Nutrition-Related Behaviors in Adult Participants of a Menu Planning and Shopping Supplemental Nutrition Assistance Program–Education Lesson Pretest
Menu Planning and Shopping Lesson
Mean ± SD
Posttest
Median (interquartile range)
Mean ± SD
P*
Median (interquartile range)
a
Questions Have enough food to last through end of month Plan menu before shopping Shop with grocery list Make food purchases based on nutrition facts label Prepare meals at home at least 3 times/wk Eat meals together as family at least 3 times/wk Eat breakfast within 2 h of waking Lesson scoreb
3.5 1.2 2.7 1.2 3.4 1.2 2.8 1.2 3.9 1.2 3.8 1.2 3.3 1.3 3.3 0.8
3 (3–5) 3 (2–5) 3 (3–4) 3 (2–4) 4 (3–5) 4 (3–5) 3.5 (3–5)
4.1 1.0 3.9 0.9 4.1 0.9 3.7 1.0 4.3 1.0 4.2 1.1 4.0 1.1 4.0 0.8
4 (4–5) 4 (3–5) 4 (4–5) 4 (3–5) 5 (4–5) 4 (4–5) 4 (4–5)
< .001 < .001 < .001 < .001 < .001 < .001 < .001 < .001
*P < .05 is considered significant. a Comparisons were performed using Wilcoxon signed-rank test; bComparisons were performed using paired t test. Note: Values are from a Likert scale (1 ¼ never, 2 ¼ seldom, 3 ¼ sometimes, 4 ¼ usually, 5 ¼ always). positive association between changes in behavioral intentions and subsequent behavior change, which supports this proposition.15,16 However, it has also been argued that estimates of intention to change can overestimate actual behavior changes. A recent meta-analysis of randomized controlled trials examining this association reported that on average, medium to large changes in intention generate small to medium changes in behavior.17 Thus, the effect sizes presented in this study might be overestimating the impact of the intervention
on actual behavior change. The post-then-pre questionnaire used in this study is a self-reported instrument. Self-reported instruments are subjective measurements that estimate participant report of intention to change behavior, which increases the risk for bias and may cause overestimation of behavioral intention.11 The researchers did not use a control group, which limits the causal inference of the results. Improvements in future behaviors reported by participants might be a consequence of their awareness of being studied
(Hawthorne effect).18 Owing to the unique design of the study, participants may not have fully understood how to fill out the questionnaires, which could also contribute to measurement error. This study focused on the Menu Planning and Shopping and the MyPlate lessons only; therefore, the impact of the greater nutrition education curricula of the SNAP-Ed program was not evaluated. Data were collected from participants during 1 data collection period. Therefore, no follow-up data were available to demonstrate the
Table 3. Reported Intention to Change Nutrition-Related Behaviors in Adult Participants of a MyPlate Supplemental Nutrition Assistance Program–Education Lesson in Utah Pretest
MyPlate Lesson Questionsa Eat at least 2.5 cups/d of vegetables Eat at least 2 cups/d of fruit Eat more whole grains than refined grains Eat at least 3 cups/d or equivalent of dairy Choose at least 2 servings of lean protein Replace saturated and trans fats with heart-healthy fats Choose to be physically active for at least 30 min, 5 d/wk Prepare foods without adding salt Lesson scoreb
Posttest
P
Median Median (interquartile (interquartile range) range) Mean ± SD Mean ± SD 3.4 1.1 3.4 1.0 3.5 1.1 3.3 1.2 3.4 1.1 3.1 1.2 3.4 1.2 3.3 1.1 3.3 0.8
4 (3–4) 3 (3–4) 4 (3–4) 3 (3–4) 3 (3–4) 3 (2–4) 4 (3–4) 3 (2–4)
4.3 0.9 4.1 0.9 4.2 0.9 4.0 1.0 4.2 0.9 4.1 1.0 4.2 1.0 4.0 1.1 4.0 0.8
4 (4–5) 4 (4–5) 4 (4–5) 4 (4–5) 4 (4–5) 4 (4–5) 4 (4–5) 4 (3–5)
< .001 < .001 < .001 < .001 < .001 < .001 < .001 < .001 < .001
Comparisons were performed using Wilcoxon signed-rank test; bComparisons were performed using paired t test; P < .05 is considered significant. Note: Values are from a Likert scale (1 ¼ never, 2 ¼ seldom, 3 ¼ sometimes, 4 ¼ usually, 5 ¼ always). a
Journal of Nutrition Education and Behavior Volume 47, Number 1, 2015 frequency of SNAP-Ed participation after attending either of the 2 selected lessons. Data on continued participation would have strengthened the results of this study by further concluding intention to change nutrition-related behaviors. Finally, although over half of participants reported this was their first SNAP-Ed lesson, there was diversity in the number of lessons participants had taken previously. It is possible that participants who had attended multiple lessons had greater intentions to change compared with new participants.
IMPLICATIONS FOR RESEARCH AND PRACTICE Findings from this study have direct implications for continued implementation of these SNAP-Ed lessons. Supplemental Nutrition Assistance Program–Education programs may consider replicating the retrospective post-then-pre design as an initial measurement to determine behavioral intentions of SNAP-Ed participants given the readability, convenience, and cost effectiveness of this design.19 The positive association between changes in behavioral intentions and subsequent behavior change makes this design a feasible and valuable evaluation method for SNAP-Ed programs. Future research should investigate the long-term indication of behavior change. Programs could enhance research by collecting data on actual behavior change to investigate how intentions to change nutrition-related behaviors translate into behavior change in this population. Future studies may incorporate telephone call surveys several weeks after completion of the SNAP-Ed class for follow-up data. Participants could also send in grocery receipts to determine purchasing behavior before and after SNAP-Ed education. This study provides validated evaluation materials that can be adopted by SNAP-Ed programs to assess similar interventions. Results from this study, together with those previously demonstrating a positive impact of nutrition education on nutrition-related behaviors, highlight the importance of continued research of federally funded nutrition education programs.
Demonstrating program impact through research and program evaluation supports future research opportunities for the SNAP-Ed program. Further studies should focus on assessing the long-term effect of nutrition education on nutrition-related behavior change among SNAP-Ed participants and their families and evaluate the impact of individual components of nutrition education on actual behavior change.
ACKNOWLEDGMENTS The authors greatly appreciate Teri Peterson for assistance with statistical analysis. The authors also thank Utah State University for allowing researchers to use the Utah SNAP-Ed data for this study, and SNAP-Ed participants in Utah for willingness to complete this survey. This study was completed as a thesis by Mateja R. Savoie and faculty at Idaho State University.
REFERENCES 1. US Department of Agriculture, Food, and Nutrition Service. Supplemental nutrition assistance program education guidance: nutrition education and obesity prevention grant program. https://www .extension.org/sites/default/files/FY13 Guidance.pdf. Accessed July 12, 2014. 2. US Department of Agriculture, Food, and Nutrition Service. Supplemental nutrition assistance program: number of persons participating. http://www .fns.usda.gov/pd/29SNAPcurrPP.htm. Accessed July 12, 2014. 3. US Department of Agriculture, Food, and Nutrition Service. EARS overview. http://snap.nal.usda.gov/national-snaped/education-and-administrative-repor ting-system/ears-overview. Accessed July 12, 2014. 4. Christofferson D, Christensen N, LeBlanc H, Bunch M. Developing an online certification program for nutrition education assistants. J Nutr Educ Behav. 2012;44:407-414. 5. US Department of General Accounting Office. Nutrition education: USDA provides services through multiple programs but stronger linkage among efforts are needed. http://www.gao.gov/assets/250/ 242135.pdf. Accessed July 12, 2014. 6. Guthrie JF, Stommes E, Voichick J. Evaluating food stamp nutrition
Savoie et al 85
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
education: issues and opportunities. J Nutr Educ Behav. 2006;38:6-11. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50:179-211. Kothe EJ, Mullan BA, Butow P. Promoting fruit and vegetable consumption: testing an intervention based on the theory of planned behaviour. Appetite. 2012;58:997-1004. Howard G, Dailey P. Response-shift bias: a source of contamination of selfreport measures. J Appl Psychol. 1979; 64:144-150. Howard G, Ralph K, Gulanick N, Maxwell S, Nance D, Gerber S. Internal invalidity in pretest-posttest selfreport evaluations and a re-evaluation of retrospective pretests. Appl Psychol Meas. 1979;3:1-23. Pratt CC, McGuigan WM, Katzev AR. Measuring program outcomes: using retrospective pretest methodology. Am J Eval. 2000;21:341-349. Raidl M, Johnson S, Gardiner K, et al. Use retrospective surveys to obtain complete data sets and measure impact in extension programs. J Extension. 2004;42(2). Heneman K, Block-Joy A, ZidenbergCherr S, et al. A ‘‘Contract for Change’’ increases produce consumption in lowincome women: a pilot study. A Am Diet Assoc. 2005;105:1793-1796. Ritchie LD, Whaley SE, Spector P, Gomez J, Crawford PB. Favorable impact of nutrition education on California WIC families. J Nutr Educ Behav. 2010;42:S2-S10. Kim KW, Reicks M, Sjoberg S. Applying the theory of planned behavior to predict dairy product consumption by older adults. J Nutr Educ Behav. 2003;35:294-301. Armitage CJ, Conner M. Efficacy of the theory of planned behaviour: a meta-analytic review. Br J Soc Psychol. 2001;40:471-499. Webb TL, Sheeran P. Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychol Bull. 2006;132:249-268. Campbell MK, Honess-Morreale L, Farrell D, Carbone E, Brasure M. A tailored multimedia nutrition education pilot program for low-income women receiving food assistance. Health Educ Res. 1999;14:257-267. Davis G. Using a retrospective pre-post questionnaire to determine program impact. J Extension. 2003;41(4).
