This article was downloaded by: [ECU Libraries] On: 19 April 2015, At: 09:07 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of American College Health Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/vach20
In the Eye of the Beholder: Pretesting the Effectiveness of Health Education Materials a
Rita Hirsch MS & Myra Ellen Edelstein MS
a
a
Division of Health Education in the university health services , University of Massachusetts , Amherst, USA Published online: 09 Jul 2010.
To cite this article: Rita Hirsch MS & Myra Ellen Edelstein MS (1992) In the Eye of the Beholder: Pretesting the Effectiveness of Health Education Materials, Journal of American College Health, 40:6, 292-293, DOI: 10.1080/07448481.1992.9936296 To link to this article: http://dx.doi.org/10.1080/07448481.1992.9936296
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
CLINICAL & ~IOGRW NOTES
Downloaded by [ECU Libraries] at 09:07 19 April 2015
In the Eye of the Beholder: Pretesting the Effectiveness of Health Education Materials Rita Hirsch, MS; and Myra Ellen Edelstein, MS
ABSTRACT. Health educators at the University of Massachusetts evaluated five posters to determine whether students comprehended, identified with, and were motivated by the messages, which dealt with aspects of safer sex, alcohol use, and stress management. Pretesting, they suggest, provides an opportunity for remedying design flaws before distributing materials, improving effectiveness, and saving money by assuring that the target audience receives the intended message. Six suggestions for improving effectiveness and acceptability of printed health education materials are offered. Key Words. graphic arts, health education, print media
ince the 197Os, college health education divisions have become important sources of health information for students. At the same time, a profusion and refinement of high-technology office equipment, including word processing, desk-top publishing, and high quality, high speed copying, have made in-house printing practical. These technologies provide the health educator with the power to produce and distribute printed materials easily and swiftly. With this ability comes the temptation to distribute health information to the target audience without pretesting materials. Graphic arts theory, however, appropriately cautions that all messages are not received equally by an intended audience, even if the messages are presented with wit, beauty, or imagination. If the information or design is controversial, inconclusive, likely to arouse uncomfort-
Rita Hirsch is a health educator with the Division of Health Education in the university health services at the University of Massachusetts in Amherst, where Myra Ellen Edelstein is a doctoral candidate.
292
able reactions, or if people do not identify with the message, the printed material will be ineffective or unacceptable. Expensive pamphlets will remain unopened and posters will remain unnoticed. Although some items go to press without rigorous pretesting, health educators at the health services of the University of Massachusetts at Amherst periodically slow down production to evaluate the effectiveness of the materials. With thousands of printed materials leaving the division each year, it is imperative that we check with our consumers, the students, on the issue of acceptability. Pretesting provides health educators with “objective information and direction for improving concepts, messages, and materials while revisions still are possible and We conducted an evaluation of five posters during the 1989 spring semester. The evaluation was designed to answer the following questions: (1) Do students comprehend the messages addressed in the posters? (2) Do students identify with and accept the posters’ messages? (3) Are students motivated by the posters? METHOD
During the semester, trained peer educators administered questionnaires to 106 students who lived in residence halls and off-campus housing. The questionnaires measured the students’ reactions to five different posters distributed by the health education division of the university health services. The main messages of the five posters dealt with practicing safer sex, monitoring alcohol use, practicing sober sex, practicing stress management, and considering the danger of driving while intoxicated. The questionnaire was adapted from a United States Department of Health and Human Services survey inJACH
PRETESTING COLLEGE HEALTH MATERIALS
strument.yp*9) We modified the instrument for the UMass project and pretested it to address issues of reliability and validity.
Downloaded by [ECU Libraries] at 09:07 19 April 2015
RESULTS
We coded 106 evaluations and analyzed them, using the Statistical Package for the Social Sciences (SPSS); 67% of the respondents were women and 33% were men. The mean age was 20 years. The sample consisted of first-year students (l8%), sophomores (20%), juniors (26Vo), seniors (32%), and graduate students (4Vo). Virtually all (97%) of the students understood or partially understood the messages presented in the posters. Although nearly all messages were interpreted correctly, most people indicated that they did not acquire new information from any of the posters. The posters were typically rated, however, as both motivating and interesting. These results were encouraging because motivation or an attitudinal change are expected outcomes of a poster campaign. Health educators realize that absorbing new information or adopting new behavior from an intervention as limited as a poster is unrealistic. Even pretesting the intervention has limitations: “Pretesting is neither an absolute predictor nor a guarantee of success in terms of learning, persuasion, behavior change, or other measures of communication Pretesting does, however, provide an opportunity to fur subtle design flaws and to save precious dollars by effectively addressing the target audience with the intended message. A majority of students identified with the posters and agreed that the messages were directed toward them. When rating the poster stating, “AIDS-Now the choice is yours-practice safer sex-use condoms and spermicides,” 73% of the students responded that the poster was “talking to someone like me.” This, too, was encouraging because it has been demonstrated that some students feel AIDS is a problem only among those who are gay or use IV drugs. Perhaps the most interesting and surprising results were the differences found in the ways men and women responded to the posters. In general, men were more critical. Compared with the women, the men ranked all five posters lower on design quality, indicated they were less interested and less motivated by the messages, and said they had received less information. The poster carrying the message “Idiot on Board-If you’re going to drink and drive at least let the rest of us know” consistently generated statistically significant differences between men and women, based upon chisquare tests (p < .05). Although 29% of the male respondents indicated that this poster would bother or offend people they knew, only 8% of the women agreed with this statement (p = .0084); 89% of the women agreed that this poster was motivating, but only 63% of
VOL 40, MAY 1992
the men agreed (p = .0046). Of the men, 40% reported that this poster was alarming, compared with only 22% of the women (p = .0289);26% of the men said the poster was informative, compared with 66% of the women (p = .OOO7); and 51% of the men reported that it was tasteful, compared with 82% of the women (p = .0185). In addition to gender differences, it is likely that other sociodemographic variables, including race, ethnicity, religion, and family income, could have a similar impact on a student’s interpretation of health education posters. This assumption, however, would require further study. DISCUSSION
Although this was a small-scale evaluation, it gave us an opportunity to assess the effectiveness and acceptability of selected health education interventions. The staff time and effort required to create, design, produce, and distribute such materials is obviously costly. We were pleased that this project helped us identify design flaws that we could easily remedy. Our experience with pretesting leads us to suggest that taking the following steps will help campus health educators increase the effectiveness and acceptability of printed materials: 1. Select the target audience you hope to reach and ask members of that group to assist you in creating and pretesting your design ideas. 2. Consider using different images, phrases, and colors for various segments of your target audience-and pretest the design while you have time to make changes. 3. Set goals and objectives before designing printed materials. Decide whether the purpose of the piece is educational, motivational, or “product” acceptability (eg, using condoms and spermicides, letting a friend drive if you’re intoxicated, etc).* 4. Take time to pretest your design before you spend hundreds of dollars producing an item that has little consumer acceptability or effectiveness. Faster is not always better! 5. Test purchased materials for consumer acceptability and effectiveness. This could save additional costs to your organization. 6. Pretest, using small-scale and informal surveys or focus groups. The results can optimize the efficiency, effectiveness, and acceptability of your printed materials. REFERENCES 1. Pretesting in Health Communications.Washington, DC: US Department of Health and Human Services; NIH Publication 84-1493; 1984. 2. Aldrich-Ruenzel N. How design can convey health risk. Step-by-step Graphics. 1988;qS): 139- 142.
293
Journal of American College Health Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/vach20
In the Eye of the Beholder: Pretesting the Effectiveness of Health Education Materials a
Rita Hirsch MS & Myra Ellen Edelstein MS
a
a
Division of Health Education in the university health services , University of Massachusetts , Amherst, USA Published online: 09 Jul 2010.
To cite this article: Rita Hirsch MS & Myra Ellen Edelstein MS (1992) In the Eye of the Beholder: Pretesting the Effectiveness of Health Education Materials, Journal of American College Health, 40:6, 292-293, DOI: 10.1080/07448481.1992.9936296 To link to this article: http://dx.doi.org/10.1080/07448481.1992.9936296
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
CLINICAL & ~IOGRW NOTES
Downloaded by [ECU Libraries] at 09:07 19 April 2015
In the Eye of the Beholder: Pretesting the Effectiveness of Health Education Materials Rita Hirsch, MS; and Myra Ellen Edelstein, MS
ABSTRACT. Health educators at the University of Massachusetts evaluated five posters to determine whether students comprehended, identified with, and were motivated by the messages, which dealt with aspects of safer sex, alcohol use, and stress management. Pretesting, they suggest, provides an opportunity for remedying design flaws before distributing materials, improving effectiveness, and saving money by assuring that the target audience receives the intended message. Six suggestions for improving effectiveness and acceptability of printed health education materials are offered. Key Words. graphic arts, health education, print media
ince the 197Os, college health education divisions have become important sources of health information for students. At the same time, a profusion and refinement of high-technology office equipment, including word processing, desk-top publishing, and high quality, high speed copying, have made in-house printing practical. These technologies provide the health educator with the power to produce and distribute printed materials easily and swiftly. With this ability comes the temptation to distribute health information to the target audience without pretesting materials. Graphic arts theory, however, appropriately cautions that all messages are not received equally by an intended audience, even if the messages are presented with wit, beauty, or imagination. If the information or design is controversial, inconclusive, likely to arouse uncomfort-
Rita Hirsch is a health educator with the Division of Health Education in the university health services at the University of Massachusetts in Amherst, where Myra Ellen Edelstein is a doctoral candidate.
292
able reactions, or if people do not identify with the message, the printed material will be ineffective or unacceptable. Expensive pamphlets will remain unopened and posters will remain unnoticed. Although some items go to press without rigorous pretesting, health educators at the health services of the University of Massachusetts at Amherst periodically slow down production to evaluate the effectiveness of the materials. With thousands of printed materials leaving the division each year, it is imperative that we check with our consumers, the students, on the issue of acceptability. Pretesting provides health educators with “objective information and direction for improving concepts, messages, and materials while revisions still are possible and We conducted an evaluation of five posters during the 1989 spring semester. The evaluation was designed to answer the following questions: (1) Do students comprehend the messages addressed in the posters? (2) Do students identify with and accept the posters’ messages? (3) Are students motivated by the posters? METHOD
During the semester, trained peer educators administered questionnaires to 106 students who lived in residence halls and off-campus housing. The questionnaires measured the students’ reactions to five different posters distributed by the health education division of the university health services. The main messages of the five posters dealt with practicing safer sex, monitoring alcohol use, practicing sober sex, practicing stress management, and considering the danger of driving while intoxicated. The questionnaire was adapted from a United States Department of Health and Human Services survey inJACH
PRETESTING COLLEGE HEALTH MATERIALS
strument.yp*9) We modified the instrument for the UMass project and pretested it to address issues of reliability and validity.
Downloaded by [ECU Libraries] at 09:07 19 April 2015
RESULTS
We coded 106 evaluations and analyzed them, using the Statistical Package for the Social Sciences (SPSS); 67% of the respondents were women and 33% were men. The mean age was 20 years. The sample consisted of first-year students (l8%), sophomores (20%), juniors (26Vo), seniors (32%), and graduate students (4Vo). Virtually all (97%) of the students understood or partially understood the messages presented in the posters. Although nearly all messages were interpreted correctly, most people indicated that they did not acquire new information from any of the posters. The posters were typically rated, however, as both motivating and interesting. These results were encouraging because motivation or an attitudinal change are expected outcomes of a poster campaign. Health educators realize that absorbing new information or adopting new behavior from an intervention as limited as a poster is unrealistic. Even pretesting the intervention has limitations: “Pretesting is neither an absolute predictor nor a guarantee of success in terms of learning, persuasion, behavior change, or other measures of communication Pretesting does, however, provide an opportunity to fur subtle design flaws and to save precious dollars by effectively addressing the target audience with the intended message. A majority of students identified with the posters and agreed that the messages were directed toward them. When rating the poster stating, “AIDS-Now the choice is yours-practice safer sex-use condoms and spermicides,” 73% of the students responded that the poster was “talking to someone like me.” This, too, was encouraging because it has been demonstrated that some students feel AIDS is a problem only among those who are gay or use IV drugs. Perhaps the most interesting and surprising results were the differences found in the ways men and women responded to the posters. In general, men were more critical. Compared with the women, the men ranked all five posters lower on design quality, indicated they were less interested and less motivated by the messages, and said they had received less information. The poster carrying the message “Idiot on Board-If you’re going to drink and drive at least let the rest of us know” consistently generated statistically significant differences between men and women, based upon chisquare tests (p < .05). Although 29% of the male respondents indicated that this poster would bother or offend people they knew, only 8% of the women agreed with this statement (p = .0084); 89% of the women agreed that this poster was motivating, but only 63% of
VOL 40, MAY 1992
the men agreed (p = .0046). Of the men, 40% reported that this poster was alarming, compared with only 22% of the women (p = .0289);26% of the men said the poster was informative, compared with 66% of the women (p = .OOO7); and 51% of the men reported that it was tasteful, compared with 82% of the women (p = .0185). In addition to gender differences, it is likely that other sociodemographic variables, including race, ethnicity, religion, and family income, could have a similar impact on a student’s interpretation of health education posters. This assumption, however, would require further study. DISCUSSION
Although this was a small-scale evaluation, it gave us an opportunity to assess the effectiveness and acceptability of selected health education interventions. The staff time and effort required to create, design, produce, and distribute such materials is obviously costly. We were pleased that this project helped us identify design flaws that we could easily remedy. Our experience with pretesting leads us to suggest that taking the following steps will help campus health educators increase the effectiveness and acceptability of printed materials: 1. Select the target audience you hope to reach and ask members of that group to assist you in creating and pretesting your design ideas. 2. Consider using different images, phrases, and colors for various segments of your target audience-and pretest the design while you have time to make changes. 3. Set goals and objectives before designing printed materials. Decide whether the purpose of the piece is educational, motivational, or “product” acceptability (eg, using condoms and spermicides, letting a friend drive if you’re intoxicated, etc).* 4. Take time to pretest your design before you spend hundreds of dollars producing an item that has little consumer acceptability or effectiveness. Faster is not always better! 5. Test purchased materials for consumer acceptability and effectiveness. This could save additional costs to your organization. 6. Pretest, using small-scale and informal surveys or focus groups. The results can optimize the efficiency, effectiveness, and acceptability of your printed materials. REFERENCES 1. Pretesting in Health Communications.Washington, DC: US Department of Health and Human Services; NIH Publication 84-1493; 1984. 2. Aldrich-Ruenzel N. How design can convey health risk. Step-by-step Graphics. 1988;qS): 139- 142.
293
