A healthlpatient education database for family practice*t By Patricia A. Gibson, Ph.D.
Vice-President, Information Services and Systems Carolyn Ruby, M.A.L.S. Medical Librarian American Academy of Family Physicians Foundation
Huffington Library 8880 Ward Parkway P.O. Box 8418 Kansas City, Missouri 64114-0418 Marian D. Craig, M.A.L.S.t Information Resources Manager
Marion Merrill Dow, Inc. Baptiste-P.O. Box 9627 Kansas City, Missouri 64134-0627
Using pilot project funding from the W. K. Kellogg Foundation, the American Academy of Family Physicians Foundation (AAFP/F) developed a program by which health/patient education print materials were reviewed. Favorably reviewed materials were entered into a database accessible through the AAFP/F's Huffington Library. The review service and resulting database were designed to help the busy clinician identify scientifically accurate, reliable materials for use in patient education. The review process developed for the project is described, as is the database and its use by family physicians. Research findings from the pilot project are discussed, some of which assisted in planning the self-supporting second phase of the program.
In 1981 the Board of Trustees of the American Academy of Family Physicians Foundation (AAFP/F) began to plan a national library and information center
for family physicians. The plan included a clearinghouse of evaluated patient education materials. The trustees viewed evaluation as a critical element in this effort, because they perceived no other source that evaluated health/patient education materials from the perspective of the family physician. The specialty of family practice, established in 1969, has a strong commitment to disease prevention and * This program was supported in part by a grant from the W. K. Kellogg Foundation. t Based on a paper presented May 24, 1988, at the Eighty-Eighth Annual Meeting of the Medical Library Association, New Orleans, Louisiana. t Formerly technical services librarian, American Academy of Family Physicians Foundation.
Bull Med Libr Assoc 79(4) October 1991
patient education. The "Special Requirements for Residency Training in Family Practice" specifically stated that residents must receive instruction in health assessment and maintenance, prevention, behavioral counseling, and health education [1]. Elaborating on this document in 1985, AAFP formulated Recommended Core Curriculum Guidelines on Health Promotion for Family Practice Residents, which detailed specific attitudes, knowledge, and skills required for health promotion [2]. In 1986 AAFP developed Recommended Core Curriculum Guidelines on Disease Prevention for Family Practice Residents, which detailed the attitudes, knowledge, and skills required for disease prevention [3]. Recognizing, however, in 1981 that 58% of AAFP's overall membership of 70,000 was already in practice and, therefore, beyond residency training, AAFP created the Preventive Health Care Subcommittee of the Commission on Public Health and Scientific Affairs. 357
Gibson et al.
Its purpose was to study methods for introducing preventive medicine techniques into family practice, to acquaint existing practitioners with health promotion and preventive medicine measures, and to address the continuing education needs of residents completing their training. In 1984, AAFP also created the Health Education Committee to identify and develop health education methods and materials for family physicians, both in practice and in training. Both these groups provided useful input as this project evolved.
The specialty of family practice, established in 1969, has a strong commitment to disease prevention and patient education. THE PROBLEM
The dilemma faced by busy clinicians as they expand their practices to include a health promotion/disease prevention orientation has been vividly described by Davies as primarily a problem of an overabundance of health promotion/patient education materials, few of which have been appropriately evaluated. Davies described the health promotion literature as an "overwhelming assortment of science, pseudo-science, tenuous theory and unmitigated clap-trap" [4]. Much is product advertising and therefore suspect even when basically sound. Furthermore, the health promotion program literature is largely ephemeral, consisting primarily of pamphlets of all shapes and sizes. A cursory review of standard reference works in the patient education/health promotion literature revealed many sources of materials, including voluntary health agencies, pharmaceutical companies, professional associations, governmental agencies, and companies whose sole purpose is to publish patient education literature [5-6]. NEEDS ASSESSMENT In order to identify the specific concerns of family physicians in health promotion and patient education, the AAFP/F library staff conducted a needs assessment survey during November 1985. A questionnaire sent to 2,258 AAFP member users of the Huffington Library elicited a response rate of 39%. The questionnaire was designed to answer the following questions. Do family physicians believe patients want to assume an active role in their own health care? Do family physicians need evaluated health promotion materials? Would they use the health education clearinghouse? What health promotion programs have they developed in their own 358
communities? What topics would they like to see addressed first by the clearinghouse? The responses to the questionnaire provided a strong statement of need for assistance with health promotion in the family practice setting. Responses to selected questions appear in Table 1.
The dilemma faced by busy clinicians as they expand their practices to include a health promotion! disease prevention orientation has been vividly described by Davies as primarily a problem of an overabundance of health promotion/patient education materials, few of which have been appropriately evaluated. Respondents were also given a list of fifty-one disease-related topics and thirty-eight health promotion/wellness topics. The highest-ranked topics, followed by the number and percentage of respondents checking the topic, appear in Tables 2 and 3. THE GRANT PROPOSAL
Based upon the directive from AAFP/F's Board of Trustees, a review of the literature, and the results of the needs assessment, AAFP/F submitted a grant proposal to the W. K. Kellogg Foundation, from which funding was received in November 1986. The project was designed as a two-year pilot to develop a mechanism for evaluating patient education materials, placing favorably reviewed materials in a database available to family physicians, and testing the effectiveness of the review process and the resulting database as a service to family practice. Only brief print materials (brochures and short booklets) were to be reviewed.
The responses to the questionnaire provided a strong statement of need for assistance with health promotion in the family practice setting. INITIAL IMPLEMENTATION A steering committee was formed to advise the project director in developing the program, selecting reviewers for the program, and monitoring the overall progress of the project. The committee consisted of five family physicians, two of whom were in full-time family practice and two of whom were faculty of Bull Med Libr Assoc 79(4) October 1991
Patient education database
Table 1
Response to selected questions from needs assessment (n = 873*)
1. Most patients want to assume an increasingly active and participative role in their own health care (both prevention and cure). 2. It will be increasingly important for family physicians to provide patients with health promotion and patient education materials that are accurate and easy to comprehend. 3. Health promotion and patient education materials that have been evaluated for accuracy and clarity by either a family physician or a heaith educator are more reliable than materials evaluated by others or not evaluated at all. 4. My health promotion and patient education efforts would be improved if materials were prescreened by health care professionals for accuracy, style, readability, etc.
No Strongly disagree opinion Number (%) Number (%) Number (%)
Moderately disagree
Strongly agree Number (%)
Moderately agree Number (%)
236(27.0%)
567(65.0%)
59(6.8%)
7(0.8%)
2(0.2%)
871 (99.8%)
617 (70.7%)
232 (26.6%)
15 (1.7%)
4(0.5%)
4(0.5%)
872 (100.0%)
471 (54.0%)
314 (36.0%)
29 (3.3%)
1 (0.1%)
55(6.3%)
870(99.7%)
454 (52.0%/a)
351 (40.2%)
27 (3.1%)
6(0.7%)
29(3.3%)
867 (99.3%)
Total Number (%)
Not all respondents answered all questions.
family practice residency programs, and a nonphysician patient educator. The committee was chaired by the AAFP/F's library committee chair, a family physician in private practice. Because family practice residency curricula emphasize patient education, AAFP/F contracted with three residency programs to complete the reviews. To identify three residencies with strong patient education/health promotion programs, all 382 U.S. family practice residencies were invited to complete an extensive application, mailed in January 1987. A total of sixty-eight applications was received. After a blind review of the applications, the steering committee Table 2 Ranked list of disease-related topics (n
=
873*) Number of respondents checking
Number of respondents checking
Topic
Backachet Hypertensiont Depressiont Headachet
Arthritist
Diabetes mellitust
Obesityt Osteoporosist
Irritable bowel syndrome Alzheimer's disease Premenstrual tension Fibrocystic disease of breast
Vaginitis Asthma *
(%)
Topic
519 (59.4%) 511 (58.5%) 475 (54.4%) 416 (47.6%) 407 (46.6%)
Sexually transmitted diseases Otitis media Allergic rhinitis Emphysema
406(46.5%) 406(46.5%) 361 (41.3%)
Acne Insomnia
306 (35.00/a) 278 (31.9%) 246 (28.2%b) 244 (28.0%) 237 (27.2%) 225 (25.8%)
identified six family practice residencies as finalists. After a site visit to each of these residencies, followed by an interview in Kansas City with representatives of each of the finalists, the steering committee selected three family practice residencies to participate in the project: East Carolina University Department of Family Medicine, Greenville, North Carolina; Forbes Health System, Monroeville, Pennsylvania; and University of Colorado Department of Family Medicine, Denver, Colorado. AAFP/F paid each participating residency the equivalent of a patient educator's salary to evaluate patient education materials and provided funds for procuring materials.
Menstruation disorders
AlDSt Cystitis Vertigo Peptic ulcer Esophageal reflux Pregnancy complications
Topic
(%)
218 (25.0Yo) 211 (24.2Yo) 208 (23.8%) 203(23.2%) 197 (22.6%) 168 (19.3%) 163 (18.7%) 161 (18.4%) 141 (16.2°h) 124 (14.2%) 101 (11.6%)
100(11.5%) 97 (11.1%)
Coronary bypass Prostate disease Hyperactive children Cervical strain Stroke rehabilitation Food allergies
Temperomandibular joint syndrome Anorexia nervosa and bulimia Insect bites
Hyperventilation syndrome Tinnitus Infertility Eczema
Number of respondents checking (%)
93(10.6%) 86(9.9%) 85 (9.7%) 79 (9.1%) 78 (8.9%) 74 (8.5%) 67 (7.7%) 53 (6.1%) 52 (6.0%)
51(6.0%) 47(5.4%) 47(5.4%) 43 (4.9%)
Respondents could check any number of categories.
t Topics covered by pilot project.
Bull Med Libr Assoc 79(4) October 1991
359
Gibson et al.
Figure 1 Procedures flow chart
THE REVIEW PROCESS
The patient educator in each participating residency was responsible for locating as many print materials as possible on three health promotion and five dis-
ease-related topics, corresponding to the ranked topics on the needs assessment. Some adjustments were made to obtain a good match with the special content expertise in each residency. The disease-related list was reordered to include AIDS. A Delphi-like process was adopted through which each residency identified as many materials as possible on its designated subjects. The patient educator eliminated those materials not deserving further review, most frequently because of content inaccuracy
or bias. The patient educator mailed copies of materials worthy of review to the other two residencies. The same materials were then evaluated by all three residencies (Figure 1). The patient educators helped design the two evaluation forms used in the project, one form completed by health professionals and the other by patients. Field tests of the forms in the residencies resulted in minor changes. To determine the reliability or internal consistency of the forms, a Cronbach's alpha was calculated on both the professional and patient evaluation forms. The professional form scored a 0.94 reliability coefficient (Figure 2), and the patient form scored 0.92 (Figure 3). The health professional form asked the reviewer to state the objective of the piece and then to respond on a five-point scale to thirty questions relating to the appearance of the piece, its content, its usefulness, the reviewer's overall reaction to the piece, and the most appropriate placement for the piece (e.g., waiting room, exam room). At the bottom of the form, the health professional was asked to supply demographic data (Appendix A). On the patient review form, respondents were asked to state the overall subject of the piece and then were asked to respond on a five-point scale to fifteen questions corresponding to questions on the health professional's form. The questions related to the piece's appearance, content, and usefulness. Patients were asked to state the purpose of the item and were invited to comment on the piece. At the bottom of the form, the patient was also asked to supply demographic data (Appendix B).
Table 3 Ranked list of health promotion/wellness and other topics (n = 873*)
Topic
(%)
Smoking cessationt Mammography and breast
768(88%)
self-examt Stress managementt
716 (82%) 698 (80%) 681 (78%) 637 (73%) 629 (72%) 611 (70%) 594 (68%)
Alcohol abuset Hemoccult blood testt
Contraceptiont
Pap smear and pelvic examt
Sigmoidoscopyt
Car safety, including seat beltst Drug abuset Child developmentt Childhood immunizationt Home accident preventiont *
585(67%) 567 (65%) 559 (64%) 550(63%) 550 (63%)
Number of
Number of respondents checking
Number of respondents checking
Topic Aerobic exerciset
Calcium, dietaryt Fiber, dietaryt
Adult immunizationt Adoescence Divorce Menopause, male and female Athletic injuries, including prevention Prostate physical exam Retirement and old age Breast feeding Single parenting Grieving and bereavement
(%) 541 (62%) 524 (60%) 524 (60%) 515 (59%) 515 (59%) 514 (59%)
506(58%) 489 (56%) 489(56%) 489 (56%) 471 (54%) 445 (51%)
444 (51%)
Topic Diet fads Nutrition for specific disease prevention
Prenatal care Empty nest syndrome Vitamin and mineral supplements
Outdoor safety, including camping DPT immunization General meal planning Intemational travel Natural childbirth Food additives Insemination, artificial
respondents checking (%)
437(50%h) 410 (47%) 375 (43%/6) 367 (42%) 340(39%) 314(36%) 313(36%) 312(36%) 236(27%) 192 (22%) 175 (20%) 44(5%)
Respondents could check any number of categories.
t Topics covered by pilot project.
360
Bull Med Libr Assoc 79(4) October 1991
.0.54 7
Patient education database
Figure 2 Reliability of professional rating form (n = 143) Total Form Reinforces Information Provides New Information Stimulates Behavior Change_ Attracts Attention Holds Attention Overall Appearance_ Quality of Illustrations_ Use of Color Typeface Use of Highlighting Current Information Accuracy _ Adequate Scope_ Overall Organization_ Flow of Ideas Needed Background Given Use of Summaries
in QA
522
527
JO 731
tD52 6JU I652
593
/92
=0 796 =0 786
Usefulness Believable Understandable Requires Little Explanation_ Recommend Use with Patients Meets its Objective
D 7,42
) 742
-I 769
0.1
0
0.2
0.3
04
0.6
05
0.7
0.8
0.9
1
Reliability
Figure 3 Reliability of patient rating form (n = 992) Total Form Accomplishes Main Purpose Right Brochure for Me Recommend Use with Others Changed My Behavior Changed My Thinking Provided New Information Reminded Me of Things Contains Important Information Understandable I'd Recommend to Another Believable Quality of Illustrations Usefulness Held My Attention Attracted My Attention
10 635
U0 74
o0.643 ] 0.639
=0.675
0.69
l 0634
0.592
0.54
-0 647
=0.677
I.
0
Bull Med Libr Assoc 79(4) October 1991
I.I
0.1
0.2
03
0.4
06
05 0.6 Reliability
0.7
0.8
0
os9
1
361
Gibson et al.
The pilot project aimed to obtain a minimum of twenty-one patient evaluations and the same number of health professional evaluations for each patient education piece to adequately test the evaluation forms and the overall review process. Each patient educator completed a form supplying descriptive information on all materials reviewed in the eight assigned topic areas and submitted two copies of all materials reviewed to the AAFP/F library. If that piece was eliminated, the patient educator indicated the reason for its elimination, e.g., bias, scientific inaccuracy, unattractive format.
On the patient review form, respondents were asked to state the overall subject of the piece and then were asked to respond on a five-point scale to fifteen questions corresponding to questions on the health professional's form. If the piece deserved further review, the full descriptive form was completed and the reading level of the piece was computed with microcomputer software, using the Dale-Chall Scale for materials written for fifth grade and above and the Spache Scale for materials below the fifth grade level. DATABASE CREATION The AAFP/F technical services librarian created the database record, adding descriptive catalog information to the data supplied by the patient educator. The database was built on a DEC VAX 11/780 minicomputer. Written in COBOL, the database program was called FPB (Family Practice Base), an interactive query and file maintenance system designed for use on VAX/VMS systems. The software package Datatrieve®§ performed the statistical calculations required to score each review. A mean score of 3.5 or higher assigned by health professionals constituted a favorable review and resulted in inclusion in the database. Datatrieve calculated mean scores for each question and also totaled scores for each category on the evaluation form, for both health professional and patient reviewers. The health education database record was displayed on a series of three screens. A maximum of six subject headings or key words were allowed per record, of which a maximum of three were MeSH ® * * § Datatrieve is a registered trademark of Digital Equipment Corporation. * * MeSH is a registered trademark of the National Library of Medicine.
362
terms. A maximum of three were lay subject terms suggested by the patient educators. Reports were generated through the software's "query" mode, which allowed Boolean searching on any field or combination of fields in the record. Statistical reports were generated through SPSS®t.
RESEARCH FINDINGS AAFP/F wanted the project to determine if there was general agreement between health professionals and patients regarding the accuracy, physical appeal, and usefulness of a particular patient education piece. Therefore, the evaluation forms used by the two groups of reviewers were designed to contain equivalent questions on these aspects of patient education materials. Materials reviewed by patients had a mean score of 3.99; those evaluated by health professional reviewers scored a mean of 3.91. Although these scores suggested that health professionals tended to be more critical than patients, the differences were not statistically significant.
The mean reading level of materials in the database was 9.7, reflecting a commonly reported problem that materials are frequently written above the U.S. population's mean reading level of 7.5. The mean reading level of materials in the database was 9.7, reflecting a commonly reported problem that materials are frequently written above the U.S. population's mean reading level of 7.5 [7-9]. A related question asked whether health professionals accurately predicted the patient impact of a patient education piece. Pearson product-moment correlations were calculated on the major sections of the evaluation form: usefulness, understandability, and physical appeal of a patient education piece. Only one dimension showed a statistically significant correlation between the professional and patient evaluations: usefulness (a moderate correlation of 0.38, significant at the .05 level) (Figure 4). When pursued further to see if certain variables, such as professional group (e.g., practicing physician, resident, nurse) or gender differences accounted for this low correlation, no such relationships were found. The study also sought to determine if there was a statistically significant difference between evaluations completed by residents and by practicing family physicians. This would help plan contracting rela-
if
SPSS is
a
registered trademark of SPSS, Inc. Bull Med Libr Assoc 79(4) October 1991
Figure 4 Correlation of professional and patient evaluations
.
Patient education database
patient education/health promotion materials. Because patient education materials are by their nature ephemeral, constant updating of the database was required to add new materials and delete those that had gone out of print. The AAFP/F library does not serve as a warehouse for the materials listed, but rather directs the user to the source of the material and provides as much order information as is available.
Correlation of Professional and Patient Evaluations 4.6
-
y = 2.614 + 0.3908 x R = 0.38
4.4 a
"
4.2-
*
0 ~~~~~ mE
H 0
a
U
U
3.8
*
U~~~~~U
U
4.0-
U
3.6 3.4
3.6 3.8 4.0 4.2 4.4 Professional Ratings of Usefulness
3.4
Library staff anticipated that users would exercise the sophisticated search capability of the DEC software and request materials written about particular subjects, targeted to specific age groups, or written on a certain reading level. Surprisingly, most users have requested the entire printout. 4.6
Users have been able to access the database through AAFP/ F's Huffington Library since mid-1989. Use has accelerated over time as AAFP / F has promoted the service and explained how it may be of use to the practicing family physician. By mid-1990, it had become the second most frequently searched database in the AAFP/F library, second only to MEDLINE®::tt.
tionships with reviewers in the future. The analysis found no significant differences in the mean evaluations of the two groups (Figure 5). THE DATABASE
By August 1990, the health education database consisted of 302 records, all favorably reviewed print Figure 5 Average ratings by residents and practicing physicians
44 MEDLINE is a registered trademark of the National Library of Medicine.
_~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~....... -I
..
Appearance
,,.,
,.,,,,
0
Content
c 0
E C ._
Useful I ..... ..........................
Overall
3.74
,, ; ,
,,,,,,j,
El
...i...... t
,
,
,I
0.5
.
I,
1
1.5
Third Year Residents
3.94
.....3...75
1~~~~~~~~~~~~~~~~~3.83 4.05 13.76 3 76 7
9
~~~~~~~~~~~~~~~~~~3.96 3.71
I,
.
Second Year Residents
Practicing Physicians
13.72
3 ~~~~~~~~~~~~3~~~~~~~~~. 3.83
0
First Year Residents
3..77,,f,;,,S
.~~~~~~~~........ ..............
.
o
13.57
,.,,,.,...,
c
0
~~~~~~~~~~~~~3.53
13.52
.,
2
,II.
2.5
3
35
4
45
Mean Evaluations
Bull Med Libr Assoc 79(4) October 1991
363
Gibson et al.
Figure 6 Sample short report TIT[LE
DIET FOR THE YOUNG AT HEART
ORDER FROM:
§81-940
Library staff anticipated that users would exercise the sophisticated search capability of the DEC software and request materials written about particular subjects, targeted to specific age groups, or written on a certain reading level. Surprisingly, most users have requested the entire printout. Typically, family physicians are either starting or updating patient education libraries in their offices and want to review the entire listing of favorably reviewed items. Two types of reports are generated for users: a short report containing simple order information (Figure 6) and a long report with detailed information on each piece, including target audience and reading level (Figure 7).
BEYOND THE PILOT The pilot phase of the review project ended in mid1989. Because no other nationally accessible review service of patient education materials targeted at the family physician existed, and because the database was well received by family physicians in need of assistance in identifying accurate and useful patient education material, AAFP/F was committed to continuing the review process. Without grant funding, a self-supporting mechanism was necessary. A fee-based peer-review process was implemented in July 1989. At least three health professionals now review each piece. The panel of reviewers consists of 114 family physicians and other health professionals in family practice. Reviewers indicate their areas of special expertise and interest. Library staff seek to match the brochure's content with the reviewers' content expertise. The cut-off point for favorable review status remains an overall mean score of 3.5. Most producers choose to submit materials in final draft form, enabling them to factor reviewers' comments and concerns into final revisions of the patient education pieces. Favorably reviewed materials are entered in the database and are eligible to bear AAFP/F's seal and citation: "This health education material has been 364
PUBLICATION
DATE
SUMMARY
PRESENTS A BRIEF EXPLANATION OF THE PART DIET PLAYS IN HEART DISEASE. HOST OF THE BOOKLET IS MADE UP OF MENUS FOR ABOUT 2000 CALORIES A DAY, AND RECIPES WITH LESS THAN 2000 MILLIGRAMS OF SODIUMP 30X CALORIES FROM FAT. 172 FROM PROTEIN. BEST FOODS DIET FOR THE YOUNG AT HEART BOX 307 - DEPT. AAFP SQVENTRYP CT 06239
1986c1983
UNIT COST .50
NOTE COSTS ARE .50 EACH TO COVER POSTAGE AND HANDLING.
DATE
ENTEREDI
98/09/08
reviewed favorably by the American Academy of Family Physicians Foundation." To date, all pieces reviewed under the self-supporting program have come from for-profit producers, primarily pharmaceutical and nutrition companies. In order to preserve a balance in the database for materials from nonprofit agencies as well, a protocol for the review of nonprofit materials is under development. MOVE TO MICROCOMPUTER ENVIRONMENT In conjunction with an overall move of the AAFP/ F's computer files from a DEC VAX 11/780 to a microcomputer-based system, the health education database has been rebuilt, using Paradox® §§, version 3.5. Paradox was selected because of its searching capabilities and ease of use. This full-featured, high-end database program works with data tables in the way that a spreadsheet does, and with database forms as a database management system does. Tables containing the same data as in the DEC system have been developed. Information found in these tables is entered from screens similar to the DEC screens. This allows the same criteria to be used in evaluating materials as was used in the initial phase of the project. The tables have been interconnected to assist in tracking the status of a pamphlet under review, current reviewers of the piece, and its final score. The data from the DEC system have been downloaded to floppy disks and installed on the microcomputer, allowing library staff to continue to build on previous records. Paradox allows for Boolean searching, truncation of terms, and use of "GREATER THAN," "LESS THAN," and "EQUAL TO" operations. It is possible to search any field in a record and print a report based on the search results. The system will produce short and long reports similar to those produced by the §§ Paradox is a registered trademark of Borland International. Bull Med Libr Assoc 79(4) October 1991
Patient education database Figure 7 Sample long report
HYPERTENSION; HIGH BLOOD PRESSURE; BLOOD PRESSURE;
KEY WORDS:
TITLE: BLOOD PRESSURE CONTROL .50
UNIT COST:
ORDER FROM:
LANGUAGE: SIZE:
PUBLICATION DATE:
BULK AVAILABLE:
Y
REPRODUCTION RESTRICTIONS:
KRAMES COMMUNICATIONS 312 90TH ST. DALY CITY, CA 940151898 MEDIUM:
ENG
ILLUSTRATIONS:
Y
PHONE: 415-994-8800
PAMPHLET
READING GRADE LEVEL:
23 cm.
c1985
LENGTH:
[6 P.]
11
COLOR; CARTOONS;
SUMMARY EXPLAINS THE WAY TO CONTROL BLOOD PRESSURE IS, BY FOLLOWING THE PHYSICIAN'S ADVI CE AND RECEIVING SUPPORT FROM FAMILY AND FRIENDS. INCLUDES DIET, EXERCISE, AND HINTS FOR MEDICATION COMPLIANCE.
AUDIENCE:
PATIENT;
OTHER:
GENDER: EITHER
AGE: ELDERLY; ADULT; OTHER: ETHNIC FOCUS: OTHER:
NONE;
NOTE BULK RATES: 1-49 .50 EACH / 50-299 .35 EACH / 300-599 5 EACH / 1000+ .23 EACH / TRI-FOLD
PLACEMENT:
WAITING ROOM; EXAM ROOM; STORE ROOM;
.30 EACH / 600-999
.2
OTHER:
AUXILIARY MATERIAL:
PERSONALIZATION ALLOWED*: Y DESCRIBE: CHARlS FOR MEDICATION AND BP DOCUMENTATION TRANSLATION:
READING SCALE: DALE-CHALL AUTHORS: SERIES: WELLNESS WAY DATE ENTERED*: 88/09/08
Bull Med Libr Assoc 79(4) October 1991
365
Gibson et al.
DEC system, but in an enhanced format, either on a laser jet printer or exported to disk, either 3.5-inch or 5.25-inch. Paradox has networking capabilities, and the AAFP/F library plans to develop a local area network to allow multiple staff access to this and other inhouse databases. Dial access for outside users is also being explored.
CONCLUSION The health education database has filled a need within the family practice community, serving as a quality filter for patient education materials. Although targeted at family physicians, the database has potential usefulness for all health professionals engaged in primary care; it identifies pieces that are scientifically accurate and useful in the clinical setting from the enormous numbers of patient education pieces being published. However useful the database may be, AAFP/F recognizes that the database only directs the user to reliable pieces. AAFP/F does not purport in any way to have made a comprehensive review of all patient education materials on a given topic.
practice. In: American Medical Association. Directory of graduate medical education programs, 1989-90. Chicago: The Association, 1989:25-6. 2. AMEIUcAN ACADEMY OF FAMILY PHYsicLANs. Recommended core curriculum guidelines on health promotion for family practice residents. Kansas City, MO: The Academy, 1985. 3. AMRIcAN ACADEMY OF FAMILY PHYsICIANs. Recommended core curriculum guidelines on disease prevention for family practice residents. Kansas City, MO: The Academy, 1986. 4. DAviEs NE. Bringing order to the literature of health education. N Engl J Med 1980 Jun 26;302(26):1476. 5. REEs AM, HoFFMAN C. Consumer health information source book. 3d ed. Phoenix: Oryx Press, 1990. 6. 1983 guide to health information resources in print. Daly City, CA: PAS, 1983. 7. VIVLAN AS, ROBERTSON EJ. Reliability of patient education materials. Clin Ther 1980;3(2):129-36. 8. TAYLOR AG, SKELTON JA, CzAjKowsKI RW. Do patients understand patient education brochures? Nurs Health Care 1982 Jun;(6):305-10. 9. DoAK LG, DoAK CC. Patient comprehension profiles: recent findings and strategies. Patient Couns Health Educ 1980;2(3):101.
Received September 1990; accepted January 1991
REFERENCES 1. ACCREDITATION COUNCIL FOR GRADUATE MEDICAL EDUCA-
TION. Special requirements for residency training in family
366
Bull Med Libr Assoc 79(4) October 1991
Patient education database APPENDIX A* Health Promotion Project Materials Evaluation Questionnaire
Directions:
Title: Objective:
After reading over the material attached to this survey, please indicate your opinion by answering the questions below. If the question does not seem to apply, please circle NA. Comments are welcome. Please comment if you disagree or strongly disagree with any of the statements. Thank you. In a few words, what do you think is the main purpose of this material?
It is designed to 1. reinforce information. 2. provide new information. 3. stimulate behavior change.
Appearance:
Please rate the appearance of the material by answering each question below. 4. At first glance it attracted my attention. 5. It held my attention. 6. Overall appearance 7. Quality of illustrations 8. Use of color 9. Typeface (large enough, attractive, etc.) 10. Highlighting of major concepts Comments
Content: Please rate the content of the material. 1 1. Up-to-date 12. Scientifically accurate 13. Adequate scope for objective(s) 14. Overall organization 15. Logical flow of ideas 16. Needed background given to enable understanding 17. Summary(ies) given when needed 18. The management of fear content 19. Fair presentation given (e.g., avoids sexism, ethnic bias, ageism, manufacturer bias, etc.) If a fair presentation is not given: 20. Does the bias interfere with the intent of the item?
Usefulness:
Strongly disagree Disagree 1 2 1 2 2 1
Very low Low appeal appeal 1 2 2 1 2 1 1 2 1 2 2 1 1 2
3 3 3
Agree 4 4 4
Strongly agree 5 5 5
NA NA NA
3 3 3 3 3 3 3
High Very high appeal appeal 4 5 4 5 5 4 4 5 5 4 4 5 4 5
NA NA NA NA NA NA NA
Very
Very poor
Poor
1 1 1 1 1
2 2 2 2 2
1 1 1 1
Good
good
3 3 3 3 3
4 4 4 4 4
5 5 5 5
NA NA NA NA NA
2 2 2
3 3 3
4 4 4
5 5 5
NA NA NA
2
3
4
5
NA
Yes No __
Please respond to the statements below by circling the best response. Strongly
Strongly
disagree Disagree
21. 22. 23. 24. *
It is useful for its intended audience. It is believable. The material is understandable. Requires little or no explanation.
1 1 1 1
2 2 2 2
(1) (2)
3 3 3 3
Agree
agree
4 4 4 4
5 5 5 5
NA NA NA NA
For use by health professionals.
Bull Med Libr Assoc 79(4) October 1991
367
Gibson et al. Comments
Overall: Please respond to the statements below. 25. Overall I would recommend that physicians use this material with patients. 26. Overall this material meets its objectives. Remarks:
Placement: Please circle all that apply. 27. Should be placed in the physician's waiting
Strongly disagree Disagree 1 1
Strongly disagree Disagree 1 1 1
room.
28. Should be placed in the exam room. 29. Should be stored for occasional use. 30. Other suggested settings Comments:
Please take a few seconds and complete the following: Professional status: a. Resident b. Family physician faculty c. Practicing physician d. R.N. e. Other (Specify)
3 3
2 2
O E O O E
PGY 1 under 6 under 6 under 6 under 6
2 2 2
O O l E O
3 3 3
Agree
Strongly agree
4 4
5 5
Agree
Strongly agree
4 4 4
5 5 5
NA NA
NA NA NA
Number of years PGY2 O PGY 3 O 26 or over 6-15 O 16-25 E 26 or over 6-15 O 16-25 O 26 or over 6-15 O 16-25 E 26 or over E 16-25 6-15
Gender:
Male Female
(1) (2)
Age Range:
Under 25 25-35
(1) (2) (3) (4) (5) (6)
36-45 46-55 56-65 over 65 (Please print)
368
Bull Med Libr Assoc 79(4) October 1991
Patient education database APPENDIX B* Directions:
Please tell us what you think of the material attached to this form by answering all the questions below. For each question circle the number that best describes how you feel. If the question does not seem to apply to the material you are reviewing, please circle NA. If you wish to write comments, we would value your additional ideas.
Title: Topic:
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.
Strongly disagree Disagree At first glance it attracted my attention. 1 2 It held my attention. 1 2 It is useful. 1 2 I like the illustrations. 1 2 I believe what it has to say. 1 2 I would recommend it to a friend or relative to read. 1 2 It is easy to understand. 1 2 What it says is important. 1 2 It reminds me of some things I need to think about. 1 2 It gives me some new things to think about. 1 2 It changes some of my thinking. 1 2 It could change how I do things. 1 2 Overall I recommend that doctors use this material with patients. 1 2 Overall I am the right person to get this material from my doctor. 1 2 Overall this material accomplishes its main purpose. 1 2
Purpose:
In just a few words, please write what you think is th ie main
purpose
Strongly
3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
Agree
agree
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
5 5 5 5 5 5 5 5 5 5 5 5 5 5 5
NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
of this material.
Comments:
We would appreciate learning
a
little
more
about you.
Male
Female Occunation
Years of school completed Age range: ( ) under 12 ( ) 12-15 ) 16-24 ( ) 25-35 ) 36-45 ( ) 46-55 ( ) 56-65 ( ) Over 65
Thank you! *
For use by patients.
Bull Med Libr Assoc 79(4) October 1991
369
Vice-President, Information Services and Systems Carolyn Ruby, M.A.L.S. Medical Librarian American Academy of Family Physicians Foundation
Huffington Library 8880 Ward Parkway P.O. Box 8418 Kansas City, Missouri 64114-0418 Marian D. Craig, M.A.L.S.t Information Resources Manager
Marion Merrill Dow, Inc. Baptiste-P.O. Box 9627 Kansas City, Missouri 64134-0627
Using pilot project funding from the W. K. Kellogg Foundation, the American Academy of Family Physicians Foundation (AAFP/F) developed a program by which health/patient education print materials were reviewed. Favorably reviewed materials were entered into a database accessible through the AAFP/F's Huffington Library. The review service and resulting database were designed to help the busy clinician identify scientifically accurate, reliable materials for use in patient education. The review process developed for the project is described, as is the database and its use by family physicians. Research findings from the pilot project are discussed, some of which assisted in planning the self-supporting second phase of the program.
In 1981 the Board of Trustees of the American Academy of Family Physicians Foundation (AAFP/F) began to plan a national library and information center
for family physicians. The plan included a clearinghouse of evaluated patient education materials. The trustees viewed evaluation as a critical element in this effort, because they perceived no other source that evaluated health/patient education materials from the perspective of the family physician. The specialty of family practice, established in 1969, has a strong commitment to disease prevention and * This program was supported in part by a grant from the W. K. Kellogg Foundation. t Based on a paper presented May 24, 1988, at the Eighty-Eighth Annual Meeting of the Medical Library Association, New Orleans, Louisiana. t Formerly technical services librarian, American Academy of Family Physicians Foundation.
Bull Med Libr Assoc 79(4) October 1991
patient education. The "Special Requirements for Residency Training in Family Practice" specifically stated that residents must receive instruction in health assessment and maintenance, prevention, behavioral counseling, and health education [1]. Elaborating on this document in 1985, AAFP formulated Recommended Core Curriculum Guidelines on Health Promotion for Family Practice Residents, which detailed specific attitudes, knowledge, and skills required for health promotion [2]. In 1986 AAFP developed Recommended Core Curriculum Guidelines on Disease Prevention for Family Practice Residents, which detailed the attitudes, knowledge, and skills required for disease prevention [3]. Recognizing, however, in 1981 that 58% of AAFP's overall membership of 70,000 was already in practice and, therefore, beyond residency training, AAFP created the Preventive Health Care Subcommittee of the Commission on Public Health and Scientific Affairs. 357
Gibson et al.
Its purpose was to study methods for introducing preventive medicine techniques into family practice, to acquaint existing practitioners with health promotion and preventive medicine measures, and to address the continuing education needs of residents completing their training. In 1984, AAFP also created the Health Education Committee to identify and develop health education methods and materials for family physicians, both in practice and in training. Both these groups provided useful input as this project evolved.
The specialty of family practice, established in 1969, has a strong commitment to disease prevention and patient education. THE PROBLEM
The dilemma faced by busy clinicians as they expand their practices to include a health promotion/disease prevention orientation has been vividly described by Davies as primarily a problem of an overabundance of health promotion/patient education materials, few of which have been appropriately evaluated. Davies described the health promotion literature as an "overwhelming assortment of science, pseudo-science, tenuous theory and unmitigated clap-trap" [4]. Much is product advertising and therefore suspect even when basically sound. Furthermore, the health promotion program literature is largely ephemeral, consisting primarily of pamphlets of all shapes and sizes. A cursory review of standard reference works in the patient education/health promotion literature revealed many sources of materials, including voluntary health agencies, pharmaceutical companies, professional associations, governmental agencies, and companies whose sole purpose is to publish patient education literature [5-6]. NEEDS ASSESSMENT In order to identify the specific concerns of family physicians in health promotion and patient education, the AAFP/F library staff conducted a needs assessment survey during November 1985. A questionnaire sent to 2,258 AAFP member users of the Huffington Library elicited a response rate of 39%. The questionnaire was designed to answer the following questions. Do family physicians believe patients want to assume an active role in their own health care? Do family physicians need evaluated health promotion materials? Would they use the health education clearinghouse? What health promotion programs have they developed in their own 358
communities? What topics would they like to see addressed first by the clearinghouse? The responses to the questionnaire provided a strong statement of need for assistance with health promotion in the family practice setting. Responses to selected questions appear in Table 1.
The dilemma faced by busy clinicians as they expand their practices to include a health promotion! disease prevention orientation has been vividly described by Davies as primarily a problem of an overabundance of health promotion/patient education materials, few of which have been appropriately evaluated. Respondents were also given a list of fifty-one disease-related topics and thirty-eight health promotion/wellness topics. The highest-ranked topics, followed by the number and percentage of respondents checking the topic, appear in Tables 2 and 3. THE GRANT PROPOSAL
Based upon the directive from AAFP/F's Board of Trustees, a review of the literature, and the results of the needs assessment, AAFP/F submitted a grant proposal to the W. K. Kellogg Foundation, from which funding was received in November 1986. The project was designed as a two-year pilot to develop a mechanism for evaluating patient education materials, placing favorably reviewed materials in a database available to family physicians, and testing the effectiveness of the review process and the resulting database as a service to family practice. Only brief print materials (brochures and short booklets) were to be reviewed.
The responses to the questionnaire provided a strong statement of need for assistance with health promotion in the family practice setting. INITIAL IMPLEMENTATION A steering committee was formed to advise the project director in developing the program, selecting reviewers for the program, and monitoring the overall progress of the project. The committee consisted of five family physicians, two of whom were in full-time family practice and two of whom were faculty of Bull Med Libr Assoc 79(4) October 1991
Patient education database
Table 1
Response to selected questions from needs assessment (n = 873*)
1. Most patients want to assume an increasingly active and participative role in their own health care (both prevention and cure). 2. It will be increasingly important for family physicians to provide patients with health promotion and patient education materials that are accurate and easy to comprehend. 3. Health promotion and patient education materials that have been evaluated for accuracy and clarity by either a family physician or a heaith educator are more reliable than materials evaluated by others or not evaluated at all. 4. My health promotion and patient education efforts would be improved if materials were prescreened by health care professionals for accuracy, style, readability, etc.
No Strongly disagree opinion Number (%) Number (%) Number (%)
Moderately disagree
Strongly agree Number (%)
Moderately agree Number (%)
236(27.0%)
567(65.0%)
59(6.8%)
7(0.8%)
2(0.2%)
871 (99.8%)
617 (70.7%)
232 (26.6%)
15 (1.7%)
4(0.5%)
4(0.5%)
872 (100.0%)
471 (54.0%)
314 (36.0%)
29 (3.3%)
1 (0.1%)
55(6.3%)
870(99.7%)
454 (52.0%/a)
351 (40.2%)
27 (3.1%)
6(0.7%)
29(3.3%)
867 (99.3%)
Total Number (%)
Not all respondents answered all questions.
family practice residency programs, and a nonphysician patient educator. The committee was chaired by the AAFP/F's library committee chair, a family physician in private practice. Because family practice residency curricula emphasize patient education, AAFP/F contracted with three residency programs to complete the reviews. To identify three residencies with strong patient education/health promotion programs, all 382 U.S. family practice residencies were invited to complete an extensive application, mailed in January 1987. A total of sixty-eight applications was received. After a blind review of the applications, the steering committee Table 2 Ranked list of disease-related topics (n
=
873*) Number of respondents checking
Number of respondents checking
Topic
Backachet Hypertensiont Depressiont Headachet
Arthritist
Diabetes mellitust
Obesityt Osteoporosist
Irritable bowel syndrome Alzheimer's disease Premenstrual tension Fibrocystic disease of breast
Vaginitis Asthma *
(%)
Topic
519 (59.4%) 511 (58.5%) 475 (54.4%) 416 (47.6%) 407 (46.6%)
Sexually transmitted diseases Otitis media Allergic rhinitis Emphysema
406(46.5%) 406(46.5%) 361 (41.3%)
Acne Insomnia
306 (35.00/a) 278 (31.9%) 246 (28.2%b) 244 (28.0%) 237 (27.2%) 225 (25.8%)
identified six family practice residencies as finalists. After a site visit to each of these residencies, followed by an interview in Kansas City with representatives of each of the finalists, the steering committee selected three family practice residencies to participate in the project: East Carolina University Department of Family Medicine, Greenville, North Carolina; Forbes Health System, Monroeville, Pennsylvania; and University of Colorado Department of Family Medicine, Denver, Colorado. AAFP/F paid each participating residency the equivalent of a patient educator's salary to evaluate patient education materials and provided funds for procuring materials.
Menstruation disorders
AlDSt Cystitis Vertigo Peptic ulcer Esophageal reflux Pregnancy complications
Topic
(%)
218 (25.0Yo) 211 (24.2Yo) 208 (23.8%) 203(23.2%) 197 (22.6%) 168 (19.3%) 163 (18.7%) 161 (18.4%) 141 (16.2°h) 124 (14.2%) 101 (11.6%)
100(11.5%) 97 (11.1%)
Coronary bypass Prostate disease Hyperactive children Cervical strain Stroke rehabilitation Food allergies
Temperomandibular joint syndrome Anorexia nervosa and bulimia Insect bites
Hyperventilation syndrome Tinnitus Infertility Eczema
Number of respondents checking (%)
93(10.6%) 86(9.9%) 85 (9.7%) 79 (9.1%) 78 (8.9%) 74 (8.5%) 67 (7.7%) 53 (6.1%) 52 (6.0%)
51(6.0%) 47(5.4%) 47(5.4%) 43 (4.9%)
Respondents could check any number of categories.
t Topics covered by pilot project.
Bull Med Libr Assoc 79(4) October 1991
359
Gibson et al.
Figure 1 Procedures flow chart
THE REVIEW PROCESS
The patient educator in each participating residency was responsible for locating as many print materials as possible on three health promotion and five dis-
ease-related topics, corresponding to the ranked topics on the needs assessment. Some adjustments were made to obtain a good match with the special content expertise in each residency. The disease-related list was reordered to include AIDS. A Delphi-like process was adopted through which each residency identified as many materials as possible on its designated subjects. The patient educator eliminated those materials not deserving further review, most frequently because of content inaccuracy
or bias. The patient educator mailed copies of materials worthy of review to the other two residencies. The same materials were then evaluated by all three residencies (Figure 1). The patient educators helped design the two evaluation forms used in the project, one form completed by health professionals and the other by patients. Field tests of the forms in the residencies resulted in minor changes. To determine the reliability or internal consistency of the forms, a Cronbach's alpha was calculated on both the professional and patient evaluation forms. The professional form scored a 0.94 reliability coefficient (Figure 2), and the patient form scored 0.92 (Figure 3). The health professional form asked the reviewer to state the objective of the piece and then to respond on a five-point scale to thirty questions relating to the appearance of the piece, its content, its usefulness, the reviewer's overall reaction to the piece, and the most appropriate placement for the piece (e.g., waiting room, exam room). At the bottom of the form, the health professional was asked to supply demographic data (Appendix A). On the patient review form, respondents were asked to state the overall subject of the piece and then were asked to respond on a five-point scale to fifteen questions corresponding to questions on the health professional's form. The questions related to the piece's appearance, content, and usefulness. Patients were asked to state the purpose of the item and were invited to comment on the piece. At the bottom of the form, the patient was also asked to supply demographic data (Appendix B).
Table 3 Ranked list of health promotion/wellness and other topics (n = 873*)
Topic
(%)
Smoking cessationt Mammography and breast
768(88%)
self-examt Stress managementt
716 (82%) 698 (80%) 681 (78%) 637 (73%) 629 (72%) 611 (70%) 594 (68%)
Alcohol abuset Hemoccult blood testt
Contraceptiont
Pap smear and pelvic examt
Sigmoidoscopyt
Car safety, including seat beltst Drug abuset Child developmentt Childhood immunizationt Home accident preventiont *
585(67%) 567 (65%) 559 (64%) 550(63%) 550 (63%)
Number of
Number of respondents checking
Number of respondents checking
Topic Aerobic exerciset
Calcium, dietaryt Fiber, dietaryt
Adult immunizationt Adoescence Divorce Menopause, male and female Athletic injuries, including prevention Prostate physical exam Retirement and old age Breast feeding Single parenting Grieving and bereavement
(%) 541 (62%) 524 (60%) 524 (60%) 515 (59%) 515 (59%) 514 (59%)
506(58%) 489 (56%) 489(56%) 489 (56%) 471 (54%) 445 (51%)
444 (51%)
Topic Diet fads Nutrition for specific disease prevention
Prenatal care Empty nest syndrome Vitamin and mineral supplements
Outdoor safety, including camping DPT immunization General meal planning Intemational travel Natural childbirth Food additives Insemination, artificial
respondents checking (%)
437(50%h) 410 (47%) 375 (43%/6) 367 (42%) 340(39%) 314(36%) 313(36%) 312(36%) 236(27%) 192 (22%) 175 (20%) 44(5%)
Respondents could check any number of categories.
t Topics covered by pilot project.
360
Bull Med Libr Assoc 79(4) October 1991
.0.54 7
Patient education database
Figure 2 Reliability of professional rating form (n = 143) Total Form Reinforces Information Provides New Information Stimulates Behavior Change_ Attracts Attention Holds Attention Overall Appearance_ Quality of Illustrations_ Use of Color Typeface Use of Highlighting Current Information Accuracy _ Adequate Scope_ Overall Organization_ Flow of Ideas Needed Background Given Use of Summaries
in QA
522
527
JO 731
tD52 6JU I652
593
/92
=0 796 =0 786
Usefulness Believable Understandable Requires Little Explanation_ Recommend Use with Patients Meets its Objective
D 7,42
) 742
-I 769
0.1
0
0.2
0.3
04
0.6
05
0.7
0.8
0.9
1
Reliability
Figure 3 Reliability of patient rating form (n = 992) Total Form Accomplishes Main Purpose Right Brochure for Me Recommend Use with Others Changed My Behavior Changed My Thinking Provided New Information Reminded Me of Things Contains Important Information Understandable I'd Recommend to Another Believable Quality of Illustrations Usefulness Held My Attention Attracted My Attention
10 635
U0 74
o0.643 ] 0.639
=0.675
0.69
l 0634
0.592
0.54
-0 647
=0.677
I.
0
Bull Med Libr Assoc 79(4) October 1991
I.I
0.1
0.2
03
0.4
06
05 0.6 Reliability
0.7
0.8
0
os9
1
361
Gibson et al.
The pilot project aimed to obtain a minimum of twenty-one patient evaluations and the same number of health professional evaluations for each patient education piece to adequately test the evaluation forms and the overall review process. Each patient educator completed a form supplying descriptive information on all materials reviewed in the eight assigned topic areas and submitted two copies of all materials reviewed to the AAFP/F library. If that piece was eliminated, the patient educator indicated the reason for its elimination, e.g., bias, scientific inaccuracy, unattractive format.
On the patient review form, respondents were asked to state the overall subject of the piece and then were asked to respond on a five-point scale to fifteen questions corresponding to questions on the health professional's form. If the piece deserved further review, the full descriptive form was completed and the reading level of the piece was computed with microcomputer software, using the Dale-Chall Scale for materials written for fifth grade and above and the Spache Scale for materials below the fifth grade level. DATABASE CREATION The AAFP/F technical services librarian created the database record, adding descriptive catalog information to the data supplied by the patient educator. The database was built on a DEC VAX 11/780 minicomputer. Written in COBOL, the database program was called FPB (Family Practice Base), an interactive query and file maintenance system designed for use on VAX/VMS systems. The software package Datatrieve®§ performed the statistical calculations required to score each review. A mean score of 3.5 or higher assigned by health professionals constituted a favorable review and resulted in inclusion in the database. Datatrieve calculated mean scores for each question and also totaled scores for each category on the evaluation form, for both health professional and patient reviewers. The health education database record was displayed on a series of three screens. A maximum of six subject headings or key words were allowed per record, of which a maximum of three were MeSH ® * * § Datatrieve is a registered trademark of Digital Equipment Corporation. * * MeSH is a registered trademark of the National Library of Medicine.
362
terms. A maximum of three were lay subject terms suggested by the patient educators. Reports were generated through the software's "query" mode, which allowed Boolean searching on any field or combination of fields in the record. Statistical reports were generated through SPSS®t.
RESEARCH FINDINGS AAFP/F wanted the project to determine if there was general agreement between health professionals and patients regarding the accuracy, physical appeal, and usefulness of a particular patient education piece. Therefore, the evaluation forms used by the two groups of reviewers were designed to contain equivalent questions on these aspects of patient education materials. Materials reviewed by patients had a mean score of 3.99; those evaluated by health professional reviewers scored a mean of 3.91. Although these scores suggested that health professionals tended to be more critical than patients, the differences were not statistically significant.
The mean reading level of materials in the database was 9.7, reflecting a commonly reported problem that materials are frequently written above the U.S. population's mean reading level of 7.5. The mean reading level of materials in the database was 9.7, reflecting a commonly reported problem that materials are frequently written above the U.S. population's mean reading level of 7.5 [7-9]. A related question asked whether health professionals accurately predicted the patient impact of a patient education piece. Pearson product-moment correlations were calculated on the major sections of the evaluation form: usefulness, understandability, and physical appeal of a patient education piece. Only one dimension showed a statistically significant correlation between the professional and patient evaluations: usefulness (a moderate correlation of 0.38, significant at the .05 level) (Figure 4). When pursued further to see if certain variables, such as professional group (e.g., practicing physician, resident, nurse) or gender differences accounted for this low correlation, no such relationships were found. The study also sought to determine if there was a statistically significant difference between evaluations completed by residents and by practicing family physicians. This would help plan contracting rela-
if
SPSS is
a
registered trademark of SPSS, Inc. Bull Med Libr Assoc 79(4) October 1991
Figure 4 Correlation of professional and patient evaluations
.
Patient education database
patient education/health promotion materials. Because patient education materials are by their nature ephemeral, constant updating of the database was required to add new materials and delete those that had gone out of print. The AAFP/F library does not serve as a warehouse for the materials listed, but rather directs the user to the source of the material and provides as much order information as is available.
Correlation of Professional and Patient Evaluations 4.6
-
y = 2.614 + 0.3908 x R = 0.38
4.4 a
"
4.2-
*
0 ~~~~~ mE
H 0
a
U
U
3.8
*
U~~~~~U
U
4.0-
U
3.6 3.4
3.6 3.8 4.0 4.2 4.4 Professional Ratings of Usefulness
3.4
Library staff anticipated that users would exercise the sophisticated search capability of the DEC software and request materials written about particular subjects, targeted to specific age groups, or written on a certain reading level. Surprisingly, most users have requested the entire printout. 4.6
Users have been able to access the database through AAFP/ F's Huffington Library since mid-1989. Use has accelerated over time as AAFP / F has promoted the service and explained how it may be of use to the practicing family physician. By mid-1990, it had become the second most frequently searched database in the AAFP/F library, second only to MEDLINE®::tt.
tionships with reviewers in the future. The analysis found no significant differences in the mean evaluations of the two groups (Figure 5). THE DATABASE
By August 1990, the health education database consisted of 302 records, all favorably reviewed print Figure 5 Average ratings by residents and practicing physicians
44 MEDLINE is a registered trademark of the National Library of Medicine.
_~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~....... -I
..
Appearance
,,.,
,.,,,,
0
Content
c 0
E C ._
Useful I ..... ..........................
Overall
3.74
,, ; ,
,,,,,,j,
El
...i...... t
,
,
,I
0.5
.
I,
1
1.5
Third Year Residents
3.94
.....3...75
1~~~~~~~~~~~~~~~~~3.83 4.05 13.76 3 76 7
9
~~~~~~~~~~~~~~~~~~3.96 3.71
I,
.
Second Year Residents
Practicing Physicians
13.72
3 ~~~~~~~~~~~~3~~~~~~~~~. 3.83
0
First Year Residents
3..77,,f,;,,S
.~~~~~~~~........ ..............
.
o
13.57
,.,,,.,...,
c
0
~~~~~~~~~~~~~3.53
13.52
.,
2
,II.
2.5
3
35
4
45
Mean Evaluations
Bull Med Libr Assoc 79(4) October 1991
363
Gibson et al.
Figure 6 Sample short report TIT[LE
DIET FOR THE YOUNG AT HEART
ORDER FROM:
§81-940
Library staff anticipated that users would exercise the sophisticated search capability of the DEC software and request materials written about particular subjects, targeted to specific age groups, or written on a certain reading level. Surprisingly, most users have requested the entire printout. Typically, family physicians are either starting or updating patient education libraries in their offices and want to review the entire listing of favorably reviewed items. Two types of reports are generated for users: a short report containing simple order information (Figure 6) and a long report with detailed information on each piece, including target audience and reading level (Figure 7).
BEYOND THE PILOT The pilot phase of the review project ended in mid1989. Because no other nationally accessible review service of patient education materials targeted at the family physician existed, and because the database was well received by family physicians in need of assistance in identifying accurate and useful patient education material, AAFP/F was committed to continuing the review process. Without grant funding, a self-supporting mechanism was necessary. A fee-based peer-review process was implemented in July 1989. At least three health professionals now review each piece. The panel of reviewers consists of 114 family physicians and other health professionals in family practice. Reviewers indicate their areas of special expertise and interest. Library staff seek to match the brochure's content with the reviewers' content expertise. The cut-off point for favorable review status remains an overall mean score of 3.5. Most producers choose to submit materials in final draft form, enabling them to factor reviewers' comments and concerns into final revisions of the patient education pieces. Favorably reviewed materials are entered in the database and are eligible to bear AAFP/F's seal and citation: "This health education material has been 364
PUBLICATION
DATE
SUMMARY
PRESENTS A BRIEF EXPLANATION OF THE PART DIET PLAYS IN HEART DISEASE. HOST OF THE BOOKLET IS MADE UP OF MENUS FOR ABOUT 2000 CALORIES A DAY, AND RECIPES WITH LESS THAN 2000 MILLIGRAMS OF SODIUMP 30X CALORIES FROM FAT. 172 FROM PROTEIN. BEST FOODS DIET FOR THE YOUNG AT HEART BOX 307 - DEPT. AAFP SQVENTRYP CT 06239
1986c1983
UNIT COST .50
NOTE COSTS ARE .50 EACH TO COVER POSTAGE AND HANDLING.
DATE
ENTEREDI
98/09/08
reviewed favorably by the American Academy of Family Physicians Foundation." To date, all pieces reviewed under the self-supporting program have come from for-profit producers, primarily pharmaceutical and nutrition companies. In order to preserve a balance in the database for materials from nonprofit agencies as well, a protocol for the review of nonprofit materials is under development. MOVE TO MICROCOMPUTER ENVIRONMENT In conjunction with an overall move of the AAFP/ F's computer files from a DEC VAX 11/780 to a microcomputer-based system, the health education database has been rebuilt, using Paradox® §§, version 3.5. Paradox was selected because of its searching capabilities and ease of use. This full-featured, high-end database program works with data tables in the way that a spreadsheet does, and with database forms as a database management system does. Tables containing the same data as in the DEC system have been developed. Information found in these tables is entered from screens similar to the DEC screens. This allows the same criteria to be used in evaluating materials as was used in the initial phase of the project. The tables have been interconnected to assist in tracking the status of a pamphlet under review, current reviewers of the piece, and its final score. The data from the DEC system have been downloaded to floppy disks and installed on the microcomputer, allowing library staff to continue to build on previous records. Paradox allows for Boolean searching, truncation of terms, and use of "GREATER THAN," "LESS THAN," and "EQUAL TO" operations. It is possible to search any field in a record and print a report based on the search results. The system will produce short and long reports similar to those produced by the §§ Paradox is a registered trademark of Borland International. Bull Med Libr Assoc 79(4) October 1991
Patient education database Figure 7 Sample long report
HYPERTENSION; HIGH BLOOD PRESSURE; BLOOD PRESSURE;
KEY WORDS:
TITLE: BLOOD PRESSURE CONTROL .50
UNIT COST:
ORDER FROM:
LANGUAGE: SIZE:
PUBLICATION DATE:
BULK AVAILABLE:
Y
REPRODUCTION RESTRICTIONS:
KRAMES COMMUNICATIONS 312 90TH ST. DALY CITY, CA 940151898 MEDIUM:
ENG
ILLUSTRATIONS:
Y
PHONE: 415-994-8800
PAMPHLET
READING GRADE LEVEL:
23 cm.
c1985
LENGTH:
[6 P.]
11
COLOR; CARTOONS;
SUMMARY EXPLAINS THE WAY TO CONTROL BLOOD PRESSURE IS, BY FOLLOWING THE PHYSICIAN'S ADVI CE AND RECEIVING SUPPORT FROM FAMILY AND FRIENDS. INCLUDES DIET, EXERCISE, AND HINTS FOR MEDICATION COMPLIANCE.
AUDIENCE:
PATIENT;
OTHER:
GENDER: EITHER
AGE: ELDERLY; ADULT; OTHER: ETHNIC FOCUS: OTHER:
NONE;
NOTE BULK RATES: 1-49 .50 EACH / 50-299 .35 EACH / 300-599 5 EACH / 1000+ .23 EACH / TRI-FOLD
PLACEMENT:
WAITING ROOM; EXAM ROOM; STORE ROOM;
.30 EACH / 600-999
.2
OTHER:
AUXILIARY MATERIAL:
PERSONALIZATION ALLOWED*: Y DESCRIBE: CHARlS FOR MEDICATION AND BP DOCUMENTATION TRANSLATION:
READING SCALE: DALE-CHALL AUTHORS: SERIES: WELLNESS WAY DATE ENTERED*: 88/09/08
Bull Med Libr Assoc 79(4) October 1991
365
Gibson et al.
DEC system, but in an enhanced format, either on a laser jet printer or exported to disk, either 3.5-inch or 5.25-inch. Paradox has networking capabilities, and the AAFP/F library plans to develop a local area network to allow multiple staff access to this and other inhouse databases. Dial access for outside users is also being explored.
CONCLUSION The health education database has filled a need within the family practice community, serving as a quality filter for patient education materials. Although targeted at family physicians, the database has potential usefulness for all health professionals engaged in primary care; it identifies pieces that are scientifically accurate and useful in the clinical setting from the enormous numbers of patient education pieces being published. However useful the database may be, AAFP/F recognizes that the database only directs the user to reliable pieces. AAFP/F does not purport in any way to have made a comprehensive review of all patient education materials on a given topic.
practice. In: American Medical Association. Directory of graduate medical education programs, 1989-90. Chicago: The Association, 1989:25-6. 2. AMEIUcAN ACADEMY OF FAMILY PHYsicLANs. Recommended core curriculum guidelines on health promotion for family practice residents. Kansas City, MO: The Academy, 1985. 3. AMRIcAN ACADEMY OF FAMILY PHYsICIANs. Recommended core curriculum guidelines on disease prevention for family practice residents. Kansas City, MO: The Academy, 1986. 4. DAviEs NE. Bringing order to the literature of health education. N Engl J Med 1980 Jun 26;302(26):1476. 5. REEs AM, HoFFMAN C. Consumer health information source book. 3d ed. Phoenix: Oryx Press, 1990. 6. 1983 guide to health information resources in print. Daly City, CA: PAS, 1983. 7. VIVLAN AS, ROBERTSON EJ. Reliability of patient education materials. Clin Ther 1980;3(2):129-36. 8. TAYLOR AG, SKELTON JA, CzAjKowsKI RW. Do patients understand patient education brochures? Nurs Health Care 1982 Jun;(6):305-10. 9. DoAK LG, DoAK CC. Patient comprehension profiles: recent findings and strategies. Patient Couns Health Educ 1980;2(3):101.
Received September 1990; accepted January 1991
REFERENCES 1. ACCREDITATION COUNCIL FOR GRADUATE MEDICAL EDUCA-
TION. Special requirements for residency training in family
366
Bull Med Libr Assoc 79(4) October 1991
Patient education database APPENDIX A* Health Promotion Project Materials Evaluation Questionnaire
Directions:
Title: Objective:
After reading over the material attached to this survey, please indicate your opinion by answering the questions below. If the question does not seem to apply, please circle NA. Comments are welcome. Please comment if you disagree or strongly disagree with any of the statements. Thank you. In a few words, what do you think is the main purpose of this material?
It is designed to 1. reinforce information. 2. provide new information. 3. stimulate behavior change.
Appearance:
Please rate the appearance of the material by answering each question below. 4. At first glance it attracted my attention. 5. It held my attention. 6. Overall appearance 7. Quality of illustrations 8. Use of color 9. Typeface (large enough, attractive, etc.) 10. Highlighting of major concepts Comments
Content: Please rate the content of the material. 1 1. Up-to-date 12. Scientifically accurate 13. Adequate scope for objective(s) 14. Overall organization 15. Logical flow of ideas 16. Needed background given to enable understanding 17. Summary(ies) given when needed 18. The management of fear content 19. Fair presentation given (e.g., avoids sexism, ethnic bias, ageism, manufacturer bias, etc.) If a fair presentation is not given: 20. Does the bias interfere with the intent of the item?
Usefulness:
Strongly disagree Disagree 1 2 1 2 2 1
Very low Low appeal appeal 1 2 2 1 2 1 1 2 1 2 2 1 1 2
3 3 3
Agree 4 4 4
Strongly agree 5 5 5
NA NA NA
3 3 3 3 3 3 3
High Very high appeal appeal 4 5 4 5 5 4 4 5 5 4 4 5 4 5
NA NA NA NA NA NA NA
Very
Very poor
Poor
1 1 1 1 1
2 2 2 2 2
1 1 1 1
Good
good
3 3 3 3 3
4 4 4 4 4
5 5 5 5
NA NA NA NA NA
2 2 2
3 3 3
4 4 4
5 5 5
NA NA NA
2
3
4
5
NA
Yes No __
Please respond to the statements below by circling the best response. Strongly
Strongly
disagree Disagree
21. 22. 23. 24. *
It is useful for its intended audience. It is believable. The material is understandable. Requires little or no explanation.
1 1 1 1
2 2 2 2
(1) (2)
3 3 3 3
Agree
agree
4 4 4 4
5 5 5 5
NA NA NA NA
For use by health professionals.
Bull Med Libr Assoc 79(4) October 1991
367
Gibson et al. Comments
Overall: Please respond to the statements below. 25. Overall I would recommend that physicians use this material with patients. 26. Overall this material meets its objectives. Remarks:
Placement: Please circle all that apply. 27. Should be placed in the physician's waiting
Strongly disagree Disagree 1 1
Strongly disagree Disagree 1 1 1
room.
28. Should be placed in the exam room. 29. Should be stored for occasional use. 30. Other suggested settings Comments:
Please take a few seconds and complete the following: Professional status: a. Resident b. Family physician faculty c. Practicing physician d. R.N. e. Other (Specify)
3 3
2 2
O E O O E
PGY 1 under 6 under 6 under 6 under 6
2 2 2
O O l E O
3 3 3
Agree
Strongly agree
4 4
5 5
Agree
Strongly agree
4 4 4
5 5 5
NA NA
NA NA NA
Number of years PGY2 O PGY 3 O 26 or over 6-15 O 16-25 E 26 or over 6-15 O 16-25 O 26 or over 6-15 O 16-25 E 26 or over E 16-25 6-15
Gender:
Male Female
(1) (2)
Age Range:
Under 25 25-35
(1) (2) (3) (4) (5) (6)
36-45 46-55 56-65 over 65 (Please print)
368
Bull Med Libr Assoc 79(4) October 1991
Patient education database APPENDIX B* Directions:
Please tell us what you think of the material attached to this form by answering all the questions below. For each question circle the number that best describes how you feel. If the question does not seem to apply to the material you are reviewing, please circle NA. If you wish to write comments, we would value your additional ideas.
Title: Topic:
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.
Strongly disagree Disagree At first glance it attracted my attention. 1 2 It held my attention. 1 2 It is useful. 1 2 I like the illustrations. 1 2 I believe what it has to say. 1 2 I would recommend it to a friend or relative to read. 1 2 It is easy to understand. 1 2 What it says is important. 1 2 It reminds me of some things I need to think about. 1 2 It gives me some new things to think about. 1 2 It changes some of my thinking. 1 2 It could change how I do things. 1 2 Overall I recommend that doctors use this material with patients. 1 2 Overall I am the right person to get this material from my doctor. 1 2 Overall this material accomplishes its main purpose. 1 2
Purpose:
In just a few words, please write what you think is th ie main
purpose
Strongly
3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
Agree
agree
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
5 5 5 5 5 5 5 5 5 5 5 5 5 5 5
NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
of this material.
Comments:
We would appreciate learning
a
little
more
about you.
Male
Female Occunation
Years of school completed Age range: ( ) under 12 ( ) 12-15 ) 16-24 ( ) 25-35 ) 36-45 ( ) 46-55 ( ) 56-65 ( ) Over 65
Thank you! *
For use by patients.
Bull Med Libr Assoc 79(4) October 1991
369
