Alcohol Abuse Management in Primary Care: An e-Learning Course
Celina Andrade Pereira, MS,1 Chao Lung Wen, PhD,2 and Hermano Tavares, PhD1 1
Department of Psychiatry, University of Sa˜o Paulo Medical School, Sa˜o Paulo, Brazil. 2 Division of Telemedicine, Department of Pathology, University of Sa˜o Paulo Medical School, Sa˜o Paulo, Brazil.
Abstract Background: The mental health knowledge gap challenges public health. The Alcohol Abuse Management in Primary Care (AAMPC) is an e-learning course designed to cover alcohol-related problems from the primary care perspective. The goal of this study was to verify if the AAMPC was able to enhance healthcare professionals’ alcohol-related problems knowledge. Materials and Methods: One hundred subscriptions for the AAMPC were offered through the federal telehealth program. The course was instructor-led and had nine weekly classes, delivered synchronously or asynchronously, at the students’ convenience, using a varied array of learning tools. At the beginning, students took a test that provided a positive score, related to critical knowledge for clinical management, and a negative score, related to misconceptions about alcohol-related problems. The test was repeated 2 months after course completion. Results: Thirty-three students completed the course. The positive score improved significantly (p < 0.001), but not the negative score. Students with previous experience with e-courses presented greater improvement on the positive score (p < 0.036). Eighty-percent of the students thought the course excelled in meeting its objectives. Web conferences and video and audio recordings were the most appreciated learning tools. Course satisfaction was negatively related to frequency of Internet access (Spearman’s rho = - 0.455, p = 0.022). Conclusions: E-learning was highly appreciated as a learning tool, especially by students with the least frequency of Internet use. Nonetheless, it worked better for those previously familiar with e-courses. The AAMPC ecourse provided effective knowledge transmission and retention. Complementary strategies to reduce misconceptions about alcohol-related problems must be developed for the training of primary care staff. Key words: e-health, mobile health, telepsychiatry, distance learning, education
Introduction
T
he use of e-learning for continuing education of health professionals is on the rise.1 E-learning presents several advantages: it saves money related to transportation costs, published material, and physical space. If combined with recorded material, the learning experience can be reproduced sev-
200 TELEMEDICINE and e-HEALTH M A R C H 2 0 1 5
eral times, while securing procedural uniformity,2,3 resulting in an appealing time-flexible cost-effective strategy.4 Such accessibility makes it particularly interesting for the continuing education of either hard-to-reach or large health professional groups, such as primary care health teams in charge of a whole region.5,6 The gap between the high incidence of mental disorders and access to treatment is a well-established fact, reinforcing the World Health Organization declaration that there can be no health without mental health.7 Regarding the three most prevalent mental health conditions, the treatment gap for mood and anxiety disorders ranges between 50% and 60% (i.e., more than half of individuals bearing such conditions were not in treatment). The gap for alcohol abuse and dependence, the highest, was estimated at nearly 80%. This gap is probably underestimated because of lack of data from low- and middle-income countries.8 Such a gap is in contrast with the burden that alcohol represents to society. It is estimated that alcohol-related problems are responsible for 3% of the disease burden around the world and that it is the first cause of days lost to disability or early death for individuals 10–24 years of age, accounting for 7% of the disease burden in this age bracket.9 Alcohol abuse and dependence is widespread and a burden upon society, and the primary level should be the preferred instance to deal with it within the healthcare system.10 Therefore, the World Health Organization has proposed that strengthening the mental health component by means of training health personnel and investing in telemedicine are two of a 25-item list of top challenges for global mental health.11 Several countries, such as Brazil, Canada, Australia, and Germany, among others, are currently exploring initiatives in continuing education for health professionals through e-learning that includes mental health.1,5,12–15 A recent meta-analysis showed large effect sizes for Internet-based learning for health professionals. Knowledge and test performance improvement were associated with interactivity, practice exercises, repetition, and feedback.16 Specific initiatives for training in mental health are sparse, although encouraging. Efficacy of computer-based learning was reported for training of nursing homes’ staff on depression and dementia17 and training of community mental health professionals on Dialectical Behavior Therapy.18 Conversely, Beidas et al.19 reported limited gains in therapist adherence, skill, and knowledge on the treatment of anxiety among youth. This apparent conflict points to the need of investigating variations on suitability of e-learning according to professional profile and specific themes. We found only two reports on computer-based learning for addiction: one that described the demographic profile of physicians using an Internet course on Alcoholics Anonymous20 and another describing a computer-based
DOI: 10.1089/tmj.2014.0042
ALCOHOL ABUSE MANAGEMENT IN PRIMARY CARE
program to train addiction counselors to help patients cope with cravings.21 The authors of the latter found significant but modest improvement in the counselors’ test performance.21 Generally, the reports sound unanimous regarding acceptance of computer-based distance learning for mental health, but they are not specific to the factors that modulate this appreciation (e.g., previous experience with e-learning), or what are the preferred tools and methods. Brazil’s public health system has been undergoing major restructuring with the progressive implementation of a new program at the primary care level based on family doctors, nurses, and community health agents—the so-called Family Health Program.22 Alcohol abuse and addiction is the third most common mental disorder among users of primary care centers, being reported by over 6% of this population,23 and it is also a frequent cause of requests for mental health specialist’s supervision from the primary care staff. Hence, we have devised and piloted an Alcohol Abuse Management in Primary Care (AAMPC) e-course, designed for (but not exclusive to) primary care staff. The content of the AAMPC covers key aspects of alcohol-related problems from the perspective of family medicine such as basic knowledge about alcohol abuse, management of patients and family members, and available community resources. Its structure is short and simple, so it can be administered at a distance, in keeping with the intrinsic qualities of e-learning: flexibility and accessibility. The primary goal of this study was to determine if the AAMPC was able to enhance healthcare professionals’ general knowledge about alcohol abuse. As secondary goals, we wanted to determine, among the topics of AAMPC, which ones presented greater knowledge improvement, to investigate if students’ characteristics (demographics, professional trade, previous education, and experience with e-learning) would modulate knowledge acquisition and course appreciation and, finally, which of the learning tools used in the course were most valued by the students.
Materials and Methods RECRUITMENT AND PROFILING OF PARTICIPANTS/STUDENTS The AAMPC e-course was announced in the Internet networks of the Federal Tele-health Program. The announcement underscored its content directed for primary care staff; however, participation of health professionals from other levels within the public healthcare system was welcome, if they had been previously registered as primary care providers. Participation was free of charge, and the course was certified by the Commission for Culture and Extension of the Medical School of the University of Sa˜o Paulo, Sa˜o Paulo, Brazil. Following directions from the institutional ethics board, anonymity was secured, and students provided electronic consent for the use of their data. The enrollment period lasted from August 2010 until October 2010. The course was delivered from October to December 2010. Of the 100 initial subscriptions, 67 students initiated the course, and 33 effectively concluded it. Figure 1 charts the pathways leading to the final sample. To control for course attrition, completers and noncompleters were compared regarding their demographic profiles, and no differences
Fig. 1. Prevention of alcohol abuse distance learning course. CCEX, Commission for Culture and Extension. were found. The mean age of course completers was 34.0 years old (standard deviation = 10.3 years); 81.8% were female; 57.6% were single, divorced, or widowed; 57.6% had a university degree; and 64.7% were health professionals currently involved with administration and/or clinical care at primary centers, whereas the remaining 35.3% were health professionals not currently involved with primary care, but with a record of having previously worked in primary care centers. Seventy-three percent of the students had daily access to Internet, 87.8% had access to high-quality Internet connection, and 66% declared they had had previous experience with e-courses.
COURSE STRUCTURE The course had nine classes lasting 1 h each; all were delivered by the Internet, except for classes #1, #5, and #9, which lasted 1.5 h and were transmitted from the Web conference room. Because of space limitation, only four spots were available for students to follow the sessions face to face from the conference room. This was previously scheduled, and each student was allowed to follow face to face just one conference—only eight students applied. Different sets of tools were used throughout the course, including web conferences, video exhibitions, text reading followed by multiple-choice questions (MCQs), e-chats and audio chats (in which students sent written questions and received oral answers from the speaker), and Internet forums for requests and doubts. The AAMPC was an instructor-led
ª M A R Y A N N L I E B E R T , I N C . VOL. 21
NO. 3 MARCH 2015
TELEMEDICINE and e-HEALTH 201
PEREIRA ET AL.
(synchronous learning) e-course; however, if the student missed one activity, he or she could access an audio or video recording of it (asynchronous learning). Additionally, the Web conferences and parts of the audio chats with the most frequently asked questions were edited and made available at the course site for further consultation, named, respectively, video and audio tips. Table 1 presents a description of the classes’ content and materials applied.
ALCOHOL KNOWLEDGE ASSESSMENT Before beginning the course, the students answered an Alcohol Knowledge Questionnaire (AKQ) covering all the content of the course, designed by the authors, to gauge the students’ previous
knowledge about clinical management of alcohol abuse. The test contained 10 questions, covering the course content, with a variable number of statements that the reader had to rate as either true or false, except items #3 (approaching alcohol problems during home visiting), #6 (first aids for alcohol intoxication), and #9 (approaching alcohol problems with the patient), which were MCQs. Two months after the AAMPC e-course completion, the students repeated the AKQ to assess retention of knowledge. This time frame was adopted in order to provide enough distance from the end of the course and yet not too much so we would not risk losing contact with the students. The AKQ is available upon request from the corresponding author.
Table 1. Course Objectives and Content LESSON Class 1
DURATION 1.5 h
TITLE The effects of alcohol upon the individual
OBJECTIVE
METHODOLOGY
Introduce the Web-based learning tools and dynamics
Face-to-face class or Web meetinga
Appreciate alcohol-related harm
Video exhibition
Class 2
1h
The alcohol-related burden upon society
Provide information about alcohol consumption in Brazil and worldwide, highlighting key risk and protection factors for alcohol abuse Introduce alcohol law
Text reading followed by MCQ completion and chat with the teacher
Class 3
1h
Risk and protective factors for alcohol abuse
Identify risk and protective factors
Text reading followed by MCQ completion
How to minimize risk factors and to strengthen protective factors
Web meeting
Truths and myths about alcohol issues
Provide information about alcohol metabolism, alcohol abuse consequences. and alcohol association with other drugs
Text reading followed by MCQ completion and chat with the teacher
Alcohol and family: how to proceed Introduction of Community Reinforcement and Family Training (CRAFT) and the alcohol screening questionnaire (CAGE)
How to identify people with alcohol abuse symptoms
Text reading followed by MCQ completion
Facilitate regular alcohol screening and management of family-related issues
Face-to-face class or Web meetinga
How investigate alcohol problems
Provide guidelines on how to deal with alcohol abusers
Text reading followed by MCQ completion, video exhibition, and audio chat with teacher
Class 4
1h
Class 5
1.5 h
Class 6
1h
Provide first aid techniques in case of alcohol intoxication Class 7
1h
Alcohol and motivation to change
The Stages of Change model
Web meeting
Class 8
1h
Alcohol and motivation to change
The Stages of Change model
Text reading followed by MCQ completion and audio chat with the teacher
Class 9
1.5 h
Seeking solutions to alcohol-related problems within community resources
Debate and reflect upon real cases followed by the students and introduce 12-step–based programs and the most common resources available in the community
Face-to-face class or Web meetinga
a Those students who were not able to attend the face-to-face class, the Web conference, or chat could watch the recorded video made available 4 days after the presentation.
MCQ, multiple-choice questions (correct answers were provided right after completion).
202 TELEMEDICINE and e-HEALTH M A R C H 2 0 1 5
ALCOHOL ABUSE MANAGEMENT IN PRIMARY CARE
COURSE APPRECIATION ASSESSMENT At course completion, the students also filled out another questionnaire, designed by the authors, to assess the course appreciation, divided into two parts: satisfaction and relevance of the e-tools used for learning. Questions addressing satisfaction asked about how well the course met its objectives, its applicability, study load, course length, site navigation, and quality of texts, Internet forums, video and audio materials. The rates were organized in an increasing fashion, using a scale of poor (1), regular (2), good (3) and excellent (4). A list of the e-tools used during the course was presented, and course completers rated their perceived relevance for learning from 0 (not important at all) to 10 (most important). The course appreciation questionnaire is available upon request from the corresponding author.
STATISTICAL ANALYSIS Two measures were defined for each true-or-false item from the AKQ to assess general knowledge about alcohol: a positive score and a negative score. The positive score was based on the proportion of true assertions endorsed by the student and represented knowledge considered critical for optimal clinical management of alcohol-related problems (e.g., ‘‘Inquiring about general health and stress related symptoms is a good ‘ice-breaker.’ A too confronting style should be avoided.’’). The negative score was based on the proportion of false assertions endorsed by the student and represented common misconceptions and biases in approaching patients with alcohol problems (i.e., ‘‘The best thing to do is to wait for a health problem to show up, so you can suggest a visit to the doctor.’’). A final score was calculated subtracting the negative score from the positive score for each item, except for items #3, #6, and #9, which were scored either 1 or 0, because they had a MCQ format and were analyzed separately. The positive and negative item scores were summed up to tally, respectively, a total positive score and a total negative score; then a final total score was calculated by subtracting the total negative score from the total positive score and adjusting the raw outcome to vary between 0 and 10 (standardized values). Total positive, total negative, and final total scores were calculated for pre- and post-course assessments. Association measures between performance on the AKQ pre- and post-course and sample demographics were performed using correlation analysis (either Pearson’s or Spearman’s when appropriate) and mean comparison (t test) for, respectively, continuous and categorical variables. The distribution of pre- and post-course scores was checked with the Kolmogorov–Smirnov test; then an analysis of variance for repeated measures was performed. Any variable reaching significance at the association analysis was introduced as either a cofactor (if categorical) or a covariable (if continuous). Performance variation for each item from the AKQ between pre- and post-course was analyzed using Wilcoxon’s test because the majority of the item scores had distributions other than normal. Two performance rankings for the AKQ items at pre- and post-course were built. Then, a third ranking for knowledge improvement was built based on the Z statistics from the Wilcoxon’s test (a lower Z value indicates a higher knowledge improvement from pre- to post-course). McNemar’s test was used for questions #3, #6, and #9 because they were in MCQ format.
The scoring of the satisfaction items were summed up and then divided by the number of valid items for each individual in order to build a mean global score for course satisfaction. The same procedure was adopted for the relevance items in order to build a mean global score for relevance attributed to the learning tools used during the course. Satisfaction and Relevance mean scores were compared with demographics and the final total score of the AKQ, using mean comparison (analysis of variance) and correlation analysis (either Pearson’s or Spearman’s when appropriate) for, respectively, categorical and continuous variables. Exploratory analyses were performed for each item of the satisfaction and relevance assessments searching for associations with demographics and performance on the AKQ. Relevance scores attributed for each e-tool were compared between themselves using Wilcoxon’s test to verify the most important ones for learning according to the students. A significance level ( p) equal to 0.05 was adopted for all statistical testing.
Results The investigation of potential relationships between sample characteristics and course performance did not reveal significant associations, except for previous experiences with e-courses. No difference was found for the total final score between students with and without previous experiences with an e-course (t31 = 0.689, p = 0.496); however, students without previous experience performed better at the pretest in trend levels (t31 = - 1.973, p = 0.057). A 2 · 2 analysis of variance for repeated measures, having time as withinsubjects factor and previous experience with e-course as betweensubjects factor, revealed a significant improvement in the AKQ score from pre- to post-course (F1,31 = 15.243, p < 0.001) and a significant interaction between time and group (i.e., students with previous experience with e-courses had a greater variation on the AKQ score from pre- to post-course) (F1,31 = 4.809, p = 0.036) (Fig. 2). Questions #3, #6, and #9, analyzed separately because of their MCQ format, did not show significant variation at McNemar’s test; however, they already had a high frequency of correct answers at the
Fig. 2. Performance on the Alcohol Knowledge Questionnaire and previous e-course experience.
ª M A R Y A N N L I E B E R T , I N C . VOL. 21
NO. 3 MARCH 2015
TELEMEDICINE and e-HEALTH 203
PEREIRA ET AL.
Table 2. Pre- and Post-test Scores Comparison, by Item (n = 33) MEAN (SD) SCORE THEME
PRE-TEST
POST-TEST
Z
P
Question #1: Patient approach (range, 0–1) Positive score
0.51 (0.30)
0.80 (0.21)
- 3.919
< 0.001
Negative score
0.06 (0.13)
0.15 (0.44)
- 0.897
0.429
0.45 (0.36)
0.76 (0.24)
- 3.832
< 0.001
Final score
a
Question #2: Family guidance (range, 0–1) Positive score
0.55 (0.26)
0.83 (0.20)
- 4.422
< 0.001
Negative score
0.05 (0.10)
0.04 (0.09)
- 0.707
0.480
Final scorea
0.50 (0.28)
0.80 (0.22)
- 4.398
< 0.001
Question #4: 12-steps program (range, 0–1) Positive score
0.51 (0.32)
0.74 (0.29)
- 3.194
0.001
Negative score
0.14 (0.14)
0.14 (0.13)
- 0.302
0.763
0.36 (0.36)
0.61 (0.35)
- 3.264
0.001
0.58 (0.24)
- 0.077
0.938
Final score
a
Question #5: Alcohol metabolism (range, 0–1) Positive score
0.58 (0.24)
Negative score
0.13 (0.11)
0.12 (0.12)
- 1.117
0.264
Final scorea
0.46 (0.22)
0.46 (0.23)
- 0.204
0.838
Question #7: Risk and protection (range, 0–1) Positive score
0.29 (0.28)
0.47 (0.35)
- 2.449
0.014
Negative score
0.55 (0.28)
0.53 (0.21)
- 0.333
0.739
- 0.26 (0.33)
- 0.06 (0.34)
- 2.229
0.026
Final score
a
Question #8: Addiction consequences (range, 0–1) Positive score
0.61 (0.26)
0.73 (0.27)
- 2.016
0.044
Negative score
0.02 (0.73)
0.05 (0.10)
- 1.342
0.180
0.59 (0.26)
0.68 (0.28)
- 1.799
0.072
Final score
a
Question #10: Alcohol problems signs (range, 0–1) Positive score
0.76 (0.21)
0.83 (0.16)
- 1.454
0.146
Negative score
0.03 (0.10)
0.05 (0.18)
- 0.378
0.705
0.73 (0.22)
0.78 (0.22)
- 1.051
0.293
Final score
a
Total score (range, 0–10)
b
Positive score Negative score a
Final total score
5.45
7.11
- 4.038
< 0.001
1.40
1.54
- 0.617
0.537
4.04
5.58
- 3.672
< 0.001
Comparisons were performed with Wilcoxon’s test.
pre-course assessment: 60.6% ( p = 0.109), 78.9% ( p = 0.754), and 90.9% ( p = 1.00), respectively, suggesting that a probable ceiling effect precluded knowledge gain appreciation in this case. Conversely, the remaining items from the test, which were scored on a continuous fashion, displayed a significant variation between preand post-course final scores, except for questions #5 and #10 and for question #8, which approached significance. As can be seen from Table 2, the final scores that varied from pre- to post-course did so because of the variation on the corresponding positive score, as no negative score was significantly modified. The comparison of the pre- and post-course scores suggests that general knowledge about alcohol addiction (alcohol problem signs and addiction consequences, respectively, questions #10 and #8), which was already good before the course, remained good; thus they did not vary over time. Themes related to practical management of patients and their families (questions #1 and #2) presented the highest improvement. Knowledge related to risk and protection factors and 12step program significantly varied, but their intermediate position in the knowledge improvement ranking (third column in Table 3) suggests that there could be further knowledge advancement. Finally, knowledge related to alcohol metabolism presented the smallest improvement between pre- and post-course. Table 3 shows the ranking of knowledge for the continuously scored questions for pre- and postcourse and for knowledge improvement over the course. Regarding the course appreciation, participants had a very positive opinion about the course: 84% thought that the objectives were excellently achieved, and 80% thought that the course had an optimal applicability. The less appreciated aspects were course study load (24% rated ‘‘excellent’’ and 64% rated ‘‘good’’) and course length (20% rated ‘‘excellent’’ and 68% rated ‘‘good’’). About the perceived importance of didactic materials and interventions, participants attributed the highest relevance to Web conferences and recorded videos and audios and the least relevance to face-to-face classes. Table 4 summarizes the main findings. No significant associations between global satisfaction and relevance mean scores and demographics and performance at the AKQ were found, except for a significant relationship between Satisfaction mean score and Frequency of Internet Access—students with the least frequency of Internet access were the most satisfied with the e-course (Spearman’s rho = - 0.455, p = 0.022). At the exploratory analysis, relevance attributed to face-to-face classes was negatively correlated with the final score at the AKQ (Spearman’s rho = - 0.466, p = 0.019). In the ranking of relevance for learning, new e-tools (Web conference, video and audio tips, and text reading followed by MCQ completion) performed better than traditional ones (chats and forums), with face-to-face encounters with teachers being the least relevant, according to the students.
a
Final scores were calculated by subtracting the negative score from the corresponding positive score.
b
Standardized values.
SD, standard deviation.
204 TELEMEDICINE and e-HEALTH M A R C H 2 0 1 5
Discussion The data show that e-learning is an effective way of transmitting information about alcohol-related problems to primary care staff. Indeed, a significant increase in alcohol knowledge was found even 2 months after the course completion. Thus, e-learning secured both
ALCOHOL ABUSE MANAGEMENT IN PRIMARY CARE
dice.24 Therefore, it is imperative that educational initiatives with a focus PRE- AND POST-COURSE on changing misconceptions that PRE-COURSE RANKING POST-COURSE RANKING IMPROVEMENT RANKINGa hinder optimal clinical management Question #10: Alcohol problem signs Question #2: Family guidance Question #2: Family guidance be developed and tested in future studies. The identification of this Question #8: Addiction consequences Question #10: Alcohol problem signs Question #1: Patient approach blind spot was only possible because Question #2: Family guidance Question #1: Patient approach Question #4: 12-steps program of the adoption of true-or-false type Question #1: Patient approach Question #8: Addiction consequences Question #7: Risk and protection of questions for the assessment of the learning process. Indeed, this type of Question #5: Alcohol metabolism Question #4: 12-steps program Question #8: Addiction consequences question proved more useful in proQuestion #4: 12-steps program Question #5: Alcohol metabolism Question #10: Alcohol problem signs viding a balanced evaluation about Question #7: Risk and protection Question #7: Risk and protection Question #5: Alcohol metabolism knowledge and biases and more a sensitive to changes in the cognitive Based on the Z coefficient from the Wilcoxon test displayed on Table 2. status of students than MCQs. The comparison between pre- and post-course scores suggests that genknowledge gain and retention. No variable, except previous experience eral knowledge about alcohol addiction did not improve over time. with e-learning, was related to course performance, indicating that This lack of improvement was probably due to a ceiling effect because e-learning is an accessible mean for teaching mental health topics. the pre-course scores in this case were already high. Despite presenting Despite profiting less from the course, students without previous exthe third highest score improvement, the 12-step programs score was perience enjoyed e-learning the most, probably because of a novelty not well positioned in the post-course score ranking, meaning that effect. Offering a practice pre-course and extending the tutorial time there is still room and need for improvement in knowledge about 12may help inexperienced students in future versions of the course. step programs and other community resources to support addiction However, only the positive subscores improved. In other words, patients. At the same time, the topics related to practical management the course was successful in providing new knowledge, but was not of addiction patients and their families were optimally classified both effective in dispelling misconceptions regarding alcohol-related at the post-course ranking and at the improvement ranking. Thus, the problems. The field of addiction is plagued with stigma and prejuAAMPC may help accomplish a key directive of the Family Health Program, which is to enable primary care staff and services to deal with highly prevalent behavioral challenges, such as addictions. Nonetheless, it is up to future studies to establish if e-learning is capable of Table 4. Relevance Attributed to Tools Used actually shaping addiction clinical management toward best evidencein Distance Learning based and cost-effective practices. MEAN SD MINIMUM MAXIMUM Z P It is interesting that the students who attributed more relevance to a Web conference 9.76 0.663 7.0 10.0 — — face-to-face classes profited less from the course. This reinforces the Video and 9.76 0.597 8.0 10.0 0b 1.00 perception that students who are less familiar with e-learning may a audio tips benefit from blended models and then smoothly progressing to Text reading 9.72 0.613 8.0 10.0 - 0.431b 0.666 strictly e-learning as they become less dependent on face-to-face a with MCQ interaction. It is interesting that taking the knowledge test was among AKQ completiona 9.44 1.003 6.0 10.0 - 1.510b 0.131 the four most appreciated activities by the students, possibly because it enables students to gauge by themselves how well the course Audio chat 9.36 1.186 5.0 10.0 - 1.930b 0.054 worked for them. b 0.041 Chat 9.20 1.384 5.0 10.0 - 2.047 The current study has important limitations, mainly its small Forum 8.72 1.514 4.0 10.0 - 3.093b 0.002 sample size and the attrition rate, which, although high, is in keeping with previous reports of e-learning dropout.25 Besides, the followFace-to-face 8.04 2.950 0 10.0 - 3.017b 0.003 class up period was short, and no further assessment was conducted to ascertain that knowledge retention is secured for a longer period. Scores ranged from 0 to 10 (n = 25). a Finally, the lack of a control group does not yield the comparison of Most relevant learning tools. b e-learning performance against traditional in-person teaching, or Wilcoxon’s test for related measures: compared with the highest score (Web conference). simple text reading followed by test completion. Thus, the power and AKQ, Alcohol Knowledge Questionnaire; MCQ, multiple-choice questions; SD, cost-effectiveness of e-learning to teach and engage students in standard deviation. mental health in comparison with other methods remain to be clarified. However, the current data allow the conclusion that e-learning
Table 3. Proficiency Ranking of Themes
ª M A R Y A N N L I E B E R T , I N C . VOL. 21
NO. 3 MARCH 2015
TELEMEDICINE and e-HEALTH 205
PEREIRA ET AL.
is effective in increasing the knowledge of primary care professionals about the clinical management of alcohol-related problems. Future studies should focus on strategies to foster long-term retention of knowledge, removal of misconceptions, actual change for better clinical practices, and cost-effectiveness of e-learning.
Disclosure Statement No competing financial interests exist.
REFERENCES 1. Curran VR. Tele-education. J Telemed Telecare 2006;12:57–63. 2. Khanna M, Aschenbrand SG, Kendall PC. New frontiers: Computer technology in the treatment of anxious youth. Behav Ther 2007;30:22–25. 3. Abeles P, Verduyn C, Robinson A. Computerized CBT for adolescent depression (‘‘Stress-busters’’) and its initial evaluation through an extended case series. Behav Cogn Psychother 2009;37:151–165. 4. Calear AL, Christensen H. Review of internet-based prevention and treatment programs for anxiety and depression in children and adolescents. Med J Aust 2010;192(11 Suppl):S12–S14. 5. Eikelboom RH, Weber S, Atlas MD, Dinh Q, Mbao MN, Gallop MA. A tele-otology course for primary care providers. J Telemed Telecare 2003;9(Suppl 2):S19–S22. 6. Griffiths F, Lindenmeyer A, Powell J, Low P, Thorogood M. Why are health care interventions delivered over the Internet? A systematic review of the published literature. J Med Internet Res 2006;8:e12. 7. Prince M, Patel V, Saxena S, et al. No health without mental health. Lancet 2007;370:859–877. 8. Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bull World Health Organ 2004;82:858–866. 9. Gore FM, Bloem PJ, Patton GC, et al. Global burden of disease in young people aged 10–24 years: A systematic analysis. Lancet 2011;377:2093–2102. 10. Gual A, Sabadini MB. Implementing alcohol disorders treatment throughout the community. Curr Opin Psychiatry 2011;24:203–207. 11. Collins PY, Patel V, Joestl S, et al. Grand challenges in global mental health. Nature 2011;475:27–30. 12. dos Santos AeF, Haddad SC, Alves HJ, Torres RM, de Souza C, de Melo MoC. Evaluating the experience of training through videoconferences in primary care. Telemed J E Health 2011;17:722–726. 13. Joshi A, Novaes MA, Iyengar S, et al. Evaluation of a tele-education programme in Brazil. J Telemed Telecare 2011;17:341–345. 14. Kind T. The Internet as an adjunct for pediatric primary care. Curr Opin Pediatr 2009;21:805–810.
206 TELEMEDICINE and e-HEALTH M A R C H 2 0 1 5
15. Paixa˜o MP, Miot HA, Wen CL. Tele-education on leprosy: Evaluation of an educational strategy. Telemed J E Health 2009;15:552–559. 16. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internetbased learning in the health professions: A meta-analysis. JAMA 2008;300:1181–1196. 17. McPhaul KM, Rosen J, Bobb S, et al. An exploratory study of mandated safety measures for home visiting case managers. Can J Nurs Res 2007;39:173–189. 18. Dimeff LA, Woodcock EA, Harned MS, Beadnell B. Can dialectical behavior therapy be learned in highly structured learning environments? Results from a randomized controlled dissemination trial. Behav Ther 2011;42: 263–275. 19. Beidas RS, Birkett M, Newcomb ME, Mustanski B. Do psychiatric disorders moderate the relationship between psychological distress and sexual risk-taking behaviors in young men who have sex with men? A longitudinal perspective. AIDS Patient Care STDS 2012;26:366–374. 20. Sellers B, Galanter M, Dermatis H, Nachbar M. Enhancing physicians’ use of Alcoholics Anonymous: Internet-based training. J Addict Dis 2005;24:77–86. 21. Cucciare MA, Weingardt KR, Greene CJ, Hoffman J. Current trends in using Internet and mobile technology to support the treatment of substance use disorders. Curr Drug Abuse Rev 2012;5:172–177. 22. Paim J, Travassos C, Almeida C, Bahia L, Macinko J. The Brazilian health system: History, advances, and challenges. Lancet 2011;377:1778–1797. 23. Gonc¸alves DM, Kapczinski F. [Prevalence of mental disorders at a referral center for the Family Health Program in Santa Cruz do Sul, Rio Grande do Sul State, Brazil]. Cad Saude Publica 2008;24:2043–2053. 24. Williamson L. Destigmatizing alcohol dependence: The requirement for an ethical (not only medical) remedy. Am J Public Health 2012;102:e5–e8. 25. Levy Y. Comparing dropouts and persistence in e-learning courses. Comput Educ 2007;48:185–204.
Address correspondence to: Celina Andrade Pereira, MS Rua Jose´ Coimbra, 120 – apto 6, Vila Andrade Sa˜o Paulo, SP, CEP 05726-110 Brasil E-mail: [email protected] Received: February 23, 2014 Revised: June 12, 2014 Accepted: June 16, 2014
Copyright of Telemedicine & e-Health is the property of Mary Ann Liebert, Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Celina Andrade Pereira, MS,1 Chao Lung Wen, PhD,2 and Hermano Tavares, PhD1 1
Department of Psychiatry, University of Sa˜o Paulo Medical School, Sa˜o Paulo, Brazil. 2 Division of Telemedicine, Department of Pathology, University of Sa˜o Paulo Medical School, Sa˜o Paulo, Brazil.
Abstract Background: The mental health knowledge gap challenges public health. The Alcohol Abuse Management in Primary Care (AAMPC) is an e-learning course designed to cover alcohol-related problems from the primary care perspective. The goal of this study was to verify if the AAMPC was able to enhance healthcare professionals’ alcohol-related problems knowledge. Materials and Methods: One hundred subscriptions for the AAMPC were offered through the federal telehealth program. The course was instructor-led and had nine weekly classes, delivered synchronously or asynchronously, at the students’ convenience, using a varied array of learning tools. At the beginning, students took a test that provided a positive score, related to critical knowledge for clinical management, and a negative score, related to misconceptions about alcohol-related problems. The test was repeated 2 months after course completion. Results: Thirty-three students completed the course. The positive score improved significantly (p < 0.001), but not the negative score. Students with previous experience with e-courses presented greater improvement on the positive score (p < 0.036). Eighty-percent of the students thought the course excelled in meeting its objectives. Web conferences and video and audio recordings were the most appreciated learning tools. Course satisfaction was negatively related to frequency of Internet access (Spearman’s rho = - 0.455, p = 0.022). Conclusions: E-learning was highly appreciated as a learning tool, especially by students with the least frequency of Internet use. Nonetheless, it worked better for those previously familiar with e-courses. The AAMPC ecourse provided effective knowledge transmission and retention. Complementary strategies to reduce misconceptions about alcohol-related problems must be developed for the training of primary care staff. Key words: e-health, mobile health, telepsychiatry, distance learning, education
Introduction
T
he use of e-learning for continuing education of health professionals is on the rise.1 E-learning presents several advantages: it saves money related to transportation costs, published material, and physical space. If combined with recorded material, the learning experience can be reproduced sev-
200 TELEMEDICINE and e-HEALTH M A R C H 2 0 1 5
eral times, while securing procedural uniformity,2,3 resulting in an appealing time-flexible cost-effective strategy.4 Such accessibility makes it particularly interesting for the continuing education of either hard-to-reach or large health professional groups, such as primary care health teams in charge of a whole region.5,6 The gap between the high incidence of mental disorders and access to treatment is a well-established fact, reinforcing the World Health Organization declaration that there can be no health without mental health.7 Regarding the three most prevalent mental health conditions, the treatment gap for mood and anxiety disorders ranges between 50% and 60% (i.e., more than half of individuals bearing such conditions were not in treatment). The gap for alcohol abuse and dependence, the highest, was estimated at nearly 80%. This gap is probably underestimated because of lack of data from low- and middle-income countries.8 Such a gap is in contrast with the burden that alcohol represents to society. It is estimated that alcohol-related problems are responsible for 3% of the disease burden around the world and that it is the first cause of days lost to disability or early death for individuals 10–24 years of age, accounting for 7% of the disease burden in this age bracket.9 Alcohol abuse and dependence is widespread and a burden upon society, and the primary level should be the preferred instance to deal with it within the healthcare system.10 Therefore, the World Health Organization has proposed that strengthening the mental health component by means of training health personnel and investing in telemedicine are two of a 25-item list of top challenges for global mental health.11 Several countries, such as Brazil, Canada, Australia, and Germany, among others, are currently exploring initiatives in continuing education for health professionals through e-learning that includes mental health.1,5,12–15 A recent meta-analysis showed large effect sizes for Internet-based learning for health professionals. Knowledge and test performance improvement were associated with interactivity, practice exercises, repetition, and feedback.16 Specific initiatives for training in mental health are sparse, although encouraging. Efficacy of computer-based learning was reported for training of nursing homes’ staff on depression and dementia17 and training of community mental health professionals on Dialectical Behavior Therapy.18 Conversely, Beidas et al.19 reported limited gains in therapist adherence, skill, and knowledge on the treatment of anxiety among youth. This apparent conflict points to the need of investigating variations on suitability of e-learning according to professional profile and specific themes. We found only two reports on computer-based learning for addiction: one that described the demographic profile of physicians using an Internet course on Alcoholics Anonymous20 and another describing a computer-based
DOI: 10.1089/tmj.2014.0042
ALCOHOL ABUSE MANAGEMENT IN PRIMARY CARE
program to train addiction counselors to help patients cope with cravings.21 The authors of the latter found significant but modest improvement in the counselors’ test performance.21 Generally, the reports sound unanimous regarding acceptance of computer-based distance learning for mental health, but they are not specific to the factors that modulate this appreciation (e.g., previous experience with e-learning), or what are the preferred tools and methods. Brazil’s public health system has been undergoing major restructuring with the progressive implementation of a new program at the primary care level based on family doctors, nurses, and community health agents—the so-called Family Health Program.22 Alcohol abuse and addiction is the third most common mental disorder among users of primary care centers, being reported by over 6% of this population,23 and it is also a frequent cause of requests for mental health specialist’s supervision from the primary care staff. Hence, we have devised and piloted an Alcohol Abuse Management in Primary Care (AAMPC) e-course, designed for (but not exclusive to) primary care staff. The content of the AAMPC covers key aspects of alcohol-related problems from the perspective of family medicine such as basic knowledge about alcohol abuse, management of patients and family members, and available community resources. Its structure is short and simple, so it can be administered at a distance, in keeping with the intrinsic qualities of e-learning: flexibility and accessibility. The primary goal of this study was to determine if the AAMPC was able to enhance healthcare professionals’ general knowledge about alcohol abuse. As secondary goals, we wanted to determine, among the topics of AAMPC, which ones presented greater knowledge improvement, to investigate if students’ characteristics (demographics, professional trade, previous education, and experience with e-learning) would modulate knowledge acquisition and course appreciation and, finally, which of the learning tools used in the course were most valued by the students.
Materials and Methods RECRUITMENT AND PROFILING OF PARTICIPANTS/STUDENTS The AAMPC e-course was announced in the Internet networks of the Federal Tele-health Program. The announcement underscored its content directed for primary care staff; however, participation of health professionals from other levels within the public healthcare system was welcome, if they had been previously registered as primary care providers. Participation was free of charge, and the course was certified by the Commission for Culture and Extension of the Medical School of the University of Sa˜o Paulo, Sa˜o Paulo, Brazil. Following directions from the institutional ethics board, anonymity was secured, and students provided electronic consent for the use of their data. The enrollment period lasted from August 2010 until October 2010. The course was delivered from October to December 2010. Of the 100 initial subscriptions, 67 students initiated the course, and 33 effectively concluded it. Figure 1 charts the pathways leading to the final sample. To control for course attrition, completers and noncompleters were compared regarding their demographic profiles, and no differences
Fig. 1. Prevention of alcohol abuse distance learning course. CCEX, Commission for Culture and Extension. were found. The mean age of course completers was 34.0 years old (standard deviation = 10.3 years); 81.8% were female; 57.6% were single, divorced, or widowed; 57.6% had a university degree; and 64.7% were health professionals currently involved with administration and/or clinical care at primary centers, whereas the remaining 35.3% were health professionals not currently involved with primary care, but with a record of having previously worked in primary care centers. Seventy-three percent of the students had daily access to Internet, 87.8% had access to high-quality Internet connection, and 66% declared they had had previous experience with e-courses.
COURSE STRUCTURE The course had nine classes lasting 1 h each; all were delivered by the Internet, except for classes #1, #5, and #9, which lasted 1.5 h and were transmitted from the Web conference room. Because of space limitation, only four spots were available for students to follow the sessions face to face from the conference room. This was previously scheduled, and each student was allowed to follow face to face just one conference—only eight students applied. Different sets of tools were used throughout the course, including web conferences, video exhibitions, text reading followed by multiple-choice questions (MCQs), e-chats and audio chats (in which students sent written questions and received oral answers from the speaker), and Internet forums for requests and doubts. The AAMPC was an instructor-led
ª M A R Y A N N L I E B E R T , I N C . VOL. 21
NO. 3 MARCH 2015
TELEMEDICINE and e-HEALTH 201
PEREIRA ET AL.
(synchronous learning) e-course; however, if the student missed one activity, he or she could access an audio or video recording of it (asynchronous learning). Additionally, the Web conferences and parts of the audio chats with the most frequently asked questions were edited and made available at the course site for further consultation, named, respectively, video and audio tips. Table 1 presents a description of the classes’ content and materials applied.
ALCOHOL KNOWLEDGE ASSESSMENT Before beginning the course, the students answered an Alcohol Knowledge Questionnaire (AKQ) covering all the content of the course, designed by the authors, to gauge the students’ previous
knowledge about clinical management of alcohol abuse. The test contained 10 questions, covering the course content, with a variable number of statements that the reader had to rate as either true or false, except items #3 (approaching alcohol problems during home visiting), #6 (first aids for alcohol intoxication), and #9 (approaching alcohol problems with the patient), which were MCQs. Two months after the AAMPC e-course completion, the students repeated the AKQ to assess retention of knowledge. This time frame was adopted in order to provide enough distance from the end of the course and yet not too much so we would not risk losing contact with the students. The AKQ is available upon request from the corresponding author.
Table 1. Course Objectives and Content LESSON Class 1
DURATION 1.5 h
TITLE The effects of alcohol upon the individual
OBJECTIVE
METHODOLOGY
Introduce the Web-based learning tools and dynamics
Face-to-face class or Web meetinga
Appreciate alcohol-related harm
Video exhibition
Class 2
1h
The alcohol-related burden upon society
Provide information about alcohol consumption in Brazil and worldwide, highlighting key risk and protection factors for alcohol abuse Introduce alcohol law
Text reading followed by MCQ completion and chat with the teacher
Class 3
1h
Risk and protective factors for alcohol abuse
Identify risk and protective factors
Text reading followed by MCQ completion
How to minimize risk factors and to strengthen protective factors
Web meeting
Truths and myths about alcohol issues
Provide information about alcohol metabolism, alcohol abuse consequences. and alcohol association with other drugs
Text reading followed by MCQ completion and chat with the teacher
Alcohol and family: how to proceed Introduction of Community Reinforcement and Family Training (CRAFT) and the alcohol screening questionnaire (CAGE)
How to identify people with alcohol abuse symptoms
Text reading followed by MCQ completion
Facilitate regular alcohol screening and management of family-related issues
Face-to-face class or Web meetinga
How investigate alcohol problems
Provide guidelines on how to deal with alcohol abusers
Text reading followed by MCQ completion, video exhibition, and audio chat with teacher
Class 4
1h
Class 5
1.5 h
Class 6
1h
Provide first aid techniques in case of alcohol intoxication Class 7
1h
Alcohol and motivation to change
The Stages of Change model
Web meeting
Class 8
1h
Alcohol and motivation to change
The Stages of Change model
Text reading followed by MCQ completion and audio chat with the teacher
Class 9
1.5 h
Seeking solutions to alcohol-related problems within community resources
Debate and reflect upon real cases followed by the students and introduce 12-step–based programs and the most common resources available in the community
Face-to-face class or Web meetinga
a Those students who were not able to attend the face-to-face class, the Web conference, or chat could watch the recorded video made available 4 days after the presentation.
MCQ, multiple-choice questions (correct answers were provided right after completion).
202 TELEMEDICINE and e-HEALTH M A R C H 2 0 1 5
ALCOHOL ABUSE MANAGEMENT IN PRIMARY CARE
COURSE APPRECIATION ASSESSMENT At course completion, the students also filled out another questionnaire, designed by the authors, to assess the course appreciation, divided into two parts: satisfaction and relevance of the e-tools used for learning. Questions addressing satisfaction asked about how well the course met its objectives, its applicability, study load, course length, site navigation, and quality of texts, Internet forums, video and audio materials. The rates were organized in an increasing fashion, using a scale of poor (1), regular (2), good (3) and excellent (4). A list of the e-tools used during the course was presented, and course completers rated their perceived relevance for learning from 0 (not important at all) to 10 (most important). The course appreciation questionnaire is available upon request from the corresponding author.
STATISTICAL ANALYSIS Two measures were defined for each true-or-false item from the AKQ to assess general knowledge about alcohol: a positive score and a negative score. The positive score was based on the proportion of true assertions endorsed by the student and represented knowledge considered critical for optimal clinical management of alcohol-related problems (e.g., ‘‘Inquiring about general health and stress related symptoms is a good ‘ice-breaker.’ A too confronting style should be avoided.’’). The negative score was based on the proportion of false assertions endorsed by the student and represented common misconceptions and biases in approaching patients with alcohol problems (i.e., ‘‘The best thing to do is to wait for a health problem to show up, so you can suggest a visit to the doctor.’’). A final score was calculated subtracting the negative score from the positive score for each item, except for items #3, #6, and #9, which were scored either 1 or 0, because they had a MCQ format and were analyzed separately. The positive and negative item scores were summed up to tally, respectively, a total positive score and a total negative score; then a final total score was calculated by subtracting the total negative score from the total positive score and adjusting the raw outcome to vary between 0 and 10 (standardized values). Total positive, total negative, and final total scores were calculated for pre- and post-course assessments. Association measures between performance on the AKQ pre- and post-course and sample demographics were performed using correlation analysis (either Pearson’s or Spearman’s when appropriate) and mean comparison (t test) for, respectively, continuous and categorical variables. The distribution of pre- and post-course scores was checked with the Kolmogorov–Smirnov test; then an analysis of variance for repeated measures was performed. Any variable reaching significance at the association analysis was introduced as either a cofactor (if categorical) or a covariable (if continuous). Performance variation for each item from the AKQ between pre- and post-course was analyzed using Wilcoxon’s test because the majority of the item scores had distributions other than normal. Two performance rankings for the AKQ items at pre- and post-course were built. Then, a third ranking for knowledge improvement was built based on the Z statistics from the Wilcoxon’s test (a lower Z value indicates a higher knowledge improvement from pre- to post-course). McNemar’s test was used for questions #3, #6, and #9 because they were in MCQ format.
The scoring of the satisfaction items were summed up and then divided by the number of valid items for each individual in order to build a mean global score for course satisfaction. The same procedure was adopted for the relevance items in order to build a mean global score for relevance attributed to the learning tools used during the course. Satisfaction and Relevance mean scores were compared with demographics and the final total score of the AKQ, using mean comparison (analysis of variance) and correlation analysis (either Pearson’s or Spearman’s when appropriate) for, respectively, categorical and continuous variables. Exploratory analyses were performed for each item of the satisfaction and relevance assessments searching for associations with demographics and performance on the AKQ. Relevance scores attributed for each e-tool were compared between themselves using Wilcoxon’s test to verify the most important ones for learning according to the students. A significance level ( p) equal to 0.05 was adopted for all statistical testing.
Results The investigation of potential relationships between sample characteristics and course performance did not reveal significant associations, except for previous experiences with e-courses. No difference was found for the total final score between students with and without previous experiences with an e-course (t31 = 0.689, p = 0.496); however, students without previous experience performed better at the pretest in trend levels (t31 = - 1.973, p = 0.057). A 2 · 2 analysis of variance for repeated measures, having time as withinsubjects factor and previous experience with e-course as betweensubjects factor, revealed a significant improvement in the AKQ score from pre- to post-course (F1,31 = 15.243, p < 0.001) and a significant interaction between time and group (i.e., students with previous experience with e-courses had a greater variation on the AKQ score from pre- to post-course) (F1,31 = 4.809, p = 0.036) (Fig. 2). Questions #3, #6, and #9, analyzed separately because of their MCQ format, did not show significant variation at McNemar’s test; however, they already had a high frequency of correct answers at the
Fig. 2. Performance on the Alcohol Knowledge Questionnaire and previous e-course experience.
ª M A R Y A N N L I E B E R T , I N C . VOL. 21
NO. 3 MARCH 2015
TELEMEDICINE and e-HEALTH 203
PEREIRA ET AL.
Table 2. Pre- and Post-test Scores Comparison, by Item (n = 33) MEAN (SD) SCORE THEME
PRE-TEST
POST-TEST
Z
P
Question #1: Patient approach (range, 0–1) Positive score
0.51 (0.30)
0.80 (0.21)
- 3.919
< 0.001
Negative score
0.06 (0.13)
0.15 (0.44)
- 0.897
0.429
0.45 (0.36)
0.76 (0.24)
- 3.832
< 0.001
Final score
a
Question #2: Family guidance (range, 0–1) Positive score
0.55 (0.26)
0.83 (0.20)
- 4.422
< 0.001
Negative score
0.05 (0.10)
0.04 (0.09)
- 0.707
0.480
Final scorea
0.50 (0.28)
0.80 (0.22)
- 4.398
< 0.001
Question #4: 12-steps program (range, 0–1) Positive score
0.51 (0.32)
0.74 (0.29)
- 3.194
0.001
Negative score
0.14 (0.14)
0.14 (0.13)
- 0.302
0.763
0.36 (0.36)
0.61 (0.35)
- 3.264
0.001
0.58 (0.24)
- 0.077
0.938
Final score
a
Question #5: Alcohol metabolism (range, 0–1) Positive score
0.58 (0.24)
Negative score
0.13 (0.11)
0.12 (0.12)
- 1.117
0.264
Final scorea
0.46 (0.22)
0.46 (0.23)
- 0.204
0.838
Question #7: Risk and protection (range, 0–1) Positive score
0.29 (0.28)
0.47 (0.35)
- 2.449
0.014
Negative score
0.55 (0.28)
0.53 (0.21)
- 0.333
0.739
- 0.26 (0.33)
- 0.06 (0.34)
- 2.229
0.026
Final score
a
Question #8: Addiction consequences (range, 0–1) Positive score
0.61 (0.26)
0.73 (0.27)
- 2.016
0.044
Negative score
0.02 (0.73)
0.05 (0.10)
- 1.342
0.180
0.59 (0.26)
0.68 (0.28)
- 1.799
0.072
Final score
a
Question #10: Alcohol problems signs (range, 0–1) Positive score
0.76 (0.21)
0.83 (0.16)
- 1.454
0.146
Negative score
0.03 (0.10)
0.05 (0.18)
- 0.378
0.705
0.73 (0.22)
0.78 (0.22)
- 1.051
0.293
Final score
a
Total score (range, 0–10)
b
Positive score Negative score a
Final total score
5.45
7.11
- 4.038
< 0.001
1.40
1.54
- 0.617
0.537
4.04
5.58
- 3.672
< 0.001
Comparisons were performed with Wilcoxon’s test.
pre-course assessment: 60.6% ( p = 0.109), 78.9% ( p = 0.754), and 90.9% ( p = 1.00), respectively, suggesting that a probable ceiling effect precluded knowledge gain appreciation in this case. Conversely, the remaining items from the test, which were scored on a continuous fashion, displayed a significant variation between preand post-course final scores, except for questions #5 and #10 and for question #8, which approached significance. As can be seen from Table 2, the final scores that varied from pre- to post-course did so because of the variation on the corresponding positive score, as no negative score was significantly modified. The comparison of the pre- and post-course scores suggests that general knowledge about alcohol addiction (alcohol problem signs and addiction consequences, respectively, questions #10 and #8), which was already good before the course, remained good; thus they did not vary over time. Themes related to practical management of patients and their families (questions #1 and #2) presented the highest improvement. Knowledge related to risk and protection factors and 12step program significantly varied, but their intermediate position in the knowledge improvement ranking (third column in Table 3) suggests that there could be further knowledge advancement. Finally, knowledge related to alcohol metabolism presented the smallest improvement between pre- and post-course. Table 3 shows the ranking of knowledge for the continuously scored questions for pre- and postcourse and for knowledge improvement over the course. Regarding the course appreciation, participants had a very positive opinion about the course: 84% thought that the objectives were excellently achieved, and 80% thought that the course had an optimal applicability. The less appreciated aspects were course study load (24% rated ‘‘excellent’’ and 64% rated ‘‘good’’) and course length (20% rated ‘‘excellent’’ and 68% rated ‘‘good’’). About the perceived importance of didactic materials and interventions, participants attributed the highest relevance to Web conferences and recorded videos and audios and the least relevance to face-to-face classes. Table 4 summarizes the main findings. No significant associations between global satisfaction and relevance mean scores and demographics and performance at the AKQ were found, except for a significant relationship between Satisfaction mean score and Frequency of Internet Access—students with the least frequency of Internet access were the most satisfied with the e-course (Spearman’s rho = - 0.455, p = 0.022). At the exploratory analysis, relevance attributed to face-to-face classes was negatively correlated with the final score at the AKQ (Spearman’s rho = - 0.466, p = 0.019). In the ranking of relevance for learning, new e-tools (Web conference, video and audio tips, and text reading followed by MCQ completion) performed better than traditional ones (chats and forums), with face-to-face encounters with teachers being the least relevant, according to the students.
a
Final scores were calculated by subtracting the negative score from the corresponding positive score.
b
Standardized values.
SD, standard deviation.
204 TELEMEDICINE and e-HEALTH M A R C H 2 0 1 5
Discussion The data show that e-learning is an effective way of transmitting information about alcohol-related problems to primary care staff. Indeed, a significant increase in alcohol knowledge was found even 2 months after the course completion. Thus, e-learning secured both
ALCOHOL ABUSE MANAGEMENT IN PRIMARY CARE
dice.24 Therefore, it is imperative that educational initiatives with a focus PRE- AND POST-COURSE on changing misconceptions that PRE-COURSE RANKING POST-COURSE RANKING IMPROVEMENT RANKINGa hinder optimal clinical management Question #10: Alcohol problem signs Question #2: Family guidance Question #2: Family guidance be developed and tested in future studies. The identification of this Question #8: Addiction consequences Question #10: Alcohol problem signs Question #1: Patient approach blind spot was only possible because Question #2: Family guidance Question #1: Patient approach Question #4: 12-steps program of the adoption of true-or-false type Question #1: Patient approach Question #8: Addiction consequences Question #7: Risk and protection of questions for the assessment of the learning process. Indeed, this type of Question #5: Alcohol metabolism Question #4: 12-steps program Question #8: Addiction consequences question proved more useful in proQuestion #4: 12-steps program Question #5: Alcohol metabolism Question #10: Alcohol problem signs viding a balanced evaluation about Question #7: Risk and protection Question #7: Risk and protection Question #5: Alcohol metabolism knowledge and biases and more a sensitive to changes in the cognitive Based on the Z coefficient from the Wilcoxon test displayed on Table 2. status of students than MCQs. The comparison between pre- and post-course scores suggests that genknowledge gain and retention. No variable, except previous experience eral knowledge about alcohol addiction did not improve over time. with e-learning, was related to course performance, indicating that This lack of improvement was probably due to a ceiling effect because e-learning is an accessible mean for teaching mental health topics. the pre-course scores in this case were already high. Despite presenting Despite profiting less from the course, students without previous exthe third highest score improvement, the 12-step programs score was perience enjoyed e-learning the most, probably because of a novelty not well positioned in the post-course score ranking, meaning that effect. Offering a practice pre-course and extending the tutorial time there is still room and need for improvement in knowledge about 12may help inexperienced students in future versions of the course. step programs and other community resources to support addiction However, only the positive subscores improved. In other words, patients. At the same time, the topics related to practical management the course was successful in providing new knowledge, but was not of addiction patients and their families were optimally classified both effective in dispelling misconceptions regarding alcohol-related at the post-course ranking and at the improvement ranking. Thus, the problems. The field of addiction is plagued with stigma and prejuAAMPC may help accomplish a key directive of the Family Health Program, which is to enable primary care staff and services to deal with highly prevalent behavioral challenges, such as addictions. Nonetheless, it is up to future studies to establish if e-learning is capable of Table 4. Relevance Attributed to Tools Used actually shaping addiction clinical management toward best evidencein Distance Learning based and cost-effective practices. MEAN SD MINIMUM MAXIMUM Z P It is interesting that the students who attributed more relevance to a Web conference 9.76 0.663 7.0 10.0 — — face-to-face classes profited less from the course. This reinforces the Video and 9.76 0.597 8.0 10.0 0b 1.00 perception that students who are less familiar with e-learning may a audio tips benefit from blended models and then smoothly progressing to Text reading 9.72 0.613 8.0 10.0 - 0.431b 0.666 strictly e-learning as they become less dependent on face-to-face a with MCQ interaction. It is interesting that taking the knowledge test was among AKQ completiona 9.44 1.003 6.0 10.0 - 1.510b 0.131 the four most appreciated activities by the students, possibly because it enables students to gauge by themselves how well the course Audio chat 9.36 1.186 5.0 10.0 - 1.930b 0.054 worked for them. b 0.041 Chat 9.20 1.384 5.0 10.0 - 2.047 The current study has important limitations, mainly its small Forum 8.72 1.514 4.0 10.0 - 3.093b 0.002 sample size and the attrition rate, which, although high, is in keeping with previous reports of e-learning dropout.25 Besides, the followFace-to-face 8.04 2.950 0 10.0 - 3.017b 0.003 class up period was short, and no further assessment was conducted to ascertain that knowledge retention is secured for a longer period. Scores ranged from 0 to 10 (n = 25). a Finally, the lack of a control group does not yield the comparison of Most relevant learning tools. b e-learning performance against traditional in-person teaching, or Wilcoxon’s test for related measures: compared with the highest score (Web conference). simple text reading followed by test completion. Thus, the power and AKQ, Alcohol Knowledge Questionnaire; MCQ, multiple-choice questions; SD, cost-effectiveness of e-learning to teach and engage students in standard deviation. mental health in comparison with other methods remain to be clarified. However, the current data allow the conclusion that e-learning
Table 3. Proficiency Ranking of Themes
ª M A R Y A N N L I E B E R T , I N C . VOL. 21
NO. 3 MARCH 2015
TELEMEDICINE and e-HEALTH 205
PEREIRA ET AL.
is effective in increasing the knowledge of primary care professionals about the clinical management of alcohol-related problems. Future studies should focus on strategies to foster long-term retention of knowledge, removal of misconceptions, actual change for better clinical practices, and cost-effectiveness of e-learning.
Disclosure Statement No competing financial interests exist.
REFERENCES 1. Curran VR. Tele-education. J Telemed Telecare 2006;12:57–63. 2. Khanna M, Aschenbrand SG, Kendall PC. New frontiers: Computer technology in the treatment of anxious youth. Behav Ther 2007;30:22–25. 3. Abeles P, Verduyn C, Robinson A. Computerized CBT for adolescent depression (‘‘Stress-busters’’) and its initial evaluation through an extended case series. Behav Cogn Psychother 2009;37:151–165. 4. Calear AL, Christensen H. Review of internet-based prevention and treatment programs for anxiety and depression in children and adolescents. Med J Aust 2010;192(11 Suppl):S12–S14. 5. Eikelboom RH, Weber S, Atlas MD, Dinh Q, Mbao MN, Gallop MA. A tele-otology course for primary care providers. J Telemed Telecare 2003;9(Suppl 2):S19–S22. 6. Griffiths F, Lindenmeyer A, Powell J, Low P, Thorogood M. Why are health care interventions delivered over the Internet? A systematic review of the published literature. J Med Internet Res 2006;8:e12. 7. Prince M, Patel V, Saxena S, et al. No health without mental health. Lancet 2007;370:859–877. 8. Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bull World Health Organ 2004;82:858–866. 9. Gore FM, Bloem PJ, Patton GC, et al. Global burden of disease in young people aged 10–24 years: A systematic analysis. Lancet 2011;377:2093–2102. 10. Gual A, Sabadini MB. Implementing alcohol disorders treatment throughout the community. Curr Opin Psychiatry 2011;24:203–207. 11. Collins PY, Patel V, Joestl S, et al. Grand challenges in global mental health. Nature 2011;475:27–30. 12. dos Santos AeF, Haddad SC, Alves HJ, Torres RM, de Souza C, de Melo MoC. Evaluating the experience of training through videoconferences in primary care. Telemed J E Health 2011;17:722–726. 13. Joshi A, Novaes MA, Iyengar S, et al. Evaluation of a tele-education programme in Brazil. J Telemed Telecare 2011;17:341–345. 14. Kind T. The Internet as an adjunct for pediatric primary care. Curr Opin Pediatr 2009;21:805–810.
206 TELEMEDICINE and e-HEALTH M A R C H 2 0 1 5
15. Paixa˜o MP, Miot HA, Wen CL. Tele-education on leprosy: Evaluation of an educational strategy. Telemed J E Health 2009;15:552–559. 16. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internetbased learning in the health professions: A meta-analysis. JAMA 2008;300:1181–1196. 17. McPhaul KM, Rosen J, Bobb S, et al. An exploratory study of mandated safety measures for home visiting case managers. Can J Nurs Res 2007;39:173–189. 18. Dimeff LA, Woodcock EA, Harned MS, Beadnell B. Can dialectical behavior therapy be learned in highly structured learning environments? Results from a randomized controlled dissemination trial. Behav Ther 2011;42: 263–275. 19. Beidas RS, Birkett M, Newcomb ME, Mustanski B. Do psychiatric disorders moderate the relationship between psychological distress and sexual risk-taking behaviors in young men who have sex with men? A longitudinal perspective. AIDS Patient Care STDS 2012;26:366–374. 20. Sellers B, Galanter M, Dermatis H, Nachbar M. Enhancing physicians’ use of Alcoholics Anonymous: Internet-based training. J Addict Dis 2005;24:77–86. 21. Cucciare MA, Weingardt KR, Greene CJ, Hoffman J. Current trends in using Internet and mobile technology to support the treatment of substance use disorders. Curr Drug Abuse Rev 2012;5:172–177. 22. Paim J, Travassos C, Almeida C, Bahia L, Macinko J. The Brazilian health system: History, advances, and challenges. Lancet 2011;377:1778–1797. 23. Gonc¸alves DM, Kapczinski F. [Prevalence of mental disorders at a referral center for the Family Health Program in Santa Cruz do Sul, Rio Grande do Sul State, Brazil]. Cad Saude Publica 2008;24:2043–2053. 24. Williamson L. Destigmatizing alcohol dependence: The requirement for an ethical (not only medical) remedy. Am J Public Health 2012;102:e5–e8. 25. Levy Y. Comparing dropouts and persistence in e-learning courses. Comput Educ 2007;48:185–204.
Address correspondence to: Celina Andrade Pereira, MS Rua Jose´ Coimbra, 120 – apto 6, Vila Andrade Sa˜o Paulo, SP, CEP 05726-110 Brasil E-mail: [email protected] Received: February 23, 2014 Revised: June 12, 2014 Accepted: June 16, 2014
Copyright of Telemedicine & e-Health is the property of Mary Ann Liebert, Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
