The Health Care Manager Volume 34, Number 4, pp. 337–349 Copyright # 2015 Wolters Kluwer Health, Inc. All rights reserved.
User Satisfaction With Telehealth Study of Patients, Providers, and Coordinators Mirna Becevic, PhD; Suzanne Boren, PhD; Rachel Mutrux, BA; Zalak Shah, HI; Sruti Banerjee, MHA, HI The aim of this study was to examine the satisfaction with telehealth technologies of all users— patients, health care providers, and telehealth presenters. As the use of videoconferencing in health care is rapidly increasing to allow adequate and timely access to care for patients from rural areas, it is important to examine how these technologies are perceived and utilized. Three separate surveys were used to collect data: patient, provider, and telehealth coordinator. Patient surveys were collected in a paper format, while provider and coordinator surveys were done using REDCap (Research Electronic Data Capture) application. Findings indicate high satisfaction with telehealth, as well as confidence in providing care via distance. While this is encouraging for both patients and health care organizations, further studies should be done to include urban telehealth as well as other types of health care organizations utilizing videoconferencing for clinical appointments. Key words: health informatics, health services management, quality improvement, telehealth, telemedicine
IFFERENT TELEHEALTH TECHNOLOGIES have been used for decades to bridge the distance and close the gap between rural populations and urban medical centers.1 From the early 2-way videoconferencing in 1970s to improve mental health, to the current use of robotics in intensive care and everything in between, telehealth has become the go-to technology for increasing access to care.2,3 With the increased acceptance of health care technologies, there was a rapid growth of and interest in the adoption of telehealth in different specialties over the last few decades.4,5 The rise in use and overall interest in telehealth are accompanied by a growing number of studies focusing on provider and patient satisfaction,
D
Author Affiliations: Departments of Dermatology (Dr Becevic) and Health Management and Informatics (Dr Boren and Mss Banerjee and Shah) and Missouri Telehealth Network (Mss Mutrux, Banerjee and Shah), University of Missouri, Columbia. The authors have no conflicts of interest. Correspondence: Mirna Becevic, PhD, University of Missouri, Department of Dermatology, Missouri Telehealth Network ([email protected]). DOI: 10.1097/HCM.0000000000000081
costs, outcomes, and so on.1,6-10 Whitten and Love1s found that although both patients and providers generally show high satisfaction with and acceptance of telehealth, their perceptions of this technology somewhat differ. Patients seem to view telehehealth as a great tool for overcoming the physical distance to get the necessary care.1 However, providers show a level of skepticism due to technological challenges, such as lack of adequate training, or difficulty of handling the telehealth equipment.1 The University of Missouri (MU) Missouri Telehealth Network (MTN) was established in 1995 with the mission to improve access to care for rural Missourians. The MTN began with the hub-and-spoke model, with only 1 county connected to the MU Hospital and Clinics. Over the years, the network grew to connect 202 telehealth sites in 62 counties and offer 29 different clinical specialties and subspecialties.11 The number of telehealth appointments on the MTN network grew exponentially as the number of sites and specialties increased, with 34 792 appointments in year 2013 conducted on the network. In order to fully understand the perceived quality of these appointments, adequacy of training provided by MTN staff, 337
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
338
THE HEALTH CARE MANAGER/OCTOBER-DECEMBER 2015
and the overall perception of telehealth services by all users, the MTN conducted a satisfaction study that included all telehealth providers, telehealth coordinators, and patients who used telehealth within the study period. This was a quality improvement project aimed at examining the satisfaction, usability, effectiveness, and comfort level by all telehealth users with this technology in order to better understand how the services offered are perceived. The expansion of telehealth is dependent on the acceptance and satisfaction of the providers, telehealth coordinators, and consumers. This is a critical issue for health care organizations because of the investments made in information technology and quality improvement. The current pressure in health care environment is to do more with less, leaving health care administrators to find adequate solutions to provide specialty care to rural and underserved. Telehealth in itself may not offer all answers to the physician shortage crisis; however, when implemented correctly and utilized to its full potential, telehealth has shown the potential to bring the timely, affordable, and quality health care to rural communities. MATERIALS AND METHODS The study was performed at MU, Columbia, the MTN, and was approved by the MU Health Sciences institutional review board. Data were collected between July 1, 2013, and August 9, 2013. The MTN is an administrative department within the Office of Rural Health Programs, School of Medicine, MU, Columbia. Missouri Telehealth Network mission corresponds with the mission of the MU School of Medicine’s 3 tenets: providing clinical care, education, and research to all Missourians.12 Within the clinical care piece, MTN’s role is to provide start-up assistance for telehealth projects (internally and our hub sites), as well as ongoing support in clinical and operations, legal and regulatory, and technical aspects.13 Missouri Telehealth Network also provides continuing medical education opportunities to participating sites.14 This is done in 2 ways: enabling them to
connect to MU’s regularly scheduled grand rounds and requesting continuing medical education on a specific topic (1 time or series).14 This study focused on improving current services by examining the user perceptions, attitudes, and acceptance of telehealth in various specialties. For this purpose, 3 separate surveys were developed: patient, provider, and coordinator (Surveys 1-3). A review of literature was performed, as well as analyses of the roles of patients, coordinators, and providers in the telehealth appointment to determine the validity of the questions. A 5-point Likert scale was used for the questions pertaining to satisfaction with the telehealth services. Providers and coordinators completed survey electronically via the REDCap (Research Electronic Data Capture). REDCap is a secure, Web-based application designed to support data capture for research studies.15 It provides an intuitive interface for validating data entry, audit trails for tracking data manipulation, and export procedures; automated export procedures for seamless data downloads to common statistical packages; and procedures for importing data from external sources.15 Patients completed paper surveys. Patient survey (see Survey 1). The patient survey consisted of 13 questions and a comment section. The questions captured the information about demographic, specialty, frequency, and satisfaction levels in specific areas. The questions regarding the quality and effectiveness of services were divided into 4 categories: (1) risks and benefits, (2) effectiveness, (3) efficiency, and (4) future adoption. Provider survey (see Survey 2). The provider survey also contained 18 questions. The questions focused on the type of the facility they are employed in as well as the provider title and specialty. Providers were also asked about the ease of use of telehealth equipment, if the images and sounds were of good quality, how well they can communicate with and treat their patients via telehealth, and how satisfied they are with telehealth appointments.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
User Satisfaction With Telehealth
339
SURVEY 1 Patient Survey Today you saw the doctor over a video connection instead of an in-person visit. This is commonly known as telehealth. Thank you for being a telehealth patient! It is very important for us to learn all that we can about telehealth. We need your help. Please take a minute to complete this survey. Your answers will be kept private. Date: _________________________ Gender: & Male & Female & Age: _______ How many times have you seen a doctor through telehealth? Place a check in the box next to your choice. & This is my first time & 2 to 5 times & More than 5 times Check all the telehealth specialties that you have used & Autism and neurodevelopmental disorders & Physical medicine and rehabilitation & Dermatology & Endocrinology & Nephrology & Orthopedics & Pediatric specialty & Burn/wound care & Psychiatry & Rheumatology & Urology & Others ____________ (Please specify) Risks and Benefits – Place a check in the box next to your choice. 1. I felt good about my telehealth doctor’s medical skill. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 2. Lack of physical contact was fine with me. & Strongly agree & Agree & Neutra & Disagree & Strongly disagree 3. I was fine with being on camera. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 4. I felt that things I shared were kept private. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Effectiveness – Place a check in the box next to your choice. 1. My telehealth visit was as good as one that is in person. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 1. It was easy for me to state my concerns and ask questions through telehealth. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 2. My doctor took the time needed to answer my questions. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Efficiency – Place a check in the box next to your choice. 1. I had to wait less time for the telehealth visit than I do for one that is in person. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 2. The staff at my clinic knew how to handle the equipment. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 3. I like the way the staff explained how telehealth works. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Future Adoption – Place a check in the box next to your choice. 1. The use of telehealth saved me travel time and money. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 2. I would use telehealth to have my doctor see me again. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 3. I will tell the people I know about telehealth. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Comments and suggestions are welcome!
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
340
THE HEALTH CARE MANAGER/OCTOBER-DECEMBER 2015
SURVEY 2 Provider Survey Thank you for being a telehealth user! It is very important for us to learn all that we can about your experience. Please take 15 minutes to complete the survey. Your answers will be kept private. Place a check in the box next to your option. Gender & Male & Female Age ____________________________________________ I work at a & Community mental health center & FQHC & Rural clinics & Critical access hospital & Academic center & Other & I work at a ____________________________________________ & I am a/an & MD & PhD & NP & (Other) I am a ______________________________________________ The telehealth specialties that I am involved with are (check all that apply) & Autism and neurodevelopmental disorders & Burn/wound care & Dermatology & Endocrinology & Health ethics consultations & Nephrology & Orthopedics & Pediatric specialty & Physical medicine and rehabilitation & Psychiatry & Rheumatology & Urology & Other The telehealth specialties that I am involved with are _____________________________________. I have used/ been using telehealth for & Less than 2 years & 2 to 5 years & More than 5 years I use telehealth & Once a week & Once a month & Once a year & Other I use telehealth _______________________________________________. It is easy to run and use the telehealth system. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree I am confident and feel at ease when I use the telehealth system. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree (continues)
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
User Satisfaction With Telehealth
341
SURVEY 2, Continued Telehealth gives me the chance to build and keep a personal bond with each of my patients. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Telehealth fits well with each day’s workflow. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree The images and sounds of telehealth gear are clear and crisp. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree I get more done in my day when I see patients through telehealth. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Telehealth helps me to converse with my patients. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Telehealth allows me to see more patients. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree I am able to treat my patients’ need well through telehealth & Strongly agree & Agree & Neutral & Disagree & Strongly disagree I prefer telehealth visits over visits that are in person. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree For the most, I am satisfied with the work I’ve done through telehealth. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree
Coordinator survey (see Survey 3). The coordinator survey consisted of 18 questions aimed at finding the type of facility where they were employed as well as their specific function within the organization. They were also asked about the ease of contacting MTN staff, satisfaction with telehealth training, ease of setting up telehealth appointments, and how well telehealth fits with the clinic flow. A total of 830 patient paper surveys were sent to the behavioral health and other medical clinic sites to be distributed to after their telehealth appointments. Ninety-five providers at the MU Hospital and Clinics and behavioral health organizations were invited to complete the questionnaire. Also, 75 site coordinators at behavioral health and other medical clinic sites were invited to complete the online survey. Data analysis Upon receipt, the numerical data obtained from the closed-ended questions on the surveys were analyzed, and the answers were coded and entered into a spreadsheet for statistical analysis. Simple descriptive statistics were performed to analyze the results. Excel (2003; Microsoft, Redmond, Washington) and SPSS for
Windows (version 16.0, released 2007; SPSS Inc, Chicago, Illinois.) were used for data analysis. RESULTS Patient survey A total of 286 patients participated in the study. The majority of them (63%) were female, 33% were male, whereas 11% chose not to identify their gender. Table 1 shows the participants’ data and the response rate. Eighty percent of the respondents used telehealth for behavioral health appointments, followed by autism and dermatology. The majority of the patients (n = 235) felt that their information is kept private, with a mean of 4.14. Only 6% of the patients did not like the lack of physical contact, whereas 38% agreed and 26% strongly agreed that the lack of contact was fine. One hundred ninety-three patients considered telemedicine visit is as good as an in-person visit, with a mean of 3.79. Patients had high confidence level in telehealth skills of the clinic staff, with a mean of 4.35, as well as their explanation of how telehealth works, with a mean of 4.17. Almost 78% of patients would use telehealth again to see their doctor, whereas 68% agreed
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
342
THE HEALTH CARE MANAGER/OCTOBER-DECEMBER 2015
SURVEY 3 Coordinator Survey Thank you for being a telehealth user! It is very important for us to learn all that we can about your experience. Please take 15 minutes to complete the survey. Your answers will be kept private. Place a check in the box next to your option. Gender & Male & Female Age ____________________________________________ I work at a & Community mental health center & FQHC & Rural clinics & Critical access hospital & Academic center & Other I work at a ____________________________________________. My position at the facility is & Administrator & Nurse & Other My position at the facility is ______________________________________________. The telehealth specialties that I am involved with are (check all that apply) & Autism and neurodevelopmental disorders & Burn/wound care & Dermatology & Endocrinology & Health ethics consultations & Nephrology & Orthopedics & Pediatric specialty & Physical medicine and rehabilitation & Psychiatry & Rheumatology & Urology & Other The telehealth specialties that I am involved with are ___________________________. I have used/ been using telehealth for & Less than 2 years & 2 to 5 years & More than 5 years I use telehealth & Once a week & Once a month & Once a year & Other I use telehealth _______________________________________________. (continues)
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
User Satisfaction With Telehealth
343
SURVEY 3, Continued For the most part, I prefer to send and receive telehealth request forms by & Fax & E-mail It is easy to reach MTN (Missouri Telehealth Network) agents, and they respond when we need them. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree It is easy to set up a telehealth appointment. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree It is easy to set up and use telehealth system. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Telehealth is a useful tool for us when we see patients. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Telehealth fits well with everyday’s workflow. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree I have been trained well to use telehealth. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree The images and sounds of the telehealth gear are clear and crisp. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Telehealth records are easy to use and work with. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree The time I give to telehealth is used in an effective way. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree For the most part, I am satisfied with the work I’ve done through telehealth. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Comments and suggestions are welcome! that telehealth saved them travel time and money. More than 85% felt confident about the clinicians’ ability to use telemedicine devices and equipment effectively and safely; approximately 70% of the patients indicated that they will recommend telemedicine to people they know and will see their doctor again via telemedicine. Table 2 shows the patients’ responses to the questionnaire. Provider survey A total of 21 providers participated in the survey, all of whom (100%) worked in the academic medical center. The majority of the providers (76%) were medical doctors, 19% were
nurses, and 5% had a doctor of philosophy degree. Six providers (28%) worked in autism and neurodevelopmental disorders clinic, followed by 19% endocrinologists, with the rest being divided among dermatology, pediatric specialty, physical medicine and rehabilitation, psychiatry, and rheumatology. Providers had very strong confidence levels in the ease of use of telehealth equipment, with a mean of 4.29. They also reported a good quality of images and sounds (mean, 4.00). None of the providers reported preference of the telehealth visits over in-person visits, with a mean of 2.0. Providers tend to have neutral opinions toward the question if telehealth
Table 1. Participants
Stakeholders Patients Providers Coordinators
Invited
Responded
Response Rate
830 95 75
286 21 12
34.5% 22.1% 16%
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
344
THE HEALTH CARE MANAGER/OCTOBER-DECEMBER 2015
Table 2. Patient Survey n (%) Risks and benefits I felt good about my telehealth doctor’s medical skill (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) Lack of physical contact was fine with me (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) I was fine with being on camera (n = 282) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) I felt that things I shared were kept private (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) Effectiveness My telehealth visit was as good as one that is in person (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) It was easy for me to state my concerns and ask questions through telehealth (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) My doctor took the time needed to answer my questions (n = 281) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) Efficiency I had to wait less time for the telehealth visit than I do for one that is in person (n = 278) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank)
Mean 4.28
135 103 32 5 5 6
(47.2) (36.0) (11.2) (1.7) (1.8) (2.1)
75 111 50 26 18 6
(26.2) (38.8) (17.5) (9.1) (6.3) (2.1)
78 125 36 26 17 4
(27.3) (43.7) (12.5) (9.1) (5.9) (1.4)
108 127 29 9 7 6
(37.8) (44.4) (10.1) (3.2) (2.4) (2.1)
92 101 42 27 18 6
(32.2) (35.3) 3.79 (14.7) (9.4) (6.3) (2.1) 3.91 (32.5) (41.3) (11.5) (7.7) (4.9) (2.1) 4.25 (43.7) (42.3) (7) (3.1) (2.1) (1.7)
3.71
3.78
4.14
93 118 33 22 14 6 125 121 20 9 6 5
3.71 75 96 70 25 12 8
(26.2) (33.6) (24.5) (8.7) (4.2) (2.8) (continues)
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
User Satisfaction With Telehealth
345
Table 2. Patient Survey, Continued n (%) The staff at my clinic knew how to handle the equipment (n = 278) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) I like the way the staff explained how telehealth works (n = 278) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) Future adoption The use of telehealth saved me travel time and money (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) I would use telehealth to have my doctor see me again (n = 278) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) I will tell people I know about telehealth (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank)
would allow them to see more patients (43%). Fifty-seven percent of the providers agreed that they are able to treat patients via telehealth, whereas 62% agreed that they are satisfied with the work done through telehealth, with a mean of 4.05. Table 3 shows the patients’ responses to the questionnaire. Coordinators survey Twelve telehealth coordinators completed the survey. Thirty-three percent worked at a critical access hospital, 33% indicated ‘‘other’’ as a place of employment, whereas the reminder of the respondents were employed at rural clinics, federally qualified health care centers, and an academic health center. Of the
Mean 4.35
122 (42.7) 132 (46.2) 24 (8.4) 0 0 8 (2.8) 4.17 107 125 35 10 1 8
(37.4) (43.7) (12.2) (3.5) (0.4) (2.8)
112 83 51 24 10 6
(39.2) (29.0) (17.8) (8.4) (3.5) (2.1)
110 113 41 5 9 8
(38.5) (39.5) (14.3) (1.7) (3.2) (2.8)
94 106 45 20 15 6
(32.8) (37.1) (15.7) (7) (5.2) (2.1)
3.93
4.11
3.87
12 respondents, 58% use telehealth once a week, and 25% use telehealth once a month. Coordinators reported high confidence levels with the ease of use and setup of telehealth appointments, with a mean of 4.36. Similarly, 67% of the respondents indicated it was easy to contact and reach MTN when needed. Ninety-one percent of the respondents felt they were trained well on how to use the telehealth equipment and that they are satisfied with the work they do over telehealth, with a mean of 4.42. Table 4 shows the coordinators’ responses to the survey. DISCUSSION This study examined satisfaction with telehealth services provided on the MTN. Data
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
346
THE HEALTH CARE MANAGER/OCTOBER-DECEMBER 2015
Table 3. Provider Survey
It is easy to run and use a telehealth system (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree I am confident and feel at ease when I use the telehealth system (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth gives me the chance to build and keep a personal bond with each of my patients (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth fits well with each day’s workflow (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree The images and sounds of telehealth gear are crisp and clear (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree I get more done in my day when I see patients through telehealth (n = 20) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth helps me to converse with my patients (n = 20) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth allows me to see more patients (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree I am able to treat my patients’ need well through telehealth (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree I prefer telehealth visits over visits that are in person (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree For the most, I am satisfied with the work I’ve done through telehealth (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree
n (%)
Mean
7 (33.3) 13 (61.9) 1 (4.8) 0 (0.0) 0 (0.0)
4.29
10 (47.6) 9 (42.9) 0 (0.0) 2 (9.5) 0 (0.0)
4.29
6 (28.6) 9 (42.9) 3 (14.3) 3 (14.3) 0 (0.0)
3.86
6 (28.6) 7 (33.3) 4 (19) 2 (9.5) 2 (9.5)
3.62
6 (28.6) 11 (52.4) 2 (9.5) 2 (9.5) 0 (0.0)
4.00
1 (5) 3 (15) 11 (55) 5 (25) 0 (0.0)
3.00
3 (15) 11 (55) 5 (25) 1 (5) 0 (0.0) 1 (4.8) 4 (19) 9 (42.9) 6 (28.6) 1 (4.8)
2.90
4 (19) 12 (57.1) 2 (9.5) 3 (14.3) 0 (0.0)
3.81
0 (0.0) 0 (0.0) 8 (38.1) 5 (23.8) 8 (38.1)
2.00
5 (23.8) 13 (61.9) 2 (9.5) 1 (4.8) 0 (0.0)
4.05
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
3.80
347
User Satisfaction With Telehealth Table 4. Coordinator Survey n (%) It is easy to reach MTN (Missouri Telehealth Network) agents and they respond when we need them. . . (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree It is easy to set up a telehealth appointment (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree It is easy to set up and use telehealth system (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth is a useful tool for us when we see patients (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth fits well with everyday’s workflow (n = 11) Strongly agree Agree Neutral Disagree Strongly disagree I have been trained to well to use telehealth (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree The images and sounds of the telehealth gear are clear and crisp (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth records are easy to use and work with (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree The time I give to telehealth is used in an effective way (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree For the most part, I am satisfied with the work I’ve done through telehealth (n = 12) Strongly agree Agree Neutral Disagree
Mean 4.33
8 0 4 0 0
(66.7) (0.0) (33.3) (0.0) (0.0)
4 4 4 0 0
(33.3) (33.3) (33.3) (0.0) (0.0)
5 5 1 0 0
(45.5) (45.5) (9.1) (0.0) (0.0)
5 5 2 0 0
(41.7) (41.7) (16.7) (0.0) (0.0)
4 5 2 0 0
(36.4) (45.5) (18.2) (0.0) (0.0)
4.0
4.36
4.25
4.18
4.42 6 (50) 5 (41.7) 1 (8.3) 0 (0.0) 0 (0.0) 4.25 4 7 1 0 0
(33.3) (58.3) (8.3) (0.0) (0.0) 3.83
3 (25) 4 (33.3) 5 (41.7) 0 (0.0) 0 (0.0) 4.25 4 (33.3) 7 (58.3) 1 (8.3) 0 (0.0) 0 (0.0) 4.42 6 (50) 5 (41.7) 1 (8.3) 0 (0.0) (continues)
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
348
THE HEALTH CARE MANAGER/OCTOBER-DECEMBER 2015
Table 4. Coordinator Survey, Continued Telehealth records are easy to use and work with (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree The time I give to telehealth is used in an effective way (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree For the most part, I am satisfied with the work I’ve done through telehealth (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree
were collected from the patients, telehealth coordinators, and providers. Patient and employee satisfaction scores have been used by hospitals across country not only to evaluate and improve clinical care, but also to help plan the growth of services. In this case, findings indicated high satisfaction by all 3 surveyed groups, which is consistent with many other telehealth satisfaction studies. The MTN holds a biannual training conference for telehealth coordinators, which focuses on hands-on equipment training, legal and regulatory issues, and telehealth reimbursement. This study also aimed at finding out how the training is perceived and whether the MTN staff were easily reachable in case of technical or scheduling difficulties by providers and telehealth coordinators. The study found that all of the telehealth coordinators who responded to the survey felt that the MTN staff was easy to reach and responsive, and 67% of them responded that the telehealth appointments are easy to set up. The majority of the providers (76%) felt that they can treat their patients well via telehealth and that they are satisfied with the work they have done via telehealth (86%). However, they still do not prefer telehealth over inperson visits (62%). Physicians are trained to provide direct care to patients, and although
3.83 3 (25) 4 (33.3) 5 (41.7) 0 (0.0) 0 (0.0) 4.25 4 (33.3) 7 (58.3) 1 (8.3) 0 (0.0) 0 (0.0) 4.42 6 (50) 5 (41.7) 1 (8.3) 0 (0.0) 0 (0.0)
doing so via video may be an adequate option, it will never replace a hands-on examination. The study had several limitations. Providers that offer telehealth are located and affiliated with different health care organizations; however, the only ones that responded to the survey were at an academic medical center. On the other hand, the majority of the patients who participated in this study were behavioral health patients, with only a few who used telehealth to see other specialists. Because of time and other limitations, the study did not include in-person visits from patients from the same counties to compare the telehealth visits to those that make the trip to see their providers. This quality improvement study was instrumental in that it allowed the MTN and the University of Missouri to evaluate the perceptions and opinions of telehealth users. It also confirmed and validated the importance of a comprehensive training and a whole team approach to the success of the overall acceptance of telehealth. With such high satisfaction, as well as the addition of other specialties offering telehealth services during the past decade, it was not surprising to see a corresponding growth of patients using the servies. However, to understand the full impact on telehealth, it is proposed to expand this study to include a
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
User Satisfaction With Telehealth comparative study of telehealth patients and in-person patients in each specialty. It is also proposed to survey providers from different types of health care organizations.
349
ACKNOWLEDGMENT The authors thank the staff of the MTN for their help and support during the study period.
REFERENCES 1. Whitten P, Love B. Patient and provider satisfaction with the use of telemedicine: overview and rationale for cautious enthusiasm. J Postgrad Med. 2005;51(4): 294-300. 2. Wittson CL, Benschoter R. Two way television: helping the medical center reach out. Am J Psychiatry. 1972;129(5):624-627. 3. Breen GM, Matusitz J. An evolutionary examination of telemedicine: a health and computer-mediated communication perspective. Soc Work Public Health. 2010; 25(1):59-71. 4. Ho K, Karlinsky H, Jarvis-Selinger S, May J. Videoconferencing for telehealth: unexpected challenges and unprecedented opportunities. BCMJ. 2004;46(6): 285-289. 5. Thomas CL, Man MS, O’Cathain A, et al. Effectiveness and cost-effectiveness of a telehealth intervention to support the management of long-term conditions: study protocol for two linked randomized trials. Trials. 2014;15:36. 6. Wade VA, Karnon J, Elshaug AG, Hiller JE. A systematic review of economic analyses of telehealth services using real time video communication. BMC Health Serv Res. 2010;10:233. 7. Moeckli J, Cram P, Cunningham C, Reisinger HS. Staff acceptance of a telemedicine intensive care unit program: a qualitative study. J Crit Care. 2013;28(6): 890-901. 8. Avey JP, Hobbs RL. Dial in: fostering the use of
9.
10.
11.
12.
13.
14.
15.
telebehavioral healt hservices in frontier Alaska. Psychol Serv. 2013;10(3):289-297. Thaker DA, Monypenny R, Olver I, Sabesan S. Cost savings from a telemedicine model of care in northern Queensland, Australia. Med J Aust. 2013;199(6):414-417. Kumar G, Falk DM, Bonello RS, Kahn JM, Perencevich E, Cram P. The cost of critical care telemedicine programs: a systematic review and analysis. Chest. 2013; 143(1):19-29. Missouri Telehealth Network Telehealth Sites. Missouri Telehealth Network [online]. 2014. Cited January 22, 2015. http://medicine.missouri.edu/telehealth/sites.html. Accessed January 22, 2015. School of Medicine University of Missouri Health System [online]. Cited January 22, 2015. http://medicine. missouri.edu/. Accessed January 22, 2015. Missouri Telehealth Network General Information. Missouri Telehealth Network [online]. Cited January 23, 2015. http://medicine.missouri.edu/telehealth/general.html. Accessed January 22, 2015. Missouri Telehealth Network Educational Programs. Missouri Telehealth Network [online]. Cited January 22, 2015. http://medicine.missouri.edu/telehealth/educationalprograms.html. Accessed January 22, 2015. REDCap (Research Electronic Data Capture). University of Missouri Institute for Clinical and Translational Science [online]. December 30, 2013. Cited January 22, 2015. http://icats.missouri.edu/portal/redcap.php. Accessed January 22, 2015.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
User Satisfaction With Telehealth Study of Patients, Providers, and Coordinators Mirna Becevic, PhD; Suzanne Boren, PhD; Rachel Mutrux, BA; Zalak Shah, HI; Sruti Banerjee, MHA, HI The aim of this study was to examine the satisfaction with telehealth technologies of all users— patients, health care providers, and telehealth presenters. As the use of videoconferencing in health care is rapidly increasing to allow adequate and timely access to care for patients from rural areas, it is important to examine how these technologies are perceived and utilized. Three separate surveys were used to collect data: patient, provider, and telehealth coordinator. Patient surveys were collected in a paper format, while provider and coordinator surveys were done using REDCap (Research Electronic Data Capture) application. Findings indicate high satisfaction with telehealth, as well as confidence in providing care via distance. While this is encouraging for both patients and health care organizations, further studies should be done to include urban telehealth as well as other types of health care organizations utilizing videoconferencing for clinical appointments. Key words: health informatics, health services management, quality improvement, telehealth, telemedicine
IFFERENT TELEHEALTH TECHNOLOGIES have been used for decades to bridge the distance and close the gap between rural populations and urban medical centers.1 From the early 2-way videoconferencing in 1970s to improve mental health, to the current use of robotics in intensive care and everything in between, telehealth has become the go-to technology for increasing access to care.2,3 With the increased acceptance of health care technologies, there was a rapid growth of and interest in the adoption of telehealth in different specialties over the last few decades.4,5 The rise in use and overall interest in telehealth are accompanied by a growing number of studies focusing on provider and patient satisfaction,
D
Author Affiliations: Departments of Dermatology (Dr Becevic) and Health Management and Informatics (Dr Boren and Mss Banerjee and Shah) and Missouri Telehealth Network (Mss Mutrux, Banerjee and Shah), University of Missouri, Columbia. The authors have no conflicts of interest. Correspondence: Mirna Becevic, PhD, University of Missouri, Department of Dermatology, Missouri Telehealth Network ([email protected]). DOI: 10.1097/HCM.0000000000000081
costs, outcomes, and so on.1,6-10 Whitten and Love1s found that although both patients and providers generally show high satisfaction with and acceptance of telehealth, their perceptions of this technology somewhat differ. Patients seem to view telehehealth as a great tool for overcoming the physical distance to get the necessary care.1 However, providers show a level of skepticism due to technological challenges, such as lack of adequate training, or difficulty of handling the telehealth equipment.1 The University of Missouri (MU) Missouri Telehealth Network (MTN) was established in 1995 with the mission to improve access to care for rural Missourians. The MTN began with the hub-and-spoke model, with only 1 county connected to the MU Hospital and Clinics. Over the years, the network grew to connect 202 telehealth sites in 62 counties and offer 29 different clinical specialties and subspecialties.11 The number of telehealth appointments on the MTN network grew exponentially as the number of sites and specialties increased, with 34 792 appointments in year 2013 conducted on the network. In order to fully understand the perceived quality of these appointments, adequacy of training provided by MTN staff, 337
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
338
THE HEALTH CARE MANAGER/OCTOBER-DECEMBER 2015
and the overall perception of telehealth services by all users, the MTN conducted a satisfaction study that included all telehealth providers, telehealth coordinators, and patients who used telehealth within the study period. This was a quality improvement project aimed at examining the satisfaction, usability, effectiveness, and comfort level by all telehealth users with this technology in order to better understand how the services offered are perceived. The expansion of telehealth is dependent on the acceptance and satisfaction of the providers, telehealth coordinators, and consumers. This is a critical issue for health care organizations because of the investments made in information technology and quality improvement. The current pressure in health care environment is to do more with less, leaving health care administrators to find adequate solutions to provide specialty care to rural and underserved. Telehealth in itself may not offer all answers to the physician shortage crisis; however, when implemented correctly and utilized to its full potential, telehealth has shown the potential to bring the timely, affordable, and quality health care to rural communities. MATERIALS AND METHODS The study was performed at MU, Columbia, the MTN, and was approved by the MU Health Sciences institutional review board. Data were collected between July 1, 2013, and August 9, 2013. The MTN is an administrative department within the Office of Rural Health Programs, School of Medicine, MU, Columbia. Missouri Telehealth Network mission corresponds with the mission of the MU School of Medicine’s 3 tenets: providing clinical care, education, and research to all Missourians.12 Within the clinical care piece, MTN’s role is to provide start-up assistance for telehealth projects (internally and our hub sites), as well as ongoing support in clinical and operations, legal and regulatory, and technical aspects.13 Missouri Telehealth Network also provides continuing medical education opportunities to participating sites.14 This is done in 2 ways: enabling them to
connect to MU’s regularly scheduled grand rounds and requesting continuing medical education on a specific topic (1 time or series).14 This study focused on improving current services by examining the user perceptions, attitudes, and acceptance of telehealth in various specialties. For this purpose, 3 separate surveys were developed: patient, provider, and coordinator (Surveys 1-3). A review of literature was performed, as well as analyses of the roles of patients, coordinators, and providers in the telehealth appointment to determine the validity of the questions. A 5-point Likert scale was used for the questions pertaining to satisfaction with the telehealth services. Providers and coordinators completed survey electronically via the REDCap (Research Electronic Data Capture). REDCap is a secure, Web-based application designed to support data capture for research studies.15 It provides an intuitive interface for validating data entry, audit trails for tracking data manipulation, and export procedures; automated export procedures for seamless data downloads to common statistical packages; and procedures for importing data from external sources.15 Patients completed paper surveys. Patient survey (see Survey 1). The patient survey consisted of 13 questions and a comment section. The questions captured the information about demographic, specialty, frequency, and satisfaction levels in specific areas. The questions regarding the quality and effectiveness of services were divided into 4 categories: (1) risks and benefits, (2) effectiveness, (3) efficiency, and (4) future adoption. Provider survey (see Survey 2). The provider survey also contained 18 questions. The questions focused on the type of the facility they are employed in as well as the provider title and specialty. Providers were also asked about the ease of use of telehealth equipment, if the images and sounds were of good quality, how well they can communicate with and treat their patients via telehealth, and how satisfied they are with telehealth appointments.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
User Satisfaction With Telehealth
339
SURVEY 1 Patient Survey Today you saw the doctor over a video connection instead of an in-person visit. This is commonly known as telehealth. Thank you for being a telehealth patient! It is very important for us to learn all that we can about telehealth. We need your help. Please take a minute to complete this survey. Your answers will be kept private. Date: _________________________ Gender: & Male & Female & Age: _______ How many times have you seen a doctor through telehealth? Place a check in the box next to your choice. & This is my first time & 2 to 5 times & More than 5 times Check all the telehealth specialties that you have used & Autism and neurodevelopmental disorders & Physical medicine and rehabilitation & Dermatology & Endocrinology & Nephrology & Orthopedics & Pediatric specialty & Burn/wound care & Psychiatry & Rheumatology & Urology & Others ____________ (Please specify) Risks and Benefits – Place a check in the box next to your choice. 1. I felt good about my telehealth doctor’s medical skill. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 2. Lack of physical contact was fine with me. & Strongly agree & Agree & Neutra & Disagree & Strongly disagree 3. I was fine with being on camera. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 4. I felt that things I shared were kept private. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Effectiveness – Place a check in the box next to your choice. 1. My telehealth visit was as good as one that is in person. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 1. It was easy for me to state my concerns and ask questions through telehealth. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 2. My doctor took the time needed to answer my questions. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Efficiency – Place a check in the box next to your choice. 1. I had to wait less time for the telehealth visit than I do for one that is in person. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 2. The staff at my clinic knew how to handle the equipment. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 3. I like the way the staff explained how telehealth works. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Future Adoption – Place a check in the box next to your choice. 1. The use of telehealth saved me travel time and money. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 2. I would use telehealth to have my doctor see me again. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree 3. I will tell the people I know about telehealth. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Comments and suggestions are welcome!
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
340
THE HEALTH CARE MANAGER/OCTOBER-DECEMBER 2015
SURVEY 2 Provider Survey Thank you for being a telehealth user! It is very important for us to learn all that we can about your experience. Please take 15 minutes to complete the survey. Your answers will be kept private. Place a check in the box next to your option. Gender & Male & Female Age ____________________________________________ I work at a & Community mental health center & FQHC & Rural clinics & Critical access hospital & Academic center & Other & I work at a ____________________________________________ & I am a/an & MD & PhD & NP & (Other) I am a ______________________________________________ The telehealth specialties that I am involved with are (check all that apply) & Autism and neurodevelopmental disorders & Burn/wound care & Dermatology & Endocrinology & Health ethics consultations & Nephrology & Orthopedics & Pediatric specialty & Physical medicine and rehabilitation & Psychiatry & Rheumatology & Urology & Other The telehealth specialties that I am involved with are _____________________________________. I have used/ been using telehealth for & Less than 2 years & 2 to 5 years & More than 5 years I use telehealth & Once a week & Once a month & Once a year & Other I use telehealth _______________________________________________. It is easy to run and use the telehealth system. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree I am confident and feel at ease when I use the telehealth system. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree (continues)
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
User Satisfaction With Telehealth
341
SURVEY 2, Continued Telehealth gives me the chance to build and keep a personal bond with each of my patients. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Telehealth fits well with each day’s workflow. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree The images and sounds of telehealth gear are clear and crisp. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree I get more done in my day when I see patients through telehealth. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Telehealth helps me to converse with my patients. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Telehealth allows me to see more patients. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree I am able to treat my patients’ need well through telehealth & Strongly agree & Agree & Neutral & Disagree & Strongly disagree I prefer telehealth visits over visits that are in person. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree For the most, I am satisfied with the work I’ve done through telehealth. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree
Coordinator survey (see Survey 3). The coordinator survey consisted of 18 questions aimed at finding the type of facility where they were employed as well as their specific function within the organization. They were also asked about the ease of contacting MTN staff, satisfaction with telehealth training, ease of setting up telehealth appointments, and how well telehealth fits with the clinic flow. A total of 830 patient paper surveys were sent to the behavioral health and other medical clinic sites to be distributed to after their telehealth appointments. Ninety-five providers at the MU Hospital and Clinics and behavioral health organizations were invited to complete the questionnaire. Also, 75 site coordinators at behavioral health and other medical clinic sites were invited to complete the online survey. Data analysis Upon receipt, the numerical data obtained from the closed-ended questions on the surveys were analyzed, and the answers were coded and entered into a spreadsheet for statistical analysis. Simple descriptive statistics were performed to analyze the results. Excel (2003; Microsoft, Redmond, Washington) and SPSS for
Windows (version 16.0, released 2007; SPSS Inc, Chicago, Illinois.) were used for data analysis. RESULTS Patient survey A total of 286 patients participated in the study. The majority of them (63%) were female, 33% were male, whereas 11% chose not to identify their gender. Table 1 shows the participants’ data and the response rate. Eighty percent of the respondents used telehealth for behavioral health appointments, followed by autism and dermatology. The majority of the patients (n = 235) felt that their information is kept private, with a mean of 4.14. Only 6% of the patients did not like the lack of physical contact, whereas 38% agreed and 26% strongly agreed that the lack of contact was fine. One hundred ninety-three patients considered telemedicine visit is as good as an in-person visit, with a mean of 3.79. Patients had high confidence level in telehealth skills of the clinic staff, with a mean of 4.35, as well as their explanation of how telehealth works, with a mean of 4.17. Almost 78% of patients would use telehealth again to see their doctor, whereas 68% agreed
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
342
THE HEALTH CARE MANAGER/OCTOBER-DECEMBER 2015
SURVEY 3 Coordinator Survey Thank you for being a telehealth user! It is very important for us to learn all that we can about your experience. Please take 15 minutes to complete the survey. Your answers will be kept private. Place a check in the box next to your option. Gender & Male & Female Age ____________________________________________ I work at a & Community mental health center & FQHC & Rural clinics & Critical access hospital & Academic center & Other I work at a ____________________________________________. My position at the facility is & Administrator & Nurse & Other My position at the facility is ______________________________________________. The telehealth specialties that I am involved with are (check all that apply) & Autism and neurodevelopmental disorders & Burn/wound care & Dermatology & Endocrinology & Health ethics consultations & Nephrology & Orthopedics & Pediatric specialty & Physical medicine and rehabilitation & Psychiatry & Rheumatology & Urology & Other The telehealth specialties that I am involved with are ___________________________. I have used/ been using telehealth for & Less than 2 years & 2 to 5 years & More than 5 years I use telehealth & Once a week & Once a month & Once a year & Other I use telehealth _______________________________________________. (continues)
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
User Satisfaction With Telehealth
343
SURVEY 3, Continued For the most part, I prefer to send and receive telehealth request forms by & Fax & E-mail It is easy to reach MTN (Missouri Telehealth Network) agents, and they respond when we need them. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree It is easy to set up a telehealth appointment. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree It is easy to set up and use telehealth system. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Telehealth is a useful tool for us when we see patients. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Telehealth fits well with everyday’s workflow. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree I have been trained well to use telehealth. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree The images and sounds of the telehealth gear are clear and crisp. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Telehealth records are easy to use and work with. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree The time I give to telehealth is used in an effective way. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree For the most part, I am satisfied with the work I’ve done through telehealth. & Strongly agree & Agree & Neutral & Disagree & Strongly disagree Comments and suggestions are welcome! that telehealth saved them travel time and money. More than 85% felt confident about the clinicians’ ability to use telemedicine devices and equipment effectively and safely; approximately 70% of the patients indicated that they will recommend telemedicine to people they know and will see their doctor again via telemedicine. Table 2 shows the patients’ responses to the questionnaire. Provider survey A total of 21 providers participated in the survey, all of whom (100%) worked in the academic medical center. The majority of the providers (76%) were medical doctors, 19% were
nurses, and 5% had a doctor of philosophy degree. Six providers (28%) worked in autism and neurodevelopmental disorders clinic, followed by 19% endocrinologists, with the rest being divided among dermatology, pediatric specialty, physical medicine and rehabilitation, psychiatry, and rheumatology. Providers had very strong confidence levels in the ease of use of telehealth equipment, with a mean of 4.29. They also reported a good quality of images and sounds (mean, 4.00). None of the providers reported preference of the telehealth visits over in-person visits, with a mean of 2.0. Providers tend to have neutral opinions toward the question if telehealth
Table 1. Participants
Stakeholders Patients Providers Coordinators
Invited
Responded
Response Rate
830 95 75
286 21 12
34.5% 22.1% 16%
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
344
THE HEALTH CARE MANAGER/OCTOBER-DECEMBER 2015
Table 2. Patient Survey n (%) Risks and benefits I felt good about my telehealth doctor’s medical skill (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) Lack of physical contact was fine with me (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) I was fine with being on camera (n = 282) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) I felt that things I shared were kept private (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) Effectiveness My telehealth visit was as good as one that is in person (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) It was easy for me to state my concerns and ask questions through telehealth (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) My doctor took the time needed to answer my questions (n = 281) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) Efficiency I had to wait less time for the telehealth visit than I do for one that is in person (n = 278) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank)
Mean 4.28
135 103 32 5 5 6
(47.2) (36.0) (11.2) (1.7) (1.8) (2.1)
75 111 50 26 18 6
(26.2) (38.8) (17.5) (9.1) (6.3) (2.1)
78 125 36 26 17 4
(27.3) (43.7) (12.5) (9.1) (5.9) (1.4)
108 127 29 9 7 6
(37.8) (44.4) (10.1) (3.2) (2.4) (2.1)
92 101 42 27 18 6
(32.2) (35.3) 3.79 (14.7) (9.4) (6.3) (2.1) 3.91 (32.5) (41.3) (11.5) (7.7) (4.9) (2.1) 4.25 (43.7) (42.3) (7) (3.1) (2.1) (1.7)
3.71
3.78
4.14
93 118 33 22 14 6 125 121 20 9 6 5
3.71 75 96 70 25 12 8
(26.2) (33.6) (24.5) (8.7) (4.2) (2.8) (continues)
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
User Satisfaction With Telehealth
345
Table 2. Patient Survey, Continued n (%) The staff at my clinic knew how to handle the equipment (n = 278) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) I like the way the staff explained how telehealth works (n = 278) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) Future adoption The use of telehealth saved me travel time and money (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) I would use telehealth to have my doctor see me again (n = 278) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank) I will tell people I know about telehealth (n = 280) Strongly agree Agree Neutral Disagree Strongly disagree No response (blank)
would allow them to see more patients (43%). Fifty-seven percent of the providers agreed that they are able to treat patients via telehealth, whereas 62% agreed that they are satisfied with the work done through telehealth, with a mean of 4.05. Table 3 shows the patients’ responses to the questionnaire. Coordinators survey Twelve telehealth coordinators completed the survey. Thirty-three percent worked at a critical access hospital, 33% indicated ‘‘other’’ as a place of employment, whereas the reminder of the respondents were employed at rural clinics, federally qualified health care centers, and an academic health center. Of the
Mean 4.35
122 (42.7) 132 (46.2) 24 (8.4) 0 0 8 (2.8) 4.17 107 125 35 10 1 8
(37.4) (43.7) (12.2) (3.5) (0.4) (2.8)
112 83 51 24 10 6
(39.2) (29.0) (17.8) (8.4) (3.5) (2.1)
110 113 41 5 9 8
(38.5) (39.5) (14.3) (1.7) (3.2) (2.8)
94 106 45 20 15 6
(32.8) (37.1) (15.7) (7) (5.2) (2.1)
3.93
4.11
3.87
12 respondents, 58% use telehealth once a week, and 25% use telehealth once a month. Coordinators reported high confidence levels with the ease of use and setup of telehealth appointments, with a mean of 4.36. Similarly, 67% of the respondents indicated it was easy to contact and reach MTN when needed. Ninety-one percent of the respondents felt they were trained well on how to use the telehealth equipment and that they are satisfied with the work they do over telehealth, with a mean of 4.42. Table 4 shows the coordinators’ responses to the survey. DISCUSSION This study examined satisfaction with telehealth services provided on the MTN. Data
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
346
THE HEALTH CARE MANAGER/OCTOBER-DECEMBER 2015
Table 3. Provider Survey
It is easy to run and use a telehealth system (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree I am confident and feel at ease when I use the telehealth system (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth gives me the chance to build and keep a personal bond with each of my patients (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth fits well with each day’s workflow (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree The images and sounds of telehealth gear are crisp and clear (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree I get more done in my day when I see patients through telehealth (n = 20) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth helps me to converse with my patients (n = 20) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth allows me to see more patients (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree I am able to treat my patients’ need well through telehealth (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree I prefer telehealth visits over visits that are in person (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree For the most, I am satisfied with the work I’ve done through telehealth (n = 21) Strongly agree Agree Neutral Disagree Strongly disagree
n (%)
Mean
7 (33.3) 13 (61.9) 1 (4.8) 0 (0.0) 0 (0.0)
4.29
10 (47.6) 9 (42.9) 0 (0.0) 2 (9.5) 0 (0.0)
4.29
6 (28.6) 9 (42.9) 3 (14.3) 3 (14.3) 0 (0.0)
3.86
6 (28.6) 7 (33.3) 4 (19) 2 (9.5) 2 (9.5)
3.62
6 (28.6) 11 (52.4) 2 (9.5) 2 (9.5) 0 (0.0)
4.00
1 (5) 3 (15) 11 (55) 5 (25) 0 (0.0)
3.00
3 (15) 11 (55) 5 (25) 1 (5) 0 (0.0) 1 (4.8) 4 (19) 9 (42.9) 6 (28.6) 1 (4.8)
2.90
4 (19) 12 (57.1) 2 (9.5) 3 (14.3) 0 (0.0)
3.81
0 (0.0) 0 (0.0) 8 (38.1) 5 (23.8) 8 (38.1)
2.00
5 (23.8) 13 (61.9) 2 (9.5) 1 (4.8) 0 (0.0)
4.05
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
3.80
347
User Satisfaction With Telehealth Table 4. Coordinator Survey n (%) It is easy to reach MTN (Missouri Telehealth Network) agents and they respond when we need them. . . (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree It is easy to set up a telehealth appointment (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree It is easy to set up and use telehealth system (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth is a useful tool for us when we see patients (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth fits well with everyday’s workflow (n = 11) Strongly agree Agree Neutral Disagree Strongly disagree I have been trained to well to use telehealth (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree The images and sounds of the telehealth gear are clear and crisp (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree Telehealth records are easy to use and work with (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree The time I give to telehealth is used in an effective way (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree For the most part, I am satisfied with the work I’ve done through telehealth (n = 12) Strongly agree Agree Neutral Disagree
Mean 4.33
8 0 4 0 0
(66.7) (0.0) (33.3) (0.0) (0.0)
4 4 4 0 0
(33.3) (33.3) (33.3) (0.0) (0.0)
5 5 1 0 0
(45.5) (45.5) (9.1) (0.0) (0.0)
5 5 2 0 0
(41.7) (41.7) (16.7) (0.0) (0.0)
4 5 2 0 0
(36.4) (45.5) (18.2) (0.0) (0.0)
4.0
4.36
4.25
4.18
4.42 6 (50) 5 (41.7) 1 (8.3) 0 (0.0) 0 (0.0) 4.25 4 7 1 0 0
(33.3) (58.3) (8.3) (0.0) (0.0) 3.83
3 (25) 4 (33.3) 5 (41.7) 0 (0.0) 0 (0.0) 4.25 4 (33.3) 7 (58.3) 1 (8.3) 0 (0.0) 0 (0.0) 4.42 6 (50) 5 (41.7) 1 (8.3) 0 (0.0) (continues)
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
348
THE HEALTH CARE MANAGER/OCTOBER-DECEMBER 2015
Table 4. Coordinator Survey, Continued Telehealth records are easy to use and work with (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree The time I give to telehealth is used in an effective way (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree For the most part, I am satisfied with the work I’ve done through telehealth (n = 12) Strongly agree Agree Neutral Disagree Strongly disagree
were collected from the patients, telehealth coordinators, and providers. Patient and employee satisfaction scores have been used by hospitals across country not only to evaluate and improve clinical care, but also to help plan the growth of services. In this case, findings indicated high satisfaction by all 3 surveyed groups, which is consistent with many other telehealth satisfaction studies. The MTN holds a biannual training conference for telehealth coordinators, which focuses on hands-on equipment training, legal and regulatory issues, and telehealth reimbursement. This study also aimed at finding out how the training is perceived and whether the MTN staff were easily reachable in case of technical or scheduling difficulties by providers and telehealth coordinators. The study found that all of the telehealth coordinators who responded to the survey felt that the MTN staff was easy to reach and responsive, and 67% of them responded that the telehealth appointments are easy to set up. The majority of the providers (76%) felt that they can treat their patients well via telehealth and that they are satisfied with the work they have done via telehealth (86%). However, they still do not prefer telehealth over inperson visits (62%). Physicians are trained to provide direct care to patients, and although
3.83 3 (25) 4 (33.3) 5 (41.7) 0 (0.0) 0 (0.0) 4.25 4 (33.3) 7 (58.3) 1 (8.3) 0 (0.0) 0 (0.0) 4.42 6 (50) 5 (41.7) 1 (8.3) 0 (0.0) 0 (0.0)
doing so via video may be an adequate option, it will never replace a hands-on examination. The study had several limitations. Providers that offer telehealth are located and affiliated with different health care organizations; however, the only ones that responded to the survey were at an academic medical center. On the other hand, the majority of the patients who participated in this study were behavioral health patients, with only a few who used telehealth to see other specialists. Because of time and other limitations, the study did not include in-person visits from patients from the same counties to compare the telehealth visits to those that make the trip to see their providers. This quality improvement study was instrumental in that it allowed the MTN and the University of Missouri to evaluate the perceptions and opinions of telehealth users. It also confirmed and validated the importance of a comprehensive training and a whole team approach to the success of the overall acceptance of telehealth. With such high satisfaction, as well as the addition of other specialties offering telehealth services during the past decade, it was not surprising to see a corresponding growth of patients using the servies. However, to understand the full impact on telehealth, it is proposed to expand this study to include a
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
User Satisfaction With Telehealth comparative study of telehealth patients and in-person patients in each specialty. It is also proposed to survey providers from different types of health care organizations.
349
ACKNOWLEDGMENT The authors thank the staff of the MTN for their help and support during the study period.
REFERENCES 1. Whitten P, Love B. Patient and provider satisfaction with the use of telemedicine: overview and rationale for cautious enthusiasm. J Postgrad Med. 2005;51(4): 294-300. 2. Wittson CL, Benschoter R. Two way television: helping the medical center reach out. Am J Psychiatry. 1972;129(5):624-627. 3. Breen GM, Matusitz J. An evolutionary examination of telemedicine: a health and computer-mediated communication perspective. Soc Work Public Health. 2010; 25(1):59-71. 4. Ho K, Karlinsky H, Jarvis-Selinger S, May J. Videoconferencing for telehealth: unexpected challenges and unprecedented opportunities. BCMJ. 2004;46(6): 285-289. 5. Thomas CL, Man MS, O’Cathain A, et al. Effectiveness and cost-effectiveness of a telehealth intervention to support the management of long-term conditions: study protocol for two linked randomized trials. Trials. 2014;15:36. 6. Wade VA, Karnon J, Elshaug AG, Hiller JE. A systematic review of economic analyses of telehealth services using real time video communication. BMC Health Serv Res. 2010;10:233. 7. Moeckli J, Cram P, Cunningham C, Reisinger HS. Staff acceptance of a telemedicine intensive care unit program: a qualitative study. J Crit Care. 2013;28(6): 890-901. 8. Avey JP, Hobbs RL. Dial in: fostering the use of
9.
10.
11.
12.
13.
14.
15.
telebehavioral healt hservices in frontier Alaska. Psychol Serv. 2013;10(3):289-297. Thaker DA, Monypenny R, Olver I, Sabesan S. Cost savings from a telemedicine model of care in northern Queensland, Australia. Med J Aust. 2013;199(6):414-417. Kumar G, Falk DM, Bonello RS, Kahn JM, Perencevich E, Cram P. The cost of critical care telemedicine programs: a systematic review and analysis. Chest. 2013; 143(1):19-29. Missouri Telehealth Network Telehealth Sites. Missouri Telehealth Network [online]. 2014. Cited January 22, 2015. http://medicine.missouri.edu/telehealth/sites.html. Accessed January 22, 2015. School of Medicine University of Missouri Health System [online]. Cited January 22, 2015. http://medicine. missouri.edu/. Accessed January 22, 2015. Missouri Telehealth Network General Information. Missouri Telehealth Network [online]. Cited January 23, 2015. http://medicine.missouri.edu/telehealth/general.html. Accessed January 22, 2015. Missouri Telehealth Network Educational Programs. Missouri Telehealth Network [online]. Cited January 22, 2015. http://medicine.missouri.edu/telehealth/educationalprograms.html. Accessed January 22, 2015. REDCap (Research Electronic Data Capture). University of Missouri Institute for Clinical and Translational Science [online]. December 30, 2013. Cited January 22, 2015. http://icats.missouri.edu/portal/redcap.php. Accessed January 22, 2015.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
