The Journal of Emergency Medicine, Vol. 46, No. 3, pp. 404–409, 2014 Copyright Ó 2014 Elsevier Inc. Printed in the USA. All rights reserved 0736-4679/$ - see front matter
http://dx.doi.org/10.1016/j.jemermed.2013.08.077
Administration of Emergency Medicine
WEB-BASED EMERGENCY DEPARTMENT PATIENT SATISFACTION SURVEYS MAY INTRODUCE POTENTIAL FOR BIAS Camille Broadwater-Hollifield, MPH, James Fair, MD, Susan Podolsky, MD, Jessica Carey, BS, Kajsa Vlasic, Robert Stephen, MD, Troy Madsen, MD, and Michael Mallin, MD Division of Emergency Medicine, University of Utah, Salt Lake City, Utah Reprint Address: Michael Mallin, MD, University of Utah, 30N 1900E Rm 1C26, Salt Lake City, UT 84132
, Abstract—Background: Emergency departments (ED) have proposed utilizing a Web-based format to distribute patient satisfaction surveys, but the potential for bias in this distribution method has not been assessed. Objective: The aim of this study was to evaluate the characteristics of ED patients who have access to the Internet to better understand potential bias in Web-based patient satisfaction surveys. Methods: We distributed a 20-question survey to consenting, English-speaking adult patients presenting to the ED from December 2010 to March 2012. Patients reported demographic information and answered questions related to their access and use of the Internet. Results: Seven hundred four patients participated in the study; 90% of Whites reported Internet access, vs. 82% of Hispanics (p = 0.034). Ninetytwo percent of patients with at least some college education had Internet access, compared to 79% of those with a high school education level or lower (p # 0.001). Of households reporting an income of > $22,000/year, 95% had Internet access, compared to 77% of those reporting a household income < $22,000/year (p # 0.001). Ninety-four percent of participants < 40 years of age had Internet access, compared to 83% between the ages of 40 and 56 years, and 77% for those over 56 years of age (p < 0.001). Conclusion: A Web-based distribution of ED patient satisfaction surveys may underrepresent minorities, patients without college education, those with lower income, and patients older than 40 years. This information may provide guidance in interpreting results of
Web-based patient satisfaction surveys and may suggest the need for multiple sampling methods. Ó 2014 Elsevier Inc. , Keywords—patient satisfaction; Internet access; patient demographics; survey bias
INTRODUCTION Patient satisfaction surveys have become an important emergency department (ED) quality measure (1–3). A patient’s perception of the care they receive in the ED may not only affect their likelihood to return to that ED, but may also influence their perceptions of that hospital system as a whole (3–5). Satisfied patients are more likely to follow their discharge instructions, follow up with recommended care, utilize referrals, and remain in a coordinated system of care (5–7). ED and hospital administrators may utilize patient satisfaction surveys to better understand methods of improving the patient experience, such as decreasing perceived wait times, keeping the patient informed about the plan of care, and improving physician and staff interpersonal skills (2,5–8). Patient satisfaction surveys have gained a new importance with the passing of the Patient Protection and Affordable Care Act. The new law will reimburse hospitals based, in part, on reported quality of care, using patient satisfaction as one of the primary metrics (9). Due to the impact of patient satisfaction surveys on
This work was presented as a poster at the American College of Emergency Physicians Research Forum, Denver, CO, October 8, 2012.
RECEIVED: 14 March 2013; FINAL SUBMISSION RECEIVED: 1 July 2013; ACCEPTED: 15 August 2013 404
Possible Bias in ED Patient Satisfaction Surveys
reimbursement and quality measures, it follows that the information collected by these surveys should be as accurate as possible. The distribution and collection of patient satisfaction surveys may be cumbersome, with mail-in methods often yielding low return rates. For this reason, many EDs, including the University of Utah, have opted to utilize an Internet-based satisfaction survey method. To better understand how this format might affect the results of the surveys, we conducted a study to identify the demographics of patients presenting to the ED and their reported Internet access. We hypothesized that minorities, patients with less education, individuals with a lower reported income, and older patients would be less likely to report Internet access and thus, less likely to be represented in ED patient satisfaction results. MATERIALS AND METHODS We surveyed a convenience sample of patients presenting to the University of Utah ED between December 2010 and March 2012. The University of Utah institutional review board approved the study prior to subject enrollment. Inclusion criteria for the study were as follows: English speaking, $ 18 years of age, clinically stable, and deemed mentally competent. Exclusion criteria included any individual that was sent to the ED by a primary care physician or was brought in by emergency medical services or law enforcement. Trained research associates were present in the ED 18 h per day, 7 days per week. Research associates approached patients during their ED stay and asked if they would be willing to participate in the study. The survey was designed at a 7th grade reading level and consisted of 20 behavioral, attitudinal, and demographic questions distributed to all consenting patients. For demographic information, respondents provided details about their age, gender, race, Hispanic (yes/no), years of education, income, and household size. Internet access was assessed with a dichotomous variable of yes/no ‘‘Do you have access to the Internet?’’, with additional questions pertaining to the location of Internet access (home, work, phone, public library, other). To measure behaviors about Internet use and attitudes about reliability, patients were asked, ‘‘Have you ever used the Internet to look for information about your health care?’’, and an ordered Likert scale question: ‘‘Do you trust the information you find on the Internet?’’ These questions were included to ascertain patients’ perceptions of Internet-based information reliability and use of the Internet in health care decision-making. See the Appendix for the full survey. We evaluated results utilizing descriptive statistics and chi-squared test statistic or Fisher’s exact test statistic for
405
categorical variables. The alpha level of significance for all statistical tests was # 0.05. A p-value of # 0.05 was considered statistically significant. Study data were collected and managed using REDCap (Research Electronic Data Capture) electronic data capture tools hosted at the University of Utah. REDCap is a secure, Webbased application designed to support data capture for research studies. Data were exported from REDCap to Stata/IC 12.1 for analysis (StataCorp, 2011; Stata Statistical Software: Release 12; College Station, TX: StataCorp LP). RESULTS Over the 16-month study period, 704 ED patients agreed to participate in the study. The median age was 43.6 years, 58% of participants were female, and 82% of patients self-identified their race as Caucasian. Eighty-seven percent of participants reported that they have some means of regularly accessing the Internet (Table 1). Hispanic patients represented the predominant minority ethnic or racial group in our ED surveys, and we compared Internet access among Caucasian and Hispanic patients. Ninety percent of patients who self-identified their race as Caucasian reported Internet access, vs. 82% of individuals who self-identified as Hispanic (p = 0.034). For the 7% of the study population that selfidentified as a minority race, there was a significant difference (p # 0.001) in Internet access, with only 88% of African Americans, 83% of Pacific Islanders, and 46% of Native Americans/Alaska Natives (46%) reporting having Internet access (Figure 1). Similarly, we noted significant disparities when evaluating Internet access by patient age. Of patients < 40 years of age, 94% had access to the Internet, whereas of individuals between the ages of 40 and 56 years, 83% had access to the Internet, and 77% of those over age 56 years had access to the Internet (p # 0.001) (Figure 2). Finally, both education and income levels served as markers of disparity in Internet access. Of those reporting an education level including some college or higher, 92% had Internet access, and of those with a high school education level or lower, 79% had access to the Internet (p # 0.001). Similarly, 95% of those reporting an income of > $22,000/year had Internet access, whereas 77% of those with a household income < $22,000/year reported access to the Internet (p # 0.001) (Figure 1). The most common sites from which patients accessed the Internet were home (32.4%) or their telephone (22.6%). A significant number of patients reported their Internet access in public or semi-public settings, with 15.1% of patients reporting their access through work and 9.5% of patients accessing the Internet at a public library (Table 1).
406
C. Broadwater-Hollifield et al.
Table 1. Patient Demographics No. Age, years Gender Female Male Race Caucasian African-American Asian/Pacific Islander American Indian/Alaska Native Other Missing Hispanic Yes No Missing Education Some high school High school degree Some college College degree Graduate, professional, doctorate Missing Income $0–10,999 $11,000–21,999 $22,000–32,999 $33,000–49,999 $50,000–74,999 $75,000–99,999 $100,000+ Missing Household size 1 2 3 4+ Missing Do you have Internet access Yes No Where do you access the Internet? Home Work My phone Public library Other Has Internet access (by race*) Caucasian African-American Asian/Pacific Islander American Indian/Alaska Native Other
(%)
43.64 (28.99–56.15 IQR) 407 297
57.81% 42.19%
580 17 23 14 46 24
82.39% 2.41% 3.27% 1.99% 6.53% 3.41%
70 610 24
9.94% 86.65% 3.41%
75 165 216 155 86 7
10.65% 23.44% 30.68% 22.02% 12.22% 0.99%
167 107 82 82 80 56 74 56
23.72% 15.20% 11.65% 11.65% 11.36% 7.95% 10.51% 7.95%
136 219 126 209 14
19.32% 31.11% 17.90% 29.69% 1.99%
614 89
87.34% 12.66%
228 106 159 67 48
37.50% 17.43% 26.15% 11.02% 7.89%
525 15 19 6 33
90.36% 88.24% 82.61% 46.15% 70.21%
Figure 1. Internet access by ethnicity, education, and income.
obtained. Although the use of Internet-based survey tools may eliminate postage costs and improve response rates, this survey method may also exclude the opinions of those without access to the Internet and may skew the actions of those making changes to care delivery due to misinformation. Our results demonstrate significant disparities in Internet access when evaluating access by age, race/ ethnicity, education, and income. Presumably, this disparity in Internet access would also be reflected in responses to patient satisfaction surveys administered via the Internet. Our review of the literature on this topic produced only one similar research study by Pourmand and Sikka (10). By surveying 489 urban ED patients in Washington, DC, they found similar rates of Internet access; 92.6% amongst all demographics. This result is similar to the Internet access reported by Caucasians and those of higher socioeconomic status in Utah. However, the Washington, DC study did not report Internet access as a function of demographics. Therefore, our study represents a novel dataset that can help us further understand the discrepancy of Internet access as related to demographic factors such as socioeconomic status, race, and education. These variables highlight the potential for bias intrinsic to Internet-based surveys. Implementing exclusively Internet-based patient satisfaction surveys may bias the measures, results, and
IQR = interquartile range.
DISCUSSION Increasingly, patient satisfaction surveys are used as a quality measure and as a foundation to improve the patient experience in the ED, and patient satisfaction scores will soon affect hospital reimbursement. The prominence of patient satisfaction surveys in hospital quality metrics and reimbursement necessitates accuracy and full representation in the sampling methods employed and results
Figure 2. Internet access by age.
Possible Bias in ED Patient Satisfaction Surveys
conclusions of the survey with the exclusion of groups with lower socioeconomic status. Introducing bias to the survey may result in the loss of opportunities for improvements that serve these specific groups. Although we did not specifically evaluate methods to eliminate bias in survey methods, EDs may wish to employ multiple sampling strategies or provide patients the option of completing their patient satisfaction survey on a computer at the time of ED discharge. Limitations Our study was limited to a convenience sample of individuals who were not acutely ill, were not sent to the ED by their primary care physician, were deemed mentally competent by the attending physician, and presented to a single academic ED. Additionally, the survey was conducted only in English. We elected to include only English-speaking patients, given that the patient satisfaction surveys are only distributed in English at our institution. We presume that the inclusion of patients who speak only Spanish would demonstrate further disparity in survey access for those who self-identify as Hispanic. Additionally, we did not evaluate patients’ likelihood to respond to patient satisfaction surveys and utilized only their access to the Internet as a surrogate for potential sampling bias. Finally, our study relies on self-reporting of race, education, and income, and did not employ objective measures in the assessment of these variables. CONCLUSION In conclusion, a Web-based format for the distribution of patient satisfaction surveys in the ED may underrepresent minorities, patients without college education, those with lower income, and patients older than 40 years of age. This information may provide guidance in interpreting results of Web-based patient satisfaction surveys and may suggest the need for multiple sampling methods. REFERENCES 1. Hansagi H, Carlsson B, Brismar B. The urgency of care need and patient satisfaction at a hospital emergency department. Health Care Manage Rev 1992;17:71–5. 2. Matulich E, Fin DW. Determinant criteria in patient satisfaction surveys. J Ambul Care Manage 1989;12:45–51. 3. Trout A, Magnusson AR, Hedges JR. Patient satisfaction investigations and the emergency department: what does the literature say? Acad Emerg Med 2000;7:695–709. 4. Hostutler JJ, Taft SH, Snyder C. Patient needs in the emergency department. Nurses’ and patients’ perceptions. J Nurs Admin 1999;29:43–50. 5. Rydman RJ, Roberts RR, Albrecht GL, Zalenski RJ, McDermott M. Patient satisfaction with an emergency department asthma observation unit. Acad Emerg Med 1999;6:178–83.
407 6. Bjorvell H, Stieg J. Patients’ perceptions of the health care received in an emergency department. Ann Emerg Med 1991;20:734–8. 7. Waggoner DM, Jackson EB, Kern DE. Physician influence on patient compliance: a clinical trial. Ann Emerg Med 1981;10:348–52. 8. Boudreaux ED, O’Hea EL. Patient satisfaction in the Emergency Department: a review of the literature and implications for practice. J Emerg Med 2004;26:13–26. 9. U.S. Department of Health and Human Services. Administration implements Affordable Care Act provision to improve care, lower costs. Available at: http://www.hhs.gov/news/press/2011pres/04/ 20110429a.html. February 2, 2013. 10. Pourmand A, Sikka N. Online health information impacts patients’ decisions to seek emergency department care. West J Emerg Med 2011;12:174–7.
APPENDIX The Influence of Internet Medical Information on Emergency Department Visits Instructions: Please complete the following survey. If you have any questions, please ask the researcher who gave you this survey. Thank you for your time! Demographics. Name:__________________Phone number:_________ _______ Race (check as many as apply):
Annual Household Income: , Caucasian , $0 – 10,999 , African-American , $11,000 – 21,999 , Asian/Pacific Islander , $22,000 – 32,999 , American Indian/Alaskan Native , $33,000 – 49,999 , Other: ____________ , $50,000 – 74,999 Are you Hispanic/Latino? , $75,000 – 99,999 , Yes , $100,000+ , No How many people are in your household? What is your highest level of ,1 education? , Some high school ,2 , High school degree ,3 , Some college , 4+ , College degree , Graduate, professional, or doctoral
Internet Utilization Questions. 1. How do you typically obtain your health information? (books, internet, TV, family, be very specific) ______________________________________ ________________________________________ ________________________________________ ________________________________________ ________________________________________ ________________________________________ ____________ 2. Do you have access to the internet? , Yes , No—Please skip to question #14
408
3. How often do you access the internet? , Once a month , Once a week , Several times a week , Every day , Several times a day 4. Where do you access the internet? (Check all that apply.) , Home , Work , My phone , Public library , Other_________ 5. Have you ever used the internet to look for information about your health care? , Yes—Please continue to question #5 , No—Please skip to question #13 6. How often do you look for health care information on the internet? , Once a year , Several times a year , Once a month , Several times a month , Once a week , Several times a week , Every day 7. When did you most recently look for health care information on the internet? , In the last 48 hours , In the last week , In the last month , More than one month ago 8. When you last looked for health care information on the internet, were you researching the same health problem that brought you to the Emergency Department today? , Yes—Please continue to question #8 , No—Please continue to question #11 9. How much did your internet reading influence your decision to come to the Emergency Department today? , It was the main reason I decided to come , It made me more likely to come , It made me less likely to come , It had no influence at all
C. Broadwater-Hollifield et al.
10. Did the information you received about your health in the Emergency Department today agree with the information you found online? , Completely agreed—the information from the Emergency Department and the information from the internet were the same , Somewhat agreed—the information from the Emergency Department agreed somewhat with the information from the internet , Somewhat disagreed—the information from the Emergency Department disagreed somewhat with the information from the internet , Completely disagreed—the information from the Emergency Department was completely different from what I read online 11. Did you tell your Emergency Department health care provider about your internet research? , Yes , No 12. What websites do you use most often to look for health care information? a. Website:_______________________________ b. Website:_______________________________ c. Website:_______________________________ 13. How much do you trust the health care information you receive from the internet? , Completely trust , Somewhat trust , Neither trust nor , Somewhat distrust , Completely distrust 14. How much do you trust the health care information you received today from your doctor in the Emergency Department? , Completely trust , Somewhat trust , Neither trust nor distrust , Somewhat distrust , Completely distrust 15. Now that you have been seen by a health care provider, do you think you needed to come to the Emergency Department today? , Yes , No Thank you for completing this survey!
Possible Bias in ED Patient Satisfaction Surveys
ARTICLE SUMMARY 1. Why is this topic important? Many emergency departments (EDs) are utilizing a Web-based format for patient satisfaction surveys without adequate information regarding their patient population’s access to the Internet. 2. What does this study attempt to show? This study attempts to evaluate the characteristics of ED patients who have access to the Internet to better understand potential bias in Web-based patient satisfaction surveys. 3. What are the key findings? There are significant disparities in Internet access when evaluating access by age, race/ethnicity, education, and income. 4. How is patient care impacted? A Web-based format for the distribution of patient satisfaction surveys in the ED may underrepresent minorities, patients without college education, those with lower income, and patients older than 40 years of age.
409
http://dx.doi.org/10.1016/j.jemermed.2013.08.077
Administration of Emergency Medicine
WEB-BASED EMERGENCY DEPARTMENT PATIENT SATISFACTION SURVEYS MAY INTRODUCE POTENTIAL FOR BIAS Camille Broadwater-Hollifield, MPH, James Fair, MD, Susan Podolsky, MD, Jessica Carey, BS, Kajsa Vlasic, Robert Stephen, MD, Troy Madsen, MD, and Michael Mallin, MD Division of Emergency Medicine, University of Utah, Salt Lake City, Utah Reprint Address: Michael Mallin, MD, University of Utah, 30N 1900E Rm 1C26, Salt Lake City, UT 84132
, Abstract—Background: Emergency departments (ED) have proposed utilizing a Web-based format to distribute patient satisfaction surveys, but the potential for bias in this distribution method has not been assessed. Objective: The aim of this study was to evaluate the characteristics of ED patients who have access to the Internet to better understand potential bias in Web-based patient satisfaction surveys. Methods: We distributed a 20-question survey to consenting, English-speaking adult patients presenting to the ED from December 2010 to March 2012. Patients reported demographic information and answered questions related to their access and use of the Internet. Results: Seven hundred four patients participated in the study; 90% of Whites reported Internet access, vs. 82% of Hispanics (p = 0.034). Ninetytwo percent of patients with at least some college education had Internet access, compared to 79% of those with a high school education level or lower (p # 0.001). Of households reporting an income of > $22,000/year, 95% had Internet access, compared to 77% of those reporting a household income < $22,000/year (p # 0.001). Ninety-four percent of participants < 40 years of age had Internet access, compared to 83% between the ages of 40 and 56 years, and 77% for those over 56 years of age (p < 0.001). Conclusion: A Web-based distribution of ED patient satisfaction surveys may underrepresent minorities, patients without college education, those with lower income, and patients older than 40 years. This information may provide guidance in interpreting results of
Web-based patient satisfaction surveys and may suggest the need for multiple sampling methods. Ó 2014 Elsevier Inc. , Keywords—patient satisfaction; Internet access; patient demographics; survey bias
INTRODUCTION Patient satisfaction surveys have become an important emergency department (ED) quality measure (1–3). A patient’s perception of the care they receive in the ED may not only affect their likelihood to return to that ED, but may also influence their perceptions of that hospital system as a whole (3–5). Satisfied patients are more likely to follow their discharge instructions, follow up with recommended care, utilize referrals, and remain in a coordinated system of care (5–7). ED and hospital administrators may utilize patient satisfaction surveys to better understand methods of improving the patient experience, such as decreasing perceived wait times, keeping the patient informed about the plan of care, and improving physician and staff interpersonal skills (2,5–8). Patient satisfaction surveys have gained a new importance with the passing of the Patient Protection and Affordable Care Act. The new law will reimburse hospitals based, in part, on reported quality of care, using patient satisfaction as one of the primary metrics (9). Due to the impact of patient satisfaction surveys on
This work was presented as a poster at the American College of Emergency Physicians Research Forum, Denver, CO, October 8, 2012.
RECEIVED: 14 March 2013; FINAL SUBMISSION RECEIVED: 1 July 2013; ACCEPTED: 15 August 2013 404
Possible Bias in ED Patient Satisfaction Surveys
reimbursement and quality measures, it follows that the information collected by these surveys should be as accurate as possible. The distribution and collection of patient satisfaction surveys may be cumbersome, with mail-in methods often yielding low return rates. For this reason, many EDs, including the University of Utah, have opted to utilize an Internet-based satisfaction survey method. To better understand how this format might affect the results of the surveys, we conducted a study to identify the demographics of patients presenting to the ED and their reported Internet access. We hypothesized that minorities, patients with less education, individuals with a lower reported income, and older patients would be less likely to report Internet access and thus, less likely to be represented in ED patient satisfaction results. MATERIALS AND METHODS We surveyed a convenience sample of patients presenting to the University of Utah ED between December 2010 and March 2012. The University of Utah institutional review board approved the study prior to subject enrollment. Inclusion criteria for the study were as follows: English speaking, $ 18 years of age, clinically stable, and deemed mentally competent. Exclusion criteria included any individual that was sent to the ED by a primary care physician or was brought in by emergency medical services or law enforcement. Trained research associates were present in the ED 18 h per day, 7 days per week. Research associates approached patients during their ED stay and asked if they would be willing to participate in the study. The survey was designed at a 7th grade reading level and consisted of 20 behavioral, attitudinal, and demographic questions distributed to all consenting patients. For demographic information, respondents provided details about their age, gender, race, Hispanic (yes/no), years of education, income, and household size. Internet access was assessed with a dichotomous variable of yes/no ‘‘Do you have access to the Internet?’’, with additional questions pertaining to the location of Internet access (home, work, phone, public library, other). To measure behaviors about Internet use and attitudes about reliability, patients were asked, ‘‘Have you ever used the Internet to look for information about your health care?’’, and an ordered Likert scale question: ‘‘Do you trust the information you find on the Internet?’’ These questions were included to ascertain patients’ perceptions of Internet-based information reliability and use of the Internet in health care decision-making. See the Appendix for the full survey. We evaluated results utilizing descriptive statistics and chi-squared test statistic or Fisher’s exact test statistic for
405
categorical variables. The alpha level of significance for all statistical tests was # 0.05. A p-value of # 0.05 was considered statistically significant. Study data were collected and managed using REDCap (Research Electronic Data Capture) electronic data capture tools hosted at the University of Utah. REDCap is a secure, Webbased application designed to support data capture for research studies. Data were exported from REDCap to Stata/IC 12.1 for analysis (StataCorp, 2011; Stata Statistical Software: Release 12; College Station, TX: StataCorp LP). RESULTS Over the 16-month study period, 704 ED patients agreed to participate in the study. The median age was 43.6 years, 58% of participants were female, and 82% of patients self-identified their race as Caucasian. Eighty-seven percent of participants reported that they have some means of regularly accessing the Internet (Table 1). Hispanic patients represented the predominant minority ethnic or racial group in our ED surveys, and we compared Internet access among Caucasian and Hispanic patients. Ninety percent of patients who self-identified their race as Caucasian reported Internet access, vs. 82% of individuals who self-identified as Hispanic (p = 0.034). For the 7% of the study population that selfidentified as a minority race, there was a significant difference (p # 0.001) in Internet access, with only 88% of African Americans, 83% of Pacific Islanders, and 46% of Native Americans/Alaska Natives (46%) reporting having Internet access (Figure 1). Similarly, we noted significant disparities when evaluating Internet access by patient age. Of patients < 40 years of age, 94% had access to the Internet, whereas of individuals between the ages of 40 and 56 years, 83% had access to the Internet, and 77% of those over age 56 years had access to the Internet (p # 0.001) (Figure 2). Finally, both education and income levels served as markers of disparity in Internet access. Of those reporting an education level including some college or higher, 92% had Internet access, and of those with a high school education level or lower, 79% had access to the Internet (p # 0.001). Similarly, 95% of those reporting an income of > $22,000/year had Internet access, whereas 77% of those with a household income < $22,000/year reported access to the Internet (p # 0.001) (Figure 1). The most common sites from which patients accessed the Internet were home (32.4%) or their telephone (22.6%). A significant number of patients reported their Internet access in public or semi-public settings, with 15.1% of patients reporting their access through work and 9.5% of patients accessing the Internet at a public library (Table 1).
406
C. Broadwater-Hollifield et al.
Table 1. Patient Demographics No. Age, years Gender Female Male Race Caucasian African-American Asian/Pacific Islander American Indian/Alaska Native Other Missing Hispanic Yes No Missing Education Some high school High school degree Some college College degree Graduate, professional, doctorate Missing Income $0–10,999 $11,000–21,999 $22,000–32,999 $33,000–49,999 $50,000–74,999 $75,000–99,999 $100,000+ Missing Household size 1 2 3 4+ Missing Do you have Internet access Yes No Where do you access the Internet? Home Work My phone Public library Other Has Internet access (by race*) Caucasian African-American Asian/Pacific Islander American Indian/Alaska Native Other
(%)
43.64 (28.99–56.15 IQR) 407 297
57.81% 42.19%
580 17 23 14 46 24
82.39% 2.41% 3.27% 1.99% 6.53% 3.41%
70 610 24
9.94% 86.65% 3.41%
75 165 216 155 86 7
10.65% 23.44% 30.68% 22.02% 12.22% 0.99%
167 107 82 82 80 56 74 56
23.72% 15.20% 11.65% 11.65% 11.36% 7.95% 10.51% 7.95%
136 219 126 209 14
19.32% 31.11% 17.90% 29.69% 1.99%
614 89
87.34% 12.66%
228 106 159 67 48
37.50% 17.43% 26.15% 11.02% 7.89%
525 15 19 6 33
90.36% 88.24% 82.61% 46.15% 70.21%
Figure 1. Internet access by ethnicity, education, and income.
obtained. Although the use of Internet-based survey tools may eliminate postage costs and improve response rates, this survey method may also exclude the opinions of those without access to the Internet and may skew the actions of those making changes to care delivery due to misinformation. Our results demonstrate significant disparities in Internet access when evaluating access by age, race/ ethnicity, education, and income. Presumably, this disparity in Internet access would also be reflected in responses to patient satisfaction surveys administered via the Internet. Our review of the literature on this topic produced only one similar research study by Pourmand and Sikka (10). By surveying 489 urban ED patients in Washington, DC, they found similar rates of Internet access; 92.6% amongst all demographics. This result is similar to the Internet access reported by Caucasians and those of higher socioeconomic status in Utah. However, the Washington, DC study did not report Internet access as a function of demographics. Therefore, our study represents a novel dataset that can help us further understand the discrepancy of Internet access as related to demographic factors such as socioeconomic status, race, and education. These variables highlight the potential for bias intrinsic to Internet-based surveys. Implementing exclusively Internet-based patient satisfaction surveys may bias the measures, results, and
IQR = interquartile range.
DISCUSSION Increasingly, patient satisfaction surveys are used as a quality measure and as a foundation to improve the patient experience in the ED, and patient satisfaction scores will soon affect hospital reimbursement. The prominence of patient satisfaction surveys in hospital quality metrics and reimbursement necessitates accuracy and full representation in the sampling methods employed and results
Figure 2. Internet access by age.
Possible Bias in ED Patient Satisfaction Surveys
conclusions of the survey with the exclusion of groups with lower socioeconomic status. Introducing bias to the survey may result in the loss of opportunities for improvements that serve these specific groups. Although we did not specifically evaluate methods to eliminate bias in survey methods, EDs may wish to employ multiple sampling strategies or provide patients the option of completing their patient satisfaction survey on a computer at the time of ED discharge. Limitations Our study was limited to a convenience sample of individuals who were not acutely ill, were not sent to the ED by their primary care physician, were deemed mentally competent by the attending physician, and presented to a single academic ED. Additionally, the survey was conducted only in English. We elected to include only English-speaking patients, given that the patient satisfaction surveys are only distributed in English at our institution. We presume that the inclusion of patients who speak only Spanish would demonstrate further disparity in survey access for those who self-identify as Hispanic. Additionally, we did not evaluate patients’ likelihood to respond to patient satisfaction surveys and utilized only their access to the Internet as a surrogate for potential sampling bias. Finally, our study relies on self-reporting of race, education, and income, and did not employ objective measures in the assessment of these variables. CONCLUSION In conclusion, a Web-based format for the distribution of patient satisfaction surveys in the ED may underrepresent minorities, patients without college education, those with lower income, and patients older than 40 years of age. This information may provide guidance in interpreting results of Web-based patient satisfaction surveys and may suggest the need for multiple sampling methods. REFERENCES 1. Hansagi H, Carlsson B, Brismar B. The urgency of care need and patient satisfaction at a hospital emergency department. Health Care Manage Rev 1992;17:71–5. 2. Matulich E, Fin DW. Determinant criteria in patient satisfaction surveys. J Ambul Care Manage 1989;12:45–51. 3. Trout A, Magnusson AR, Hedges JR. Patient satisfaction investigations and the emergency department: what does the literature say? Acad Emerg Med 2000;7:695–709. 4. Hostutler JJ, Taft SH, Snyder C. Patient needs in the emergency department. Nurses’ and patients’ perceptions. J Nurs Admin 1999;29:43–50. 5. Rydman RJ, Roberts RR, Albrecht GL, Zalenski RJ, McDermott M. Patient satisfaction with an emergency department asthma observation unit. Acad Emerg Med 1999;6:178–83.
407 6. Bjorvell H, Stieg J. Patients’ perceptions of the health care received in an emergency department. Ann Emerg Med 1991;20:734–8. 7. Waggoner DM, Jackson EB, Kern DE. Physician influence on patient compliance: a clinical trial. Ann Emerg Med 1981;10:348–52. 8. Boudreaux ED, O’Hea EL. Patient satisfaction in the Emergency Department: a review of the literature and implications for practice. J Emerg Med 2004;26:13–26. 9. U.S. Department of Health and Human Services. Administration implements Affordable Care Act provision to improve care, lower costs. Available at: http://www.hhs.gov/news/press/2011pres/04/ 20110429a.html. February 2, 2013. 10. Pourmand A, Sikka N. Online health information impacts patients’ decisions to seek emergency department care. West J Emerg Med 2011;12:174–7.
APPENDIX The Influence of Internet Medical Information on Emergency Department Visits Instructions: Please complete the following survey. If you have any questions, please ask the researcher who gave you this survey. Thank you for your time! Demographics. Name:__________________Phone number:_________ _______ Race (check as many as apply):
Annual Household Income: , Caucasian , $0 – 10,999 , African-American , $11,000 – 21,999 , Asian/Pacific Islander , $22,000 – 32,999 , American Indian/Alaskan Native , $33,000 – 49,999 , Other: ____________ , $50,000 – 74,999 Are you Hispanic/Latino? , $75,000 – 99,999 , Yes , $100,000+ , No How many people are in your household? What is your highest level of ,1 education? , Some high school ,2 , High school degree ,3 , Some college , 4+ , College degree , Graduate, professional, or doctoral
Internet Utilization Questions. 1. How do you typically obtain your health information? (books, internet, TV, family, be very specific) ______________________________________ ________________________________________ ________________________________________ ________________________________________ ________________________________________ ________________________________________ ____________ 2. Do you have access to the internet? , Yes , No—Please skip to question #14
408
3. How often do you access the internet? , Once a month , Once a week , Several times a week , Every day , Several times a day 4. Where do you access the internet? (Check all that apply.) , Home , Work , My phone , Public library , Other_________ 5. Have you ever used the internet to look for information about your health care? , Yes—Please continue to question #5 , No—Please skip to question #13 6. How often do you look for health care information on the internet? , Once a year , Several times a year , Once a month , Several times a month , Once a week , Several times a week , Every day 7. When did you most recently look for health care information on the internet? , In the last 48 hours , In the last week , In the last month , More than one month ago 8. When you last looked for health care information on the internet, were you researching the same health problem that brought you to the Emergency Department today? , Yes—Please continue to question #8 , No—Please continue to question #11 9. How much did your internet reading influence your decision to come to the Emergency Department today? , It was the main reason I decided to come , It made me more likely to come , It made me less likely to come , It had no influence at all
C. Broadwater-Hollifield et al.
10. Did the information you received about your health in the Emergency Department today agree with the information you found online? , Completely agreed—the information from the Emergency Department and the information from the internet were the same , Somewhat agreed—the information from the Emergency Department agreed somewhat with the information from the internet , Somewhat disagreed—the information from the Emergency Department disagreed somewhat with the information from the internet , Completely disagreed—the information from the Emergency Department was completely different from what I read online 11. Did you tell your Emergency Department health care provider about your internet research? , Yes , No 12. What websites do you use most often to look for health care information? a. Website:_______________________________ b. Website:_______________________________ c. Website:_______________________________ 13. How much do you trust the health care information you receive from the internet? , Completely trust , Somewhat trust , Neither trust nor , Somewhat distrust , Completely distrust 14. How much do you trust the health care information you received today from your doctor in the Emergency Department? , Completely trust , Somewhat trust , Neither trust nor distrust , Somewhat distrust , Completely distrust 15. Now that you have been seen by a health care provider, do you think you needed to come to the Emergency Department today? , Yes , No Thank you for completing this survey!
Possible Bias in ED Patient Satisfaction Surveys
ARTICLE SUMMARY 1. Why is this topic important? Many emergency departments (EDs) are utilizing a Web-based format for patient satisfaction surveys without adequate information regarding their patient population’s access to the Internet. 2. What does this study attempt to show? This study attempts to evaluate the characteristics of ED patients who have access to the Internet to better understand potential bias in Web-based patient satisfaction surveys. 3. What are the key findings? There are significant disparities in Internet access when evaluating access by age, race/ethnicity, education, and income. 4. How is patient care impacted? A Web-based format for the distribution of patient satisfaction surveys in the ED may underrepresent minorities, patients without college education, those with lower income, and patients older than 40 years of age.
409
