Health Care Manag Sci DOI 10.1007/s10729-015-9329-z
The effect of information technology on hospital performance Cynthia Williams 1 & Yara Asi 2 & Amanda Raffenaud 2 & Matt Bagwell 2 & Ibrahim Zeini 2
Received: 23 December 2014 / Accepted: 19 May 2015 # Springer Science+Business Media New York 2015
Abstract While healthcare entities have integrated various forms of health information technology (HIT) into their systems due to claims of increased quality and decreased costs, as well as various incentives, there is little available information about which applications of HIT are actually the most beneficial and efficient. In this study, we aim to assist administrators in understanding the characteristics of top performing hospitals. We utilized data from the Health Information and Management Systems Society and the Center for Medicare and Medicaid to assess 1039 hospitals. Inputs considered were full time equivalents, hospital size, and technology inputs. Technology inputs included personal health records (PHR), electronic medical records (EMRs), computerized physician order entry systems (CPOEs), and electronic access to diagnostic results. Output variables were measures of quality, hospital readmission and mortality rate. The analysis was conducted in a two-stage methodology: Data Envelopment Analysis (DEA) and Automatic Interaction Detector Analysis (AID), decision tree regression (DTreg). Overall, we found that electronic access to diagnostic results systems was the most influential technological characteristics; however organizational characteristics were more important than technological inputs. Hospitals that had the highest levels of quality indicated no excess in the use of technology input, averaging one use of a technology component. This study indicates that prudent
* Cynthia Williams [email protected] 1
Department of Public Health, Brooks College of Health, University of North Florida, 1 UNF Drive, Jacksonville, FL 32224-2646, USA
2
College of Health & Public Affairs, University of Central Florida, Orlando, Fl, USA
consideration of organizational characteristics and technology is needed before investing in innovative programs. Keywords DEA . AID . Hospital . Health information technology . Efficiency . Quality
1 Introduction The health care industry is seeking to improve the quality of patient care in U.S. hospitals. To alleviate these difficulties, the U.S. health care system is undergoing significant change as different applications of health information technology (HIT) have captivated the medical community. While some attention is given to HIT and its contribution to patient quality of care, a paucity of literature examines the relationship between how much HIT is truly needed and what HIT applications are most effective in leading to quality outcomes. The benefits of HIT are likely to be underutilized without a sufficient understanding of its value. By examining hospitals that adopt technology, we aim to assist administrators in understanding the characteristics of the hospitals that demonstrate quality of care outcomes. Most health care commentators would agree that the U.S. health care system suffers from significant quality deficits; therefore as innovative methodologies enter into the health care industry, it is critical that we examine the efficaciousness of these tools to affect positive change. Such practices are able to demonstrate if resources should be devoted to these tools or if other innovations should be sought. When advanced information and communication technology is combined with the quality improvement process, it is effective in increasing the quality of patient care [1]. The decision to adopt new technology and consider its influence on quality of patient care is critical to a hospital’s viability [2]. However, the successful
C. Williams et al.
implementation and use of a HIT application is not solely related to financial gains. In health care, the benefits of technology may be intangible, such as access to care, quality of patient care, and the potential for clinical collaborations [2, 3]. While there are several types and uses of HIT, the literature has shown potential for the following HIT applications in particular to increase patient quality of care: personal health records (PHRs), electronic medical records (EMRs), computerized physician order entry systems (CPOEs), and electronic access to diagnostic results. Personal health records (PHRs) are Belectronic platform(s) that allows patients to manage [and view] their health information^ [4]. Physicians’ use of PHRs is postulated to increase clinician-patient communication, increase patient engagement, and produce better outcomes at lower costs [5, 6]. This fosters a shared decision-making relationship between physicians and patients. The benefits to quality are especially evident in patient groups that are traditionally underserved, such as those with chronic or mental illness and where complex comorbidities exist [7, 8]. Electronic medical records (EMRs) are touted as one of the Bmost promising components of health information technology^ [9]. Replacing paper records, EMRs are useful for information storage and retrieval, as well as increasing information exchange at the point of care [2]. Goodwin et al. (2013) suggest that EMR adoption demonstrates statistically significant decreases in hospital utilization. As EMRs become more readily integrated into different types of facilities and are coupled with trained on-site staff, increases in the quality of care and of the records themselves are evident [10–12]. Computerized physician order entry systems (CPOEs) focus on improving safety and accuracy in ordering medications, laboratory tests, and other medical procedures [13]. It gives providers the ability to order medications, tests, and procedures electronically, Bproviding significantly enhanced decision support capabilities as compared to traditional handwritten orders^ [14]. The literature supports the argument that CPOEs increase quality care and safety. It decreases medical errors and adverse events while improving the process of providing care [15–17]. The final application of HIT assessed in this study was electronic access to diagnostic results. Electronic access is beneficial for patients with conditions that have the potential to rapidly deteriorate because it decreases the time to diagnosis [18, 19]. It may improve the use of diagnostic imaging results in clinical practice [18]. Electronic access to diagnostic results may facilitate communication between providers and patients, resulting in a decrease in rehospitalizations and duplications of diagnostic procedures [20]. Timely access to diagnostic results may decrease patient anxiety and increase the decision-making capabilities of the patient care team [21]. Patience et al., (2015) suggest that timely reassurance is an important intervention strategy in medical consultation and
reduces subsequent needless investigations and clinical appointments. While total patient quality of care cannot be based solely on technology, we can speculate on its contribution to patient perception of quality and hospital indicators of patient quality [1]. A summary of each HIT applications’ link to patient quality of care outcomes can be found in Table 1.
2 Theoretical framework The Institute of Medicine (IOM) conceptualizes efficiency as a reduction of waste, decreasing unnecessary hospital utilizations, medication overuse, and duplication of services [22]. Efficiency refers to the combined sequence of services (inputs) in the most efficacious manner to produce the greatest increment of health (output) given specified expenditure [23]. However, efficiency optimization alone is not an adequate indicator of organizational success because it examines the health care organization as a business unit and fails to fully explore the outcomes of health care provision; thus, quality of patient care, the intangible benefit, is omitted from the analysis [1, 3, 24]. The IOM report, Crossing the Quality Chasm (2002), suggests that efficiency is one of the six domains of quality, as seen in Fig. 1. Efficiency and quality are interdependent concepts as the goal of health care is to provide Bthe highest quality of care … that yields the greatest expected improvement in health status…^ [25]. Health care that is efficient optimizes patient care while minimizing expenditures. Care that is harmless but fails to improve health is inefficient [23]. Health care organizations are tasked with providing parsimonious care without sacrificing quality [23]. In this way, information technology should increase efficiency’s contributions to quality models. In health care, it is not unreasonable to include quality measures that classify hospital readmissions and emergency room visits as undesirable outputs. The literature supports the concept of user-perceived quality indicators; therefore we include patient perception of services as a measure of quality [26]. In this study, we conceptualize efficiency as the quality of patient care and not technical efficiency in the conventional sense. In the production of care, hospitals seek to minimize future costs yet yield the greatest benefit. The increasing pace of HIT development and its influx into the marketplace requires providential consideration by health care administrators. Administrators must ask whether more technology is better and if it will lead to better patient quality of care. The IOM suggests that the focus of the health care system should be on the delivery of safer, more efficacious care using existing technologies rather than the development and implementation of new technologies [27]. While the debate about the productivity yields of technology is new to health care, other industries have realized (in the 1970s and 1980s) the BIT productivity
The effect of information technology on hospital performance Table 1
Summary of HIT applications on quality of patient care
HIT application
Patient Perception of Quality
CPOEs
Enhance care coordination, Decrease medication errors, Increase in favorable heart failure and COPD clinical outcomes EMRs Patient perception of care increases; Increase in favorable heart failure and COPD clinical outcomes PHRs Favorable Patient Satisfaction, Increase self- management, Increased patient engagement Electronic access Decreased patient anxiety, to diagnostic Increased access and quality results in chronic diseases, mental illness
Hospital Readmissions
Morality Rate
References
Decrease hospitalization via a reduction in adverse drug events
Decrease in hospital wide mortality rate
Minesh et al., (2012); McCullough et al., (2010); Longhurst et al., (2010); Miller et al., (2011);Charles et al., (2014)
Decrease in 30 day hospital readmission
No change in mortality Lee, Kuo & Goodwin, (2013); Sibona et al., (2011); McCullough et al., (2010)
Indirect benefits of chronic NA disease management, Contributed to decrease hospital length of stay
paradox^ [28]. The term was coined by Robert Solow, a Nobel Laureate Economist. Solow realized that the technological age was everywhere but in productivity statistics [28]. Since that time, much progress has been made in our understanding of how HIT affects productivity. Research suggests that health care professionals are able to accomplish more tasks, yet unintended consequences, such as decreased time in patient care services and patient satisfaction, were found [29]. Thus, keeping up with the pace of HIT is not advantageous if links to quality of care are not established. 2.1 Health policy HIT has gained considerable national and federal attention. The IOM and the federal government have both called for the adoption of HIT. In 2004, President George Bush established the Office of the National Coordination for Health
Safe
Efficiency
Patient Centered
Equitable
Effective
Time
Fig. 1 Efficiency included in the quality paradigm as defined by IOM
Archer et al., (2011),Davis et al., (2014); Tang et al., (2006).
Decrease mortality rate Hurlen et al., (2010): Patience et al., in Breast cancer (2015); Arnaout et al., (2013) patient
Information Technology which promotes HIT to improve the health care system. In 2009, President Barack Obama signed the Health Information Technology of Economic and Clinical Health Act (HITECH). This initiative distributes over $27 billion to providers who adopted HIT [30]. As a result, 72 % of office-based physicians adopted EHRs. Providers who received incentive payments were required to demonstrate Bmeaningful use^ [31, 32]. Meaningful use requirements are a measured set of objectives defined as: 1) the use of a certified EHR in a meaningful manner; 2) the use of a certified EHR to electronically exchange health information for the purposes of quality of care; and 3) the use of a certified EHR to report clinical quality measures [31]. Stages 1 and 2 passed in 2011 and 2014, respectively, and stage 3 will be enacted in 2016 [31]. The encouragement of EHR adoption through incentive payments goes beyond mere technology adoption and considers quality of patient care. The Center for Medicare and Medicaid Services (CMS) has also linked efficiency and quality. The Medicare Payment Advisory Commission reported to the U.S. Congress that HIT adoption was a key driver in improving quality of care in U.S. hospitals [33]. The Hospital Readmission Payment Penalty highlighted the critical link between efficiency and quality to the extent that penalties were imposed if quality outcomes were inadequate. CMS made the decision to cease paying for care for Bpreventable complications,^ which emphasized that efficiency is closely linked to quality [34]. At the state level, New York State, realizing the potential for increased quality of care, enacted the Healthcare Efficiency and Affordability Law for New Yorkers (HEAL NY). This piece of legislation invested nearly $440 million towards interoperable HIT and focused on EHRs and health information exchanges. HEAL NY represents a sizeable state-based investment into HIT [35]. From current federal and state initiatives, perceptions of quality by the IOM, and presented literature, we suggest that
C. Williams et al.
it is possible to have efficiency without quality in patient care. Including quality of patient care in the efficiency discussion improves the literature by considering the efficiency and quality goals required by hospital organizations. The literature suggests that technology adopters are experiencing better patient quality of care [32]. Studies demonstrate that there is a positive correlation between HIT and patient quality of care, despite mitigating factors such as provider training and integration of the HIT within the greater hospital system. While the literature supports the concept that HIT is beneficial in reducing costs and increasing the quality of care, the extent to which it contributes to such improvements has not been established [36, 37]. This paper aims to measure efficiency in U.S. hospitals by considering quality of patient care indicators in the context of live and available HIT in the environment. We seek to answer the questions: (1) To what degree does the intensity of a technologically rich environment affect positive change in hospital utilizations and patient perceived quality of care? In other words, does more technology lead to better outcomes? And, (2) Among hospitals that adopt technology, which technology applications contribute the most to patient quality of care?
3 Methods We take a unique methodological approach by providing a comparative measure of quality performance through a twostage analysis: Data Envelopment Analysis (DEA) and Automatic Interaction Detector Analysis (AID) using Decision Tree Regression (DTreg). By using both analyses, we seek a comprehensive approach to decision-making. DEA assisted in determining what inputs contributed to the quality of patient care; DTreg is used to determine which HIT components were most characteristic of the hospitals that DEA ranked highly among the decision making units (DMUs) [38]. This provides a clearer sense of the reasoning behind the quality scores and enhances decision-making capabilities for hospital administrators. Chuang et al. (2011) suggest that while DEA distinguishes between varying levels of quality outcomes, DEA does very little to assist with decision-making abilities. Studies from Rahimi & Behmanesh (2012) and Chuang et al. (2011) suggest that DEA combined with DTreg enhances the ability to evaluate quality and presents decision-making rules that may improve resource allocation [38, 39]. Data Envelopment Analysis, an application of linear programming, compares DMUs by considering inputs and outputs of the organization and establishes an efficient frontier (technical efficiency, or as conceptualized in this study, technical quality). Technical quality is the minimum number of inputs to achieve desired outputs, a ratio analysis of output over input. DEA provides a single measure of relative quality (a score between 0 and 1) for each DMU; a score of 1 indicates
optimal quality. Quality scores of less than 1 indicated more resources were used than needed to achieve desired results. Once the ranking is established, DEA identifies potential areas of improvement. By highlighting relatively quality scores, administrators may consider resource allocation strategies. This model is well-suited to guide hospital administrators, who are required to increase or maintain service outputs with diminishing resources. As a productivity management tool, DEA assists decision makers in determining the influence of HIT and its contribution to patient quality of care. This analysis uses an input oriented DEA model with CRS (constant return to scale). Using an input orientation model emphasizes managerial control and manipulation of inputs to improve the quality of patient care [40]. By incorporating a CRS, we assume that a change in the inputs has a proportional effect on outputs. After DEA, AID was performed using decision tree regression analysis (DTreg). DTreg is an open source software tool that generates a single decision tree structured model using regression. The aim of AID DTreg is to Bsplit the data successively by binary division into a number of subgroups^ [41]. In AID DTreg, the iterations stop when it does not find significant values between the determinant and predictor variables. Decision tree analysis provides for greater interpretability and visualization than is allowed by classical regression approaches. 3.1 Data sources The data were provided by and compiled from the Health Information and Management Systems Society (HIMSS) and the Center for Medicare and Medicaid Services (CMS). The CMS data file contained information on hospital readmission and mortality rates for 4791 hospitals. We obtained information on the utilization of HIT components employed by hospitals from the HIMSS analytics survey of 2011. At the time of analysis, this was the most recent data available on current HIT adoption, demographics, and organizational characteristics by 5339 U.S. hospitals. The HIMSS data files that were used included the acute care information file, hospital characteristics file, and the use of HIT components file. The hospital characteristics file contained the following information: number of FTE (NofFTE), ownership status, number of beds, number of staffed beds (Nofstaffedbeds), year established, and type of facility. The acute care information file contained hospital name, address, phone number, and Medicare number. The acute care information file (5339 hospitals) and hospital characteristic file (4908 hospitals) were merged, leaving a sample size of 4908. The use of HIT component file contained 1550 hospitals, however only 1433 reported live and operational use; this file was merged with previous file of 4908 hospitals, leaving a sample size of 1152 hospitals. This new file of 1152 hospitals was merged with CMS hospital readmission and mortality rates file (4791 hospitals) leaving 1039 hospitals to be analyzed.
The effect of information technology on hospital performance
The resultant file contained only: (1) hospitals that reported live and operational HIT applications, (2) hospitals that were classified as general medical and surgical facilities, and (3) hospitals whose data were complete across CMS and HIMSS files. Input variables Table 2 summarizes the study variables. Staffing levels (full time equivalent of nursing and physician staff), as a predictor of quality, have significant implications on patient perception of quality of care [42–45]. McHugh et al. (2013) suggested that hospitals with higher staff to patient ratios had decreased hospital readmissions and improved patient perception of quality [45]. McFarland et al., (2015) suggest that hospital size was a significant predictor of hospital quality, thus included in the analysis [46]. HIT modalities (PHRs, EMRs, CPOEs, and electronic access to diagnostic results) were included as inputs. Output variables CMS provides data on hospital readmission and mortality rates. Hospital readmission rates and mortality rates are quality indicators used by CMS to monitor chronic conditions such as pneumonia and heart failure. This publicly available data set represents the time period included in the HIMSS database (updated October 11, 2012).
4 Results A correlational analysis was performed using SPSS version 18. Results suggest that there is a high correlation between the
Table 2
Summary of input/output variables
Input variables
Description
Classification of hospital
1 - General Medical & Surgical; 2 - Academic; 3 - Critical Care; 4 - General Medical Number of Staffed Bed
Beds Full Time Equivalent Technology inputs
Nof Staffed Beds NofFTE
PHR
EMR CPOE Output variables Hospital Readmission Rates Mortality rates
Full Time Equivalent of Nursing and Physician Staff Personal Health Record For Retrieving Diagnostic Results Electronic Medical Records For Entering Orders
HRA
Number of 30 Day Hospital Readmission Rates
MR
Number of Death in 30 Days from Admission
number of beds and number of staffed beds, thus the number of beds variable was removed from the analysis. The average years of hospital operation was 52 years and average hospital size, based on the number of staffed beds, was 236 beds. There were no significant associations among HIT applications, thus all were included in the analysis. Average hospital readmission rates are 18.45 % for pneumonia and 24.56 % for heart failure: 30-day mortality rate was 11.90 % and 11.62 % for pneumonia and heart failure, respectively. 4.1 DEA The number of DMUs analyzed by the model was 1039; 20 DMUs (1.9 %) reported a quality score of 1. The average quality score among all DMUs was 0.226. The summary of quality outcomes is presented in Table 3. 4.2 DTreg For this analysis, a quality score of 1 was set as the target variable. Number of FTE, number of staffed beds, and each type of technology, were set as predictor variables. Results of the DTreg suggest that the number of staffed beds, an indicator of hospital size, was the most influential variable in the model; electronic access to diagnostic results was the most influential technological characteristic of the hospitals that had a quality score of 1. The DTreg model results suggest that there is relationship between quality and the number of staffed beds (R-squared=25.1 %). As seen in Table 4, the degree of variable importance to the model is seen in rankings of 0 to 100 %. Higher numbers (closer to 100 %) suggest stronger relationships between quality and inputs into the model. Likewise, values closer to zero suggest a weaker association. A graphical display of the decision tree is seen in Fig. 2. 4.3 Sensitivity analysis We performed a sensitivity analysis to measure uncertainty in the results. To run a sensitivity analysis, we used the results of the DEA and DTreg to examine the least important and most important variable in the model. Results of the DTreg suggest that the electronic access to diagnostic results variable is the least important variable in the model, but the most important of the HIT variables. We stratified the electronic access to diagnostic results variable: Yes, DMUs who used electronic access to diagnostic results and No, DMUs who did not have electronic access to diagnostic results. After running the DEA again for each of the stratified groups, we found that the group who had electronic access to diagnostic results had an average quality score of 1.0; while the group that did not have electronic access to diagnostic results received an average quality score of 0.000. Quality results are shown in Table 5
C. Williams et al. Table 3
Comparative results of quality scores of 1 and
The effect of information technology on hospital performance Cynthia Williams 1 & Yara Asi 2 & Amanda Raffenaud 2 & Matt Bagwell 2 & Ibrahim Zeini 2
Received: 23 December 2014 / Accepted: 19 May 2015 # Springer Science+Business Media New York 2015
Abstract While healthcare entities have integrated various forms of health information technology (HIT) into their systems due to claims of increased quality and decreased costs, as well as various incentives, there is little available information about which applications of HIT are actually the most beneficial and efficient. In this study, we aim to assist administrators in understanding the characteristics of top performing hospitals. We utilized data from the Health Information and Management Systems Society and the Center for Medicare and Medicaid to assess 1039 hospitals. Inputs considered were full time equivalents, hospital size, and technology inputs. Technology inputs included personal health records (PHR), electronic medical records (EMRs), computerized physician order entry systems (CPOEs), and electronic access to diagnostic results. Output variables were measures of quality, hospital readmission and mortality rate. The analysis was conducted in a two-stage methodology: Data Envelopment Analysis (DEA) and Automatic Interaction Detector Analysis (AID), decision tree regression (DTreg). Overall, we found that electronic access to diagnostic results systems was the most influential technological characteristics; however organizational characteristics were more important than technological inputs. Hospitals that had the highest levels of quality indicated no excess in the use of technology input, averaging one use of a technology component. This study indicates that prudent
* Cynthia Williams [email protected] 1
Department of Public Health, Brooks College of Health, University of North Florida, 1 UNF Drive, Jacksonville, FL 32224-2646, USA
2
College of Health & Public Affairs, University of Central Florida, Orlando, Fl, USA
consideration of organizational characteristics and technology is needed before investing in innovative programs. Keywords DEA . AID . Hospital . Health information technology . Efficiency . Quality
1 Introduction The health care industry is seeking to improve the quality of patient care in U.S. hospitals. To alleviate these difficulties, the U.S. health care system is undergoing significant change as different applications of health information technology (HIT) have captivated the medical community. While some attention is given to HIT and its contribution to patient quality of care, a paucity of literature examines the relationship between how much HIT is truly needed and what HIT applications are most effective in leading to quality outcomes. The benefits of HIT are likely to be underutilized without a sufficient understanding of its value. By examining hospitals that adopt technology, we aim to assist administrators in understanding the characteristics of the hospitals that demonstrate quality of care outcomes. Most health care commentators would agree that the U.S. health care system suffers from significant quality deficits; therefore as innovative methodologies enter into the health care industry, it is critical that we examine the efficaciousness of these tools to affect positive change. Such practices are able to demonstrate if resources should be devoted to these tools or if other innovations should be sought. When advanced information and communication technology is combined with the quality improvement process, it is effective in increasing the quality of patient care [1]. The decision to adopt new technology and consider its influence on quality of patient care is critical to a hospital’s viability [2]. However, the successful
C. Williams et al.
implementation and use of a HIT application is not solely related to financial gains. In health care, the benefits of technology may be intangible, such as access to care, quality of patient care, and the potential for clinical collaborations [2, 3]. While there are several types and uses of HIT, the literature has shown potential for the following HIT applications in particular to increase patient quality of care: personal health records (PHRs), electronic medical records (EMRs), computerized physician order entry systems (CPOEs), and electronic access to diagnostic results. Personal health records (PHRs) are Belectronic platform(s) that allows patients to manage [and view] their health information^ [4]. Physicians’ use of PHRs is postulated to increase clinician-patient communication, increase patient engagement, and produce better outcomes at lower costs [5, 6]. This fosters a shared decision-making relationship between physicians and patients. The benefits to quality are especially evident in patient groups that are traditionally underserved, such as those with chronic or mental illness and where complex comorbidities exist [7, 8]. Electronic medical records (EMRs) are touted as one of the Bmost promising components of health information technology^ [9]. Replacing paper records, EMRs are useful for information storage and retrieval, as well as increasing information exchange at the point of care [2]. Goodwin et al. (2013) suggest that EMR adoption demonstrates statistically significant decreases in hospital utilization. As EMRs become more readily integrated into different types of facilities and are coupled with trained on-site staff, increases in the quality of care and of the records themselves are evident [10–12]. Computerized physician order entry systems (CPOEs) focus on improving safety and accuracy in ordering medications, laboratory tests, and other medical procedures [13]. It gives providers the ability to order medications, tests, and procedures electronically, Bproviding significantly enhanced decision support capabilities as compared to traditional handwritten orders^ [14]. The literature supports the argument that CPOEs increase quality care and safety. It decreases medical errors and adverse events while improving the process of providing care [15–17]. The final application of HIT assessed in this study was electronic access to diagnostic results. Electronic access is beneficial for patients with conditions that have the potential to rapidly deteriorate because it decreases the time to diagnosis [18, 19]. It may improve the use of diagnostic imaging results in clinical practice [18]. Electronic access to diagnostic results may facilitate communication between providers and patients, resulting in a decrease in rehospitalizations and duplications of diagnostic procedures [20]. Timely access to diagnostic results may decrease patient anxiety and increase the decision-making capabilities of the patient care team [21]. Patience et al., (2015) suggest that timely reassurance is an important intervention strategy in medical consultation and
reduces subsequent needless investigations and clinical appointments. While total patient quality of care cannot be based solely on technology, we can speculate on its contribution to patient perception of quality and hospital indicators of patient quality [1]. A summary of each HIT applications’ link to patient quality of care outcomes can be found in Table 1.
2 Theoretical framework The Institute of Medicine (IOM) conceptualizes efficiency as a reduction of waste, decreasing unnecessary hospital utilizations, medication overuse, and duplication of services [22]. Efficiency refers to the combined sequence of services (inputs) in the most efficacious manner to produce the greatest increment of health (output) given specified expenditure [23]. However, efficiency optimization alone is not an adequate indicator of organizational success because it examines the health care organization as a business unit and fails to fully explore the outcomes of health care provision; thus, quality of patient care, the intangible benefit, is omitted from the analysis [1, 3, 24]. The IOM report, Crossing the Quality Chasm (2002), suggests that efficiency is one of the six domains of quality, as seen in Fig. 1. Efficiency and quality are interdependent concepts as the goal of health care is to provide Bthe highest quality of care … that yields the greatest expected improvement in health status…^ [25]. Health care that is efficient optimizes patient care while minimizing expenditures. Care that is harmless but fails to improve health is inefficient [23]. Health care organizations are tasked with providing parsimonious care without sacrificing quality [23]. In this way, information technology should increase efficiency’s contributions to quality models. In health care, it is not unreasonable to include quality measures that classify hospital readmissions and emergency room visits as undesirable outputs. The literature supports the concept of user-perceived quality indicators; therefore we include patient perception of services as a measure of quality [26]. In this study, we conceptualize efficiency as the quality of patient care and not technical efficiency in the conventional sense. In the production of care, hospitals seek to minimize future costs yet yield the greatest benefit. The increasing pace of HIT development and its influx into the marketplace requires providential consideration by health care administrators. Administrators must ask whether more technology is better and if it will lead to better patient quality of care. The IOM suggests that the focus of the health care system should be on the delivery of safer, more efficacious care using existing technologies rather than the development and implementation of new technologies [27]. While the debate about the productivity yields of technology is new to health care, other industries have realized (in the 1970s and 1980s) the BIT productivity
The effect of information technology on hospital performance Table 1
Summary of HIT applications on quality of patient care
HIT application
Patient Perception of Quality
CPOEs
Enhance care coordination, Decrease medication errors, Increase in favorable heart failure and COPD clinical outcomes EMRs Patient perception of care increases; Increase in favorable heart failure and COPD clinical outcomes PHRs Favorable Patient Satisfaction, Increase self- management, Increased patient engagement Electronic access Decreased patient anxiety, to diagnostic Increased access and quality results in chronic diseases, mental illness
Hospital Readmissions
Morality Rate
References
Decrease hospitalization via a reduction in adverse drug events
Decrease in hospital wide mortality rate
Minesh et al., (2012); McCullough et al., (2010); Longhurst et al., (2010); Miller et al., (2011);Charles et al., (2014)
Decrease in 30 day hospital readmission
No change in mortality Lee, Kuo & Goodwin, (2013); Sibona et al., (2011); McCullough et al., (2010)
Indirect benefits of chronic NA disease management, Contributed to decrease hospital length of stay
paradox^ [28]. The term was coined by Robert Solow, a Nobel Laureate Economist. Solow realized that the technological age was everywhere but in productivity statistics [28]. Since that time, much progress has been made in our understanding of how HIT affects productivity. Research suggests that health care professionals are able to accomplish more tasks, yet unintended consequences, such as decreased time in patient care services and patient satisfaction, were found [29]. Thus, keeping up with the pace of HIT is not advantageous if links to quality of care are not established. 2.1 Health policy HIT has gained considerable national and federal attention. The IOM and the federal government have both called for the adoption of HIT. In 2004, President George Bush established the Office of the National Coordination for Health
Safe
Efficiency
Patient Centered
Equitable
Effective
Time
Fig. 1 Efficiency included in the quality paradigm as defined by IOM
Archer et al., (2011),Davis et al., (2014); Tang et al., (2006).
Decrease mortality rate Hurlen et al., (2010): Patience et al., in Breast cancer (2015); Arnaout et al., (2013) patient
Information Technology which promotes HIT to improve the health care system. In 2009, President Barack Obama signed the Health Information Technology of Economic and Clinical Health Act (HITECH). This initiative distributes over $27 billion to providers who adopted HIT [30]. As a result, 72 % of office-based physicians adopted EHRs. Providers who received incentive payments were required to demonstrate Bmeaningful use^ [31, 32]. Meaningful use requirements are a measured set of objectives defined as: 1) the use of a certified EHR in a meaningful manner; 2) the use of a certified EHR to electronically exchange health information for the purposes of quality of care; and 3) the use of a certified EHR to report clinical quality measures [31]. Stages 1 and 2 passed in 2011 and 2014, respectively, and stage 3 will be enacted in 2016 [31]. The encouragement of EHR adoption through incentive payments goes beyond mere technology adoption and considers quality of patient care. The Center for Medicare and Medicaid Services (CMS) has also linked efficiency and quality. The Medicare Payment Advisory Commission reported to the U.S. Congress that HIT adoption was a key driver in improving quality of care in U.S. hospitals [33]. The Hospital Readmission Payment Penalty highlighted the critical link between efficiency and quality to the extent that penalties were imposed if quality outcomes were inadequate. CMS made the decision to cease paying for care for Bpreventable complications,^ which emphasized that efficiency is closely linked to quality [34]. At the state level, New York State, realizing the potential for increased quality of care, enacted the Healthcare Efficiency and Affordability Law for New Yorkers (HEAL NY). This piece of legislation invested nearly $440 million towards interoperable HIT and focused on EHRs and health information exchanges. HEAL NY represents a sizeable state-based investment into HIT [35]. From current federal and state initiatives, perceptions of quality by the IOM, and presented literature, we suggest that
C. Williams et al.
it is possible to have efficiency without quality in patient care. Including quality of patient care in the efficiency discussion improves the literature by considering the efficiency and quality goals required by hospital organizations. The literature suggests that technology adopters are experiencing better patient quality of care [32]. Studies demonstrate that there is a positive correlation between HIT and patient quality of care, despite mitigating factors such as provider training and integration of the HIT within the greater hospital system. While the literature supports the concept that HIT is beneficial in reducing costs and increasing the quality of care, the extent to which it contributes to such improvements has not been established [36, 37]. This paper aims to measure efficiency in U.S. hospitals by considering quality of patient care indicators in the context of live and available HIT in the environment. We seek to answer the questions: (1) To what degree does the intensity of a technologically rich environment affect positive change in hospital utilizations and patient perceived quality of care? In other words, does more technology lead to better outcomes? And, (2) Among hospitals that adopt technology, which technology applications contribute the most to patient quality of care?
3 Methods We take a unique methodological approach by providing a comparative measure of quality performance through a twostage analysis: Data Envelopment Analysis (DEA) and Automatic Interaction Detector Analysis (AID) using Decision Tree Regression (DTreg). By using both analyses, we seek a comprehensive approach to decision-making. DEA assisted in determining what inputs contributed to the quality of patient care; DTreg is used to determine which HIT components were most characteristic of the hospitals that DEA ranked highly among the decision making units (DMUs) [38]. This provides a clearer sense of the reasoning behind the quality scores and enhances decision-making capabilities for hospital administrators. Chuang et al. (2011) suggest that while DEA distinguishes between varying levels of quality outcomes, DEA does very little to assist with decision-making abilities. Studies from Rahimi & Behmanesh (2012) and Chuang et al. (2011) suggest that DEA combined with DTreg enhances the ability to evaluate quality and presents decision-making rules that may improve resource allocation [38, 39]. Data Envelopment Analysis, an application of linear programming, compares DMUs by considering inputs and outputs of the organization and establishes an efficient frontier (technical efficiency, or as conceptualized in this study, technical quality). Technical quality is the minimum number of inputs to achieve desired outputs, a ratio analysis of output over input. DEA provides a single measure of relative quality (a score between 0 and 1) for each DMU; a score of 1 indicates
optimal quality. Quality scores of less than 1 indicated more resources were used than needed to achieve desired results. Once the ranking is established, DEA identifies potential areas of improvement. By highlighting relatively quality scores, administrators may consider resource allocation strategies. This model is well-suited to guide hospital administrators, who are required to increase or maintain service outputs with diminishing resources. As a productivity management tool, DEA assists decision makers in determining the influence of HIT and its contribution to patient quality of care. This analysis uses an input oriented DEA model with CRS (constant return to scale). Using an input orientation model emphasizes managerial control and manipulation of inputs to improve the quality of patient care [40]. By incorporating a CRS, we assume that a change in the inputs has a proportional effect on outputs. After DEA, AID was performed using decision tree regression analysis (DTreg). DTreg is an open source software tool that generates a single decision tree structured model using regression. The aim of AID DTreg is to Bsplit the data successively by binary division into a number of subgroups^ [41]. In AID DTreg, the iterations stop when it does not find significant values between the determinant and predictor variables. Decision tree analysis provides for greater interpretability and visualization than is allowed by classical regression approaches. 3.1 Data sources The data were provided by and compiled from the Health Information and Management Systems Society (HIMSS) and the Center for Medicare and Medicaid Services (CMS). The CMS data file contained information on hospital readmission and mortality rates for 4791 hospitals. We obtained information on the utilization of HIT components employed by hospitals from the HIMSS analytics survey of 2011. At the time of analysis, this was the most recent data available on current HIT adoption, demographics, and organizational characteristics by 5339 U.S. hospitals. The HIMSS data files that were used included the acute care information file, hospital characteristics file, and the use of HIT components file. The hospital characteristics file contained the following information: number of FTE (NofFTE), ownership status, number of beds, number of staffed beds (Nofstaffedbeds), year established, and type of facility. The acute care information file contained hospital name, address, phone number, and Medicare number. The acute care information file (5339 hospitals) and hospital characteristic file (4908 hospitals) were merged, leaving a sample size of 4908. The use of HIT component file contained 1550 hospitals, however only 1433 reported live and operational use; this file was merged with previous file of 4908 hospitals, leaving a sample size of 1152 hospitals. This new file of 1152 hospitals was merged with CMS hospital readmission and mortality rates file (4791 hospitals) leaving 1039 hospitals to be analyzed.
The effect of information technology on hospital performance
The resultant file contained only: (1) hospitals that reported live and operational HIT applications, (2) hospitals that were classified as general medical and surgical facilities, and (3) hospitals whose data were complete across CMS and HIMSS files. Input variables Table 2 summarizes the study variables. Staffing levels (full time equivalent of nursing and physician staff), as a predictor of quality, have significant implications on patient perception of quality of care [42–45]. McHugh et al. (2013) suggested that hospitals with higher staff to patient ratios had decreased hospital readmissions and improved patient perception of quality [45]. McFarland et al., (2015) suggest that hospital size was a significant predictor of hospital quality, thus included in the analysis [46]. HIT modalities (PHRs, EMRs, CPOEs, and electronic access to diagnostic results) were included as inputs. Output variables CMS provides data on hospital readmission and mortality rates. Hospital readmission rates and mortality rates are quality indicators used by CMS to monitor chronic conditions such as pneumonia and heart failure. This publicly available data set represents the time period included in the HIMSS database (updated October 11, 2012).
4 Results A correlational analysis was performed using SPSS version 18. Results suggest that there is a high correlation between the
Table 2
Summary of input/output variables
Input variables
Description
Classification of hospital
1 - General Medical & Surgical; 2 - Academic; 3 - Critical Care; 4 - General Medical Number of Staffed Bed
Beds Full Time Equivalent Technology inputs
Nof Staffed Beds NofFTE
PHR
EMR CPOE Output variables Hospital Readmission Rates Mortality rates
Full Time Equivalent of Nursing and Physician Staff Personal Health Record For Retrieving Diagnostic Results Electronic Medical Records For Entering Orders
HRA
Number of 30 Day Hospital Readmission Rates
MR
Number of Death in 30 Days from Admission
number of beds and number of staffed beds, thus the number of beds variable was removed from the analysis. The average years of hospital operation was 52 years and average hospital size, based on the number of staffed beds, was 236 beds. There were no significant associations among HIT applications, thus all were included in the analysis. Average hospital readmission rates are 18.45 % for pneumonia and 24.56 % for heart failure: 30-day mortality rate was 11.90 % and 11.62 % for pneumonia and heart failure, respectively. 4.1 DEA The number of DMUs analyzed by the model was 1039; 20 DMUs (1.9 %) reported a quality score of 1. The average quality score among all DMUs was 0.226. The summary of quality outcomes is presented in Table 3. 4.2 DTreg For this analysis, a quality score of 1 was set as the target variable. Number of FTE, number of staffed beds, and each type of technology, were set as predictor variables. Results of the DTreg suggest that the number of staffed beds, an indicator of hospital size, was the most influential variable in the model; electronic access to diagnostic results was the most influential technological characteristic of the hospitals that had a quality score of 1. The DTreg model results suggest that there is relationship between quality and the number of staffed beds (R-squared=25.1 %). As seen in Table 4, the degree of variable importance to the model is seen in rankings of 0 to 100 %. Higher numbers (closer to 100 %) suggest stronger relationships between quality and inputs into the model. Likewise, values closer to zero suggest a weaker association. A graphical display of the decision tree is seen in Fig. 2. 4.3 Sensitivity analysis We performed a sensitivity analysis to measure uncertainty in the results. To run a sensitivity analysis, we used the results of the DEA and DTreg to examine the least important and most important variable in the model. Results of the DTreg suggest that the electronic access to diagnostic results variable is the least important variable in the model, but the most important of the HIT variables. We stratified the electronic access to diagnostic results variable: Yes, DMUs who used electronic access to diagnostic results and No, DMUs who did not have electronic access to diagnostic results. After running the DEA again for each of the stratified groups, we found that the group who had electronic access to diagnostic results had an average quality score of 1.0; while the group that did not have electronic access to diagnostic results received an average quality score of 0.000. Quality results are shown in Table 5
C. Williams et al. Table 3
Comparative results of quality scores of 1 and
