International Journal of Medical Informatics 84 (2015) 683–693

Contents lists available at ScienceDirect

International Journal of Medical Informatics journal homepage: www.ijmijournal.com

Integrating commercial ambulatory electronic health records with hospital systems: An evolutionary process Susan A. Sherer a,∗ , Chad D. Meyerhoefer b , Michael Sheinberg c , Donald Levick c a b c

Lehigh University Department of Management, Bethlehem, PA 18015, United States Lehigh University Department of Economics, Bethlehem, PA 18015, United States Lehigh Valley Health Network, Allentown, PA 18105, United States

a r t i c l e

i n f o

Article history: Received 16 March 2015 Received in revised form 15 May 2015 Accepted 18 May 2015 Keywords: Electronic health records Care coordination Clinical information retrieval

a b s t r a c t Objective: The increase in electronic health record implementation in all treatment venues has led to greater demands for integration within and across practice settings with different work cultures. We study the evolution of coordination processes when integrating ambulatory-specific electronic health records with hospital systems. Materials and methods: Longitudinal qualitative study using semi-structured interviews and archival documentation throughout a 5-year implementation and integration of obstetrical ambulatory and hospital records with a goal of achieving a perinatal continuum of care. Results: As users implement and integrate electronic health records, there is an evolution in their focus from technology acceptance to structural adaptation to coordination. The users’ perspective on standardization evolves from initial concern about the unintended consequences of standardization to recognition of its importance and then finally to more active acceptance. The system itself cannot drive all reengineering; the organization must impose specific work process changes and as the user’s perspective evolves, more individually adapted and aligned change will occur. Computer integration alone does not result in coordination; users must value integrated information and incorporate this information within their workflows. Discussion: Users initially view electronic health records as a documentation tool, but over time they come to recognize the benefits of the system for clinical information retrieval, and finally, for care coordination after the integrated information provided through electronic health records becomes more complete, accessible and adapted to meet user needs. As this occurs, coordination mechanisms move beyond pooled standardization through sequential plans coordinated by the organization to reciprocal mutual adjustments for clinical decision making by individuals. Trust in the information source, not software interoperability, is critical for information sharing. Conclusions: Organizations implementing commercial electronic health records cannot simply assume that reciprocal coordination will immediately occur. It takes time for users to adjust, and enculturate coordination goals, during which time there are adaptive structurations that require organizational response, and changes in mechanisms for achieving coordination. © 2015 Elsevier Ireland Ltd. All rights reserved.

1. Introduction Adoption of electronic health records (EHRs) in the U.S. is increasing as a result of the incentives created by the HITECH Act [14,15,29]. As these basic systems are adopted, true meaningful use will require integration of information with other systems

∗ Corresponding author. Tel.: +1 6107583424; fax: +1 6107586941. E-mail address: [email protected] (S.A. Sherer). http://dx.doi.org/10.1016/j.ijmedinf.2015.05.010 1386-5056/© 2015 Elsevier Ireland Ltd. All rights reserved.

across ambulatory practices, hospitals, and other facilities, both within a health care network and across networks. Furthermore, as healthcare organizations assume financial/clinical risk for populations, the need for integration of clinical data across the continuum is critical to achieve financial success and improve clinical quality. While EHR implementation challenges have been documented [18,25], there have been few studies of integration challenges [5]. Custom developed systems that support existing work processes were the basis for most early reported successful implementations

684

S.A. Sherer et al. / International Journal of Medical Informatics 84 (2015) 683–693

[6], but developing these systems is simply not cost effective for most health care organizations. As a result, commercial vendorsupplied applications are the mainstream of health information technology (HIT) adoption [1]. Integration of these non-customized applications with other existing applications and work processes across multiple care settings that often have different organizational cultures can be challenging. Patient care coordination through integrated systems requires not only user acceptance, but user technology appropriation and work process adaptation aligned with organizational integration goals. EHR workarounds within a single facility have been found to have unexpected consequences [20,24,60] and users can adapt technology in ways that challenge organizational goals [12,21,42,43]. There are many sociotechnical changes accompanying EHR use [20,24,35,37,38,44] and factors impeding implementation and use [19]. A systematic review of research on EHR implementations between 2001 and 2011 reported a lack of socio-technical connections between the clinicians, the patients and the technology when developing and implementing EHRs [41]. Drivers of workarounds and sociotechnical changes are operational, cultural, organizational, and technical, and include the need to maintain productivity when acclimating to new systems, discomfort with new work processes, and the potential for information overload [8,39,50]. However, there are few reports of integrated EHR programs [18], and in particular the process changes required to achieve coordinated care benefits [1]. Our study addresses the research questions: how do users achieve coordinated care goals via information sharing with integrated health records? How do their views evolve and how do they adapt their usage of EHRs to achieve their integration goals? We use a grounded theory approach, using longitudinal case studies to build theory.

2. Background An integrated EHR that makes a patient’s clinical data instantaneously available to all providers throughout a given episode of care, regardless of the service location (hospital, primary care practice) or provider (specialist, primary care physician), holds great promise. It could facilitate coordination of care, improve clinical decision-making, and reduce underuse, overuse, and inappropriate use of medications and diagnostic tests. Many medical errors are attributable to poor communication when a patient transfers from primary care to acute care and back again [40,45]. Studies of chronically ill patients have shown that for 22% of visits to a single physician, prior test results were missing and duplicate tests were ordered, with this percentage rising to 43% of visits when patients saw four or more physicians. Furthermore, over 33% of patients discharged from inpatient facilities reported no subsequent follow-up arrangements [47,48] and fewer than 50% of primary care physicians (PCPs) reported receiving any discharge information on recently released patients [24]. A review of the literature on communication between hospitalbased physicians and PCPs reported that only 3% of PCPs had any conversations with inpatient physicians about discharge planning; that only 17–20% of outpatient doctors received any discharge notifications; and only 20% had received a discharge summary at one week following discharge. In fact, 66% of PCPs had at least one post-discharge visit from most patients before receiving any discharge information from the hospital, and 25% of discharge summaries never reached the PCP. Of those summaries that did arrive, 38% did not list test results and 21% did not include medications prescribed at discharge [34]. This has been identified as such an important issue that successful management of “transitions of care”

and appropriate “hand-off communication” are part of the requirements for physicians and hospitals to achieve Stage 2 of Meaningful Use.1 In response to such breakdowns in physician communication, integrated EHRs have emerged as a priority among the possible HIT applications. However, full implementation of EHRs even within a single facility involves high initial costs, changes in workflow, training, and adoption-related inefficiencies [4,20,60]. As a result, PCPs and hospitals have been slow to adopt these technologies [13,15,28,31], which has hampered efforts to integrate EHRs across care settings. And while there is a growing literature on the implementation issues for EHRs, there is limited literature on the organizational, workflow, or cultural changes that must take place to achieve coordinated care through integration of records across multiple care settings. 3. Methods 3.1. Research setting Our field site is the Lehigh Valley Health Network (LVHN), a regional health system located in Allentown, PA, where we analyzed the implementation of ambulatory EHRs within all obstetric (OB/GYN) practices of the hospital system’s physician group and its integration with Lehigh Valley Hospital’s labor and delivery (L&D) unit. The perinatal continuum of care is a microcosm of what might occur with integration of all health records for the general population once all lifetime health information is captured. It involves a defined population that is guaranteed to cross the various locations of care (hospital, ambulatory) within a relatively defined time period (gestation). LVHN delivers nearly 3900 babies each year, almost all of which are managed by network physicians and providers. The information flow is shown in Fig. 1. There are two primary ambulatory obstetrics care practices in the network (Practice A and B), one outpatient clinic, and one high-risk inpatient unit that provides complex care either in place of or in collaboration with one of the primary obstetrics practices. When expectant mothers who are patients of any of the network obstetrics practices think they may be in labor, they go to the Labor & Delivery (L&D) triage sub-unit, where their condition is evaluated. The physician on duty either formally admits the patient to the L&D unit or discharges them back home. Medical records and information must flow from the four obstetric/gynecological (OB/GYN) practices to the triage subunit of the L&D unit, from triage back to the practices every time a patient is seen but is not admitted, and again from the practices, through triage, to L&D once a patient is formally admitted. Prior to this implementation, courier or fax was used to accomplish this information flow. The process of installing and linking EHR systems took place in phases; the timeline is shown in Fig. 2. First, all four OB/GYN practices implemented GE’s Centricity Physician Office (CPO), the ambulatory EHR, some migrating from paper, others from a legacy EHR system. Integration took place in three phases, beginning in late 2009 when discrete data elements in CPO, including medical and antenatal histories, were integrated into the inpatient perinatal information system, GE Centricity Perinatal (CPN). This was followed by triage summary documents recording new treatments and diagnoses recorded in CPN moving into the ambulatory EHR,

1 Demonstration of meaningful use, as defined by the Office of the National Coordinator for Health IT along with the Center for Medicare and Medicaid Services, is achieved in three stages corresponding to increasing level of EHR functionality. In order to qualify for federal subsidies and avoid future penalties, providers must demonstrate that their systems meet the meaningful use criteria.

S.A. Sherer et al. / International Journal of Medical Informatics 84 (2015) 683–693

685

Fig. 1. Information flow between ambulatory and hospital units.

CPO. And finally, discrete data elements recorded in the triage unit in CPN were integrated directly into CPO, the ambulatory EHR. 3.2. Study design We conducted a qualitative study using an interpretive philosophical perspective, with a longitudinal multiple case embedded design [59]. Each case was an individual practice location and each participant was a unit of analysis. The study was embedded in a large scale mixed method analysis of the benefits and challenges associated with the introduction and integration of EHRs. In addition to 76 one-hour interviews with both clinical and non-clinical staff participants, we reviewed workflow change documentation

and meeting minutes of various committees including EHR Oversight, OB Steering, OBGYN Clinical Content Workgroup, CPN Leadership, CPN Interoperability, Mother Baby Unit Leadership, and Cross Divisional Content throughout a five-year implementation and integration process. We transcribed and coded all qualitative data with NVIVO software, using two coders to ensure reliability and consistency. Interpretations evolved through the iterative analysis of the information within this context. 3.3. Data collection We captured user perceptions with three rounds of interviews as shown on the timeline in Fig. 2. Interviews were based upon

Fig. 2. Timeline.

686

S.A. Sherer et al. / International Journal of Medical Informatics 84 (2015) 683–693

semi-structured questions, shown in Appendix A. The first group of interviewees included key participants in each location who were expected to have the broadest understanding of the organizational changes associated with EHR adoption, such as physicians in administrative roles, nurse coordinators, and EHR implementation staff. We then sampled all users on several dimensions including roles and location (see Fig. 2 for a breakdown), so that we could capture as much variation as possible. Finally, we transitioned to theoretical sampling to guide additional interviews [16,36], as we expanded our theoretical interpretation.

Table 1 Density for major coding concepts nodes (percentage of coded references). Nodes

Facilitation Unexpected consequences Reengineering Value

Phase I (Acceptance)

II (Adaptation)

III (Coordination)

23 23

14 44

11 48

41 13

30 13

30 11

4. Results 3.4. Analytical approach Data collection and analysis occurred iteratively, consistent with grounded theory procedures [23,53]. The lead qualitative investigator (SAS) coded the first round of interviews. After an extensive training period, two graduate research assistants independently coded each of the transcribed interviews. The coders were trained to adhere to interpretive field research principles [32]. The first round of interviews were coded using open coding [52], using concepts initially identified via the technology acceptance [10,26,27,57,58] and complementarity literatures [3,9,46]. The coders, guided by the lead qualitative investigator, moved to axial and then selective coding [52], iterating between the interdependent meaning of the individual comments and general principles. After each round of coding, the lead qualitative investigator reviewed and organized the coding themes, with input from the coders. Four core conceptual categories emerged: value, reengineering, facilitators, and unexpected consequences, similar to four of the Cs in Glaser’s coding family [22]: consequences, covariances, causes, and contingencies. Coding the respondent’s location and attributes such as job title, office location, gender, and age captured the other two Cs: conditions and contexts. Examples of the coded subcategories are included in Appendix B. Many of these subcategories are consistent with concepts identified in previous studies such as communication, information availability, workflow, and trust [17]. The lead qualitative investigator analyzed inter-rater reliability and selected only nodes where there was strong reliability, to ensure sensitivity to bias, multiple interpretations, and dialogical reasoning [32]. The investigators performed two different types of analyses. First, we looked for relationships between subnodes of the four major categories where there was high inter-rater reliability. For example, the investigators found a strong relationship between one of the reengineering sub-nodes, policies, and one of the value nodes, standardization, suggesting a strong relationship between the role of policies in enforcing standards. The lead qualitative investigator then focused on differences in coding patterns and content over the three different time periods. We expanded our theoretical foundation as we analyzed the data. For example we recognized a need to draw upon the situated change perspective [42] to understand organizational transformation, analyzing the adaptive structuration of the users [12,43]. Additionally, we drew upon coordination theory [55] to interpret the impact of these adaptations on coordination. In order to ensure that our conclusions are robust, we used multiple sources of data (archival data and interviews) for construct validity, and multiple informants from different hierarchical levels and functional areas to limit bias [16]. We used multiple coders to increase the reliability and consistency of our qualitative themes, and had key hospital informants review our analysis for internal validity. Finally, our analysis is based on multiple cases in order to increase external validity, although we acknowledge that our study is limited to a single hospital system.

4.1. Changes in user focus across three phases of EHR implementation: acceptance to adaptation to coordination Analysis of the density of coded comments within the four major conceptual categories revealed an evolution of user focus across three phases, from acceptance to adaptation, and finally coordination. Table 1 summarizes the coding density in terms of the percentage of nodes within the four major categories in each phase. 4.1.1. Phase I: acceptance Almost one half of the nodes coded during Phase I of the study were related to (1) facilitating acceptance of an automated clinical documentation tool as a result of performance and usage expectations derived from current work practices, or (2) unexpected consequences that arose when these individual expectations did not match the system and organization goals. While there were a significant proportion of nodes coded to reengineering concepts, our analysis showed that many of these concepts focused on changes required by the system, not on process improvements that could support better integration and use. While the system was introduced within existing work practices, it required providers to make substantial changes to their workflows. Process reengineering to support integration occurred later as users became more familiar with the system and its value. Users were primarily focused on key facilitators of technology acceptance, particularly performance and effort expectancy [10,11,54,57,58] and work practice compatibility [30,56] to support the use of the electronic documentation tool. This varied between physicians and other staff, as physicians expressed higher effort expectancy and lower performance expectancy for the record systems compared to staff. Physician effort expectancy was particularly high due to (1) new work requirements and (2) navigation difficulties, as one physician noted: “It’s a big effort to enter everything into this and make sure it’s accurate.” Physicians now were required to complete billing codes themselves as well as additional documentation that were previously not required entries for them. “It takes more time to create the same document that I could either have hand written or, more frequently, dictated. It just takes more time, more time to go from screen-to-screen, more time to generate the diagnosis code, the orders, the dialogue of the plan, the plan of management.” (Physician) Many found the system difficult to navigate: “So your ability to get a quick sense of this patient, I think is harder with EMR, because no matter you’re looking at you have to page through document after document after document to get the sense of everything.” (Physician) Physician performance expectancy was low. Some felt that the system would interfere with patient care:

S.A. Sherer et al. / International Journal of Medical Informatics 84 (2015) 683–693

“I really felt it was going to detract from me having eye contact with the patient and, you know, reading the body language.” (Physician) Charting procedures had traditionally been the province of individual physicians, who did not necessarily distinguish this from their clinical responsibilities. They felt that the new procedures did not match their work processes: “It’s like it’s all about billing, and it’s not really about the medical problems. . . it’s not very physician friendly. It’s very billing friendly.” (Physician) In contrast, performance expectancy was much higher while effort expectancy was lower among other staff, particularly due to two major features of the EHRs: (1) integrated and timely lab reporting and (2) lack of need to search for files. “I like the idea of the labs and everything integrating from one system to the other like, you know, from [Lab X], basically anything from the hospital, and as well as the other facilities. . .that everything just flows right in, that we can see a lot more information.” (Nurse) “As soon as that [imaging] result is put into the hospital’s system, it then gets pushed into ours, into CPO, and we see it....Now that we went to this, we’re calling patients before they’re getting their call [from the imaging center], so our efficiency really has gone up.” (Nurse) “You have no idea how much time was used looking for charts...The nurses, they get things done, I think, faster. I don’t think that the majority of them are here as long or longer than their scheduled time, let’s put it that way. I think they’re getting out at their designated time more now than they had.” (Nurse Administration)

4.1.2. Phase II: adaptation By Phase II, they focused on what was necessary to adapt the system to meet their needs. Most of the coded nodes were focused on unexpected consequences and reengineering to accommodate these outcomes. Many of the unexpected consequences involved conflicts between system usage and optimal cognitive workflow. In addition to adaptation in data entry, many unexpected consequences resulted from issues involved in finding information. In fact, the users focused more on the information within the system, including what had to change to use this information, and how they had to adapt to better utilize this information. There were several types of adaptive structuration: alternative data entry modes through time and field shifting, and poor information retrieval processes. Data entry processes were adapted via time shifting, for example entering patient information at the end of the day rather than during a patient visit, or field shifting, for example, entering free text rather than (or in addition to) requested entry formats. Time Shifting: “The docs, nurse practitioners, whatever, need to spend other time just simply finishing up the documentation, nighttime, weekends, lunchtime, other times, ...I would say this, there is no way, and I would say this to anybody, there’s no way that most of the providers are able to complete their documentation from their patient encounters during the normal quote, “work time”. This is happening on other extra time.” (Physician) Field Shifting: “All these templates that exist that I find kind of cumbersome. So I end up just free texting a lot of stuff... But I don’t think the CPO people like that because they want you to use those fields that they created, that’s how they track it. You know, can’t track my freehand texting.” (Physician)

687

Dysfunctional data retrieval practices involved system underutilization or avoidance or inordinate amount of time allocated to information retrieval. Previously when the information was unavailable, the providers just “did the best they could” with the information that they had. However, once they knew that more information was available, they felt compelled (medically and legally) to find and review this information. This often resulted in inefficient data retrieval processes accompanied by frustration and sometimes a learned helplessness (I might as well not try). Too Much Time: “So now I’m a data miner, so I spend a lot of time data mining and wasting time trying to find data that’s there when before it was worse because the data wasn’t there... So now the data’s there and you can’t find it or you waste a lot of time to find it,...we didn’t move to the next step where it’s synthesizing that data into a usable force.” (Physician) Underutilization: “I’m just now accommodating for my slowness, as opposed to thinking how can I use the system to make me more efficient.” (Physician)

4.1.3. Phase III: coordination One year after completion of phase III, the density of coding continued to focus on unexpected consequences and reengineering, as the system continued to be adapted. During this phase, we saw system design changes required to make the system more useable, particularly for care coordination. While the density of coding to value concepts remained fairly constant, within the sub-node classifications a move occurred toward greater focus on the value of the information in order to achieve coordinated care, rather than on productivity and efficiency improvements. “It is flowing through and I think it’s great. That’s a wonderful thing that you guys have done. It was very much needed, because patients could’ve been in before and we wouldn’t have even known it except for a phone note and now everything is described as far as what happened.” (Physician) However, there were limits to the acceptance and use of the data itself, as many comments centered on issues with trust of data submitted by others. In fact, even when data such as lab test results were available directly within discrete system fields, data verification was still typically done against source lab results, which were now scanned as complete documents within the record. In some cases, this was the result of training: “We’re taught to, you know, look at it ourselves and make sure that we physically saw the test results. . .We have to look at the actual source document. ... I think it’s more so from the standpoint of, OK make sure that you look at it and make sure they’re not missing anything. I guess, later on, I guess you can rely on it a little bit more. But at this point, each resident’s kind of held to a high standard in terms of verifying . . ..everything they see.. Because potentially you’ll base a clinical decision on that, you want to make sure that it’s absolutely right, to the best of your abilities”. (Resident) In other cases, users did not trust the veracity of the information entered by others. “I have a concern that errors can happen too easily just from clicking the wrong drop-down bar, from being tired. . .. And then it generates through”. (Nurse) If a provider noted that they personally viewed a report, then another provider reading that report would more readily trust the data. If the provider did not make this note in the record, then the system was used to alert the second provider to search for

688

S.A. Sherer et al. / International Journal of Medical Informatics 84 (2015) 683–693

more source information, thereby creating a secondary means of communication:

of standardization and its value for coordinated care, as users came to appreciate its importance.

“Sometimes I trust what the person wrote in their note. Like, if the person says, “I’ve reviewed the operative report, and this is what the operative report said.” I would trust that. But if they don’t specifically write, “I viewed the operative report,” then I would try to find it myself.” . . .. “in the same way, when I’m the person seeing them in the office I’ll write that: “Personally reviewed operative report”. (Resident)

“I’m not saying that everybody has to be totally standardized, but I think what goes into some things that are very routine things like new OB write-ups, first initial doctor visits, those should be standardized. I’m not saying that our practice should be cookie cutter, but there are certain things that should be in visits. One of the problems here, with CPO, was, is there was too much customization. Everybody wanted something a little bit different because their practice functioned a little bit differently.” (Nurse practitioner)

4.2. User perspective on standardization Since standardization is a critical concept in integration, we analyzed relationships between standardization as a facilitator and value concepts over time. We found that there was a significant change in how users perceived standardization. In the beginning, as the EHR was introduced into the offices, there were concerns about whether and how clinical documentation could be standardized. Providers were accustomed to documenting in their own way, personalizing their notes, in ways that provided visual memory cues. “It is very individualized. I like to do something and he likes to do it a different way. Then you come in and you say, ‘okay, we’re going to do it this way because this is the standard,’ and then it’s not easy for me....We liked our ‘individualness’. We’re taught to be skeptical and not believe things that are there and then when you sort of have different mindsets about what the goal of the medical record really is, that part’s hard.” (Physician) By the end of Phase II, as providers began to use the system and adapt it to their needs, they focused on the unexpected consequences of lack of standardization for information retrieval. If providers documented differently or the scanned documents were entered with non-standard labels, information was not easily located. “We had a real problem in the beginning, no one understood the purpose of committing the assessment2 , so it took us several months to really get people documenting consistently across all of our providers to help the subsequent people who see them”. (Physician) “There’s no standard for the labeling of the documents. . .. . .. . . There is a standard, but I don’t know that everybody’s familiar with the standardization of it.” (Physician) Finally, lack of standardization was recognized as a problem for mining the information to improve care: “You can take for instance like the standard form for Obstetrics. Every group, the group practitioner documents on it in a different way, even though the key elements are there. . . so a lot of people use the flow sheet to document all those things and when you print the prenatal record that comes out. Other people use the assessment and plan portion to put their whole part, which doesn’t get printed out of the prenatal record and the standard pages I get... so when they come to Labor and Delivery and they hit print you just get that sort of Ob flow sheet view and it has all the flow sheet records, which it was intended to be, but no one ever came out and said, ‘okay, this is exactly how you want to document Ob patients. This is our standard.’” (Physician)

“I hope one day there’s just one system where we can see whatever we want to see, you know, and somehow they come up with some kind of standards that people are held accountable to make sure it’s a clean, accurate record because I think one of the things the docs complain about the most is they see things in the chart because other people are entering things that maybe might not be accurate but it’s really not for them to say because it’s not their practice or their specialty.” (Nurse administrator)

4.3. Coordination By analyzing the relationships between facilitation and reengineering concepts with coordination value, we found changes in the impetus for process reengineering to achieve coordination. In Phase I, most of the complementary changes in roles, work processes, and relationships that typically accompany new systems [2,9,46] were driven directly by the system itself. For example, providers had to document using standard templates and complete billing codes in order to document within the system. As the system was rolled out, ensuring completion of standard fields for quality reporting and the availability of records across multiple locations required organizationally imposed work processes, including specific policies for completion and sign-off, organizational follow-up via the quality department, and additional training. For example, to enforce standardization, the quality control department had to audit documentation to identify missing or incorrect information. Throughout phases II and III, there were more individually initiated process changes to work toward improved coordination. By the completion of Phase II, there were many individual workarounds to accommodate integrated system problems. For example, problem lists became unwieldy as the records captured information from all specialties, so individuals identified high-risk pregnancies by adding that information to the patients’ banner headings. However, due to the significant deep misalignment of the integrated system with work requirements [51], the system was modified within Phase III, to create a prioritized OB pregnancy problem list that was exchanged with the hospital system. This required group retraining for all the providers in order to manage the problem list appropriately. One year after completion of Phase III and the system changes, individual adaptation was more aligned with organizational goals for coordination. As users began to value the information that flowed back and forth, they adapted their behaviors to match coordination goals. For example, as the office-based users incorporated review of triage information within their workflow, triage-based users sent fewer notes in the form of flags, and relied on the system to communicate issues.3

By the time that information was flowing in both directions after Phase III concluded, there began to be much more active acceptance

2 Committing an assessment was one of several ways to document the provider’s care plan.

3 Flags are an EHR function that allows providers to send free text messages to one another. When the receiver opens the flag they are re-directed to the patient’s chart, but flags are not part of the official medical record because the sender can delete them.

S.A. Sherer et al. / International Journal of Medical Informatics 84 (2015) 683–693

“There’s more of a communication within the flow of what we’re doing. . ..I mean, flags are not part of the flow. So we have better communication in some respects because of it’s built in now. We now see, because I can rely on it. I know it’s going to happen.” (Physician) 4.4. Interoperability Analyzing patterns of discussion related to interoperability, trust, and work process concepts, we found that individuals did not rely on computer integration when data were available through more familiar sources. It was originally expected that direct computer-to-computer exchange of data from the office into the hospital system would be a source of efficiency, since it could enable providers to find information within the same system in which they documented their hospital visits. However, the transfer of data from the office system (CPO) directly to the triage system CPN never became a useful feature for the providers because (1) dataflow were unreliable; (2) access was not within standard workflow; and (3) CPO office records provided alternative access to the same information and became directly accessible from the hospital when CPO terminals were installed on inpatient L&D unit. Very few respondents knew where to find the discrete data elements from the office (the prenatal tab) directly within the hospital system, even by Phase III. Furthermore, if they learned how to use the tab and found the data incomplete, they were hesitant to do this again. And the data were not within workflow. “Unless there’s a button right on that screen that says, ‘show me the old OB history,’ nobody’s going to go up to the menu and record and then pull down the submenu that says past OB history.” (Physician) Instead, as CPO availability on the triage unit increased, it became the primary data source for OB history. We confirmed this theme with 32 completed user surveys from triage that we administered in 2012 and 2013. The majority of respondents received most of their historical data for a patient by reviewing the office record in CPO (59%). Some relied on both CPO and patient interview (6%) or primarily on patient interview (12.5%). Only 12.5% said that they received their information from the CPO-CPN Interfaced Data. On the other hand, the direct computer-to-computer data transfer from the triage CPN system to the CPO system was much more valued because the triage summary provided information that was previously unavailable in the ambulatory practices and the discrete data were directly within the workflow of the providers. “Now with the new ability to flow everything in from triage, it’s a good thing. You can see the whole note, what was done, any changes, any labs. It’s good. . .. . ..I’ve noted that the triage visit comes right into the OB flow sheet, and you can read it right off the OB flow sheet. You can see that the note is a triage note, because it’s dedicated on the first page of the flow, and you can see the documentation is slightly different as well so it automatically cues in your brain, oh, they were in triage. Then you look at the body of the note on the second page, and if there’s

689

a lab you need to look for whatever, you can do that. But I don’t know, I don’t seem to have a problem with that. I’m just thankful it’s there. . ...It’s kind of like one of those things like if we didn’t have the information, yeah, you can still look after the patient, but do I think the quality of the care that we would give would be the same? The answer is no, because it’s like working in the dark a little bit. You can focus on the specific issue of why the patient’s here, but now when we have the complete picture of the patient, the context is very different.” (Physician) 5. Discussion Our results show that implementing and integrating EHRs is an evolutionary process. The users’ vision evolves from viewing EHRs as an automated clinical document tool (Phase I) to a clinical information retrieval and management tool (Phase II) to a system for care coordination (Phase III). This is a multi-step process, requiring system use and complementary process change, first requiring acceptance, then adaptation to individual needs, especially regarding information retrieval, and finally, learning what data to trust. In healthcare institutions where users have traditionally been accustomed to more autonomy, the value of standardization may initially be challenged [7,33,49]. But with continued use and complementary process change, accompanied by management of user expectations, views evolve. Our results also show that software interoperability is not the only way to coordinate. In fact, because users trust familiar sources, electronic exchange may not be the most useful mechanism for information sharing, particularly if the information exchange is not trusted. Even if information is provided in more efficient ways, it is trust in the information source, not software interoperability, that is critical for information sharing. Simple introduction of a system does not enable complete integration and coordination. While the goal of health care integration is reciprocal interdependence, we found that it required time and organizational effort with progression first through several stages of coordination. Coordination theory suggests that there are three types of interdependence and three corresponding types of coordination [55]. The three types of coordination, summarized in Table 2, place increasingly heavy burdens on communication and decisions and require different types of coordination mechanisms. As the user’s vision of the system evolves, we found that more complex mechanisms for coordination can be introduced. 5.1. Pooled coordination Pooled coordination through standardization, the first coordination mechanism accompanying the implementation of the EHRs, was initially threatened by user response to the system. To generate standard metrics, LVHN instituted a single patient ambulatory EHR used by all specialties. These included specific fields that the providers were to use in order to capture and report on quality statistics. However, as LVHN rolled out the system, they found that some providers preferred free text input, which threatened pooled coordination.

Table 2 Evolution of coordinationa . Phase

User vision

Primary type of interdependence

Coordination mechanism

I

Automated clinical documentation

Standardization: Routines or rules

II

Clinical information management

III

Care coordination

Pooled: Each unit contributes to the whole Sequential: One unit produces output needed by another Reciprocal: Outputs of each unit become inputs for others (cyclical)

a

Categorization of types of interdependence and coordination mechanisms from [55].

Plan: Schedule Mutual adjustment: Feedback

690

S.A. Sherer et al. / International Journal of Medical Informatics 84 (2015) 683–693

“If we start free texting too much then we can’t go back and look at our data, so you can’t find all those elements because that’s the whole reason they want to have the drop-down menus so that we can do research and also do quality controls. If everybody just quick texted stuff, now when we search for a mammogram or people who got GBS4 screening and everyone’s quick texting it in and you run the report, 75% of the people don’t have it. You’re going to have this big alarm go off saying, oh my gosh, we’re not screening for GBS when really everyone’s just putting it in in quick text so you can’t find it when you’re running the reports.” (Physician) To insure pooled coordination, LVHN had to enforce the use of standards. The quality control department audited documentation to find missing or incorrect information.

5.2. Sequential coordination As the system was integrated, sequential coordination mechanisms were required. Final completion of the documentation of the office visit or encounter by the physician, referred to as a “sign-off,” was required to make the system available to other users. Also the triage provider had to select an ambulatory practice in order for that practice to receive the triage information. If she failed to do so, the information would not flow to any of the outpatient providers. The organization initially had to develop policies to enforce sequential coordination. Sign-offs served several purposes: (1) unlock records, (2) trigger information transfer, and (3) supervise physician extenders and residents. While timely sign-off was required for sequential coordination, it was not easy to achieve. “Ideally, right, everything should be signed off as soon as you’re done with the patients. They walk out of the room, you sign off, done. Reality is that, well, you have four more people waiting, I’ll finish this later.” (Physician) “She keeps it as a reminder to herself, almost like when she used to hang onto that paper chart because she wanted to call back to the patient in maybe three days and see how they were doing, or you know, she wanted to make sure to check to see if they went for their blood work, or whatever the case may be. That was her reminder. You know, it’s sort of like that little Post It note. She’s doing the same thing with the electronic medical records. She keeps it on her desktop as a reminder.” (Non-clinical staff, referring to physician) Non-timely sign-offs affected sequential coordination and timely care: “I hate to be on the receiving end of a call coming in from a physician, “Dr. So-and-so, you failed to sign off this record. I’m now seeing this patient. I need that information.” And that, unfortunately, has happened. And sometimes that’s what needs to happen for that record to get signed off, so that that other physician could treat the patient.” (Non-clinical staff) To promote sequential coordination, strict sign-off policies had to be created and enforced: “So your first offense is if you have 30 office visits or more open for a period of greater than a week... you’ll get written up. Second offense, you know, I think is, again, you have to talk to your department chairperson. Third offense is a little bit more egregious, and it just keeps going. And then fourth time, whatever,

4

GBS is group beta strep.

then I think that there might even be like a monetary sanction.” (Non-clinical staff) Several system modifications were required to achieve sequential coordination once cultural differences surfaced that were not immediately evident. Since the hospital focused on episodic care, default reporting included sequential periods of data collection within a very compressed amount of time (minutes to hours). The ambulatory culture, which focused on lifetime care, involved systems whose default data reporting usually occurred once per day. When hospital data began to flow over to the office records, the visibility of the details recorded sequentially over a short time period was lost. The ambulatory system had to be modified by adding an alert notification in the ambulatory record whenever data flowed from the hospital as a result of a triage visit. A new flow sheet with detailed time period reporting during the episode was then added to the ambulatory record. This sequential coordination mechanism was necessary to accommodate flow of data between the two systems.

5.3. Reciprocal coordination As physicians began to use data in the record as a source of information, particularly as the office-based physicians saw the value of the data flowing back from the hospital, there were additional adaptations to standardize where and when data were recorded. Individuals were more apt to work together to develop standards that enabled improved information retrieval. “I’m not surprised because what I’m hearing in talking to people is that they’re realizing that what somebody else does is what’s making it harder for them, so I think they’re starting to think, well, you know, maybe if we did it this way and I knew where to look, so I think that connection, they’re starting to realize because it’s impacting them, so then they’re thinking, well, maybe if we had a way that I knew where to look and we all could come together on that, it would save me time and effort. So I can see that that’s happening.” (Administration) One of the major issues impacting the usefulness of the information for reciprocal coordination was unwieldy problem lists in triage that might include information irrelevant to a pregnancy. Both technical and organizational changes were made to create a problem list specific to pregnancy: “We actually implemented a process within CPO that allows us to identify the pregnancy-related problems... because the system is universal, anybody can enter a problem and it may not be relevant from the obstetrical perspective. So one of the things we’ve done is develop this process where we can prioritize what the pregnancy-related problems are. Then the next step was we developed the process that allows those prioritized problems to get transferred into CPN.” (Physician) And finally, as providers saw the value of the data, they adapted their behaviors. For example, they realized that if they did not include the office location in the CPN system, then they would not have that information in the office. Therefore, their value of this data began to drive their behavior.

6. Conclusions While national incentives suggest EHRs will address healthcare delivery problems, there remain many challenges, particularly as these systems are integrated with existing systems and processes. We studied how implementation and then integration needs to be supported by both system and process change. This is an important

S.A. Sherer et al. / International Journal of Medical Informatics 84 (2015) 683–693

step toward understanding how to achieve benefits from integrating EHR systems. A commercial off-the-shelf EHR, which is expected to be the most common form of implementation as health networks scramble to comply with federal regulations, is challenging to integrate within existing applications and health care processes. Simply implementing a clinical documentation tool without explicit attention to work practice change can result in adaptations that may not match organizational aspirations. Traditional technology acceptance factors, especially performance and usage expectancy, are critical to understanding the initial acceptance and use of EHRs. Enabling individuals to customize the application can make the initial implementation more palatable to users concerned about standardization, but over time they begin to recognize the role of standardization for information retrieval, and subsequently, for care coordination. Structural adaptations will occur, with some unexpected consequences, especially for coordination. System and process changes are required to foster reciprocal coordination, some of which will require organizational imposition and system change. Over time, as more information is integrated, trust in the information becomes a critical factor. Time is needed in order to achieve the goal of reciprocal coordination, and continued training is critical. When multiple systems are used within a health network, integration of data from one system to another presents specific challenges. Information that is “pushed” from one system to another can be unreliable, in part because cultural differences may not be fully understood. Trust in information can be more useful than interoperability. People will use systems with which they are most familiar, where it is easiest to find the information, and when they can trust the veracity of the information. As providers begin to use EHR systems to coordinate care, and are expected to manage patients across the continuum of care, there is a need to develop more investigative and information retrieval skills. But providers are so busy that they often do not have the time to do this themselves. They need systems to more easily support information retrieval. A key challenge is determining what information is most important at specific times of care, so that this information is most prominently available. Additionally, there is a recognized need to provide care support.

691

their thoughts about and experience with the new systems as well as the valued input from colleagues at Lehigh University, Shin-Yi Chou and Mary E. Deily. We also acknowledge the help of several graduate research assistants who provided coding assistance: Sabrina Terrizzi, Kevin Byma, and Brittany Dawson. This study received financial support from the Agency for Health Research and Quality under Grant PARA-08-270. Appendix A. Interview questions

All authors have made contributions to (1) the conception and design of the study, acquisition of data, or analysis and interpretation of data, (2) drafting this article or revising it critically for important intellectual content, and (3) final approval of the version to be submitted.

1. How are you using the system(s) today? How does your work differ from past? 2. How involved were you in the rollout? Who assisted you and how? 3. Who do you go to if you have a problem with the system(s) now? 4. What has changed in your work environment? 5. Have there been any changes in relationships with others? (patients, others)? Who? How? Explain. 6. How have communication patterns changed? 7. Are there new work policies? 8. Have there been any changes in organizational structures? 9. How has your role changed? 10. What could be done differently to help you be more effective? 11. Did any changes have to be made to the system to make it more useful for you? 12. Do you have to do any workarounds to get the system to work the way you want it to? 13. Are there components of the system that you are not using because they do not match what you need? 14. Are you doing things today that you were not able to do in the past or conversely, are there things that you are not able to do today that you could do in the past? Explain. 15. Are you able to get all the information that you need? What is the source of most of your information? Do you use all the information that you get from the system? 16. Has your productivity increased/decreased? Explain. 17. Is there any information missing that could help you do your job better? 18. Are the system(s) difficult/easy to use? More or less than in the past? Are interactions more clear and understandable? 19. Do you feel that using the system(s) decreases your control or professional discretion over patient care decisions? Do you think that using the EHR will increase monitoring of your decisions by others? 20. Do you have any concerns about the system? Compare to past? 21. Do you like using the new system? Why or why not? 22. What are the advantages/disadvantages of the system over the old one? 23. Do you think that the system(s) is beneficial for you? Why or why not? How could it help you more? 24. How are you using the information from the system? Is the system providing you with exactly the same information as in the past or new information? 25. Are you able to do anything different with this information that you could not do in the past?

Competing interests

Appendix B. Examples of coded themes and demographics

“All you’re doing currently is putting data into a computer, and it spits out what you’ve put in. But there’s nothing in there to tell you that you have a problem. That’s where I see the biggest challenge that needs to be completed.” (Administration) Consequently, we expect that the users themselves will push for more sophisticated tools to support care, not just care coordination, including decision support and practice guideline tools. Author contributions

None. Acknowledgements We are grateful to the many physicians and staff members of the Lehigh Valley Health Network who shared their time to discuss

Value Efficiency or Productivity Information Accessibility Information Availability Information Validity

692

S.A. Sherer et al. / International Journal of Medical Informatics 84 (2015) 683–693

Interoperability Patient involvement/interaction Guidance/decision support Standardization Tracking health quality Reengineering Communication Culture Layout Organizational structure Policies Roles System Work processes Facilitators Age Awareness Champion/super user Documenting work flows Expectations – performance Expectations – use Feelings about change Readiness Scheduling Social Training Trust –positive User involvement Unexpected consequences Inefficient work Invalid information Information overload Information retrieval difficulties Interoperability problems Non-standard work Navigation problems Network or hardware issues Not using system Problem list difficulties Redundancy/duplication of work Sign off problems Too much effort required Trust – lack of User interface difficult Work practice incompatibility Roles Physician Resident Nurse Practice Manager IS Medical Assistant Quality Location Practice A Practice B

Summary points What was already known: • EHR adoption has had unexpected consequences. • Many sociotechnical changes accompany EHR adoption within just one facility, and they can be driven by operational, cultural, organizational and technical issues. What this study added to our knowledge: • Significant organizational, workflow, and cultural changes must take place in order to achieve the goal of coordinated care through EHR implementation and integration across multiple facilities. • It takes time for users to adjust and enculturate coordination goals, during which time there are adaptive structurations that require organizational response, and changes in mechanisms for achieving coordination.

Clinic High Risk Triage Administration References [1] R. Agarwal, G. Gao, C. DesRoches, A. Jha, The digital transformation of healthcare: current status and the road ahead, Inf. Syst. Res. 21 (4) (2010) 796–809. [2] A. Barua, P. Konana, A. Whinston, An empirical investigation of net-enabled business value, MIS Q. 28 (4) (2004) 585–620. [3] A. Barua, C. Lee, A. Whinston, The calculus of reengineering, Inf. Syst. Res. 6 (1) (1996) 409–428. [4] T. Bodenheimer, Coordinating care – a perilous journey through the health care system, N. Engl. J. Med. 358 (10) (2008) 1064–1071. [5] M. Buntin, M. Burke, M. Hoaglin, D. Blumenthal, The benefits of health information technology: a review of the recent literature shows predominantly positive results, Health Aff. (Millwood) 30 (2) (2011) 464–471. [6] B. Chaudhry, J. Wang, S. Wu, M. Maglione, W. Mojica, E. Roth, S. Morton, P. Shekelle, Systematic review: impact of health information technology on quality, efficiency, and costs of medical care, Ann. Intern. Med. 144 (10) (2006) 742. [7] M. Chiasson, E. Davidson, Taking industry seriously in information systems research, MIS Q. 29 (4) (2005) 591–605. [8] N. Clynch, J. Kellet, Medical documentation: part of the solution, or part of the problem? A narrative review of the literature on the time spent on and value of medical documentation, Int. J. Med. Inf. 84 (4) (2015) 221–228. [9] M. Davern, R. Kauffman, Discovering potential and realizing value from information technology investments, JMIS 16 (4) (2000) 121–143. [10] F.D. Davis, Perceived usefulness, perceived ease of use, and user acceptance of information technology, MIS Q. 13 (3) (1989) 319–339. [11] W. DeLone, E. McLean, Information systems success: the quest for the dependent variable, Inf. Syst. Res. 3 (1) (1992) 60–95. [12] G. DeSanctis, M.S. Poole, Capturing the complexity in advanced technology use: adaptive structuration theory, Organ. Sci. 5 (2) (1994) 121. [13] C. DesRoches, E. Campbell, S. Rao, K. Donelan, T. Ferris, A. Jha, R. Kaushal, D. Levy, S. Rosenbaum, A. Shields, D. Blumenthal, Electronic health records in ambulatory care – a national survey of physicians, N. Engl. J. Med. 359 (1) (2008) 50–60. [14] C. DesRoches, D. Charles, M. Furukawa, M. Joshi, P. Kralovec, F. Mostashari, C. Worzala, A. Jha, Adoption of electronic health records grows rapidly, but fewer than half of US hospitals had at least a basic system in 2012, Health Aff. (Millwood) 32 (8) (2013) 1478–1485. [15] C. DesRoches, C. Worzala, M. Joshi, P. Kralovec, A. Jha, Small, nonteaching, and rural hospitals continue to be slow in adopting electronic health record systems, Heath Aff. 31 (5) (2012) 1092–1099. [16] K.M. Eisenhardt, M. Graebner, Theory building from cases: opportunities and challenges, Acad. Manag. J. 50 (1) (2007) 25–32. [17] P. Embi, C. Weir, E. Efthimiadis, S. Thielke, A. Hedeen, K. Hammond, Computerized provider documentation: findings and implications of a multisite study of clinicians and administrators, J. Am. Med. Inf. Assoc. 20 (4) (2013) 718–726. [18] I. Featherstone, J. Keen, Do integrated record systems lead to integrated services? An observational study of a multi-professional system in a diabetes service, Int. J. Med. Inf. 81 (2012) 48–52. [19] M. Fleurant, R. Kell, C. Jenter, L. Volk, F. Zhang, D. Bates, S. Simon, Factors associated with difficult electronic health record implementation in office practice, J. Am. Med. Inf. Assoc. 19 (2012) 541–544.

S.A. Sherer et al. / International Journal of Medical Informatics 84 (2015) 683–693 [20] A. Friedman, J. Crosson, J.L. Howard, E. Clark, M. Pellerano, B. Karsh, B. Crabtree, C. Jaen, D. Cohen, A typology of electronic health record workarounds in small-to-medium size primary care practices, J. Am. Med. Inf. Assoc. 21 (2014) e78–e83. [21] R. Fuller, A. Dennis, Does fit matter: the impact of task-technology fit and appropriation on team performance in repeated tasks, Inf. Syst. Res. 20 (1) (2009) 2–17. [22] B. Glaser, Theoretical Sensitivity, Sociology Press, Mill Valley, CA, 1978. [23] B. Glaser, A. Strauss, The Discovery of Grounded Theory, Aldine Publishing Company, Chicago, 1967. [24] M.I. Harrison, R. Koppel, S. Bar-Lev, Unintended consequences of information technologies in health care – an interactive sociotechnical analysis, J. Am. Med. Inf. Assoc. 14 (5) (2007) 542–549. [25] D. Heisey-Grove, L. Danehy, M. Consolazio, K. Lynch, F. Mostashari, A national study of challenges to electronic health record adoption and meaningful use, Med. Care 52 (2) (2014) 144–148. [26] A. Hennington, B. Janz, Information systems and healthcare XVI: physician adoption of electronic records: applying the UTAUT model in a healthcare context, Commun. Assoc. Inf. Syst. 19. (2007) 5. [27] R. Holden, B.-T. Karsh, The technology acceptance model: its past and its future in health care, J. Biomed. Inform. 43 (2010) 159–172. [28] C. Hsiao, S. Decker, E. Hing, J. Sisk, Most physicians were eligible for federal incentives in 2011, but few had EHR systems that met meaningful-use criteria, Health Aff. (Millwood) 5 (2012) 1100–1107. [29] C. Hsiao, E. Hing, Use and Characteristics of Electronic Health Record Systems Among Office-Based Physician Practices: United States, 2001–2012. NCHS Data Brief. CDC, National Center for Health Statistics, Hyattsville, MD, 2012, pp. 111. [30] P.-J. Hsieh, Physician’s acceptance of electronic medical records exchange: an extension of the decomposed TPD model with institutional trust and perceived risk, Int. J. Med. Inf. 84 (4) (2015) 2–14. [31] A.K. Jha, C.M. DesRoches, E.G. Campbell, K. Donelan, S.R. Rao, T.G. Ferris, A. Shields, S. Rosenbaum, D. Blumenthal, Use of electronic health records in US hospitals, N. Engl. J. Med. 360 (16) (2009) 1628–1638. [32] H. Klein, M. Myers, A set of principles for conducting and evaluating interpretive field studies in information systems, MIS Q. 23 (1) (1999) 67. [33] R. Kohli, W. Kettinger, Informating the clan: controlling physicians’ costs and outcomes, MIS Q. 28 (3) (2004) 363. [34] S. Kripalani, F. LeFevre, C.O. Phillips, M.V. Williams, P. Basaviah, D.W. Baker, Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care, J. Am. Med. Assoc. 297 (8) (2007) 831–841. [35] H. Lanhan, L. Leykum, R. McDaniel, Same organization, same electronic health records (EHRs) system, different use: exploring the linkage between practice member communication patterns and EHR use patterns in an ambulatory setting, J. Am. Med. Inf. Assoc. 19 (2012) 382–391. [36] K. Locke, Grounded Theory in Management Research, Sage Publications, London, 2001. [37] D. Ludwick, J. Doucette, Adopting electronic medical records in primary care: lessons learned from health information systems implementation experience in seven countries, Int. J. Med. Inf. 78 (1) (2009) 22–31. [38] L. McQuaid, P. Breen, J. Grimson, C. Normand, M. Dunne, N. Delanty, D. Kalra, M. Fitzsimons, Socio-technical considerations in epilepsy electronic patient record implementation, Int. J. Med. Inf. 79 (5) (2010) 349–360. [39] R. Miller, I. Sim, Physicians’ use of electronic medical records: barriers and solutions, Health Aff. (Millwood) 23 (2) (2004) 116–126.

693

[40] C. Moore, J.W. Wisnivesky, S.T. McGinn, Medical errors related to discontinuity of care from an inpatient to an outpatient setting, J. Gen. Intern. Med. 18 (2003) 646–651. [41] L. Nguyen, E. Bellucci, L. Nguyen, Electronic health records implementation: an evaluation of information system impact and contingency factors, Int. J. Med. Inf. 83 (11) (2014) 779–796. [42] W.J. Orlikowski, Improvising organizational transformation over time: a situated change perspective, Inf. Syst. Res. 7 (1) (1996) 63–92. [43] W.J. Orlikowski, Using technology and constituting structures: a practice lens for studying technology in organizations, Organ. Sci. 11 (4) (2000) 404. [44] S.Y. Park, S.Y. Lee, Y. Chen, The effects of EMR deployment on doctors’ work practices: a qualitative study in the emergency department of a teaching hospital, Int. J. Med. Inf. 81 (2012) 204–217. [45] E.S. Patterson, R.I. Cook, D.D. Woods, M.L. Render, Examining the complexity behind a medication error: generic patterns in communication, IEEE Trans. Syst. Man Cybern. 34 (Part A) (2004) 749–756. [46] T.C. Powell, A. Dent-Micallef, Information technology as competitive advantage: the role of human, business, and technology resources, Strateg. Manag. J. 18 (5) (1997) 375–405. [47] C. Schoen, R. Osborn, P.T. Huynh, M. Doty, K. Davis, K. Zapert, J. Peugh, Primary care and health system performance: adults’ experiences in five countries, Health Aff. (Millwood) (2004) 487–503, Suppl Web Exclusives: W4. [48] C. Schoen, R. Osborn, P.T. Huynh, M. Doty, K. Zapert, J. Peugh, K. Davis, Taking the pulse of health care systems: experiences of patients with health problems in six countries, Health Aff. (2005) 509–525, Suppl Web Exclusives: W5. [49] W. Scott, M. Ruef, P. Mendel, C. Caronna, Institutional Change and Healthcare Organizations: From Professional Dominance to Managed Care, University of Chicago Press, Chicago, IL, 2000. [50] G. Ser, A. Robertson, A. Sheikh, A qualitative exploration of workarounds related to the implementation of electronic health records in early adopter mental health hospitals, PLOS ONE 9 (1) (2014). [51] S. Sia, C. Soh, An assessment of package-organisation misalignment: institutional and ontological structures, Eur. J. Inf. Syst. 16 (5) (2007) 568. [52] A. Strauss, J. Corbin, Basics of Qualitative Research, Sage, Thousand Oaks, CA, 1998. [53] D. Strong, O. Volkoff, Understanding organization-enterprise system fit: a path to theorizing the information technology artifact, MIS Q. 34 (4) (2010) 731–756. [54] S. Taylor, P.A. Todd, Understanding information technology usage: a test of competing models, Inf. Syst. Res. 6 (4) (1995) 144–176. [55] J.D. Thompson, Organizations in Action; Social Science Bases of Administrative Theory, McGraw Hill, New York, 1967. [56] B. Tulu, R. Burkhard, T. Horan, Information systems and health care XIV: continuing use of medical information systems by medical professionals: empirical evaluation of a work system model, Commun. Assoc. Inf. Syst. 18 (2006) 641–656. [57] V. Venkatesh, F.D. Davis, A theoretical extension of the technology acceptance model: four longitudinal field studies, Manag. Sci. 45 (2) (2000) 186–204. [58] V. Venkatesh, M. Morris, G. Davis, F. Davis, User acceptance of information technology: toward a unified view, MIS Q. 27 (3) (2003) 425–477. [59] R.K. Yin, Case Study Research: Design and Methods, 4th edition, Sage Publications, Los Angeles, 2009. [60] P. Yu, Y. Zhang, Y. Gong, J. Zhang, Unintended adverse consequences of introducing electronic health records in residential aged care homes, Int. J. Med. Inf. 82 (2013) 772–788.