Computers in Biology and Medicine 60 (2015) 40–50

Contents lists available at ScienceDirect

Computers in Biology and Medicine journal homepage: www.elsevier.com/locate/cbm

Mobilizing clinical decision support to facilitate knowledge translation: A case study in China Yinsheng Zhang a,c, Haomin Li b,n, Huilong Duan a, Yinhong Zhao d a

College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou 310027, China Children's Hospital, Institute of Translational Medicine, Zhejiang University, Hangzhou 310027, China c School of Computer Science and Information Engineering, Zhejiang Gongshang University, Hangzhou 310018, China d China National Center for Biotechnology Development, Beijing 100036, China b

art ic l e i nf o

a b s t r a c t

Article history: Received 4 August 2014 Accepted 14 February 2015

Background: A wide gulf remains between knowledge and clinical practice. Clinical decision support has been demonstrated to be an effective knowledge tool that healthcare organizations can employ to deliver the “right knowledge to the right people in the right form at the right time”. How to adopt various clinical decision support (CDS) systems efficiently to facilitate evidence-based practice is one challenge faced by knowledge translation research. Method: A computer-aided knowledge translation method that mobilizes evidence-based decision supports is proposed. The foundation of the method is a knowledge representation model that is able to cover, coordinate and synergize various types of medical knowledge to achieve centralized and effective knowledge management. Next, web-based knowledge-authoring and natural language processing based knowledge acquisition tools are designed to accelerate the transformation of the latest clinical evidence into computerized knowledge content. Finally, a batch of fundamental services, such as data acquisition and inference engine, are designed to actuate the acquired knowledge content. These services can be used as building blocks for various evidence-based decision support applications. Results: Based on the above method, a computer-aided knowledge translation platform was constructed as a CDS infrastructure. Based on this platform, typical CDS applications were developed. A case study of drug use check demonstrates that the CDS intervention delivered by the platform has produced observable behavior changes (89.7% of alerted medical orders were revised by physicians). Discussion: Computer-aided knowledge translation via a CDS infrastructure can be effective in facilitating knowledge translation in clinical settings. & 2015 Elsevier Ltd. All rights reserved.

Keywords: Knowledge translation Clinical decision support Knowledge representation Knowledge acquisition Inference engine Context-aware knowledge retrieval Natural language processing

1. Introduction 1.1. The knowledge-practice chasm Globally, healthcare organizations fail to use evidence optimally. In 1973, Wennberg and Gittelsohn published their landmark article demonstrating substantial variation among different healthcare service providers [1]. Forty years later, this situation has not changed. A large gulf remains between what we know and what we practice. One study in 2003 showed that only 54.9% of US patients received evidence-based care [2]. Similar findings have been reported globally in both developed and developing countries [3]. Several reasons have caused this knowledge-practice gap. First, knowledge grows exponentially. It is estimated that each year 10,000

diseases, 3000 drugs, and 400,000 articles are added to the biomedical domain [4], and every 19 years, the volume of the literature in the biomedical domain is doubled [5]. This explosive knowledge growth has inflicted a heavy cognitive load on clinicians. Second, medical knowledge itself is evolving. Former conclusions can be overturned by the latest evidence. According to a 2001 survey, 50% of guideline knowledge becomes invalid in 5.8 years, and the validity period of 90% guidelines is only 3.6 years [6]. The dynamic nature of knowledge inflicts great pressure on physicians to respond to the latest clinical evidence in a timely manner. Third, the knowledge translation course is long. The course could take as long as 10 years for newly discovered knowledge to enter textbooks [7], and it takes 10–17 years for new knowledge to be transformed into routine practice [8]. 1.2. Clinical decision support as a knowledge tool to facilitate knowledge translation

n

Corresponding author. Tel.: þ 86 571 87951792. E-mail addresses: [email protected] (Y. Zhang), [email protected] (H. Li), [email protected] (H. Duan), [email protected] (Y. Zhao). http://dx.doi.org/10.1016/j.compbiomed.2015.02.013 0010-4825/& 2015 Elsevier Ltd. All rights reserved.

To close the chasm between knowledge and practice, knowledge translation (KT) in healthcare has been widely recognized and studied

Y. Zhang et al. / Computers in Biology and Medicine 60 (2015) 40–50

CDS1

KR1 EMRS

CDS2

CDS3

EMRS

2. Methods As mentioned above, a CDS infrastructure is a promising alternative to traditional CDS systems in facilitating knowledge translation (Fig. 1). The computer-aided knowledge translation process enabled by computerized decision support tools can be summarized as three procedures: 1. Knowledge representation. Before medical knowledge can be used in various computerized decision support systems, it needs to be represented in a computer-interpretable format. Knowledge representation determines the knowledge base schema and what types of computerized decision supports can be provided. 2. Knowledge acquisition. Knowledge acquisition transforms medical knowledge embedded in the literature and textbooks into computer-interpretable formats determined by knowledge representation formalisms. This procedure is usually performed by knowledge engineers. 3. Knowledge application. Computer-interpretable knowledge contents are applied in healthcare decision making in the form of computerized decision support. To support knowledge translation better, this paper aims to design a CDS infrastructure that could benefit all of the above three procedures. Implementation of such an infrastructure would entail several distinctive characteristics compared with traditional CDS systems. First, the infrastructure needs a general-purpose knowledge

CDS2

CDS3

KE1 vendor2 KE2

KB3 vendor3 KR3

CDS1

vendor1

KB2 KR2

1.3. Needs of CDS mobilization infrastructure Traditional CDS systems are developed as independent systems by different vendors. They usually target a specific application (such as diagnostic CDS [20], lab test surveillance and alerting [21,22], or drug use checking [23]) and use private knowledge representation and implementation methods. This has led to fragmented, disintegrated and inconsistent knowledge, therefore introducing complexity in developing CDS front ends [24] and managing back-end knowledge content [16]. Because of such heterogeneity, coordinated knowledge content and unified knowledge management cannot be easily achieved in healthcare organizations [25]. As an amendment to traditional CDS systems, the 2000 AMIA symposium recommended “maintainable technical and methodological foundations for computer-based decision support” as a key factor for implementing “evidence-based (i.e., based on timely and latest knowledge)” CDS [26,27]. Therefore, the mission of this study is to design and construct a mobilization infrastructure or foundation where computerized clinical decision support is used as the major knowledge tool that facilitates knowledge translation. The infrastructure should be able to provide centralized knowledge management and efficient knowledge acquisition such that the rapidly changing clinical evidence can be quickly and effectively applied to computerized decision support.

KB1

Knowledge Execution Environment

in the past decade. According to the Canadian Institutes of Health Research (CIHR), KT has been defined as “a dynamic and iterative process that includes the synthesis, dissemination, exchange and ethically sound application of knowledge to provide more effective health services and products and strengthen the healthcare system” [9]. Because knowledge accumulates and evolves rapidly, KT is constantly faced with the challenge of synthesizing the vast body of highly dynamic knowledge content and utilizing it at the point of care [10]. To meet this challenge, healthcare organizations need effective knowledge management tools to organize and utilize the vast knowledge content to maintain competitive performance in such a highly dynamic environment [11]. CDS (clinical decision support) is such a tool that a healthcare organization can employ to deliver the “right knowledge to the right people in the right form at the right time” [12]. Many studies [13–19] have demonstrated CDS's capability to improve evidence-based practice and facilitate knowledge translation.

41

KE3

CDS Infrastructure

KB KE KR

Fig. 1. Computer-aided knowledge translation process by computerized decision support tools. (A) Knowledge translation facilitated by traditional CDS systems deployed in a healthcare organization. (B) Knowledge translation facilitated by a CDS infrastructure. KR¼ knowledge representation; KB ¼ knowledge base; KE ¼ knowledge engineer.

representation model to cover, coordinate and synergize various types of medical knowledge in computer-interpretable forms, which lays the foundation for centralized knowledge management and subsequent knowledge dissemination via computerized decision support services [28,29]. Second, the infrastructure should provide highly efficient knowledge acquisition and authoring tools to reduce the knowledge acquisition time and cost. Knowledge acquisition is a time-consuming process and can take as much as 50% of the total effort for building a CDS system [30]. Efficient and timely knowledge acquisition ensures that the provided CDS intervention is based on the latest clinical evidence. Third, the infrastructure should provide a mechanism to transfer acquired knowledge contents to evidencebased decision supports efficiently. To fulfill the above requirements, this paper designs a computeraided knowledge translation framework as such a CDS infrastructure. As shown in Fig. 2, the framework is composed of five components. The first component is knowledge representation, which defines a unified ontology to cover and coordinate multiple knowledge types. The second component is knowledge acquisition, where knowledge content is acquired and authored using computer-aided methods. The third component is the knowledge base, which stores various types of knowledge and their inter-relationships as defined by the unified ontology. The fourth is a layer of knowledge-driven services such as the inference engine, treatment recommendation, and context-aware knowledge retrieval, which are used as fundamental service modules by diverse CDS applications. The fifth is a layer of CDS applications that interact with end users. The above framework covers the three procedures (knowledge representation, knowledge acquisition and knowledge application) of computer-aided knowledge translation. The framework supports knowledge translation by the following workflow (Fig. 2). 1. Design a knowledge representation model suited for knowledge translation, and construct the corresponding knowledge base. 2. Knowledge content is continuously synthesized, curated and maintained by an expert panel through a knowledge-authoring web portal. 3. In addition to the aforementioned authoring web portal,

42

Y. Zhang et al. / Computers in Biology and Medicine 60 (2015) 40–50

LIS CDR

do free text c

RIS

CIS

Clinical Decision Support Applications Drug use Other CDS Diagnostic CDS checking applications

5

Integration Engine

14

physicians 13

4 6

8

Data Acquisition 7

Knowledge Driven Services Treatment Inference Engine Recommendation 9

Pharm Drug use rules ADE DDD

NLP Lexicon Algorithms Terminologies

Knowledge Base

Diagnosis Clinical problem Diagnostic rules Facts

1

Unified Knowledge Ontology Knowledge Representation

10

11

Knowledgeauthoring web portal

Application Store 12

Treatment Literature Standard order sets Guidelines Clinical pathways Systematic review Drug fact sheets 2

Expert Panel Clinical experts Pharmacists Medical insurance staff

Context-aware knowledge retrieval

Hardcoded Dosing advisor Formulary support Risk evaluator

3

NLP-based knowledge acquisition

Knowledge Acquisition

Public Knowledge Drug labels Cochrane reviews MOH clinical pathways

Fig. 2. Computer-aided knowledge translation framework. Each number in the figure represents a data communication: (1) database schema according to unified knowledge ontology, (2,3) knowledge content, (4) clinical event, (5,6) clinical data, (7) NLP task, (8) facts (structured clinical data), (9) drug use rules and diagnostic rules, (10) care protocol, (11) human-readable literature, (12) computer-aided tools, (13) fundamental services for building CDS applications, (14) CDS interventions.

computer-aided knowledge acquisition is enabled by technologies such as NLP (natural language processing), which extracts structured content from free text resources. 4. An integration engine [31] which connects various information systems and routes messages between them acts as an ideal clinical event source for the inference engine. For example, when a new lab test report is generated in LIS (laboratory information system), the integration engine will notify the inference engine to start a reasoning process. 5. A CDR (clinical data repository), which aggregates various clinical data through the integration engine acts as a central data source. 6. A data acquisition service retrieves both structured and free text data from CDR for various CDS applications. 7. NLP tasks for specific types of clinical documents can be defined to extract structured data from free text. 8. Structured data provided by the data acquisition service are fed into the inference engine as facts. 9. Drug use rules or diagnostic rules are loaded into the inference engine to generate drug alerts or clinical problems. 10. Care protocols associated with a detected clinical problem are recommended as normalized interventions. 11. HL7 Infobutton based context-aware knowledge retrieval provides human-readable knowledge for CDS interventions to improve user acceptance. 12. An application store that manages registered computer-aided tools pushes and executes hardcoded knowledge applications under specific contexts by a plug-in technical framework [32]. 13. The above services can be co-delivered in a specific sequence to provide collaborative decision support. For example, the data acquisition, inference engine and treatment recommendation services can be orchestrated to implement the “patient data, diagnosis, treatment” decision support process. 14. Various CDS applications enabled by the above services interact with physicians to improve decision making. In the following manuscript, we will introduce each component of the framework in detail and explain how they assist knowledge translation through the above workflow. 2.1. Knowledge representation As mentioned before, the heterogeneity of knowledge representation in traditional CDS systems has limited their ability to provide full-scale collaborative knowledge services. To address this problem, we constructed a unified ontology that incorporates various types of knowledge as well as their semantic relationships,

as shown in Fig. 3. The ontology contains the following knowledge categories: The diagnostic knowledge category defines diagnostic rules and related concepts for clinical problems. Context items (e.g., systolic blood pressure and diastolic blood pressure) that are classified into several semantic groups (e.g., lab test, vital sign and symptom) are used to define diagnostic rules (e.g., systolic blood pressure4140 mmHg). An inference engine processes the rules to drive diagnostic decision support. The treatment knowledge category defines recommended care protocols for clinical problems, such as standard order sets and clinical pathways. Care protocols are designed by clinical experts based on the latest evidence. In this ontology, the clinical pathway is modeled as a multi-phase care plan, while a standard order set is a single-phase care plan. Each phase of the care plan contains a predefined order set such as nursing, diet, medication, lab test request, and surgery request. The pharmaceutical knowledge category defines various drug use rules, such as dosage, frequency, administration route, solvent, indication, contraindication, allergy, pregnancy class, DDI (drug–drug interactions), and insurance policy, for medication-related decision support. Other pharmaceutical resources, such as ADR (adverse drug reactions) and DDD (defined daily doses) are also included. The above three categories are formally represented knowledge. However, there also exists much knowledge directly hardcoded in various clinical applications where explicit knowledge is not available. Traditionally, such hardcoded knowledge is always excluded from knowledge management. In our approach, the hardcoded knowledge embedded in various computer-aided tools can also be managed and linked to other knowledge. In this way, the hardcoded knowledge serves as a supplement to formally represented knowledge. For example, a personalized drug dosing tool can be linked to medication orders in a care protocol. When the care protocol is applied, the tool will be invoked at the point of care to fulfill personalized medication. The primary goal of including hardcoded knowledge is to overcome the limitations of the formal knowledge representation's scalability and capability [33]. In this computer-aided knowledge translation framework, these tools are managed by an application store service (refer to “Section 2.4.5”) and are integrated into the clinical workflow by a plug-in mechanism [32].

Y. Zhang et al. / Computers in Biology and Medicine 60 (2015) 40–50

dosing advisor

Domain Knowledge

nt

Formally Represented Knowledge

is a

is a ha s

drug use rule

order set checks

medication

constrain

allergy

constrain

frequency

s

ha

lab test result

phase

route in

is a

s

vital sign

a is

is a

symptom

dosage

clinical pathway

ha

clinical problem

context item

is a

has

other external applications

evaluation tools

is a

protocol has

is a

str a

metaanalysis

a is

instance of

ple sup

is a

diagnostic rule

fact

3rd-party / legacy tools

co n

a is

system review

is a

Hardcoded Knowledge

a is

is a

is a

me

a is

clinical practice guideline

Machinedoable Knowledge

reference

constrain

Humanreadable Knowledge

is a

formulary support

is a

is a

is

a

a is

drug fact sheet

43

include

medical order

include

test request

Fig. 3. A unified ontology that contains various types of knowledge.

Both formally represented knowledge and hardcoded knowledge are “machine-doable knowledge”, which are based on “human-readable knowledge”, such as clinical practice guidelines, meta-analyses, systematic reviews, and randomized clinical trials. Machine-doable knowledge is processable by computers and makes knowledge application automatic, efficient and without bias. The human-readable knowledge explains what happened in black boxes and can effectively improve user acceptance [34]. Moreover, the adverse effect of inappropriately complied knowledge could be suppressed if human-readable knowledge is available at point of care. Both machine-doable knowledge and humanreadable knowledge can be co-delivered to clinicians. This ontology not only defines various knowledge types but also the semantic relations between them. Three relations that coordinate these knowledge types should be emphasized: 1. The clinical problem acts as an intermediary between diagnostic and treatment knowledge. When a clinical problem is detected by a diagnostic rule, corresponding care protocols can be recommended. 2. Hardcoded knowledge “supplements” formally represented knowledge. 3. Machine-doable knowledge “references” human-readable knowledge. The above semantic relations link different knowledge categories and provide synergetic knowledge resources for various clinical applications. In summary, the unified ontology defines what types of knowledge are incorporated and managed. It determines the knowledge base schema and lays the foundation for computer-aided knowledge translation. 2.2. Knowledge acquisition For reliable knowledge acquisition, the involvement of clinical experts is required. In our collaborator hospital (DaYi hospital, 2600 beds, ShanXi Province, China), an expert panel was established under the supervision of the hospital director. The expert panel currently contains 35 clinicians, 6 pharmacists, and 1 medical insurance staff member. The clinicians were selected from 35 clinical departments (one for each department). They are responsible for maintaining clinical problems, literature, order sets, and clinical pathways used in each specialty. Pharmaceutical knowledge, such as drug use rules and drug labels are maintained by the pharmacists. One medical

insurance staff member was also recruited for editing insurance policies, such as required indications for specific drugs. Two knowledge acquisition modes are provided to help experts efficiently create and update knowledge content. The first is a knowledge authoring web portal and the other is computer-aided knowledge acquisition, which extracts computer-processable knowledge from public resources by technologies such as Chinese language oriented NLP. The knowledge-authoring web portal is a knowledge acquisition tool that acts as a bridge between humans and the knowledge base. These tools usually provide structured data entry templates or forms for experts to create computerized knowledge content. Fig. 4 shows the knowledge-authoring web portal developed using ASP.Net MVC technology. The web portal is hosted on an intranet IIS server, and content contributors can access the portal by web browser. The web portal supports management of following knowledge types: clinical problems (Fig. 4A and B), care protocols (Fig. 4C), literature, pharmacy, and hardcoded knowledge. Hardcoded knowledge includes external computer-aided tools, such as dosing advisors and risk evaluators, which can be registered to specific contexts in the clinical workflow. For example, they can be registered to specific medical orders in a care protocol, as shown in Fig. 4D. When physicians use the care protocol, the corresponding tools will be automatically launched. In addition to knowledge-authoring via the web portal, it is now a trend to automatically extract knowledge from narrative resources [35,36]. In this study, we designed an NLP system that depends on a Chinese medical lexicon. The lexicon is maintained by the knowledgeauthoring web portal (Fig. 5). With the help of this NLP system, we have extracted 39,324 drug-ADR associations from 3683 Chinese drug fact sheets [37]. The details of this study are provided in Supplement-I. 2.3. Knowledge base For constructing a knowledge base, this paper employed the Entity Framework [38] to generate a knowledge base schema from an ontology model. Entity Framework is Microsoft's implementation of ORM (object relation mapping) [39] technology, which enables mapping and conversion between an object-oriented model and a relational database schema. We used the visual model editor provided by Entity Framework to construct the unified ontology. Then, the

44

Y. Zhang et al. / Computers in Biology and Medicine 60 (2015) 40–50

Fig. 4. Knowledge-authoring web portal. The web portal supports creating various knowledge types, such as (A) clinical problems, (B) diagnostic rules, and (C) care protocols. D is a dialog of computer-aided tools that can be registered to a specified medical order in the care protocol.

corresponding SQL server database schema was automatically generated from the ontology. We use this database as the persistent storage for our knowledge base. The ontology model and database schema are provided in Supplement-V. Until now, through the above knowledge acquisition approaches, 27,960 clinical problems from ICD-10 (Chinese version), 463 standard care protocols created according to clinical practice guidelines and China MOH (Ministry of Health) clinical pathways [40], 5378 medical articles from public resources such as China Guideline Clearinghouse, 1986 drugs from the hospital's pharmacy system, 490 DDIs (drug-drug interactions) from related work [41], 161 drug DDDs (defined daily

doses) from WHO ATC/DDD Index, and 39,324 drug-ADR associations from drug labels have been maintained in the knowledge base. In addition, 11 computer-aided tools were collected and uploaded, including dosing advisor, formulary support, risk evaluator, BMI (body mass index) calculator, BSA (body surface area) calculator, and CrCl (creatinine clearance) calculator. 2.4. Knowledge driven services To apply the knowledge content in computerized decision support efficiently, a “knowledge execution environment” is needed to actuate

Y. Zhang et al. / Computers in Biology and Medicine 60 (2015) 40–50

45

Fig. 5. Editing concepts and relationships in the Chinese language lexicon. In the editing area, a concept was defined by its name, semantic type, coding, value type and clinical significance. Each concept was allowed to be assigned with several terms, which could be synonyms, abbreviations, and frequent typos. A pop-up dialog box that provided semantic relations defined in UMLS was used to construct relationships around this concept. The concepts and relationships also constitute a shared domain ontology among different systems, such as the inference engine and data acquisition service.

the knowledge content in the knowledge base (Fig. 1). As different knowledge types in the unified knowledge ontology use different representation formulisms, such as the rule syntax used by diagnostic rules and the multi-phase data structure used by care protocols, it is necessary to provide corresponding reasoning techniques or processing methods [42]. As a consequence, the following services are provided to actuate the knowledge content. 2.4.1. Data acquisition In clinical settings, both structured data and narrative text exist. A task-specific NLP framework was proposed to extract structured data from narrative text (Fig. 6). The framework allows a user to define specific NLP tasks for different data requirements. For each NLP task, the user can subset the homegrown domain ontology to form a task-specific lexicon composed of only task-relevant concepts and relationships. The user can also specify a targeted document type and NLP algorithms for the task. At runtime, the integration engine acts as a task monitor and notifies the NLP system when a clinical document or report was generated in the clinical setting. Then the corresponding NLP task will extract structured concepts and values to drive the subsequent services. A related case study using this NLP framework is provided in Supplement-II. 2.4.2. Inference engine A rule-based inference engine has been developed based on ANTLR [43] (ANother Tool for Language Recognition). The engine supports diagnostic rules, such as “If ([Glucose]47.0 || [OGTT120'] 411.1) then [Diabetes]”. To help clinicians edit rules, a web-based GUI

rule editor (Fig. 4 B) is provided in the knowledge-authoring web portal. In this specific implementation, the inference engine depends on a shared domain ontology between external data providers, such as CDR and NLP. This partly solves the difficulty of mapping external clinical data to referenced variables in rule expression, which is a common problem faced by existing rule engines [44]. The rule engine is published as open source and can be found on the project home page. Meanwhile, we are considering using Drools [45] as a more promising rule engine for future requirements, which supports features such as rule chaining, decision table, DSL (domain specific language) extension and temporal reasoning. 2.4.3. Treatment recommendation Care protocols, such as order set and clinical pathway, are evidence-based therapeutic treatment plans. Collecting, organizing, and distribution of these interventions are important for the meaningful use of CDS [46]. Experts from different specialties can use the knowledge-authoring web portal to define care protocols (Fig. 4C), and associate them with corresponding clinical problems. Such protocols can be recommended to physicians when specific clinical problems are detected (Fig. 7G). The knowledge-authoring web portal uses the same medical order dictionaries of EMRS, so the created care protocols can generate native medical orders in CPOE (computerized physician order entry) without term conversion. 2.4.4. Context-aware knowledge retrieval For context-aware retrieval of human-readable literature, both the local knowledge base and external knowledge sources are

46

Y. Zhang et al. / Computers in Biology and Medicine 60 (2015) 40–50

Fig. 6. Task specific NLP framework. The user defines NLP tasks based on diverse CDS data requirements and enrolls new concepts to complement the homegrown domain ontology. In each NLP task instance, targeted concepts will be extracted from specific documents and fed to CDS tasks.

used. The local knowledge base contains 5378 articles uploaded through the knowledge-authoring web portal, including 2352 drug labels and 2822 clinical practice guidelines collected from the Internet, 126 MOH-published clinical pathways [40], and 78 drug adverse event reports from the SFDA (China State Food and Drug Administration) [47]. Such local literature can be tagged to machine-doable knowledge, e.g., diagnostic rules and drug use rules. In addition, the integration of external reference sources is supported, including both HL7 Infobutton standard [48] compliant sources (such as BMJ Best Practices, MedlinePlus, UpToDate, Merck Manual and Micromedex) and non-HL7 compliant knowledge sources (such as Cochrane Reviews, FDA DailyMed, PubMed, China Guideline Clearinghouse [49] and China Cochrane/Evidence Based Medicine Center [50]). To handle various external knowledge sources, an Infobutton manager was developed to translate an Infobutton query into a resource-specific SOA or URL request. Until now, 6 HL7 compliant sources and 22 non-HL7 compliant sources have been integrated. We keep the integration of external knowledge sources as an important feature of this platform because we believe shareable and public knowledge bases will prevail in the near future. The knowledge sources are managed through the knowledge-authoring web portal (〈http://www.cktp.org:8006/ KnowledgeSource〉).

2.4.5. Application store Legacy decision support systems and various third-party applications have become commonplace within the medical field as both personal and professional tools [51]. Typical applications include dosing advisor, formulary support and risk evaluator. These tools assist physicians with decision making in daily activities. This platform

designed an application store service that allows users to upload, manage and register (Fig. 4D) such tools through the knowledgeauthoring web portal. In runtime, the application store will push corresponding tools to the specific context and execute them through a plug-in mechanism [32]. 2.5. Clinical decision support The above knowledge driven services can be used as building blocks for CDS applications. Based on these services, clinical decision support applications (Fig. 7) have been developed to effectively disseminate the knowledge content to influence healthcare decision making. Two approaches are used to disseminate diverse formatted knowledge to clinicians by computerized decision support: The first one is the on-demand approach, in which users query or trigger knowledge positively. All human-readable knowledge can be disseminated in this way via context-aware knowledge retrieval (C and D in Fig. 7) from both local and external knowledge sources. Similarly, hardcoded knowledge in the form of computer-aided tools can also be downloaded and triggered by the user via the application store service that plays the role of “Apple App Store” or “Android App Market” in EMRS. For example, E in Fig. 7 shows a micropump speed calculator invoked by the user. The other is a proactive approach in which data drives the knowledge dissemination. The formally represented knowledge such as diagnostic rules and drug use rules has the potential to be implemented in this way. Notified by clinical events and newly generated data, the inference engine fetches and executes related rules from the knowledge base. Then the inference results are pushed to users as alerts (A and B in Fig. 7) or clinical problems (F in Fig. 7). Associated care protocols (G in Fig. 7) are also provided as actionable

Y. Zhang et al. / Computers in Biology and Medicine 60 (2015) 40–50

interventions that can be directly applied in the order entry workbench. One challenge for this approach is that much of the information needed to actuate the rules is recorded in narrative text. The aforementioned task-specific NLP framework (Fig. 6) was designed to address this challenge. With this NLP framework, several free-text driven CDS applications have been developed, such as extracting medication and adverse events from clinical documents for ADE (adverse drug event) detection and extracting fetal biometry measurements (e.g., biparietal diameter and femur length) from pregnancy ultrasound reports (refer to Supplement-II).

47

drug use check CDS, and free-text driven CDS. Infobutton is another frequently used CDS application. Among the above KTP-based clinical decision support applications, some (e.g., drug use check) are able to produce observable physician behavior changes, while others (e.g., Infobutton) help to alter awareness, attitudes, and knowledge, but did not necessarily influence behavior. Considering the variety of knowledge types and decision supports covered by KTP, it is not easy to evaluate the entire platform as a whole. However, we are able to do evaluation by specific use cases. The system log shows that drug use check is frequently used by clinicians and can be analyzed as a typical case study of the platform.

3. System implementation and case study 3.2. Case study 3.1. System implementation According to the above framework, we initiated a knowledge translation platform (KTP) project in DaYi Hospital. KTP was deployed in the hospital in September 2013. As seen from Table 1, KTP has been extensively used through various CDS applications in clinical settings. During the period from 2013/09/ 01 to 2014/10/01, KTP processed 1,345,364 clinical events notified by the integration engine, and retrieved 2,427,133 facts using the data acquisition service. These events and acquired clinical data were further used by the inference engine to drive diagnostic CDS,

The drug use check CDS provides alerts in two modes: asynchronous notification (A in Fig. 7) and synchronous message (B in Fig. 7). In the current phase, synchronous mode was implemented, and the basic rules were supported, including dosage, administration route, frequency and skin test. During the period from 2013/12/25 to 2014/ 10/01, 9596 alerts were detected from 2,667,016 medical orders (1,113,705 were medication orders) by the drug use check. Among the 9596 alerts generated, we need to find out how many medication orders were actually corrected. A corrected order is a direct proof that a physician's behavior was influenced by the

Fig. 7. CDS applications in CPOE. (A) Notification area for clinical events, such as drug alert and ADT (admit, discharge, transfer); (B) drug alert message; (C) Infobutton for drug; (D) retrieved drug label by Infobutton; (E) micropump speed calculator as a hardcoded knowledge application; (F) clinical problems generated by diagnostic CDS; (G) recommended care protocols. This screen cut was produced by the Chinese CPOE system in DaYi hospital. English readers can refer to Supplement-VI for a more detailed description.

48

Y. Zhang et al. / Computers in Biology and Medicine 60 (2015) 40–50

Table 1 Log data collected from the knowledge translation platform. Log items

Period

Count

Description

Inpatient visits

2013/09/01– 2014/10/01 2013/09/01– 014/10/01 2013/09/01– 2014/10/01 2013/09/01– 2014/10/01 2013/12/25– 2014/10/01 2013/09/01– 2014/10/01

27,179

Inpatient visits enrolled by the new EMRS that is supported by KTP

Clinical events notified by the integration engine Facts retrieved by the data acquisition service Clinical problems generated by the inference engine Drug alerts generated by the inference engine Infobutton queries

1,345,364 Including 8 event types, such as lab test (trigger diagnostic CDS), medical order submission (trigger drug check use), and clinical document (trigger free-text driven CDS), etc 2,427,133 Facts retrieved from clinical events by the data acquisition service 1621

In this period, 20 diagnostic rules are defined

9596

In this period, the basic rules such as dosage, administration route, frequency and skin test are supported

23,883

User queries for drug labels and clinical practice guidelines through the Infobutton

n

More details on the usage data are provided in Supplement-III.

CDS intervention. To measure how many alerts were actually corrected, we carried out a retrospective analysis that applied the same rule set to the 2,667,016 medical orders. In this analysis, if an alert was overridden by the physician and the corresponding medication order was not modified, the same drug alert can still be detected using the same rules. In the end, we found that 987 alerts remained. That means that 8609 (9596-987) alerts were corrected by physicians. The 89.7% (8609/9596) acceptance rate indicates that the CDS intervention provided by KTP significantly influenced clinicians' behavior, i.e., led to the modification of medical orders. Because these CDS interventions are backed by drug rules that are constantly updated and curated by pharmacists, such intervention acts as an efficient knowledge tool that delivers the latest maintained evidence. In this sense, KTP has successfully assisted in translating evidence-based rules into observable clinician behavior changes. This demonstrates the capability of KTP in facilitating knowledge translation through evidence-based decision support tools. The detailed report and data set of this case study can be found in Supplement-IV.

4. Discussion The primary contribution of this study is a computer-aided knowledge translation framework that acts as a CDS mobilization infrastructure. The framework can also be extended to more hospitals and scenarios. Many modules and knowledge assets in the framework can be customized according to different local requirements. The following are some examples: 1. The current inference engine can be replaced or extended by other engines such as Drools. 2. Through a customizable lexicon and algorithms, the NLP system (Fig. 6) is expected to handle various situations, i.e., different domains or languages. A secondary contribution is a scalable and extensible knowledge management mechanism. With the knowledge-authoring tool, the knowledge content can be periodically updated to include the latest evidence and tailored to local settings, which makes dependent decision support flexible and adaptive. Through this mechanism, clinicians are both authors and users of the knowledge content. They can form an autonomous and collaborative community where knowledge content is continuously improved by a “create, use, evaluate, revise” loop. A third contribution is the integration and management of the hardcoded knowledge by the application store service. This enhances the reusability of legacy and external tools. By the knowledge-authoring web portal, users can upload various computer-aided tools into the knowledge base and register them for specific contexts. In this way, such hardcoded knowledge is integrated into the clinical workflow and can be invoked in a context-aware style. The inclusion of hardcoded knowledge into the unified ontology also extended the knowledge representation (KR) capability. As discussed in the article “Why we need

many knowledge representation formalisms” [42], different KRs serve specific needs and inference methods, therefore enabling more powerful artificial intelligence. The inclusion of hardcoded knowledge fulfills the necessity of the coexistence of various KR methods. One point that readers should bear in mind is that technology “assists” but will not “substitute” for humans [52]. The knowledge translation platform provides evidence-based CDS in EMRS to assist physicians in making better decisions. CDS interventions (such as drug alerts) are informative, rather than mandatory. Moreover, the platform supports the co-delivery of relevant human-readable literature with CDS interventions, so that users have more references for consideration at the point of care. This also helps to prevent overreliance on the system. There are several limitations to this study. First, a more systematic evaluation methodology is needed for this knowledge translation platform. One challenge in evaluation is that for certain scenarios it is difficult to determine whether user altered their behavior solely because of the provided CDS intervention. For example, after a new clinical problem is detected, the physician may take action, such as creating new medical orders or revising existing medical orders. However, there is no straightforward means to measure to what degree the action was influenced by CDS. To address this problem, we are considering several evaluation approaches, such as an observational method [53] (i.e., when a CDS alert is fired, if such corresponding behavioral changes are observed, knowledge translation is regarded as successful), KPI (key performance indicator) comparisons, and questionnaires. Second, in the current implementation of KTP, lack of standardization can be an obstacle for sharing knowledge content among different healthcare organizations. For example, the referenced variables in diagnostic rules and the medical orders defined in care protocols are based on the hospital's private terminologies. Although using institution-specific terminologies saves the efforts of data mapping between KTP and external systems, it makes the knowledge content difficult to be used and shared across different hospitals. A viable solution may be using standard terminologies for knowledge-authoring, and publishing the knowledge content as an open resource. Each hospital can use a terminology service to map the standardized knowledge content to their local terminologies. In the end, readers should be aware that knowledge translation (KT) is not merely a technical issue. The KTP proposed in this paper is not intended to be an “all-around” solution. As mentioned in [54], KT is a complex process that involves physicians' cognitive and behavioral changes. It is influenced by various psychological, social, economic, legislative and regulatory factors. Because KT is such a complicated and broad subject, the topic covered by this study is just “the tip of the iceberg”, and more research must be carried out in the future, such as integrating more types of knowledge. In recent years, genomics knowledge and high-throughput testing technology have gradually been applied in clinical practice [55]. This type of knowledge is also

Y. Zhang et al. / Computers in Biology and Medicine 60 (2015) 40–50

faced with the urgent pressure for knowledge translation [56,57]. The inclusion of such knowledge can further extend the capability of KTP in guiding personalized diagnosis and drug use.

5. Conclusion Translating knowledge into clinical practice has always been an urgent need for healthcare organizations. In this paper, we report our effort to design a computer-aided knowledge translation framework that uses computerized decision support as a knowledge tool to improve evidence-based practice. This report describes a pioneer study in China, and we hope it can promote and inspire related research in more hospitals.

Conflict of interest None declared.

Acknowledgment This study was supported in part by the National High-tech R&D Program of China (2012AA02A601) and the National Natural Science Foundation of China (30900329). Sponsors have no role in this study that could have influenced its outcome. Many thanks to the journal editor and reviewers who have provided us with valuable advice and helped us to substantially improve the manuscript.

Appendix A. Supporting information Supplementary data associated with this article can be found in the online version at http://dx.doi.org/10.1016/j.compbiomed.2015.02.013. References [1] J.E. Wennberg, A.M. Gittelsohn, Small area variations in health care delivery, Science 182 (1973) 1102–1108. [2] E.A. McGlynn, S.M. Asch, J. Adams, J. Keesey, J. Hicks, A. DeCristofaro, E.A. Kerr, The quality of health care delivered to adults in the United States, N. Engl. J. Med. 348 (2003) 2635–2645. [3] R. Grol, Successes and failures in the implementation of evidence-based guidelines for clinical practice, Med. Care 39 (2001) II-46–II-54. [4] O. Hirakis, S. Karakounos, Goals and benefits of knowledge management in healthcare, in: A.A. Lazakidou (Ed.), Handbook of Research on Informatics in Healthcare and Biomedicine, Idea Group Inc., Hershey, Pennsylvania, US, 2006, pp. 193–200. [5] T. Hongsermeier, Clinical Informatics Opportunities in Healthcare and Lifesciences 2008. Available: 〈http://www.healthcare-informatics.com/sites/healthcare-in formatics.com/files/archive/www.healthcare-informatics.com/Media/BlogTopics/ 2008%20Hongsermeier%20-%20Medical%20Informatics%20Careers%20SEAK.pdf〉. [6] P.G. Shekelle, E. Ortiz, S. Rhodes, S.C. Morton, M.P. Eccles, J.M. Grimshaw, S.H. Woolf, Validity of the Agency For Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? JAMA 286 (2001) 1461–1467. [7] E.M. Antman, J. Lau, B. Kupelnick, F. Mosteller, T.C. Chalmers, A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts: treatments for myocardial infarction, JAMA 268 (1992) 240–248. [8] T. Thompson, Health Care Information Technology, 2003. Available: 〈http:// aspe.hhs.gov/sp/nhii/Conference03/SpeechText.htm〉. [9] J. Tetroe, Knowledge translation at the Canadian Institutes of Health Research: a primer, Focus Technical Brief, 18, 2007, 1–8. [10] R.B. Haynes, Of studies, syntheses, synopses, summaries, and systems: the 5S evolution of information services for evidence-based healthcare decisions, Evid. Based Med. 11 (2006) 162–164. [11] M. Stefanelli, Knowledge and process management in health care organizations, Methods Inf. Med. 43 (2004) 525–535. [12] G. Schreiber, Knowledge engineering and management: the CommonKADS methodology, MIT Press, Cambridge, Massachusetts, USA, 2000.

49

[13] T.J. Bright, A. Wong, R. Dhurjati, E. Bristow, L. Bastian, R.R. Coeytaux, G. Samsa, V. Hasselblad, J.W. Williams, M.D. Musty, Effect of clinical decision-support systemsa systematic review, Ann. Intern. Med. 157 (2012) 29–43. [14] M.W. Jaspers, M. Smeulers, H. Vermeulen, L.W. Peute, Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings, J. Am. Med. Inform. Assoc. 18 (2011) 327–334. [15] M.E. Johnston, K.B. Langton, R.B. Haynes, A. Mathieu, Effects of computerbased clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research, Ann. Intern. Med. 120 (1994) 135–142. [16] J. H. Weber-Jahnke,G. McCallum, A light-weight component for adding decision support to electronic medical records, in: Proceedings of the 41st Annual Hawaii International Conference on System Sciences, 2008, pp. 251–251. [17] D.W. Bates, G.J. Kuperman, S. Wang, T. Gandhi, A. Kittler, L. Volk, C. Spurr, R. Khorasani, M. Tanasijevic, B. Middleton, Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality, J. Am. Med.l Inform. Assoc. 10 (2003) 523–530. [18] D. Bates, M. Cohen, L. Leape, J.M. Overhage, M.M. Shabot, T. Sheridan, Reducing the frequency of errors in medicine using information technology, J. Am. Med. Inform. Assoc. 8 (2001) 299–308. [19] B. Middleton, K. Renner, M. Leavitt, Ambulatory practice clinical information management: problems and prospects, Healthc. Inf. Manag.: J. Healthc. Inf. Manag. Syst. Soc. Am. Hosp. Assoc. 11 (1997) 97. [20] P.L. Elkin, M. Liebow, B.A. Bauer, S. Chaliki, D. Wahner-Roedler, J. Bundrick, M. Lee, S.H. Brown, D. Froehling, K. Bailey, The introduction of a diagnostic decision support system (DXplain™) into the workflow of a teaching hospital service can decrease the cost of service for diagnostically challenging Diagnostic Related Groups (DRGs), Int. J. Med. Inform. 79 (2010) 772–777. [21] ICNet International. ABX Alert. Available: 〈http://icnetplc.com/en-gb/products/ abx-alert〉. [22] E.G. Poon, S.J. Wang, T.K. Gandhi, D.W. Bates, G.J. Kuperman, Design and implementation of a comprehensive outpatient results manager, J. Biomed. Inform. 36 (2003) 80. [23] G.J. Kuperman, A. Bobb, T.H. Payne, A.J. Avery, T.K. Gandhi, G. Burns, D.C. Classen, D.W. Bates, Medication-related clinical decision support in computerized provider order entry systems: a review, J. Am. Med. Inform. Assoc. 14 (2007) 29–40. [24] R. Bose, Knowledge management-enabled health care management systems: capabilities, infrastructure, and decision-support, Expert Syst. Appl. 24 (2003) 59–71. [25] W. Sujansky, Heterogeneous database integration in biomedicine, J. Biomed. Inform. 34 (2001) 285–298. [26] I. Sim, P. Gorman, R.A. Greenes, R.B. Haynes, B. Kaplan, H. Lehmann, P.C. Tang, Clinical decision support systems for the practice of evidence-based medicine, J. Am. Med. Inform. Assoc. 8 (2001) 527–534. [27] M. Peleg, S. Tu, Decision support, knowledge representation and management in medicine, Yearb. Med. Inform. 45 (2006) 72–80. [28] A.A. Boxwala, B.H. Rocha, S. Maviglia, V. Kashyap, S. Meltzer, J. Kim, R. Tsurikova, A. Wright, M.D. Paterno, A. Fairbanks, A multi-layered framework for disseminating knowledge for computer-based decision support, J. Am. Med. Inform. Assoc. 18 (2011) i132–i139. [29] A.A. Boxwala, S. Tu, M. Peleg, Q. Zeng, O. Ogunyemi, R.A. Greenes, E.H. Shortliffe, V.L. Patel, Toward a representation format for sharable clinical guidelines, J. Biomed. Inform. 34 (2001) 157–169. [30] T.S. Field, P. Rochon, M. Lee, L. Gavendo, S. Subramanian, S. Hoover, J. Baril, J. Gurwitz, Costs associated with developing and implementing a computerized clinical decision support system for medication dosing for patients with renal insufficiency in the long-term care setting, J. Am. Med. Inform. Assoc. 15 (2008) 466–472. [31] Z. Chenhui, D. Huilong, L. Xudong, An integration approach of healthcare information system, 2008, 606–609. [32] Y. Zhang, Z. Li, H. Li, X. Zheng, X. Lu, H. Duan, A knowledge representation and translation method for clinical protocols, Chin. J. Biomed. Eng. 33 (2) (2014) 139–147. [33] C. Weng, S.W. Tu, I. Sim, R. Richesson, Formal representations of eligibility criteria: a literature review, J. Biomed. Inform. 43 (2010) 451. [34] A. L. Russ, A. J. Zillich, M. S. McManus, B. N. Doebbeling,J. J. Saleem, A human factors investigation of medication alerts: barriers to prescriber decisionmaking and clinical workflow, in: Proceedings of the AMIA Annual Symposium, 2009, 548. [35] S. L. Achour, M. Dojat, C. Rieux, P. Bierling,E. Lepage, Knowledge acquisition environment for the design of a decision support system: application in blood transfusion, in: Proceedings of the AMIA Symposium, 187–191, 1999. [36] S. Achour, M. Dojat, C. Rieux, P. Bierling, E. Lepage, A UMLS-based knowledge acquisition tool for rule-based clinical decision support system development, J. Am. Med. Inform. Assoc. 8 (2001) 351–360. [37] C. Ge, Y. Zhang, W. Chen, H. Duan and H. LI, Construction and statistical analysis of an ADR (Adverse Drug Reactions) knowledgebase, Chin. J. Hosp. Pharm. In press. [38] A. Adya, J. A. Blakeley, S. Melnik,S. Muralidhar, Anatomy of the ado. net entity framework, in: Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data, 2007, 877–888. [39] H.Z.C. Zhigang, Research and Application about object/relation mapping framework, Comput. Eng. Appl. 26 (2003) 058.

50

Y. Zhang et al. / Computers in Biology and Medicine 60 (2015) 40–50

[40] China Ministry of Health. The Medical Administrative Department of the Ministry of Health, in: Proceedings of the Clinical Pathway Management Work Conference, 2012. Available: 〈http://www.gov.cn/gzdt/2012-05/14/content_ 2136918.htm〉. [41] B.L. Hug, D.J. Witkowski, C.M. Sox, C. Yoon, M.E. Matheny, D.W. Bates, Adverse drug event rates in six community hospitals and the potential impact of computerized physician order entry for prevention, J. Gen. Intern. Med. 25 (2010) 31–38. [42] A. Sloman, Why we need many knowledge representation formalisms, in: Proceedings BCS Expert Systems Conf, (1985), pp. 163-183. [43] T.J. Parr, R.W. Quong, ANTLR: a predicated-LL (k) parser generator, Softw.: Pract. Exp. 25 (1995) 789–810. [44] C.Y. Jung, K.A. Sward, P.J. Haug, Executing medical logic modules expressed in ArdenML using Drools, J. Am. Med. Inform. Assoc. 19 (2012) 533–536. [45] P. Browne, JBoss Drools business rules, Packt Publishing Ltd., Birmingham, UK, 2009. [46] J. Osheroff, J. Teich, B. Middleton, A roadmap for national action on clinical decision support, J. Am. Med. Inform. Assoc 14 (2) (2007) 141–145. [47] SFDA. China State Food and Drug Administration. Available: 〈http://www.sda. gov.cn/WS01/CL0001/〉. [48] G. Del Fiol, V. Huser, H.R. Strasberg, S.M. Maviglia, C. Curtis, J.J. Cimino, Implementations of the HL7 context-aware knowledge retrieval (infobutton) standard: challenges, strengths, limitations, and uptake, J. Biomed. Inform. 45 (2012) 726–735.

[49] Evidence-Based Medicine Committee of China Medical Association. China Guideline Clearinghouse. Available: 〈http://cgc.bjmu.edu.cn:820〉. [50] Chinese Journal of Evidence-Based Medicine. China Cochrane/Evidence Based Medicine Center. Available: 〈http://www.cjebm.org.cn/docs/yx/details.aspx? documentid=101〉. [51] O.I. Franko, T.F. Tirrell, Smartphone app use among medical providers in ACGME training programs, J. Med. Syst. 36 (2012) 3135–3139. [52] C.P. Friedman, A fundamental theorem of biomedical informatics, J. Am. Med. Inform. Assoc. 16 (2009) 169–170. [53] Centers for Disease Control and Prevention, Data Collection Methods for Program Evaluation: Observation, 2008. Available: 〈http://www.cdc.gov/ healthyyouth/evaluation/pdf/brief16.pdf〉. [54] J. Lomas, Diffusion, dissemination, and implementation: who should do what? Ann. NY Acad. Sci. 703 (2006) 226–237. [55] X. Li, R.J. Quigg, J. Zhou, W. Gu, P.N. Rao, E.F. Reed, Clinical utility of microarrays: current status, existing challenges and future outlook, Curr. Genomics 9 (2008) 466. [56] M.T. Scheuner, P. Sieverding, P.G. Shekelle, Delivery of genomic medicine for common chronic adult diseases: a systematic review, JAMA 299 (2008) 1320–1334. [57] F.W. Frueh, D. Gurwitz, From pharmacogenetics to personalized medicine: a vital need for educating health professionals and the community, Pharmacogenomics 5 (2004) 571–579.