Conceptualization and application of an approach for designing healthcare software interfaces.

Journal of Biomedical Informatics xxx (2014) xxx–xxx

Contents lists available at ScienceDirect

Journal of Biomedical Informatics journal homepage: www.elsevier.com/locate/yjbin

Conceptualization and application of an approach for designing healthcare software interfaces Ajit Kumar a, Reena Maskara b, Sanjeev Maskara c, I-Jen Chiang a,d,⇑ a

Graduate Institute of Biomedical Informatics, Taipei Medical University, Taiwan Department of Foreign Languages and Applied Linguistics, Yuan Ze University, Taiwan c Ovens and King Community Health Services, Wangaratta, Victoria, Australia d Institute of Biomedical Engineering, National Taiwan University, Taiwan b

a r t i c l e

i n f o

Article history: Received 25 February 2013 Accepted 1 February 2014 Available online xxxx Keywords: Interface design Rhetorical situation Move analysis Stereotype Healthcare Outpatient clinic

a b s t r a c t Objective: The aim of this study is to conceptualize a novel approach, which facilitates us to design prototype interfaces for healthcare software. Methods: Concepts and techniques from various disciplines were used to conceptualize an interface design approach named MORTARS (Map Original Rhetorical To Adapted Rhetorical Situation). The concepts and techniques included in this approach are (1) rhetorical situation – a concept of philosophy provided by Bitzer (1968); (2) move analysis – an applied linguistic technique provided by Swales (1990) and Bhatia (1993); (3) interface design guidelines – a cognitive and computer science concept provided by Johnson (2010); (4) usability evaluation instrument – an interface evaluation questionnaire provided by Lund (2001); (5) user modeling via stereotyping – a cognitive and computer science concept provided by Rich (1979). A prototype interface for outpatient clinic software was designed to introduce the underlying concepts of MORTARS. The prototype interface was evaluated by thirty-two medical informaticians. Results: The medical informaticians found the designed prototype interface to be useful (73.3%), easy to use (71.9%), easy to learn (93.1%), and satisfactory (53.2%). Conclusions: MORTARS approach was found to be effective in designing the prototype user interface for the outpatient clinic software. This approach might be further used to design interfaces for various software pertaining to healthcare and other domains. Ó 2014 Elsevier Inc. All rights reserved.

1. Introduction A user interface is the space, where interaction between humans and computers occurs. The interface allows a user to manipulate a system (input) and shows the effects of the user’s manipulation (output) [1]. In the recent years, the interface design of healthcare software has been a hot topic of discussion in various national and international forums because it plays a vital role in the success, as well as the failure of healthcare Information and Communication Technology (ICT) systems. A study in Germany reported that a small improvement in interface design enhanced the quality of healthcare data being captured [2]. In another study at the Academic Medical Center of Amsterdam, Netherlands, the pre and post usability of the former and redesigned Electronic Health Record (EHR) system was evaluated. The redesigned EHR system ⇑ Corresponding author at: Graduate Institute of Biomedical Informatics, Taipei Medical University, 250, Wuxing Street, Taipei 11031, Taiwan. Fax: +886 2 27392914. E-mail addresses: [email protected], [email protected] (I.-J. Chiang).

was appreciated because of its user-friendly interfaces, enhanced functionality, and capability [3]. Similarly, the failure of several healthcare ICT systems has been attributed to non-friendly design of user interface. Several studies have shown that designing a userfriendly interface has been one of the biggest challenges faced by healthcare ICT systems [4]. A study in Finland has reported that the increasing use of computers stole the time and attention of doctors, nurses, and other healthcare professionals and distracted them from patients in delivering quality care [5]. In addition, several other studies have reported that doctors and nurses have faced numerous difficulties in their routine work due to non-friendly user interface [6–9]. To rectify the interface design problems, the involvement of users has been always recommended during the software development [10,11]. However, users’ involvement of has been always given a low priority. Therefore, the strategy to involve users has remained unexplored until now [12]. After the advent of Graphical User Interface, their use has been suggested in healthcare ICT systems. However, due to the high complexity of the healthcare

http://dx.doi.org/10.1016/j.jbi.2014.02.007 1532-0464/Ó 2014 Elsevier Inc. All rights reserved.

Please cite this article in press as: Kumar A et al. Conceptualization and application of an approach for designing healthcare software interfaces. J Biomed Inform (2014), http://dx.doi.org/10.1016/j.jbi.2014.02.007

2

A. Kumar et al. / Journal of Biomedical Informatics xxx (2014) xxx–xxx

domain, even Graphical User Interface cannot bring much change in the usability of healthcare software [7,13]. Therefore, the healthcare domain has continuously faced the interface design related problems. Our literature searches uncovered some of the possible causes behind the poor usability of healthcare ICT products, as explained below [14–18]. First, the evolution of ICT took place ‘inside-out’ manner. In the earlier days, computer systems were extremely expensive; therefore, the system efficiency usually had priority over the convenience [19–21]. At that time, finding cost-effective and efficient hardware solutions was the focus of research and development [22–26]. Over the time, the computer hardware turned out to be cheaper, effective, and efficient [27]. On the other hand, due to ever-growing software complexity, the failure of the software became a common phenomenon [28–31]. These intermittent software failures motivated its designers to involve users in the process of software development; however, users were always the last one to be involved [32,33]. The partial involvement of users

works well in developing ICT products for domains such as banking, finance, insurance, and telecommunication because users of these domains are not diverse and have clearly defined system workflow. Nevertheless, about 68% ICT projects failed in these domains too, which caused the loss of nearly 63 billion USD [34,35]. Ironically, in spite of such failure prone situations, designers of healthcare ICT products continued to stick with the same traditional ‘inside-out’ approach with the minimal participation of users [36]. Second, the healthcare is a complex adaptive system – dynamic, nonlinear, consists of a number of independent agents, diverse settings such as clinics, polyclinics, hospitals, and a wide variety of stakeholders such as doctors, nurses, patients. Therefore, healthcare systems face several limitations pertaining to their design and management [37–39]. Very often, interface designers are not aware of the underlying complexity of the health care organization. In addition, the time allocated to the interface designer is usually not sufficient to understand complex requirements of

Fig. 1. Typical approach for interface design along with popular approaches such as structured, task-based, and scenario-based.



healthcare users, which prevents designers from developing userfriendly interfaces. Third, the ICT domain is still maturing to provide a systematic approach for the interface design. As shown in Fig. 1, interfaces are typically designed through two iterative steps: (1) prototype design, and (2) prototype testing. Some of the popular interface design approaches are structured, task-based, and scenario-based. In the structured approach, system functionality is developed first, followed by prototype interface design, then heuristic evaluation. The task-based approach first focuses on the user’s tasks, then cognitive walk-through evaluation that allows the interface designer to test the organized series of tasks. The scenario-based approach is a variant of task-oriented prototyping, which considers a realistic scenario – how the ICT system would be used in the real-world settings followed by think-aloud evaluation. In these approaches, the role of various factors, such as usability, user involvement, testing in real-world settings, cognitive aspects, time and cost of development, and complexity, increases from structured to scenario-based approach, as shown in Fig. 1. The bottom line of these approaches is to design human-centered interfaces by modeling human perception, cognitive process, and behavior. Due to a high diversity in the healthcare domain (diverse end-users, settings, and so on), the scenario-based approach has been highly recommended for designing interfaces for healthcare ICT systems. The scenario-based approach performs testing of prototype interfaces using the think-aloud method, wherein the cognitive aspects of clinicians are captured in their natural work settings using video recordings. Subsequently, the analyses of these video recordings reveal their perception, cognitive process, and behavior [11,40–45]. As the think-aloud method heavily relies on video recordings and their analysis, this method has been reported having several problems such as subjectivity of the experimenter, time-consuming, disturbances in the clinicians’ cognitive process, and high-cost [46]. We argue that documents written by clinicians in healthcare settings (medical records) are a true representation of clinicians’ cognitive processes and behavior; therefore, we can make use of medical records instead of video-recordings. Several problems encountered in the think-aloud method could be resolved by resorting to concepts such as the rhetorical situation (a concept of philosophy), and the move analysis (a concept of applied linguistics) [47–50]. Therefore, we conceptualized a novel approach using concepts and techniques from various disciplines, which can systematically design a prototype interface, evaluate, and continually

3

improve it. This approach was applied to design the interface for the outpatient clinic software. In this research paper, we have described the newly conceptualized interface design approach and its application.

2. Methods This study was carried out with the institutional approval (study reference number – SIEC/12/01/001) from Suraksha Independent Ethics Committee, The Committee for Scientific Review and Evaluation of Biomedical Research, Hyderabad, India. Informed consents were taken from all participants prior to the study. In addition, the necessary measures were taken to ensure the privacy and confidentiality of the collected data. In this study, a new approach named MORTARS (Map Original Rhetorical To Adapted Rhetorical Situation) was conceptualized, which provides a systematic way to design medical software interfaces. As shown in Table 1, MORTARS consists of four iterative phases – development of a prototype interface, evaluation, individualization, and improvement [51]. Interface designers usually deploy iterative design because it allows them to identify the usability issues that may arise in the user interface before its wide usage. As it would be quite difficult even for the best usability experts to design a perfect user interface in a single attempt, the usability engineering life-cycle needs to be built around the concept of iteration [52]. The four phases of MORTARS include nine activities. The schematic diagram of Activity 1–6, which models users’ cognitive process, is shown in Fig. 2 [44]. Activity 7 (evaluation), Activity 8 (individualization), and Activity 9 (improvement) are carried out after the modeling of users’ cognitive process (Activity 1–6). These nine activities make use of concepts and techniques from various disciplines such as (1) the rhetorical situation – a concept of philosophy provided by Lloyd Bitzer, (2) move analysis – an applied linguistic technique provided by Swales and Bhatia, (3) interface design guidelines – a cognitive and computer science concept provided by Johnson, (4) usability evaluation instrument – an interface evaluation questionnaire provided by Lund, and (5) user modeling via stereotyping – a cognitive and computer science concept provided by Rich [40,47–50,53–57]. In the coming sections, nine activities, involved concepts, techniques, and tools are explained and applied to design a prototype interface of outpatient clinic software.

Table 1 MORTARS conceptualization. Phases

Activities

1. Prototype interface design

Activity 1: Understand a rhetorical situation named original rhetorical situation Activity 2: Imagine a rhetorical situation named adapted rhetorical situation (which is superior to the original rhetorical situation)

Rhetorical situation provided by Lloyd Bitzer (Cognitive process and some common human behavior)

Activity 3: Collate texts from the original rhetorical situation Activity 4: Analyze the collated text using move analysis Activity 5: Map Original Rhetorical To Adapted Rhetorical Situation to obtain a preliminary interface

Move analysis provided by Swales and Bhatia (Cognitive process and some common human behavior) (Cognitive process and some common human behavior)

– Mental model (user’s mental image of the interface)

– Design model (design realization of the user model) – Implementation model (look and feel of an interface)

Activity 6: Fine-tune the preliminary interface using existing interface design guidelines to obtain a prototype interface

Concepts, techniques or tools

Interface design guideline provided by Johnson

Mock-up screen tool (Human perception) 2. Evaluation 3. Individualization

Activity 7: Perform a usability evaluation of the prototype interface Activity 8: Individualize the prototype interface

4. Improvement

Activity 9: Improve the prototype interface

Usability evaluation provided by Lund Stereotyping provided by Elaine Rich (Specific human behavior) Techniques used in Activity 1–8


4


Fig. 2. Schematic diagram of human cognitive system and Activity 1–6 of MORTARS (Fig. 2 is modified from [44]).

2.1. Activity 1: Understand a rhetorical situation called original rhetorical situation

purpose of writing. Working hours, location, and facilities are the settings of the outpatient clinic.

A proper understanding of the rhetorical situation is a vital activity of MORTARS. Lloyd Bitzer was the first person to define the rhetorical situation. The rhetorical situation is an act of communication that takes place in relation to various elements, and it affects contents, comprehension of the contents, identity of the author, communicative purpose, and audience. The rhetorical situation consists of seven basic elements: (1) text – communication instance; (2) genre – distinctive type of text; (3) medium – print, spoken, and digital; (4) author – someone, who writes communication instance; (5) audience – recipient of communication instance; (6) purpose – varied reasons for communication between author and audience; (7) setting – time, place, and an environment. The rhetorical situations exhibit structures, which may be (1) simple or complex, and (2) more or less organized. The situation is said to be highly structured when all elements are properly located and ready for the task to be performed. A courtroom is an example of a complex and highly organized structure. An outpatient clinic is an example of simple, but highly organized structure. In this study, the outpatient clinic was considered as a rhetorical situation called ‘original rhetorical situation,’ as shown in Part A of Fig. 3. In the outpatient clinic, a doctor usually writes the text that includes patient details, problems, diagnoses, and management. The outpatient visit record is the genre of the outpatient clinic. A pen and paper are the media of recording. A doctor performs the task of writing; therefore, the doctor is the author. A patient, nurse, administrative clerk, pharmacist, pathologist, and radiologist are the audience. To provide the instructions to the audience is the

2.2. Activity 2: Imagine a rhetorical situation called adapted rhetorical situation ‘Adapted rhetorical situation’ is defined as an imagined situation, which is superior to the ‘original rhetorical situation.’ Part B of Fig. 3 shows an example of ‘adapted rhetorical situation.’ As can be seen in Part A of Fig. 3, a pen and paper are the main recording media during doctor–patient’s consultations. These consultations generate paper-based medical records, which have several limitations and problems of their own such as illegible writing, long-term storage, accessibility, and possible delays in communicating with other healthcare stakeholders. Therefore, let us imagine a better situation, as shown in Part B of Fig. 3, wherein the use of computer systems instead of traditional pen and paper style of recording would be encouraged. This paperless system of recording consultations would help to overcome a few of the above-mentioned problems and limitations. 2.3. Activity 3: Collate texts from the original rhetorical situation In the outpatient clinic, medical records are generated as a result of doctor–patient’s consultations. These records contain texts that need to be analyzed. One hundred seven outpatient medical records were collected. We removed all the identifying and confidential details of doctors and patients from the medical records. Out of one hundred seven medical records, only fifty-nine met the inclusion criteria of this study, as shown in Fig. 4.



5

Fig. 3. Mapping Original Rhetorical To Adapted Rhetorical Situation (MORTARS).

2.4. Activity 4: Analyze the collated text using move analysis According to Swales and Bhatia, the structural interpretation of the genre’s text highlights the cognitive aspects of text

organization. They further describe that a specialist writer (for example, doctor) remains consistent in organizing the overall message in a particular genre; therefore, the structural analysis of the genre would reveal their preferred ways of communication. At this


6


Fig. 4. Sample texts inclusion.

juncture, it is relevant to realize that, for an outpatient clinic, a doctor remains consistent in organizing the overall text of the outpatient medical record. This particular organization of the text in the medical record is a true representation of the doctor’s cognitive processes [44]. When we switch from the traditional pen and paper method to digital media, doctors would prefer recording the text in their accustomed cognitive style in the digital media too. Therefore, the textual analysis of the ‘original rhetorical situation’ is crucial in revealing the doctor’s cognitive processes. Swales and Bhatia proposed a technique called ‘move analysis,’ which is useful for the textual analysis of the genre. Move analysis is a top down approach (where the focus is on function or meaning, or both), which analyzes the discourse structure of texts from a genre. Move analysis identifies various key elements: (1) move – the entire text is described as a sequence of moves, where each move represents ‘a stretch of text’ serving a particular communicative function and subordinate to the overall communicative purpose of the genre; (2) sub-move – the text of a ‘move’ may contain ‘sub-function’ and can be divided further called sub-move; (3) step – the group of functional or semantic, or both themes, which are in ‘relative proximity’ to each other, often occur in similar locations in the representative texts and help to realize a broader move called step. Usually, a ‘move’ may contain submoves or steps, or both; a sub-move may contain sub-sub-moves or steps, or both; so on; whereas a step does not contain any further branch. To identify moves, sub-moves, and steps in the texts (outpatient medical records), eight coding guidelines introduced by Biber et al. in 2007 can be used [58]. The first five guidelines categorize the text from its functional–semantic perspective in order to develop an analytical framework (development of coding protocol, including potential moves, sub-moves, and steps) [59]. The last three guidelines perform coding (segment, classify, verify, and review) of the text [59]. The further details of guidelines are described using an example of the outpatient medical record genre. A few key points should be kept in mind while applying for the coding guidelines. First, an applied linguistics expert who possesses skills,

such as genre, discourse, and text analysis, should preferably carry out the coding. Second, in the coding process, the involvement of professionals, such as computer (human–computer interaction), medical, and cognitive science, are recommended. Third, to avoid the subjectivity of coders, two or more similar groups of coders (applied linguistics, computer, medical, and cognitive science) should be recruited, and the inter-rater reliability needs to be verified. Fourth, we finally intend to move to a computer-based system due to several of its advantages over the paper-based system. In addition, we want to retain the advantages of the existing paper-based system, which has been in place for a long time, and medical professionals traditionally prefer it. Therefore, people responsible for coding of the text should not only try to retain the unique features of the paper-based system, but also avoid over fitting it. Otherwise, it can be a limitation of the MORATRS approach. For instance, a computer-based system might have the capability to facilitate an individualized human–computer interface. However, to bring this capability, we should provide a mechanism, which ensure flexibility in the interface, as discussed in Activity 8. Similarly, in the paper-based system, serial number, identity, date, and time are manually specified, which could be automated in the computer-based system without any hassle. Therefore, unique features of the computer-based system (flexibility, accessibility, automation, storage, and so on) should be properly considered while coding. Guideline 1: Determine rhetorical purposes of the genre. First, in order to identify moves’ categories for a genre, it is important to understand the ‘big-picture’ of the overall rhetorical purpose of the texts in the genre. For instance, the rhetorical purposes of outpatient medical record can be (1) to document doctor–patient consultations for future reference, (2) to share medical records with other healthcare stakeholders, and so on. Guideline 2: Look at the function of each text segment, and evaluate its local purpose after the overall rhetorical purpose is understood. We have to identify moves, which need to be



distinctive. Multiple readings and reflections on the texts are needed before the clear moves with their defining function emerge. During this process, one need to look for any common functional–semantic themes, represented by the various text segments that have been identified, especially those that are in ‘relative proximity’ to each other, or often occur in about ‘the same location’ in various texts representing the genre. For instance, various sections of outpatient medical records should be carefully investigated to figure out their functions such as patient details, diagnosis, and so on. Guideline 3: Group functional–semantic themes that reflect the various steps of a broader move, with each move having its own functional–semantic contribution to the overall rhetorical purpose of the text. A stretch of text can possibly have a functional–semantic theme, as well as sub-themes (possibly subsub-themes) inside it. For instance, in the outpatient record, a stretch of text describes patient details, wherein we can find sub-themes such as demographic details, vital sign, and so on. We can designate a sub-theme as a sub-move. Moreover, a sub-theme, for example – demographic detail, can have subsub-themes such as name, age, sex, and so on. We can designate a sub-sub-theme as a step. Guideline 4: Conduct pilot coding to test and fine-tune the definition of moves, sub-moves, and steps. It is the best to begin with a pilot coding – ideally, with at least two coders. Initial analyses are discussed then fine-tuned until there is agreement on the functional–semantic purposes that are being realized by the text segments. Guideline 5: Develop coding protocol with a clear definition of moves (themes), sub-moves (sub-themes), and steps (subsub-themes). The protocol can be used as an initial baseline. Guideline 6: Code full set of texts with inter-rater reliability checks to confirm that there is a good understanding about the definition of moves, including how moves, sub-moves, and steps are realized in texts. Guideline 7: Add any additional move, sub-move, or step, which is revealed in the full analysis of texts. At times, a move, submove, or step could be missed in the baseline coding protocol, and later it comes to knowledge. In such situations, coders should consider incorporating the newly identified move, submove, or step to the baseline protocol. Guideline 8: Review the baseline coding protocol to resolve any discrepancies revealed by the inter-rater reliability checks.

Considering the above-mentioned eight guidelines, a team of researchers (applied linguistics, computer, and medical) coded fifty-nine samples of outpatient medical record to identify moves, sub-moves, and steps. Descriptive names were assigned to the identified moves, sub-moves, and steps. Fig. 5 in the Result section shows an example of the coding process, and Fig. 6 in the Result section shows descriptive names assigned to various moves, submoves, and steps. Another team of independent researchers (applied linguistics, computer, and medical) verified the coding, and a few minor changes were made based upon their recommendations. Frequencies of various moves, sub-moves, and steps were calculated after the completion of coding and its verification. The coding and frequency calculations were performed using NVivo 10 tools. Table 2 in the Result section shows the codes, descriptive names, and frequencies of moves, sub-moves, and steps. The interrater reliability was also calculated, as shown in Table 2. The overall inter-rater reliability score of the genre (outpatient medical record) was 99.21%. The high score (99.21%) could be due to the analysis of the highly structured genre (outpatient medical record). Some minor discrepancies were detected in inter-rater reliability, which could be due to illegible writings of a few doctors in some medical records.

7

2.5. Activity 5: Map Original Rhetorical To Adapted Rhetorical Situation to obtain a preliminary interface As shown in Fig. 3, MORTARS maps the original rhetorical situation (Part A of Fig. 3) to the adapted rhetorical situation (Part B of Fig. 3). The transition from the original (Part A) to the adapted (Part B) needs a careful mapping of all seven rhetorical situation elements. As Activity 5 (Map Original Rhetorical To Adapted Rhetorical Situation) is the novel, central and highly important concept, we used its abbreviation, MORTARS, for the entire approach. As can be seen in the outpatient clinic scenario, we intend to replace the traditional pen and paper with the digital artifacts that cause a change in the ‘medium.’ Thus, the pen and paper need to be properly mapped with the digital media. Some basic understanding of the digital media and the involvement of a computer professional would be helpful in the mapping process. The codes in Fig. 6 were utilized to derive a preliminary interface, as shown in Fig. 7 in the Result section. As the structure of the genre is a true representation of doctor’s cognitive process, the codes as shown in Fig. 6 were positioned in such a way that the preliminary interface (Fig. 7) preserved the structure of the genre. 2.6. Activity 6: Fine-tune the preliminary interface using the existing interface design guidelines to obtain the prototype interface As illustrated in Block ‘B’ of Fig. 2, Activity 1–5 of MORTARS approach models the cognitive process and common behavior of authors (or users). However, the consideration of human perception (as illustrated in Block ‘A’ of Fig. 2) is highly desirable during the transition – from the preliminary interface to the digital media based interface design [11,60]. Fortunately, several guidelines, principles, and mockup screen tools exist, which consider the human perception. These design guidelines and tools could be applied during the transition (from preliminary to digital media based interface). In this study, we deployed Johnson’s interface design guidelines and a mockup screen tool to fine-tune the preliminary interface (Fig. 7) [40]. Fig. 8 in the Result section shows the prototype interface obtained after the fine-tuning of the preliminary interface. 2.7. Activity 7: Perform the usability evaluation of the prototype interface In this study, as the prototype was nonfunctional, the evaluation could not be carried out by the real-time users. A questionnaire along with the prototype interface was sent to thirty-two medical informaticians for prototype usability evaluation [57]. These medical informaticians were chosen after a careful search of the various medical informatics organizational websites. The criteria to choose these medical informaticians were based on their educational qualifications and healthcare ICT experiences. Out of thirty-two medical informaticians, twenty-five had a background of medical science education, whereas seven had the background of computer science. They were engaged in different professions such as clinicians, healthcare ICT consultants, professors, and healthcare ICT researchers. These thirty-two medical informaticians resided in different parts of India. Informed consents were obtained prior to the study. They were also informed that the collected data would be used solely for the research purposes. Table 3 in the Result section shows the response summary of the usability evaluation questionnaire. In the questionnaire, an open-ended question was also included for providing comments. We received 28 comments, which were analyzed using NVivo 10 tools. Table 4 in the Result section shows themes and sub-themes derived from medical informaticians’ comments.


8


Fig. 5. An example demonstrating moves, sub-moves, and steps.

2.8. Activity 8: Individualize the interface A common mistake, which is usually made during the development of healthcare ICT solutions, is an assumption that the users are ‘homogeneous.’ However, this assumption is not particularly true while developing interfaces for healthcare stakeholders [61]. In the currently available approaches, interfaces are usually developed in a standard manner, and it is highly desirable that all users should fit within the standard. Standard interface design approaches might have worked well in relatively lesser complex industries such as banking, finance, and insurance. However, as mentioned before, the healthcare sector has diverse settings and stakeholders; therefore, we cannot apply the one-size-fits-all approach [61]. The comments from medical informaticians, as described in Table 4, also support this argument. Therefore, the need for an individualized interface is felt.

Stereotyping can be deployed as the useful mechanism to fulfill the need of an individualized interface. The stereotype collects frequently occurring characteristics of users called facets [53,54]. The facets are categorized into two types – composite and atomic. The composite facet may contain the further composite or atomic facets whereas the atomic facet does not contain any. The composite and atomic facets are given a value and rating respectively. The value of the composite facet varies from 1 to 10, and the rating of the atomic facet varies from 100 to 1000. The higher value or rating shows a greater degree of certainty than its lower value or rating. For all facets, we must provide an initial value, rating, and proper justification. In addition, any change in the value or rating of the facet needs to be justified further. In the outpatient clinic, the stereotype of doctors needs to be defined, which should include composite or atomic facets, initial values of composite facets, ratings of atomic facets, and justifications



9

Fig. 6. Code for moves, sub-moves, steps, and their descriptive names.

for the inclusion of facets. Fig. 6 is assumed as the initial facets of doctor’s stereotype, wherein ‘moves or sub-moves’ and steps might be considered as composite and atomic facets respectively. Moves or sub-moves were assigned a value from 1 to 10, and steps were assigned a rating from 100 to 1000. Initially, we assigned the value ‘5’ for all moves or sub-moves, and the rating ‘500’ for all steps. Tables 5 and 6 in the Result section show examples of Pediatricians and General practitioner’s stereotype respectively. Later, the value and rating might be adjusted based on the usability evaluation, or users’ behavior while interacting with the interface. The shaded parts of Tables 5 and 6 show examples of rating adjustment. Table 5

is the stereotype of pediatricians, wherein the ratings of height and weight were adjusted from 500 to 600 based on the usability evaluation and detailed analysis of the outpatient medical records issued by pediatricians. Similarly, Table 6 is the stereotype of general practitioners, wherein the ratings of height and weight were adjusted from 500 to 400 based on the usability evaluation and detailed analysis of outpatient medical records issued by general practitioners. A stereotype-based interfacing system would require maintaining a set of stereotypes such as Heart Specialist, Pediatrician, Psychiatrist, and General Practitioner. These stereotypes are


10


Table 2 Code, description, frequency, inter-rater reliability of moves, sub-moves, and steps (total sample size = 59). Code of moves, sub-moves, or steps

Description of moves, sub-moves, or steps

Coding group 1 (frequency)

Coding group 2 (frequency)

Inter-rater reliability (%)a

Move 1 Step 1.1 Step 1.2 Step 1.3 Step 1.4 Step 1.5 Move 2 Step 2.1 Step 2.2 Step 2.3 Step 2.4 Step 2.5 Step 2.6 Move 3 Move 4 Sub-move 4.1 Step 4.1.1 Step 4.1.2 Step 4.1.3 Step 4.1.4 Step 4.1.5 Step 4.1.6 Step 4.1.7 Step 4.1.8 Sub-move 4.2 Step 4.2.1 Step 4.2.2 Step 4.2.3 Step 4.2.4 Sub-move 4.3 Move 5 Move 6 Step 6.1 Step 6.2 Move 7 Step 7.1 Step 7.2 Move 8 Move 9 Move 10 Move 11 Move 12

About clinic Logo of clinic Name of clinic Address of clinic Contact numbers of clinic Timing of clinic About doctor Name of doctor Registration number of doctor Qualification of doctor Specialty of doctor Contact numbers of doctor Timing of doctor Date of patient visit About patient Demographic profile Name of patient Father or guardian Name Age of patient Sex of patient Referred by Address Height Weight Vital sign Temperature Blood pressure Pulse rate Respiratory rate History of patient/symptom Clinical signs Diagnosis Doctor’s clinical opinion/diagnosis Diagnostic test Treatment Prescription/advice Minor procedure Referral Next appointment Signature of doctor Rules of clinic or medicine Extra facilities available in clinic

51 45 49 49 45 15 29 15 29 27 26 9 9 53 59 59 59 1 47 39 1 3 7 16 11 7 6 4 5 13 34 32 30 9 57 53 16 9 6 59 53 51

51 45 49 49 45 15 29 15 29 27 25 9 9 53 59 59 59 1 47 39 1 3 8 17 11 6 5 5 4 11 31 33 29 9 57 53 15 10 7 57 53 51

100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 98.31 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 98.31 98.31 100.00 98.31 98.31 98.31 98.31 96.61 94.92 98.31 98.31 100.00 100.00 100.00 98.31 98.31 98.31 96.61 100.00 100.00

Inter-rater reliability (%) at move or sub-move level Inter-rater reliability (%) at sub-move level Inter-rater reliability (%) step level Overall inter-rater reliability (%) a

Inter-rater reliability (%) = 100

[ABS (Expert 1

98.87 98.87 99.42 99.21

Expert 2)/n * 100] where ABS means absolute value.

integrated with the interfacing system, and triggers would be required to invoke the stereotypes. The easily identifiable characteristics of users may serve as triggers. For instance, the minimum information is compulsorily taken when the user logs into the system for the first time. The system would establish a match between trigger and stored stereotypes; then the matched stereotype would invoke the associated interface. For instance, when a user enters ‘Pediatrics,’ this could be a trigger. The trigger ‘Pediatrics’ would establish the match with stereotype Pediatrician, then stereotype (Pediatricians) would invoke the associated interface.

2.9. Activity 9: Improve the prototype interface Several iterations (full or partial) from Activity 1 to 8 are required, which would depend upon the complexity of the rhetorical situation. The prototype interface needs continual improvement until a user-friendly interface is obtained. However, we could accomplish the only-one iteration while designing the interface for the outpatient clinic software.

3. Results Figs. 5–8 and Table 2 were obtained after accomplishing prototype interface design phase (Activity 1–6), as shown below. Fig. 5 shows an example of the coding process. Fig. 6 shows the descriptive name assigned to various moves, sub-moves, and steps. Table 2 shows the frequencies of moves, sub-moves, and steps. Figs. 7 and 8 show preliminary and prototype interface respectively. The prototype interface was evaluated in the evaluation phase (Activity 7). The results are shown the concept of interface personalization was introduced in the individualization phase (Activity 8) using examples of pediatricians and general practitioners. The results are shown in Table 5 and 6.

4. Discussion A novel approach called MORTARS was introduced to provide a systematic approach for prototype interface design, which consists of four phases – prototype interface design, usability evaluation,



11

Fig. 7. Obtaining preliminary interface by Mapping Original Rhetorical To Adapted Rhetorical Situation.

individualization, and continual improvement. These four phases consist of nine activities, which suggest the use of various tools and techniques. These phases and activities provide a foundation to include a comprehensive range of features that would be required for designing a user-friendly interface. Five highly desired features for interface design are made available through MORTARS, as described below [44–46,51,61–67]. First, the transparency of working environment suggests a good understanding of healthcare settings and various human factors. The working environment and human factors of the users were well represented by seven elements of the rhetorical situation, as

described in Activity 1–3 with two illustrative figures (Figs. 3 and 4). Second, the user-centeredness suggests the incorporation of (1) cognitive process of healthcare users by understanding their requirements with their maximum involvement, and (2) perception of users regarding the interfaces. As can be seen in MORTARS approach, Activity 4 provides actions and techniques (for instance, move analysis) needed to consider cognitive aspects of the users (Figs. 5 and 6, Table 2). Activity 5 and 6 consider the human perceptions by adhering to the best practices, guidelines, tools, and techniques of interface design, thereby produces the preliminary and prototype interface designs (Figs. 7 and 8) [40]. Third, the high


12


Fig. 8. Prototype interface (clinic and doctor details are fictitious).

usability suggests efficiency, effectiveness, and satisfaction for interfaces. Activity 7 focuses on the usability evaluation of the prototype interface (Table 3 and 4). The results of the usability evaluation provide insight into the usability of the designed interface in the current form and suggest some further scope of improvement in order to attain higher usability. Fourth, the individualization of

an interface suggests customizing the interface according to the users need and their behavior. A technique called stereotyping, described in Activity 8, is proposed for individualizing the prototype interface. The results of usability evaluation (Activity 7) might also be used for the individualization of the interface. For instance, we readjusted the ratings of height and weight based on comments


13

A. Kumar et al. / Journal of Biomedical Informatics xxx (2014) xxx–xxx Table 3 Usability evaluation (total number of respondents = 32). Category

Strongly disagree (%)

Somewhat disagree (%)

Neither agree nor disagree (%)

Somewhat agree (%)

Strongly agree (%)

Somewhat agree + Strongly agree (%)

Usefulness Ease of use Ease of learning Satisfaction

3.3 3.1 0

6.7 9.4 0

16.7 15.6 6.3

50.0 43.8 28.1

23.3 28.1 65.6

50.0 + 23.3 = 73.3 43.8 + 28.1 = 71.9 28.1 + 65.6 = 93.1

3.1

18.8

25.0

31.3

21.9

31.3 + 21.9 = 53.2

Table 4 Comments (28 respondents provided comments). Themes or subthemes 1. Strengths – Simple – Easy to learn – Easy to use – Simple data entry – Easy for novices 2. Area of improvement – Data capturing details

– Unstructured form of data capturing

– Intelligence enhancement

– Improvement in interface elements positioning, sequencing, and connectivity – Appearance of interface

– Additional features

Explanation The respondents admitted that the designed interface appeared simple, easy to learn, and use. They were highly impressed with the simplicity of data entry. One of the respondents mentioned that MORTARS produced simple interfaces might be useful for novices

Two key points were pointed out to improve data capturing. First, insufficient details were captured in the prototype interface (Fig. 8). The details, such as diagnosis or procedure codes, time of the patient’s visit, details of patient’s previous visits, clinic website, email address, and patient’s address, were suggested to be included in the prototype interface. In addition, some of the unnecessary patients’ details, such as height and weight, were suggested to be removed. However, it transpired from Table 2 that height (12%) and weight (27%) related data were recorded in the outpatient medical records mainly issued by pediatricians As unstructured data are usually difficult to analyze; therefore, incorporation of template features, such as drop-down menus, combo-boxes, pick-lists, auto-complete options, and voice recognition tools, were recommended. Moreover, the structured data capturing would further reduce the extensive typing involved The respondents suggested inclusion of various clinical decision support features such as drug interactions, drug–food interactions, dosage checks, duplicate order checks, drug allergy, and dosage calculation for different medications based on the child’s weight. Besides, some respondents felt the need for inclusion of specialtybased dictionaries for clinical entries, ICD-10 library for diagnoses entries, interface between outpatient medical records and approved pharmacy database Some respondents suggested interfacing of electronic pads or similar devices with the prototype interface to avoid doctor’s distraction from the patient, which might be used as a virtual prescription pad. In addition, the sequencing of interface elements should be flexible enough to meet real-world changes Some respondents suggested renaming of some of the data entry fields such as DOB instead of Age, and History of Patient instead of Past Medical History. The respondents also recommended providing wider spaces for some of the data entry fields The respondents recommended some additional features such as a facility to view old outpatient medical records, integration of computerized physician order entry (CPOE) to in-house pharmacy or lab, accounts like billing, medicine stock, and report generation, provision of security, and error trap mechanisms

received from respondents, as shown in the shaded region of Tables 5 and 6. Fifth, the continual improvement suggests improving the interface over the time. Activity 9 of the MORTARS strongly recommends multiple iterations of Activity 1–8 for the continual improvement of the interface. We used MORTARS to design an interface for the outpatient clinic software. Unlike other approaches such as structured, taskbased, scenario-based, we found that MORTARS assimilates most of the above-mentioned desired features for interface design with simplicity and by consuming fewer resources. In fact, the four phases, nine activities, and a range of various tools and techniques make MORTARS a robust and unique approach. We expect that MORTARS may be utilized to design software interfaces for other rhetorical situations related to healthcare and other domains such as the legal system (advocate and client consultations at an advocate’s office), accounting (chartered accountant and client consultations at an accountant’s office), and police services (police officer and victim consultations at police station). 4.1. Limitation and future studies There are some limitations in this study, which need further research and exploration. First, Activity 7 (Perform the usability

evaluation of the prototype interface) was carried out to evaluate the usability, ease of use, ease of learning, and satisfaction of the prototype interface with the help of non-real time users (medical informaticians). The real-time users (clinician) interact with the interfaces in a complex work environment; therefore, various aspects of the interface (organization of elements on the screen, screen contents, and so on) and its relation to healthcare-related activities need to be tested in the real-time settings (the outpatient clinic) and users (clinician). Therefore, some pilot testing in realtime settings would expand Activity 7 and provide more evidence about the utility and effectiveness of the MORTARS approach. In addition, the pilot testing would give more evidence whether the interface fits with healthcare professional’s workflow in their real-time settings. Second, in Activity 8, an attempt was made to individualize the prototype interface using the stereotyping. In this activity, the underlying theories of stereotyping and triggers have been defined; however, their applications remain unexplored. In addition, some of the assumptions, such as range, magnitude, and increment of value and rating, were arbitrary. Therefore, we feel the need for further research to explore this concept. The study performed by Zheng et al. has found that clinicians show consistent behavior while navigating the interfaces, and the interface designer does


14


Table 5 An example of trigger for pediatricians.a Facet

Rating

Justification

1st

2nd

3rd and so forth

1st

2nd

3rd and so forth

1st

2nd

3rd and so forth

About clinic

5

5

...

–

–

...

Based on prescription analysis

...

Logo of clinic Name of clinic Address of clinic Contact numbers of clinic Timing of clinic About doctor Name of doctor Registration number of doctor Qualification of doctor Specialty of doctor Contact numbers of doctor Timing of doctor Date of patient visit About patient Demographic profile Name of patient Father or guardian Name Age of patient Sex of patient Referred by Address Height

– – – –

– – – –

... ... ... ...

500 500 500 500

500 500 500 500

... ... ... ...

Based on prescription analysis ’’ ’’ ’’ ’’

’’ ’’ ’’ ’’

... ... ... ...

– 5 – –

– 5 – –

... ... ... ...

500 – 500 500

500 – 500 500

... ... ... ...

’’ ’’ ’’ ’’

’’ ’’ ’’ ’’

... ... ... ...

–

–

...

500

500

...

’’

’’

...

– –

– –

... ...

500 500

500 500

... ...

’’ ’’

’’ ’’

... ...

– 5 5 5 – –

– 5 5 5 – –

... ... ... ... ... ...

500 – – – 500 500

500 – – – 500 500

... ... ... ... ... ...

’’ ’’ ’’ ’’ ’’ ’’

’’ ’’ ’’ ’’ ’’ ’’

... ... ... ... ... ...

– – – – –

– – – – –

... ... ... ... ...

500 500 500 500 500

500 500 500 500 600

... ... ... ... ...

’’ ’’ ’’ ’’ Based on evaluation (Activity 5) and pediatrician issued prescription analysis

... ... ... ... ...

Weight

–

–

...

500

600

...

Based on user evaluation (Activity 5) and pediatrician issued prescription analysis

...

Vital sign

5

5

...

–

–

...


...

– – – – 5

– – – – 5

... ... ... ... ...

500 500 500 500 –

500 500 500 500 –

... ... ... ... ...

’’ ’’ ’’ ’’ Based on prescription analysis Based on prescription analysis Based on prescription analysis ’’ ’’ ’’ ’’ ’’

’’ ’’ ’’ ’’ ’’

... ... ... ... ...

5 5 –

5 5 –

... ... ...

– – 500

– – 500

... ... ...

’’ ’’ ’’

’’ ’’ ’’

... ... ...

– 5 – – 5 5 5 5

– 5 – – 5 5 5 5

... ... ... ... ... ... ... ...

500 – 500 500 – – – –

500 – 500 500 – – – –

... ... ... ... ... ... ... ...

’’ ’’ ’’ ’’ ’’ ’’ ’’ ’’

’’ ’’ ’’ ’’ ’’ ’’ ’’ ’’

... ... ... ... ... ... ... ...

5

5

...

–

–

...

’’

’’

...

Temperature Blood pressure Pulse rate Respiratory rate History of patient/ symptom Clinical Signs Diagnosis Doctor’s clinical opinion/diagnosis Diagnostic test Treatment Prescription/advice Minor procedure Referral Next appointment Signature of doctor Rules of clinic or medicine Extra facilities available in clinic a

Value

(–) not applicable, (. . .) pattern keep repeating, (’’) value same as above the cell.

not anticipate some of the behavior. Zheng et al. used sequential pattern analysis and a first-order Markov chain model to find the recurring navigational patterns and unanticipated clinicians’ behavior. The findings of Zheng et al. may help us to overcome some of the limitations of Activity 8, in the MORATRS approach. Third, Activity 1–8 were accomplished out of proposed nine activities. Activity 9 advocates multiple iterations (full or partial) of Activity 1–8 depending upon the complexity of rhetorical situations [7,68]. We also found some indication of the need to carry out

Activity 9. For instance, in Activity 6, the designed interface is found to be useful (73.3%), easy to use (71.9%), easy to learn (93.7%), and satisfactory (53.2%) upon usability evaluation performed by medical informaticians, as shown in Table 3. In addition, the analysis of comments received from 28 medical informaticians has revealed two broad themes – strengths and possible areas of improvements in the prototype interface (Table 4). The prototype interface has scored highly regarding the usefulness, learnability, and usage; however, the satisfaction score is relatively modest.


15

A. Kumar et al. / Journal of Biomedical Informatics xxx (2014) xxx–xxx Table 6 An example of trigger for general practitioners.a Facet

Rating

Justification

1st

2nd

3rd and so forth

1st

2nd

3rd and so forth

1st

2nd

3rd and so forth

About clinic

5

5

...

–

–

...


...

Logo of clinic Name of clinic Address of clinic Contact numbers of clinic Timing of clinic About doctor Name of doctor Registration number of doctor Qualification of doctor Specialty of doctor Contact numbers of doctor Timing of doctor Date of patient visit About patient Demographic profile Name of patient Father or guardian name Age of patient Sex of patient Referred by Address Height

– – – –

– – – –

... ... ... ...

500 500 500 500

500 500 500 500

... ... ... ...

Based on prescription analysis ’’ ’’ ’’ ’’

’’ ’’ ’’ ’’

... ... ... ...

– 5 – –

– 5 – –

... ... ... ...

500 – 500 500

500 – 500 500

... ... ... ...

’’ ’’ ’’ ’’

’’ ’’ ’’ ’’

... ... ... ...

–

–

...

500

500

...

’’

’’

...

– –

– –

... ...

500 500

500 500

... ...

’’ ’’

’’ ’’

... ...

– 5 5 5 – –

– 5 5 5 – –

... ... ... ... ... ...

500 – – – 500 500

500 – – – 500 500

... ... ... ... ... ...

’’ ’’ ’’ ’’ ’’ ’’

’’ ’’ ’’ ’’ ’’ ’’

... ... ... ... ... ...

– – – – –

– – – – –

... ... ... ... ...

500 500 500 500 500

500 500 500 500 400

... ... ... ... ...

’’ ’’ ’’ ’’ Based on user evaluation (Activity 5) and general practitioner issued prescription analysis

... ... ... ... ...

Weight

–

–

...

500

400

...

Based on user evaluation (Activity 5) and general practitioner issued prescription analysis

...

Vital sign

5

5

...

–

–

...


...

– – – – 5

– – – – 5

... ... ... ... ...

500 500 500 500 –

500 500 500 500 –

... ... ... ... ...

’’ ’’ ’’ ’’ Based on prescription analysis Based on prescription analysis Based on prescription analysis ’’ ’’ ’’ ’’ ’’

’’ ’’ ’’ ’’ ’’

... ... ... ... ...

5 5 –

5 5 –

... ... ...

– – 500

– – 500

... ... ...

’’ ’’ ’’

’’ ’’ ’’

... ... ...

– 5 – – 5 5 5 5

– 5 – – 5 5 5 5

... ... ... ... ... ... ... ...

500 – 500 500 – – – –

500 – 500 500 – – – –

... ... ... ... ... ... ... ...

’’ ’’ ’’ ’’ ’’ ’’ ’’ ’’

’’ ’’ ’’ ’’ ’’ ’’ ’’ ’’

... ... ... ... ... ... ... ...

5

5

...

–

–

...

’’

’’

...

Temperature Blood pressure Pulse rate Respiratory rate History of patient/ symptom Clinical Signs Diagnosis Doctor’s clinical opinion/diagnosis Diagnostic test Treatment Prescription/advice Minor procedure Referral Next appointment Signature of doctor Rules of clinic/ medicine Extra facilities available in clinic a

Value

(–) not applicable, (. . .) pattern keep repeating, (’’) value same as above the cell.

We can see that one-fourth medical informaticians chose to be neutral (‘neither agree nor disagree’ category). These medical informaticians could be motivated to shift towards ‘somewhat agree’ or ‘strongly agree’ categories provided Activity 9 is carried out, and the interface is improved further based on their comments (Table 4). Therefore, in the future studies, we strongly recommend for carrying out Activity 9, which would show how the interface improves through iterations.

5. Conclusions An interface design approach named MORTARS was conceptualized, which can design healthcare software interfaces. MORTARS approach was applied to design a prototype interface for the outpatient clinic software. The designed prototype interface was evaluated by medical informaticians. The evaluation has shown the prototype interface to be simple, easy to use and learn. This


16


approach might be further used to design interfaces for various software pertaining to healthcare and other domains. Acknowledgments Our special thanks to thirty-two medical informaticians for their valuable feedback. Many thanks to two anonymous reviewers for their invaluable comments and suggestions, which further helped us to improve the overall quality of this manuscript. Appendix A. Supplementary material Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.jbi.2014.02.007. References [1] Holzinger A, Kosec P, Schwantzer G, Debevc M, Hofmann-Wellenhof R, Frühauf J. Design and development of a mobile computer application to reengineer workflows in the hospital and the methodology to evaluate its effectiveness. J Biomed Inform 2011;44:968–77. [2] Ahlbrandt J, Henrich M, Hartmann BA, Bundschuh BB, Schwarz J, Klasen J, et al. Small cause – big effect: improvement in interface design results in improved data quality – a multicenter crossover study. Stud Health Technol Inform 2012;180:393–7. [3] Jaspers MW, Peute LW, Lauteslager A, Bakker PJ. Pre-post evaluation of physicians’ satisfaction with a redesigned electronic medical record system. Stud Health Technol Inform 2008;136:303–8. [4] Sittig DF, Wright A, Osheroff JA, Middleton B, Teich JM, Ash JS, et al. Grand challenges in clinical decision support. J Biomed Inform 2008;41:387–92. [5] Viitanen J, Nieminen M, Hypponen H, Laaveri T. Finnish physicians? Experiences with computer supported patient information exchange and communication in clinical work. Int J Electron Healthc 2011;6:153–73. [6] Johnson M, Willey ND. Usability failures and healthcare data hemorrhages. Secur Privacy IEEE 2011;9:35–42. [7] Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform 2012;45:1202–16. [8] Mchome S, Sachdeva S, Bhalla S. A brief survey: usability in healthcare. In: 2010 International conference on electronics and information engineering (ICEIE). IEEE; 2010. p. V1-463–7. [9] Viitanen J, Kuusisto A, Nykänen P. Usability of electronic nursing record systems: definition and results from an evaluation study in Finland. Int Perspect Health Inform Stud Health Technol Inform 2011;164:333–8. [10] Udsen LE, Jorgensen AH. The aesthetic turn: unravelling recent aesthetic approaches to human-computer interaction. Digit Creativity 2005;16:205–16. [11] Patel VL, Kushniruk AW. Interface design for health care environments: the role of cognitive science. In: Proceedings/AMIA annual symposium AMIA symposium; 1998. p. 29–37. [12] Niès J, Pelayo S. From users involvement to users’ needs understanding: a case study. Int J Med Inform 2010;79:e76–82. [13] Svanæs D, Alsos OA, Dahl Y. Usability testing of mobile ICT for clinical settings: methodological and practical challenges. Int J Med Inform 2010;79:e24–34. [14] Viitanen J, Hyppönen H, Lääveri T, Vänskä J, Reponen J, Winblad I. National questionnaire study on clinical ICT systems proofs: physicians suffer from poor usability. Int J Med Inform 2011;80:708–25. [15] Clarke AM. The reality of ICT use is failing to meet the user’s requirements. Interactions 2006;13:26–9. [16] Martikainen S, Viitanen J, Korpela M, Lääveri T. Physicians’ experiences of participation in healthcare IT development in Finland: willing but not able. Int J Med Inform 2012;81:98–113. [17] Karlsson IM, Engelbrektsson P, Larsson LE, Pareto L, Snis U, Berndtsson B, et al. Use-centred design of medical and healthcare technology: a pilot study of field tests as a development tool. Int J Biomed Eng Technol 2011;5:11–28. [18] Kushniruk AW, Borycki EM, Kannry J. Commercial versus in-situ usability testing of healthcare information systems: towards ‘‘public’’ usability testing in healthcare organizations. Stud Health Technol Inform 2013:157–61. [19] Silberschatz A, Peterson JL. Operating system concepts. Reading, Mass.: Addison-Wesley; 1988. [20] Bergmann GJ. A programming language and operating system for parallel processing computing machines; 1992. [21] Gabbrielli M, Martini S. Programming languages principles and paradigms. London: Springer-Verlag London Limited; 2010. [22] Accardi J. Encyclopedia of computers and computer history. Libr J 2002;127:86. [23] de Chambeau A, Radcliff CJ. Encyclopedia of computers and computer history. Ref User Serv Q 2001;41:188. [24] Markgren S. Computer history museum. Choice Curr Rev Acad Libr 2006;43:1574. [25] Plosker G. Computer history living well in San Francisco. EContent 2005;28:12–3. [26] Winward KD. Core memory: a visual survey of vintage computers featuring machines from the computer history museum. Choice Curr Rev Acad Libr 2007;45:663.

[27] Clements A. The principles of computer hardware. Oxford University Press; 2000. [28] Charette RN. Why software fails [software failure]. Spectr IEEE 2005;42:42–9. [29] Pressman RS. Software engineering: a practitioner’s approach. 6th McGrawHill Higher Education; 2005. [30] Kannampallil TG, Schauer GF, Cohen T, Patel VL. Considering complexity in healthcare systems. J Biomed Inform 2011;44:943–7. [31] Banker RD, Datar SM, Kemerer CF, Zweig D. Software complexity and maintenance costs. Commun ACM 1993;36:81–94. [32] Olson MH, Ives B. User involvement in system design: AN empirical test of alternative approaches. Info Manag 1981;4:183–95. [33] Tait P, Vessey I. The effect of user involvement on system success: a contingency approach. MIS Q 1988;12:91–108. [34] Standish. Chaos. Standish Group; 1995. [35] McCafferty D. Why IT projects fail; 2010. [36] Morgan M, Mates J, Chang P. Toward a user-driven approach to radiology software solutions: putting the wag back in the dog. J Digit Imaging 2006;19:197–201. [37] Begun JW, Zimmerman B, Dooley K. Health care organizations as complex adaptive systems. Adv Health Care Organ Theory 2003;253:288. [38] Rouse WB. Health care as a complex adaptive system: implications for design and management. Bridge 2008;38:17. [39] Rouse WB. Engineering complex systems: implications for research in systems engineering. IEEE Trans Syst Man Cybern C Appl Rev 2003;33:154–6. [40] Johnson J. Designing with the mind in mind: simple guide to understanding user interface design rules. Amsterdam; Boston: Morgan Kaufmann Publishers/Elsevier; 2010. [41] Kushniruk A, Patel V. Cognitive computer-based video analysis: its application in assessing the usability of medical systems. Medinfo MEDINFO 1995;8:1566. [42] Kushniruk A, Patel V, Cimino J, Barrows R. Cognitive evaluation of the user interface and vocabulary of an outpatient information system. In: Proceedings of the AMIA annual fall symposium. American Medical Informatics Association; 1996. p. 22. [43] Kushniruk AW, Patel VL. Cognitive evaluation of decision making processes and assessment of information technology in medicine. Int J Med Inform 1998;51:83–90. [44] Jaspers M, Steen T, Van Den Bos C, Geenen M. The think aloud method: a guide to user interface design. Int J Med Inform 2004;73:781. [45] Martijn van W. Task-based user interface design. Amsterdam: UV University; 2001. [46] Jaspers MW. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform 2009;78:340–53. [47] Bitzer LF. The rhetorical situation. Philos Rhetoric 1968;1:1–14. [48] Swales J. Occluded genres in the academy: the case of the submission letter. John Benjamins; 1996. p. 45–58. [49] Swales J. Genre analysis: English in academic and research settings. Cambridge England; New York: Cambridge University Press; 1990. [50] Swales J. Research genres: explorations and applications. Cambridge, UK; New York: Cambridge University Press; 2004. [51] Pressman R. Software Engineering – a practitioner’s approach. 7th ed. London: McGraw-Hill; 2010. [52] Nielsen J. Iterative user-interface design. Computer 1993;26:32–41. [53] Rich E. User modeling via stereotypes. Cogn Sci 1979;3:329–54. [54] Rich E. Users are individuals: individualizing user models. Int J Man Mach Stud 1983;18:199–214. [55] Bhatia VK. Analysing genre: language use in professional settings. London; New York: Longman; 1993. [56] Bhatia VK. Interdiscursivity in professional communication. Discourse Commun 2010;4:32–50. [57] Lund AM. Measuring usability with the USE questionnaire. STC Usability SIG Newslett 2001:8. [58] Biber D, Connor U, Upton TA. Discourse on the move: using corpus analysis to describe discourse structure. Amsterdam; Philadelphia: John Benjamins Pub.; 2007. [59] Upton TA, Cohen MA. An approach to corpus-based discourse analysis: the move analysis as example. Discourse Stud 2009;11:585–605. [60] ETSI E. Human Factors (HF). Guidelines for ICT products and services. [61] Zheng K, Padman R, Johnson MP, Diamond HS. An interface-driven analysis of user interactions with an electronic health records system. J Am Med Inform Assoc 2009;16:228–37. [62] Kaipio J. Usability in healthcare: overcoming the mismatch between information systems and clinical work [Doctoral Thesis]. Helsinki: School of Computer Science and Engineering, Aalto University; 2011. [63] Cooper A, Reimann R, Cronin D. About face 3: the essentials of interaction design. Wiley; 2012. [64] Norman DA, Collyer B. The design of everyday things. Tantor Media, Incorporated; 2011. [65] Bennett KB, Flach JM. Display and interface design: subtle science, exact art. CRC Press; 2011. [66] Eberts RE. User interface design. Prentice-Hall, Inc.; 1994. [67] Hinze-Hoare V. The review and analysis of human computer interaction (HCI) principles. arXiv, preprint arXiv:07073638. 2007. [68] Taylor HA, Sullivan D, Mullen C, Johnson CM. Implementation of a usercentered framework in the development of a web-based health information database and call center. J Biomed Inform 2011;44:897–908.


Designing behavioral self-regulation application for preventive personal mental healthcare.

Designing Guiding Systems for Brain-Computer Interfaces.

PRISE2: software for designing sequence-selective PCR primers and probes.

[Interfaces in healthcare].

MCHS: an application software for family welfare programmes.

Designing and optimizing a healthcare kiosk for the community.

Taking ergonomics to the bedside--a multi-disciplinary approach to designing safer healthcare.

Lean methodology: an evidence-based practice approach for healthcare improvement.

Development and application of new quality model for software projects.

An updated approach to healthcare-associated meningitis.

sgRNAcas9: a software package for designing CRISPR sgRNA and evaluating potential off-target cleavage sites.

From cineradiography to biorobots: an approach for designing robots to emulate and study animal locomotion.

Transdiagnostic Case Conceptualization of Emotional Problems in Youth with ASD: An Emotion Regulation Approach.

Designing of a Computer Software for Detection of Approximal Caries in Posterior Teeth.

Making healthcare safer by understanding, designing and buying better IT.

Fluorescence modulation by absorbent on solid surface: an improved approach for designing fluorescent sensor.

In silico approach towards designing virtual oligopeptides for HRSV.

Modelface: an Application Programming Interface (API) for Homology Modeling Studies Using Modeller Software.

A multi-scale approach to designing therapeutics for tuberculosis.

User-specific interfaces for clinical data-management systems: an object-based approach.

Scattering approach for fluctuation-induced interactions at fluid interfaces.

Designing Psychological Treatments for Scalability: The PREMIUM Approach.

An innovative approach to teaching bioethics in management of healthcare.

Designing an architectural style for dynamic medical Cross-Organizational Workflow management system: an approach based on agents and web services.