CIN: Computers, Informatics, Nursing
& Vol. 32, No. 12, 596–605 & Copyright B 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins
F E A T U R E A R T I C L E
Measuring Nursing Informatics Competencies of Practicing Nurses in Korea Nursing Informatics Competencies Questionnaire SEON YOON CHUNG, MSN, RN NANCY STAGGERS, PhD, RN, FAAN
In today’s rapidly changing and technologically rich healthcare delivery system, nurses worldwide are incorporating information and communication technologies (ICTs) into clinical settings. In the early 2000s, the Institute of Medicine in the US strongly recommended that hospitals use electronic solutions to improve the quality of care provided to hospitalized patients.1 As part of the American Recovery and Reinvestment Act of 2009, healthcare facilities across US are expected to use electronic documentation systems to meet Meaningful Use criteria and to be able to receive the Centers for Medicare & Medicaid Services financial incentive payments authorized by the Health Information Technology for Economic and Clinical Health Act.2,3 Recent literature on the effects of health information technology (IT) suggested a large number of positive impacts from the expansion of health IT in the healthcare system.4 Incorporating ICT such as electronic health records (EHRs) in clinical settings assisted clinical decision making,5,6 enabled evidence-based practice,7 improved the efficiency of nursing practice,8,9 and increased patient safety.10 Electronic health records can have unintentional consequences11 and can be more controversial among physicians in ambulatory practices (who may have rushed to install EHRs to meet Meaningful Use requirements),12 but EHR use among nurses is becoming ubiquitous. No matter the setting, nurses are using ICTs in their practices, and this will only continue to expand in the future. 596
Informatics competencies are a necessity for contemporary nurses. However, few researchers have investigated informatics competencies for practicing nurses. A full set of Informatics competencies, an instrument to measure these competencies, and potential influencing factors have yet to be identified for practicing nurses. The Nursing Informatics Competencies Questionnaire was designed, tested for psychometrics, and used to measure beginning and experienced levels of practice. A pilot study using 54 nurses ensured item comprehension and clarity. Internal consistency and face and content validity were established. A crosssectional survey was then conducted on 230 nurses in Seoul, Korea, to determine construct validity, describe a complete set of informatics competencies, and explore possible influencing factors on existing informatics competencies. Principal components analysis, descriptive statistics, and multiple regression were used for data analysis. Principal components analysis gives support for the Nursing Informatics Competencies Questionnaire construct validity. Survey results indicate that involvement in a managerial position and self-directed informaticsrelated education may be more influential for improving informatics competencies, whereas general clinical experience and workplace settings are not. This study provides a foundation for understanding how informatics competencies might be integrated throughout nurses’ work lives and how to develop appropriate strategies to support nurses in their informatics practice in clinical settings. KEY WORDS Competency assessment & Cross-sectional studies & Nurses & Nursing informatics
Because of the inevitable integration of health IT worldwide in healthcare, competencies for ICTs have become essential requirements for nurses as they fulfill their roles as healthcare providers.13,14 The requirement for informatics Author Affiliation: School of Nursing, University of Maryland, Baltimore, MD. The authors have disclosed that they have no significant relationship with, or financial interest in, any commercial companies pertaining to this article. Corresponding author: Seon Yoon Chung, MSN, RN, 655 W Lombard St, Baltimore, MD 21201 ([email protected]). DOI: 10.1097/CIN.0000000000000114
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
competencies (ICs) in nursing education is now well accepted in numerous countries. In the US, accrediting agencies such as the American Academy of Colleges of Nursing include informatics as core content at all levels of education. Despite these requirements, ICs for some groups of nurses, such as practicing nurses, are not well known.
Background Competence is defined as the state of being well qualified. It refers to an individual’s capability to understand and do certain tasks consistent with the expectations for a particular specialty. Competencies are elements of competence,15 composed of knowledge, skills, and attitudes necessary for professional practice that are observable, measurable, and practical.16 They correlate with demonstrated ability or performance that can be evaluated against expectations and can be improved through training.17 Nursing Informatics is an ‘‘integration of nursing science, computer and information science, and cognitive science to manage, communicate, and expand the data, information, knowledge, and wisdom of nursing practice.’’13(p1) Identifying, collecting, processing, and managing data and information support administration, education, and research of all nursing specialties, in all sites and settings, at all basic to advanced level.18 Nursing ICs, the competencies necessary for nursing informatics practice, can therefore be defined as the knowledge, skills, and attitudes to integrate nursing science, computer, and information science to Identify, collect, process, manage, communicate, and expand data, information, knowledge, and wisdom in nursing practice.13,17,19 Since the mid-80s, nurse researchers and educators have completed various efforts on identifying core IC for nurses20–24 as well as specific competencies needed by nurse practitioners,25 managers,26 nurse leaders,27 and nursing informatics specialists.13,23,28 Public organizations including the American Nurses Association, the American Medical Informatics Association, Healthcare Information Management and Systems Society (HIMSS), the National League for Nursing, the American Association of Colleges of Nursing, and the Technology Informatics Guiding Educational Reform (TIGER) Initiative have put efforts to identify core ICs for nurses.29 However, across the myriad efforts, the majority of past private and public organizations concentrated on developing and evaluating ICs for entry-level undergraduate nurses30–35 or graduate nurses’ education.36 These multiple consensus and research efforts resulted in a wide variety of lists of ICs, but very little attention has been paid to the competencies needed by practicing nurses. Schleyer and colleagues37 described a multiyear operational effort to translate Staggers’ work23 and later the TIGER competencies24 into nursing practice. Only a handful of researchers investigated the ICs of practicing nurses. Of these, authors examined only partial components of essential ICs such as computer skills or information literacy.38–40 One set of
authors, Hwang and Park,41 investigated ICs and associated factors of practicing nurses using a limited set of seven items on security and confidentiality, knowledge management, information management, communication, clinical service audit, clinical information systems, and telehealth. The larger set of ICs,13,20,23 potential influencing factors such as clinical experience or computer use at work, has yet to be identified, and an instrument to measure full set of ICs has not been developed and tested for practicing nurses in Korea or elsewhere in the world. The purpose of this study was therefore to develop an instrument to measure full set of ICs at the beginning and experienced levels for practicing nurses, to examine the competencies at a large hospital, and to explore the possible influence of selected clinical factors on the level of ICs. Identifying the status of ICs among practicing nurses could be the basis to develop job-specific standardized ICs for practicing nurses along with the appropriate evaluation tools. If gaps are identified, then targeted training can be offered to each practicing nurse to obtain the necessary skills in their given positions.42 Knowing the ICs for practicing nurses might also form the foundation for understanding how ICs might be integrated throughout nurses’ work lives as well as how informatics support can be provided in the clinical setting.
Development of the Nursing Informatics Competencies Questionnaire The Nursing Informatics Competencies Questionnaire (NICQ) included a total of 112 items with 53 competencies for beginning and 59 competencies for the experienced nurse. Competencies were presented as items on a five-point Likert scale (1 = very low, 2 = low, 3 = average, 4 = high, 5 = very high), and participants were asked to rank their perceived level of competency on each item. Beginning and experienced levels of nursing ICs were derived from Chang and colleagues.20 This master list is an updated set of 318 competencies modified for the Asian population based on (1) a master list of nursing ICs for nurses at four levels in the US23,43 and (2) 45 new or edited items derived from a literature review. Chang and colleagues’ work is based on an original master list of competencies developed by Staggers et al,23,43 who used a Delphi technique to construct the list, representing a high level of content validity for this initial list. The master list consisted of three major domains: computer skills, informatics knowledge, and informatics skills.
These three major domains included 39 categories of ICs necessary for nurses at four levels of practice (Table 1). The
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
597
Ta b l e 1
ways to protect data and general applications to support clinical care.
Map of Domains and Categories of the Original Master List of Competencies by Staggers et al23,43 No. of Items Domain
Category
Beginning Experienced Nurse Nurse
Nursing ICs Computer skills Administration Communication (e-mail, Internet, telecommunications) Data access Documentation Education Monitoring Basic desktop software Systems Quality improvement Research
37 25
32 14
2 3
5 —
4 3 1 1 3 8 — — 12
3 — 3 1 — — 1 1 11
Data Research Impact Privacy/security Systems
1 — 5 2 4 —
3 1 1 2 4 7
Evaluation Role Systems maintenance
— — —
2 4 1
Informatics knowledge
Informatics skills
Informatics skills are applied only to the experienced level of practice. These include assessing the accuracy of health information on the Internet, acting as an advocate of system users including patients or clients, and performing basic troubleshooting in applications (Table 1).
In the NICQ, two categories (evidence-based and information literacy) and 49 items were added to the original list of competencies for the beginning and experienced level of practice (Table 2). For this study, the 112 NICQ items for beginning and experienced level of nurses were translated into Korean and back-translated to validate that the questionnaire was reflective of the original items. T a b l e 2 Map of Domains and Categories of the NICQ List of Competencies by Chang et al20 No. of Items Domain
administrative applications, telecommunication devices, database applications, operating systems, peripheral devices, and information management technologies for patient education.
598
53 29
59 19
2 3
6 1
5 3 1 1 4 9 1 24
3 — 4 1 — — 1 3 23
Data Research Evidence based Information literacy Impact Privacy/security Systems Education
1 — — 7 6 4 4 2 —
3 7 2 1 3 2 4 1 17
Evaluation Role Systems maintenance
— — —
2 5 10
Administration Communication (e-mail, Internet, telecommunications) Data access Documentation Education Monitoring Basic desktop software Systems Quality improvement Research Informatics knowledge
At the beginning and experienced nurse levels, informatics knowledge includes recognizing the use and/or importance of nursing data for improving practice; recognizing the value of clinicians’ involvement in the design, selection, implementation, and evaluation of applications and systems in healthcare; describing general applications available for research; and
Beginning Experienced Nurse Nurse
Nursing IC Computer skills
beginning- and experienced-level computer skills (basic system competencies for interacting with computers) concentrate on technology usage. For example, computer skills for nurses at beginning and experienced levels of practice include using
Category
Informatics skills
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
PILOT TEST A pilot test was conducted to clarify any ambiguity in the items, to ensure item comprehension, and to assess initial internal consistency and content validity. The pilot study participants were a convenience sample of 54 nurses who were taking courses in a graduate nursing school. More than half of the nurses (53.6%) worked in medicalsurgical unit setting, whereas the others worked in specialty units or outpatient departments. The mean age was 30.4 (SD, 4.9) years (range, 24–44 years), and their mean number of years of work experience was 3.8 (SD, 1.0) years (range, 2–5 years). Instrument clarity. Major comments from the pilot study were that the nurses wanted examples for the system competencies (eg, technology resources refer to computers,
EHR, Internet) or informatics skills (eg, manage data refers to collect, analyze, evaluate, report, apply data). Changes were made to the NICQ after the pilot study that included minor changes to item wording. Brief examples were added to the NICQ with help from a panel of four experts, one of whom was an informatics expert, one was a nursing expert, and two were informatics clinicians. For instance, telecommunication devices were changed from ‘‘modem and other devices’’ to ‘‘LAN and other devices.’’ Minor additions were made such as adding definitions to terms like ‘‘administrative applications.’’ Examples were added to terms such as ‘‘telecommunication,’’ ‘‘peripheral bedside terminals,’’ or ‘‘handhelds’’ (Table 3). Initial psychometric testing. Internal consistency of the NICQ for the overall instrument was assessed using
T a b l e 3 Examples of Edited 112- item nursing Informatics Competencies Questionnaire (NICQ) Beginning Nurse Competencies Computer skills
Communication (e-mail, Internet, telecommunications) Data access Documentation Monitoring Basic desktop software Systems
Informatics knowledge Information literacy Privacy/security Education
Uses telecommunication devices (eg, LAN or other devices) to communicate with other systems (eg, access data, upload, download) Uses decision support systems, expert systems and other aids for clinical decision making and care planning (eg, drug interaction or dosage alert) Uses an application (eg, electronic medical record [EMR]) to enter patient data (eg, vital signs, demographic, and physiological data) Uses computerized patient monitoring systems (eg, electrocardiogram [ECG]) Uses spreadsheet application (table calculation software), such as Microsoft Excel Operates peripheral devices—hardware connected to computer such as monitor, keyboard, printer, scanner (eg, bedside terminals [screen, keyboard], handhelds [mobile tablets]) Understands and applies essential information-seeking concepts and practices (such as recognizing problems, interpret, search, and implementing) Describes patients’ rights (eg, view, copy, request correction, confidentiality) as they pertain to computerized information management Analyzes patient information needs, accesses technology resources (eg, computer, Internet, EMR) to meet needs and evaluates effectiveness
Experienced Nurse Competencies Computer skills
Administration Communication
Data access Monitoring Research
Informatics knowledge Research Evidence based Impact Informatics skills
Role Systems maintenance
Uses administrative applications (software to analyze, predict, and report data) for forecasting Uses telecommunications (telephone, messaging, e-mail), to support care delivery, empowers the consumer, transforms education, and enhances decision making Accesses shared data sets (eg, hospital information, scholarly information) Applies monitoring system (eg, ECG) appropriately according to the data needed Uses applications to aggregate and analyzes data for forecasting, accreditation, clinician value, nurse-sensitive outcomes (NOC), evidence-based practice, and quality improvement Describes general applications available for research (eg, Excel, MS Word) Synthesizes data from 91 source (eg, EMR, Internet) and applies to practice Acts as an advocate of system users (eg, know and deliver users’ need, increase perception of system) including patient and colleagues Participates in quality management initiatives related to patient and nursing data in practice (eg, accuracy, reliability, completeness, integrity) Evaluates the appropriateness of the monitoring systems (eg, ECG) for the type of data needed
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
599
Cronbach’s !. An ! of .7 or more was considered to be evidence of reliability.44 The calculated Cronbach’s ! coefficient indicated high initial internal consistency (total instrument = .981). Internal consistency of the NICQ for the three subscales (computer skills, informatics knowledge, informatics skills) was assessed later as is described in the following sections. Two methods were used to assess the initial validity of the instrument: face validity for content validity and construct validity. The original list of competencies was derived from literature review and Delphi method supporting content validity. A panel of nursing experts assessed the instrument for content validity, and the NICQ was deemed adequate. Construct validity was assessed as part of the survey and initial application of the NICQ.
INITIAL APPLICATION OF THE NICQ
INSTRUMENTATION A demographic form and the NICQ were given to participants. A self-report questionnaire was designed to collect data on sociodemographics, clinical factors, and nursing ICs for the sample. The specific sociodemographic factors and nursing ICs are explained in the following sections. Demographic factors. Demographic factors were carefully chosen to include in this study: clinical experience, staff position, working site, hours of computer use at work, and the prior informatics-related education received. These factors were based on a previous study.41 PROCEDURES
Methods This phase of the research used a cross-sectional, descriptive, correlational design, and the study was approved by the local institutional review board. SETTING The setting was a tertiary teaching hospital located in Seoul, Korea, a facility accredited by the Joint Commission International. This facility has about 850 inpatient beds with more than 600 RNs and 400 physicians. The hospital offers a comprehensive range of primary and specialized care services including maternity, oncology, psychiatric, orthopaedics, and neurology. At the time of study, this facility was at stage 2–3 of the HIMSS EHR Adoption Model45 with a house-wide, vendor-based computerized order entry including decision support capabilities, medical imaging using picture archiving and communication systems (PACS), computerized admission, billing and patient management, and ancillary clinical systems (ie, pharmacy, laboratory, and radiology) in place for approximately 10 years. Computerized nursing and other clinical documentation were in the process of being installed. PARTICIPANTS The participants in this phase were a convenience sample recruited through the unit managers. The inclusion criteria for this study were all practicing clinical nurses with greater than 6 months of experience and included both full-time and part-time nurses. The focus of this work was on practicing nurses’ ICs to include experienced nurses as defined in the four different levels of nurses described originally by Staggers and colleagues.23,43 Defining nursing ICs and developing standardized nursing informatics education based on nursing 600
ICs were both proposed in Korea,46 but the integration of nursing ICs into education was not an official national direction. Instead, the choice was up to individual faculty and schools. The practicing nurses who participated in the study are not likely to have had informatics education integrated into their curricula during their entry-level preparation.
Data collection. Data were collected using a survey method and printed forms. The researcher explained the questionnaires and gave them to nurse managers in each department to distribute to eligible participants. All managers and participants were provided with an explanation about the study in the cover letter, and informed consents were collected. DATA ANALYSES Demographics and nursing informatics competency. Participants’ sociodemographics, clinical factors, and ICs were summarized using descriptive statistics. Multiple regression models were used to determine how much unique variance of the nursing ICs each clinically related factor explained. Mean scores for the 112 nursing ICs items and each of the three subscales were used for regression analysis. Five factors were included in regression model: years of clinical work experience, staff position, work site, hours of computers use at work, and any previous informaticsrelated education. A sample size of 90 was needed for sufficient power to conduct a regression analysis using the five independent variables in the regression model: N 9 50 + 8 (no. of IV: 5).47 Assumptions for multiple regression were tested including multicolinearity (r G 0.9, variance inflation factor G2, tolerance 90.5). The significance level was set at P G .05. The statistical software SPSS version 21 (IBM Corp, Armonk, NY) was used for data analyses.
RESULTS Participant characteristics A total of 230 surveys were distributed, and 228 were retrieved, a 99.1% response rate. Twenty surveys were excluded because of incomplete data or issues with
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
eligibility criteria; 208 nurses were included in final data set. As shown on Table 4, majority of the participants were female (97.6%) with mean age of 30.64 (SD, 6.09) years. Nurses’ work sites in Korea are commonly grouped as medical-surgical and other areas in contrast to inpatient and outpatient seen in other countries. Less than half of the participants (40.4%) worked in a medical-surgical unit setting, whereas the others worked in specialty units or outpatient departments. Mean number of years of clinical experience was 7.78 (SD, 5.36) years, and 13.9% of the participants were in a position involving managerial tasks such as nurse manager.
Psychometrics The Cronbach’s ! coefficient was reinforced with high internal consistency (total instrument = .981, computer skills = .957, information knowledge = .965, and information skills = .944). Construct validity was examined more formally by using principal component analysis. Factors with eigenvalue of 1 or greater were extracted as valid based on the Kaiser criterion. Initially, a scree plot was used to determine the
number of factors to be retained. Significant findings from Bartlett test of sphericity (# 26216 = 20138.642, P G .001) supported the factorability of the correlation matrix, and the high value of the Kaiser-Meyer-Olkin test (0.90) showed adequate sampling.48 Initially, 21 factors explained 72.81% of the total variance in the scale. Based on the Scree plot (Figure 1), the number of domains in the original list of competencies (computer skills, informatics knowledge, informatics skills), and the comprehensibility of the results, three factors were extracted from 112 items. In general, items representing informatics skills loaded onto factor 1 (factor loadings = 0.476–0.732) explaining 34.4% of the variance, items on informatics knowledge loaded onto factor 2 (factor loadings = 0.019–0.700) explaining 6.7%, and the items on computer skills loaded onto factor 3 (factor loadings= 0.102–0.674) explaining 3.8% of variance in total ICs, adding up to 44.9%. Factor 1 (informatics skills): all items about informatics skills loaded onto factor 1 Factor 2 (informatics knowledge): all items about informatics knowledge at beginning nurse level loaded onto factor 2, except for six items (items 32, 33,
T a b l e 4 Demographics, Clinical Factors, and ICs in Practicing Nurses (n = 208) Characteristics Gender Education
Previous informatics education Work site Staff position Age, y
Clinical work experience, y
Use of computers at work NICQ Computer skills Informatics knowledge Informatics skills
Categories
n
%
Male Female Associate degree Bachelor’s degree Master’s degree and above Yes No Medical-surgical unit Others Nonmanaging Manager and above Total G30 30–40 Q40 Total 0.5 e to G 5 5 e to G10 Q10 Total Sum Item Sum Item Sum Item Sum Item
5 203 94 98 16 140 68 84 124 179 29
2.4 97.6 45.2 47.1 7.7 67.3 32.7 40.4 59.6 86.1 13.9
110 76 22
52.9 36.5 10.6
73 75 60
35.1 36.1 28.8
Range
Mean (SD)
22.00–51.00
30.64 (6.09)
0.67–25.08
7.78 (5.36)
0.50–13.00 200–509 1.79–4.54 77–224 1.60–4.67 97–218 2.06–4.64 20–82 1.18–4.82
5.65 369.44 3.30 157.88 3.29 158.73 3.38 52.84 3.11
(3.18) (56.37) (0.50) (26.77) (0.56) (23.47) (0.50) (10.10) (0.59)
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
601
FIGURE 1. Scree plot of NICQ
34, 35, 37, and 52), which loaded onto computer skills, and three items (items 44, 51, and 53), which loaded onto informatics skills. As for items about informatics knowledge at experienced nurse level, only three items loaded onto the informatics knowledge (items 73, 74, and 75 on data). All the other items loaded to informatics skills instead of informatics knowledge. Factor 3 (computer skills): all items on computer skills at beginning nurse level loaded to factor 3 except for seven items, which loaded onto informatics knowledge. As for items on computer skills at experienced nurse level, only two items loaded onto computer skills (items 65 and 67). All the other items loaded to informatics skills instead of computer skills, except for one item (item 60), which loaded onto informatics knowledge.
Overall, this analysis supports adequate construct validity for the NICQ accounting for different levels of competencies in each domain, although some specific items were rearranged. The specific results are available from the first author.
Nursing Informatics Competencies and Relationship Between Participant Characteristics The beginning and experienced levels of nursing ICs in this sample were just above midrange for the mean total scale score (mean NIC, 3.30 [SD, 0.503]). The informatics knowledge subscale was the highest (mean, 3.38 [SD, 0.499]) followed by computer skills (mean, 3.29 [SD, 0.558]) and informatics skills (mean, 3.11 [SD, 0.594]). In the regression analysis, age was excluded from further consideration because of its suspected multicolinearity (r = 0.949, P G .001) with clinical work experience. For the total NICQ instrument, significant associations were 602
shown with staff position (beta = j.304, P G .001), previous informatics-related education (beta = .187, P = .005), and hours of computer use at work (beta = .144, P = .045), while controlling for all other variables in the model (Table 5). Hours of computer use at work increased mean scores on the NICQ, although minimal (beta = .023; 95% confidence interval [CI], G.001-to .045). Nurses not involved in a managerial role had lower scores on the NICQ and so had lower scores compared with nurses involved in a managerial role (beta = j.441; 95% CI, j.680 to j.202). Nurses with previous self-directed informatics-related education had higher scores for nursing ICs (beta = .200; 95% CI, .062–.338). Results were similar for the computer skills subscale. Significant associations were shown with staff position (beta = j.305, P G .001), previous informatics-related education (beta = .162, P = .014), and hours of computer use at work (beta = .154, P = .031). Likewise, results were similar for the informatics knowledge and informatics skills subscales, except that the hours of computer use at work was not statistically significant for both informatics knowledge (beta =.123, P = .090) and informatics skills (beta =.109, P = .146).
DISCUSSION The purposes of this research were to develop an instrument to measure a full set of ICs at the beginning and experienced levels among practicing nurses, to examine the competencies at a large hospital, and to explore the possible influence of selected clinical factors on the level of informatics competency. The measure has adequate reliability as assessed by a high internal consistency (! = .98) and no multicolinearity indicated (no bivariate correlations among items, r 9 0.90). Initial instrument validity was supported with face, content, and construct validity. Three factors matching three domains of informatics competency explained 44.9% of variance in total ICs. Originally, items about informatics skills were only available as ICs for the experienced nurse level. Interestingly, most of the items on computer skills and informatics knowledge at the experienced nurse level loaded more onto informatics skills than within their own domain. This may be because informatics skills are indeed experienced level ICs, and perhaps informatics knowledge and computer skills at experienced level are more representative of informatics skills. This sample of practicing nurses had an above-average level of informatics competency at beginning and experienced levels on the NICQ. Findings also show that staff position (involvement in managerial position), previous informatics-related education, and use of computers at work (more hours) significantly account for the variance
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
" is unstandardized b; beta, standardized b. Predictors in the model: years of clinical work experience, work position, work site, use of computers at work, and any previous informatics-related education. NICQ model (F = 6.685, P e .01; R 2 = 0.142 [adjusted R 2 = 0.377]); computer skill model (F = 7.247, P e .01; R 2 = 0.152 [adjusted R 2 = 0.131]); informatics knowledge model (F = 5.997, P e .01; R 2 = 0.129 [adjusted R 2 = 0.108]); informatics skill model (F = 2.873, P = .016; R 2 = 0.066 [adjusted R 2 = 0.043]). a P G .05.
Nonmanager vs manager j.441 0.121 j.304 j3.636 Previous informatics-related .200 0.070 .187 2.862 education vs no education Computer use at work, h/d .023 0.011 .144 2.013 Clinical work experience, y .000 0.008 j.004 j0.042 Medical-surgical unit vs others j.028 0.074 j.028 j0.379
.045a .027 .012 .154 2.175 .031a .019 0.011 .123 1.702 .090 .020 0.014 .109 1.460 .146 .967 .004 .009 .035 0.415 .679 j.001 0.008 j.014 j0.161 .872 j.009 0.010 j.081 j0.911 .364 .705 j.015 .082 j.013 j0.183 .855 j.028 0.074 j.027 j0.376 .708 j.066 0.091 j.055 j0.726 .469
G.001 j.491 .134 j.305 j3.674 G.001 j.415 0.121 j.288 j3.424 .001 j.372 0.149 j.218 j2.495 .013a .005a .192 .077 .162 2.486 .014a .213 0.070 .200 3.037 .003a .191 0.086 .151 2.218 .028a
P t beta SE "
SE
beta
t
P
a
"
SE beta
t
P
a
"
SE
beta
t
P
a
"
Informatics Skill Informatics Knowledge Computer Skill NICQ
Factors Predicting NIC, Computer Skills, Informatics Knowledge, and Informatics Skills Controlling for Years of Clinical Work Experience, Work Position, Work Site, Use of Computers at Work, and Any Previous Informatics-Related Education (n = 208)
T a b l e 5
in nursing ICs of nurses, whereas general clinical experience and workplace do not. Involvement in managerial position and informatics-related education significantly accounted for variance on all subscales (computer skills, informatics knowledge, informatics skills), whereas use of computers at work accounted solely for computer skills. Because few studies are available about nursing ICs in practicing nurses, our ability to compare findings across studies is somewhat limited. The level of nursing informatics competency in this facility may be higher than other settings because the facility is an urban, teaching hospital with an EHR installed base for over 10 years. The finding about self-directed informatics education being related to higher nursing ICs is congruent with a former study in Korean nurses.41 Although this is not a surprising finding, it should be noted that no information nurse specialists were included in this sample, so the higher level is more impressive among this group of nurses who had little informatics training during their initial nursing preparation. On the other hand, Hwang and Park41 reported higher nursing ICs in staff nurses compared with nurse managers— the opposite of the findings here. Administrators likely use computers more in depth and breadth because of their tasks and roles. Because the demographic characteristics were similar between the two study samples, further study is warranted to reveal the true relationship between staff position or their related tasks and ICs in nurses. Perhaps the difference in the tasks completed for each position should be identified to illuminate the true influencing factors on informatics competency. The factor concerning use of computers at work was not measured in other studies. Although this factor accounted for significant variance in the total score for nursing ICs of this sample, only the computer skill subscale was a significant predictor among the three subscales. Computer use is a complex concept related to both the depth and breadth of computer use as well as technology knowledge and skills.49 Why computer use is not related to informatics knowledge and skills needs to be explored in the future. Perhaps the informatics education of practicing nurses is similar among most nurses currently, resulting in the small variation seen in this sample. At any rate, further studies are needed to differentiate the influence of computer use in informatics knowledge and informatics skills and their relation to nursing ICs. Nurses are the largest part of the healthcare workforce and are the major users of the clinical information system. Nurses’ informatics competency is considered crucial for the success of clinical information systems use and for patient safety in computerized environments. The roles of the information nurses and information nurse specialists will evolve in conjunction with new technologies and changing healthcare delivery models. Continued work on ICs will be required to match the evolving nature of nursing informatics, and planned, formal education is recommended across nurses’ work lives.
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
603
Knowing the extent of ICs among practicing nurses could provide the basis for developing job-specific standardized ICs in practicing nurses along with the appropriate evaluation tools. Moreover, knowing ICs for practicing nurses could form the foundation for understanding how ICs might be integrated throughout nurses’ work lives and how to develop appropriate strategies to support nurses in their informatics practice in clinical settings. Targeted training could then be provided to facilitate nurses to obtain the necessary skills in their positions.42 Nursing educational programs may vary in what they teach about nursing ICs without central guidance. Job-specific standardized ICs in practicing nurses could be helpful in meeting informatics needs and an organized/central effort for NI competencies in the future.
Study Limitations and Future Research Several limitations to this study include that the sample was fairly homogeneous. Selecting a homogeneous sample was purposeful to increase the internal validity of the study but could have limited the extent to which the results can be generalized. The sample also had an above-average NICQ score, potentially limiting generalizability. Other settings, units, and hospitals could be used for future studies. Although the internal consistency, face, content, and construct validity were ensured for the NICQ in this study, stability and additional psychometrics could be assessed in the future. For future studies, experienced level competencies related to computer skills and informatics knowledge could be reconsidered as informatics skills, and the informatics skills domain itself could be thought of as an ‘‘experienced level and above’’ competency. The idea of recategorizing experienced level computer skills and informatics knowledge competences into informatics skills is supported by the fact that informatics skills accounted for the largest portion of the variance (34.4%) in the ICs present in NICQ, compared with informatics knowledge (6.7%) or computer skills (3.8%). Following the recategorization, the number of items in NICQ could be reduced considering the high internal consistency value. Last, the instrument could be refined to be more parsimonious in the future.
Acknowledgments The authors thank Dr Shijun Zhu at University of Maryland Baltimore, School of Nursing, for his insightful review of the statistical portion of this article, and Dr Elizabeth Borycki at University of Victoria, School of Health Information Science, for her thoughtful review of a previous version of this article. Guidance throughout the study from Drs Kyung Rim Shin, Youn Hee Kang, and Ok Su Kim at Ewha Womans University, Division of Nursing Science, as well as Dr Sun Mi Lee at The Catholic University of Korea, College of Nursing, is greatly appreciated. 604
REFERENCES 1. Institute of Medicine (IOM). Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001. 2. Kelley TF, Brandon DH, Docherty SL. Electronic nursing documentation as a strategy to improve quality of patient care. J Nurs Scholarsh. 2011;43(2):154–162. 3. Blumenthal D, Tavenner M. The ‘‘Meaningful Use’’ regulation for electronic health records. N Engl J Med. 2010;363(6):501–504. 4. Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff. 2011;30(3):464–471. 5. Chae YM, Yoo KB, Kim ES, Chae H. The adoption of electronic medical records and decision support systems in Korea. Healthc Inform Res. 2011;17(3):172–177. 6. Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc. 2011;18(3):327–334. 7. Bloomrosen M, Detmer DE. Informatics, evidence-based care, and research; implications for national policy: a report of an American Medical Informatics Association health policy conference. J Am Med Inform Assoc. 2010;17(2):115–123. 8. Banet GA, Jeffe DB, Williams JA, Asaro PV. Effects of implementing computerized practitioner order entry and nursing documentation on nursing workflow in an emergency department. J Healthc Inform Manage. 2006;20(2):45. 9. Hoo WES, Parisi LL. Nursing informatics approach to analyzing staffing effectiveness indicators. J Nurs Care Qual. 2005;20(3):215–219. 10. Parente ST, McCullough JS. Health information technology and patient safety: evidence from panel data. Health Aff. 2009;28(2):357–360. 11. Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. Int J Medical Inform. 2009;78: S69–S76. 12. Ruder DB. Malpractice claims analysis confirms risks in EHRs. Patient Saf Qual Healthc. 2014;11(1):20–23. 13. American Nurses Association. Nursing Informatics: Scope and Standards of Practice. Silver Spring, MD: American Nurses Association; 2008. 14. Nagle L. Making Informatics Competency Development Explicit. Online J Nurs Inform. 2013;17(1). 15. Kaslow NJ, Borden KA, Collins FL, et al. Competencies conference: future directions in education and credentialing in professional psychology. J Clin Psychol. 2004;60(7):699–712. 16. Stratford R. A competency approach to educational psychology practice: the implications for quality. Educ Child Psychology. 1994. 17. Parry SB. The quest for competencies. Training. 1996;33(7):48–54. 18. Staggers N, Thompson CB. The evolution of definitions for nursing informatics: a critical analysis and revised definition. J Am Med Inform Assoc. 2002;9(3):255–261. 19. Graves JR, Corcoran S. The study of nursing informatics. J Nurs Scholarsh. 1989;21(4):227–231. 20. Chang J, Poynton MR, Gassert CA, Staggers N. Nursing informatics competencies required of nurses in Taiwan. Int J Med Inform. 2011;80(5):332–340. 21. Foster J, Bryce J. Australian nursing informatics competency project. In: Proceedings of the 10th International Congress on Nursing Informatics: Connecting Health and Humans. June 8–July 1, 2009. Helsinki. QUT Digital Repository. http://eprints.qut.edu.au/31927/1/c31927a.pdf. 22. Grobe SJ. Nursing informatics competencies. Methods Inform Med. 1989;28(4):267–269. 23. Staggers N, Gassert CA, Curran C. A Delphi study to determine informatics competencies for nurses at four levels of practice. Nurs Res. 2002;51(6):383–390. 24. TIGER Initiative. Collaborating to integrate evidence and informatics into nursing practice and education: an executive summary. 2009. 25. Curran CR. Informatics competencies for nurse practitioners. AACN Adv Crit Care. 2003;14(3):320–330. 26. Hart MD. A Delphi study to determine baseline informatics competencies for nurse managers. Comput Inform Nurs. 2010;28(6): 364–370.
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
27. Westra BL, Delaney CW. Informatics competencies for nursing and healthcare leaders. AMIA Annual Symposium Proceedings. 2008:804–808. 28. Hebert M. A national education strategy to develop nursing informatics competencies. Can J Nurs Leadersh. 2000;13(2):11. 29. Hunter KM, McGonigle DM, Hebda TL. TIGER-based measurement of nursing informatics competencies: the development and implementation of an online tool for self-assessment. J Nurs Educ Pract. 2013;3(12):70–80. 30. Borycki EM, Foster J, Sahama T, Frisch N, Kushniruk AW. Developing national level informatics competencies for undergraduate nurses: methodological approaches from Australia and Canada. Stud Health Technol Inform. 2013;183:345–349. 31. Choi J. Comparative assessment of informatics competencies in three undergraduate programs. Online J Nurs Inform. 2012;16(2). 32. Choi J, de Martinis JE. Nursing informatics competencies: assessment of undergraduate and graduate nursing students. J Clin Nurs. 2013;22(13–14):1970–1976. 33. Fetter MS. Graduating nurses’ self-evaluation of information technology competencies. J Nurs Educ. 2009;48(2):86. 34. Hunter K, McGonigle D, Hebda T. The integration of informatics content in baccalaureate and graduate nursing education: a status report. Nurse Educ. 2013;38(3):110–113. 35. Kleib M, Zimka O, Olson K. Status of informatics integration in baccalaureate nursing education: a systematic review. Can J Nurs Res. 2013;45(1):138–154. 36. McBride SG, Tietze M, Fenton MV. Developing an applied informatics course for a doctor of nursing practice program. Nurse Educ. 2013;38(1):37–42. 37. Schleyer RH, Burch CK, Schoessler MT. Defining and integrating informatics competencies into a hospital nursing department. Comput Inform Nurs. 2011;29(3):167–173.
38. Campbell CJ, McDowell DE. Computer literacy of nurses in a community hospital: where are we today? J Contin Educ Nurs. 2011; 42(8):365. 39. Gilmour JA, Scott SD, Huntington N. Nurses and Internet health information: a questionnaire survey. J Adv Nurs. 2008;61(1):19–28. 40. Huang H, Lee TT. Evaluation of ICU nurses’ use of the clinical information system in Taiwan. Comput Inform Nurs. 2011;29(4): 221–229. 41. Hwang JI, Park HA. Factors associated with nurses’ informatics competency. Comput Inform Nurs. 2011;29(4):256–262. 42. Hart MD. Informatics competency and development within the US nursing population workforce: a systematic literature review. Comput Inform Nurs. 2008;26(6):320–329. 43. Staggers N, Gassert CA, Curran C. Informatics competencies for nurses at four levels of practice. J Nurs Educ. 2001;40(7):303–316. 44. Nunnally J, Bernstein I. Psychometric Theory. New York, NY: McGrawHill; 1994. 45. HIMSS Analytics. The EMR Adoption Model (EMRAM). 2014. Available at http://www.himssanalytics.org/emram/emram.aspx. Accessed November 1, 2014. 46. Park H. Direction of nursing informatics education in Korea: suggestion for nursing informatics education model. (Korean) Publications from The Research Institute of Nursing Science, Seoul National University. 2001;2001(1):43–64. 47. Tabachnick BG, Fidell LS, Osterlind SJ. Using multivariate statistics, Profile analysis: the multivariate approach to repeated measures. New York, NY: Pearson; 2001. 48. Stevens J. Applied Multivariate Statistics for the Social Sciences. New York, NY: Routledge; 2009. 49. Staggers N. The Staggers nursing computer experience questionnaire. Applied Nurs Res. 1994;7(2):97–106.
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
605
& Vol. 32, No. 12, 596–605 & Copyright B 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins
F E A T U R E A R T I C L E
Measuring Nursing Informatics Competencies of Practicing Nurses in Korea Nursing Informatics Competencies Questionnaire SEON YOON CHUNG, MSN, RN NANCY STAGGERS, PhD, RN, FAAN
In today’s rapidly changing and technologically rich healthcare delivery system, nurses worldwide are incorporating information and communication technologies (ICTs) into clinical settings. In the early 2000s, the Institute of Medicine in the US strongly recommended that hospitals use electronic solutions to improve the quality of care provided to hospitalized patients.1 As part of the American Recovery and Reinvestment Act of 2009, healthcare facilities across US are expected to use electronic documentation systems to meet Meaningful Use criteria and to be able to receive the Centers for Medicare & Medicaid Services financial incentive payments authorized by the Health Information Technology for Economic and Clinical Health Act.2,3 Recent literature on the effects of health information technology (IT) suggested a large number of positive impacts from the expansion of health IT in the healthcare system.4 Incorporating ICT such as electronic health records (EHRs) in clinical settings assisted clinical decision making,5,6 enabled evidence-based practice,7 improved the efficiency of nursing practice,8,9 and increased patient safety.10 Electronic health records can have unintentional consequences11 and can be more controversial among physicians in ambulatory practices (who may have rushed to install EHRs to meet Meaningful Use requirements),12 but EHR use among nurses is becoming ubiquitous. No matter the setting, nurses are using ICTs in their practices, and this will only continue to expand in the future. 596
Informatics competencies are a necessity for contemporary nurses. However, few researchers have investigated informatics competencies for practicing nurses. A full set of Informatics competencies, an instrument to measure these competencies, and potential influencing factors have yet to be identified for practicing nurses. The Nursing Informatics Competencies Questionnaire was designed, tested for psychometrics, and used to measure beginning and experienced levels of practice. A pilot study using 54 nurses ensured item comprehension and clarity. Internal consistency and face and content validity were established. A crosssectional survey was then conducted on 230 nurses in Seoul, Korea, to determine construct validity, describe a complete set of informatics competencies, and explore possible influencing factors on existing informatics competencies. Principal components analysis, descriptive statistics, and multiple regression were used for data analysis. Principal components analysis gives support for the Nursing Informatics Competencies Questionnaire construct validity. Survey results indicate that involvement in a managerial position and self-directed informaticsrelated education may be more influential for improving informatics competencies, whereas general clinical experience and workplace settings are not. This study provides a foundation for understanding how informatics competencies might be integrated throughout nurses’ work lives and how to develop appropriate strategies to support nurses in their informatics practice in clinical settings. KEY WORDS Competency assessment & Cross-sectional studies & Nurses & Nursing informatics
Because of the inevitable integration of health IT worldwide in healthcare, competencies for ICTs have become essential requirements for nurses as they fulfill their roles as healthcare providers.13,14 The requirement for informatics Author Affiliation: School of Nursing, University of Maryland, Baltimore, MD. The authors have disclosed that they have no significant relationship with, or financial interest in, any commercial companies pertaining to this article. Corresponding author: Seon Yoon Chung, MSN, RN, 655 W Lombard St, Baltimore, MD 21201 ([email protected]). DOI: 10.1097/CIN.0000000000000114
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
competencies (ICs) in nursing education is now well accepted in numerous countries. In the US, accrediting agencies such as the American Academy of Colleges of Nursing include informatics as core content at all levels of education. Despite these requirements, ICs for some groups of nurses, such as practicing nurses, are not well known.
Background Competence is defined as the state of being well qualified. It refers to an individual’s capability to understand and do certain tasks consistent with the expectations for a particular specialty. Competencies are elements of competence,15 composed of knowledge, skills, and attitudes necessary for professional practice that are observable, measurable, and practical.16 They correlate with demonstrated ability or performance that can be evaluated against expectations and can be improved through training.17 Nursing Informatics is an ‘‘integration of nursing science, computer and information science, and cognitive science to manage, communicate, and expand the data, information, knowledge, and wisdom of nursing practice.’’13(p1) Identifying, collecting, processing, and managing data and information support administration, education, and research of all nursing specialties, in all sites and settings, at all basic to advanced level.18 Nursing ICs, the competencies necessary for nursing informatics practice, can therefore be defined as the knowledge, skills, and attitudes to integrate nursing science, computer, and information science to Identify, collect, process, manage, communicate, and expand data, information, knowledge, and wisdom in nursing practice.13,17,19 Since the mid-80s, nurse researchers and educators have completed various efforts on identifying core IC for nurses20–24 as well as specific competencies needed by nurse practitioners,25 managers,26 nurse leaders,27 and nursing informatics specialists.13,23,28 Public organizations including the American Nurses Association, the American Medical Informatics Association, Healthcare Information Management and Systems Society (HIMSS), the National League for Nursing, the American Association of Colleges of Nursing, and the Technology Informatics Guiding Educational Reform (TIGER) Initiative have put efforts to identify core ICs for nurses.29 However, across the myriad efforts, the majority of past private and public organizations concentrated on developing and evaluating ICs for entry-level undergraduate nurses30–35 or graduate nurses’ education.36 These multiple consensus and research efforts resulted in a wide variety of lists of ICs, but very little attention has been paid to the competencies needed by practicing nurses. Schleyer and colleagues37 described a multiyear operational effort to translate Staggers’ work23 and later the TIGER competencies24 into nursing practice. Only a handful of researchers investigated the ICs of practicing nurses. Of these, authors examined only partial components of essential ICs such as computer skills or information literacy.38–40 One set of
authors, Hwang and Park,41 investigated ICs and associated factors of practicing nurses using a limited set of seven items on security and confidentiality, knowledge management, information management, communication, clinical service audit, clinical information systems, and telehealth. The larger set of ICs,13,20,23 potential influencing factors such as clinical experience or computer use at work, has yet to be identified, and an instrument to measure full set of ICs has not been developed and tested for practicing nurses in Korea or elsewhere in the world. The purpose of this study was therefore to develop an instrument to measure full set of ICs at the beginning and experienced levels for practicing nurses, to examine the competencies at a large hospital, and to explore the possible influence of selected clinical factors on the level of ICs. Identifying the status of ICs among practicing nurses could be the basis to develop job-specific standardized ICs for practicing nurses along with the appropriate evaluation tools. If gaps are identified, then targeted training can be offered to each practicing nurse to obtain the necessary skills in their given positions.42 Knowing the ICs for practicing nurses might also form the foundation for understanding how ICs might be integrated throughout nurses’ work lives as well as how informatics support can be provided in the clinical setting.
Development of the Nursing Informatics Competencies Questionnaire The Nursing Informatics Competencies Questionnaire (NICQ) included a total of 112 items with 53 competencies for beginning and 59 competencies for the experienced nurse. Competencies were presented as items on a five-point Likert scale (1 = very low, 2 = low, 3 = average, 4 = high, 5 = very high), and participants were asked to rank their perceived level of competency on each item. Beginning and experienced levels of nursing ICs were derived from Chang and colleagues.20 This master list is an updated set of 318 competencies modified for the Asian population based on (1) a master list of nursing ICs for nurses at four levels in the US23,43 and (2) 45 new or edited items derived from a literature review. Chang and colleagues’ work is based on an original master list of competencies developed by Staggers et al,23,43 who used a Delphi technique to construct the list, representing a high level of content validity for this initial list. The master list consisted of three major domains: computer skills, informatics knowledge, and informatics skills.
These three major domains included 39 categories of ICs necessary for nurses at four levels of practice (Table 1). The
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
597
Ta b l e 1
ways to protect data and general applications to support clinical care.
Map of Domains and Categories of the Original Master List of Competencies by Staggers et al23,43 No. of Items Domain
Category
Beginning Experienced Nurse Nurse
Nursing ICs Computer skills Administration Communication (e-mail, Internet, telecommunications) Data access Documentation Education Monitoring Basic desktop software Systems Quality improvement Research
37 25
32 14
2 3
5 —
4 3 1 1 3 8 — — 12
3 — 3 1 — — 1 1 11
Data Research Impact Privacy/security Systems
1 — 5 2 4 —
3 1 1 2 4 7
Evaluation Role Systems maintenance
— — —
2 4 1
Informatics knowledge
Informatics skills
Informatics skills are applied only to the experienced level of practice. These include assessing the accuracy of health information on the Internet, acting as an advocate of system users including patients or clients, and performing basic troubleshooting in applications (Table 1).
In the NICQ, two categories (evidence-based and information literacy) and 49 items were added to the original list of competencies for the beginning and experienced level of practice (Table 2). For this study, the 112 NICQ items for beginning and experienced level of nurses were translated into Korean and back-translated to validate that the questionnaire was reflective of the original items. T a b l e 2 Map of Domains and Categories of the NICQ List of Competencies by Chang et al20 No. of Items Domain
administrative applications, telecommunication devices, database applications, operating systems, peripheral devices, and information management technologies for patient education.
598
53 29
59 19
2 3
6 1
5 3 1 1 4 9 1 24
3 — 4 1 — — 1 3 23
Data Research Evidence based Information literacy Impact Privacy/security Systems Education
1 — — 7 6 4 4 2 —
3 7 2 1 3 2 4 1 17
Evaluation Role Systems maintenance
— — —
2 5 10
Administration Communication (e-mail, Internet, telecommunications) Data access Documentation Education Monitoring Basic desktop software Systems Quality improvement Research Informatics knowledge
At the beginning and experienced nurse levels, informatics knowledge includes recognizing the use and/or importance of nursing data for improving practice; recognizing the value of clinicians’ involvement in the design, selection, implementation, and evaluation of applications and systems in healthcare; describing general applications available for research; and
Beginning Experienced Nurse Nurse
Nursing IC Computer skills
beginning- and experienced-level computer skills (basic system competencies for interacting with computers) concentrate on technology usage. For example, computer skills for nurses at beginning and experienced levels of practice include using
Category
Informatics skills
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
PILOT TEST A pilot test was conducted to clarify any ambiguity in the items, to ensure item comprehension, and to assess initial internal consistency and content validity. The pilot study participants were a convenience sample of 54 nurses who were taking courses in a graduate nursing school. More than half of the nurses (53.6%) worked in medicalsurgical unit setting, whereas the others worked in specialty units or outpatient departments. The mean age was 30.4 (SD, 4.9) years (range, 24–44 years), and their mean number of years of work experience was 3.8 (SD, 1.0) years (range, 2–5 years). Instrument clarity. Major comments from the pilot study were that the nurses wanted examples for the system competencies (eg, technology resources refer to computers,
EHR, Internet) or informatics skills (eg, manage data refers to collect, analyze, evaluate, report, apply data). Changes were made to the NICQ after the pilot study that included minor changes to item wording. Brief examples were added to the NICQ with help from a panel of four experts, one of whom was an informatics expert, one was a nursing expert, and two were informatics clinicians. For instance, telecommunication devices were changed from ‘‘modem and other devices’’ to ‘‘LAN and other devices.’’ Minor additions were made such as adding definitions to terms like ‘‘administrative applications.’’ Examples were added to terms such as ‘‘telecommunication,’’ ‘‘peripheral bedside terminals,’’ or ‘‘handhelds’’ (Table 3). Initial psychometric testing. Internal consistency of the NICQ for the overall instrument was assessed using
T a b l e 3 Examples of Edited 112- item nursing Informatics Competencies Questionnaire (NICQ) Beginning Nurse Competencies Computer skills
Communication (e-mail, Internet, telecommunications) Data access Documentation Monitoring Basic desktop software Systems
Informatics knowledge Information literacy Privacy/security Education
Uses telecommunication devices (eg, LAN or other devices) to communicate with other systems (eg, access data, upload, download) Uses decision support systems, expert systems and other aids for clinical decision making and care planning (eg, drug interaction or dosage alert) Uses an application (eg, electronic medical record [EMR]) to enter patient data (eg, vital signs, demographic, and physiological data) Uses computerized patient monitoring systems (eg, electrocardiogram [ECG]) Uses spreadsheet application (table calculation software), such as Microsoft Excel Operates peripheral devices—hardware connected to computer such as monitor, keyboard, printer, scanner (eg, bedside terminals [screen, keyboard], handhelds [mobile tablets]) Understands and applies essential information-seeking concepts and practices (such as recognizing problems, interpret, search, and implementing) Describes patients’ rights (eg, view, copy, request correction, confidentiality) as they pertain to computerized information management Analyzes patient information needs, accesses technology resources (eg, computer, Internet, EMR) to meet needs and evaluates effectiveness
Experienced Nurse Competencies Computer skills
Administration Communication
Data access Monitoring Research
Informatics knowledge Research Evidence based Impact Informatics skills
Role Systems maintenance
Uses administrative applications (software to analyze, predict, and report data) for forecasting Uses telecommunications (telephone, messaging, e-mail), to support care delivery, empowers the consumer, transforms education, and enhances decision making Accesses shared data sets (eg, hospital information, scholarly information) Applies monitoring system (eg, ECG) appropriately according to the data needed Uses applications to aggregate and analyzes data for forecasting, accreditation, clinician value, nurse-sensitive outcomes (NOC), evidence-based practice, and quality improvement Describes general applications available for research (eg, Excel, MS Word) Synthesizes data from 91 source (eg, EMR, Internet) and applies to practice Acts as an advocate of system users (eg, know and deliver users’ need, increase perception of system) including patient and colleagues Participates in quality management initiatives related to patient and nursing data in practice (eg, accuracy, reliability, completeness, integrity) Evaluates the appropriateness of the monitoring systems (eg, ECG) for the type of data needed
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
599
Cronbach’s !. An ! of .7 or more was considered to be evidence of reliability.44 The calculated Cronbach’s ! coefficient indicated high initial internal consistency (total instrument = .981). Internal consistency of the NICQ for the three subscales (computer skills, informatics knowledge, informatics skills) was assessed later as is described in the following sections. Two methods were used to assess the initial validity of the instrument: face validity for content validity and construct validity. The original list of competencies was derived from literature review and Delphi method supporting content validity. A panel of nursing experts assessed the instrument for content validity, and the NICQ was deemed adequate. Construct validity was assessed as part of the survey and initial application of the NICQ.
INITIAL APPLICATION OF THE NICQ
INSTRUMENTATION A demographic form and the NICQ were given to participants. A self-report questionnaire was designed to collect data on sociodemographics, clinical factors, and nursing ICs for the sample. The specific sociodemographic factors and nursing ICs are explained in the following sections. Demographic factors. Demographic factors were carefully chosen to include in this study: clinical experience, staff position, working site, hours of computer use at work, and the prior informatics-related education received. These factors were based on a previous study.41 PROCEDURES
Methods This phase of the research used a cross-sectional, descriptive, correlational design, and the study was approved by the local institutional review board. SETTING The setting was a tertiary teaching hospital located in Seoul, Korea, a facility accredited by the Joint Commission International. This facility has about 850 inpatient beds with more than 600 RNs and 400 physicians. The hospital offers a comprehensive range of primary and specialized care services including maternity, oncology, psychiatric, orthopaedics, and neurology. At the time of study, this facility was at stage 2–3 of the HIMSS EHR Adoption Model45 with a house-wide, vendor-based computerized order entry including decision support capabilities, medical imaging using picture archiving and communication systems (PACS), computerized admission, billing and patient management, and ancillary clinical systems (ie, pharmacy, laboratory, and radiology) in place for approximately 10 years. Computerized nursing and other clinical documentation were in the process of being installed. PARTICIPANTS The participants in this phase were a convenience sample recruited through the unit managers. The inclusion criteria for this study were all practicing clinical nurses with greater than 6 months of experience and included both full-time and part-time nurses. The focus of this work was on practicing nurses’ ICs to include experienced nurses as defined in the four different levels of nurses described originally by Staggers and colleagues.23,43 Defining nursing ICs and developing standardized nursing informatics education based on nursing 600
ICs were both proposed in Korea,46 but the integration of nursing ICs into education was not an official national direction. Instead, the choice was up to individual faculty and schools. The practicing nurses who participated in the study are not likely to have had informatics education integrated into their curricula during their entry-level preparation.
Data collection. Data were collected using a survey method and printed forms. The researcher explained the questionnaires and gave them to nurse managers in each department to distribute to eligible participants. All managers and participants were provided with an explanation about the study in the cover letter, and informed consents were collected. DATA ANALYSES Demographics and nursing informatics competency. Participants’ sociodemographics, clinical factors, and ICs were summarized using descriptive statistics. Multiple regression models were used to determine how much unique variance of the nursing ICs each clinically related factor explained. Mean scores for the 112 nursing ICs items and each of the three subscales were used for regression analysis. Five factors were included in regression model: years of clinical work experience, staff position, work site, hours of computers use at work, and any previous informaticsrelated education. A sample size of 90 was needed for sufficient power to conduct a regression analysis using the five independent variables in the regression model: N 9 50 + 8 (no. of IV: 5).47 Assumptions for multiple regression were tested including multicolinearity (r G 0.9, variance inflation factor G2, tolerance 90.5). The significance level was set at P G .05. The statistical software SPSS version 21 (IBM Corp, Armonk, NY) was used for data analyses.
RESULTS Participant characteristics A total of 230 surveys were distributed, and 228 were retrieved, a 99.1% response rate. Twenty surveys were excluded because of incomplete data or issues with
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
eligibility criteria; 208 nurses were included in final data set. As shown on Table 4, majority of the participants were female (97.6%) with mean age of 30.64 (SD, 6.09) years. Nurses’ work sites in Korea are commonly grouped as medical-surgical and other areas in contrast to inpatient and outpatient seen in other countries. Less than half of the participants (40.4%) worked in a medical-surgical unit setting, whereas the others worked in specialty units or outpatient departments. Mean number of years of clinical experience was 7.78 (SD, 5.36) years, and 13.9% of the participants were in a position involving managerial tasks such as nurse manager.
Psychometrics The Cronbach’s ! coefficient was reinforced with high internal consistency (total instrument = .981, computer skills = .957, information knowledge = .965, and information skills = .944). Construct validity was examined more formally by using principal component analysis. Factors with eigenvalue of 1 or greater were extracted as valid based on the Kaiser criterion. Initially, a scree plot was used to determine the
number of factors to be retained. Significant findings from Bartlett test of sphericity (# 26216 = 20138.642, P G .001) supported the factorability of the correlation matrix, and the high value of the Kaiser-Meyer-Olkin test (0.90) showed adequate sampling.48 Initially, 21 factors explained 72.81% of the total variance in the scale. Based on the Scree plot (Figure 1), the number of domains in the original list of competencies (computer skills, informatics knowledge, informatics skills), and the comprehensibility of the results, three factors were extracted from 112 items. In general, items representing informatics skills loaded onto factor 1 (factor loadings = 0.476–0.732) explaining 34.4% of the variance, items on informatics knowledge loaded onto factor 2 (factor loadings = 0.019–0.700) explaining 6.7%, and the items on computer skills loaded onto factor 3 (factor loadings= 0.102–0.674) explaining 3.8% of variance in total ICs, adding up to 44.9%. Factor 1 (informatics skills): all items about informatics skills loaded onto factor 1 Factor 2 (informatics knowledge): all items about informatics knowledge at beginning nurse level loaded onto factor 2, except for six items (items 32, 33,
T a b l e 4 Demographics, Clinical Factors, and ICs in Practicing Nurses (n = 208) Characteristics Gender Education
Previous informatics education Work site Staff position Age, y
Clinical work experience, y
Use of computers at work NICQ Computer skills Informatics knowledge Informatics skills
Categories
n
%
Male Female Associate degree Bachelor’s degree Master’s degree and above Yes No Medical-surgical unit Others Nonmanaging Manager and above Total G30 30–40 Q40 Total 0.5 e to G 5 5 e to G10 Q10 Total Sum Item Sum Item Sum Item Sum Item
5 203 94 98 16 140 68 84 124 179 29
2.4 97.6 45.2 47.1 7.7 67.3 32.7 40.4 59.6 86.1 13.9
110 76 22
52.9 36.5 10.6
73 75 60
35.1 36.1 28.8
Range
Mean (SD)
22.00–51.00
30.64 (6.09)
0.67–25.08
7.78 (5.36)
0.50–13.00 200–509 1.79–4.54 77–224 1.60–4.67 97–218 2.06–4.64 20–82 1.18–4.82
5.65 369.44 3.30 157.88 3.29 158.73 3.38 52.84 3.11
(3.18) (56.37) (0.50) (26.77) (0.56) (23.47) (0.50) (10.10) (0.59)
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
601
FIGURE 1. Scree plot of NICQ
34, 35, 37, and 52), which loaded onto computer skills, and three items (items 44, 51, and 53), which loaded onto informatics skills. As for items about informatics knowledge at experienced nurse level, only three items loaded onto the informatics knowledge (items 73, 74, and 75 on data). All the other items loaded to informatics skills instead of informatics knowledge. Factor 3 (computer skills): all items on computer skills at beginning nurse level loaded to factor 3 except for seven items, which loaded onto informatics knowledge. As for items on computer skills at experienced nurse level, only two items loaded onto computer skills (items 65 and 67). All the other items loaded to informatics skills instead of computer skills, except for one item (item 60), which loaded onto informatics knowledge.
Overall, this analysis supports adequate construct validity for the NICQ accounting for different levels of competencies in each domain, although some specific items were rearranged. The specific results are available from the first author.
Nursing Informatics Competencies and Relationship Between Participant Characteristics The beginning and experienced levels of nursing ICs in this sample were just above midrange for the mean total scale score (mean NIC, 3.30 [SD, 0.503]). The informatics knowledge subscale was the highest (mean, 3.38 [SD, 0.499]) followed by computer skills (mean, 3.29 [SD, 0.558]) and informatics skills (mean, 3.11 [SD, 0.594]). In the regression analysis, age was excluded from further consideration because of its suspected multicolinearity (r = 0.949, P G .001) with clinical work experience. For the total NICQ instrument, significant associations were 602
shown with staff position (beta = j.304, P G .001), previous informatics-related education (beta = .187, P = .005), and hours of computer use at work (beta = .144, P = .045), while controlling for all other variables in the model (Table 5). Hours of computer use at work increased mean scores on the NICQ, although minimal (beta = .023; 95% confidence interval [CI], G.001-to .045). Nurses not involved in a managerial role had lower scores on the NICQ and so had lower scores compared with nurses involved in a managerial role (beta = j.441; 95% CI, j.680 to j.202). Nurses with previous self-directed informatics-related education had higher scores for nursing ICs (beta = .200; 95% CI, .062–.338). Results were similar for the computer skills subscale. Significant associations were shown with staff position (beta = j.305, P G .001), previous informatics-related education (beta = .162, P = .014), and hours of computer use at work (beta = .154, P = .031). Likewise, results were similar for the informatics knowledge and informatics skills subscales, except that the hours of computer use at work was not statistically significant for both informatics knowledge (beta =.123, P = .090) and informatics skills (beta =.109, P = .146).
DISCUSSION The purposes of this research were to develop an instrument to measure a full set of ICs at the beginning and experienced levels among practicing nurses, to examine the competencies at a large hospital, and to explore the possible influence of selected clinical factors on the level of informatics competency. The measure has adequate reliability as assessed by a high internal consistency (! = .98) and no multicolinearity indicated (no bivariate correlations among items, r 9 0.90). Initial instrument validity was supported with face, content, and construct validity. Three factors matching three domains of informatics competency explained 44.9% of variance in total ICs. Originally, items about informatics skills were only available as ICs for the experienced nurse level. Interestingly, most of the items on computer skills and informatics knowledge at the experienced nurse level loaded more onto informatics skills than within their own domain. This may be because informatics skills are indeed experienced level ICs, and perhaps informatics knowledge and computer skills at experienced level are more representative of informatics skills. This sample of practicing nurses had an above-average level of informatics competency at beginning and experienced levels on the NICQ. Findings also show that staff position (involvement in managerial position), previous informatics-related education, and use of computers at work (more hours) significantly account for the variance
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
" is unstandardized b; beta, standardized b. Predictors in the model: years of clinical work experience, work position, work site, use of computers at work, and any previous informatics-related education. NICQ model (F = 6.685, P e .01; R 2 = 0.142 [adjusted R 2 = 0.377]); computer skill model (F = 7.247, P e .01; R 2 = 0.152 [adjusted R 2 = 0.131]); informatics knowledge model (F = 5.997, P e .01; R 2 = 0.129 [adjusted R 2 = 0.108]); informatics skill model (F = 2.873, P = .016; R 2 = 0.066 [adjusted R 2 = 0.043]). a P G .05.
Nonmanager vs manager j.441 0.121 j.304 j3.636 Previous informatics-related .200 0.070 .187 2.862 education vs no education Computer use at work, h/d .023 0.011 .144 2.013 Clinical work experience, y .000 0.008 j.004 j0.042 Medical-surgical unit vs others j.028 0.074 j.028 j0.379
.045a .027 .012 .154 2.175 .031a .019 0.011 .123 1.702 .090 .020 0.014 .109 1.460 .146 .967 .004 .009 .035 0.415 .679 j.001 0.008 j.014 j0.161 .872 j.009 0.010 j.081 j0.911 .364 .705 j.015 .082 j.013 j0.183 .855 j.028 0.074 j.027 j0.376 .708 j.066 0.091 j.055 j0.726 .469
G.001 j.491 .134 j.305 j3.674 G.001 j.415 0.121 j.288 j3.424 .001 j.372 0.149 j.218 j2.495 .013a .005a .192 .077 .162 2.486 .014a .213 0.070 .200 3.037 .003a .191 0.086 .151 2.218 .028a
P t beta SE "
SE
beta
t
P
a
"
SE beta
t
P
a
"
SE
beta
t
P
a
"
Informatics Skill Informatics Knowledge Computer Skill NICQ
Factors Predicting NIC, Computer Skills, Informatics Knowledge, and Informatics Skills Controlling for Years of Clinical Work Experience, Work Position, Work Site, Use of Computers at Work, and Any Previous Informatics-Related Education (n = 208)
T a b l e 5
in nursing ICs of nurses, whereas general clinical experience and workplace do not. Involvement in managerial position and informatics-related education significantly accounted for variance on all subscales (computer skills, informatics knowledge, informatics skills), whereas use of computers at work accounted solely for computer skills. Because few studies are available about nursing ICs in practicing nurses, our ability to compare findings across studies is somewhat limited. The level of nursing informatics competency in this facility may be higher than other settings because the facility is an urban, teaching hospital with an EHR installed base for over 10 years. The finding about self-directed informatics education being related to higher nursing ICs is congruent with a former study in Korean nurses.41 Although this is not a surprising finding, it should be noted that no information nurse specialists were included in this sample, so the higher level is more impressive among this group of nurses who had little informatics training during their initial nursing preparation. On the other hand, Hwang and Park41 reported higher nursing ICs in staff nurses compared with nurse managers— the opposite of the findings here. Administrators likely use computers more in depth and breadth because of their tasks and roles. Because the demographic characteristics were similar between the two study samples, further study is warranted to reveal the true relationship between staff position or their related tasks and ICs in nurses. Perhaps the difference in the tasks completed for each position should be identified to illuminate the true influencing factors on informatics competency. The factor concerning use of computers at work was not measured in other studies. Although this factor accounted for significant variance in the total score for nursing ICs of this sample, only the computer skill subscale was a significant predictor among the three subscales. Computer use is a complex concept related to both the depth and breadth of computer use as well as technology knowledge and skills.49 Why computer use is not related to informatics knowledge and skills needs to be explored in the future. Perhaps the informatics education of practicing nurses is similar among most nurses currently, resulting in the small variation seen in this sample. At any rate, further studies are needed to differentiate the influence of computer use in informatics knowledge and informatics skills and their relation to nursing ICs. Nurses are the largest part of the healthcare workforce and are the major users of the clinical information system. Nurses’ informatics competency is considered crucial for the success of clinical information systems use and for patient safety in computerized environments. The roles of the information nurses and information nurse specialists will evolve in conjunction with new technologies and changing healthcare delivery models. Continued work on ICs will be required to match the evolving nature of nursing informatics, and planned, formal education is recommended across nurses’ work lives.
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
603
Knowing the extent of ICs among practicing nurses could provide the basis for developing job-specific standardized ICs in practicing nurses along with the appropriate evaluation tools. Moreover, knowing ICs for practicing nurses could form the foundation for understanding how ICs might be integrated throughout nurses’ work lives and how to develop appropriate strategies to support nurses in their informatics practice in clinical settings. Targeted training could then be provided to facilitate nurses to obtain the necessary skills in their positions.42 Nursing educational programs may vary in what they teach about nursing ICs without central guidance. Job-specific standardized ICs in practicing nurses could be helpful in meeting informatics needs and an organized/central effort for NI competencies in the future.
Study Limitations and Future Research Several limitations to this study include that the sample was fairly homogeneous. Selecting a homogeneous sample was purposeful to increase the internal validity of the study but could have limited the extent to which the results can be generalized. The sample also had an above-average NICQ score, potentially limiting generalizability. Other settings, units, and hospitals could be used for future studies. Although the internal consistency, face, content, and construct validity were ensured for the NICQ in this study, stability and additional psychometrics could be assessed in the future. For future studies, experienced level competencies related to computer skills and informatics knowledge could be reconsidered as informatics skills, and the informatics skills domain itself could be thought of as an ‘‘experienced level and above’’ competency. The idea of recategorizing experienced level computer skills and informatics knowledge competences into informatics skills is supported by the fact that informatics skills accounted for the largest portion of the variance (34.4%) in the ICs present in NICQ, compared with informatics knowledge (6.7%) or computer skills (3.8%). Following the recategorization, the number of items in NICQ could be reduced considering the high internal consistency value. Last, the instrument could be refined to be more parsimonious in the future.
Acknowledgments The authors thank Dr Shijun Zhu at University of Maryland Baltimore, School of Nursing, for his insightful review of the statistical portion of this article, and Dr Elizabeth Borycki at University of Victoria, School of Health Information Science, for her thoughtful review of a previous version of this article. Guidance throughout the study from Drs Kyung Rim Shin, Youn Hee Kang, and Ok Su Kim at Ewha Womans University, Division of Nursing Science, as well as Dr Sun Mi Lee at The Catholic University of Korea, College of Nursing, is greatly appreciated. 604
REFERENCES 1. Institute of Medicine (IOM). Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001. 2. Kelley TF, Brandon DH, Docherty SL. Electronic nursing documentation as a strategy to improve quality of patient care. J Nurs Scholarsh. 2011;43(2):154–162. 3. Blumenthal D, Tavenner M. The ‘‘Meaningful Use’’ regulation for electronic health records. N Engl J Med. 2010;363(6):501–504. 4. Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff. 2011;30(3):464–471. 5. Chae YM, Yoo KB, Kim ES, Chae H. The adoption of electronic medical records and decision support systems in Korea. Healthc Inform Res. 2011;17(3):172–177. 6. Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc. 2011;18(3):327–334. 7. Bloomrosen M, Detmer DE. Informatics, evidence-based care, and research; implications for national policy: a report of an American Medical Informatics Association health policy conference. J Am Med Inform Assoc. 2010;17(2):115–123. 8. Banet GA, Jeffe DB, Williams JA, Asaro PV. Effects of implementing computerized practitioner order entry and nursing documentation on nursing workflow in an emergency department. J Healthc Inform Manage. 2006;20(2):45. 9. Hoo WES, Parisi LL. Nursing informatics approach to analyzing staffing effectiveness indicators. J Nurs Care Qual. 2005;20(3):215–219. 10. Parente ST, McCullough JS. Health information technology and patient safety: evidence from panel data. Health Aff. 2009;28(2):357–360. 11. Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. Int J Medical Inform. 2009;78: S69–S76. 12. Ruder DB. Malpractice claims analysis confirms risks in EHRs. Patient Saf Qual Healthc. 2014;11(1):20–23. 13. American Nurses Association. Nursing Informatics: Scope and Standards of Practice. Silver Spring, MD: American Nurses Association; 2008. 14. Nagle L. Making Informatics Competency Development Explicit. Online J Nurs Inform. 2013;17(1). 15. Kaslow NJ, Borden KA, Collins FL, et al. Competencies conference: future directions in education and credentialing in professional psychology. J Clin Psychol. 2004;60(7):699–712. 16. Stratford R. A competency approach to educational psychology practice: the implications for quality. Educ Child Psychology. 1994. 17. Parry SB. The quest for competencies. Training. 1996;33(7):48–54. 18. Staggers N, Thompson CB. The evolution of definitions for nursing informatics: a critical analysis and revised definition. J Am Med Inform Assoc. 2002;9(3):255–261. 19. Graves JR, Corcoran S. The study of nursing informatics. J Nurs Scholarsh. 1989;21(4):227–231. 20. Chang J, Poynton MR, Gassert CA, Staggers N. Nursing informatics competencies required of nurses in Taiwan. Int J Med Inform. 2011;80(5):332–340. 21. Foster J, Bryce J. Australian nursing informatics competency project. In: Proceedings of the 10th International Congress on Nursing Informatics: Connecting Health and Humans. June 8–July 1, 2009. Helsinki. QUT Digital Repository. http://eprints.qut.edu.au/31927/1/c31927a.pdf. 22. Grobe SJ. Nursing informatics competencies. Methods Inform Med. 1989;28(4):267–269. 23. Staggers N, Gassert CA, Curran C. A Delphi study to determine informatics competencies for nurses at four levels of practice. Nurs Res. 2002;51(6):383–390. 24. TIGER Initiative. Collaborating to integrate evidence and informatics into nursing practice and education: an executive summary. 2009. 25. Curran CR. Informatics competencies for nurse practitioners. AACN Adv Crit Care. 2003;14(3):320–330. 26. Hart MD. A Delphi study to determine baseline informatics competencies for nurse managers. Comput Inform Nurs. 2010;28(6): 364–370.
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
27. Westra BL, Delaney CW. Informatics competencies for nursing and healthcare leaders. AMIA Annual Symposium Proceedings. 2008:804–808. 28. Hebert M. A national education strategy to develop nursing informatics competencies. Can J Nurs Leadersh. 2000;13(2):11. 29. Hunter KM, McGonigle DM, Hebda TL. TIGER-based measurement of nursing informatics competencies: the development and implementation of an online tool for self-assessment. J Nurs Educ Pract. 2013;3(12):70–80. 30. Borycki EM, Foster J, Sahama T, Frisch N, Kushniruk AW. Developing national level informatics competencies for undergraduate nurses: methodological approaches from Australia and Canada. Stud Health Technol Inform. 2013;183:345–349. 31. Choi J. Comparative assessment of informatics competencies in three undergraduate programs. Online J Nurs Inform. 2012;16(2). 32. Choi J, de Martinis JE. Nursing informatics competencies: assessment of undergraduate and graduate nursing students. J Clin Nurs. 2013;22(13–14):1970–1976. 33. Fetter MS. Graduating nurses’ self-evaluation of information technology competencies. J Nurs Educ. 2009;48(2):86. 34. Hunter K, McGonigle D, Hebda T. The integration of informatics content in baccalaureate and graduate nursing education: a status report. Nurse Educ. 2013;38(3):110–113. 35. Kleib M, Zimka O, Olson K. Status of informatics integration in baccalaureate nursing education: a systematic review. Can J Nurs Res. 2013;45(1):138–154. 36. McBride SG, Tietze M, Fenton MV. Developing an applied informatics course for a doctor of nursing practice program. Nurse Educ. 2013;38(1):37–42. 37. Schleyer RH, Burch CK, Schoessler MT. Defining and integrating informatics competencies into a hospital nursing department. Comput Inform Nurs. 2011;29(3):167–173.
38. Campbell CJ, McDowell DE. Computer literacy of nurses in a community hospital: where are we today? J Contin Educ Nurs. 2011; 42(8):365. 39. Gilmour JA, Scott SD, Huntington N. Nurses and Internet health information: a questionnaire survey. J Adv Nurs. 2008;61(1):19–28. 40. Huang H, Lee TT. Evaluation of ICU nurses’ use of the clinical information system in Taiwan. Comput Inform Nurs. 2011;29(4): 221–229. 41. Hwang JI, Park HA. Factors associated with nurses’ informatics competency. Comput Inform Nurs. 2011;29(4):256–262. 42. Hart MD. Informatics competency and development within the US nursing population workforce: a systematic literature review. Comput Inform Nurs. 2008;26(6):320–329. 43. Staggers N, Gassert CA, Curran C. Informatics competencies for nurses at four levels of practice. J Nurs Educ. 2001;40(7):303–316. 44. Nunnally J, Bernstein I. Psychometric Theory. New York, NY: McGrawHill; 1994. 45. HIMSS Analytics. The EMR Adoption Model (EMRAM). 2014. Available at http://www.himssanalytics.org/emram/emram.aspx. Accessed November 1, 2014. 46. Park H. Direction of nursing informatics education in Korea: suggestion for nursing informatics education model. (Korean) Publications from The Research Institute of Nursing Science, Seoul National University. 2001;2001(1):43–64. 47. Tabachnick BG, Fidell LS, Osterlind SJ. Using multivariate statistics, Profile analysis: the multivariate approach to repeated measures. New York, NY: Pearson; 2001. 48. Stevens J. Applied Multivariate Statistics for the Social Sciences. New York, NY: Routledge; 2009. 49. Staggers N. The Staggers nursing computer experience questionnaire. Applied Nurs Res. 1994;7(2):97–106.
CIN: Computers, Informatics, Nursing & December 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
605
