Int J Clin Exp Med 2014;7(10):3543-3549 www.ijcem.com /ISSN:1940-5901/IJCEM0001883
Original Article Reliability and validity of the Chinese version of reporting of clinical adverse events scale (C-RoCAES) Xiao Sun1, Yan Shi1,2, Shuying Zhang3, Meimei Tian1, Yafen Mao1, Qian Wu1, Xiaoping Zhu1, Meifang Gong1 Department of Nursing, Tenth People’s Hospital, Tongji University, Shanghai, China; 2Shanghai Nursing Quality Control Centre, Shanghai, China; 3Tongji University School of Medicine, Shanghai, China 1
Received August 13, 2014; Accepted September 20, 2014; Epub October 15, 2014; Published October 30, 2014 Abstract: Background: Adverse event is a crucial issue affecting patient’s safety of healthcare services. To assess nurses’ attitude of reporting adverse events is important to establish a safe environment for patients. However, no relevant instrument has been validated and used in China. This study was to examine validity and reliability of Chinese version of Reporting of Clinical adverse Event Scale (C-RoCAES). Material and methods: Chinese version of 25-item RoCAES was used in a sample of 1557 nurses. Confirmatory factor analysis (CFA) and exploratory factor analysis (EFA) were selected for construct validity test. Internal consistency was also examined. Results: After CFA and EFA, two items were removed and two items loaded on different factors in our sample. Five factors were generated, including perceived blame, perceived criteria for identifying events that should be reported, perceptions of colleagues’ expectations, perceived benefits of reporting and perceived clarity of reporting procedures. Cronbach’s alpha for the total scale and subscales ranged from 0.70 to 0.85. Conclusion: C-RoCAES is applicable to healthcare services of China. The instrument provide information for the providers of healthcare services to develop staff education regarding patient safety, and also help them to evaluate strategies of preventing adverse events in clinical practice in China. Keywords: Reporting of clinical adverse events scale (RoCAES), confirmatory factor analysis, exploratory factor analysis
Introduction Adverse events (AEs) have become a significant issue influencing hospitalized patients safety . An AE is usually defined “a process or act of omission or commission that resulted in hazardous health care conditions and/or unintended harm to the patient” . AEs caused various outcomes for patients [3-6] and organizations [2, 7, 8]. For patients outcomes, AEs contributed to the increasing harm from no or minor harm [3, 4]; to temporary or permanent disability [3, 5, 6], and even death [3, 5, 6]. For organization outcomes, AEs were associated with high direct medical costs due to prolonged stay of hospitalization or readmission [7, 8]; adverse media outcome, or legal action . Comprehensive strategies have been recommended to prevent AEs, including quality assurance or peer review, staff education, evaluation of safety behavior, and procedures improvement ; voluntary and computerized reporting systems reporting for improving health profes-
sionals’ reporting attitude and behavior [6, 10]; Of the effective strategies, improving reporting status was an essential strategy to detect problems related with patients safety and learning from error . However, The researchers have identified the following main barriers to AEs reporting [12-23]: fear of blame, sanctions or dishonor, threat peer relationships, lawsuits [12-20]; lacking of AEs reporting knowledge [13, 14, 16, 18, 21, 22]; lacking of support or expectation from colleagues [12, 23]; and reporting system issues such as tedious processes, lacking of confidentiality and anonymity for reporting, lacking of efficient and timely feedback [15, 16, 18, 22]. In 2008, WHO launched an International Reporting and Learning Systems (RLS) with the purpose of sharing learning, innovations, solutions and best practices . The Institute of Medicine (IOM) issued a report “Health IT and Patient Safety: Building Safer Systems for Better Care ” and emphasized that “reporting of
Reliability and validity of C-RoCAES safety problems should be voluntary, share lessons learned, confidential, and non-punitive” . “The patient safety target” drew up by China Hospital Association (CHA) also advocated to set up a voluntary, non-punitive reporting system and to take measures to encourage active reporting AEs. Nurses play a major role in reporting AEs because they work at the first line of patient care . According the data, nursing safety events accounted for 40% of AEs in China. It is necessary to understand the reporting attitude of nurses for improving the reporting status of AEs in China. There were some studies to investigate influence factors of AEs under-reporting for nurses in China [16, 17, 26, 27]. Most of the studies focused one or two aspects of reporting attitude of nurses [16, 17, 26], such as their perceptions of blame culture [16, 17, 26], or information cognition , or reporting system . There is a paucity of research on their perception of expectations from administrators and co-workers to reporting attitude. Moreover, to improve the reporting statue, an applicable measurement can provide valuable insight into nurses’ attitudes of reporting AEs . However, there has been no validated Chinese-version instrument to examine main aspects of nurses’ AEs reporting attitudes. From the literature, we found some different version instruments (Medication Administration Error Reporting scale; Nursing Staff/Pharmacist/Medical Staff Questionnaire Regarding Error reporting; Hospital Survey on Patient Safety Culture; Incident Reporting Culture Questionnaires) for measuring nurses’ reporting status. The English-version RoCAES was more specific and comprehensive than others, so we chose it to examine nurses’ attitude and behavior for reporting of clinical adverse events in China. The English version of Reporting of Clinical Adverse Events Scale (RoCAES)  developed by Wilson et al. assessed the five main aspects of attitudes of reporting AEs: perceived blame (six items), perceived criteria for identifying events that should be reported (six items), perceptions of colleagues’ expectations (six items), perceived benefits of reporting (five items) and perceived clarity of reporting procedures (two items). The scale had been compared the attitude and behavior of nurses/ nurse-midwives with that of doctors at Leeds, Hull and York Universities in August September 2003. The scale had better reliability and validity. The Cronbach’s alphas for the five subscales 3544
and total scale ranged from 0.66 to 0.84. The 6-week test-retest reliability coefficients for five subscales ranged from 0.48 to 0.62. The main purpose of the study was to examine the reliability and validation of the Chinese version of RoCAES (C-RoCAES). Material and methods Design and sampling This study was a cross-sectional survey. The participants were nurses recruited from four general hospitals of Shanghai, including two tertiary (Shanghai Tenth People’s Hospital, Shanghai Yangpu District Central Hospital) and two secondary (Shanghai Kongjiang Hospital, Shanghai Antu Hospital) hospitals. The inclusion criterion was: the nurses providing direct care for patients independently. The exclusion criteria were: directors of nursing department of the hospitals, and nurses having training courses. Procedure Data collection was conducted between 7th, May to 31th, December 2012. Permission to use the scale  was obtained from the original author before the study. Translation and back-translation were conducted by a translator with bilingual education backgrounds. Content validity test was conducted by an expert committee involving six experienced clinical nursing professionals. Item-Content Validity Index (I-CVI) was computed as the number of experts giving a rating of either 3 or 4, divided by the total number of experts. In our study, I-CVI of the scale ranged from 0.83 to 1.00, indicating adequate content validity . Prior to the investigation, permissions to conduct the study were approved from the hospitals. Then the principal investigator (the first author) contacted the nursing departments of the recruited hospitals. The numbers of potential participants at each wards or nursing unit were obtained from the nursing departments. A total of 1557 survey packages with return stamped envelopes were mailed to the potential participants. The survey package included: a) an introduction describing the purposes and procedure of the study and emphasizing voluntary, anonymous and whether the nurse participated into this survey or not would not affect Int J Clin Exp Med 2014;7(10):3543-3549
Reliability and validity of C-RoCAES their work at the hospital; b) a written consent form; and c) questionnaires including demographic information and the initial version of C-RoCAES. Of 1557 nurses, 1257 completed the survey with a response rate of 80.7%. Measures Demographic information includes age, gender, education level, year of working at ward and hospital, and category of wards. The nurses’ attitudes of reporting adverse events were measured by the initial version of Chinese version of Reporting of Adverse Events Scale (C-RoCAES). The C-RoCAES was translated and back-translated from the English version (RoCAES) developed by Wilson et al. . The RoCAES consisted of 25 items assessing five aspects of attitudes of reporting adverse events: perceived blame (six items), perceived criteria for identifying events that should be reported (six items), perceptions of colleagues’ expectations (six items), perceived benefits of reporting (five items) and perceived clarity of reporting procedures (two items). Each item rates the attitude with a 4-point Likert points from “1” to “4” (from “strongly agree”, “agree”, “disagree” to “strongly disagree”). Items include both positive and negative statements, and the negative statements were reverse-scored. The higher score of each item indicates more negative attitude to reporting adverse events. The RoCAES also comprises two questions regarding an individual’s response to witnessing or involving an adverse event (“Have you ever witnessed, or been involved in, an adverse event? If yes, have you ever reported an adverse event?”) and the intension of reporting future adverse events (“How likely are you to report an adverse event in the future?”). Construct validity was established by exploratory factor analysis. The Cronbach’s alphas for the five subscales and total scale ranged from 0.66 to 0.84. The 6-week test-retest reliability coefficients for five subscales ranged from 0.48 to 0.62. Data analysis Preliminary data analysis was conducted. Confirmatory factor analysis (CFA) was used to test whether the hypothesized factor structure provided a good fit for the research data. The criteria of model fit used for CFA were exam-
ined, including: a) the goodness of fit index (GFI) ≥ 0.90 and adjusted goodness of fit index (AGFI) ≥ 0.90; b) the root-mean-square error of approximation (RMSEA) with a 90% confidence interval with a cut-off value of 0.06 or less; c) insignificance of chi-square test (χ2); d) comparative fit index (CFI) of ≥ 0.90; and e) normed-fit index (NFI) of ≥ 0.90 . However, the factor structure of the C-RoCAES may be influenced by the culture differences and disparities of hospital policies between two countries. If the factor structure is not confirmed by CFA, exploratory factor analysis (EFA) can be used to improve and modify the model . Then principal component analysis with varimax orthogonal rotation was conducted for finding a satisfactory factor structure. The reliability was assessed by internal consistency (Cronbach’s alpha). Data were analyzed using SPSS version 19.0 and Amos version 17.0. Results The mean age of nurses was 28.4 years old (SD=7.03), ranging from 19 years old to 55 years old. 97.6% of the nurses were females. Nearly 50.6% (788) of nurses had junior college level, 36.6% (570) had tertiary or higher education level, and the remaining (12.8%, 199) were graduated from technical secondary school. The average years of working was 7.6 years (SD=7.52). 43.1% of them had been working for six years or longer, and 28.2% for two years or less. Most of nurses were working at general medical (31.5%, 396) and general surgical wards (27.3%, 343) and intensive care units (23.5%, 296). The remaining (17.7%, 222) were working at special wards including obstetrics, gynecology and pediatric wards. Confirmatory factor analysis Prior to CFA, normality distribution of each item of C-RoCAES was examined, and no item had a skewness value greater than the cut-offs of |3| . Therefore, all items were retained for CFA. Then offending estimates of this model were examined, including negative error variance, excessive standardized coefficients (≥ 1) and large standardize errors . None of them were presented. Based on the factor structure of English version RoCAES, CFA was conducted. The results showed poor fit of the original model, indicating EFA to further modifying the factor structure (Table 1).
Int J Clin Exp Med 2014;7(10):3543-3549
Reliability and validity of C-RoCAES Table 1. The fit indices of C-RoCAES factor model
and original factors, the two items were retained.
Fit indices χ2 (265) GFI AGFI CFI NFI
The final principal component analysis with varimax orthogonal rotation resulted in a solution involving 23 items loading on the five factors. The pattern matrix of factors, descriptions of the item content and corresponding factor loadings were listed in Table 2. The cumulative proportions for the first five factors were increased to 61.77% of the 23-item version from 53.61% of the initial version.
Criteria value ≥ 0.09 ≥ 0.09 ≥ 0.09 ≥ 0.09
Results 4311.78 0.77 0.72 0.71 0.70
Exploratory factor analysis Prior to conducting EFA, Kaiser-Meyer-Olkin (KMO) index, Barlett’s test of sphericity and correlation matrix of items was tested in order to make the sample size and intercorrelation strength suitable for factor analysis. The KMO was 0.82 which was greater than 0.60, and Barlett’s test of sphericity was significant (P < 0.000). Most of the coefficients in the correction matrix were greater than 0.30, with an acceptable value for factor analysis . A principal component analysis without rotation solution was conducted. The eigenvalues for six factors were greater than 1.0. But scree plot had flat-lined beginning the fifth factor, and the total cumulative proportion accounted for 53.61%. Therefore, five factors were retained for the next factor analysis. A second principal component analysis with varimax orthogonal rotation was run on the first five factors. The criteria used to remove the items included: if the item loading was less than ±0.04, or if items were loaded simultaneously on two factors . On the basis of the criteria, item 2 (Whether or not to report an adverse event depends on how many people are aware the error has taken place) was deleted as there was no loading on any of the five factors; item 21 (Reporting adverse events is a method through which to pinpoint blame) was removed as the item was loaded simultaneously on two factors. Item 6 (As long as those around me learn from adverse events there is no need to report them) and 16 (Receiving encouragement from senior clinical staff encourages me to report adverse events) were not loaded on the original factors (Item 6: “perception of colleagues’ expectation”; Item 16: “perception benefits of reporting”). The two items were loaded on “perception criteria for identifying events that should be reported” and “perception of colleagues’ expectation”, respectively. After comparing the content between newly 3546
Reliability of C-RoCAES Cronbach’s alpha for the C-RoCAES was 0.85, and for each subscales were all over 0.70 (ranging from 0.70 to 0.85). Discussion Reliability and validity are essential qualities of a good instrument. The aim of the present study was to examine the reliability and validity of the Chinese version of RoCAES. The results of CFA failed to confirm the original factor structure for our sample data. One interpretation for the result could be that the RoCAES contained some items that could not identify the reporting attitude in either culture. Then the results of EFA identified five main domains which were the same as the original scale. However, two items were excluded in the C-RoCAES according to the result of factor analysis, and factor loading of the two items were a little low in English-version RoCAES. Two items (item 6: As long as those around me learn from adverse events there is no need to report them; item 16: receiving encouragement from senior clinical staff encourages me to report adverse events) loaded on different factors in our sample. From the meaning of items, we found they were related to the “which incidents should be reported” (the meaning of perception criteria for identifying events that should be reported) and “how colleagues view reporting” (the meaning of perception of colleagues’ expectation) respectively. So they can be included in the factor of C-RoCAES. The result revealed that the C-RoCAES accounted for 61.77% of the variance which were better than the original version (41%), and the reliability of the subscale scores was higher than the original version scale. Therefore, the results of our study showed that the C-RoCAES was a Int J Clin Exp Med 2014;7(10):3543-3549
Reliability and validity of C-RoCAES Table 2. Factor loadings for each of the 23 attitude items in C-RoCAES C-RoCAES items Factor 1. Perceived blame 5. Reporting adverse events lets others check up on me 12. Reporting adverse events creates problems for me 18. Reporting adverse events lets everyone know I have made a mistake 7. The careers of staff who report adverse events suffer 23. Reporting adverse events lets colleagues gossip about my involvement in the event Factor 2: Perceived criteria for identifying events that should be reported 14. Only uncommon adverse events should be reported 20. You should only report those adverse events where something can be learnt from them 10. Minor adverse events should not be reported 15. Writing in a patient’s notes that an adverse event has happened is just as good as filing in a separate reporting form 3. It is not my responsibility to report adverse events involving colleagues 6. As long as those around me learn from adverse events there is no need to report them Factor 3: Perceptions of colleagues’ expectations 19. I am not permitted to report adverse events 11. My colleagues expect me to report adverse events 22. Adverse events can’t be prevented so there is no point in reporting them 25. Colleagues seem unconcerned when adverse events occur 9. I am not doing my job properly unless I report adverse events 16. Receiving encouragement from senior clinical staff encourages me to report adverse events Factor 4: Perceived benefits of reporting 4. Reporting adverse events protects patients 17. Having an external adverse event monitoring unit based in the hospital encourage staff to report error Reporting adverse events helps identify staff who need additional training 24. Reporting adverse events makes people accountable for their actions Factor 5: Perceived clarity of reporting procedures 8. The procedures in this hospital are clear on how to report adverse events 13. The procedures in this hospital are clear on what sort of adverse events should be reported
better applicable instrument in assessing the main attitudes of nurses’ reporting AEs in China. The strength of the C-RoCAES was the multidimensional assessment of the attitude and behavior of Chinese nurses toward AEs report3547
Loading 0.887 0.636 0.847 0.676 0.621 0.790 0.802 0.644 0.534 0.467 0.591 0.762 0.548 0.481 0.426 0.777 0.748 0.573 0.545 0.573 0.593
ing, which included five subscales (perceived blame, perceived criteria for identifying events that should be reported, perceptions of colleagues’ expectations, perceived benefits of reporting and perceived clarity of reporting procedures). These dimensions were similar with the results of previous literature review and Int J Clin Exp Med 2014;7(10):3543-3549
Reliability and validity of C-RoCAES focus group interview carried out before selecting the research scale. One of the limitations in this study was lack of external constructs from which the C-RoCAES could be cross-validated across different groups. Further work is needed to validate the C-RoCAES with other well-developed measures with proven cross-cultural validity and reliability. As previous research had found that the attitude of AEs reporting were associated with demographic characteristics such as educational background, tenure of present work and professional title [12, 14], the future work will examine the impact of demographic characteristics on the attitude and behavior of AEs reporting. The results from C-RoCAES applications can recognize the reporting barriers, which will be used to focus interventions to improve reporting culture and to change reporting attitude and behavior; the results can also evaluate the success of interventions . Conclusions The finding of this study provided that the final C-RoCAES has a better validity and reliability according the scientific factor analysis. It will help the hospital management to take interventions for changing adverse events reporting environment, improving reporting behavior, and evaluating the effectiveness of interventions. In addition, the C-RoCAES need to be further tested in different nurse populations from multicenter hospitals for a better validity and reliability.
References   
The study was supported by the Health Bureau, Shanghai, China (grant no: 2010254). The authors thank Dr. B Wilson, School of Nursing, Midwifery and Social Work, the University of Manchester, and H L Bekker, Institute of Health Sciences and Public Health Research, School of Medicine, University of Leeds.
Disclosure of conflict of interest None. Address correspondence to: Yan Shi or Shuying Zhang, No. 301 Yanchang Middle Road, Zhabei District, Shanghai 200072, China. E-mail: sy1880178@ 163.com (YS); [email protected]
; [email protected]
Adhikari NK. Patient safety without borders: measuring the global burden of adverse events. BMJ Qual Saf 2013; 22: 798-801. Organization TWH; The Conceptual Framework of an International Patient Safety Event Classification Executive Summary. 2006. Baines RJ, Langelaan M, de Bruijne MC, Asscheman H, Spreeuwenberg P, van de Steeg L, Siemerink KM, van Rosse F, Broekens M and Wagner C. Changes in adverse event rates in hospitals over time: a longitudinal retrospective patient record review study. BMJ Qual Saf 2013; 22: 290-298. de Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ and Boermeester MA. The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 2008; 17: 216-223. Soop M, Fryksmark U, Koster M and Haglund B. The incidence of adverse events in Swedish hospitals: a retrospective medical record review study. Int J Qual Health Care 2009; 21: 285-291. Forster AJ, Worthington JR, Hawken S, Bourke M, Rubens F, Shojania K and van Walraven C. Using prospective clinical surveillance to identify adverse events in hospital. BMJ Qual Saf 2011; 20: 756-763. Hoonhout LH, de Bruijne MC, Wagner C, Zegers M, Waaijman R, Spreeuwenberg P, Asscheman H, van der Wal G and van Tulder MW. Direct medical costs of adverse events in Dutch hospitals. BMC Health Serv Res 2009; 9: 27. Lee SM, Kim CM, Lim SJ and Suh DC. Impact of adverse drug events on hospital length of stay and hospitalization costs in hospitals for 20032009 in the United States. Valu in heal 2013; 16: A202. Smits M, Zegers M, Groenewegen PP, Timmermans DR, Zwaan L, van der Wal G and Wagner C. Exploring the causes of adverse events in hospitals and potential prevention strategies. Qual Saf Health Care 2010; 19: e5. Wagner LM, Castle NG and Handler SM. Use of HIT for adverse event reporting in nursing homes: barriers and facilitators. Geriatr Nurs 2013; 34: 112-115. World Health Organization; Patients Safety: From information to action. 2014. Heard GC, Sanderson PM and Thomas RD. Barriers to adverse event and error reporting in anesthesia. Anesth Analg 2012; 114: 604614. Chiang HY and Pepper GA. Barriers to nurses’ reporting of medication administration errors in Taiwan. J Nurs Scholarsh 2006; 38: 392399.
Int J Clin Exp Med 2014;7(10):3543-3549
Reliability and validity of C-RoCAES  Chiang HY, Lin SY, Hsu SC and Ma SC. Factors determining hospital nurses’ failures in reporting medication errors in Taiwan. Nurs Outlook 2010; 58: 17-25.  Moumtzoglou A. Factors impeding nurses from reporting adverse events. J Nurs Manag 2010; 18: 542-547.  Li YM and Shi RF. Investigation and analysis on nursing error concealment. Chin Nurs Mana 2011; 11: 13-16.  Fang LY, Wang JN and Qiao GH. Investigation on factors influencing nurses’ report of adverse events. J Nurs Scie 2010; 25: 54-56.  Sanghera IS, Franklin BD and Dhillon S. The attitudes and beliefs of healthcare professionals on the causes and reporting of medication errors in a UK Intensive care unit. Anaesthesia 2007; 62: 53-61.  Karsh BT, Escoto KH, Beasley JW and Holden RJ. Toward a theoretical approach to medical error reporting system research and design. Appl Ergon 2006; 37: 283-295.  Hartnell N, MacKinnon N, Sketris I and Fleming M. Identifying, understanding and overcoming barriers to medication error reporting in hospitals: a focus group study. BMJ Qual Saf 2012; 21: 361-368.  Evans SM, Berry JG, Smith BJ, Esterman A, Selim P, O’Shaughnessy J and DeWit M. Attitudes and barriers to incident reporting: a collaborative hospital study. Qual Saf Health Care 2006; 15: 39-43.  Handler SM, Perera S, Olshansky EF, Studenski SA, Nace DA, Fridsma DB and Hanlon JT. Identifying modifiable barriers to medication error reporting in the nursing home setting. J Am Med Dir Assoc 2007; 8: 568-574.  Wu JH, Shen WS, Lin LM, Greenes RA and Bates DW. Testing the technology acceptance model for evaluating healthcare professionals’ intention to use an adverse event reporting system. Int J Qual Health Care 2008; 20: 123129.  Committee on Patient S, Health Information T and Institute of M. Health IT and Patient Safety: Building Safer Systems for Better Care. Washington (DC): National Academies Press (US); 2011. Copyright 2012 by the National Academy of Sciences. All rights reserved.
 Keeping patients safe: Institute of Medicine looks at transforming nurses’ work environment. Qual Lett Healthc Lead 2004; 16: 9-11.  Chen L, Zhu ZM and Wang RY. Investigation and Countermeasures of Influence Factors for Adverse Event Report. Nurs J Chin PLA 2010; 27: 628-629.  Long YF, Li YL and Guo YH. The status quo investigation and analysis of nursing adverse events report in China. Chin Nurs Mana 2011; 11: 16-20.  Chiang HY, Hsiao YC, Lin SY and Lee HF. Incident reporting culture: scale development with validation and reliability and assessment of hospital nurses in Taiwan. Int J Qual Health Care 2011; 23: 429-436.  Wilson B, Bekker HL and Fylan F. Reporting of Clinical Adverse Events Scale: a measure of doctor and nurse attitudes to adverse event reporting. Qual Saf Health Care 2008; 17: 364-367.  Polit DF and Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health 2006; 29: 489-497.  Wu ML. Structural equation modeling-peration and use of Amos. China: Chongqing University Press; 2010.  Ozer ZC, Firat MZ and Bektas HA. Confirmatory and exploratory factor analysis of the caregiver quality of life index-cancer with Turkish samples. Qual Life Res 2009; 18: 913-921.  Kline RB. Principles and practice of structural equation modeling. Guilford Press; 2011.  Hair JF, Tatham RL, Anderson RE and Black W. Muitivariate data analysis. USA: Prentice Hall; 1998.  Wakefield DS, Wakefield BJ, Uden-Holman T, Borders T, Blegen M and Vaughn T. Understanding why medication administration errors may not be reported. Am J Med Qual 1999; 14: 81-88.
Int J Clin Exp Med 2014;7(10):3543-3549