Intensive and Critical Care Nursing (2015) 31, 165—170

Available online at www.sciencedirect.com

ScienceDirect journal homepage: www.elsevier.com/iccn

Reliability and validity assessment of the Japanese version of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) Yuji Koga a,∗, Ryosuke Tsuruta b, Hiroaki Murata c, Koji Matsuo d, Tatsuhiko Ito e, E. Wesley Ely f,g, Ayumi Shintani h,i,j, Hiroya Wakamatsu k, Masamitsu Sanui l, Hiroaki Yamase m a

Department of Nursing, Yamaguchi University Hospital, Yamaguchi, Japan Advanced Medical Emergency and Critical Care Center, Yamaguchi University, Yamaguchi, Japan c Department of Nursing, Jikei University School of Medicine, Tokyo, Japan d Division of Neuropsychiatry, Department of Neuroscience, Yamaguchi University Graduate School of Medicine, Yamaguchi, Japan e Department of Psychiatry, Jikei University School of Medicine, Tokyo, Japan f Department of Medicine, Division of Allergy, Pulmonary and Critical Care Medicine Center for Health Services Research, Vanderbilt School of Medicine, Nashville, TN, USA g Geriatric Research, Education and Clinical Center (GRECC), Department of Veterans Affairs Medical Center, Tennessee Valley Healthcare System, Nashville, USA h Department of Biostatistics, Vanderbilt University School of Medicine, Nashville, USA i Center for Health Services Research, Vanderbilt University School of Medicine, Nashville, USA j Center for Quantitative Sciences, Vanderbilt University School of Medicine, Nashville, USA k Department of Intensive Care, Yamaguchi University, Yamaguchi, Japan l Department of Anesthesiology and Critical Care Medicine, Jichi Medical University Saitama Medical Center, Saitama, Japan m Department of Clinical Nursing, Yamaguchi University Graduate School of Medicine, Yamaguchi, Japan b

Accepted 20 October 2014

KEYWORDS CAM-ICU; Delirium; ICU; Japan;

Summary Objective: Delirium may lead to adverse outcomes in patients with serious conditions, but is often under-diagnosed due to inadequate screening. The Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) is an established method for assessing delirium in the ICU. The validity and reliability of the Japanese version of the CAM-ICU has not, however, been verified, and we undertook this study to verify these parameters.



Corresponding author at: Department of Nursing, Yamaguchi University Hospital, 1-1-1, Minami-Kogushi, Ube 755-8505, Yamaguchi, Japan. Tel.: +81 836 22 2111; fax: +81 836 22 2729. E-mail address: [email protected] (Y. Koga). http://dx.doi.org/10.1016/j.iccn.2014.10.002 0964-3397/© 2014 Elsevier Ltd. All rights reserved.

166 Reliability; Validity

Y. Koga et al. Research methodology: CAM-ICU validity and reliability were assessed in two Japanese ICUs. Using the evaluation of the DMS-IV-TR in the psychiatrists group as the standard criteria for delirium diagnosis, we compared the evaluation of the Japanese version of the CAM-ICU between the research nurses group and the staff nurses group. Results: According to DSM-IV-TR criteria, the prevalence of delirium was 22.0%, and according to CAM-ICU delirium was found in 22.0% with Research Nurses and 19.5% with Staff Nurses. CAM-ICU sensitivity ratings were 83% and 78%, while their specificity ratings were 95% and 97%, respectively. The Kappa inter-rater reliability was good ( = 0.85), and Cronbach’s alpha coefficient was 0.69 (95% CI: 0.57—0.79). Mean rating time for the CAM-ICU was 2.5—2.8 minutes for Research Nurses and Staff Nurses, respectively. Conclusion: The Japanese version of the CAM-ICU has comparable validity and reliability as a delirium assessment tool in surgical patients in two Japanese ICUs. With training, CAM-ICU can be incorporated into daily clinical practice. © 2014 Elsevier Ltd. All rights reserved.

Implications for Clinical Practice • The risk of delirium in seriously ill ICU patients is high and proper evaluation of ICU delirium is important. • Japanese ICUs, do not follow a standardised method for diagnosing delirium, and the diagnosis is solely made by psychiatrists, who are not often present in the ICU. • We evaluated the validity and reliability of the Japanese version of the confusion assessment method for the ICU (CAM-ICU) in order to standardise critical care in Japan. • Using a valid and reliable instrument may contribute to developing a national database that could contribute internationally to epidemiology on this important form of organ dysfunction in critical illness.

Introduction Delirium is highly prevalent in intensive care units (ICUs) (Morandi et al., 2008; Ouimet et al., 2007), and up to 80% of intubated patients may develop its symptoms (Ely et al., 2001a). Risk factors for delirium include host, severe disease, and iatrogenic factors (Smith et al., 2009). The condition is associated with increased mortality, prolonged hospitalisation, and long-term cognitive impairment (Ely et al., 2004; Pisani et al., 2009; van Rompaey et al., 2009). In addition, it is reported that delirium is associated with longer ICU stay duration and higher medical expenses (Ely et al., 2001a; Milbrandt et al., 2004). Over the past decade, the assessment standard for delirium has been the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition, Text Revision (DSM-IV-TR). Although the DSM-IV-TR has been supplanted by the DSM-5, the current edition does not substantially alter the delirium diagnosis. The manual lists the following constellation of symptoms with a diagnosis of delirium: changes and acute variations of mental state, upset mental state, attention deficit disorders, altered level of consciousness, hallucinations, delusions, illusions, and thought disorders (APA, 2000, 2013; Morandi et al., 2008). Because delirium has, by definition, a fluctuating course, it is often underdiagnosed in many clinical cases (Girard et al., 2008). Without the use of screening tools, research has shown that delirium is overlooked in 75% of ICU admissions (Spronk et al., 2009). The Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) is a widely accepted measure—–alongside the

Intensive Care Delirium Screening Checklist—–as a standard method of evaluating delirium in intensive care facilities. The current pain, agitation, and delirium (PAD) guideline recommends that these measures be used as delirium monitoring tools (Barr et al., 2013; Bergeron et al., 2001; Ely et al., 2001b). The CAM-ICU, a modified version of the Confusion Assessment Method (CAM), was designed for healthcare professionals other than psychiatrists (Inouye et al., 1990), and it has both high sensitivity and specificity when compared to the DSM-IV (Ely et al., 2001b). Furthermore, research has found it to have excellent inter-rater reliability when used by staff nurses (Pun et al., 2005). In Japan, for both research and clinical purposes, a valid and reliable delirium diagnostic-scale is also required. It is true that a Japanese translation of the CAM-ICU and a training manual endorsed by the original authors (Tsuruta et al., 2002) has become available, but its validity and reliability have not been evaluated. Moreover, the Japanese version contains changes made to two items concerning attention deficits. The original version employs an alphabet test (grasp hand when saying ‘‘A’’ in ‘‘SAVEAHAART’’), which the Japanese version substitutes for a number test (grasp hand when saying ‘‘1’’ in ‘‘6153191124’’) because the English alphabet is not common in Japan (Tsuruta et al., 2002).

Research question The purpose of this research is to verify the validity and reliability of the Japanese version of the CAM-ICU.

Reliability and validity assessment of the Japanese version of CAM-ICU Therefore, the sensitivity and specificity of the CAM-ICU in relation to the DSM-IV-TR are calculated. Moreover, the reliability between evaluators of CAM-ICU and inter-rater reliability (the kappa coefficient) and internal consistency (Cronbach’s ˛ coefficient) of the CAM-ICU are calculated.

Methods Sample size Sample size calculations were performed based on sensitivity. We computed the sample size for this study to obtain the necessary number of patients to yield a lower confidence interval (CI) of 70%. Assuming a prevalence of 30% for delirium, and a point estimate for sensitivity of 85%, we found that a sample size of 80 patients was required. The lower 95% CI for specificity was estimated to be 75% for a point estimate of 85% specificity.

Participants This study was conducted at two university-affiliated, tertiary care centre ICUs—–one in Yamaguchi Prefecture (with 12 beds) and the other in Tokyo (with 10 beds). The clinical research ethics committee of each facility approved the study, and this study conforms to the guidelines set by the Declaration of Helsinki. Elective surgical patients who were 20 years of age or older and were scheduled for an ICU admission duration of 24 hours or more were included in this study. Participants gave their written consent during the lead-up to their surgeries. Patients’ level of sedation was assessed by intensive care medical specialists using the RASS. Those with a score of −3 to +4 were then evaluated for delirium by the research team comprising psychiatrists, research nurses, and staff nurses. We excluded patients who had histories of mental illness, a poor grasp of Japanese, significant hearing or visual impairments (in order to ensure appropriate communication), a necessity to use muscle relaxants, and RASS scores lower than −3. We further excluded those who developed a cerebral infarction during or after surgery.

Measurements We noted each patient’s demographic data, medical history (including history of mental illness), in addition to their reasons for, and duration of, ICU admission. Additionally, we employed the following measures: (a) The Japanese version of the CAM-ICU for a dichotomous classification of delirium vs. non-delirium. The Japanese version of the CAM-ICU (Tsuruta et al., 2002) was created using the original version of the CAM-ICU (Ely et al., 2001b), which was translated into Japanese, and then was back-translated into English by one of the original measure’s Japanese-fluent authors. The original CAMICU creators then checked any discrepancies between the original and the back-translated versions.

167

(b) The Richmond Agitation-Sedation Scale (RASS) to evaluate participants’ abilities to complete the CAM-ICU (Sessler et al., 2002). (c) APACHE II, to evaluate participants’ illness severity (Knaus et al., 1985). (d) DSM-IV-TR diagnostic criteria for delirium, which enabled a licensed psychiatrist to make a dichotomous classification (delirium/no delirium) (APA, 2000).

Procedure Three groups performed the delirium diagnosis, and the group of psychiatrists diagnosed patients daily. The group of research nurses all had master’s degrees in nursing science, and they received CAM-ICU evaluation training by one of the authors of the Japanese version of the CAM-ICU. The group of staff nurses all had more than four years of ICU clinical experience, and they received CAM-ICU evaluation training from the research nurse group. CAM-ICU evaluation training was performed using the Japanese version of the CAM-ICU training manual (Tsuruta et al., 2002). Psychiatrists performed a delirium assessment using DSMIV-TR criteria within an arbitrarily chosen time frame (10 AM to 5 PM). After the psychiatrist’s evaluation, the research nurse performed a delirium assessment using the CAM-ICU within a window of two hours. Following this assessment, the staff nurse performed a second CAM-ICU evaluation. Delirium is, by definition, a fluctuating condition; therefore, all delirium evaluations were completed within two hours to get the most accurate possible assessment. Each subsequent delirium evaluator was unaware of previous evaluation results, and the assessment was performed once for each patient.

Statistical analysis To determine the validity of the Japanese version of CAM-ICU, we calculated its sensitivity, specificity, positive predictive value, and negative predictive value in relation to the DSM-IV-TR. In addition, as an index of reliability, we calculated inter-rater reliability between the research nurse and the staff nurse for CAM-ICU through the Kappa coefficient (). Internal consistency was evaluated by Cronbach’s ˛. We performed the analysis using R version 3.0.1 (www.r-project.org), with continuous variables expressed as mean ± standard deviation. Probability and coefficients were expressed using a 95% CI.

Results Participants From the two ICUs, a total of six psychiatrists, three research nurses, and five staff nurses participated in this study. All participants understood the purpose of this research and consented to participate in the study. Ninety-nine patients were admitted to the two ICUs between September 2012 and June 2013. Of these, we excluded two (2%) due to post-surgical cerebral infarction,

168 Table 1

Y. Koga et al. Table 3 Reliability analysis of the Japanese version of the CAM-ICU.

Primary patient characteristics. n (%)

 (95% CI)

Gender Male Female Age (mean ± SD) APACHE II (mean ± SD)

48 (58.5) 34 (41.5) 68.5 ± 10.3 14.7 ± 4.7

Intubation Intubation No intubation

11 (11.8) 71 (88.2)

Use of analgesics and/or sedation Analgesic use Sedatives use Analgesic and sedatives use None

18 (22.0) 7 (8.5) 4 (4.9) 53 (64.6)

Sedation and agitation Level Mean RASS RASS ≥+2 RASS +1 RASS 0 RASS −1 RASS −2

−.33 ± 2.5 0 1 (1.2) 54 (65.9) 27 (32.9) 1 (1.2)

Main reason for ICU admission Cardiovascular surgery Abdominal surgery Thoracic surgery

68 (82.9%) 13 (15.9%) 1 (1.2%)

Complete FEATURE 1 ‘‘Acute onset or fluctuating course’’ FEATURE 2 ‘‘Inattention’’ FEATURE 3 ‘‘Altered level of consciousness’’ FEATURE 4 ‘‘Disorganized thinking’’

0.90 (0.68—1.11) 0.53 (0.32—0.75) 0.69 (0.47—0.90)

interval between the psychiatrist and the research nurse was 20.7 ± 21.4 min, and the evaluation interval between the research and staff nurses was 24.1 ± 26.5 min.

Validity

and 15 patients (15.5%) were found to be in the state of RASS

Reliability and validity assessment of the Japanese version of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU).

Delirium may lead to adverse outcomes in patients with serious conditions, but is often under-diagnosed due to inadequate screening. The Confusion Ass...
421KB Sizes 1 Downloads 8 Views