International Journal of Psychiatry in Clinical Practice, 2009; 13: 303306

ORIGINAL ARTICLE

The reliability and validity of the Chinese version of the Modified Overt Aggression Scale

HUI CHUN HUANG1, YU-TING WANG1,5, KAO CHING CHEN1,2, TZUNG LIEH YEH1,2, I HUI LEE1,2, PO SEE CHEN1,2,4, YEN KUANG YANG1,2,4 & RU BAND LU13 1

Department of Psychiatry, National Cheng Kung University Hospital, 2Department of Psychiatry, College of Medicine, National Cheng Kung University, Tainan, Taiwan, 3Institute of Behavioral Medicine, College of Medicine, National Cheng Kung University, 4National Cheng Kung University Hospital, Dou-liou Branch, Yunlin, Taiwan, and 5Department of Psychiatry, St. Martin De Porres Hospital, Chiayi, Taiwan

Abstract Objective. Instruments to assess aggressive behaviors in the psychiatric ward are crucial for monitoring risky behaviors. The purpose of this study was to assess the validity and reliability of the Chinese version of the Modified Overt Aggression Scale (MOAS). Methods. We translated the English version of MOAS into Chinese. We interviewed and trained two volunteers to act as agitated patients in the seclusion room. One senior psychiatrist, experienced in using the MOAS, directed scenarios of different aggressive intensity and established the standard scores. The validity was assessed by comparing the other raters’ scores with those of the director’s. Inter-rater reliability was also assessed. Results. Inter-rater reliability, based on Intra-class correlation coefficient (ICC0.94, P B0.001) and Kendall’s W coefficient of concordance (W0.83, P0.001), was high. The validity was assessed by the MannWhitney test. The results showed that the raters adequately differentiated (z2.89, P0.002) between the above-average and below-average scores of the MOAS. Conclusion. The Chinese version of the MOAS has modest psychometric properties. The model, which used trained volunteers acting as patients and compared their scores with those of the director, may be used in further studies for developing psychometric instruments to assess abrupt behaviors.

Key Words: Aggression, Modified Overt Aggression Scale, reliability, validity

Introduction Aggression in psychiatric wards represents a major concern for mental health professionals [1]. Instruments for recording and evaluating aggressive behaviors in the psychiatric ward are crucial for monitoring risky behaviors and evaluating intervention strategies. However, the psychometric for assessing aggressive events in the Chinese version is still lacking. Yudofsky et al. [2] developed the overt aggression scale (OAS) in order to assess and record the frequency and severity of aggressive events in psychiatric wards. The Modified Overt Aggression Scale (MOAS), a modified version of the OAS [3,4], is an inpatient rating scale that contains four

aggression categories: verbal aggression, aggression against objects, aggression against self, and aggression against others. For each category of aggressive behaviors, the rater checks the highest applicable rating point to describe the most serious act of aggression during the specified observation period. Each category is weighted using a psychometrically validated method developed by the MOAS authors. Specifically, a scale of increasing severity with verbal aggression is assigned the lowest weight and physical aggression the highest. The sum of the four categories is the total MOAS score, which indicates the severity of the aggressive behaviors. The different weight of each category allows the MAOS to provide more precise assessment than the OAS. The MOAS

Correspondence: Dr Kao Ching Chen, Department of Psychiatry, National Cheng Kung University Hospital, 138 Sheng Li Road, Tainan 70428, Taiwan. Tel: 886 6 235 3535, ext. 5189. Fax: 886 6 275 9259. E-mail: [email protected]

(Received 22 December 2008; accepted 13 May 2009) ISSN 1365-1501 print/ISSN 1471-1788 online # 2009 Informa UK Ltd. DOI: 10.3109/13651500903056533

304

H.C. Huang et al.

has been utilized widely in psychiatric patients to assess aggressive behaviors [5,6], risk [7] and medication effect [8,9]. Preliminary results of MOAS studies indicated good inter-rater reliability and validity of the English version [3,10] and the Italian version [11]. However, the methods of assessing inter-rater reliability in previous studies had several limitations. First of all, the scenarios observed by the raters about the same patient may differ since aggressive events in the psychiatric wards can be abrupt and influenced by the environment as well as staff intervention. The differences among the scenarios observed by the raters, may influence the rater’s scoring and affect the inter-rater reliability. Secondly, using a video camera to record patients’ behaviors without their permission is prohibited by law in many countries, including Taiwan. Therefore, it is impossible to use real patient material for aggressive behaviors for study or teaching purposes. Thirdly, the lack of a precise reference for validity should be considered when comparing the MOAS scores of psychiatric acute ward patients with those of chronic care patients. In this study, we translated the English version of the MOAS into a Chinese version. Our purpose was to propose a novel model of assessing the validity and reliability of MOAS by combining trained volunteers who acted as aggressive patients with the use of a video monitoring system.

Methods Instruments We translated the English version of MOAS into a Chinese version after a series of discussions by the staff in our psychiatric ward and our research team members. The Chinese version of the MOAS was applied. Participants and setting We recruited volunteers to act as patients with psychiatric disorders from the medical school by advertisements. Volunteers were interviewed by senior psychiatrists to select two suitable participants (Participant 1: aged 22, male, with college education; Participant 2: aged 22, female, with college education). One senior psychiatrist, who acted as the director, designed and directed the scenarios of different aggressive intensity. The selected actor and actress were trained to portray different intensity of aggression in the seclusion room of our psychiatric acute ward. There were a total of 16 scenarios of different aggressive intensity. Each scenario was recorded by the video monitoring system in the

seclusion room. The director arranged scenarios of different intensity and rated each scenario after directing each scenario according to the volunteer’s performance. Another three senior psychiatrists, who were blinded to the scenario setting, rated the scores of the MOAS by watching each scenario from the video monitoring system without any discussion with each other. The director’s rating scores were used as the validity reference. Data analysis Data were analyzed using the Statistical Package for Social Sciences (Version 12.0, SPSS, Chicago, IL). Intra-class correlation coefficient and Kendall’s W coefficient of concordance were used to determine the inter-rater reliability of the Chinese version of the MOAS. The MannWhitney test was applied to validate the theoretical structures of the scale. The director’s ratings were used as the validity reference. The other three other psychiatrists’ ratings were compared to the director’s ratings.

Results Reliability of the MOAS The mean MOAS score of the director’s rating toward all the 16 senarios was 13.6910.34. The mean MOAS score of all the 16 scenarios was 12.9298.53. Interrater reliability, measured by Intra-class correlation coefficient (ICC0.94, PB0.001) and Kendall’s W coefficient of concordance (W0.83, P0.001), was high. Validity of the MOAS The validity was assessed by the MannWhitney test. The average of the director’s MOAS rating scores toward the 16 senarios was 13. We devided the 16 senarios according to the director’s average score into the above-average group (five senarios) and the below-average group (11 senarios). The results showed that the other raters adequately (z2.89, P0.002) differentiated between the senarios with above-average scores and senarios with belowaverage scores. Discussion Our results showed that the Chinese version of the MOAS has modest psychometric properties. The Chinese version of the MOAS may be helpful for managing and monitoring inpatient aggressive behaviors. It may also be applied in future studies aimed at establishing local data.

305

Chinese version of MOAS and validity and reliability In this study, we demonstrated a novel model for assessing the reliability and validity of psychometrics. This model is useful for measuring aggressive events. Aggressive behaviors are often abrupt in their onset and thus difficult to predict. In addition, different raters may observe different characteristics about the same aggressive event. Moreover, the condition of the aggressive event is also influenced by the environment as well as staff intervention in the psychiatric service setting. The staff who had to provide intervention may not rate the score objectively. Combining volunteers who act as psychiatric patients with a video monitored system may improve the objectivity of MOAS ratings. A previous study used standard patients to be interviewed by two raters to assess the inter-rater reliability of the Overt Aggression Scale-Modified [12], another modified version of OAS. Andreasen et al. [13] proposed a model of assessing inter-rater reliability across multiple sites based on the exchange of videotaped interviews. However, they did not discuss the validity of their model. In the present study, we present a novel way to assess the validity of the MOAS by using a director’s rating as validity reference. The model of using scenarios directed by a senior psychiatrist and trained volunteers who acted as patients to assess the reliability and validity of the MOAS may be useful for future developments and assessments of psychometrics. The results of the present study should be interpreted with caution due to the following limitations. Firstly, the raters in the study were psychiatrists. In the clinical setting, the observers include nursing staff, psychologists and even social workers. Secondly, the number of raters was relatively small and the study was performed in only one medical center, instead of multiple sites. Thirdly, the method using volunteers as a substitute for real patients may have the drawback of not appearing like a real event in the psychiatric ward. Therefore, the training of the volunteers is very important. In our study, even the director selected suitable volunteers and they discussed each scenario together. The volunteers also practiced each scenario several times and received the director’s feedback. However, a discrepancy between the real clinical situation and the created scenario may still exist. Fourthly, this new evaluation method we developed for this study may need to be assessed further to establish its own reliability and validity. Further studies may apply this method for other psychometrics of which the validity and reliability are already confirmed. In conclusion, the present study demonstrated a novel way to assess the reliability and validity of the Chinese version of the MOAS. It is the first validation and reliability study of MOAS for the

Taiwanese population, highlighting the significance of risk behavior monitoring, which will contribute to further studies on MOAS in Taiwan in different settings. Key points . The Chinese version of psychometrics for assessing aggressive events is lacking . In the present study, we propose a novel model of assessing the validity and reliability of MOAS by combining trained volunteers who acted as aggressive patients with the use of a video monitoring system . Our results showed that the Chinese version of the MOAS has modest psychometric properties Acknowledgements This work was supported by a grant from the National Science Council of Taiwan (NSC 942218-E-006-043, NSC 95-2218-E-006-020). The authors wish to thank Linda Chang and Hsiu Yu Lo for their administrative assistance. Statement of Interest The authors have no conflict of interest with any commercial or other associations in connection with the submitted article.

References [1] Poster EC. A multinational study of psychiatric nursing staffs’ beliefs and concerns about work safety and patient assault. Arch Psychiatr Nurs 1996;10(6):36573. [2] Yudofsky SC, Silver JM, Jackson W, Endicott J, Williams D. The Overt Aggression Scale for the objective rating of verbal and physical aggression. Am J Psychiatry 1986;143(1):359. [3] Kay SR, Wolkenfeld F, Murrill LM. Profiles of aggression among psychiatric patients. II. Covariates and predictors. J Nerv Ment Dis 1988;176(9):54757. [4] Knoedler DW. The Modified Overt Aggression Scale. Am J Psychiatry 1989;146(8):10812. [5] Colasanti A, Natoli A, Moliterno D, Rossattini M, De Gaspari IF, Mauri MC. Psychiatric diagnosis and aggression before acute hospitalisation. Eur Psychiatry 2008;23(6): 4418. [6] Foley SR, Kelly BD, Clarke M, McTigue O, Gervin M, Kamali M, et al. Incidence and clinical correlates of aggression and violence at presentation in patients with first episode psychosis. Schizophr Res 2005;72(23):1618. [7] Troisi A, Kustermann S, Di Genio M, Siracusano A. Hostility during admission interview as a short-term predictor of aggression in acute psychiatric male inpatients. J Clin Psychiatry 2003;64(12):14604. [8] Buitelaar JK, van der Gaag RJ, Cohen-Kettenis P, Melman CT. A randomized controlled trial of risperidone in the treatment of aggression in hospitalized adolescents with /

/

/

/

/

/

/

/

/

/

/

/

306

H.C. Huang et al.

subaverage cognitive abilities. J Clin Psychiatry 2001;62(4): 23948. [9] Krakowski MI, Czobor P, Citrome L, Bark N, Cooper TB. Atypical antipsychotic agents in the treatment of violent patients with schizophrenia and schizoaffective disorder. Arch Gen Psychiatry 2006;63(6):6229. [10] Steinert T, Wolfle M, Gebhardt RP. Measurement of violence during in-patient treatment and association with psychopathology. Acta Psychiatr Scand 2000;102:10712. [11] Margari F, Matarazzo R, Casacchia M, et al. Italian validation of MOAS and NOSIE: a useful package for /

/

/

/

/

/

psychiatric assessment and monitoring of aggressive behaviours. Int J Methods Psychiatr Res 2005;14(2):10918. [12] Endicott J, Tracy K, Burt D, Olson E, Coccaro EF. A novel approach to assess inter-rater reliability in the use of the Overt Aggression Scale-Modified. Psychiatry Res 2002; 112(2):1539. [13] Andreasen NC, McDonald-Scott P, Grove WM, Keller MB, Shapiro RW, Hirschfeld RM. Assessment of reliability in multicenter collaborative research with a videotape approach. Am J Psychiatry 1982;139(7):87682. /

/

/

/

Copyright of International Journal of Psychiatry in Clinical Practice is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.

The reliability and validity of the Chinese version of the Modified Overt Aggression Scale.

Objective. Instruments to assess aggressive behaviors in the psychiatric ward are crucial for monitoring risky behaviors. The purpose of this study wa...
52KB Sizes 3 Downloads 6 Views