Qual Life Res DOI 10.1007/s11136-015-0986-7
BRIEF COMMUNICATION
Validation of the Athens Insomnia Scale for screening insomnia in South Korean firefighters and rescue workers Hyeonseok S. Jeong1 • Yujin Jeon2 • Jiyoung Ma2 • Yera Choi2 • Soonhyun Ban2,4 Sooyeon Lee2,4 • Bora Lee2,4 • Jooyeon Jamie Im2 • Sujung Yoon2 • Jieun E. Kim2,4 • Jae-ho Lim5 • In Kyoon Lyoo2,3,4
•
Accepted: 3 April 2015 Ó Springer International Publishing Switzerland 2015
Abstract Purpose Sleep problems are a major cause of occupational stress in firefighters and rescue workers. We evaluated the psychometric properties of the Athens Insomnia Scale (AIS) among South Korean firefighters and rescue workers. Methods Structured clinical interviews and self-report questionnaires were administered to 221 firefighters and rescue workers. The Pittsburgh Sleep Quality Index (PSQI), Insomnia Severity Index (ISI), Epworth Sleepiness Scale (ESS), Short-Form 36-item Health Survey (SF36), and Alcohol Use Disorder Identification Test—Consumption (AUDIT-C) were used to examine convergent and divergent validity. Test–retest reliability was calculated from a subsample (n = 24). Analysis of internal
consistency, factor analysis, and receiver operator characteristic curve analysis were conducted. Results Cronbach’s alpha was 0.88. The mean item-total correlation coefficient was 0.73. The test–retest reliability was excellent (ICC = 0.94). Significant correlations of the AIS with the PSQI, ISI, ESS, and SF36 confirmed convergent validity. Nonsignificant associations of the AIS with the AUDIT-C and socioeconomic status showed divergent validity. Factor analysis revealed a one-factor structure. For groups with different symptom severity, group-specific cutoff scores which may improve positive predictive values were suggested. Conclusions The AIS may be a useful tool with good reliability and validity for screening insomnia symptoms in firefighters and rescue workers. Keywords Insomnia Firefighter Rescue worker Validation Athens Insomnia Scale
Hyeonseok S. Jeong and Yujin Jeon contributed equally to this work.
Electronic supplementary material The online version of this article (doi:10.1007/s11136-015-0986-7) contains supplementary material, which is available to authorized users. & In Kyoon Lyoo [email protected] 1
Department of Radiology, Incheon St. Mary’s Hospital, The Catholic University of Korea, Incheon, South Korea
2
Ewha Brain Institute, Ewha W. University, Seoul, South Korea
3
Graduate School of Pharmaceutical Sciences, Ewha W. University, Seoul, South Korea
4
Department of Brain and Cognitive Sciences, Ewha W. University, Seoul, South Korea
5
Central Officials Training Institute, Gwacheon, Gyeonggi-do, South Korea
Introduction Firefighters and rescue workers have been considered as a high-risk occupational group due to their chronic exposure to stressful work environments [1, 2]. Previous literature have shown that they tend to develop more frequent and severe mental and sleep problems [3, 4]. Particularly, insomnia may threaten their safety by increasing the risk of being involved in accidents during work [5, 6]. Therefore, establishing an effective and accurate measure to assess insomnia in firefighters and rescue workers is necessary to identify high-risk individuals with insomnia and provide appropriate interventions. The Athens Insomnia Scale (AIS) is a brief measure which has been validated in the general population,
123
Qual Life Res
insomnia patients, and psychiatric patients [7–9]. As the AIS contains only a moderate number of items and uses simple scoring methods, it can be used as a convenient and effective diagnostic tool based on the International Classification of Diseases-10 (ICD-10) [10]. Notwithstanding its potential usefulness for clinical and research purposes, the AIS has not yet been validated in firefighters or rescue workers. The aim of the current study is to validate the AIS in South Korean firefighters and rescue workers and to determine the optimal cutoff scores to discriminate between insomnia groups with different severity.
Materials and methods Participants Participants were recruited from four fire departments in Seoul, South Korea. After strict confidentiality was assured, written informed consent was voluntarily obtained. The study protocol was reviewed and approved by the Bioethics Committee of Ewha W. University. Self-report questionnaires The AIS contains eight items, which are scored on a fourpoint Likert scale. Total AIS scores range from 0 to 24 [8], with a higher score indicating greater insomnia symptom severity. The widely accepted cutoff score for the diagnosis of insomnia is 6 [9]. The Pittsburgh Sleep Quality Index (PSQI) and Insomnia Severity Index (ISI) were used to assess insomnia symptoms [11, 12]. To assess daytime sleepiness, the Epworth Sleepiness Scale (ESS) was used [13]. The Short-Form 36-Item Health Survey (SF36) was used to evaluate physical and mental functioning [14]. The Alcohol Use Disorder Identification Test—Consumption (AUDIT-C) assessed alcohol use problems [15], while the Hollingshead Four-Factor Index of Socioeconomic Status was used to evaluate socioeconomic status [16]. Structured clinical interviews The Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders-IV—Text Revision (DSM-IV-TR) [17] and a structured clinical interview for insomnia were administered by clinicians trained by a board-certified psychiatrist (Y. J.). The structured clinical interview for insomnia was developed based on the ICD10, DSM-IV-TR, and Research Diagnostic Criteria [10, 18, 19]. According to the interview results, participants were divided into insomnia (n = 76) and non-insomnia (n = 138) groups. Individuals with insomnia were
123
subdivided into three groups: group 1 (n = 76), participants with insomnia symptoms including difficulty initiating sleep, difficulty maintaining sleep, non-restorative sleep, or early morning awakening that persisted for at least 1 month; group 2 (n = 40), those in group 1 whose daily functioning had been significantly disturbed; and group 3 (n = 21), those in group 2 who had symptoms even during off-duty periods. Data analysis The relationship between the AIS scores and age was examined using linear regression. Internal consistency and item-total correlation were evaluated using Cronbach’s alpha and Pearson’s correlation coefficient, respectively. One-week test–retest reliability was assessed with the intraclass correlation coefficient in a subsample. Linear trends of the AIS scores across the non-insomnia group and three insomnia groups were confirmed using orthogonal polynomial contrasts. As an item-level analysis, we estimated the effect size between the insomnia and non-insomnia groups and the linear trends across the noninsomnia group and three insomnia groups. To evaluate convergent validity, Pearson’s correlations between the AIS scores and the scores of the PSQI, ISI, ESS, and SF36 were assessed. Divergent validity was evaluated by correlating the AIS scores with the AUDIT-C scores and socioeconomic status. The adequacy of the data for factor analysis was supported by the Kaiser–Meyer–Olkin measure of sampling adequacy and Bartlett’s test of sphericity. Principal component analysis with varimax rotation was conducted to examine the factor structure. The diagnostic accuracy was determined by calculating the area under the curve (AUC) after a receiver operating characteristic (ROC) curve was constructed. Sensitivity, specificity, likelihood ratio, and predictive values of each insomnia group were calculated for each cutoff score. The optimal cutoff score for each insomnia group was identified as the point with the maximal Youden’s index. Stata version 12.1 (StataCorp., College Station, TX, USA) was used for statistical analyses. A two-sided p value of 0.05 was considered significant.
Results A total of 221 participants were enrolled (207 men and 14 women), and test–retest reliability was assessed in a subsample of 24 participants. The mean age was 40.3 ± 9.2 years. The mean years of service as a firefighter or a rescue worker was 12.7 ± 8.8 years. The mean AIS score was 6.0 ± 4.1. The AIS score was positively associated with age (b = 0.59, R2 = 0.02, p = 0.03). The AIS total
Qual Life Res
score did not show a statistically significant association with the recent work demand which was assessed by the average number of emergency-response dispatches per week (r = -0.04, p = 0.58). The detailed characteristics of the participants are summarized in the Supplementary Table 1. Cronbach’s alpha of the AIS was 0.88, and the mean item-total correlation coefficient was 0.73. Table 1 presents descriptive statistics, item-total correlation, and oneweek test–retest reliability of the AIS. The total and itemlevel scores of the groups showed significant linear trends (all p for trend \ 0.001): Total AIS scores—non-insomnia group, 4.1 ± 2.9; group 1, 9.3 ± 3.8; group 2, 10.1 ± 3.9; group 3, 11.1 ± 4.1. Results from the tests to examine convergent and divergent validity are summarized in Table 2. (See the Supplementary Table 2 for the item-level analysis). The Kaiser–Meyer–Olkin statistic was 0.86, and Bartlett’s test of sphericity was significant (p \ 0.001). Factor analysis revealed a one-factor structure explaining 95.73 % of the variance (eigenvalue = 3.89). The factor loading of each item ranged from 0.51 to 0.82. The ROC curves for each insomnia group are shown in Fig. 1. The optimal cutoff scores were 6 for group 1, 8 for group 2, and 9 for group 3 (Table 3).
Discussion The current study examined the psychometric properties of the AIS in South Korean firefighters and rescue workers, using validated structured clinical interviews and self-report measures. The results suggest that the AIS is a reliable and valid measure to evaluate insomnia symptoms in this occupation group. Age was positively correlated with insomnia symptom severity, as reported in previous research [20]. This effect
Table 2 Convergent and divergent validity of the Athens Insomnia Scale Measure
r
p
Convergent validity PSQIa
0.82
\0.001
ISIb
0.85
\0.001
ESSb
0.29
\0.001
SF36 MCSc
-0.53
\0.001
SF36 PCSc
-0.37
\0.001
Divergent validity AUDIT-Cd Socioeconomic statuse
0.10
0.13
0.01
0.92
PSQI Pittsburgh Sleep Quality Index, ISI Insomnia Severity Index, ESS Epworth Sleepiness Scale, SF36 Short-Form 36-item Health Survey, MCS mental component summary, PCS physical component summary, AUDIT-C Alcohol Use Disorder Identification Test— Consumption a
n = 214,
b
n = 221,
c
n = 218,
d
n = 220, e n = 213
of age may be due to the cumulative effects of occupational stress including stress from shift work or traumatic experiences [21]. On the other side, the recent work demand did not show a significant association with insomnia symptoms in this sample. This finding is in line with a previous observation that work-related impact on sleep has been reported as a cumulative rather than short-term effect [22]. In the current study, Cronbach’s alpha was similar to that of the original scale and the mean item-total correlation was higher than that of the original scale [8], suggesting satisfactory internal consistency. The one-week test–retest reliability coefficients for the total score (0.94) and for each individual item (range 0.61–0.95) were higher than those of the original scale [8]. The AIS scores were significantly correlated with alternative measures evaluating sleep quality or insomnia
Table 1 Descriptive statistics, item-total correlation, and test–retest reliability of the Athens Insomnia Scale Item (range 0–3)a
Mean
Standard deviation
Item-total correlation
One-week test–retest reliability
1
‘‘Sleep induction’’
0.8
0.8
0.82
0.81
2
‘‘Awakenings during the night’’
0.8
0.7
0.76
0.82
3
‘‘Final awakening earlier than desired’’
0.5
0.7
0.61
0.95
4
‘‘Total sleep duration’’
1.0
0.8
0.75
0.61
5
‘‘Overall quality of sleep’’
0.9
0.7
0.84
0.78
6
‘‘Sense of well-being during the day’’
0.5
0.7
0.75
0.67
7
‘‘Functioning (physical and mental) during the day’’ 0.5
0.6
0.76
0.73
8
‘‘Sleepiness during the day’’
1.0
0.5
0.56
0.58
6.0
4.1
Total score (range 0–24) a
0.94
All items were adopted from the original scale in Soldatos et al. [8]
123
Qual Life Res Table 3 Diagnostic performance of the AIS for detecting insomnia Cutoff scorea
Sensitivity
Specificity
?LR
-LR
PPV
NPV
4
0.99
0.44
1.75
0.03
0.49
0.98
5
0.92
0.59
2.23
0.14
0.55
0.93
6
0.87
0.72
3.07
0.18
0.63
0.91
7
0.75
0.80
3.83
0.31
0.68
0.85
8
0.67
0.90
6.62
0.37
0.79
0.83
6
0.90
0.60
2.27
0.17
0.34
0.96
7
0.80
0.70
2.68
0.29
0.38
0.94
8
0.73
0.79
3.50
0.35
0.45
0.93
9 10
0.60 0.53
0.87 0.91
4.75 6.09
0.46 0.52
0.52 0.58
0.91 0.89
7
0.86
0.66
2.51
0.22
0.21
0.98
8
0.76
0.75
3.00
0.32
0.25
0.97
9
0.71
0.84
4.45
0.34
0.33
0.96
10
0.57
0.88
4.60
0.49
0.33
0.95
11
0.57
0.92
6.89
0.47
0.43
0.95
Group 1b
Group 2c
Group 3d
?LR positive likelihood ratio, -LR negative likelihood ratio, PPV positive predictive value, NPV negative predictive value a Optimal cutoff scores determined based on Youden’s index appear in bold b
Group 1 included the participants with insomnia symptoms (i.e., difficulty initiating sleep, difficulty maintaining sleep, non-restorative sleep, and early morning awakening) that had persisted for at least 1 month
c
Group 2 included the participants in group 1 whose daily functioning had been significantly impaired by insomnia symptoms
d
Group 3 included the participants in group 2 whose insomnia symptoms remained even when they could have enough time for sleep
Fig. 1 Receiver operating characteristic curves of the AIS. AUC Area under the receiver operating characteristics curve. a Group 1 included the participants with insomnia symptoms (i.e., difficulty initiating sleep, difficulty maintaining sleep, non-restorative sleep, and early morning awakening) that had persisted for at least 1 month (AUC = 0.87). b Group 2 included the participants in group 1 whose daily functioning had been significantly impaired by insomnia symptoms (AUC = 0.84). c Group 3 included the participants in group 2 whose insomnia symptoms remained even when they could have enough time for sleep (AUC = 0.85)
symptoms (PSQI, ISI, and ESS) and with questionnaires assessing overall health status (SF36). In accordance with the previous reports [7, 23], these results imply good
123
convergent validity. The AIS scores were neither associated with the AUDIT-C scores nor with socioeconomic status, supporting the divergent validity of the AIS. The positive linear trend of the AIS scores was found among the non-insomnia group and the insomnia groups as the symptom severity increased, indicating additional evidence of external validity. Factor analysis supports the homogeneity of the AIS, which replicates the findings from the study that validated the original scale [8] and the study by Go´mez-Benito et al. [24]. In a study by Soldatos et al. [9] in which the study sample was mainly composed of primary care clinical outpatients with a complaint of insomnia and psychiatric patients, the cutoff score to distinguish patients with clinically significant insomnia symptoms was suggested to be 6. This insomnia group can be considered comparable to group 2 in the current study, which was best distinguished with a higher cutoff score of 8. Considering that firefighters
Qual Life Res
and rescue workers generally experience frequent and prolonged work-related sleep interference, a relatively high cutoff score may have a better positive predictive value to discern those who are in need of clinical care. Although insomnia symptom severity was thoroughly assessed through face-to-face interviews and self-report questionnaires in the current study, objective sleep measures such as polysomnography and actigraphy would have provided important additional information. In conclusion, the current study suggests that the AIS is suitable for diagnosing insomnia in firefighters and rescue workers and thus may be considered as a valid and effective screening tool for insomnia symptoms. The usage of AIS could also be expanded into studies addressing sleep problems in other high-stress occupational groups (e.g., police officers and military personnel). Further studies exploring the different impacts on sleep according to types of work and intensity of workloads are needed. Acknowledgments This study was supported by the National Emergency Management Agency of South Korea. Conflict of interest interest.
The authors have no financial conflicts of
References 1. Vargas de Barros, V., Martins, L. F., Saitz, R., Bastos, R. R., & Ronzani, T. M. (2013). Mental health conditions, individual and job characteristics and sleep disturbances among firefighters. Journal of Health Psychology, 18(3), 350–358. doi:10.1177/ 1359105312443402. 2. Ohayon, M. M., Smolensky, M. H., & Roth, T. (2010). Consequences of shiftworking on sleep duration, sleepiness, and sleep attacks. Chronobiology International, 27(3), 575–589. doi:10. 3109/07420521003749956. 3. Beaton, R. D., & Murphy, S. A. (1993). Sources of occupational stress among firefighter/EMTs and firefighter/paramedics and correlations with job-related outcomes. Prehospital and Disaster Medicine, 8(2), 140–150. doi:10.1017/S1049023X00040218. 4. Bos, J., Mol, E., Visser, B., & Frings-Dresen, M. (2004). Risk of health complaints and disabilities among Dutch firefighters. International Archives of Occupational and Environmental Health, 77(6), 373–382. doi:10.1007/s00420-004-0537-y. 5. Patterson, P. D., Weaver, M. D., Frank, R. C., Warner, C. W., Martin-Gill, C., Guyette, F. X., et al. (2011). Association between poor sleep, fatigue, and safety outcomes in emergency medical services providers. Prehospital Emergency Care, 16(1), 86–97. doi:10.3109/10903127.2011.616261. 6. Saijo, Y., Ueno, T., & Hashimoto, Y. (2008). Twenty-four-hour shift work, depressive symptoms, and job dissatisfaction among Japanese firefighters. American Journal of Industrial Medicine, 51(5), 380–391. doi:10.1002/ajim.20571. 7. Okajima, I., Nakajima, S., Kobayashi, M., & Inoue, Y. (2013). Development and validation of the Japanese version of the Athens Insomnia Scale. Psychiatry and Clinical Neurosciences, 67(6), 420–425. doi:10.1111/pcn.12073.
8. Soldatos, C. R., Dikeos, D. G., & Paparrigopoulos, T. J. (2000). Athens Insomnia Scale: Validation of an instrument based on ICD-10 criteria. Journal of Psychosomatic Research, 48(6), 555–560. doi:10.1016/S0022-3999(00)00095-7. 9. Soldatos, C. R., Dikeos, D. G., & Paparrigopoulos, T. J. (2003). The diagnostic validity of the Athens Insomnia Scale. Journal of Psychosomatic Research, 55(3), 263–267. doi:10.1016/s00223999(02)00604-9. 10. World Health Organization. (1992). The ICD-10 classification of mental and behavioural disorders: Clinical descriptions and diagnostic guidelines. Geneva: World Health Organization. 11. Bastien, C. H., Vallieres, A., & Morin, C. M. (2001). Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Medicine, 2(4), 297–307. doi:10.1016/ S1389-9457(00)00065-4. 12. Buysse, D. J., Reynolds, C. F, 3rd, Monk, T. H., Berman, S. R., & Kupfer, D. J. (1989). The Pittsburgh Sleep Quality Index: A new instrument for psychiatric practice and research. Psychiatry Research, 28(2), 193–213. doi:10.1016/0165-1781(89)90047-4. 13. Johns, M. W. (1991). A new method for measuring daytime sleepiness: The Epworth sleepiness scale. Sleep, 14(6), 540–545. 14. Ware, J. E., Kosinski, M., & Dewey, J. E. (2000). How to score version 2 of the SF-36Ò health survey. Lincoln, RI: QualityMetric Incorporated. 15. Bush, K., Kivlahan, D. R., McDonell, M. B., Fihn, S. D., Bradley, K. A., & Project, A. C. Q. I. (1998). The AUDIT alcohol consumption questions (AUDIT-C)—An effective brief screening test for problem drinking. Archives of Internal Medicine, 158(16), 1789–1795. doi:10.1001/archinte.158.16.1789. 16. Hollingshead, A. B. (1975). Four factor index of social status. Unpublished manuscript. New Haven: Yale University. 17. First, M., Gibbon, M., Spitzer, R. L., & Williams, J. (1996). User’s guide for the structured clinical interview for DSM-IV axis I disorders—research version. New York: Biometrics Research. 18. American Psychiatric Association. (2000). Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR). Wasington, DC: American Psychiatric Association. 19. Edinger, J. D., Bonnet, M. H., Bootzin, R. R., Doghramji, K., Dorsey, C. M., Espie, C. A., et al. (2004). Derivation of research diagnostic criteria for insomnia: Report of an American Academy of Sleep Medicine Work Group. Sleep, 27(8), 1567–1596. 20. Akerstedt, T., Knutsson, A., Westerholm, P., Theorell, T., Alfredsson, L., & Kecklund, G. (2002). Sleep disturbances, work stress and work hours: A cross-sectional study. Journal of Psychosomatic Research, 53(3), 741–748. doi:10.1016/S00223999(02)00333-1. 21. Turner, R. J., & Lloyd, D. A. (1995). Lifetime traumas and mental health: The significance of cumulative adversity. Journal of Health and Social Behavior, 36(4), 360–376. doi:10.2307/ 2137325. 22. De Lange, A. H., Kompier, M. A., Taris, T. W., Geurts, S. A., Beckers, D. G., Houtman, I. L., et al. (2009). A hard day’s night: A longitudinal study on the relationships among job demands and job control, sleep quality and fatigue. Journal of Sleep Research, 18(3), 374–383. 23. Sun, J. L., Chiou, J. F., & Lin, C. C. (2011). Validation of the Taiwanese version of the Athens Insomnia Scale and assessment of insomnia in Taiwanese cancer patients. Journal of Pain and Symptom Management, 41(5), 904–914. doi:10.1016/j.jpainsym man.2010.07.021. 24. Gomez-Benito, J., Ruiz, C., & Guilera, G. (2011). A Spanish version of the Athens Insomnia Scale. Quality of Life Research, 20(6), 931–937. doi:10.1007/s11136-010-9827-x.
123
BRIEF COMMUNICATION
Validation of the Athens Insomnia Scale for screening insomnia in South Korean firefighters and rescue workers Hyeonseok S. Jeong1 • Yujin Jeon2 • Jiyoung Ma2 • Yera Choi2 • Soonhyun Ban2,4 Sooyeon Lee2,4 • Bora Lee2,4 • Jooyeon Jamie Im2 • Sujung Yoon2 • Jieun E. Kim2,4 • Jae-ho Lim5 • In Kyoon Lyoo2,3,4
•
Accepted: 3 April 2015 Ó Springer International Publishing Switzerland 2015
Abstract Purpose Sleep problems are a major cause of occupational stress in firefighters and rescue workers. We evaluated the psychometric properties of the Athens Insomnia Scale (AIS) among South Korean firefighters and rescue workers. Methods Structured clinical interviews and self-report questionnaires were administered to 221 firefighters and rescue workers. The Pittsburgh Sleep Quality Index (PSQI), Insomnia Severity Index (ISI), Epworth Sleepiness Scale (ESS), Short-Form 36-item Health Survey (SF36), and Alcohol Use Disorder Identification Test—Consumption (AUDIT-C) were used to examine convergent and divergent validity. Test–retest reliability was calculated from a subsample (n = 24). Analysis of internal
consistency, factor analysis, and receiver operator characteristic curve analysis were conducted. Results Cronbach’s alpha was 0.88. The mean item-total correlation coefficient was 0.73. The test–retest reliability was excellent (ICC = 0.94). Significant correlations of the AIS with the PSQI, ISI, ESS, and SF36 confirmed convergent validity. Nonsignificant associations of the AIS with the AUDIT-C and socioeconomic status showed divergent validity. Factor analysis revealed a one-factor structure. For groups with different symptom severity, group-specific cutoff scores which may improve positive predictive values were suggested. Conclusions The AIS may be a useful tool with good reliability and validity for screening insomnia symptoms in firefighters and rescue workers. Keywords Insomnia Firefighter Rescue worker Validation Athens Insomnia Scale
Hyeonseok S. Jeong and Yujin Jeon contributed equally to this work.
Electronic supplementary material The online version of this article (doi:10.1007/s11136-015-0986-7) contains supplementary material, which is available to authorized users. & In Kyoon Lyoo [email protected] 1
Department of Radiology, Incheon St. Mary’s Hospital, The Catholic University of Korea, Incheon, South Korea
2
Ewha Brain Institute, Ewha W. University, Seoul, South Korea
3
Graduate School of Pharmaceutical Sciences, Ewha W. University, Seoul, South Korea
4
Department of Brain and Cognitive Sciences, Ewha W. University, Seoul, South Korea
5
Central Officials Training Institute, Gwacheon, Gyeonggi-do, South Korea
Introduction Firefighters and rescue workers have been considered as a high-risk occupational group due to their chronic exposure to stressful work environments [1, 2]. Previous literature have shown that they tend to develop more frequent and severe mental and sleep problems [3, 4]. Particularly, insomnia may threaten their safety by increasing the risk of being involved in accidents during work [5, 6]. Therefore, establishing an effective and accurate measure to assess insomnia in firefighters and rescue workers is necessary to identify high-risk individuals with insomnia and provide appropriate interventions. The Athens Insomnia Scale (AIS) is a brief measure which has been validated in the general population,
123
Qual Life Res
insomnia patients, and psychiatric patients [7–9]. As the AIS contains only a moderate number of items and uses simple scoring methods, it can be used as a convenient and effective diagnostic tool based on the International Classification of Diseases-10 (ICD-10) [10]. Notwithstanding its potential usefulness for clinical and research purposes, the AIS has not yet been validated in firefighters or rescue workers. The aim of the current study is to validate the AIS in South Korean firefighters and rescue workers and to determine the optimal cutoff scores to discriminate between insomnia groups with different severity.
Materials and methods Participants Participants were recruited from four fire departments in Seoul, South Korea. After strict confidentiality was assured, written informed consent was voluntarily obtained. The study protocol was reviewed and approved by the Bioethics Committee of Ewha W. University. Self-report questionnaires The AIS contains eight items, which are scored on a fourpoint Likert scale. Total AIS scores range from 0 to 24 [8], with a higher score indicating greater insomnia symptom severity. The widely accepted cutoff score for the diagnosis of insomnia is 6 [9]. The Pittsburgh Sleep Quality Index (PSQI) and Insomnia Severity Index (ISI) were used to assess insomnia symptoms [11, 12]. To assess daytime sleepiness, the Epworth Sleepiness Scale (ESS) was used [13]. The Short-Form 36-Item Health Survey (SF36) was used to evaluate physical and mental functioning [14]. The Alcohol Use Disorder Identification Test—Consumption (AUDIT-C) assessed alcohol use problems [15], while the Hollingshead Four-Factor Index of Socioeconomic Status was used to evaluate socioeconomic status [16]. Structured clinical interviews The Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders-IV—Text Revision (DSM-IV-TR) [17] and a structured clinical interview for insomnia were administered by clinicians trained by a board-certified psychiatrist (Y. J.). The structured clinical interview for insomnia was developed based on the ICD10, DSM-IV-TR, and Research Diagnostic Criteria [10, 18, 19]. According to the interview results, participants were divided into insomnia (n = 76) and non-insomnia (n = 138) groups. Individuals with insomnia were
123
subdivided into three groups: group 1 (n = 76), participants with insomnia symptoms including difficulty initiating sleep, difficulty maintaining sleep, non-restorative sleep, or early morning awakening that persisted for at least 1 month; group 2 (n = 40), those in group 1 whose daily functioning had been significantly disturbed; and group 3 (n = 21), those in group 2 who had symptoms even during off-duty periods. Data analysis The relationship between the AIS scores and age was examined using linear regression. Internal consistency and item-total correlation were evaluated using Cronbach’s alpha and Pearson’s correlation coefficient, respectively. One-week test–retest reliability was assessed with the intraclass correlation coefficient in a subsample. Linear trends of the AIS scores across the non-insomnia group and three insomnia groups were confirmed using orthogonal polynomial contrasts. As an item-level analysis, we estimated the effect size between the insomnia and non-insomnia groups and the linear trends across the noninsomnia group and three insomnia groups. To evaluate convergent validity, Pearson’s correlations between the AIS scores and the scores of the PSQI, ISI, ESS, and SF36 were assessed. Divergent validity was evaluated by correlating the AIS scores with the AUDIT-C scores and socioeconomic status. The adequacy of the data for factor analysis was supported by the Kaiser–Meyer–Olkin measure of sampling adequacy and Bartlett’s test of sphericity. Principal component analysis with varimax rotation was conducted to examine the factor structure. The diagnostic accuracy was determined by calculating the area under the curve (AUC) after a receiver operating characteristic (ROC) curve was constructed. Sensitivity, specificity, likelihood ratio, and predictive values of each insomnia group were calculated for each cutoff score. The optimal cutoff score for each insomnia group was identified as the point with the maximal Youden’s index. Stata version 12.1 (StataCorp., College Station, TX, USA) was used for statistical analyses. A two-sided p value of 0.05 was considered significant.
Results A total of 221 participants were enrolled (207 men and 14 women), and test–retest reliability was assessed in a subsample of 24 participants. The mean age was 40.3 ± 9.2 years. The mean years of service as a firefighter or a rescue worker was 12.7 ± 8.8 years. The mean AIS score was 6.0 ± 4.1. The AIS score was positively associated with age (b = 0.59, R2 = 0.02, p = 0.03). The AIS total
Qual Life Res
score did not show a statistically significant association with the recent work demand which was assessed by the average number of emergency-response dispatches per week (r = -0.04, p = 0.58). The detailed characteristics of the participants are summarized in the Supplementary Table 1. Cronbach’s alpha of the AIS was 0.88, and the mean item-total correlation coefficient was 0.73. Table 1 presents descriptive statistics, item-total correlation, and oneweek test–retest reliability of the AIS. The total and itemlevel scores of the groups showed significant linear trends (all p for trend \ 0.001): Total AIS scores—non-insomnia group, 4.1 ± 2.9; group 1, 9.3 ± 3.8; group 2, 10.1 ± 3.9; group 3, 11.1 ± 4.1. Results from the tests to examine convergent and divergent validity are summarized in Table 2. (See the Supplementary Table 2 for the item-level analysis). The Kaiser–Meyer–Olkin statistic was 0.86, and Bartlett’s test of sphericity was significant (p \ 0.001). Factor analysis revealed a one-factor structure explaining 95.73 % of the variance (eigenvalue = 3.89). The factor loading of each item ranged from 0.51 to 0.82. The ROC curves for each insomnia group are shown in Fig. 1. The optimal cutoff scores were 6 for group 1, 8 for group 2, and 9 for group 3 (Table 3).
Discussion The current study examined the psychometric properties of the AIS in South Korean firefighters and rescue workers, using validated structured clinical interviews and self-report measures. The results suggest that the AIS is a reliable and valid measure to evaluate insomnia symptoms in this occupation group. Age was positively correlated with insomnia symptom severity, as reported in previous research [20]. This effect
Table 2 Convergent and divergent validity of the Athens Insomnia Scale Measure
r
p
Convergent validity PSQIa
0.82
\0.001
ISIb
0.85
\0.001
ESSb
0.29
\0.001
SF36 MCSc
-0.53
\0.001
SF36 PCSc
-0.37
\0.001
Divergent validity AUDIT-Cd Socioeconomic statuse
0.10
0.13
0.01
0.92
PSQI Pittsburgh Sleep Quality Index, ISI Insomnia Severity Index, ESS Epworth Sleepiness Scale, SF36 Short-Form 36-item Health Survey, MCS mental component summary, PCS physical component summary, AUDIT-C Alcohol Use Disorder Identification Test— Consumption a
n = 214,
b
n = 221,
c
n = 218,
d
n = 220, e n = 213
of age may be due to the cumulative effects of occupational stress including stress from shift work or traumatic experiences [21]. On the other side, the recent work demand did not show a significant association with insomnia symptoms in this sample. This finding is in line with a previous observation that work-related impact on sleep has been reported as a cumulative rather than short-term effect [22]. In the current study, Cronbach’s alpha was similar to that of the original scale and the mean item-total correlation was higher than that of the original scale [8], suggesting satisfactory internal consistency. The one-week test–retest reliability coefficients for the total score (0.94) and for each individual item (range 0.61–0.95) were higher than those of the original scale [8]. The AIS scores were significantly correlated with alternative measures evaluating sleep quality or insomnia
Table 1 Descriptive statistics, item-total correlation, and test–retest reliability of the Athens Insomnia Scale Item (range 0–3)a
Mean
Standard deviation
Item-total correlation
One-week test–retest reliability
1
‘‘Sleep induction’’
0.8
0.8
0.82
0.81
2
‘‘Awakenings during the night’’
0.8
0.7
0.76
0.82
3
‘‘Final awakening earlier than desired’’
0.5
0.7
0.61
0.95
4
‘‘Total sleep duration’’
1.0
0.8
0.75
0.61
5
‘‘Overall quality of sleep’’
0.9
0.7
0.84
0.78
6
‘‘Sense of well-being during the day’’
0.5
0.7
0.75
0.67
7
‘‘Functioning (physical and mental) during the day’’ 0.5
0.6
0.76
0.73
8
‘‘Sleepiness during the day’’
1.0
0.5
0.56
0.58
6.0
4.1
Total score (range 0–24) a
0.94
All items were adopted from the original scale in Soldatos et al. [8]
123
Qual Life Res Table 3 Diagnostic performance of the AIS for detecting insomnia Cutoff scorea
Sensitivity
Specificity
?LR
-LR
PPV
NPV
4
0.99
0.44
1.75
0.03
0.49
0.98
5
0.92
0.59
2.23
0.14
0.55
0.93
6
0.87
0.72
3.07
0.18
0.63
0.91
7
0.75
0.80
3.83
0.31
0.68
0.85
8
0.67
0.90
6.62
0.37
0.79
0.83
6
0.90
0.60
2.27
0.17
0.34
0.96
7
0.80
0.70
2.68
0.29
0.38
0.94
8
0.73
0.79
3.50
0.35
0.45
0.93
9 10
0.60 0.53
0.87 0.91
4.75 6.09
0.46 0.52
0.52 0.58
0.91 0.89
7
0.86
0.66
2.51
0.22
0.21
0.98
8
0.76
0.75
3.00
0.32
0.25
0.97
9
0.71
0.84
4.45
0.34
0.33
0.96
10
0.57
0.88
4.60
0.49
0.33
0.95
11
0.57
0.92
6.89
0.47
0.43
0.95
Group 1b
Group 2c
Group 3d
?LR positive likelihood ratio, -LR negative likelihood ratio, PPV positive predictive value, NPV negative predictive value a Optimal cutoff scores determined based on Youden’s index appear in bold b
Group 1 included the participants with insomnia symptoms (i.e., difficulty initiating sleep, difficulty maintaining sleep, non-restorative sleep, and early morning awakening) that had persisted for at least 1 month
c
Group 2 included the participants in group 1 whose daily functioning had been significantly impaired by insomnia symptoms
d
Group 3 included the participants in group 2 whose insomnia symptoms remained even when they could have enough time for sleep
Fig. 1 Receiver operating characteristic curves of the AIS. AUC Area under the receiver operating characteristics curve. a Group 1 included the participants with insomnia symptoms (i.e., difficulty initiating sleep, difficulty maintaining sleep, non-restorative sleep, and early morning awakening) that had persisted for at least 1 month (AUC = 0.87). b Group 2 included the participants in group 1 whose daily functioning had been significantly impaired by insomnia symptoms (AUC = 0.84). c Group 3 included the participants in group 2 whose insomnia symptoms remained even when they could have enough time for sleep (AUC = 0.85)
symptoms (PSQI, ISI, and ESS) and with questionnaires assessing overall health status (SF36). In accordance with the previous reports [7, 23], these results imply good
123
convergent validity. The AIS scores were neither associated with the AUDIT-C scores nor with socioeconomic status, supporting the divergent validity of the AIS. The positive linear trend of the AIS scores was found among the non-insomnia group and the insomnia groups as the symptom severity increased, indicating additional evidence of external validity. Factor analysis supports the homogeneity of the AIS, which replicates the findings from the study that validated the original scale [8] and the study by Go´mez-Benito et al. [24]. In a study by Soldatos et al. [9] in which the study sample was mainly composed of primary care clinical outpatients with a complaint of insomnia and psychiatric patients, the cutoff score to distinguish patients with clinically significant insomnia symptoms was suggested to be 6. This insomnia group can be considered comparable to group 2 in the current study, which was best distinguished with a higher cutoff score of 8. Considering that firefighters
Qual Life Res
and rescue workers generally experience frequent and prolonged work-related sleep interference, a relatively high cutoff score may have a better positive predictive value to discern those who are in need of clinical care. Although insomnia symptom severity was thoroughly assessed through face-to-face interviews and self-report questionnaires in the current study, objective sleep measures such as polysomnography and actigraphy would have provided important additional information. In conclusion, the current study suggests that the AIS is suitable for diagnosing insomnia in firefighters and rescue workers and thus may be considered as a valid and effective screening tool for insomnia symptoms. The usage of AIS could also be expanded into studies addressing sleep problems in other high-stress occupational groups (e.g., police officers and military personnel). Further studies exploring the different impacts on sleep according to types of work and intensity of workloads are needed. Acknowledgments This study was supported by the National Emergency Management Agency of South Korea. Conflict of interest interest.
The authors have no financial conflicts of
References 1. Vargas de Barros, V., Martins, L. F., Saitz, R., Bastos, R. R., & Ronzani, T. M. (2013). Mental health conditions, individual and job characteristics and sleep disturbances among firefighters. Journal of Health Psychology, 18(3), 350–358. doi:10.1177/ 1359105312443402. 2. Ohayon, M. M., Smolensky, M. H., & Roth, T. (2010). Consequences of shiftworking on sleep duration, sleepiness, and sleep attacks. Chronobiology International, 27(3), 575–589. doi:10. 3109/07420521003749956. 3. Beaton, R. D., & Murphy, S. A. (1993). Sources of occupational stress among firefighter/EMTs and firefighter/paramedics and correlations with job-related outcomes. Prehospital and Disaster Medicine, 8(2), 140–150. doi:10.1017/S1049023X00040218. 4. Bos, J., Mol, E., Visser, B., & Frings-Dresen, M. (2004). Risk of health complaints and disabilities among Dutch firefighters. International Archives of Occupational and Environmental Health, 77(6), 373–382. doi:10.1007/s00420-004-0537-y. 5. Patterson, P. D., Weaver, M. D., Frank, R. C., Warner, C. W., Martin-Gill, C., Guyette, F. X., et al. (2011). Association between poor sleep, fatigue, and safety outcomes in emergency medical services providers. Prehospital Emergency Care, 16(1), 86–97. doi:10.3109/10903127.2011.616261. 6. Saijo, Y., Ueno, T., & Hashimoto, Y. (2008). Twenty-four-hour shift work, depressive symptoms, and job dissatisfaction among Japanese firefighters. American Journal of Industrial Medicine, 51(5), 380–391. doi:10.1002/ajim.20571. 7. Okajima, I., Nakajima, S., Kobayashi, M., & Inoue, Y. (2013). Development and validation of the Japanese version of the Athens Insomnia Scale. Psychiatry and Clinical Neurosciences, 67(6), 420–425. doi:10.1111/pcn.12073.
8. Soldatos, C. R., Dikeos, D. G., & Paparrigopoulos, T. J. (2000). Athens Insomnia Scale: Validation of an instrument based on ICD-10 criteria. Journal of Psychosomatic Research, 48(6), 555–560. doi:10.1016/S0022-3999(00)00095-7. 9. Soldatos, C. R., Dikeos, D. G., & Paparrigopoulos, T. J. (2003). The diagnostic validity of the Athens Insomnia Scale. Journal of Psychosomatic Research, 55(3), 263–267. doi:10.1016/s00223999(02)00604-9. 10. World Health Organization. (1992). The ICD-10 classification of mental and behavioural disorders: Clinical descriptions and diagnostic guidelines. Geneva: World Health Organization. 11. Bastien, C. H., Vallieres, A., & Morin, C. M. (2001). Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Medicine, 2(4), 297–307. doi:10.1016/ S1389-9457(00)00065-4. 12. Buysse, D. J., Reynolds, C. F, 3rd, Monk, T. H., Berman, S. R., & Kupfer, D. J. (1989). The Pittsburgh Sleep Quality Index: A new instrument for psychiatric practice and research. Psychiatry Research, 28(2), 193–213. doi:10.1016/0165-1781(89)90047-4. 13. Johns, M. W. (1991). A new method for measuring daytime sleepiness: The Epworth sleepiness scale. Sleep, 14(6), 540–545. 14. Ware, J. E., Kosinski, M., & Dewey, J. E. (2000). How to score version 2 of the SF-36Ò health survey. Lincoln, RI: QualityMetric Incorporated. 15. Bush, K., Kivlahan, D. R., McDonell, M. B., Fihn, S. D., Bradley, K. A., & Project, A. C. Q. I. (1998). The AUDIT alcohol consumption questions (AUDIT-C)—An effective brief screening test for problem drinking. Archives of Internal Medicine, 158(16), 1789–1795. doi:10.1001/archinte.158.16.1789. 16. Hollingshead, A. B. (1975). Four factor index of social status. Unpublished manuscript. New Haven: Yale University. 17. First, M., Gibbon, M., Spitzer, R. L., & Williams, J. (1996). User’s guide for the structured clinical interview for DSM-IV axis I disorders—research version. New York: Biometrics Research. 18. American Psychiatric Association. (2000). Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR). Wasington, DC: American Psychiatric Association. 19. Edinger, J. D., Bonnet, M. H., Bootzin, R. R., Doghramji, K., Dorsey, C. M., Espie, C. A., et al. (2004). Derivation of research diagnostic criteria for insomnia: Report of an American Academy of Sleep Medicine Work Group. Sleep, 27(8), 1567–1596. 20. Akerstedt, T., Knutsson, A., Westerholm, P., Theorell, T., Alfredsson, L., & Kecklund, G. (2002). Sleep disturbances, work stress and work hours: A cross-sectional study. Journal of Psychosomatic Research, 53(3), 741–748. doi:10.1016/S00223999(02)00333-1. 21. Turner, R. J., & Lloyd, D. A. (1995). Lifetime traumas and mental health: The significance of cumulative adversity. Journal of Health and Social Behavior, 36(4), 360–376. doi:10.2307/ 2137325. 22. De Lange, A. H., Kompier, M. A., Taris, T. W., Geurts, S. A., Beckers, D. G., Houtman, I. L., et al. (2009). A hard day’s night: A longitudinal study on the relationships among job demands and job control, sleep quality and fatigue. Journal of Sleep Research, 18(3), 374–383. 23. Sun, J. L., Chiou, J. F., & Lin, C. C. (2011). Validation of the Taiwanese version of the Athens Insomnia Scale and assessment of insomnia in Taiwanese cancer patients. Journal of Pain and Symptom Management, 41(5), 904–914. doi:10.1016/j.jpainsym man.2010.07.021. 24. Gomez-Benito, J., Ruiz, C., & Guilera, G. (2011). A Spanish version of the Athens Insomnia Scale. Quality of Life Research, 20(6), 931–937. doi:10.1007/s11136-010-9827-x.
123
