Available online at www.sciencedirect.com
ScienceDirect Comprehensive Psychiatry xx (2015) xxx – xxx www.elsevier.com/locate/comppsych
Psychometric properties of the self-reporting questionnaire (SRQ-20): Measurement invariance across women from Brazilian community settings Felipe Paraventi⁎, Hugo Cogo-Moreira, Cristiane Silvestre Paula, Jair de Jesus Mari Department of Psychiatry, Federal University of São Paulo, São Paulo, Brazil
Abstract Background: SRQ-20 is a validated screening tool for common psychiatric disorders in several countries. Exploration of the latent structure of this instrument resulted in conflicting evidence. This study aimed to explore the latent structure of SRQ-20 among Brazilian women from community settings. We also tested the model invariance across different sociodemographic conditions. Methods: Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were conducted on a sample of 1668 women from four different geographical regions of Brazil. Invariance of the model was tested through multi-group CFA according to sociodemographic variables. Results: EFA has shown two potential solutions with two and three factors. CFA resulted in indices of the two-factor solution slightly worse than the three-factor solution. Invariance testing has shown this model was not invariant across cities, but was invariant across different social classes. The structure was also invariant for the two lower educated groups. The respecified model (i.e., excluding item 16) was not invariant across groups with different educational levels. Conclusion: The three-factor solution seems to be the most suitable model of SRQ-20 for Brazilian women in community settings. Furthermore, sociodemographic variables seem to reflect on the latent structure of this instrument. Validation of screening tools should consider sociodemographic variables. © 2014 Elsevier Inc. All rights reserved.
1. Introduction Mental and behavioral disorders are the largest cause of Years Lost Due to Disability (YLDs) globally [1], more than half of this burden is attributable to depressive and anxiety disorders [2]. In spite of that, the treatment gap for psychiatric disorders is a worldwide phenomenon and several barriers to the progress of mental health care are still present, especially in low-income and middle-income countries [3,4]. Lack of awareness of psychiatric conditions is a critical step for the treatment, thus screening instruments might contribute to this context [5,6]. Patel et al. [5] suggested that increasing recognition of depression might be achieved through community- and practice-based screening by general practitioners as well as trained interviewers. Mateus et al. [7] investigated the panorama of the mental health system of Brazil; these researchers recommended, for
⁎ Corresponding author. E-mail address: [email protected] (F. Paraventi). http://dx.doi.org/10.1016/j.comppsych.2014.11.020 0010-440X/© 2014 Elsevier Inc. All rights reserved.
future mental health policies in this country, a larger focus on community and primary care. The Self-Reporting Questionnaire (SRQ) seems to be a suitable tool for this purpose. The SRQ was developed by WHO as an instrument to screen for common mental disorders in primary care— especially in developing countries [8,9]. Originally, the SRQ consisted of 25 questions to be answered ‘yes’ or ‘no’; 20 questions related to neurotic symptoms, 4 items representative of psychotic symptoms and one about convulsive episodes. For several reasons, attention has been concentrated only on “neurotic items”, expressed as SRQ-20 [10]. SRQ-20 is a brief and simple screening tool that covers important areas of psychopathology. This tool is easily trainable and may be applied as a ‘self-administered’ or ‘interviewer administered’ instrument, the latter being very relevant in settings of high rates of illiteracy [10]. SRQ-20 has been validated in several cultures and settings as a cost-effective manner to screen for psychiatric disorders [10–15]; the Brazilian version is also validated [16–18]. Though several articles discuss distinct aspects of validity (e.g. face validity, content validity, criterion validity, etc.)
2
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
there is conflicting evidence concerning the structure of this questionnaire. Tafari et al. [19] found four-factors of SRQ-24, being the psychotic scale a single factor clearly distinguished from the neurotic section (i.e. SRQ-20), in which three different factors were derived (I—“Cognitive”, II—“Anxiety and Depression” and III—“Somatic Symptoms”). Sen [20] derived seven distinct factors, while Iacoponi and Mari [21] suggested four factors for the SRQ-20 construct (I—“Decreased energy”, II—“Somatic symptoms”, III—“Depressive mood” and IV—“Depressive thoughts”). At the time, the hypothesis raised by WHO was that Sen' and Iacoponi' studies were developed in a primary care setting, while Tafari et al. used a community sample [10]. Chen and collaborators [11] proposed a three-factor model for this instrument in a community population in China, describing ‘depressive symptoms’, ‘anxiety symptoms’ and ‘somatic symptoms’ as clustered components. In a Vietnamese sample, a three-factor model was also found, but this study raised evidences for a bi-factor model as well (a single general factor, i.e. “general distress”, and additional sub-domain or group factors labeled: “negative affect”, “somatic complaints”, and “hopelessness”) [22]. These findings highlight a poor consistency of the latent structure of SRQ-20 across studies; hence, it is not reliable to use SRQ factors as subscales at this moment in time. Invariance across groups is another relevant point, which has not been well explored yet. In other words, whether the questionnaire structure is the same for different populations is not known. In terms of cut-off points, there is a large variation across settings; sometimes the optimum cut-off point is significantly higher, for instance it was 10/11 in Afghanistan [23] and 11/12 in India [20], while in other settings it was considerably lower, for instance 6/7 in Zambia [24]. Previous studies demonstrated that gender and educational level could have an important influence on misclassification by SRQ-20. For instance, results have shown a larger tendency of males to be classified as “false negatives”, while poorly educated people were more likely to be classified as “false positives” [14,23,25,26]. A recent study pointed out the importance of considering gender in validation of screening instruments as well as the benefits of locally-developed measures of mental health [27]. Pendergast and colleagues [28] explored the cross-cultural invariance and invariance over time of SRQ-20 on a specific population of postpartum women; these authors suggested that removing four items of SRQ-20 (the somatic symptoms) would lead to a more suitable structure across those women. They also investigated the invariance of SRQ-20 over eight different sites; responses patterns from Brazil were clearly different from the other seven sites. Validity of SRQ-20 over a specific population (i.e., postpartum women from Northern region of Brazil) was questioned by the cited study. Given the conflicting evidence regarding the latent structure of SRQ-20 and the underexplored invariance across populations, this study aimed to (a) explore and confirm the latent structure of SRQ-20 among Brazilian
women in community settings, and (b) test the invariance of the best-confirmed model through different sociodemographic conditions (i.e. site, social economical status and educational level).
2. Methods 2.1. Procedures and instruments This analysis is part of a large cross-sectional study— designed to explore the mental health service use among Brazilian children and adolescents. The research team randomly approached 2000 families in order to interview and apply standardized instruments to main caregivers, which included: SRQ-20, family's socioeconomic status and sociodemographic factors (i.e. area of residence, age and educational level) [29]. Family's socioeconomic status was assessed by a questionnaire developed by the Brazilian Association of Research Companies according to family purchasing power; this tool is highly used in Brazil and categorizes five social strata, from A (highest purchase power) to E (lowest purchase power) [30]. SRQ-20 questionnaires (previously described) were administered to all mothers or main caregivers only by the interviewer, thus the methods of application (‘self-administered’ and ‘interviewer administered’) were not mixed according to WHO's recommendations [10]. SRQ-20 items are scored ‘0’ if answered ‘no’ (symptom was absent during past month) and ‘1’ if answered ‘yes’ (symptom was present during past month), consequently the maximum score is 20. A cut-off value of 7/8 (‘7’—probable non-case; ‘8’—probable case) is commonly used in Brazil; Mari and Williams [17] proposed a different cut-off for each sex (5/6—male and 7/8—female). Prior to data collection, a team of mental health experts trained and supervised the selected psychologists to conduct the questionnaires application. Participants were individually assessed in private rooms at buildings rented for this particular survey. Data collection took 15 months to be fully completed by December 2011. Written informed consent from caregivers was obtained. The Research Ethics Committee of University of São Paulo approved this study under the process number 0301/09. Local governmental agencies were contacted and agreed to support this research as well. 2.2. Setting Brazil is the fifth largest country in the world with 195 million inhabitants spread over five heterogeneous geographical areas. The Southeast and South regions of Brazil are the two most developed and wealthy [31]. This multicenter research included four towns, from four out of the five Brazilian geographical regions (Caeté—Southeast; Goianira—Central-West; Itaitinga—Northeast; Rio Preto da Eva—North). The selection criteria of the four sites were: (1) being near to a State Capital, (2) having a Human Development Index (HDI) similar to the Brazilian average;
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
and (3) having approximately 30,000 inhabitants, since 84.7% of Brazilian municipalities have less than 50,000 inhabitants [32].
2.3. Sample This study included all the 1668 female caregivers who represented four different geographical areas of Brazil—402 women (24.10%) from Itatinga (Northeast region), 446 women (26.74%) from Goianira (Central-West region), 454 women (27.22%) from Caete (Southeast region) and 366 women (21.94%) from Rio Preto da Eva (North region).
2.4. Statistical analysis Initially, we conducted an exploratory factor analysis (EFA); one to five factors were extracted considering a random sampling of 40% of the original data set. We used the robust weighted least squares estimator and oblique rotation (geomin, default in Mplus), allowing the factors to be correlated. The loadings were considered prominent when larger than 0.32 [33]. Statistical parameters (i.e. model fit indices, magnitude and significance of factor loading across the number of the factor solutions) and interpretability of the factors were considered to achieve the best solution. Based on that solution, we conducted a confirmatory factor analysis on the other 60% of the random sample. Both EFA and CFA model fit indices were evaluated considering the following cut-offs: Root Mean Square Error of Approximation (RMSEA) less or equal to 0.06, Tucker–Lewis Index (TLI) and Comparative Fit Index (CFI) close to 0.95 [34]. Lastly, we executed a multi-group CFA of the confirmed model using 60% of the random sample (i.e. invariance testing). This analysis was included to ensure that theoretical constructs of SRQ-20's items are measured equally across different groups. Two levels of invariance were tested: configural invariance (also called “pattern invariance”) and scalar invariance (also “strong invariance”). Configural invariance must be established in order that subsequent tests can be meaningful. Configural invariance implies that the items comprising the instrument should exhibit the same pattern across groups, but it does not indicate that people respond to those items in the same way. On the other hand, the presence of scalar invariance implies that the instrument's measurement capacity is the same across groups, allowing the comparison of scores between them [35,36]. The scalar against configural invariance testing – through Χ 2 – evaluates if the constraints from more restrictive models [scalar] worsen the fit. Invariance testing considered the four cities (Itatinga, Goianira, Caete and Rio Preto da Eva), years of schooling and SES. We used a one-way ANOVAS (with Bonferroni's correction) to explore the sociodemographic characteristics of each city. All analyses were conducted in Mplus 7.11 [37].
3
3. Results 3.1. Sociodemographic indicators Sociodemographic characteristics of the four cities are presented in Table 1. The highest average indices of age, SES and educational level were seen in Caeté (Southeast region): 35.13 [SD = 6.58], 17.58 [SD = 5.37] and 8.26 [SD = 3.97] respectively. On the contrary, the lowest indices were observed in Itatinga (Northeast region): 32.25 [SD = 6.80], 12.78 [SD = 4.08] and 5.06 [SD = 3.67] respectively. After a multiple group comparison (with Bonferroni correction), we found that statistical differences were explained as follows: (a) the four cities are statistically different regarding SES (Caeté N Goianira N Rio Preto da Eva N Itaitinga); (b) the cities had statistical difference in terms of schooling, with exception of Rio Preto da Eva when compared to Goianira (p = 0.999); and (c) the cities were comparable in regard to age, with exception of Goianira [Goianira vs Caeté (p b 0.001), Goianira vs Itatinga (p b 0.001), and Goianira vs Rio Preto da Eva (p = 0.011)]. 3.2. EFA Table 2 shows EFA solutions from one to five geomin rotated factor loadings (model fit indices and correlations between each respective latent factor may be seen at the bottom of the table). The one-factor solution was the only model to not show good fit indices (CFI/TLI lower than 0.95). Nonetheless, the solutions with four and five factors had a number of cross loadings in six items (5, 6, 11, 13, 15, 19) and eight items (3, 4, 5, 6, 11, 13, 15, 16, 20) respectively. Moreover, the interpretability of such factors is not plausible as they are configured, thus both solutions were invalidated conceptually. The two-factor and three-factor solutions were considered plausible. Observing the two-factor solution, item 8 “Do you have trouble to think clearly?” and item 18 “Do you feel tired all the time?” showed cross loadings. For the three-factor solution, item 3 “Do you sleep badly?” did not reach a loading higher than 0.32 in any of the factors, and four items (15, 16, 17 and 20) showed cross loadings—i.e. commonality of different factors which cannot be accounted for a single factor correlation [38]. Table 1 Mean and standard deviation [SD] of age, SES and years of schooling (YoS) by region. Age (mean [SD]) SES a (mean [SD]) YoS (mean [SD]) Caeté Goianira Itaitinga Rio Preto da Eva F p-Value
35.13 [6.58] 32.87 [6.00] 32.25 [6.80] 34.40 [7.11] 11.31 b0.001
17.58 [5.37] 15.63 [4.64] 12.78 [4.08] 13.93 [5.19] 76,858 b0.001
8.26 [3.97] 7.04 [3.44] 5,06 [3,67] 6.88 [3.9] 48.47 b0.001
a In the multigroup analysis, SES was categorized as follows: 35–46 = A; 23–34 = B; 14–22 = C; 08–13 = D; 00–07 = E.
4
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
Table 2 EFA with one to five factor solutions, correlation of factors and model fit indices. Solution Item
One factor
Two Factors 1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Correlation
1 2 3 4 5
0.479 0.519 0.525 0.522 0.532 0.761 0.619 0.669 0.813 0.686 0.684 0.536 0.606 0.564 0.722 0.760 0.627 0.794 0.705 0.661 1
2
Three Factors 1
2
0.346 0.381 0.331
Four Factors
3
1
4
0.378 0.307
0.440 0.466 0.751 0.422 0.329 0.979 0.913 0.531 0.500 0.468 0.479 0.629 0.801 0.633 0.464 0.434 0.847 0.538 1 2 0.597 1
3
1
2
3
0.304
0.317 0.404 0.329
4
5
0.377 0.351
0.560
2
Five factors
0.492 0.490 0.400
0.337
0.366 0.326
0.763
0.490 0.508
0.605
0.578
0.748 0.902
0.566
0.932 0.901
0.626 0.383 0.741 0.554 0.341 0.413 0.343 0.687 0.377 0.367 0.681 0.604 2 1 0.565
0.337
0.632
0.407
1 0.552 0.446
0.370 0.482 0.426 0.314
0.340
0.847 0.354 3 1
0.931 0.906 0.424 0.672 0.377 0.663 0.331 0.745 0.447
0.321 0.450 0.372 0.341
0.669 1 0.623 0.373 0.508
0.778 0.401 0.634 4
2 3 1,000 0.350 1,000 1 0.514 0.304 1,000
0.393 0.644 0.318 0.610 0.309 0.688 0.426
0.470 0.507
0.725
1 0.292 0.263 0.349 0.075
2 1,000 0.550 0.388 0.560
3
0.943 0.394 4
0.518 5
1,000 0.444 1,000 0.463 0.319 1,000
Model Fit Indices Χ 2 (df), p-value RMSEA (90% IC) CFI TLI
465.162 (170), b0.001 0.052 (0.046–0.057) 0.943 0.936
313.601 (151), b0.001 0.041 (0.034–0.047) 0.969 0.960
Fig. 1 illustrates the result of scree-plot test; we used this tool to provide more information regarding the number of factors to be extracted. According to scree-plot criterion [39], the two- or three-factor solution would be admissible, hence both solutions were considered for CFA.
232.882 (133), b0.001 0.034 (0.027–0.041) 0.981 0.972
152.250 (116), 0.013 0.022 (0.011–0.031) 0.993 0.989
114.610 (100), 0.15 0.015 (0.000–0.026) 0.997 0.995
the three derived factors of this model might be clinically labeled as: Factor 1—“Anxiety/Depression”, Factor 2—“Disability” and Factor 3—“Somatic symptoms”. Table 4 illustrates this configuration. 3.4. Invariance testing (multi-group CFA)
3.3. CFA The two- and three-factor solutions were modeled in a CFA keeping all the 20-items exactly as obtained from EFA, which in the case of the three-factor solution had five items with cross loadings. Indeed, an additional CFA was conducted testing a three-factor solution without the item 3, because this item had a loading lower than 0.32 in all the factors. Model fit indices for these CFAs are presented in the Table 3. Analysis of the results of CFA solutions indicates the restriction worsens the model fit indices (Χ 2(5) = 100.810, p b 0.001). Consequently, we decided to follow through invariance testing (multi-group CFA) considering the three-factor solution. In our point of view,
3.4.1. City invariance Individually, the baseline model's fit indices were good for each city as follows—(a) Rio Preto da Eva (Χ 2(162) = 237.479, p b 0.001), RMSEA = 0.051, CFI = 0.923, TLI = 0.91, and WRMR = 1.0; (b) Caete (Χ 2(162) = 230.030, p b 0.001), RMSEA = 0.042, CFI = 0.958, TLI = 0.951, WRMR = 0.952; (c) Goianira (Χ 2 (162) = 235.802, p b 0.001), RMSEA = 0.042, CFI = 0.975, TLI = 971, WRMR =0.915, and (d) Taitinga (Χ 2(162) = 210.544, p = 0.006), RMSEA = 0.037, CFI = 0.971, TLI = 0.966, WRMR = 0.860. Once the baseline models were determined and showed good fit indices, they were integrated into a multi-group CFA model. Considering the cities all together, scalar invariance versus
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
5
Fig. 1. Scree-plot test
configural invariance was statistically significant (Χ 2(57) = 93.737, p = 0.0016). Therefore, the observed non-invariance indicates that scores may not be comparable across the four cities. 3.4.2. SES invariance Highest strata (A, B, and C) and lowest strata (D and E) were merged in order to obtain a reasonable sample size to conduct the invariance testing. The upper class (n = 527) and the lower class (n = 374) showed good fit indices separately. The configural model indices were: (Χ 2(324) = 617.206, p b 0.001), RMSEA = 0.045, CFI = 0.958, and TLI = 0.951, while the scalar model indices were: (Χ 2(343) = 627.280, p b 0.001), RMSEA = 0.043, CFI = 0.959, and TLI = 0.955. The scalar against configural model was not statistically significant (Χ 2(19) = 21.065, p = 0.3332), thus factor loadings and thresholds across different social classes are invariant. Consequently, scores are comparable between these groups. 3.4.3. Schooling invariance Based on percentiles of ages of schooling, three cut-offs were considered for schooling invariance testing: less than or equal to four years of schooling (25th percentile; n = 302), more than four and less than ten years of schooling (n = 348), and more than or equal to ten years of schooling (75th percentile, n = 251). Women with less than or equal to four years of schooling returned good fit indices: (Χ 2(162) = 280.458, p b 0.001), RMSEA = 0.049 (90% IC = 0.039– 0.059), CFI = 0.957, TLI = 0.950. Taking the population with ‘more than four and less than ten years of schooling’ into account, the tested model returned good indices: (Χ 2 (162) = 248.678, p b 0.001), RMSEA = 0.039 (95% IC = 0.029–0.049), CFI = 0.963, TLI = 0.957, and WRMR = 0.967. Lastly, women with ‘more than or equal to ten years of schooling’ showed a perfect correlation between items in two different situations, as follows: (a) item 6 (“Do you feel
nervous, tense or worried?”) and item 16 (“Do you feel that you are a worthless person?”); and (b) item 16 and item 12 (“Do you find it difficult to make decisions”?). Both situations are considered inadmissible, so each pair should have one variable excluded. This finding means that the baseline model of this group is not comparable to the others; therefore, a respecified model needed to be achieved. In order to preserve the maximum of the instrument's original items, we took the decision to exclude the item 16—which is present in both pairs. The respecified model showed good statistical fit after exclusion of the item 16 as follows: (Χ 2 (145) = 189.380, p = 0.008), RMSEA = 0.039 (90% IC = 0.021–0.053), CFI = 0.952, TLI = 0.943, and WRMR = 0.918. Comparing the two lower groups under the original structure (with 20 items), configural and scalar invariance returned good fit indices. The scalar against configural invariance was not statistically significant (Χ 2(19) = 16.488, p = 0.6245) hence this model is invariant across women with less than ten years of schooling. We also tested the invariance across the three groups under the respecified model (i.e., without item 16). In this case, configural and scalar invariance returned good fit indices, but the scalar against configural testing was statistically significant (Χ 2(36) = 52.007, p = 0.0410). In reason of that, the respecified model is not invariant across those groups.
4. Discussion 4.1. Factorial structure and invariance This study explored the underlying structure of SRQ-20 through EFA and CFA. Results suggested the three-factor solution seems to be the most suitable model for a female community sample in Brazil. From our point of view, the factors “Anxiety/Depression”, “Disability” and “Somatic symptoms” represented the latent structure of SRQ-20 in this
6
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
Table 3 CFA of two- and three-factor solutions. Model Fit index
Two-factor
Three-factor (all items)
Three-factor (without item3)
Χ 2 (df), p-value RMSEA (90% IC) CFI TLI
556.965 (167), p b 0.001 0.051 (0.046–0.056) 0.946 0.939
438.606 (162), p b 0.001 0.044 (0.039–0.048) 0.962 0.955
413.498 (145), p b 0.001 0.045 (0.040–0.050) 0.961 0.954
population. Further investigation of this model through multi-group CFA (invariance testing) has shown that this model is only invariant across social classes; when schooling and cities' location were considered the expected invariance was not observed. As stated by van de Schoot and colleagues [40], when measurement invariance is not achieved, groups or subjects respond differently to the items and, as consequence, factor scores may not be compared. In the case of geographical regions, the pattern of factors is equivalent, but those items and factors are not measured in the same scale across the four cities. In other words, women responded to the instrument in different ways across the cities, so direct comparison of scores is not possible when comparing women from one city to the other. With regard to schooling, the baseline model does not fit the group with highest educational level, which demanded a respecified model (e.g. without item 16). Therefore, in this situation, the comparison of scores between groups is not possible because the models are not equivalent between them. The structure was not invariant even considering the respecified model. Reasons why the proposed model had a variance across geographical regions of Brazil remain unclear. Differences regarding sociodemographic indicators could partially justify this issue. For example, cities were different in terms of schooling, and – as seen in the Results section –the structure was not invariant when this variable was considered. Several studies have raised questions about cultural impact on how emotions, thoughts and behaviors are noticed and manifested [41–43]. Draguns and Tanaka-Matsumi [41], explored the cultural influence in psychopathology highlighting the variation of psycho-emotional manifestations across and within cultures. Other studies showed the benefit of assessing some items within cultural contexts in order to improve the instrument's validity [44]. Youngmann and colleagues [45] demonstrated that the validity of the culturally adapted SRQ was superior to the original structure. As long as Brazil is marked by a very heterogeneous cultural pattern across the territory, we hypothesize that this massive difference might also be an explanation for the non-observed invariances. Such an unexplored area represents a potential field for future research in Brazil. Our results support a three-factor structure of SRQ-20 among women in community settings. These findings resemble solutions proposed by other authors, as described
in the Introduction section (Tafari et al. [19], Chen et al. [11], Stratton et al. [22]). Moreover, in a recent work, Rasmussen and colleagues [27] found a latent structure of SRQ-20 that has some resemblance to our findings (“Negative affect”, “Emotional Numbing” and “Somatic complaints”). When researchers compared SRQ-20 to a culturally grounded tool, they found out that SRQ-20 was a worse measure of distress for women, while both instruments were similar for men. Those authors once more highlighted that validity of screening instruments might be gender-differentiated. Our study design takes this proposition into account; furthermore, our results suggested that not only gender and setting should be considered when validating screening tools, but also, wider characteristics of each population. For instance, Pendergast and collaborators [28] have studied the SRQ-20 structure among postpartum women; their findings suggested the four somatic items – i.e., headaches, stomach-
Table 4 The three-factor solution of SRQ-20. SRQ-20 item
Factor I Factor II Factor III “Anxiety“Disability” “Somatic Depression” Symptoms”
2. Is your appetite poor? 6. Do you feel nervous, tense or worried? 9. Do you feel unhappy? 10. Do you cry more than usual? 15. Have you lost interest in things? 16. Do you feel that you are a worthless person? 17. Has the thought of ending your life been on your mind? 8. Do you have trouble thinking clearly? 11. Do you find it difficult to enjoy your daily activities? 12. Do you find it difficult to make decisions? 13. Is your daily work suffering? 14. Are you unable to play a useful part in life? 18. Do you feel tired all the time? 20. Are you easily tired? 1. Do you often have headaches? 4. Are you easily frightened? 5. Do your hands shake? 7. Is your digestion poor? 19. Do you have uncomfortable feelings in your stomach? 3. Do you sleep badly?
0.351 0.400
– –
– –
0.748 0.902
– –
– –
0.341
0.413
–
0.343
0.687
–
0.377
0.367
–
–
0.407
–
–
0.626
–
–
0.383
–
–
0.741
–
–
0.554
–
–
0.681
–
– –
0.604 –
0.354 0.377
– – – –
– – – –
0.492 0.490 0.763 0.847
–
–
–
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
aches, digestive difficulties and sleep difficulties – were not representative of internalizing symptoms for this sample. In other words, such somatic complaints may be typical of women recovering from childbirth, not reflecting mental distress. Consequently, when screening mental illness among postpartum women, use of SRQ-20 in its original form might not be as appropriated as the proposed one-factor model, which excludes the somatic items (i.e. SRQ-16) [28]. These findings go along with our hypothesis that screening tools validation should take more specific characteristics of each population into consideration. That is, probably many factors (e.g. sociodemographics characteristics as proposed by this work) or even different groups may influence the understanding of mental illness' construct; consequently, they may respond unequally to the instrument's items. In conclusion, overall or factor scores might not be comparable across distinct populations. 4.2. Limitations and strengths Some limitations may apply to this study. First, our sample is restricted to a female population from Brazilian community settings. Moreover, participants are considered “mothers” or “main caregivers” of children and adolescents, once this investigation was part of a large cross-sectional study. We have only assessed invariance considering site, educational level and SES, so many other social and cultural parameters might have been explored. Religion seems to be a particular example to highlight, especially because it is very heterogeneous in Brazil. Limitations notwithstanding, this work suggests a threefactor structure of SRQ-20 in a community population of Brazilian women and highlights that SRQ-20 has a variable measurement capacity of psycho-emotional symptoms across different sociodemographic conditions. Results seem particularly important to Brazil, where a noticeable diversity of cultural or social factors is observed. A better quality of screening in this country might be helpful in reducing its treatment gap. On the other hand, our findings also reflect some limitations on practical use of SRQ-20. As argued by Kagee and colleagues [6], integration of routine screening into primary care has some concerns. Our results indicate that the underlying construct of SRQ-20 may not be comparable between some populations, it possibly affects the instrument's reliability, and consequently, the integration of this tool into primary care practice would not represent a reasonable strategy. When compared to other measures (e.g. clinical interview), the possible misclassification associated to SRQ-20 could raise the treatment gap on false-negative individuals. On the contrary, detection of false-positive individuals (i.e. those who do not need treatment) would raise the assistance needs within a mental health system that is already overloaded [7,25]. From our point of view, future validation of screening questionnaires should consider sociodemographic variables in order to improve the screening quality.
7
5. Conclusion The three-factor model of SRQ-20 seems to be suitable for a community population of Brazilian women. In spite of that, our results suggested that the measurement capacity and understanding of the mental health construct might vary across the country and some social conditions. At this moment in time, benefits of screening mental disorders in Brazil through SRQ-20 should be weighted and further explored. Future validation of screening instruments with a focus on the sociodemographic characteristics of specific populations seems to be plausible and recommended. Acknowledgment The National Research Council (CNPq) funded this project by a grant from “INCT—Psiquiatria do Desenvolvimento para Crianças e Adolescentes” (No. 573974/2008-0). H.C.M. is a post-doc researcher from Coordination for the Improvement of Higher Education Personnel (PNPD/ CAPES). C.S.P. was partially supported by CNPq. J.J.M. is a senior researcher from CNPq. We are very grateful to the reviewers for the detailed work they conducted. References [1] Vos T, Flaxman AD, Naghavi M, Lozano R, Michaud C, Ezzati M, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012;380:2163-96. [2] Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 2013;382:1575-86. [3] Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bull World Health Organ 2004;82:858-66. [4] Saraceno B, van Ommeren M, Batniji R, Cohen A, Gureje O, Mahoney J, et al. Barriers to improvement of mental health services in lowincome and middle-income countries. Lancet 2007;370:1164-74. [5] Patel V, Simon G, Chowdhary N, Kaaya S, Araya R. Packages of care for depression in low- and middle-income countries. PLoS Med 2009;6:e1000159. [6] Kagee A, Tsai AC, Lund C, Tomlinson M. Screening for common mental disorders in low resource settings: reasons for caution and a way forward. Int Health 2013;5:11-4. [7] Mateus MD, Mari JJ, Delgado PG, Almeida-Filho N, Barrett T, Gerolin J, et al. The mental health system in Brazil: policies and future challenges. Int J Ment Heal Syst 2008;2:12, http://dx.doi.org/10.1186/1752-4458-2-12. [8] Harding TW, Climent CE, Diop M, Giel R, Ibrahim HH, Murthy RS, et al. The WHO collaborative study on strategies for extending mental health care, II: the development of new research methods. Am J Psychiatry 1983;140:1474-80. [9] Harding TW, de Arango MV, Baltazar J, Climent CE, Ibrahim HH, Ladrido-Ignacio L, et al. Mental disorders in primary health care: a study of their frequency and diagnosis in four developing countries. Psychol Med 1980;10:231-41. [10] WHO. A User's Guide to the Self-Reporting Questionnaire (SRQ). Geneva: World Health Organization; 1994. [11] Chen S, Zhao G, Li L, Wang Y, Chiu H, Caine E. Psychometric properties of the Chinese version of the Self-Reporting Questionnaire 20 (SRQ-20) in community settings. Int J Soc Psychiatry 2009;55:538-47.
8
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
[12] Scholte WF, Verduin F, van Lammeren A, Rutayisire T, Kamperman AM. Psychometric properties and longitudinal validation of the self-reporting questionnaire (SRQ-20) in a Rwandan community setting: a validation study. BMC Med Res Methodol 2011;11:116, http://dx.doi.org/10.1186/1471-2288-11-116. [13] Giang KB, Allebeck P, Kullgren G, Tuan NV. The Vietnamese version of the Self Reporting Questionnaire 20 (SRQ-20) in detecting mental disorders in rural Vietnam: a validation study. Int J Soc Psychiatry 2006;52:175-84. [14] Araya R, Wynn R, Lewis G. Comparison of two self administered psychiatric questionnaires (GHQ-12 and SRQ-20) in primary care in Chile. Soc Psychiatry Psychiatr Epidemiol 1992;27:168-73. [15] Harpham T, Reichenheim M, Oser R, Thomas E, Hamid N, Jaswal S, et al. Measuring mental health in a cost-effective manner. Health Policy Plan 2003;18:344-9. [16] Mari JJ, Williams P. A comparison of the validity of two psychiatric screening questionnaires (GHQ-12 and SRQ-20) in Brazil, using Relative Operating Characteristic (ROC) analysis. Psychol Med 1985;15:651-9. [17] Mari JJ, Williams P. A validity study of a psychiatric screening questionnaire (SRQ-20) in primary care in the city of Sao Paulo. Br J Psychiatry 1986;148:23-6. [18] Scazufca M, Menezes PR, Vallada H, Araya R. Validity of the self reporting questionnaire-20 in epidemiological studies with older adults: results from the Sao Paulo Ageing & Health Study. Soc Psychiatry Psychiatr Epidemiol 2009;44:247-54. [19] Tafari S, Aboud FE, Larson CP. Determinants of mental illness in a rural Ethiopian adult population. Soc Sci Med 1991;32:197-201. [20] Sen B. Psychiatric phenomena in primary health care their extent and nature. Indian J Psychiatry 1987;29:33-40. [21] Iacoponi E, Mari JJ. Reliability and factor structure of the Portuguese version of Self-Reporting Questionnaire. Int J Soc Psychiatry 1989;35: 213-22. [22] Stratton KJ, Aggen SH, Richardson LK, Acierno R, Kilpatrick DG, Gaboury MT, et al. Evaluation of the psychometric properties of the Self-Reporting Questionnaire (SRQ-20) in a sample of Vietnamese adults. Compr Psychiatry 2013;54:398-405. [23] Ventevogel P, De Vries G, Scholte WF, Shinwari NR, Faiz H, Nassery R, et al. Properties of the Hopkins Symptom Checklist-25 (HSCL-25) and the Self-Reporting Questionnaire (SRQ-20) as screening instruments used in primary care in Afghanistan. Soc Psychiatry Psychiatr Epidemiol 2007;42:328-35. [24] Aidoo M. Explanations for the causes of mental ill-health among lowincome women in an urban area: the case of Zambia. PhD thesis London: South Bank University; 1998. [25] Mari JJ, Williams P. Misclassification by psychiatric screening questionnaires. J Chronic Dis 1986;39:371-8. [26] Ludermir AB, Lewis G. Investigating the effect of demographic and socioeconomic variables on misclassification by the SRQ-20 compared with a psychiatric interview. Soc Psychiatry Psychiatr Epidemiol 2005;40:36-41. [27] Rasmussen A, Ventevogel P, Sancilio A, Eggerman M, Panter-Brick C. Comparing the validity of the self reporting questionnaire and the Afghan symptom checklist: dysphoria, aggression, and gender in transcultural assessment of mental health. BMC Psychiatry 2014;14:206, http:// dx.doi.org/10.1186/1471-244X-14-206.
[28] Pendergast LL, Scharf RJ, Rasmussen ZA, Seidman JC, Schaefer BA, Svensen E, et al. Postpartum depressive symptoms across time and place: structural invariance of the Self-Reporting Questionnaire among women from the international, multi-site MAL-ED study. J Affect Disord 2014;167:178-86. [29] Paula CS, Bordin IAS, Mari JJ, Velasque L, Rohde LA, Coutinho ESF. The mental health care gap among children and adolescents: data from an epidemiological survey from four Brazilian regions. PLoS One 2014;9:e88241, http://dx.doi.org/10.1371/journal.pone.0088241. [30] Associaçao Brasileira de Pesquisa de Mercado A. Critério de classificação econômica Brasil. Available http://www.abep.org/codigosguias/ABEP_ CCEB.pdf 2007. [31] (IBGE). IBdGeE. Coordenação de Trabalho e Rendimento, Pesquisa Nacional por Amostra de Domicílios. Available http://www.ibge.gov. br/home2013 [Brasil]. [32] Instituto Brasileiro de Geografia e Estatística I.. Séries historicas e estatísticas. Available http://seriesestatisticas.ibge.gov.br/series. aspx?no=10&op=0&vcodigo=CD97&t=numero-municipios-censodemografico-classes-tamanho2010 [Brasil]. [33] Comrey AL, Lee HB. A first course in factor analysis. (2nd ed.). Hillsdale, NJ: Erlbaum; 1992. [34] Hu LT, Bentler P. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model 1999;6:1-55. [35] Vandenberg RJ, Lance CE. A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organ Res Methods 2000;3:4-70. [36] Steenkamp J-BE, Baumgartner H. Assessing measurement invariance in cross-national consumer research. J Consum Res 1998;25:78-107. [37] Muthén L, Muthén B. Mplus User's guide. Los Angeles, CA: Muthén & Muthén; 1998-2012. [38] Raykov T, Marcoulides G. Exploratory factor Analysis. Introduction to applied multivariate analysis: Routledge; 2008267. [39] Costello AB, Osborne J. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Pract Assess Res Eval 2005;10:1-9. [40] van de Schoot R, Lugtig P, Hox J. A checklist for testing measurement invariance. Eur J Dev Psychol 2012;9:486-92. [41] Draguns JG, Tanaka-Matsumi J. Assessment of psychopathology across and within cultures: issues and findings. Behav Res Ther 2003;41:755-76. [42] Bass JK, Bolton PA, Murray LK. Do not forget culture when studying mental health. Lancet 2007;370:918-9. [43] Steel Z, Silove D, Giao NM, Phan TT, Chey T, Whelan A, et al. International and indigenous diagnoses of mental disorder among Vietnamese living in Vietnam and Australia. Br J Psychiatry 2009;194:326-33. [44] Das-Munshi J, Castro-Costa E, Dewey ME, Nazroo J, Prince M. Crosscultural factorial validation of the Clinical Interview Schedule– Revised (CIS-R); findings from a nationally representative survey (EMPIRIC). Int J Methods Psychiatr Res 2014;23:229-44. [45] Youngmann R, Zilber N, Workneh F, Giel R. Adapting the SRQ for Ethiopian populations: a culturally-sensitive psychiatric screening instrument. Transcult Psychiatry 2008;45:566-89.
ScienceDirect Comprehensive Psychiatry xx (2015) xxx – xxx www.elsevier.com/locate/comppsych
Psychometric properties of the self-reporting questionnaire (SRQ-20): Measurement invariance across women from Brazilian community settings Felipe Paraventi⁎, Hugo Cogo-Moreira, Cristiane Silvestre Paula, Jair de Jesus Mari Department of Psychiatry, Federal University of São Paulo, São Paulo, Brazil
Abstract Background: SRQ-20 is a validated screening tool for common psychiatric disorders in several countries. Exploration of the latent structure of this instrument resulted in conflicting evidence. This study aimed to explore the latent structure of SRQ-20 among Brazilian women from community settings. We also tested the model invariance across different sociodemographic conditions. Methods: Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were conducted on a sample of 1668 women from four different geographical regions of Brazil. Invariance of the model was tested through multi-group CFA according to sociodemographic variables. Results: EFA has shown two potential solutions with two and three factors. CFA resulted in indices of the two-factor solution slightly worse than the three-factor solution. Invariance testing has shown this model was not invariant across cities, but was invariant across different social classes. The structure was also invariant for the two lower educated groups. The respecified model (i.e., excluding item 16) was not invariant across groups with different educational levels. Conclusion: The three-factor solution seems to be the most suitable model of SRQ-20 for Brazilian women in community settings. Furthermore, sociodemographic variables seem to reflect on the latent structure of this instrument. Validation of screening tools should consider sociodemographic variables. © 2014 Elsevier Inc. All rights reserved.
1. Introduction Mental and behavioral disorders are the largest cause of Years Lost Due to Disability (YLDs) globally [1], more than half of this burden is attributable to depressive and anxiety disorders [2]. In spite of that, the treatment gap for psychiatric disorders is a worldwide phenomenon and several barriers to the progress of mental health care are still present, especially in low-income and middle-income countries [3,4]. Lack of awareness of psychiatric conditions is a critical step for the treatment, thus screening instruments might contribute to this context [5,6]. Patel et al. [5] suggested that increasing recognition of depression might be achieved through community- and practice-based screening by general practitioners as well as trained interviewers. Mateus et al. [7] investigated the panorama of the mental health system of Brazil; these researchers recommended, for
⁎ Corresponding author. E-mail address: [email protected] (F. Paraventi). http://dx.doi.org/10.1016/j.comppsych.2014.11.020 0010-440X/© 2014 Elsevier Inc. All rights reserved.
future mental health policies in this country, a larger focus on community and primary care. The Self-Reporting Questionnaire (SRQ) seems to be a suitable tool for this purpose. The SRQ was developed by WHO as an instrument to screen for common mental disorders in primary care— especially in developing countries [8,9]. Originally, the SRQ consisted of 25 questions to be answered ‘yes’ or ‘no’; 20 questions related to neurotic symptoms, 4 items representative of psychotic symptoms and one about convulsive episodes. For several reasons, attention has been concentrated only on “neurotic items”, expressed as SRQ-20 [10]. SRQ-20 is a brief and simple screening tool that covers important areas of psychopathology. This tool is easily trainable and may be applied as a ‘self-administered’ or ‘interviewer administered’ instrument, the latter being very relevant in settings of high rates of illiteracy [10]. SRQ-20 has been validated in several cultures and settings as a cost-effective manner to screen for psychiatric disorders [10–15]; the Brazilian version is also validated [16–18]. Though several articles discuss distinct aspects of validity (e.g. face validity, content validity, criterion validity, etc.)
2
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
there is conflicting evidence concerning the structure of this questionnaire. Tafari et al. [19] found four-factors of SRQ-24, being the psychotic scale a single factor clearly distinguished from the neurotic section (i.e. SRQ-20), in which three different factors were derived (I—“Cognitive”, II—“Anxiety and Depression” and III—“Somatic Symptoms”). Sen [20] derived seven distinct factors, while Iacoponi and Mari [21] suggested four factors for the SRQ-20 construct (I—“Decreased energy”, II—“Somatic symptoms”, III—“Depressive mood” and IV—“Depressive thoughts”). At the time, the hypothesis raised by WHO was that Sen' and Iacoponi' studies were developed in a primary care setting, while Tafari et al. used a community sample [10]. Chen and collaborators [11] proposed a three-factor model for this instrument in a community population in China, describing ‘depressive symptoms’, ‘anxiety symptoms’ and ‘somatic symptoms’ as clustered components. In a Vietnamese sample, a three-factor model was also found, but this study raised evidences for a bi-factor model as well (a single general factor, i.e. “general distress”, and additional sub-domain or group factors labeled: “negative affect”, “somatic complaints”, and “hopelessness”) [22]. These findings highlight a poor consistency of the latent structure of SRQ-20 across studies; hence, it is not reliable to use SRQ factors as subscales at this moment in time. Invariance across groups is another relevant point, which has not been well explored yet. In other words, whether the questionnaire structure is the same for different populations is not known. In terms of cut-off points, there is a large variation across settings; sometimes the optimum cut-off point is significantly higher, for instance it was 10/11 in Afghanistan [23] and 11/12 in India [20], while in other settings it was considerably lower, for instance 6/7 in Zambia [24]. Previous studies demonstrated that gender and educational level could have an important influence on misclassification by SRQ-20. For instance, results have shown a larger tendency of males to be classified as “false negatives”, while poorly educated people were more likely to be classified as “false positives” [14,23,25,26]. A recent study pointed out the importance of considering gender in validation of screening instruments as well as the benefits of locally-developed measures of mental health [27]. Pendergast and colleagues [28] explored the cross-cultural invariance and invariance over time of SRQ-20 on a specific population of postpartum women; these authors suggested that removing four items of SRQ-20 (the somatic symptoms) would lead to a more suitable structure across those women. They also investigated the invariance of SRQ-20 over eight different sites; responses patterns from Brazil were clearly different from the other seven sites. Validity of SRQ-20 over a specific population (i.e., postpartum women from Northern region of Brazil) was questioned by the cited study. Given the conflicting evidence regarding the latent structure of SRQ-20 and the underexplored invariance across populations, this study aimed to (a) explore and confirm the latent structure of SRQ-20 among Brazilian
women in community settings, and (b) test the invariance of the best-confirmed model through different sociodemographic conditions (i.e. site, social economical status and educational level).
2. Methods 2.1. Procedures and instruments This analysis is part of a large cross-sectional study— designed to explore the mental health service use among Brazilian children and adolescents. The research team randomly approached 2000 families in order to interview and apply standardized instruments to main caregivers, which included: SRQ-20, family's socioeconomic status and sociodemographic factors (i.e. area of residence, age and educational level) [29]. Family's socioeconomic status was assessed by a questionnaire developed by the Brazilian Association of Research Companies according to family purchasing power; this tool is highly used in Brazil and categorizes five social strata, from A (highest purchase power) to E (lowest purchase power) [30]. SRQ-20 questionnaires (previously described) were administered to all mothers or main caregivers only by the interviewer, thus the methods of application (‘self-administered’ and ‘interviewer administered’) were not mixed according to WHO's recommendations [10]. SRQ-20 items are scored ‘0’ if answered ‘no’ (symptom was absent during past month) and ‘1’ if answered ‘yes’ (symptom was present during past month), consequently the maximum score is 20. A cut-off value of 7/8 (‘7’—probable non-case; ‘8’—probable case) is commonly used in Brazil; Mari and Williams [17] proposed a different cut-off for each sex (5/6—male and 7/8—female). Prior to data collection, a team of mental health experts trained and supervised the selected psychologists to conduct the questionnaires application. Participants were individually assessed in private rooms at buildings rented for this particular survey. Data collection took 15 months to be fully completed by December 2011. Written informed consent from caregivers was obtained. The Research Ethics Committee of University of São Paulo approved this study under the process number 0301/09. Local governmental agencies were contacted and agreed to support this research as well. 2.2. Setting Brazil is the fifth largest country in the world with 195 million inhabitants spread over five heterogeneous geographical areas. The Southeast and South regions of Brazil are the two most developed and wealthy [31]. This multicenter research included four towns, from four out of the five Brazilian geographical regions (Caeté—Southeast; Goianira—Central-West; Itaitinga—Northeast; Rio Preto da Eva—North). The selection criteria of the four sites were: (1) being near to a State Capital, (2) having a Human Development Index (HDI) similar to the Brazilian average;
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
and (3) having approximately 30,000 inhabitants, since 84.7% of Brazilian municipalities have less than 50,000 inhabitants [32].
2.3. Sample This study included all the 1668 female caregivers who represented four different geographical areas of Brazil—402 women (24.10%) from Itatinga (Northeast region), 446 women (26.74%) from Goianira (Central-West region), 454 women (27.22%) from Caete (Southeast region) and 366 women (21.94%) from Rio Preto da Eva (North region).
2.4. Statistical analysis Initially, we conducted an exploratory factor analysis (EFA); one to five factors were extracted considering a random sampling of 40% of the original data set. We used the robust weighted least squares estimator and oblique rotation (geomin, default in Mplus), allowing the factors to be correlated. The loadings were considered prominent when larger than 0.32 [33]. Statistical parameters (i.e. model fit indices, magnitude and significance of factor loading across the number of the factor solutions) and interpretability of the factors were considered to achieve the best solution. Based on that solution, we conducted a confirmatory factor analysis on the other 60% of the random sample. Both EFA and CFA model fit indices were evaluated considering the following cut-offs: Root Mean Square Error of Approximation (RMSEA) less or equal to 0.06, Tucker–Lewis Index (TLI) and Comparative Fit Index (CFI) close to 0.95 [34]. Lastly, we executed a multi-group CFA of the confirmed model using 60% of the random sample (i.e. invariance testing). This analysis was included to ensure that theoretical constructs of SRQ-20's items are measured equally across different groups. Two levels of invariance were tested: configural invariance (also called “pattern invariance”) and scalar invariance (also “strong invariance”). Configural invariance must be established in order that subsequent tests can be meaningful. Configural invariance implies that the items comprising the instrument should exhibit the same pattern across groups, but it does not indicate that people respond to those items in the same way. On the other hand, the presence of scalar invariance implies that the instrument's measurement capacity is the same across groups, allowing the comparison of scores between them [35,36]. The scalar against configural invariance testing – through Χ 2 – evaluates if the constraints from more restrictive models [scalar] worsen the fit. Invariance testing considered the four cities (Itatinga, Goianira, Caete and Rio Preto da Eva), years of schooling and SES. We used a one-way ANOVAS (with Bonferroni's correction) to explore the sociodemographic characteristics of each city. All analyses were conducted in Mplus 7.11 [37].
3
3. Results 3.1. Sociodemographic indicators Sociodemographic characteristics of the four cities are presented in Table 1. The highest average indices of age, SES and educational level were seen in Caeté (Southeast region): 35.13 [SD = 6.58], 17.58 [SD = 5.37] and 8.26 [SD = 3.97] respectively. On the contrary, the lowest indices were observed in Itatinga (Northeast region): 32.25 [SD = 6.80], 12.78 [SD = 4.08] and 5.06 [SD = 3.67] respectively. After a multiple group comparison (with Bonferroni correction), we found that statistical differences were explained as follows: (a) the four cities are statistically different regarding SES (Caeté N Goianira N Rio Preto da Eva N Itaitinga); (b) the cities had statistical difference in terms of schooling, with exception of Rio Preto da Eva when compared to Goianira (p = 0.999); and (c) the cities were comparable in regard to age, with exception of Goianira [Goianira vs Caeté (p b 0.001), Goianira vs Itatinga (p b 0.001), and Goianira vs Rio Preto da Eva (p = 0.011)]. 3.2. EFA Table 2 shows EFA solutions from one to five geomin rotated factor loadings (model fit indices and correlations between each respective latent factor may be seen at the bottom of the table). The one-factor solution was the only model to not show good fit indices (CFI/TLI lower than 0.95). Nonetheless, the solutions with four and five factors had a number of cross loadings in six items (5, 6, 11, 13, 15, 19) and eight items (3, 4, 5, 6, 11, 13, 15, 16, 20) respectively. Moreover, the interpretability of such factors is not plausible as they are configured, thus both solutions were invalidated conceptually. The two-factor and three-factor solutions were considered plausible. Observing the two-factor solution, item 8 “Do you have trouble to think clearly?” and item 18 “Do you feel tired all the time?” showed cross loadings. For the three-factor solution, item 3 “Do you sleep badly?” did not reach a loading higher than 0.32 in any of the factors, and four items (15, 16, 17 and 20) showed cross loadings—i.e. commonality of different factors which cannot be accounted for a single factor correlation [38]. Table 1 Mean and standard deviation [SD] of age, SES and years of schooling (YoS) by region. Age (mean [SD]) SES a (mean [SD]) YoS (mean [SD]) Caeté Goianira Itaitinga Rio Preto da Eva F p-Value
35.13 [6.58] 32.87 [6.00] 32.25 [6.80] 34.40 [7.11] 11.31 b0.001
17.58 [5.37] 15.63 [4.64] 12.78 [4.08] 13.93 [5.19] 76,858 b0.001
8.26 [3.97] 7.04 [3.44] 5,06 [3,67] 6.88 [3.9] 48.47 b0.001
a In the multigroup analysis, SES was categorized as follows: 35–46 = A; 23–34 = B; 14–22 = C; 08–13 = D; 00–07 = E.
4
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
Table 2 EFA with one to five factor solutions, correlation of factors and model fit indices. Solution Item
One factor
Two Factors 1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Correlation
1 2 3 4 5
0.479 0.519 0.525 0.522 0.532 0.761 0.619 0.669 0.813 0.686 0.684 0.536 0.606 0.564 0.722 0.760 0.627 0.794 0.705 0.661 1
2
Three Factors 1
2
0.346 0.381 0.331
Four Factors
3
1
4
0.378 0.307
0.440 0.466 0.751 0.422 0.329 0.979 0.913 0.531 0.500 0.468 0.479 0.629 0.801 0.633 0.464 0.434 0.847 0.538 1 2 0.597 1
3
1
2
3
0.304
0.317 0.404 0.329
4
5
0.377 0.351
0.560
2
Five factors
0.492 0.490 0.400
0.337
0.366 0.326
0.763
0.490 0.508
0.605
0.578
0.748 0.902
0.566
0.932 0.901
0.626 0.383 0.741 0.554 0.341 0.413 0.343 0.687 0.377 0.367 0.681 0.604 2 1 0.565
0.337
0.632
0.407
1 0.552 0.446
0.370 0.482 0.426 0.314
0.340
0.847 0.354 3 1
0.931 0.906 0.424 0.672 0.377 0.663 0.331 0.745 0.447
0.321 0.450 0.372 0.341
0.669 1 0.623 0.373 0.508
0.778 0.401 0.634 4
2 3 1,000 0.350 1,000 1 0.514 0.304 1,000
0.393 0.644 0.318 0.610 0.309 0.688 0.426
0.470 0.507
0.725
1 0.292 0.263 0.349 0.075
2 1,000 0.550 0.388 0.560
3
0.943 0.394 4
0.518 5
1,000 0.444 1,000 0.463 0.319 1,000
Model Fit Indices Χ 2 (df), p-value RMSEA (90% IC) CFI TLI
465.162 (170), b0.001 0.052 (0.046–0.057) 0.943 0.936
313.601 (151), b0.001 0.041 (0.034–0.047) 0.969 0.960
Fig. 1 illustrates the result of scree-plot test; we used this tool to provide more information regarding the number of factors to be extracted. According to scree-plot criterion [39], the two- or three-factor solution would be admissible, hence both solutions were considered for CFA.
232.882 (133), b0.001 0.034 (0.027–0.041) 0.981 0.972
152.250 (116), 0.013 0.022 (0.011–0.031) 0.993 0.989
114.610 (100), 0.15 0.015 (0.000–0.026) 0.997 0.995
the three derived factors of this model might be clinically labeled as: Factor 1—“Anxiety/Depression”, Factor 2—“Disability” and Factor 3—“Somatic symptoms”. Table 4 illustrates this configuration. 3.4. Invariance testing (multi-group CFA)
3.3. CFA The two- and three-factor solutions were modeled in a CFA keeping all the 20-items exactly as obtained from EFA, which in the case of the three-factor solution had five items with cross loadings. Indeed, an additional CFA was conducted testing a three-factor solution without the item 3, because this item had a loading lower than 0.32 in all the factors. Model fit indices for these CFAs are presented in the Table 3. Analysis of the results of CFA solutions indicates the restriction worsens the model fit indices (Χ 2(5) = 100.810, p b 0.001). Consequently, we decided to follow through invariance testing (multi-group CFA) considering the three-factor solution. In our point of view,
3.4.1. City invariance Individually, the baseline model's fit indices were good for each city as follows—(a) Rio Preto da Eva (Χ 2(162) = 237.479, p b 0.001), RMSEA = 0.051, CFI = 0.923, TLI = 0.91, and WRMR = 1.0; (b) Caete (Χ 2(162) = 230.030, p b 0.001), RMSEA = 0.042, CFI = 0.958, TLI = 0.951, WRMR = 0.952; (c) Goianira (Χ 2 (162) = 235.802, p b 0.001), RMSEA = 0.042, CFI = 0.975, TLI = 971, WRMR =0.915, and (d) Taitinga (Χ 2(162) = 210.544, p = 0.006), RMSEA = 0.037, CFI = 0.971, TLI = 0.966, WRMR = 0.860. Once the baseline models were determined and showed good fit indices, they were integrated into a multi-group CFA model. Considering the cities all together, scalar invariance versus
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
5
Fig. 1. Scree-plot test
configural invariance was statistically significant (Χ 2(57) = 93.737, p = 0.0016). Therefore, the observed non-invariance indicates that scores may not be comparable across the four cities. 3.4.2. SES invariance Highest strata (A, B, and C) and lowest strata (D and E) were merged in order to obtain a reasonable sample size to conduct the invariance testing. The upper class (n = 527) and the lower class (n = 374) showed good fit indices separately. The configural model indices were: (Χ 2(324) = 617.206, p b 0.001), RMSEA = 0.045, CFI = 0.958, and TLI = 0.951, while the scalar model indices were: (Χ 2(343) = 627.280, p b 0.001), RMSEA = 0.043, CFI = 0.959, and TLI = 0.955. The scalar against configural model was not statistically significant (Χ 2(19) = 21.065, p = 0.3332), thus factor loadings and thresholds across different social classes are invariant. Consequently, scores are comparable between these groups. 3.4.3. Schooling invariance Based on percentiles of ages of schooling, three cut-offs were considered for schooling invariance testing: less than or equal to four years of schooling (25th percentile; n = 302), more than four and less than ten years of schooling (n = 348), and more than or equal to ten years of schooling (75th percentile, n = 251). Women with less than or equal to four years of schooling returned good fit indices: (Χ 2(162) = 280.458, p b 0.001), RMSEA = 0.049 (90% IC = 0.039– 0.059), CFI = 0.957, TLI = 0.950. Taking the population with ‘more than four and less than ten years of schooling’ into account, the tested model returned good indices: (Χ 2 (162) = 248.678, p b 0.001), RMSEA = 0.039 (95% IC = 0.029–0.049), CFI = 0.963, TLI = 0.957, and WRMR = 0.967. Lastly, women with ‘more than or equal to ten years of schooling’ showed a perfect correlation between items in two different situations, as follows: (a) item 6 (“Do you feel
nervous, tense or worried?”) and item 16 (“Do you feel that you are a worthless person?”); and (b) item 16 and item 12 (“Do you find it difficult to make decisions”?). Both situations are considered inadmissible, so each pair should have one variable excluded. This finding means that the baseline model of this group is not comparable to the others; therefore, a respecified model needed to be achieved. In order to preserve the maximum of the instrument's original items, we took the decision to exclude the item 16—which is present in both pairs. The respecified model showed good statistical fit after exclusion of the item 16 as follows: (Χ 2 (145) = 189.380, p = 0.008), RMSEA = 0.039 (90% IC = 0.021–0.053), CFI = 0.952, TLI = 0.943, and WRMR = 0.918. Comparing the two lower groups under the original structure (with 20 items), configural and scalar invariance returned good fit indices. The scalar against configural invariance was not statistically significant (Χ 2(19) = 16.488, p = 0.6245) hence this model is invariant across women with less than ten years of schooling. We also tested the invariance across the three groups under the respecified model (i.e., without item 16). In this case, configural and scalar invariance returned good fit indices, but the scalar against configural testing was statistically significant (Χ 2(36) = 52.007, p = 0.0410). In reason of that, the respecified model is not invariant across those groups.
4. Discussion 4.1. Factorial structure and invariance This study explored the underlying structure of SRQ-20 through EFA and CFA. Results suggested the three-factor solution seems to be the most suitable model for a female community sample in Brazil. From our point of view, the factors “Anxiety/Depression”, “Disability” and “Somatic symptoms” represented the latent structure of SRQ-20 in this
6
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
Table 3 CFA of two- and three-factor solutions. Model Fit index
Two-factor
Three-factor (all items)
Three-factor (without item3)
Χ 2 (df), p-value RMSEA (90% IC) CFI TLI
556.965 (167), p b 0.001 0.051 (0.046–0.056) 0.946 0.939
438.606 (162), p b 0.001 0.044 (0.039–0.048) 0.962 0.955
413.498 (145), p b 0.001 0.045 (0.040–0.050) 0.961 0.954
population. Further investigation of this model through multi-group CFA (invariance testing) has shown that this model is only invariant across social classes; when schooling and cities' location were considered the expected invariance was not observed. As stated by van de Schoot and colleagues [40], when measurement invariance is not achieved, groups or subjects respond differently to the items and, as consequence, factor scores may not be compared. In the case of geographical regions, the pattern of factors is equivalent, but those items and factors are not measured in the same scale across the four cities. In other words, women responded to the instrument in different ways across the cities, so direct comparison of scores is not possible when comparing women from one city to the other. With regard to schooling, the baseline model does not fit the group with highest educational level, which demanded a respecified model (e.g. without item 16). Therefore, in this situation, the comparison of scores between groups is not possible because the models are not equivalent between them. The structure was not invariant even considering the respecified model. Reasons why the proposed model had a variance across geographical regions of Brazil remain unclear. Differences regarding sociodemographic indicators could partially justify this issue. For example, cities were different in terms of schooling, and – as seen in the Results section –the structure was not invariant when this variable was considered. Several studies have raised questions about cultural impact on how emotions, thoughts and behaviors are noticed and manifested [41–43]. Draguns and Tanaka-Matsumi [41], explored the cultural influence in psychopathology highlighting the variation of psycho-emotional manifestations across and within cultures. Other studies showed the benefit of assessing some items within cultural contexts in order to improve the instrument's validity [44]. Youngmann and colleagues [45] demonstrated that the validity of the culturally adapted SRQ was superior to the original structure. As long as Brazil is marked by a very heterogeneous cultural pattern across the territory, we hypothesize that this massive difference might also be an explanation for the non-observed invariances. Such an unexplored area represents a potential field for future research in Brazil. Our results support a three-factor structure of SRQ-20 among women in community settings. These findings resemble solutions proposed by other authors, as described
in the Introduction section (Tafari et al. [19], Chen et al. [11], Stratton et al. [22]). Moreover, in a recent work, Rasmussen and colleagues [27] found a latent structure of SRQ-20 that has some resemblance to our findings (“Negative affect”, “Emotional Numbing” and “Somatic complaints”). When researchers compared SRQ-20 to a culturally grounded tool, they found out that SRQ-20 was a worse measure of distress for women, while both instruments were similar for men. Those authors once more highlighted that validity of screening instruments might be gender-differentiated. Our study design takes this proposition into account; furthermore, our results suggested that not only gender and setting should be considered when validating screening tools, but also, wider characteristics of each population. For instance, Pendergast and collaborators [28] have studied the SRQ-20 structure among postpartum women; their findings suggested the four somatic items – i.e., headaches, stomach-
Table 4 The three-factor solution of SRQ-20. SRQ-20 item
Factor I Factor II Factor III “Anxiety“Disability” “Somatic Depression” Symptoms”
2. Is your appetite poor? 6. Do you feel nervous, tense or worried? 9. Do you feel unhappy? 10. Do you cry more than usual? 15. Have you lost interest in things? 16. Do you feel that you are a worthless person? 17. Has the thought of ending your life been on your mind? 8. Do you have trouble thinking clearly? 11. Do you find it difficult to enjoy your daily activities? 12. Do you find it difficult to make decisions? 13. Is your daily work suffering? 14. Are you unable to play a useful part in life? 18. Do you feel tired all the time? 20. Are you easily tired? 1. Do you often have headaches? 4. Are you easily frightened? 5. Do your hands shake? 7. Is your digestion poor? 19. Do you have uncomfortable feelings in your stomach? 3. Do you sleep badly?
0.351 0.400
– –
– –
0.748 0.902
– –
– –
0.341
0.413
–
0.343
0.687
–
0.377
0.367
–
–
0.407
–
–
0.626
–
–
0.383
–
–
0.741
–
–
0.554
–
–
0.681
–
– –
0.604 –
0.354 0.377
– – – –
– – – –
0.492 0.490 0.763 0.847
–
–
–
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
aches, digestive difficulties and sleep difficulties – were not representative of internalizing symptoms for this sample. In other words, such somatic complaints may be typical of women recovering from childbirth, not reflecting mental distress. Consequently, when screening mental illness among postpartum women, use of SRQ-20 in its original form might not be as appropriated as the proposed one-factor model, which excludes the somatic items (i.e. SRQ-16) [28]. These findings go along with our hypothesis that screening tools validation should take more specific characteristics of each population into consideration. That is, probably many factors (e.g. sociodemographics characteristics as proposed by this work) or even different groups may influence the understanding of mental illness' construct; consequently, they may respond unequally to the instrument's items. In conclusion, overall or factor scores might not be comparable across distinct populations. 4.2. Limitations and strengths Some limitations may apply to this study. First, our sample is restricted to a female population from Brazilian community settings. Moreover, participants are considered “mothers” or “main caregivers” of children and adolescents, once this investigation was part of a large cross-sectional study. We have only assessed invariance considering site, educational level and SES, so many other social and cultural parameters might have been explored. Religion seems to be a particular example to highlight, especially because it is very heterogeneous in Brazil. Limitations notwithstanding, this work suggests a threefactor structure of SRQ-20 in a community population of Brazilian women and highlights that SRQ-20 has a variable measurement capacity of psycho-emotional symptoms across different sociodemographic conditions. Results seem particularly important to Brazil, where a noticeable diversity of cultural or social factors is observed. A better quality of screening in this country might be helpful in reducing its treatment gap. On the other hand, our findings also reflect some limitations on practical use of SRQ-20. As argued by Kagee and colleagues [6], integration of routine screening into primary care has some concerns. Our results indicate that the underlying construct of SRQ-20 may not be comparable between some populations, it possibly affects the instrument's reliability, and consequently, the integration of this tool into primary care practice would not represent a reasonable strategy. When compared to other measures (e.g. clinical interview), the possible misclassification associated to SRQ-20 could raise the treatment gap on false-negative individuals. On the contrary, detection of false-positive individuals (i.e. those who do not need treatment) would raise the assistance needs within a mental health system that is already overloaded [7,25]. From our point of view, future validation of screening questionnaires should consider sociodemographic variables in order to improve the screening quality.
7
5. Conclusion The three-factor model of SRQ-20 seems to be suitable for a community population of Brazilian women. In spite of that, our results suggested that the measurement capacity and understanding of the mental health construct might vary across the country and some social conditions. At this moment in time, benefits of screening mental disorders in Brazil through SRQ-20 should be weighted and further explored. Future validation of screening instruments with a focus on the sociodemographic characteristics of specific populations seems to be plausible and recommended. Acknowledgment The National Research Council (CNPq) funded this project by a grant from “INCT—Psiquiatria do Desenvolvimento para Crianças e Adolescentes” (No. 573974/2008-0). H.C.M. is a post-doc researcher from Coordination for the Improvement of Higher Education Personnel (PNPD/ CAPES). C.S.P. was partially supported by CNPq. J.J.M. is a senior researcher from CNPq. We are very grateful to the reviewers for the detailed work they conducted. References [1] Vos T, Flaxman AD, Naghavi M, Lozano R, Michaud C, Ezzati M, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012;380:2163-96. [2] Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 2013;382:1575-86. [3] Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bull World Health Organ 2004;82:858-66. [4] Saraceno B, van Ommeren M, Batniji R, Cohen A, Gureje O, Mahoney J, et al. Barriers to improvement of mental health services in lowincome and middle-income countries. Lancet 2007;370:1164-74. [5] Patel V, Simon G, Chowdhary N, Kaaya S, Araya R. Packages of care for depression in low- and middle-income countries. PLoS Med 2009;6:e1000159. [6] Kagee A, Tsai AC, Lund C, Tomlinson M. Screening for common mental disorders in low resource settings: reasons for caution and a way forward. Int Health 2013;5:11-4. [7] Mateus MD, Mari JJ, Delgado PG, Almeida-Filho N, Barrett T, Gerolin J, et al. The mental health system in Brazil: policies and future challenges. Int J Ment Heal Syst 2008;2:12, http://dx.doi.org/10.1186/1752-4458-2-12. [8] Harding TW, Climent CE, Diop M, Giel R, Ibrahim HH, Murthy RS, et al. The WHO collaborative study on strategies for extending mental health care, II: the development of new research methods. Am J Psychiatry 1983;140:1474-80. [9] Harding TW, de Arango MV, Baltazar J, Climent CE, Ibrahim HH, Ladrido-Ignacio L, et al. Mental disorders in primary health care: a study of their frequency and diagnosis in four developing countries. Psychol Med 1980;10:231-41. [10] WHO. A User's Guide to the Self-Reporting Questionnaire (SRQ). Geneva: World Health Organization; 1994. [11] Chen S, Zhao G, Li L, Wang Y, Chiu H, Caine E. Psychometric properties of the Chinese version of the Self-Reporting Questionnaire 20 (SRQ-20) in community settings. Int J Soc Psychiatry 2009;55:538-47.
8
F. Paraventi et al. / Comprehensive Psychiatry xx (2015) xxx–xxx
[12] Scholte WF, Verduin F, van Lammeren A, Rutayisire T, Kamperman AM. Psychometric properties and longitudinal validation of the self-reporting questionnaire (SRQ-20) in a Rwandan community setting: a validation study. BMC Med Res Methodol 2011;11:116, http://dx.doi.org/10.1186/1471-2288-11-116. [13] Giang KB, Allebeck P, Kullgren G, Tuan NV. The Vietnamese version of the Self Reporting Questionnaire 20 (SRQ-20) in detecting mental disorders in rural Vietnam: a validation study. Int J Soc Psychiatry 2006;52:175-84. [14] Araya R, Wynn R, Lewis G. Comparison of two self administered psychiatric questionnaires (GHQ-12 and SRQ-20) in primary care in Chile. Soc Psychiatry Psychiatr Epidemiol 1992;27:168-73. [15] Harpham T, Reichenheim M, Oser R, Thomas E, Hamid N, Jaswal S, et al. Measuring mental health in a cost-effective manner. Health Policy Plan 2003;18:344-9. [16] Mari JJ, Williams P. A comparison of the validity of two psychiatric screening questionnaires (GHQ-12 and SRQ-20) in Brazil, using Relative Operating Characteristic (ROC) analysis. Psychol Med 1985;15:651-9. [17] Mari JJ, Williams P. A validity study of a psychiatric screening questionnaire (SRQ-20) in primary care in the city of Sao Paulo. Br J Psychiatry 1986;148:23-6. [18] Scazufca M, Menezes PR, Vallada H, Araya R. Validity of the self reporting questionnaire-20 in epidemiological studies with older adults: results from the Sao Paulo Ageing & Health Study. Soc Psychiatry Psychiatr Epidemiol 2009;44:247-54. [19] Tafari S, Aboud FE, Larson CP. Determinants of mental illness in a rural Ethiopian adult population. Soc Sci Med 1991;32:197-201. [20] Sen B. Psychiatric phenomena in primary health care their extent and nature. Indian J Psychiatry 1987;29:33-40. [21] Iacoponi E, Mari JJ. Reliability and factor structure of the Portuguese version of Self-Reporting Questionnaire. Int J Soc Psychiatry 1989;35: 213-22. [22] Stratton KJ, Aggen SH, Richardson LK, Acierno R, Kilpatrick DG, Gaboury MT, et al. Evaluation of the psychometric properties of the Self-Reporting Questionnaire (SRQ-20) in a sample of Vietnamese adults. Compr Psychiatry 2013;54:398-405. [23] Ventevogel P, De Vries G, Scholte WF, Shinwari NR, Faiz H, Nassery R, et al. Properties of the Hopkins Symptom Checklist-25 (HSCL-25) and the Self-Reporting Questionnaire (SRQ-20) as screening instruments used in primary care in Afghanistan. Soc Psychiatry Psychiatr Epidemiol 2007;42:328-35. [24] Aidoo M. Explanations for the causes of mental ill-health among lowincome women in an urban area: the case of Zambia. PhD thesis London: South Bank University; 1998. [25] Mari JJ, Williams P. Misclassification by psychiatric screening questionnaires. J Chronic Dis 1986;39:371-8. [26] Ludermir AB, Lewis G. Investigating the effect of demographic and socioeconomic variables on misclassification by the SRQ-20 compared with a psychiatric interview. Soc Psychiatry Psychiatr Epidemiol 2005;40:36-41. [27] Rasmussen A, Ventevogel P, Sancilio A, Eggerman M, Panter-Brick C. Comparing the validity of the self reporting questionnaire and the Afghan symptom checklist: dysphoria, aggression, and gender in transcultural assessment of mental health. BMC Psychiatry 2014;14:206, http:// dx.doi.org/10.1186/1471-244X-14-206.
[28] Pendergast LL, Scharf RJ, Rasmussen ZA, Seidman JC, Schaefer BA, Svensen E, et al. Postpartum depressive symptoms across time and place: structural invariance of the Self-Reporting Questionnaire among women from the international, multi-site MAL-ED study. J Affect Disord 2014;167:178-86. [29] Paula CS, Bordin IAS, Mari JJ, Velasque L, Rohde LA, Coutinho ESF. The mental health care gap among children and adolescents: data from an epidemiological survey from four Brazilian regions. PLoS One 2014;9:e88241, http://dx.doi.org/10.1371/journal.pone.0088241. [30] Associaçao Brasileira de Pesquisa de Mercado A. Critério de classificação econômica Brasil. Available http://www.abep.org/codigosguias/ABEP_ CCEB.pdf 2007. [31] (IBGE). IBdGeE. Coordenação de Trabalho e Rendimento, Pesquisa Nacional por Amostra de Domicílios. Available http://www.ibge.gov. br/home2013 [Brasil]. [32] Instituto Brasileiro de Geografia e Estatística I.. Séries historicas e estatísticas. Available http://seriesestatisticas.ibge.gov.br/series. aspx?no=10&op=0&vcodigo=CD97&t=numero-municipios-censodemografico-classes-tamanho2010 [Brasil]. [33] Comrey AL, Lee HB. A first course in factor analysis. (2nd ed.). Hillsdale, NJ: Erlbaum; 1992. [34] Hu LT, Bentler P. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model 1999;6:1-55. [35] Vandenberg RJ, Lance CE. A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organ Res Methods 2000;3:4-70. [36] Steenkamp J-BE, Baumgartner H. Assessing measurement invariance in cross-national consumer research. J Consum Res 1998;25:78-107. [37] Muthén L, Muthén B. Mplus User's guide. Los Angeles, CA: Muthén & Muthén; 1998-2012. [38] Raykov T, Marcoulides G. Exploratory factor Analysis. Introduction to applied multivariate analysis: Routledge; 2008267. [39] Costello AB, Osborne J. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Pract Assess Res Eval 2005;10:1-9. [40] van de Schoot R, Lugtig P, Hox J. A checklist for testing measurement invariance. Eur J Dev Psychol 2012;9:486-92. [41] Draguns JG, Tanaka-Matsumi J. Assessment of psychopathology across and within cultures: issues and findings. Behav Res Ther 2003;41:755-76. [42] Bass JK, Bolton PA, Murray LK. Do not forget culture when studying mental health. Lancet 2007;370:918-9. [43] Steel Z, Silove D, Giao NM, Phan TT, Chey T, Whelan A, et al. International and indigenous diagnoses of mental disorder among Vietnamese living in Vietnam and Australia. Br J Psychiatry 2009;194:326-33. [44] Das-Munshi J, Castro-Costa E, Dewey ME, Nazroo J, Prince M. Crosscultural factorial validation of the Clinical Interview Schedule– Revised (CIS-R); findings from a nationally representative survey (EMPIRIC). Int J Methods Psychiatr Res 2014;23:229-44. [45] Youngmann R, Zilber N, Workneh F, Giel R. Adapting the SRQ for Ethiopian populations: a culturally-sensitive psychiatric screening instrument. Transcult Psychiatry 2008;45:566-89.
