Research in Developmental Disabilities 35 (2014) 75–86
Contents lists available at ScienceDirect
Research in Developmental Disabilities
Measuring quality of life in people with intellectual and multiple disabilities: Validation of the San Martı´n scale Miguel A. Verdugo a, Laura E. Go´mez Robert L. Schalock e
b,
*, Benito Arias c, Patricia Navas d,
a
Institute on Community Integration (INICO), University of Salamanca, Avda. de la Merced, 109-131, 37005 Salamanca, Spain Department of Psychology, University of Oviedo, Plaza Feijoo s/n, 33003 Oviedo, Spain Department of Psychology, University of Valladolid, Paseo de Bele´n 1, Campus Miguel Delibes, 47011 Valladolid, Spain d Nisonger Center, The Ohio State University, 357F McCampbell Hall, 1581 Dodd Drive, Columbus, OH 43210, United States e Hastings College, Nebraska, 710 Turner Avenue, Hastings, NE 68901, United States b c
A R T I C L E I N F O
A B S T R A C T
Article history: Received 18 September 2013 Received in revised form 19 October 2013 Accepted 21 October 2013 Available online 15 November 2013
Although there are numerous quality of life instruments in the, field of intellectual disability, most of them are addressed to those, people with the highest levels of functioning, while only a few are, suitable for people with the lowest levels (i.e., people with profound, and severe intellectual disabilities, or people with intellectual and, developmental disabilities and other significant medical conditions or, disabilities). This study provides reliability and validity evidence of, the San Martı´n Scale, a 95-item Likert scale questionnaire that is, completed by a third-party respondent. The validation sample was composed, of 1770 people from Spain with intellectual and developmental, disabilities that showed extensive or pervasive support needs (8.7% had, mild intellectual disability, 28.25% moderate, 41.6% severe, and 21.4%, profound). The age of the participants ranged between 16 and 77 years old, (M = 7.78; SD = 12.32). The results suggested that the eight quality of, life domains assessed on the scale are reliable (Cronbach’s alpha ranging, from .821 to .933). Confirmatory Factor Analyses provided construct, validity evidences related to the internal structure of the San Martı´n, Scale, and indicated that the eight first-order factor solution provided, the best fit to the data over unidimensional and hierarchical solutions. Implications of these findings and guidelines for further research are, discussed. ß 2013 Elsevier Ltd. All rights reserved.
Keywords: Profound intellectual and multiple disabilities Intellectual, disability Psychometric evaluation Validation Reliability Measurement Developmental disabilities
1. Introduction Personal outcomes are important measures in the fields of education, health care, and social services that are being used not only for enhancing person well-being (Claes, van Hove, Vandevelde, van Loon, & Schalock, 2012; van Loon et al., 2013) but also becoming very useful for assessing the effectiveness of intervention programs (Go´mez, Verdugo, Arias, Navas, & Schalock, 2013; Schalock, Verdugo, & Go´mez, 2011; Schalock & Verdugo, 2012a, 2012b). Personal outcomes are typically referenced to eight core quality of life domains that reflect an individual’s self-determination (SD), emotional well-being
* Corresponding author at: Faculty of Psychology, University of Oviedo, Plaza Feijoo s/n, 33003 Oviedo, Spain. Tel.: +34 985 10 46 95. E-mail addresses: [email protected] (M.A. Verdugo), [email protected] (L.E. Go´mez), [email protected] (B. Arias), [email protected] (P. Navas), [email protected] (R.L. Schalock). 0891-4222/$ – see front matter ß 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ridd.2013.10.025
76
M.A. Verdugo et al. / Research in Developmental Disabilities 35 (2014) 75–86
(EW), physical well-being (PW), material well-being (MW), personal development (PD), rights (RI), social inclusion (SI), and interpersonal relationships (IR) (Schalock & Verdugo, 2002). These domains can be assessed either through self-reports, reports from other people, or both. Self-report forms assess the individual’s self-perception of his/her status on the respective personal outcome and reflect the values underlying the quality of life concept (e.g., inclusion, empowerment, equity, and self-determination) as well as the principles underlying the disability rights movement (Navas, Go´mez, Verdugo, & Schalock, 2012; Verdugo, Navas, Go´mez, & Schalock, 2012). The report of others assesses the respondent’s perception about the person’s status on the respective personal outcome. Though the most desirable measurement is that involving both kinds of reports (Beadle-Brown, Murphy, & DiTerlizzi, 2009; Schalock et al., 2002; van Loon, van Hove, Schalock, & Claes, 2008; Verdugo, Go´mez, & Arias, 2007; Verdugo et al., 2013), it is not always possible to get reliable and valid self-reports from those people with the lowest levels of functioning (Finlay & Lyons, 2002; Kober & Eggleton, 2002; McGillivray, Lau, Cummins, & Davey, 2009) and highest support needs (Petry & Maes, 2007; Petry, Maes, & Vlaskamp, 2010). For this reason, reports from others (such as relatives, professionals, or caretakers) are frequently used (e.g., Sines, Hogard, & Ellis, 2012; Vos, De Cock, Petry, Van den Noortgate, & Maes, 2010; Wong, Wong, Schalock, & Chou, 2011). Moreover, others who are close to people with severe and profound disabilities often make major life decisions on their behalf. Therefore, it is important to know what the perceptions are of professionals, relatives, and other significant people (i.e., those involved in providing supports), given that concordance between the formers and the judgments of people with intellectual and developmental disabilities is often low to moderate (Claes et al., 2012; Cummins, 2002; Golubovic & Sˇkrbic´, 2013; Janssen, Schuengel, & Stolk, 2005; Kane et al., 2005; Mcvilly, Burton-Smit, & Davidson, 2000; Perry & Felce, 2002; Schmidt et al., 2010; Shipman, Sheldrick, & Perrin, 2011). Although there are a very considerable number of instruments to assess quality of life for people with intellectual and developmental disabilities, almost none of them are suitable for those with the lowest levels of functioning. This is due to significant limitations in adaptive behavior (Arias, Verdugo, Navas, & Go´mez, 2013; Belva & Matson, 2013; Matson, Dixon, Matson, & Logan, 2005; Matson, Cooper, Malone, & Moskow, 2008; Navas, Verdugo, Arias, & Go´mez, 2012; Schuchardt, Maehler, & Hasselhorn, 2011; Tasse´, 2013), or other significant conditions related to language limitation, significant motor dysfunctions (Nakken & Vlaskamp, 2007), chronic and pain-related medical conditions (van der Putten & Vlaskamp, 2011), challenging behaviors (Gerber et al., 2011; Matson & Boisjoli, 2007; Matson & Minshawi, 2007; Matson & Neal, 2009; Matson et al., 2011; Poppes, van der Putten, & Vlaskamp, 2010), sensory impairments (Meule et al., 2013), or mental health problems (Horovitz et al., 2011; Kozlowski, Matson, Sipes, Hattier, & Bamburg, 2011; Morisse, Vandemaele, Claes, Claes, & Vandevelde, 2013). In addition to the lack of suitable scales to use for people with severe and profound limitations, a systematic review of self-reported quality of life measures for people with intellectual disability (Li, Tsoi, Zhang, Chen, & Wang, 2013) identified only nine instruments (in English) that measured domains or indicators that aligned with the quality of life construct currently used widely in the field. Data with regard to internal consistency was available for the nine instruments but only four of them had an excellent overall rating (lower when internal consistency was analyzed for every domain). None of the instruments applied confirmatory factor analysis or more advanced measurement methods such as item response theory. Another recent systematic review of the existent quality of life scales – also focused only on English available tools – showed that only six were valued as psychometrically sound and none of them specifically assessed quality of life for those who exhibit challenging behaviors (Townsend-White, Pham, & Vassos, 2012). According to both reviews, it can be concluded that, although there is a wealth of instruments to assess quality of life for people with intellectual disability, most of them are not well validated or not related to a clearly articulated quality of life theory. In addition, few are suitable for use regarding to the assessment of people with the lowest levels of functioning (Lyons, 2005; Petry, Maes, & Vlaskamp, 2009; Petry, Kuppens, Vos, & Maes, 2010; Ross & Oliver, 2003; Vos et al., 2010). There are several quality of life instruments available in Spanish that have been recently developed specifically for people with intellectual and developmental disabilities: (a) the INTEGRAL Scale (Go´mez, Arias, Verdugo, & Navas, 2012), for those who are able to communicate and self-report but with psychometric limitations that need to be solved (it includes both selfreport and report of others); (b) the INICO-FEAPS Scale (Verdugo et al., 2013), an instrument with the same goals and addressed to the same target population than the previous one but overcoming its limitations (self-report and report of others); (c) the GENCAT Scale (Verdugo, Arias, Go´mez, & Schalock, 2010), addressed to social service recipients, including people with intellectual disability (report of others); and (d) the FUMAT Scale (Go´mez, Verdugo, Arias, & Navas, 2008), focused on people with disabilities in aging process (i.e., over 45 years old), but its psychometric properties have been only preliminary tested (report of others). According to professionals providing supports and services to those people with intellectual and developmental disabilities in Spain, none of the former instruments is suitable for those with the lowest levels of functioning who are frequently unable to communicate their feelings, thoughts, and preferences. The San Martı´n Scale (Verdugo et al., in press) was developed with the goal of bridging this gap and satisfying the demands of practitioners that are interested on the implementation of evidence based practices to improve the quality of life of people through the provision of supports. Its development involved the suggested steps for creating multidimensional quality of life instruments focused on the context (Verdugo, Schalock, Go´mez, & Arias, 2007) and guidelines for the construction and analysis of tests (AERA, APA, NCME, 1999; Brennan, 2006; Downing & Haladyna, 2006; Evers et al., 2013; Wilson, 2005). To develop the San Martin Scale, firstly, a pool of 276 items – organized around the eight domains (Schalock & Verdugo, 2002) – was developed after an exhaustive review of the scientific literature. With the goal of selecting the best items and
M.A. Verdugo et al. / Research in Developmental Disabilities 35 (2014) 75–86
77
providing evidences based on the content of the scale, the items were evaluated by 12 experts on quality of life (among them, directors of entities, academics and researchers, psychologists). Experts valued their applicability to people with profound intellectual disability and people with extensive and pervasive support needs. We carried out a modified Delphi method in which the panel of experts assessed not only the content but also the structure of the initial pool of items, as well as added new items and reformulations. The best 118 items by suitability, importance, sensitivity, and observability according to the experts were selected using qualitative and quantitative methods (Go´mez, Arias, Verdugo, Tasse´, & Brown, 2013). Next, given that stakeholders must be involved in the selection of items, the resultant items of the Delphi study were reviewed by a focus group formed by nine direct-care professionals that worked with the target population at Fundacio´n Obra San Martı´n – a Spanish agency that provides support and services to people with intellectual and developmental disabilities. They assessed again the suitability, importance, sensitivity, and observability of the 118 items, as well as valued if the items were assessing the domain in which they were included, and if there was any indicator or important aspect not represented by the selected items. The participants in the focus group considered that all items were suitable, important, sensible, and observable, as well as well located in the domain they had been assigned to. They also reformulated some items, made some clarifications in order to facilitate the understanding of some items, and added two items in the material well-being domain with the goal of assessing a missing indicator (i.e., conditions of the center providing supports). The above process resulted in a 120 items field-test version of the scale whose validation is the object of the present study. Both reliability and validity evidences based on the internal structure of the scale will be provided. The evaluation of reliability includes Cronbach’s alpha indexes and means of polychoric correlations, while Confirmatory Factor Analysis (CFA) evaluated validity comparing the goodness-of-fit to the data of three alternative models that have been pointed out in the recent literature (Go´mez, Verdugo, Arias, & Arias, 2010): (a) a unidimensional model: quality of life is composed of only one generic domain (although there is a broad consensus about the multidimensionality of the construct, the structure will be checked since structural equation modeling emphasizes model parsimony) (Model I); (b) the hypothesized eight-correlated factors (Schalock & Verdugo, 2002) that was on the basis of the development of the scale (Model II); and (c) a hierarchical structure in which these eight domains are first-order factors with the existence of a second-order one representing a generic quality of life domain (Wang, Schalock, Verdugo, & Jenaro, 2010) (Model III). 2. Method 2.1. Participants Two selection criteria were employed. In reference to those providing information (i.e., report of others) informants could be professionals, relatives or proxies who knew well the assessed person for at least three months, and had the recent opportunity (i.e., within the last month) to observe him/her in different contexts and during prolonged periods of time (i.e., several hours a day). In reference to the person being assessed, this individual needed to: (a) show an intellectual or developmental disability and extensive or pervasive support needs (i.e., people with intellectual disability and a low level of functioning, for instance, due to a profound or severe intellectual disability, very significant limitations in adaptive behavior, multiple disabilities, chronic and severe health conditions, or mental health problems); (b) be currently receiving supports and services; and (c) be 16 years old or older and not currently engaged in the education system. The field-test version of the San Martı´n Scale was applied to a convenience sample composed of 1770 people who met the above criteria. The assessment was carried out by 399 people, most of whom (97.4%) were professionals working at 99 agencies that provided support to people with intellectual and developmental disabilities located throughout Spain. Other respondents were parents (n = 36; 2%), siblings (n = 6; 0.3%), and a guardian (0.1%). Each respondent completed the instrument on an average of four people, and an average of 18 people was assessed at each agency. People that completed the assessment had known the person for more than two years in most of the cases (85.1%), more than a half (54.2%) knew the person for more than five years, and 30.9% for between two and five years. Professionals who knew the person for between six months and a year answered the scale for 6.4% of the participants, while those for 3–6 months evaluated only 1.4% of the cases. The great majority (92.8%) had a frequency of contact with the assessed person of several times per week. With regard to the people with intellectual and developmental disabilities assessed, the number of men (n = 993; 56.1%) was lightly higher than the number of women (n = 777; 43.9%). The age of the participants ranged between 16 and 77 years old (M = 37.78; SD = 12.32). The analysis of the Pearson standardized residuals showed that the proportions of men and women by age groups were equiprobable with the only exception of a slight underrepresentation of those women under 28 (x2ð3Þ ¼ 14:658; p = .002). All clients in the sample required extensive (45.3%) or pervasive (54.7%) support needs. Although there was no specific measurement of intellectual and adaptive functioning available for most of the people, respondents estimated that 8.7% had mild intellectual disability, 28.25% moderate, 41.6% severe, and 21.4% profound. In addition, 91.6% had other associated verified conditions, such as epilepsy, physical disability or challenging behaviors (see Fig. 1). A third (34.3%) of participants had only one associated condition, 16% had two concurrent associated conditions, 16% showed three conditions, 6% presented four, and the rest of the sample had between five and seven (M = 1.85; SD = 1.25). Most of them (74.2%) were taking medication, especially antiepileptic/anti-seizures and anxiolytics.
78
M.A. Verdugo et al. / Research in Developmental Disabilities 35 (2014) 75–86
Fig. 1. Percent of persons with other associated conditions.
2.2. Instrument The field-test version of the San Martı´n Scale was a self-administered questionnaire in which a third-party respondent (i.e., staff, relative, proxy, etc. who knows well and has opportunities to observe the person assessed) must answer questions about the person’s quality of life. Administration time varied from 20 and 40 min. It was composed of eight subscales that correspond to the eight quality of life domains by Schalock and Verdugo, and a total of 120 items (SD = 12; EW = 16; PW = 14; MW = 17; RI = 16; PD = 16; SI = 11; IR = 18). All items were formulated as third person declarative statements and random organized by domains. All items had positive valence with the only exception of five (i.e., EW15, EW20, EW22, RI63, and RI73). The answer format was a frequency scale with four options (never, sometimes, often, and always). An English version of the items is available via e-mail on request. 2.3. Procedure Initially, an e-mail was sent to a large number of agencies providing services to people with intellectual and developmental disabilities throughout Spain. In the e-mail, we indicated the goals of the study and asked for collaboration. This procedure was augmented by explaining the study to participants of a number of different conferences and workshops addressed to people interested on intellectual disability. Also a recruitment form was posted on Institute on Community Integration (INICO)’s website. People expressing interest answered a survey in which they provided their contact data and the potential number of people that they might assess. We obtained a positive response from 147 agencies that were willing to potentially evaluate 4017 people. Consequently, the actual participation was 43.49% of the potential people to be assessed and 63.35% of the interested agencies. Among the reasons they used to drop out of the study were the unexpected work overload, participating in another study or assessment at that moment, and the loss of staff due to cutbacks in social service spending in Spain. Next, the research team e-mailed information to these 147 agencies providing them more detailed information about the study, as well as the way to gain access to the online version of the San Martı´n Scale, its administration manual with all the needed instructions, and the informed consent sheet that should be completed for each person prior the evaluation. In this way, informed consents were collected for all participants and the assessment was carried out guaranteeing the anonymity and confidentiality of the collected data. The Ethics Committee of the University of Salamanca granted ethical approval. Phone and e-mail contact was kept all along the process in case there were doubts or suggestions. Both were also used to remind people about deadlines when it was necessary. 3. Results 3.1. Reliability Reliability was analyzed in terms of internal consistency using corrected homogeneity index (CHI), Cronbach’s alpha, and polychoric correlations. Only five among the 120 items showed CHI below .200, and they were eliminated. Among the remaining 115 items, those 12 with the highest CHI values for each domain were retained for the final version of the scale,
M.A. Verdugo et al. / Research in Developmental Disabilities 35 (2014) 75–86
79
Table 1 Cronbach’s alpha coefficients by level of intellectual functioning.
Total Mild Moderate Severe Profound
x2(3) p
SD
EW
PW
MW
RI
PD
SI
IR
N
.884 .882 .890 .886 .870 2.934 .402
.863 .834 .866 .866 .863 2.698 .441
.821 .846 .823 .811 .823 2.690 .442
.858 .872 .860 .855 .852 1.253 .740
.832 .840 .835 .837 .810 3.059 .383
.933 .931 .936 .932 .932 0.538 .911
.904 .892 .906 .902 .908 1.488 .685
.866 .860 .889 .889 .884 3.215 .340
1770 154 500 738 378 – –
Note: SD = self-determination; EW = emotional wellbeing; PW = Physical wellbeing; MW = material wellbeing; RI = rights; PD = personal development; SI = social inclusion; IR = interpersonal relationships.
with the only exception of social inclusion that retained only 11. In this way, the final version of the San Martı´n Scale kept 95 items, with CHI values ranging from .332 to .676 (all of them had positive valence). The mean polychoric correlations among the items composing each domain ranged between .444 and .650, while Cronbach’s alpha fluctuated between .821 (physical well-being) and .933 (personal development). The Cronbach’s alpha coefficient for the total scale (i.e., 95 items) was .974. Given that the target population of this scale was those people with intellectual and developmental disabilities with lowest levels of functioning and, as it was aforementioned in the description of the sample, some of the people assessed were considered by the observers as showing mild or moderate intellectual disability, reliability was also analyzed taking into account the intellectual disability level with the goal of checking if there were significant differences between the internal consistency coefficients found for the different groups (i.e., mild, moderate, severe, profound). The results of the test of significance of the differences between independent coefficient alphas (Hakstian & Whalen, 1976) showed that internal consistency differences among groups were very small and non-significant (Table 1). 3.2. Construct validity 3.2.1. Based on the internal structure With the goal of providing evidences of validity based on the internal structure of the scale, a Confirmatory Factor Analysis (CFA) was used to evaluate the goodness-of-fit for three measurement models. Model I was unidimensional. Model II contained the eight inter-correlated factors proposed by Schalock and Verdugo. Model III hypothesized a hierarchical structure with eight first-order domains (the ones proposed by Schalock and Verdugo) and a second-order one (i.e., quality of life). Given the nature of the data, the CFA was performed implementing DWLS (Diagonally Weighted Least Squares) estimation method with the covariance and asymptotic variance–covariance matrices. LISREL 9.1 was the software used. In spite of the use of parcels is considered by some authors as a controversial practice (Little, Cunningham, Shahar, & Widaman, 2002), among the reasons to use them stand out that parcels are closer to multivariate normality than the original set of item scores and reduce models complexity (Bandalos, 2002; Bandalos & Finney, 2001; Hall, Snell, & Singer Foust, 1999). Given the high number of items in each domain, three parcels composed of four items (i.e., a total of 24 parcels) were used as indicators of the latent constructs for each quality of life domain by combining individual items and using them as the observed variables. The items were assigned to each parcel depending on their opposite skew (Temperaal, Schim, & Gijselaers, 2007) (i.e., the most and less asymmetric items were assigned to the first parcel; the next most and less asymmetric were allocated in the second one, and so on), a recommended practice with continuous or ordered categorical items (Holt, 2004). Given that item parceling should only be carried out if a clearly defined unidimensional structure has been identified (Bandalos & Finney, 2001; Little et al., 2002), unidimensionality of each parcel was guaranteed through Exploratory Factor Analysis (EFA) and Horn’s Parallel Analysis – one of the most recommendable rules for determining the number of components to retain that compares the observed eigenvalues with those obtained from uncorrelated normal variables using a Monte-Carlo simulation method. A factor is retained if the associated eigenvalue is larger than the 95th of the distribution of eigenvalues derived from the random data. In Fig. 2, it can be seen that first empirical eigenvalues were bigger than the random ones for the 24 parcels, and the second empirical eigenvalues were lower than those random generated. Parcels were also considered unidimensional because: (a) the eigenvalue for the first factor was large relative to the eigenvalue of the second factor (Reise, Moore, & Haviland, 2010), ratios ranged between 1.914 (parcel 1 in physical wellbeing) to 5.379 (parcel 1 in social inclusion); and (b) the proportion of variance explained by the first factor was greater than 40% (range from 42.62% to 69.26%). As it is shown in Table 2, all these criteria were fulfilled and supported the appropriateness of conducting CFA. Finally, another required condition for a model to be estimated is that there are more observations than parameters to be estimated, so measured variables were identified and it was confirmed that the three models (Models I, II, and III) were overidentified (df = 252, 224, and 244, respectively). According to the recommendations of Hu and Bentler (1999) and addressing the limitations of Chi-square test (x2) (e.g., it is greatly affected by sample size), model fit was evaluated using a combination of absolute and incremental goodness-of-fit
M.A. Verdugo et al. / Research in Developmental Disabilities 35 (2014) 75–86
80
Fig. 2. Parallel analysis results with empirical and random eigenvalues.
Table 2 Parcel unidimensionality. Domain
Parcel
Eig1
Eig2
E1/E2
Dif_Eig1
Dif_Eig2
Var expl 18
Var expl 28
Dif_var
SD
SD1 SD2 SD3 EW1 EW 2 EW 3 PW1 PW12 PW13 MW1 MW2 MW3 RI1 RI2 RI3 PD1 PD2 PD3 SI1 SI2 SI3 IR1 IR2 IR3
1.945 2.079 2.576 2.335 1.992 2.510 1.820 1.993 1.705 2.030 1.886 1.781 1.857 1.868 2.266 2.634 2.447 2.715 2.770 2.119 1.864 2.287 2.282 2.141
0.897 0.766 0.582 0.805 0.868 0.746 0.951 0.889 0.847 0.773 0.875 0.892 0.942 0.846 0.748 0.652 0.673 0.599 0.515 0.891 0.721 0.716 0.681 0.976
2.168 2.714 4.426 2.901 2.295 3.365 1.914 2.242 2.013 2.626 2.155 1.997 1.971 2.208 3.029 4.040 3.636 4.533 5.379 2.378 2.585 3.194 3.351 2.194
0.896 1.030 1.527 1.286 0.943 1.461 0.771 0.944 0.656 0.981 0.837 0.732 0.808 0.819 1.217 1.585 1.398 1.666 1.721 1.070 0.815 1.238 1.233 1.092
0.152 0.283 0.467 0.244 0.181 0.303 0.098 0.160 0.202 0.276 0.174 0.157 0.107 0.203 0.301 0.397 0.376 0.450 0.534 0.158 0.328 0.333 0.368 0.073
48.625 51.986 64.403 58.382 49.793 62.746 45.491 49.824 42.623 50.751 47.143 44.515 46.434 46.708 56.643 65.840 61.185 67.867 69.260 52.966 62.127 57.166 57.052 53.516
22.420 19.160 14.538 20.117 21.690 18.652 23.779 22.226 21.174 19.324 21.877 22.297 23.560 21.147 18.700 16.310 16.830 14.966 12.870 22.273 24.020 17.907 17.035 24.401
26.205 32.826 49.865 38.265 28.103 44.094 21.712 27.598 21.449 31.427 25.266 22.218 22.874 25.561 37.943 49.530 44.355 52.901 56.390 30.693 38.107 39.259 40.017 29.115
EW
PW
MW
RI
PD
SI
IR
Note: SD = self-determination; EW = emotional wellbeing; PW = Physical wellbeing; MW = material wellbeing; RI = rights; PD = personal development; SI = social inclusion; IR = interpersonal relationships; Dif_Eig1 = difference between the eigenvalue for the first factor and the first value random obtained; Dif_Eig2 = difference between eigenvalue for the second factor and the first value random obtained.
M.A. Verdugo et al. / Research in Developmental Disabilities 35 (2014) 75–86
81
Table 3 Goodness of fit indices.
2
S-Bx p df RMSEA (90% CI) CFI TLI SRMR McDonald’s Omega Composite reliability Average variance extracted
Model I
Model II
Model III
7508.356 .000 252 .066 .064–.069 .953 .949 .087 .971 .971 .588
2676.694 .000 224 .054 .051–.057 .984 .981 .045 .985 .986 .740
5745.080 .000 244 .132 .130–.135 .965 .960 .253 .964 .973 .644
Note: S-Bx2 = Satorra–Bentler adjusted Chi-square; RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker–Lewis index; SRMR = standardized root mean square residual.
Table 4 Correlations among the eight domains (Model II).
SD EW PW MW RI PD SI IR
SD
EM
PW
MW
RI
PD
SI
IR
1.000 .823 .660 .727 .682 .802 .682 .814
1.000 .848 .832 .815 .801 .650 .887
1.000 .830 .706 .663 .592 .735
1.000 .701 .709 .636 .821
1.000 .657 .502 .720
1.000 .732 .887
1.000 .782
1.000
Note: SD = self-determination; EW = emotional wellbeing; PW = Physical wellbeing; MW = material wellbeing; RI = rights; PD = personal development; SI = social inclusion; IR = interpersonal relationships.
indices (McDonald & Ho, 2002). Regarding Chi-square analysis, it is recommended to examine its magnitude rather than its level of significance (e.g., Barrett, 2007; Flora & Curran, 2004; Iacobucci, 2010; Kline, 2010). For this reason, Satorra–Bentler scaled Chi-square (Satorra & Bentler, 2010) was used since it is a correction that allows data to more closely approximate the Chi-square distribution. With regard to relative fit indexes, Root Mean Square Error of Approximation (RMSEA), Comparative Fix Index (CFI), Tucker-Lewis Index (TLI), and Standardized Root Mean Square Residual (SRMR) measure the proportionate improvement in fit by comparing a target model with a more restricted nested baseline model (a null model in which all the observed variables are uncorrelated). Model fit is indicated by Chi-square small values and non-significant levels; RMSEA values less than .060; CFI and TLI values above .950; and SRMR values less than .080 (Browne & Cudeck, 1993). On the other hand, internal consistency of each model was calculated through McDonald’s Omega, composite reliability (rc) (also known as construct reliability), and the average variance extracted (rv). McDonald’s Omega and composite reliability (rc) are similar to Cronbach’s alpha coefficients with values greater than .700 signifying that indicators are a reliable measure of the latent variable. The average variance extracted (rv), however, indicates the accuracy in which the construct is measured (i.e., validity). Hair, Ringle, and Sarstedt (2011) recommend values greater than .500. As it is shown in Table 3, the goodness-of-fit indexes of the eight-factor model – the one hypothesized to provide the best fit to the data (Model II) – were acceptable with the exception of x2. The eight-factor solution had much better indexes than the unidimensional model (Model I) and the hierarchical model (Model III). Despite Model I and Model III obtained CFI and TLI values that indicate a good model fit (Model I .950; Model III .960), all the values were smaller to the ones found for Model II (>.980). On the other hand, both Model I and Model III showed an increase of SRMR and RMSEA: both values were close to show a good model fit in the case of Model I (SRMR = .087; RMSEA = .066) but far in the case of Model III (SRMR = .253; RMSEA = .244). In contrast, both RMSEA (
Contents lists available at ScienceDirect
Research in Developmental Disabilities
Measuring quality of life in people with intellectual and multiple disabilities: Validation of the San Martı´n scale Miguel A. Verdugo a, Laura E. Go´mez Robert L. Schalock e
b,
*, Benito Arias c, Patricia Navas d,
a
Institute on Community Integration (INICO), University of Salamanca, Avda. de la Merced, 109-131, 37005 Salamanca, Spain Department of Psychology, University of Oviedo, Plaza Feijoo s/n, 33003 Oviedo, Spain Department of Psychology, University of Valladolid, Paseo de Bele´n 1, Campus Miguel Delibes, 47011 Valladolid, Spain d Nisonger Center, The Ohio State University, 357F McCampbell Hall, 1581 Dodd Drive, Columbus, OH 43210, United States e Hastings College, Nebraska, 710 Turner Avenue, Hastings, NE 68901, United States b c
A R T I C L E I N F O
A B S T R A C T
Article history: Received 18 September 2013 Received in revised form 19 October 2013 Accepted 21 October 2013 Available online 15 November 2013
Although there are numerous quality of life instruments in the, field of intellectual disability, most of them are addressed to those, people with the highest levels of functioning, while only a few are, suitable for people with the lowest levels (i.e., people with profound, and severe intellectual disabilities, or people with intellectual and, developmental disabilities and other significant medical conditions or, disabilities). This study provides reliability and validity evidence of, the San Martı´n Scale, a 95-item Likert scale questionnaire that is, completed by a third-party respondent. The validation sample was composed, of 1770 people from Spain with intellectual and developmental, disabilities that showed extensive or pervasive support needs (8.7% had, mild intellectual disability, 28.25% moderate, 41.6% severe, and 21.4%, profound). The age of the participants ranged between 16 and 77 years old, (M = 7.78; SD = 12.32). The results suggested that the eight quality of, life domains assessed on the scale are reliable (Cronbach’s alpha ranging, from .821 to .933). Confirmatory Factor Analyses provided construct, validity evidences related to the internal structure of the San Martı´n, Scale, and indicated that the eight first-order factor solution provided, the best fit to the data over unidimensional and hierarchical solutions. Implications of these findings and guidelines for further research are, discussed. ß 2013 Elsevier Ltd. All rights reserved.
Keywords: Profound intellectual and multiple disabilities Intellectual, disability Psychometric evaluation Validation Reliability Measurement Developmental disabilities
1. Introduction Personal outcomes are important measures in the fields of education, health care, and social services that are being used not only for enhancing person well-being (Claes, van Hove, Vandevelde, van Loon, & Schalock, 2012; van Loon et al., 2013) but also becoming very useful for assessing the effectiveness of intervention programs (Go´mez, Verdugo, Arias, Navas, & Schalock, 2013; Schalock, Verdugo, & Go´mez, 2011; Schalock & Verdugo, 2012a, 2012b). Personal outcomes are typically referenced to eight core quality of life domains that reflect an individual’s self-determination (SD), emotional well-being
* Corresponding author at: Faculty of Psychology, University of Oviedo, Plaza Feijoo s/n, 33003 Oviedo, Spain. Tel.: +34 985 10 46 95. E-mail addresses: [email protected] (M.A. Verdugo), [email protected] (L.E. Go´mez), [email protected] (B. Arias), [email protected] (P. Navas), [email protected] (R.L. Schalock). 0891-4222/$ – see front matter ß 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ridd.2013.10.025
76
M.A. Verdugo et al. / Research in Developmental Disabilities 35 (2014) 75–86
(EW), physical well-being (PW), material well-being (MW), personal development (PD), rights (RI), social inclusion (SI), and interpersonal relationships (IR) (Schalock & Verdugo, 2002). These domains can be assessed either through self-reports, reports from other people, or both. Self-report forms assess the individual’s self-perception of his/her status on the respective personal outcome and reflect the values underlying the quality of life concept (e.g., inclusion, empowerment, equity, and self-determination) as well as the principles underlying the disability rights movement (Navas, Go´mez, Verdugo, & Schalock, 2012; Verdugo, Navas, Go´mez, & Schalock, 2012). The report of others assesses the respondent’s perception about the person’s status on the respective personal outcome. Though the most desirable measurement is that involving both kinds of reports (Beadle-Brown, Murphy, & DiTerlizzi, 2009; Schalock et al., 2002; van Loon, van Hove, Schalock, & Claes, 2008; Verdugo, Go´mez, & Arias, 2007; Verdugo et al., 2013), it is not always possible to get reliable and valid self-reports from those people with the lowest levels of functioning (Finlay & Lyons, 2002; Kober & Eggleton, 2002; McGillivray, Lau, Cummins, & Davey, 2009) and highest support needs (Petry & Maes, 2007; Petry, Maes, & Vlaskamp, 2010). For this reason, reports from others (such as relatives, professionals, or caretakers) are frequently used (e.g., Sines, Hogard, & Ellis, 2012; Vos, De Cock, Petry, Van den Noortgate, & Maes, 2010; Wong, Wong, Schalock, & Chou, 2011). Moreover, others who are close to people with severe and profound disabilities often make major life decisions on their behalf. Therefore, it is important to know what the perceptions are of professionals, relatives, and other significant people (i.e., those involved in providing supports), given that concordance between the formers and the judgments of people with intellectual and developmental disabilities is often low to moderate (Claes et al., 2012; Cummins, 2002; Golubovic & Sˇkrbic´, 2013; Janssen, Schuengel, & Stolk, 2005; Kane et al., 2005; Mcvilly, Burton-Smit, & Davidson, 2000; Perry & Felce, 2002; Schmidt et al., 2010; Shipman, Sheldrick, & Perrin, 2011). Although there are a very considerable number of instruments to assess quality of life for people with intellectual and developmental disabilities, almost none of them are suitable for those with the lowest levels of functioning. This is due to significant limitations in adaptive behavior (Arias, Verdugo, Navas, & Go´mez, 2013; Belva & Matson, 2013; Matson, Dixon, Matson, & Logan, 2005; Matson, Cooper, Malone, & Moskow, 2008; Navas, Verdugo, Arias, & Go´mez, 2012; Schuchardt, Maehler, & Hasselhorn, 2011; Tasse´, 2013), or other significant conditions related to language limitation, significant motor dysfunctions (Nakken & Vlaskamp, 2007), chronic and pain-related medical conditions (van der Putten & Vlaskamp, 2011), challenging behaviors (Gerber et al., 2011; Matson & Boisjoli, 2007; Matson & Minshawi, 2007; Matson & Neal, 2009; Matson et al., 2011; Poppes, van der Putten, & Vlaskamp, 2010), sensory impairments (Meule et al., 2013), or mental health problems (Horovitz et al., 2011; Kozlowski, Matson, Sipes, Hattier, & Bamburg, 2011; Morisse, Vandemaele, Claes, Claes, & Vandevelde, 2013). In addition to the lack of suitable scales to use for people with severe and profound limitations, a systematic review of self-reported quality of life measures for people with intellectual disability (Li, Tsoi, Zhang, Chen, & Wang, 2013) identified only nine instruments (in English) that measured domains or indicators that aligned with the quality of life construct currently used widely in the field. Data with regard to internal consistency was available for the nine instruments but only four of them had an excellent overall rating (lower when internal consistency was analyzed for every domain). None of the instruments applied confirmatory factor analysis or more advanced measurement methods such as item response theory. Another recent systematic review of the existent quality of life scales – also focused only on English available tools – showed that only six were valued as psychometrically sound and none of them specifically assessed quality of life for those who exhibit challenging behaviors (Townsend-White, Pham, & Vassos, 2012). According to both reviews, it can be concluded that, although there is a wealth of instruments to assess quality of life for people with intellectual disability, most of them are not well validated or not related to a clearly articulated quality of life theory. In addition, few are suitable for use regarding to the assessment of people with the lowest levels of functioning (Lyons, 2005; Petry, Maes, & Vlaskamp, 2009; Petry, Kuppens, Vos, & Maes, 2010; Ross & Oliver, 2003; Vos et al., 2010). There are several quality of life instruments available in Spanish that have been recently developed specifically for people with intellectual and developmental disabilities: (a) the INTEGRAL Scale (Go´mez, Arias, Verdugo, & Navas, 2012), for those who are able to communicate and self-report but with psychometric limitations that need to be solved (it includes both selfreport and report of others); (b) the INICO-FEAPS Scale (Verdugo et al., 2013), an instrument with the same goals and addressed to the same target population than the previous one but overcoming its limitations (self-report and report of others); (c) the GENCAT Scale (Verdugo, Arias, Go´mez, & Schalock, 2010), addressed to social service recipients, including people with intellectual disability (report of others); and (d) the FUMAT Scale (Go´mez, Verdugo, Arias, & Navas, 2008), focused on people with disabilities in aging process (i.e., over 45 years old), but its psychometric properties have been only preliminary tested (report of others). According to professionals providing supports and services to those people with intellectual and developmental disabilities in Spain, none of the former instruments is suitable for those with the lowest levels of functioning who are frequently unable to communicate their feelings, thoughts, and preferences. The San Martı´n Scale (Verdugo et al., in press) was developed with the goal of bridging this gap and satisfying the demands of practitioners that are interested on the implementation of evidence based practices to improve the quality of life of people through the provision of supports. Its development involved the suggested steps for creating multidimensional quality of life instruments focused on the context (Verdugo, Schalock, Go´mez, & Arias, 2007) and guidelines for the construction and analysis of tests (AERA, APA, NCME, 1999; Brennan, 2006; Downing & Haladyna, 2006; Evers et al., 2013; Wilson, 2005). To develop the San Martin Scale, firstly, a pool of 276 items – organized around the eight domains (Schalock & Verdugo, 2002) – was developed after an exhaustive review of the scientific literature. With the goal of selecting the best items and
M.A. Verdugo et al. / Research in Developmental Disabilities 35 (2014) 75–86
77
providing evidences based on the content of the scale, the items were evaluated by 12 experts on quality of life (among them, directors of entities, academics and researchers, psychologists). Experts valued their applicability to people with profound intellectual disability and people with extensive and pervasive support needs. We carried out a modified Delphi method in which the panel of experts assessed not only the content but also the structure of the initial pool of items, as well as added new items and reformulations. The best 118 items by suitability, importance, sensitivity, and observability according to the experts were selected using qualitative and quantitative methods (Go´mez, Arias, Verdugo, Tasse´, & Brown, 2013). Next, given that stakeholders must be involved in the selection of items, the resultant items of the Delphi study were reviewed by a focus group formed by nine direct-care professionals that worked with the target population at Fundacio´n Obra San Martı´n – a Spanish agency that provides support and services to people with intellectual and developmental disabilities. They assessed again the suitability, importance, sensitivity, and observability of the 118 items, as well as valued if the items were assessing the domain in which they were included, and if there was any indicator or important aspect not represented by the selected items. The participants in the focus group considered that all items were suitable, important, sensible, and observable, as well as well located in the domain they had been assigned to. They also reformulated some items, made some clarifications in order to facilitate the understanding of some items, and added two items in the material well-being domain with the goal of assessing a missing indicator (i.e., conditions of the center providing supports). The above process resulted in a 120 items field-test version of the scale whose validation is the object of the present study. Both reliability and validity evidences based on the internal structure of the scale will be provided. The evaluation of reliability includes Cronbach’s alpha indexes and means of polychoric correlations, while Confirmatory Factor Analysis (CFA) evaluated validity comparing the goodness-of-fit to the data of three alternative models that have been pointed out in the recent literature (Go´mez, Verdugo, Arias, & Arias, 2010): (a) a unidimensional model: quality of life is composed of only one generic domain (although there is a broad consensus about the multidimensionality of the construct, the structure will be checked since structural equation modeling emphasizes model parsimony) (Model I); (b) the hypothesized eight-correlated factors (Schalock & Verdugo, 2002) that was on the basis of the development of the scale (Model II); and (c) a hierarchical structure in which these eight domains are first-order factors with the existence of a second-order one representing a generic quality of life domain (Wang, Schalock, Verdugo, & Jenaro, 2010) (Model III). 2. Method 2.1. Participants Two selection criteria were employed. In reference to those providing information (i.e., report of others) informants could be professionals, relatives or proxies who knew well the assessed person for at least three months, and had the recent opportunity (i.e., within the last month) to observe him/her in different contexts and during prolonged periods of time (i.e., several hours a day). In reference to the person being assessed, this individual needed to: (a) show an intellectual or developmental disability and extensive or pervasive support needs (i.e., people with intellectual disability and a low level of functioning, for instance, due to a profound or severe intellectual disability, very significant limitations in adaptive behavior, multiple disabilities, chronic and severe health conditions, or mental health problems); (b) be currently receiving supports and services; and (c) be 16 years old or older and not currently engaged in the education system. The field-test version of the San Martı´n Scale was applied to a convenience sample composed of 1770 people who met the above criteria. The assessment was carried out by 399 people, most of whom (97.4%) were professionals working at 99 agencies that provided support to people with intellectual and developmental disabilities located throughout Spain. Other respondents were parents (n = 36; 2%), siblings (n = 6; 0.3%), and a guardian (0.1%). Each respondent completed the instrument on an average of four people, and an average of 18 people was assessed at each agency. People that completed the assessment had known the person for more than two years in most of the cases (85.1%), more than a half (54.2%) knew the person for more than five years, and 30.9% for between two and five years. Professionals who knew the person for between six months and a year answered the scale for 6.4% of the participants, while those for 3–6 months evaluated only 1.4% of the cases. The great majority (92.8%) had a frequency of contact with the assessed person of several times per week. With regard to the people with intellectual and developmental disabilities assessed, the number of men (n = 993; 56.1%) was lightly higher than the number of women (n = 777; 43.9%). The age of the participants ranged between 16 and 77 years old (M = 37.78; SD = 12.32). The analysis of the Pearson standardized residuals showed that the proportions of men and women by age groups were equiprobable with the only exception of a slight underrepresentation of those women under 28 (x2ð3Þ ¼ 14:658; p = .002). All clients in the sample required extensive (45.3%) or pervasive (54.7%) support needs. Although there was no specific measurement of intellectual and adaptive functioning available for most of the people, respondents estimated that 8.7% had mild intellectual disability, 28.25% moderate, 41.6% severe, and 21.4% profound. In addition, 91.6% had other associated verified conditions, such as epilepsy, physical disability or challenging behaviors (see Fig. 1). A third (34.3%) of participants had only one associated condition, 16% had two concurrent associated conditions, 16% showed three conditions, 6% presented four, and the rest of the sample had between five and seven (M = 1.85; SD = 1.25). Most of them (74.2%) were taking medication, especially antiepileptic/anti-seizures and anxiolytics.
78
M.A. Verdugo et al. / Research in Developmental Disabilities 35 (2014) 75–86
Fig. 1. Percent of persons with other associated conditions.
2.2. Instrument The field-test version of the San Martı´n Scale was a self-administered questionnaire in which a third-party respondent (i.e., staff, relative, proxy, etc. who knows well and has opportunities to observe the person assessed) must answer questions about the person’s quality of life. Administration time varied from 20 and 40 min. It was composed of eight subscales that correspond to the eight quality of life domains by Schalock and Verdugo, and a total of 120 items (SD = 12; EW = 16; PW = 14; MW = 17; RI = 16; PD = 16; SI = 11; IR = 18). All items were formulated as third person declarative statements and random organized by domains. All items had positive valence with the only exception of five (i.e., EW15, EW20, EW22, RI63, and RI73). The answer format was a frequency scale with four options (never, sometimes, often, and always). An English version of the items is available via e-mail on request. 2.3. Procedure Initially, an e-mail was sent to a large number of agencies providing services to people with intellectual and developmental disabilities throughout Spain. In the e-mail, we indicated the goals of the study and asked for collaboration. This procedure was augmented by explaining the study to participants of a number of different conferences and workshops addressed to people interested on intellectual disability. Also a recruitment form was posted on Institute on Community Integration (INICO)’s website. People expressing interest answered a survey in which they provided their contact data and the potential number of people that they might assess. We obtained a positive response from 147 agencies that were willing to potentially evaluate 4017 people. Consequently, the actual participation was 43.49% of the potential people to be assessed and 63.35% of the interested agencies. Among the reasons they used to drop out of the study were the unexpected work overload, participating in another study or assessment at that moment, and the loss of staff due to cutbacks in social service spending in Spain. Next, the research team e-mailed information to these 147 agencies providing them more detailed information about the study, as well as the way to gain access to the online version of the San Martı´n Scale, its administration manual with all the needed instructions, and the informed consent sheet that should be completed for each person prior the evaluation. In this way, informed consents were collected for all participants and the assessment was carried out guaranteeing the anonymity and confidentiality of the collected data. The Ethics Committee of the University of Salamanca granted ethical approval. Phone and e-mail contact was kept all along the process in case there were doubts or suggestions. Both were also used to remind people about deadlines when it was necessary. 3. Results 3.1. Reliability Reliability was analyzed in terms of internal consistency using corrected homogeneity index (CHI), Cronbach’s alpha, and polychoric correlations. Only five among the 120 items showed CHI below .200, and they were eliminated. Among the remaining 115 items, those 12 with the highest CHI values for each domain were retained for the final version of the scale,
M.A. Verdugo et al. / Research in Developmental Disabilities 35 (2014) 75–86
79
Table 1 Cronbach’s alpha coefficients by level of intellectual functioning.
Total Mild Moderate Severe Profound
x2(3) p
SD
EW
PW
MW
RI
PD
SI
IR
N
.884 .882 .890 .886 .870 2.934 .402
.863 .834 .866 .866 .863 2.698 .441
.821 .846 .823 .811 .823 2.690 .442
.858 .872 .860 .855 .852 1.253 .740
.832 .840 .835 .837 .810 3.059 .383
.933 .931 .936 .932 .932 0.538 .911
.904 .892 .906 .902 .908 1.488 .685
.866 .860 .889 .889 .884 3.215 .340
1770 154 500 738 378 – –
Note: SD = self-determination; EW = emotional wellbeing; PW = Physical wellbeing; MW = material wellbeing; RI = rights; PD = personal development; SI = social inclusion; IR = interpersonal relationships.
with the only exception of social inclusion that retained only 11. In this way, the final version of the San Martı´n Scale kept 95 items, with CHI values ranging from .332 to .676 (all of them had positive valence). The mean polychoric correlations among the items composing each domain ranged between .444 and .650, while Cronbach’s alpha fluctuated between .821 (physical well-being) and .933 (personal development). The Cronbach’s alpha coefficient for the total scale (i.e., 95 items) was .974. Given that the target population of this scale was those people with intellectual and developmental disabilities with lowest levels of functioning and, as it was aforementioned in the description of the sample, some of the people assessed were considered by the observers as showing mild or moderate intellectual disability, reliability was also analyzed taking into account the intellectual disability level with the goal of checking if there were significant differences between the internal consistency coefficients found for the different groups (i.e., mild, moderate, severe, profound). The results of the test of significance of the differences between independent coefficient alphas (Hakstian & Whalen, 1976) showed that internal consistency differences among groups were very small and non-significant (Table 1). 3.2. Construct validity 3.2.1. Based on the internal structure With the goal of providing evidences of validity based on the internal structure of the scale, a Confirmatory Factor Analysis (CFA) was used to evaluate the goodness-of-fit for three measurement models. Model I was unidimensional. Model II contained the eight inter-correlated factors proposed by Schalock and Verdugo. Model III hypothesized a hierarchical structure with eight first-order domains (the ones proposed by Schalock and Verdugo) and a second-order one (i.e., quality of life). Given the nature of the data, the CFA was performed implementing DWLS (Diagonally Weighted Least Squares) estimation method with the covariance and asymptotic variance–covariance matrices. LISREL 9.1 was the software used. In spite of the use of parcels is considered by some authors as a controversial practice (Little, Cunningham, Shahar, & Widaman, 2002), among the reasons to use them stand out that parcels are closer to multivariate normality than the original set of item scores and reduce models complexity (Bandalos, 2002; Bandalos & Finney, 2001; Hall, Snell, & Singer Foust, 1999). Given the high number of items in each domain, three parcels composed of four items (i.e., a total of 24 parcels) were used as indicators of the latent constructs for each quality of life domain by combining individual items and using them as the observed variables. The items were assigned to each parcel depending on their opposite skew (Temperaal, Schim, & Gijselaers, 2007) (i.e., the most and less asymmetric items were assigned to the first parcel; the next most and less asymmetric were allocated in the second one, and so on), a recommended practice with continuous or ordered categorical items (Holt, 2004). Given that item parceling should only be carried out if a clearly defined unidimensional structure has been identified (Bandalos & Finney, 2001; Little et al., 2002), unidimensionality of each parcel was guaranteed through Exploratory Factor Analysis (EFA) and Horn’s Parallel Analysis – one of the most recommendable rules for determining the number of components to retain that compares the observed eigenvalues with those obtained from uncorrelated normal variables using a Monte-Carlo simulation method. A factor is retained if the associated eigenvalue is larger than the 95th of the distribution of eigenvalues derived from the random data. In Fig. 2, it can be seen that first empirical eigenvalues were bigger than the random ones for the 24 parcels, and the second empirical eigenvalues were lower than those random generated. Parcels were also considered unidimensional because: (a) the eigenvalue for the first factor was large relative to the eigenvalue of the second factor (Reise, Moore, & Haviland, 2010), ratios ranged between 1.914 (parcel 1 in physical wellbeing) to 5.379 (parcel 1 in social inclusion); and (b) the proportion of variance explained by the first factor was greater than 40% (range from 42.62% to 69.26%). As it is shown in Table 2, all these criteria were fulfilled and supported the appropriateness of conducting CFA. Finally, another required condition for a model to be estimated is that there are more observations than parameters to be estimated, so measured variables were identified and it was confirmed that the three models (Models I, II, and III) were overidentified (df = 252, 224, and 244, respectively). According to the recommendations of Hu and Bentler (1999) and addressing the limitations of Chi-square test (x2) (e.g., it is greatly affected by sample size), model fit was evaluated using a combination of absolute and incremental goodness-of-fit
M.A. Verdugo et al. / Research in Developmental Disabilities 35 (2014) 75–86
80
Fig. 2. Parallel analysis results with empirical and random eigenvalues.
Table 2 Parcel unidimensionality. Domain
Parcel
Eig1
Eig2
E1/E2
Dif_Eig1
Dif_Eig2
Var expl 18
Var expl 28
Dif_var
SD
SD1 SD2 SD3 EW1 EW 2 EW 3 PW1 PW12 PW13 MW1 MW2 MW3 RI1 RI2 RI3 PD1 PD2 PD3 SI1 SI2 SI3 IR1 IR2 IR3
1.945 2.079 2.576 2.335 1.992 2.510 1.820 1.993 1.705 2.030 1.886 1.781 1.857 1.868 2.266 2.634 2.447 2.715 2.770 2.119 1.864 2.287 2.282 2.141
0.897 0.766 0.582 0.805 0.868 0.746 0.951 0.889 0.847 0.773 0.875 0.892 0.942 0.846 0.748 0.652 0.673 0.599 0.515 0.891 0.721 0.716 0.681 0.976
2.168 2.714 4.426 2.901 2.295 3.365 1.914 2.242 2.013 2.626 2.155 1.997 1.971 2.208 3.029 4.040 3.636 4.533 5.379 2.378 2.585 3.194 3.351 2.194
0.896 1.030 1.527 1.286 0.943 1.461 0.771 0.944 0.656 0.981 0.837 0.732 0.808 0.819 1.217 1.585 1.398 1.666 1.721 1.070 0.815 1.238 1.233 1.092
0.152 0.283 0.467 0.244 0.181 0.303 0.098 0.160 0.202 0.276 0.174 0.157 0.107 0.203 0.301 0.397 0.376 0.450 0.534 0.158 0.328 0.333 0.368 0.073
48.625 51.986 64.403 58.382 49.793 62.746 45.491 49.824 42.623 50.751 47.143 44.515 46.434 46.708 56.643 65.840 61.185 67.867 69.260 52.966 62.127 57.166 57.052 53.516
22.420 19.160 14.538 20.117 21.690 18.652 23.779 22.226 21.174 19.324 21.877 22.297 23.560 21.147 18.700 16.310 16.830 14.966 12.870 22.273 24.020 17.907 17.035 24.401
26.205 32.826 49.865 38.265 28.103 44.094 21.712 27.598 21.449 31.427 25.266 22.218 22.874 25.561 37.943 49.530 44.355 52.901 56.390 30.693 38.107 39.259 40.017 29.115
EW
PW
MW
RI
PD
SI
IR
Note: SD = self-determination; EW = emotional wellbeing; PW = Physical wellbeing; MW = material wellbeing; RI = rights; PD = personal development; SI = social inclusion; IR = interpersonal relationships; Dif_Eig1 = difference between the eigenvalue for the first factor and the first value random obtained; Dif_Eig2 = difference between eigenvalue for the second factor and the first value random obtained.
M.A. Verdugo et al. / Research in Developmental Disabilities 35 (2014) 75–86
81
Table 3 Goodness of fit indices.
2
S-Bx p df RMSEA (90% CI) CFI TLI SRMR McDonald’s Omega Composite reliability Average variance extracted
Model I
Model II
Model III
7508.356 .000 252 .066 .064–.069 .953 .949 .087 .971 .971 .588
2676.694 .000 224 .054 .051–.057 .984 .981 .045 .985 .986 .740
5745.080 .000 244 .132 .130–.135 .965 .960 .253 .964 .973 .644
Note: S-Bx2 = Satorra–Bentler adjusted Chi-square; RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker–Lewis index; SRMR = standardized root mean square residual.
Table 4 Correlations among the eight domains (Model II).
SD EW PW MW RI PD SI IR
SD
EM
PW
MW
RI
PD
SI
IR
1.000 .823 .660 .727 .682 .802 .682 .814
1.000 .848 .832 .815 .801 .650 .887
1.000 .830 .706 .663 .592 .735
1.000 .701 .709 .636 .821
1.000 .657 .502 .720
1.000 .732 .887
1.000 .782
1.000
Note: SD = self-determination; EW = emotional wellbeing; PW = Physical wellbeing; MW = material wellbeing; RI = rights; PD = personal development; SI = social inclusion; IR = interpersonal relationships.
indices (McDonald & Ho, 2002). Regarding Chi-square analysis, it is recommended to examine its magnitude rather than its level of significance (e.g., Barrett, 2007; Flora & Curran, 2004; Iacobucci, 2010; Kline, 2010). For this reason, Satorra–Bentler scaled Chi-square (Satorra & Bentler, 2010) was used since it is a correction that allows data to more closely approximate the Chi-square distribution. With regard to relative fit indexes, Root Mean Square Error of Approximation (RMSEA), Comparative Fix Index (CFI), Tucker-Lewis Index (TLI), and Standardized Root Mean Square Residual (SRMR) measure the proportionate improvement in fit by comparing a target model with a more restricted nested baseline model (a null model in which all the observed variables are uncorrelated). Model fit is indicated by Chi-square small values and non-significant levels; RMSEA values less than .060; CFI and TLI values above .950; and SRMR values less than .080 (Browne & Cudeck, 1993). On the other hand, internal consistency of each model was calculated through McDonald’s Omega, composite reliability (rc) (also known as construct reliability), and the average variance extracted (rv). McDonald’s Omega and composite reliability (rc) are similar to Cronbach’s alpha coefficients with values greater than .700 signifying that indicators are a reliable measure of the latent variable. The average variance extracted (rv), however, indicates the accuracy in which the construct is measured (i.e., validity). Hair, Ringle, and Sarstedt (2011) recommend values greater than .500. As it is shown in Table 3, the goodness-of-fit indexes of the eight-factor model – the one hypothesized to provide the best fit to the data (Model II) – were acceptable with the exception of x2. The eight-factor solution had much better indexes than the unidimensional model (Model I) and the hierarchical model (Model III). Despite Model I and Model III obtained CFI and TLI values that indicate a good model fit (Model I .950; Model III .960), all the values were smaller to the ones found for Model II (>.980). On the other hand, both Model I and Model III showed an increase of SRMR and RMSEA: both values were close to show a good model fit in the case of Model I (SRMR = .087; RMSEA = .066) but far in the case of Model III (SRMR = .253; RMSEA = .244). In contrast, both RMSEA (
