INTELLECTUAL AND DEVELOPMENTAL DISABILITIES

’AAIDD

2015, Vol. 53, No. 2, 129–142

DOI: 10.1352/1934-9556-53.2.129

The Portuguese Version of Personal Outcomes Scale: A Psychometric Validity and Reliability Study Cristina Simo˜es, Sofia Santos, and Claudia Claes

Abstract This article presents the results of a pilot study to evaluate the psychometric properties of the Escala Pessoal de Resultados (EPR), the Portuguese version of the Personal Outcomes Scale (POS), to assess quality of life (QOL) in people with intellectual disability (ID) in Portugal. We present the adaptation and validation processes of the EPR. Data were collected from 1,264 adults with intermittent and limited levels of support needs and respective proxies. Results of the research are discussed in requisites of the validity and reliability. The different coefficients illustrate that this scale may be a useful addition to the QOL assessment in Portuguese people with ID. Key Words: intellectual disability; quality of life; validity; reliability; Personal Outcomes Scale; Portugal

In the last decade, considerable attention has been centered upon the quality of life (QOL) construct of the Personal Outcomes Scale (POS). There is a growing interest in the QOL’s measurement for populations with intellectual disability (ID). People with ID often experience problems concerning their participation in society and are in danger of being excluded from opportunities that typically are available to others. Therefore, adults with ID need to have greater control in their lives and should have the opportunity to voice their choices, perspectives, and concerns (Bonham et al., 2004; Schalock, Gardner, & Bradley, 2007; Shogren & Broussard, 2011; Turnbull, Turnbull, Wehmeyer, & Park, 2003). The QOL assessment is increasingly being used as a personal outcome measure, and is focused on the collection of information from the person in question about his or her own life (Bonham et al., 2004; Brown, Schalock, & Brown, 2009; Cummins, 2005; Phillips, 2006; Schalock, 2004; Schalock et al., 2007; Schalock & Verdugo, 2002; Verdugo, Schalock, Keith, & Stancliffe, 2005; Wang, Schalock, Verdugo, & Jenaro, 2010). The measurement of QOL allows for the weighting of domains and indicators according to the significance or the value assigned by each person (Schalock & Verdugo, 2002). It should also reflect each person’s individual and cultural life experiences (Claes, van Hove, van Loon, Vandevelde, & Schalock, 2010; Schalock, 2004; Schalock

C. Simo˜es, S. Santos, and C. Claes

et al., 2007; Schalock & Verdugo, 2002; van Loon, Van Hove, Schalock, & Claes, 2009). Varying numbers of QOL domains have been reported in literature; however, the number of domains is less important than the acknowledgment that each structure must include a multielement framework and that all domains must represent the QOL’s construct (Cummins, 2005; Schalock, 2004; Schalock & Verdugo, 2002; Verdugo et al., 2005). The QOL is a multidimensional concept that includes subjective and objective measures (Bonham et al., 2004; Brown et al., 2009; Claes et al., 2010; Cummins, 2005; Phillips, 2006; Schalock, 2004; Schalock et al., 2007; Schalock & Verdugo, 2002; van Loon et al., 2009; Verdugo et al., 2005; Wang et al., 2010), and is a hierarchical construct (Wang et al., 2010). Nevertheless, according to the World Health Organization (WHO), the QOL is based on subjective measures and can be understood as the satisfaction, the perception, and the interpretation by each person of the various facets of his or her own life (Skevington, Lotfy, & O’Connell, 2004; WHOQOL Group, 1998a). In the last 2 decades, many instruments have been developed for QOL assessment. This measurement is essential for research and intervention (Camfield & Skevington, 2008; Claes et al., 2010; Finlay & Lyons, 2001; Hartley & MacLean, 2006; Kayes & McPherson, 2010; Kimberlin & Winterstein, 2008; Li, Tsoi, Zhang, Chen, & Wang, 2013;

129

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES

’AAIDD

2015, Vol. 53, No. 2, 129–142

McGillivray, Lau, Cummins, & Davey, 2009; Schalock, 2004; Schwartz & Rabinovitz, 2003; van Loon et al., 2009). However, it is not clear which scale is suitable for evaluating QOL in different situations. Subjective measures (self-report) only may not be enough to assess QOL and the evaluation of individual programs (Hensel, Rose, Kroese, & BanksSmith, 2002; Schalock, 2004). As a consequence, this study investigates the validity and reliability of the Escala Pessoal de Resultados (EPR; the Portuguese version of the POS), which involves subjective and objective QOL indicators, predicting that the scale will be an appropriate measure for Portuguese people with ID. The instrument is based on eight core domains set out to assess QOL in people with ID, namely: personal development, self-determination, interpersonal relations, social inclusion, rights, emotional well-being, physical well-being, and material well-being (Schalock & Verdugo, 2002). The original scale of the EPR is the POS, which advocates international guidelines to assess QOL in people with ID (Claes et al., 2010; van Loon, van Hove, Schalock, & Claes, 2008, 2009). The POS was developed for the purpose of assessing specific indicators associated with each of the eight QOL domains that have been validated in a series of crosscultural studies (Bonham et al., 2004; Claes et al., 2010; Jenaro et al., 2005; Schalock & Verdugo, 2002; van Loon et al., 2009; Wang et al., 2010). The POS scale (in the form of the EPR) was selected because of its validity, reliability, and frequent use in studies about ID. However, to date, it has not been validated in Portuguese adults with ID. Current research emphasizes the validity and the reliability of the psychometric properties of the EPR. We hypothesized that: (1) the scale should demonstrate the construct validity through the intercorrelations between domain scores and total scores and the eight-factor model should be supported by the exploratory factor analysis (EFA); (2) the scale should demonstrate evidence of concurrent validity with a related measure of similar construct; (3) the scale should demonstrate a good test-retest reliability; (4) alpha’s coefficients and split-half should demonstrate evidence of internal consistency; and (5) interrespondent reliability coefficients should be acceptable.

Method Sample The sample comprised of 1,264 adults with intermittent (n 5 769; 60.84%) and limited levels

130

DOI: 10.1352/1934-9556-53.2.129

of support needs (n 5 495; 39.16%). The age of the participants ranged from 18 to 66 years (M 5 31.36). Regarding gender, 600 participants were female (47.47%) and 664 were male (52.53%). Living arrangements included homeownership (n 5 93; 7.36%), family home (n 5 924; 73.10%), and institutions (n 5 247; 19.54%). Eighty-seven adults with ID (6.88%) did not have an occupation (were only with the family at home), while the others worked a paid job (n 5 138; 10.92%) or were involved in professional training (n 5 481; 38.05%) or occupational activities centers (n 5 558; 44.15%). The support staff that participated for the report of others portion of the EPR (n 5 224) was connected to 45 Portuguese institutions. There was a mix of psychologists, social workers, special education teachers, and psychomotor/occupational therapists or monitors. Age ranged from 22 to 64 (M 5 41.46), and 179 were female (79.91%), while 45 were male (20.09%). The second part of the EPR (the report of others portion) also involved family members (n 5 109). Age of included family members ranged from 21 to 89 years (M 5 57.37); 74 were female (67.90%) and 35 were male (32.10%). These participants were mothers (n 5 56; 51.38%), fathers (n 5 24; 22.02%), siblings (n 5 15; 13.76%), and others members of family (n 5 14; 12.84%).

Measures The Portuguese EPR scale is divided into two parts (a self-report and a report of others). The self-report part has a set of questions to be answered directly by people with ID. The report of others part has the same indicators, but should be completed by a proxy—a family member or a caregiver who knows the person for at least 2 years. A total of 48 questions are presented in each part of the EPR, with six questions per domain. Answers on the scale are reported on a 3-point Likert type scale (i.e., frequent, sometimes, and never). Two new items about importance and satisfaction were added to EPR in the selfreport scale, which differs from the original POS. The question about importance is hierarchical (i.e., 1 corresponding to the most important domain and 8 to the lesser important). Satisfaction on each domain is rated through a 3-point Likert scale (i.e., very satisfied, not too satisfied, and unsatisfied).

Portuguese Personal Outcomes Scale

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES

’AAIDD

2015, Vol. 53, No. 2, 129–142

DOI: 10.1352/1934-9556-53.2.129

Procedures

(Verdugo et al., 2005). A content specialist reviewed the initial translation in the target language and made necessary corrections. Products were evaluated by 10 experts who agreed, after doubts and language clarification, on a pre-final version of the scale with items based on Portuguese lifestyle and culture. The experts considered all of the EPR’s questions relevant (n 5 98), but 14 items were identified as needing revision in regard to level of clarity, simplicity, and ambiguity (Simo˜es & Santos, 2014). The scores of scale content validity index-universal agreement, scale content validity index-average, and Cohen’s Kappa showed adequate evidence of the EPR’s content validity (Simo˜es & Santos, 2014). Construct validity. To evaluate the extent to which the domains of the EPR actually assess the QOL construct, Pearson’s correlations were calculated between the eight domains of the EPR and the total score and domains. The scores of Pearson’s coefficients are considered moderate (.40 to .69), high (.70 to .89), and very high (above .90; Pestana & Gageiro, 2005). As can be noted in Table 1, domain-to-domain correlations in the self-report portion showed at least one moderate correlation. The higher score was observed between the interpersonal relations and social inclusion domains (r 5 .62). The correlations between the total selfreport score and the domain scores were moderate to high. Higher Pearson’s coefficients among total QOL score and domain scores were observed in self-determination (r 5 .72), interpersonal relations (r 5 .72), social inclusion (r 5 .75), and material

Ethical approval for the study was provided by the ethic committee of Centro Hospitalar de Sa˜o Joa˜o. The study was carried out in compliance with the Helsinki Declaration (World Medical Association, 2008). Participants signed an informed consent prior to the entry of the study and were free to leave at any time. The selection of participants was based on the following criteria: formal diagnosis of intermittent or limited level of support needs of ID, and 18 years or more on age. All EPR items were administered to each participant in accordance with the guidelines of the original scale (van Loon et al., 2008). The interviewers of the self-report and report of others measures were always the same. They were qualified professionals who had been previously trained to administer the EPR. In the self-report portion, the EPR was always administered through an interview. Before the participants answered the questions, the interviewers read standard instructions and completed sociodemographic information. Statistical Package for Social Sciences (SPSS), version 21.0, was used for data analysis.

Results Validity Content validity. The cross-cultural adaptation and the content validity research were developed through five stages: (1) translation, (2) back translation, (3) comparison between the versions in Portuguese and in English, (4) content validity by group of experts, and (5) focus group Table 1 Intercorrelations of Domains (Self-Report) Domains PD SD IR SI R EW PW MW Total score

PD

SD

IR

SI

R

EW

PW

MW

.56** .38** .40** .33** .08* .05 .40** .65**

.41** .49** .35** .23** .13** .44** .72**

.62** .28** .39** .33** .38** .72**

.32** .35** .30** .37** .75**

.15** .12** .54** .62**

.48** .22** .51**

.16** .45**

.71**

Note. PD 5 Personal development; SD 5 Self-determination; IR 5 Interpersonal relations; SI 5 Social inclusion; R 5 Rights; EW 5 Emotional well-being; PW 5 Physical well-being; MW 5 Material well-being; ** p , .001; * p , .05.

C. Simo˜es, S. Santos, and C. Claes

131

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES

’AAIDD

2015, Vol. 53, No. 2, 129–142

DOI: 10.1352/1934-9556-53.2.129

well-being (r 5 .71). In the same way, significant domain-to-domain coefficients were confirmed in the report of others portion (see Table 2). Correlations among the domain scores and the total QOL scores were high on the same four domains of the self-report measure, and ranged from .44 (physical well-being) to .78 (social inclusion). The structure of the EPR was analyzed with an EFA to find out if a potential framework of the POS may emerge from the sample data on Portuguese population. We employed the maximum likelihood (ML) estimation method, with a Promax rotation. To evaluate the sampling adequacy, we used the Kaiser-Meyer-Olkin (KMO) index and the Bartlett’s test of sphericity (Pestana & Gageiro, 2005). The initial factor pattern/structure matrix, including all 48 items, showed 12 factors in the self-report portion and 11 in the report of others part. Nineteen items were removed from the self-report measure and 14 from the report of others measure. The results of the EFA showed that these unretained items had salient loadings on two or more factors (Brown, 2006), with values $ .32 on at least two factors (Costello & Osborne, 2005; Worthington & Whittaker, 2006), or failed to exceed the cut-off point of .40 (DeVon et al., 2007; Henson & Roberts, 2006; Matsunaga, 2010). The factor pattern/structure matrix is presented in Table 3 and Table 4. In the self-report portion, results of the EFA supported eight factors, accounting for 51.33% of the variance (see Table 3), and the eigenvalue of the ninth unretained factor was

1.01. The report of others portion (see Table 4) also revealed eight factors accounting for 56.62% of the overall variance. Criteria of factors retained included multiple strategies, namely: eigenvalues greater than 1, scree plot, and components loadings above .40. We removed items with low communalities and rejected a factor with fewer than three items (Brown, 2006; Costello & Osborne, 2005; Henson & Roberts, 2006; Pestana & Gageiro, 2005; Worthington & Whittaker, 2006). Although the results seem to support the eight-factor structure proposed by the POS (van Loon et al., 2008, 2009), the EFA did not confirm all items in the Portuguese sample. Concurrent validity. To determine the concurrent validity, two scales were administered to the same person, specifically: the EPR (self-report measure only) and the World Health Quality of Life-BREF (WHOQOL-BREF; Canavarro et al., 2007; Skevington et al., 2004; WHOQOL Group, 1998b). Concurrent validity evaluates the relationship between an instrument and previous standard scores obtained by another test (Claes et al., 2010; DeVon et al., 2007; Kimberlin & Winterstein, 2008; Pasquali, 2007; Schalock et al., 2007; van Loon et al., 2009). Consequently, the correlation between different dimensions of two scales shows how the same construct is measured in the same subjects (Cicchetti, 1994; DeVon et al., 2007; Kimberlin & Winterstein, 2008; Pasquali, 2007). Both scales were assessed to 216 adults with ID (M age 5 31.38; 128 with intermittent and 88 with

Table 2 Intercorrelations of Domains (Report of Others) Domains PD SD IR SI R EW PW MW Total score

PD

SD

IR

SI

R

EW

PW

MW

.52** .28** .40** .40** .13** .10* .38** .64**

.39** .48** .45** .25** .14** .46** .73**

.65** .27** .49** .37** .33** .72**

.40** .40** .28** .39** .78**

.11** .04 .55** .64**

.44** .23** .55**

15** .44**

.70**

Note. PD 5 Personal development; SD 5 Self-determination; IR 5 Interpersonal relations; SI 5 Social inclusion; R 5 Rights; EW 5 Emotional well-being; PW 5 Physical well-being; MW 5 Material well-being; ** p , .001; * p , .05.

132

Portuguese Personal Outcomes Scale

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES

’AAIDD

2015, Vol. 53, No. 2, 129–142

DOI: 10.1352/1934-9556-53.2.129

Table 3 EPR’s Factor Pattern/Structure Matrix (Self-Report)

Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8 Item 9 Item 10 Item 11 Item 12 Item 13 Item 14 Item 15 Item 16 Item 17 Item 18 Item 19 Item 20 Item 21 Item 22 Item 23 Item 24 Item 25 Item 26 Item 27 Item 28 Item 29 % of variance

Factor 1 SI

Factor 2 R

Factor 3 PD

Factor 4 MW

Factor 5 PW

Factor 6 SD

2.06 .07 .03 2.05 .01 .01 2.06 .00 .65 .26 .55 .56 .42 .14 .81 .52 2.12 .00 2.08 2.01 2.01 .01 2.02 .06 2.07 .03 .03 2.01 .05 19.62

.11 .05 2.08 2.02 .02 2.05 .01 2.06 2.07 .06 2.08 .03 .09 .06 .02 .07 .05 .97 .14 2.03 .03 2.02 .01 .05 2.05 2.03 2.03 .96 .08 7.87

.44 .54 .72 .70 2.10 .04 2.01 .14 .05 2.09 .11 2.18 .16 .15 2.10 .15 .00 .00 .06 .05 2.01 .00 .05 2.07 2.08 2.02 .19 .01 .00 5.91

.06 2.04 2.05 2.02 2.04 2.07 .05 .06 .05 .02 .00 2.01 2.04 .03 .00 2.04 .01 2.02 2.04 2.01 2.04 .03 2.02 2.07 .85 .63 .17 .00 .68 4.37

.02 2.21 .19 2.01 2.13 .07 .40 .22 2.04 2.07 2.06 .16 2.05 .39 .01 2.03 .09 .02 .03 .48 .56 .01 .03 .43 2.04 2.01 2.09 .01 .02 4.10

.21 .03 2.13 2.01 .86 .77 .12 .41 .01 .04 2.04 .01 .07 .04 2.06 .05 .02 2.02 2.05 .03 2.10 .04 .00 2.11 2.04 2.09 .01 2.01 .03 3.34

Factor 7 Factor 8 EW IR .04 .03 2.04 .02 .04 2.02 2.01 2.04 2.02 2.05 2.06 2.02 .02 2.03 .05 .00 .00 .00 .03 .01 2.06 .74 .82 .12 .01 .04 .06 2.01 2.04 3.13

2.16 2.10 .01 .19 .10 2.05 2.03 .02 .04 .40 .19 2.07 .16 2.08 2.26 .07 .43 .05 .40 2.04 .15 .01 2.01 .02 .10 .01 .40 .06 2.03 2.98

h2 .40 .30 .45 .53 .60 .60 .20 .41 .47 .27 .47 .32 .40 .32 .50 .40 .20 .95 .25 .25 .30 .57 .66 .23 .63 .36 .34 .94 .57 51.33

Note. PD 5 Personal development; SD 5 Self-determination; IR 5 Interpersonal relations; SI 5 Social inclusion; R 5 Rights; EW 5 Emotional well-being; PW 5 Physical well-being; MW 5 Material well-being; h2 5 Communality coefficient. limited levels of support needs). The WHOQOLBREF was chosen because it was the only QOL instrument standardized to the Portuguese population (Canavarro et al., 2007), and has been used in several studies. The WHOQOL-BREF was developed as a short version of the WHOQOL-100 (also validated in Portugal by Canavarro et al., 2009) to be used in situations where respondent burden must be minimized (Canavarro et al., 2007; Skevington et al., 2004; WHOQOL Group, 1998b). This scale

C. Simo˜es, S. Santos, and C. Claes

has a different QOL framework than the EPR because it consists of four domains, namely: the physical, the psychological, the social, and the environmental. While the EPR includes both subjective and objective measures (Claes et al., 2010; van Loon et al., 2008, 2009), the WHOQOL-BREF is based only on subjective indicators related to personal expectations and subjective experiences (Canavarro et al., 2007; Skevington et al., 2004; WHOQOL Group, 1998b). Table 5 summarizes the Pearson’s correlation

133

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES

’AAIDD

2015, Vol. 53, No. 2, 129–142

DOI: 10.1352/1934-9556-53.2.129

Table 4 EPR’s Factor Pattern/Structure Matrix (Report of Others)

Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8 Item 9 Item 10 Item 11 Item 12 Item 13 Item 14 Item 15 Item 16 Item 17 Item 18 Item 19 Item 20 Item 21 Item 22 Item 23 Item 24 Item 25 Item 26 Item 27 Item 28 Item 29 Item 30 Item 31 Item 32 Item 33 Item 34 % of variance

Factor 1 SI

Factor 2 SD

Factor 3 EW

Factor 4 R

Factor 5 MW

Factor 6 IR

Factor 7 PD

Factor 8 PW

h2

.02 .00 .03 2.08 .00 .03 .00 2.04 .09 .55 .74 .66 .07 .62 .62 .77 .56 2.11 .02 .03 2.01 .21 2.10 .01 .13 2.15 .29 .03 2.02 .09 .15 2.06 .01 2.01 21.13

.23 2.15 .25 .73 .87 .71 .45 .57 .71 .07 .06 .01 2.07 2.05 .01 2.07 .07 .10 .02 2.05 .10 .05 2.05 2.04 .09 2.01 2.03 2.09 2.07 2.04 .02 2.07 2.02 .26 9.73

.01 .06 .03 .04 .07 2.06 .09 2.04 2.05 .16 2.05 .03 .14 .11 2.06 .00 2.07 2.04 2.01 .01 .41 .46 .78 .76 .55 .09 2.03 .05 .08 2.01 2.08 .04 2.01 2.05 5.99

.13 2.04 2.04 .11 2.05 2.05 2.02 .04 .00 .04 2.01 .02 .03 .07 2.01 2.03 .02 .04 .98 .04 2.06 2.02 .03 2.01 2.03 .04 .01 2.03 .01 .01 2.10 .03 .93 2.01 5.28

.00 2.08 2.06 2.06 2.09 2.07 .02 .20 2.04 2.04 .03 2.05 2.12 2.03 .07 2.01 .05 2.03 .01 .00 .06 .04 .01 2.02 .07 2.05 2.18 .15 .87 .56 .17 .57 .03 .49 4.13

2.07 2.10 .17 .03 .02 .05 .03 2.02 2.13 2.17 2.07 .06 .46 .15 .08 2.05 .02 .46 .02 .57 .10 2.06 .06 .00 2.09 .04 .00 .03 2.08 2.11 .45 .09 .03 .07 3.75

.42 .84 .41 .01 2.15 2.01 2.08 .10 2.07 2.05 2.09 .03 2.18 2.10 .08 .03 .13 2.13 2.05 .09 2.16 .07 .05 .12 2.02 .16 .00 2.18 2.03 .00 .04 2.15 .02 .10 3.44

.04 .03 .10 .00 2.10 2.10 .07 2.03 .11 .00 .08 2.04 .06 2.17 2.02 2.04 .06 .10 .02 2.14 .12 2.08 .09 .02 2.03 .54 .43 .45 2.09 .18 .11 2.08 .02 .02 3.16

.38 .53 .41 .52 .57 .45 .24 .51 .50 .35 .58 .47 .30 .43 .48 .48 .48 .22 .97 .38 .35 .30 .61 .55 .39 .27 .32 .33 .65 .36 .42 .29 .91 .52 56.62

Note. PD 5 Personal development; SD 5 Self-determination; IR 5 Interpersonal relations; SI 5 Social inclusion; R 5 Rights; EW 5 Emotional well-being; PW 5 Physical well-being; MW 5 Material well-being; h2 5 Communality coefficient. coefficients between the EPR (self-report measure) and the WHOQOL-BREF. Both scales had moderate correlations between: emotional well-being in EPR and the physical (r 5 .41; p , .001) and

134

psychological (r 5 .43; p , .001) domains in the WHOQOL-BREF; physical well-being in EPR and physical (r 5 .49; p , .001) and psychological (r 5 .48; p , .001) domains in WHOQOL-BREF.

Portuguese Personal Outcomes Scale

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES

’AAIDD

2015, Vol. 53, No. 2, 129–142

DOI: 10.1352/1934-9556-53.2.129

Table 5 Concurrent Validity Coefficients WHOQOL-BREF Domains Personal development Self-determination Interpersonal relations Social inclusion Rights Emotional well-being Physical well-being Material well-being

Physical

Psychological

.16* .19*

.14*

.41** .49**

.43** .48**

Social relations

Environment

.31** .25** .20* .27**

.34** .29**

Note. ** p , .001; * p , .05.

Reliability Test-retest reliability. In order to determine the test-retest reliability, the EPR was re-administered, by the same interviewer to the same person and proxy, 2 to 3 weeks following its initial administration. A total of 160 people with ID (ages ranging from 18 to 64 years; M 5 32.23; 77 with intermittent and 83 with limited levels of support needs) and respective support staff (ages ranging from 31 to 56 years; M 5 38.17) were assessed. The period of 2 to 3 weeks between administrations was considered optimal for minimizing the likelihood that raters could remember previous responses or that the person’s behavior could change (DeVon et al., 2007; Kimberlin & Winterstein, 2008). Pearson correlations were calculated to quantify the test-retest reliability (Andresen, 2000; Cicchetti, 1994; DeVon et al., 2007; Li et al., 2013; McGillivray et al., 2009; Noonan, Miller, & Noreau, 2009). We assessed the repeatability and the stability scores of the EPR’s QOL index and domains, which were all statistically significant. Table 6 illustrates that the testretest evaluation showed high correlations in the self-report portion for all domains, except for the domain emotional well-being (r 5 .67). Correlations for the report of others measure were higher in all domains, when compared with the self-report measure. They ranged from high (r 5 .82 in interpersonal relations) to very high (r 5 .93 in rights). Test-retest reliability was also analyzed for the additional questions on importance and satisfaction. Results can be seen in Table 7. On the question of importance, Pearson correlations were weak in the emotional well-being domain (r 5 .32)

C. Simo˜es, S. Santos, and C. Claes

to moderate in the other seven domains. On the question of satisfaction, Pearson correlations ranged from weak (r 5 .35, personal development) to high (r 5 .73, interpersonal relations). Items with lower test-retest coefficients were rewritten using simple, clear, brief, and everyday language that could be understood by adults with ID. The way in which a question is phrased is of crucial importance (Andresen, 2000; Bonham et al., 2004; Finlay & Lyons, 2001; Hartley & MacLean, 2006; Li et al., 2013; McGillivray et al., 2009). Thus, some insight of acquiescence was also analyzed in the pretest from 160 adults with ID and respective proxies. According to the results, no participant responded to all the items with a favorable answer. In the selfreport portion, people with ID indicated the favorable response to 50.3%, neutral to 32.9%, and negative to 16.8% of the questions. Table 6 Test-Retest Reliability Self-report Report of others Domains Personal development Self-determination Interpersonal relations Social inclusion Rights Emotional well-being Physical well-being Material well-being QOL index

r .88** .77** .75** .79** .86** .67** .87** .89** .89**

r .92** .88** .82** .86** .93** .83** .87** .90** .92**

Note. r 5 Pearson correlations; ** p , .001.

135

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES

’AAIDD

2015, Vol. 53, No. 2, 129–142

DOI: 10.1352/1934-9556-53.2.129

Table 7 Test-Retest Reliability of the Importance and Satisfaction Questions

Domains Personal development Self-determination Interpersonal relations Social inclusion Rights Emotional well-being Physical well-being Material well-being

Importance

Satisfaction

r

r

.45** .50** .49** .46** .49** .32** .40** .46**

.35** .49** .73** .61** .50** .45** .65** .57**

Table 8 Cronbach Alpha’s Coefficients Domains

Self-report Report of others

Personal development Self-determination Interpersonal relations Social inclusion Rights Emotional well-being Physical well-being Material well-being

.65 .73

.78 .76

.48 .73 .77 .76 .46 .74

.61 .80 .79 .73 .55 .70

Note. r 5 Pearson correlations; ** p , .001. Internal consistency reliability. To evaluate the internal consistency, Cronbach’s alpha coefficients and split-half were calculated (Pestana & Gageiro, 2005). According to several authors, Cronbach’s alpha above .80 may be considered excellent (Andresen, 2000; Hartley & MacLean, 2006; Li et al., 2013; Noonan et al., 2009; Robinson, Shaver, & Wrightsman, 1991), between .70 and .80 is adequate, and below .70 is inadequate (Andresen, 2000; Li et al., 2013; Noonan et al., 2009). Following the Cronbach’s alpha guidelines, the overall instrument demonstrated good indexes of reliability, and the QOL index ranged from .87 (self-report measure) to .90 (report of others measure). The report of others portion showed higher values of internal consistency at the domain level, compared to the self-report part (see Table 8). In both parts of the EPR, the physical well-being domain has the lowest score (a 5 .46 on self-report; a 5 .55 on report of others). The highest coefficient of the self-report scale was observed in rights (a 5 .77), and social inclusion had the highest score in the report of others scale (a 5 .80). According to Pestana and Gageiro (2005), after randomly dividing all items into split-halves (split-half reliability), different indexes were estimated: the Cronbach’s alpha for each part, the correlation between forms, and the global reliability coefficients (Spearman-Brown and Guttman). As shown in Table 9, the first and the second half ranged from .74 (self-report measure) to .89 (all scale). The correlations between each half were high. Spearman-Brown and Guttman coefficients were equal or above .90.

136

Inter-rater reliability. In order to evaluate the inter-respondent reliability, a report of others measure was conducted by interviewing one institutional support worker (n 5 14; M age 5 37.28) and one family member (n 5 63; M age 5 52.75) for each person that completed the selfreport measure (n 5 63; M age 5 32.23). The advantage of the inter-respondent reliability is that it provides the ranking scores by independent evaluators and gives values of interexaminer agreement (Andresen, 2000; Cicchetti, 1994). As shown in Table 10, inter-respondent values ranged from moderate (r 5 .40) to excellent (r 5 .88), and the degree of scores differs between the three groups. Reliability normative standards were used to evaluate these coefficients, namely: poor , .40, moderate to good $ .40 , .75, and excellent $ .75 (Andresen, 2000; Li et al., 2013; Noonan et al., 2009). Cicchetti (1994) reported similar scores (i.e., poor , .40, fair $ .40 , .60, good $ .60 , .75, and excellent $ .75). Higher correlations, in general, were observed between support staff versus adults with ID or family members.

Discussion An ideal scale to assess QOL ‘‘should be one which is valid, reliable, with low administer burden, and cultural-specific to the target population involved’’ (Li et al., 2013, p. 266). The present research is a pilot study to determine the EPR as a valid and a reliable tool to evaluate the QOL in Portuguese people with ID. Concerning the construct validity, the domain-to-domain scores of the EPR of both parts had shown at least one moderate correlation. These

Portuguese Personal Outcomes Scale

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES

’AAIDD

2015, Vol. 53, No. 2, 129–142

DOI: 10.1352/1934-9556-53.2.129

Table 9 Split-Half Coefficients Domains Self-report Report of others All scale

1st half a 2nd half a Correlation 1st-2nd half a .74 .78 .86

.80 .83 .89

.82 .85 .87

Spearman-Brown Reliability Guttman .90 .92 .92

.90 .92 .92

Note. a 5 Cronbach’s alpha. results were similar to those found in the original study (van Loon et al., 2009). We reiterate that the ‘‘domains should be intercorrelated, but not highly intercorrelated’’ (van Loon et al., 2009, p. 30).The correlations domain-total scale ranged from moderate to high and were statistically significant (p , .001). In fact, the physical well-being domain had lower correlations (r 5 .35) in previous research (Claes et al., 2010; van Loon et al., 2009). According to Hartley and MacLean (2006) and Robinson et al. (1991), with r $ .60 in at least two related measures or r $ .70 in one related measure, the construct validity is strong, and is moderate when r 5 .502.59 in at least two measures and r 5 .692.60 in one related measure. The expected scores showed empirical construct validity, since correlations between domain-total scale were higher than .70 in four domains for both EPR portions. Consequently, there are different indicators for the QOL domains, but eight subscales that relate to each other and assess the same construct. The EFA seems to identify the factors that have substantive relevance and theoretical/empirical meaning (Brown, 2006; Matsunaga, 2010). Simultaneously, it is an essential step of validity

and we emphasize the importance of this information, as there are no data available on the original POS scale (Claes et al., 2010; van Loon et al., 2009). Despite the fact that we identified eight factors (Schalock & Verdugo, 2002) in a Portuguese sample that supported the original scale, the final factor pattern/structure matrix presented shortcomings. For example, we retained two factors in the self-report scale and one factor in the report of others scale with only two items. However, ‘‘it is possible to retain a factor with only two items if the items are highly correlated (i.e., r . .70) and relatively uncorrelated with other variables’’ (Worthington & Whittaker, 2006, p. 821), and the three factors retained showed this principle. Additionally, some retained items presented low communality coefficients, but they were important for the factor pattern/structure matrix. We highlight that ‘‘if an item has a communality of less than .40, it may either a) not be related to the other items, or b) suggest an additional factor that should be explored’’ (Costello & Osborne, 2005, p. 4). Finally, although the scale presented the same indicators on each domain, for both parts, we observed differences in the factor pattern/structure

Table 10 Inter-Respondent Reliability Coefficients Self-report/support staff Domains Personal development Self-determination Interpersonal relations Social inclusion Rights Emotional well-being Physical well-being Material well-being QOL index

r .82** .76** .58** .72** .76** .52** .58** .82** .82**

Self-report/family member Support staff/family member r .79** .56** .61** .62** .73** .52** .40** .86** .71**

r .78** .54** .61** .71** .75** .53** .60** .88** .76**

Note. r 5 Pearson correlations; ** p , .001.

C. Simo˜es, S. Santos, and C. Claes

137

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES 2015, Vol. 53, No. 2, 129–142

matrix between the self-report and the report of others measures. The results reflected different perceptions between people with ID and proxies, and it seems that the indicators of each domain do not necessarily have to be the same for both parts. For those reasons, it is necessary for a confirmatory factorial analysis to contribute more evidence of the Portuguese QOL’s construct in the field of ID. Regarding concurrent validity, the domains scores of the EPR were weakly (r 5 .14) to moderately (r 5 .49) correlated with the domains scores of the WHOQOL-BREF. It is difficult to stipulate scores for the concurrent validity purposes, as it depends on the characteristics of the tests used, yet the scores should not be equal to 1 (identical construct) or 0 (different construct; Cicchetti, 1994). According to DeVon et al. (2007), the concurrent validity is substantial and high when coefficients are equal or superior to .45. The concurrent validity of the POS was performed with the GENCAT-Scale (Verdugo, Arias, Go´mez, & Schalock, 2008). Although these two scales were based on the same eight domains defined by Schalock and Verdugo (2002), the correlations ranged between material well-being (r 5 .23) to self-determination (r 5 .79; Claes et al., 2010; van Loon et al., 2009). The concurrent validity between the EPR and the WHOQOL-BREF (Canavarro et al., 2007; Skevington et al., 2004; WHOQOL Group, 1998a) was a challenge, as the scales have different domains and indicators (subjective/objective versus subjective). In the present research, the relationship between the objective/subjective measures (EPR) and the personal sense of well-being (WHOQOL-BREF) was modest. It is interesting to emphasize that the subjective indicators are related to well-being and its importance, while the other indicators advocate circumstances and experiences of objective life, as well as the traditional social goals (Brown et al., 2009; Schalock, 2004; Verdugo et al., 2005). On the one hand, the satisfaction is not consistent with the objective scores (Gardner & Carran, 2005). On the other hand, the objective indicators do not capture the variability and contextual information (Kayes & McPherson, 2010). The literature also highlights the difficulty of comparing subjective and objective measures (Brown et al., 2009; Camfield & Skevington, 2008; Fakhoury & Priebe, 2002; Hensel et al., 2002; Kusel et al., 2007; Phillips, 2006). This fact is consistent with previous studies

138

’AAIDD DOI: 10.1352/1934-9556-53.2.129

that have compared the WHOQOL-BREF with other QOL scales, in which weak to moderate correlations were observed (Fakhoury & Priebe, 2002; Kusel et al., 2007). In addition, the instruments created for a certain population can be inappropriate for another (Finlay & Lyons, 2001; Kayes & McPherson, 2010; McGillivray et al., 2009). The WHOQOL-BREF is a generic tool for assessing QOL (Skevington et al., 2004). In this way, the EPR is an essential scale to measure subjective and objective QOL indicators in Portuguese adults with ID. In the field of ID, only the subjective measures may not be suitable to assess QOL and practitioners’ interventions (Hensel et al., 2002). Schalock (2004) considers that the subjective indicators are important to determine the satisfaction of people with ID, but the objective measures should also be used for evaluating environmental design and service programs. The scores presented good test-retest reliability and results of the two administrations were correlated. All of Pearson’s correlations were high to very high (Pestana & Gageiro, 2005), except for the emotional well-being domain in the self-report portion. According to Hartley and MacLean (2006), the test-retest reliability, with 1 to 3 weeks between the applications, are considered moderate (r 5 .80 and # .89) and strong (r $ .90). Based on those guidelines, the EPR can be evaluated as a valid and generic QOL scale from this psychometric point of view. The QOL index correlation was moderate (r 5 .89) in the self-report scale and strong (r 5 .92) in the report of others scale. This information is important, as there are no test-retest reliability data available on the original POS (Claes et al., 2010; van Loon et al., 2009). The two questions that were added to the EPR’s self-report portion, regarding importance and satisfaction about the domains, were analyzed separately. The test-retest coefficients showed a slight evidence of repeatability or stability. On the one hand, the questions seem to be based on feelings, opinions, and subjective interpretations (Gardner & Carran, 2005; Kayes & McPherson, 2010) that may have affected the test-retest scores. We emphasize that ‘‘in QOL theory, emotions— and related satisfaction judgments—are seen as adaptive in that they provide continuous feedback on progress toward personal goals’’ (Frisch, 2006, p. 18). However, it seems that stability is lower because the importance and the satisfaction of the eight QOL domains are related to personal

Portuguese Personal Outcomes Scale

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES

’AAIDD

2015, Vol. 53, No. 2, 129–142

circumstances, experiences, and goals. On the other hand, the hierarchical question requires abstract thinking and complex concepts (Finlay & Lyons, 2001; Hartley & MacLean, 2006; Li et al., 2013). For most people with ID, it was difficult to prioritize the domains and differentiate the eight points of response. As a result, these items will not be considered for the EPR’s QOL index. Concerning internal consistency, the lowest Cronbach’s alpha scores were observed in the physical well-being and interpersonal relations domains. Alpha coefficients in most domains were acceptable and the scale seems to be reliable in the field of ID (Andresen, 2000; Hartley & MacLean, 2006; Li et al., 2013; Noonan et al., 2009; Robinson et al., 1991; Schalock et al., 2007). Data indicated that the reliability is slightly similar to the original version. Cronbach’s alpha on self-report scale was .89 in 79 adults with ID (Claes et al., 2010) and .88 with 399 participants (van Loon et al., 2009). Internal consistency of the report of others scale was respectively .86 (Claes et al., 2010) and .85 (van Loon et al., 2009). The domains ranged from rights (n 5 79; a 5 .50) to social inclusion (n 5 79; a 5 .81), and from material well-being (n 5 399; a 5 .25) to self-determination (n 5 399; a 5 .75; Claes et al., 2010; van Loon et al., 2009). Bonham et al. (2004) also reported lower internal consistency in the report of others measure compared to the self-report measure. Scores of our study contrast with these results, as the report of others portion has a higher internal consistency. According to the guidelines of Andresen (2000), Cicchetti (1994), Li et al. (2013), and Noonan et al. (2009), inter-respondent coefficients of the domains ranged from moderate to excellent. The results were consistent with the POS, which ranged from .29 to .79 (Claes et al., 2010; van Loon et al., 2009). Our study reflected higher scores between staff members and people with ID in the personal development, self-determination, social inclusion, and rights domains. Simultaneously, we noted a better agreement between key workers and family members in the interpersonal relations and well-being domains (emotional, physical, and material). Schwartz and Rabinovitz (2003) pointed out that staff tended to overestimate life satisfaction by desire for positive outcomes of their work, and so concluded that parents seem to be better judges. However, further investigations should be performed to understand if these results can be

C. Simo˜es, S. Santos, and C. Claes

DOI: 10.1352/1934-9556-53.2.129

related to the expectations of Portuguese families with this population. The EPR is important and useful because it comprises all QOL principles of conceptualization, assessment, and intervention in the field of ID. Policy and practice have implications in the QOL of adults with ID, and should encourage individual experiences in all contexts of life. This research introduces the EPR as a tool with psychometric properties for assessing QOL in people with ID, from their own perspectives. However, self-report ratings must be complemented by objective measures (report of others). The EPR data is essential to ensure that the needs, decisions, and choices of people with ID are included in Portuguese programs and delivery of services. Adults with ID should be considered within QOL measurement and intervention plans. The QOL assessment should also be used together with other measures, such as the Adaptive Behavior Scale (Nihira, Leland, & Lambert, 1993) and the Supports Intensity Scale (Thompson et al., 2004) to increase planning designs.

Limitations Our study illustrates the challenge of assessing QOL in people with ID, with both self-report and report of others ratings. Nevertheless, there were limitations and the findings should be cautiously analyzed. As a convenience sample was used, further research is needed to confirm the results in other levels of ID. Unfortunately our research did not provide data on alternative and parallel forms of the EPR, to avoid the burden of participants. Although the selection of the indicators was based from previous literature (Schalock & Verdugo, 2002) and from previous scale (Claes et al., 2010; van Loon et al., 2008, 2009), future research should include additional items in the domains retained by the EFA, such as rights (selfreport and report of others measures), and emotional well-being (self-report measure). Future confirmatory factorial analysis research should be performed to prove the EPR structure, and help to understand the inclusion (or not) of the items with low communality coefficients. The psychometric properties of the domains such as interpersonal relations and physical well-being are also in need of improvement in future research. Nevertheless, this article represents an initial research to provide the QOL’s construct from theory to practice in Portuguese adults with ID.

139

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES 2015, Vol. 53, No. 2, 129–142

References Andresen, E. (2000). Criteria for assessing the tools of disability outcomes research. Archives of Physical Medicine and Rehabilitation, 81, S15S20. http://dx.doi.org/10.1053/apmr.2000.20619 Bonham, G., Basehart, S., Schalock, R., Marchand, C., Kirchner, N., & Rumenap, J. (2004). Consumer-based quality of life assessment: The Maryland Ask Me! Project. Mental Retardation, 42(5), 338–355. http://dx.doi.org/10. 1352/0047-6765(2004)42%3C338:CQOLAT% 3E2.0.CO;2 Brown, T. (2006). Confirmatory factor analysis for applied research. New York, NY: The Guilford Press. Brown, I., Schalock, R., & Brown, R. (2009). Quality of life: Its application to persons with intellectual disabilities and their families-introduction and overview. Journal of Policy and Practice in Intellectual Disabilities, 6(1), 2–6. http://dx.doi. org/10.1111/j.1741-1130.2008.00202.x Camfield, L., & Skevington, S. (2008). On subjective well-being and quality of life. Journal of Health Psychology, 13(6), 764–775. http://dx. doi.org/10.1177/1359105308093860 Canavarro, M., Serra, A., Simo˜es, M., Rijo, D., Pereira, M., Gameiro, S., …Carona, C. (2009). Development and psychometric properties of the World Health Organization Quality of Life Assessment Instrument (WHOQOL-100) in Portugal. International Journal Behavioral Medicine, 16, 116–124. http://dx.doi.org/10.1007/ s12529-008-9024-2 Canavarro, M., Simo˜es, M., Serra, A., Pereira, M., Rijo, D., Quartilho, M., …Carona, C. (2007). Instrumento de avaliac¸a˜o da qualidade de vida da Organizac¸a˜o Mundial de Sau´de: WHOQOLBREF. In M. Simo˜es, C. Machado, M. Gonc¸alves, & L. Almeida (Eds.), Avaliac¸a˜o psicolo´gica: Instrumentos validados para a populac¸a˜o portuguesa (pp. 77–100). Coimbra, Portugal: Quarteto Editora. Cicchetti, D. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. http:// dx.doi.org/10.1037//1040-3590.6.4.284 Claes, C., van Hove, G., van Loon, J., Vandevelde, S., & Schalock, R. (2010). Quality of life measurement in the field of intellectual disabilities: Eight principles for assessing qual-

140

’AAIDD DOI: 10.1352/1934-9556-53.2.129

ity of life-related personal outcomes. Social Indicators Research, 98, 61–72. http://dx.doi.org/ 10.1007 /s11205-009-9517-7 Costello, A., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research and Evaluation, 10(7), 1–9. Retrieved from http://pareonline. net/getvn.asp?v510&n57 Cummins, R. (2005). Moving from the quality of life concept to a theory. Journal of Intellectual Disability Research, 49(10), 699–706. http://dx. doi.org/10.1111/j.1365-2788.2005.00738.x DeVon, H., Block, M., Moyle-Wright, P., Ernst, D., Hayden, S., Lazzara D., …Kostas-Polston, E. (2007). A psychometric toolbox for testing validity and reliability. Journal of Nursing Scholarship, 39(2), 155–164. http://dx.doi.org/ 10.1111/j.1547-5069.2007.00161.x Fakhoury, W., & Priebe, S. (2002). Subjective quality of life: Its association with other constructs. International Review of Psychiatry, 14, 219–224. Finlay, W., & Lyons, E. (2001). Methodological issues in interviewing and using self-report questionnaires with people with mental retardation. Psychological Assessment, 13(3), 319– 335. http://dx.doi.org/10.1037//1040-3590.13. 3.319 Frisch, M. (2006). Quality of life therapy: Applying a life satisfaction approach to positive psychology and cognitive therapy. Hoboken, NJ: John Wiley & Sons. Gardner, J., & Carran, D. (2005). Attainment of personal outcomes by people with developmental disabilities. Mental Retardation, 43(3), 157–174. http://dx.doi.org/10.1352/0047-6765(2005)43% 5B157:AOPOBP%5D2.0.CO;2 Hartley, S., & MacLean, W. (2006). A review of the reliability and validity of Likert-type scales for people with intellectual disability. Journal of Intellectual Disability Research, 50, 813–827. http:// dx.doi.org/10.1111/j.1365-2788.2006.00844.x Hensel, E., Rose, J., Kroese, B., & J, Banks-Smith (2002). Subjective judgements of quality of life: A comparison study between people with intellectual disability and those without disability. Journal of Intellectual Disability Research, 46(2), 95–107. http://dx.doi.org/10.1046/j. 1365-2788.2002.00343.x Henson, R., & Roberts, J. (2006). Use of exploratory factor analysis in published research:

Portuguese Personal Outcomes Scale

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES 2015, Vol. 53, No. 2, 129–142

Common errors and some comment on improved practice. Educational and Psychological Measurement, 66(3), 393–416. http://dx.doi. org/10.1177/0013164405282485 IBM SPSS Statistics for Windows (Version 21.0) [Computer software]. Armonk, NY: IBM Corp. Jenaro, C., Verdugo, M., Caballo, C., Balboni, G., Lachapelle, Y., & Otrebski, W. (2005). Crosscultural study of person-centered quality of life domains and indicators: A replication. Journal of Intellectual Disability Research, 49(10), 734–739. Kayes, N., & McPherson, K. (2010). Measuring what matters: Does ‘objectivity’ mean good science? Disability and Rehabilitation, 32(12), 1011–1019. http://dx.doi.org/10.3109/09638281003775501 Kimberlin, C., & Winterstein, A. (2008). Validity and reliability of measurement instruments used in research. American Journal of HealthSystem Pharmacy, 65(1), 2276–2284. http://dx. doi.org/10.2146/ajhp070364 Kusel, Y., Laugharne, R., Perrington, S., McKendrick, J., Stephenson, D., Stockton-Henderson, J., … Burns, T. (2007). Measurement of quality of life in schizophrenia: A comparison of two scales. Social Psychiatry and Psychiatric Epidemioly, 42, 819–823. http://dx.doi.org/10.1007/ s00127-007-0249-1 Li, C., Tsoi, E., Zhang, A., Chen, S., & Wang, C. (2013). Psychometric properties of self-reported quality of life measures for people with intellectual disabilities: A systematic review. Journal of Developmental and Physical Disabilities, 25, 253–270. http://dx.doi.org/10.1007/s10882012-9297-x Matsunaga, M. (2010). How to factor-analyze your data right: Do’s, don’ts, and how-to’s. International Journal of Psychological Research, 3(1), 97–110. McGillivray, J., Lau, A., Cummins, R., & Davey, G. (2009). The utility of the Personal Wellbeing Index Intellectual Disability Scale in an Australian sample. Journal of Applied Research in Intellectual Disabilities, 22, 276–286. http://dx. doi.org/10.1111/j.1468-3148.2008.00460.x Nihira, K., Leland, H., & Lambert, N. (1993). Adaptive Behavior Scale-Residential and Community edition (ABS-RC: 2). Austin, TX: ProEd Publishing. Noonan, V., Miller, W., & Noreau, L. (2009). A review of instruments assessing participation in

C. Simo˜es, S. Santos, and C. Claes

’AAIDD DOI: 10.1352/1934-9556-53.2.129

persons with spinal cord injury. Spinal Cord, 47, 435–446. http://dx.doi.org/10.1038/sc.2008.171 Pasquali, L. (2007). Validade dos testes psicolo´gicos: Sera´ possı´vel reencontrar o caminho? Psicologia: Teoria e Pesquisa, 23, 99–107. Pestana, M., & Gageiro, J. (2005). Ana´lise de dados para cieˆncias sociais: A complementaridade do SPSS. Lisboa, Portugal: Edic¸o˜es Sı´labo. Phillips, D. (2006). Quality of life: Concept, policy and practice. London, England: Routledge. Robinson, J., Shaver, P., & Wrightsman, L. (1991). Criteria for scale selection and evaluation. In J. Robinson, P. Shaver, & L. Wrightsman (Eds.), Measures of personality social psychological attitudes (pp. 1–15). San Diego, CA: Academic Press, Inc. Schalock, R. (2004). The concept of quality of life: What we know and do not know. Journal of Intellectual Disability Research, 48(3), 203– 216. http://dx.doi.org/10.1111/j.1365-2788.2003. 00558.x Schalock, R., Gardner, J., & Bradley, V. (2007). Quality of life for people with intellectual and other developmental disabilities: Applications across individuals, organizations, communities, and systems. Washington, DC: American Association on Intellectual Disability. Schalock, R., & Verdugo, M. (2002). Handbook on quality of life for human service practitioners. Washington, DC: American Association on Mental Retardation. Schwartz, C., & Rabinovitz, S. (2003). Life satisfaction of people with intellectual disability living in community residences: Perceptions of the residents, their parents and staff members. Journal of Intellectual Disability Research, 47(2), 75–84. http://dx.doi.org/10.1046/ j.1365-2788.2003.00436.x Shogren, K., & Broussard, R. (2011). Exploring the perceptions of self-determination of individuals with intellectual disability. Intellectual and Developmental Disabilities, 49(2), 86–102. http:// dx.doi.org/10.1352/1934-9556-49.2.86 Simo˜es, C., & Santos, S. (2014). Cross cultural adaptation, validity and reliability of the Escala Pessoal de Resultados. Social Indicators Research, 119(2), 1065–1077. http://dx.doi.org/10.1007/ s11205-013-0515-4 Skevington, S., Lotfy, M., & O’Connell, K. (2004). The World Health Organization’s WHOQLBREF quality of life assessment: Psychometric

141

INTELLECTUAL AND DEVELOPMENTAL DISABILITIES 2015, Vol. 53, No. 2, 129–142

properties and results of the international field trial. Quality of Life Research, 13, 299–310. Thompson, J., Bryant, B., Campbell, E., Craig, E., Hughes, C., Rotholz, D., … Wehmeyer, M. (2004). Supports Intensity Scale: Users manual. Washington, DC: American Association on Mental Retardation. Turnbull, H., Turnbull, A., Wehmeyer, M., & Park, J. (2003). A quality of life framework for special education outcomes. Remedial and Special Education, 24(2), 67–74. http://dx.doi. org/10.1177/07419325030240020201 van Loon, J., van Hove, G., Schalock, R., & Claes, C. (2008). Personal Outcomes Scale: A scale to assess an individual’s quality of life. Ghent, Netherlands: Stichting Arduin. van Loon, J., van Hove, G., Schalock, R., & Claes, C. (2009). Personal Outcomes Scale: Administration and standardization manual. Ghent, Netherlands: Stichting Arduin. Verdugo, M., Arias, B., Go´mez, L., & Schalock, R. (2008). Formulari de l’escala GENCAT de qualitat de vida. Manual d’aplicacio´ de l’escala GENCAT de qualitat de vida. Barcelona, Spain: Departament d’Accio´ Social i Ciutadania, Generalitat de Catalun˜a. Verdugo, M., Schalock, R., Keith, K., & Stancliffe, R. (2005). Quality of life and its measurement: Important principles and guidelines. Journal of Intellectual Disability Research, 49(10), 707– 717. http://dx.doi.org/10.1111/j.1365-2788.2005. 00739.x Wang, M., Schalock, R., Verdugo, M., & Jenaro, C. (2010). Examining the factor structure and hierarchical nature of the quality of life construct. American Journal on Intellectual and Developmental Disabilities, 115(3), 218–233. http://dx.doi.org/10.1352/1944-7558-115.3.218

142

’AAIDD DOI: 10.1352/1934-9556-53.2.129

WHOQOL Group. (1998a). The World Health Organization Quality of Life Assessment (WHOQOL): Development and general psychometric properties. Social Science & Medicine, 46, 1569–1585. WHOQOL Group. (1998b). Development of the World Health Organization WHOQOL-BREF quality of life assessment. Psychological Medicine, 28, 551–558. World Medical Association. (2008). Declaration of Helsinki: Ethical principles for medical research involving human subjects. Retrieved from http:// www.bioetica.ufrgs.br/helsin7.pdf Worthington, R., & Whittaker, T. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806–838. http://dx. doi.org/10.1177/0011000006288127

Received 4/8/2014, accepted 12/14/2014. The authors would like to thank Robert Schalock for reviewing the article, spending considerable time and effort, and providing valuable comments. Authors: Cristina Simo˜es and Sofia Santos, University of Lisbon, Portugal; and Claudia Claes, University College Ghent. Correspondence concerning this article can be addressed to Cristina Simo˜es, University of Lisbon, Faculty of Human Kinetics, Estrada da Costa 1499002, Cruz Quebrada, Lisbon 1499-002, Portugal (e-mail: [email protected]).

Portuguese Personal Outcomes Scale

Copyright of Intellectual & Developmental Disabilities is the property of American Association on Intellectual & Developmental Disabilities and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.