This article was downloaded by: [Selcuk Universitesi] On: 23 January 2015, At: 16:28 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

The Clinical Neuropsychologist Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/ntcn20

Exploring the Reliability and Component Structure of the Personality Assessment Inventory in a Neuropsychological Sample a

b

a

c

Michelle Busse , Douglas Whiteside , Dana Waters , Jared Hellings & Peter Ji

b

a

Washington School of Professional Psychology, Seattle, WA, USA

b

Adler School of Professional Psychology, Chicago, IL, USA

c

Click for updates

Western Washington Medical Group, Everett, WA, USA Published online: 05 Feb 2014.

To cite this article: Michelle Busse, Douglas Whiteside, Dana Waters, Jared Hellings & Peter Ji (2014) Exploring the Reliability and Component Structure of the Personality Assessment Inventory in a Neuropsychological Sample, The Clinical Neuropsychologist, 28:2, 237-251, DOI: 10.1080/13854046.2013.876100 To link to this article: http://dx.doi.org/10.1080/13854046.2013.876100

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions

The Clinical Neuropsychologist, 2014 Vol. 28, No. 2, 237–251, http://dx.doi.org/10.1080/13854046.2013.876100

Exploring the Reliability and Component Structure of the Personality Assessment Inventory in a Neuropsychological Sample Michelle Busse1, Douglas Whiteside2, Dana Waters1, Jared Hellings3, and Peter Ji2 1

Washington School of Professional Psychology, Seattle, WA, USA Adler School of Professional Psychology, Chicago, IL, USA 3 Western Washington Medical Group, Everett, WA, USA

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

2

The current study was designed to advance general research investigating the Personality Assessment Inventory (PAI), by examining whether the psychometric properties of the PAI would generalize to a sample differing from the original standardization sample. Specifically, the reliability and factor structure of the PAI were examined in a mixed neuropsychological sample. Reliability full scale coefficients ranged from .72 to .94, and subscale coefficients ranged from .60 to .90. Confirmatory factor analysis (CFA) was conducted to test Morey’s original four-factor model (for all 22 PAI scales) and three-factor model (for the 11 clinical scales). CFA results indicated that Morey’s original factor solutions were not a good fit. Thus, following Morey’s original methodology, principal components analyses (PCA) were conducted on all 22 PAI scales and on the 11 PAI clinical scales and the results indicated evidence for a five-component solution (for all 22 PAI scales) and a two-component solution (for the 11 clinical scales). Overall, while results indicated some relatively subtle differences between the original standardization sample and the current sample, they still supported the notion that the PAI is a reliable and valid measure when used in a neuropsychological sample. This study expands upon the existing literature related to the clinical utility of the PAI in specialized samples. Keywords: Personality Assessment Inventory; Neuropsychological sample; Psychometric properties.

INTRODUCTION: THE PERSONALITY ASSESSMENT INVENTORY The Personality Assessment Inventory is becoming a widely used self-report measure of personality and psychopathology (PAI; Morey, 1991). The PAI contains 344 items that comprise 22 non-overlapping scales: 4 validity scales, 11 clinical scales, 5 treatment scales, and 2 interpersonal scales (Morey, 1991). The PAI is supported in research studies as being a reliable and valid measure of personality and psychopathology (Morey, 2003). While the use of the PAI has increased over the last 20 years (Piotrowski, 2000), research investigating and extending the clinical utility and efficacy of the PAI has lagged (Kurtz & Blais, 2007). As a result, Kurtz and Blais (2007) advocated for additional research investigating the PAI, its psychometric properties, and its utility with special populations in order to enhance confidence in the construct validity of the PAI. This is particularly important given that the PAI was originally developed

Address Correspondence to: Michelle Busse, Washington School of Professional Psychology, Seattle, WA, USA. E-mail: [email protected] (Received 19 October 2012; accepted 11 December 2013)

© 2014 Taylor & Francis

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

238

MICHELLE BUSSE ET AL.

for use in non-neurologically impaired psychiatric populations (the majority of patients had mood disorders, anxiety disorders, or substance-related disorders, and only 2.2% were coded with neurological disorders); yet it is increasingly being used in neuropsychological populations (Demakis et al., 2007). Studies have suggested that generalizability cannot be assumed when individuals in the sample are defined by specific and unique characteristics (Schinka, 1995), as some aspects of tests do not always generalize across different populations (Tasca, Wood, Demidenko, & Bissada, 2002). As a result there is a need to assess and evaluate the PAI’s psychometric properties within a neuropsychological sample given the unique characteristics of this patient population. Recently research has begun to explore the psychometric properties of the PAI in specific samples, with an emphasis being placed on the PAI’s factor structure. Generally, findings have indicated that the PAI’s dimensional structures are consistent when used with nonclinical samples and clinical samples (Hoelzle & Meyer, 2009). Past research has replicated the original factor structure of the PAI with specialized samples (e.g., TBI patients; Demakis et al., 2007); however, other research has found different factor solutions when used with substance abuse patients (Schinka, 1995), psychiatric inpatients (Boone, 1998), eating-disordered patients (Tasca et al., 2002), university counseling students (Cashel, Dollinger, & Cunningham, 2003), and patients suffering from chronic pain (Karlin et al., 2005). For example, Tasca et al. (2002) found a similar factor structure to Morey’s (1991); however, the study also found evidence for an additional or novel factor. The researchers suggested that this finding points to an important interpersonal component of eating disorder pathology and proposed that different interpretations of PAI profiles may be required for specific clinical populations (Tasca et al., 2002). Within a neuropsychological sample there are limited published studies on the reliability and validity of the PAI. In one such study Frazier, Naugle, and Haggerty (2006) studied 421 adults from an outpatient clinic who were referred for a neuropsychological evaluation. Results indicated that the Cronbach’s alpha values in the research sample were very similar to Morey’s values, ranging from .64 to .91 across the 22 PAI full scales (reliability of the subscales was not explored). The factor structure was also compared to Morey’s reported factor structure in the normal sample. With the neuropsychological sample, a four-factor solution was replicated when using all 22 PAI scales (factor structure of the clinical scales was not examined). Similar configurations of the factor scores were found for all four factors: Pearson r values = .99, .97, .87, and .96, which suggested a similar pattern of relations between scales across the two samples (Frazier et al., 2006). Therefore this research provided preliminary support for the utility of the PAI within a neuropsychological population. However, no replication of these results has been found to date, and no reliability analyses of the subscales have been published. Given the limited research on the reliability and validity of the PAI in a neuropsychological sample, the goal of the current study was to replicate and extend on previous findings to determine whether the PAI can be generalized to a neuropsychological sample. It was hypothesized that, when used in a neuropsychological sample, the PAI will have similar levels of internal consistency reliability coefficients and a similar factor structure when compared to Morey’s (1991) original results. Based on the results of the Frazier et al. (2006) study, it was expected that the factor structure of the current study

RELIABILITY AND COMPONENT STRUCTURE OF PAI

239

and Morey’s (1991) sample would be similar, with the possibility of some minor variances due to differences in the current sample’s demographics and diagnoses. METHOD

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

Participants The current study used de-identified neuropsychological assessment data collected from a multi-center clinical database that included data from patients assessed between 2001 and 2012. This database contained demographic information, diagnoses, and assessment performance results for 506 individuals, over 18 years of age, who had completed a neuropsychological evaluation. Individuals came from a variety of referral sources and were assessed in an outpatient clinic. Approval through the Institutional Review Board of the Washington School of Professional Psychology was obtained prior to data analysis. Prior to data analysis, participant profiles were screened for invalid responding and excessive nonresponse according to Morey’s (1991) criteria. Participants were excluded if they had any of the following scores: Inconsistency (ICN) scale > 73t, Infrequency (INF) scale > 75t, Negative Impression Management (NIM) scale > 92t, Positive Impression Management (PIM) scale > 68t or more than 33 omitted responses. Of the initial 506 participants, 42 were excluded due to invalid responding. Following these exclusions, the analyzed sample was comprised of 464 participants, 4 of whom had more than 10 responses omitted. See Table 1 for demographic information related to the valid and invalid responders. No individuals were systematically excluded due to demographic characteristics. Data analysis Most data analyses were completed using Statistical Package for Social Sciences 17.0 (SPSS); however, confirmatory factor analysis (CFA) was calculated with Mplus 7.11. To extend Frazier et al.’s (2006) study, Cronbach’s alpha and mean inter-item correlation coefficients were calculated for all PAI full scales as well as subscales. Participants’ individual PAI item responses were entered into a database, along with PAI raw and T scale scores. Then, for each full scale and subscale, Chronbach’s alpha coefficients were calculated to evaluate each scale’s internal consistency. After this, confirmatory factor analyses (CFA) was initially conducted to verify Morey’s four-factor model for the 22 PAI full scales and Morey’s three-factor model for the 11 clinical scales in the study’s mixed neuropsychological sample. However, as noted below, the CFA did not find evidence for a good fit. The next step was to conduct principal components analyses (PCA) to provide evidence for the most appropriate component structure. For this step the current study followed Morey’s (1991) methodology for PCA. Specifically, correlation matrices were first calculated, and then PCAs were conducted on the correlation matrices. Components with an eigenvalue > 1 were retained. To aid in interpretation and consistent with Morey’s initial methodology, varimax rotation was used. In the first PCA, all 22 PAI full scales were examined, and in a second PCA, only the 11 clinical scales were examined. These analyses reproduced the methodology used by Morey (1991), Tasca et al. (2002), and Karlin et al. (2005).

240

MICHELLE BUSSE ET AL. Table 1. Demographics for valid and invalid responders

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

Participants Demographic

Valid responders (n = 464)

Invalid responders (n = 42)

Mean age Mean education Gender Male Race Caucasian Other Referral source Health care providers Attorneys Educational settings Worker’s compensation Social security Other Primary mental health diagnosis Mood Disorder Anxiety Disorder Somatoform Disorder Cognitive Disorder NOS Adjustment Disorder Substance Use Disorder Attention Deficit Disorder Learning Disorder Dementia Other Primary medical diagnosis Traumatic Brain Injury Metabolic Disorder Sleep Apnea Dementia Epilepsy Fibromyalgia Cerebrovascular Disease Multiple Sclerosis Chronic Kidney Disease Anoxia Parkinson’s Disease Fatigue Other

46.77 years (SD = 15.00) 13.44 years (SD = 2.46)

47.28 years (SD = 15.19) 12.31 years (SD = 2.60)

48.1%

40.6%

95.7% 4.3%

87.5% 12.5%

83.6% 5.5% 3.5% 2.3% 2.3% 2.9%

62.5% 18.8% 12.5% 0% 3.1% 3.1%

29.6% 15.2% 11.7% 10.9% 5.6% 5.1% 4.3% 4.0% 4.3% 9.3%

26.2% 7.2% 2.4% 0% 9.5% 2.4% 2.4% 4.8% 9.5% 4.8%

31.4% 12.6% 5.6% 5.1% 4.9% 3.7% 4.5% 2.6% 1.9% 1.9% 1.5% 0% 26.9%

23.8% 9.5% 0% 2.4% 0% 9% 2.4% 2.4% 0% 0% 0% 2.4% 48.1%

n = Number; SD = Standard deviation.

RESULTS As hypothesized, full scale and subscale alpha coefficients were acceptable and comparable to what Morey reported in his standardization sample (see Tables 2 and 3). Full scale coefficients ranged from .72 (Negative Impression Management; NIM) to .94 (Anxiety; ANX). The mean Cronbach’s alpha value for 20 of the PAI full scales was .83; slightly lower than what was reported for Morey’s (1991) clinical sample (.86), and

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

RELIABILITY AND COMPONENT STRUCTURE OF PAI

241

slightly higher than Frazier et al.’s (2006) findings in a neuropsychological sample (.80). Not surprisingly, overall subscale coefficients were slightly lower but still acceptable; ranging from .60 (Mania-Activity Level; MAN-A) to .90 (Anxiety Related Disorders-Traumatic Stress; ARD-T and Depression-Affective; DEP-A). The mean alpha value for the subscales was .78, which is identical to the published data by Morey (1991) in his clinical sample. Frazier et al. (2006) did not examine the reliability of the subscales so no comparison can be made. The next step of the analysis involved conducting a CFA to evaluate whether Morey’s four-factor solution for the 22 PAI full scales was a good fit with the current data. Unfortunately, none of the goodness of fit parameters met acceptable levels as defined by Hu and Bentler (1999). Specifically, Hu and Bentler defined the minimum criteria for goodness of fit as RMSEA < .06, the Tucker-Lewis fit index (TLI) > .95, and the comparative fit index (CFI) > .95. For the current data, Morey’s four-factor solution did not meet these criteria; the specific values were CFI = .72, the TLI = .68, and the RMSEA = .14. A CFA was also conducted to evaluate Morey’s three-factor

Table 2. Full scale Cronbach’s Alpha coefficients and mean interitem correlations Alpha

Mean interitem correlation

Study sample

Census sample

Clinical sample

Study sample

Census sample

Clinical sample

Negative Impression Management Positive Impression Management

.72

.72

.74

.24

.24

.25

.78

.71

.77

.28

.17

.26

Somatic Complaints Anxiety Anxiety Related Disorders Depression Mania Paranoia Schizophrenia Borderline Features Antisocial Features Alcohol Problems Drug Problems

.91 .94 .86

.89 .90 .76

.92 .94 .86

.30 .39 .21

.26 .17 .13

.31 .40 .20

.93 .83 .90 .87 .91 .86 .86 .81

.87 .82 .85 .81 .87 .84 .84 .74

.93 .82 .89 .89 .91 .86 .93 .89

.35 .17 .28 .23 .29 .23 .41 .33

.24 .17 .20 .17 .22 .20 .36 .28

.36 .16 .26 .26 .29 .21 .55 .43

Aggression Suicide Ideation Stress Nonsupport Treatment Rejection

.89 .91 .79 .79 .75

.85 .85 .76 .72 .76

.90 .93 .79 .80 .80

.31 .49 .33 .32 .26

.19 .41 .30 .25 .28

.33 .56 .32 .34 .33

Dominance Warmth

.81 .83

.78 .79

.82 .83

.26 .28

.22 .24

.27 .28

Scales

Study sample = Study’s neuropsychological sample; Census sample = Morey’s census matched normative sample; Clinical sample = Morey’s clinical normative sample.

242

MICHELLE BUSSE ET AL. Table 3. Subscale Cronbach’s Alpha coefficients and mean interitem correlations Alpha

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

Subscales Somatic complaints Conversion Somatization Health Concerns Anxiety Cognitive Affective Physiological Anxiety-related disorders ObsessiveCompulsive Phobias Traumatic Stress Depression Cognitive Affective Physiological Mania Activity Level Grandiosity Irritability Paranoia Hypervigilance Persecution Resentment Schizophrenia Psychotic Experiences Social Detachment Thought Disorder Borderline features Affective Instability Identity Problems Negative Relationships Self-Harm Antisocial features Antisocial Behaviors Egocentricity Stimulus-Seeking Aggression Aggressive Attitude Verbal Aggression Physical Aggression

Mean interitem correlation

Study sample

Census sample

Clinical sample

Study sample

Census sample

Clinical sample

.81 .79 .81

.74 .68 .81

.83 .77 .83

.36 .31 .34

.27 .22 .36

.38 .29 .38

.88 .84 .83

.81 .73 .74

.87 .84 .83

.46 .40 .38

.35 .27 .29

.45 .40 .38

.68

.56

.63

.21

.14

.18

.64 .90

.58 .81

.67 .89

.18 .53

.15 .35

.20 .50

.84 .90 .78

.74 .80 .71

.84 .88 .80

.40 .54 .30

.28 .36 .23

.40 .48 .32

.60 .77 .81

.51 .73 .78

.55 .78 .81

.16 .31 .36

.12 .26 .31

.14 .31 .35

.79 .80 .73

.64 .76 .66

.75 .83 .72

.32 .36 .25

.19 .30 .21

.28 .39 .25

.68

.56

.71

.24

.16

.26

.86 .83

.79 .73

.85 .85

.44 .37

.33 .27

.41 .42

.82 .77 .75

.71 .70 .63

.81 .77 .68

.43 .35 .32

.29 .30 .22

.42 .36 .26

.76

.62

.76

.35

.22

.34

.77

.73

.80

.32

.27

.34

.66 .76

.63 .69

.63 .75

.22 .30

.20 .23

.18 .28

.81 .74 .75

.74 .67 .71

.80 .70 .84

.42 .32 .38

.32 .25 .34

.39 .28 .48

Study sample = Study’s neuropsychological sample; Census sample = Morey’s census matched normative sample; Clinical sample = Morey’s clinical normative sample.

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

RELIABILITY AND COMPONENT STRUCTURE OF PAI

243

solution for the 11 clinical scales. This CFA also did not support Morey’s three-factor model because this model did not converge, so CFI, TLI, and RMSWA could not be calculated. Other CFA results, such as standardized parameter estimates, unstandardized estimates, squared multiple correlation values were not calculated since neither of the models met goodness of fit parameters. Since the CFA results did not provide evidence for a good fit with the data, PCAs were conducted to explore the data further and to determine what factor structure was applicable to the data. For comparison purposes, Morey’s (1991) original methodology of PCA with an orthogonal rotation using varimax was conducted using the 22 PAI full scales and the 11 clinical scales. The criteria of eigenvalues > 1 was used for both analyses in order to replicate the original methodology. The rotated component loadings for the 22 PAI full scales are presented in Table 4, while the rotated component loadings for the 11 clinical scales are presented in Table 5. The 22 full scales loaded on five components, which accounted for 69% of the variance. This is in contrast to the four factors found in Morey’s (1991) clinical sample, which accounted for 65% of the variance. In the current sample, component 1 had significant loadings (> 0.40) on

Table 4. Varimax rotated component loadings of all full scales Morey’s (1991) clinical sample factors

Study sample components Scales Inconsistency Infrequency Negative Impression Positive Impression Somatic Complaints Anxiety Anxiety Related Disorders Depression Mania Paranoia Schizophrenia Borderline Features Antisocial Features Alcohol Problems Drug Problems Aggression Suicide Ideation Stress Nonsupport Treatment Rejection Dominance Warmth % of variance Factor eigenvalues

1

2

3

4

5

1

2

3

4

– – .79 –.50 .81 .82 .80 .83 – .56 .73 .67 – – – – .58 .61 .44 –.61 – – 40.13 8.83

– – – –.42 – – – – .82 – – .41 .68 – – .69 – – – – .64 – 10.73 2.36

– – – – – – – – – .42 .43 – – – – – – – .56 – –.48 –.82 7.49 1.65

.45 – – – – – – – – – – – .47 .78 .79 – – – – – – – 5.29 1.16

.47 .73 – .41 – – – – – – – – – – – – – – – – – – 4.91 1.08

– – .79 –.68 .70 .90 .85 .89 – .68 .84 .81 – – – – .70 .61 .63 –.50 –.40 –.49 41.2

– – – – – – – – – – – .41 .73 .77 .79 .48 – – – –.42 – – 11.2

– – – – – – – – .78 – – – – – – .45 – – – – .72 .40 6.9

.41 .81 – – – – – – – – – – – – – – – – – – – – 5.9

Within the study sample, Component 1 = General Distress; Component 2 = Behavioral Acting Out; Component 3 = Social Distancing; Component 4 = Substance Abuse Vulnerability; Component 5 = Random Responding.

244

MICHELLE BUSSE ET AL. Table 5. Varimax rotated component loadings of clinical scales only Morey’s (1991) clinical sample factors

Study sample components

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

Scales Somatic Complaints Anxiety Anxiety Related Disorders Depression Mania Paranoia Schizophrenia Borderline Features Antisocial Features Alcohol Problems Drug Problems % of Variance Factor Eigenvalues

1

2

1

2

3

.75 .88 .83 .88 – .65 .81 .73 – – – 49.1% 5.40

– – – – .59 .42 – .52 .84 .74 .70 16.4% 1.81

.75 .91 .87 .89 – .67 .84 .77 – – – 49.7%

– – – – .50 .40 – .45 .86 .73 .82 18.0%

– – – – .90 – – .48 .51 – – 9.5%

Within the study sample, Component 1 = Internalizing Behaviors; Component 2 = Externalizing Behaviors.

Depression (DEP), ANX, NIM, Anxiety Related Disorders (ARD), Somatic Complaints (SOM), Schizophrenia (SCZ), Borderline Features (BOR), Stress (STR), Suicide Ideation (SUI), and Paranoia (PAR) scales and significant negative loadings on Treatment Rejection (RXR) and Positive Impression Management (PIM) scales. This component accounted for the most variance (40%), similar to Morey’s Factor 1 which accounted for 41% of the variance. Component 2 had significant factor loadings on Mania (MAN), Aggression (AGG), Antisocial Features (ANT), and Dominance (DOM) scales, and it accounted for 11% of the variance. Morey’s Factor 2 accounted for 11.2% of the variance. Component 3 had a significant negative factor loading on Warmth (WRM) and a significant positive factor loading on Nonsupport (NON); it accounted for 8% of the variance. Morey’s Factor 3 accounted for 6.9% of the variance. Component 4 had significant factor loadings on Alcohol Problems (ALC) and Drug Problems (DRG) scales, and it accounted for 5% of the variance. Finally, component 5 had significant factor loadings on Infrequency (INF) and Inconsistency (INC) scales and accounted for 5% of the variance. This component accounted for 5.9% of the variance in Morey’s (1991) study. As noted above, the rotated factor loadings for the 11 clinical scales are presented in Table 5. The 11 PAI clinical scales loaded on two components, with component 1 accounting for 49% of the variance and component 2 accounting for 16% of the variance. This result was in contrast to Morey’s three-factor solution for the 11 clinical scales found in his clinical sample. In Morey’s study, Factor 1 accounted for 49.7% of the variance, Factor 2 accounted for 18.0% of the variance, and Factor 3 accounted for 9.5% of the variance. In the current study, component 1 had significant positive factor loadings on SOM, ANX, ARD, DEP, PAR, SCZ, and BOR scales. Component 2 had significant positive factor loadings on MAN, ANT, ALC, and DRG scales.

RELIABILITY AND COMPONENT STRUCTURE OF PAI

245

DISCUSSION

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

Reliability As hypothesized, full scale reliability coefficients ranged from .72 (NIM) to .94 (ANX) and were generally consistent with those found in Morey’s (1991) clinical sample. Also as hypothesized, subscale reliability coefficients ranged from .60 (MAN-A) to .90 (ARD-T and DEP-A) and were again generally consistent with the values published by Morey. The fact that this study’s internal reliability estimates were very similar to those estimates published by Morey (1991) lends further support for the reliability of the PAI in a neuropsychological sample and suggests that the PAI performs similarly in this special population. While overall reliability of the PAI full scales and subscales was acceptable, a few subscale reliability estimates fell slightly below the recommended level of reliability that ranges in the research literature from .70 to .90 (Green & Weiner, 2008; Nunnally, 1967). These scales included ARD-P (.64), MAN-A (.60), SCZ-P (.68), and ANT-E (.66). However, the lower reliability coefficients on these particular subscales is consistent with Morey’s (1991) findings in his clinical sample, and is as a result of the smaller scale sizes. It should be noted that in this study the internal consistency coefficients of the PAI scales tended to be higher than other similar measures. For example, the clinical and content scales from the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) range from .34 to .87 and .68 to .86, respectively (Butcher et al., 1989). Furthermore, the Millon Clinical Multiaxial Inventory-III (MCMI-III) has scales with internal consistency coefficients below.70 (Compulsive and Narcissistic scales; Millon, 1994). Wise, Streiner, and Walfish (2010) reported that, on the MCMI-III, 78% of the scales obtained alpha coefficients greater than .80, on the MMPI-2 27% and 29% of the scales (women and men, respectively) reached the criterion, and on the PAI 63% of the scales reached the criterion. In comparison, the current study found that 90% of the scales had Cronbach’s alpha coefficients above .70 and only 57% of the scales had alpha coefficients greater than the Wise et al. (2010) .80 defined criteria. The different criteria used to select individual test items when developing these instruments (e.g., statistical properties of items, using the theory of personality to select items, ability of items to differentiate individuals from a diagnostic group) likely accounts for the varying internal consistency coefficients (Wise et al., 2010). Factor structure The current study initially found that Morey’s (1991) original factor solution was not a good fit for the 22 PAI full scales or the 11 clinical scales. Follow-up PCA replicating Morey’s original factor analytic strategy found a five-component solution for the 22 PAI full scales using an orthogonal rotation. This was in contrast to the models found in Morey’s (1991) original clinical sample and Frazier et al.’s (2006) neuropsychological sample, both of which found evidence for a four-factor solution for the 22 PAI full scales. Although an additional component was identified in the current study, it should be noted that the overall five-component solution bears a strong resemblance to that which Morey (1991) and Frazier et al. (2006) reported.

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

246

MICHELLE BUSSE ET AL.

Specifically, the first component strongly resembles the first factor reported in other PAI studies, and this component has generally been called a general distress factor (Morey, 1996). Thus, this “General Distress” component found in the current study is consistent with Factor 1 in Morey’s (1991) clinical sample as well as other samples like Tasca et al.’s. (2002) eating-disordered sample, Schinka’s (1995) alcohol-dependent sample, Karlin et al.’s (2005) chronic pain sample, and Deisinger’s (1995) normal (non-clinical) sample. In these studies this first component consistently accounted for the largest amount of variance. This component has large loadings for many of the clinical scales purported to assess various clinical constructs, as well as several of the treatment scales. Thus, this component is hypothesized to reflect an openness to report various types of psychological distress on the part of the individual (Hoelzle & Meyer, 2009) and is consistent with Morey’s original hypothesis for this component in that it represents significant subjective distress and affective disruption. The only difference between Morey’s (1991) results and the current study’s findings were that NON and WRM scales did not heavily load on this component, instead these two scales loaded on their own component, which is hypothesized to be more of a “Social Distancing” component. The second component, labeled “Behavioral Acting Out”, is hypothesized to measure impulsivity, hostility, dominance, and poor anger control, in other words, externalizing behaviors. This component is almost identical to the third factor reported by Karlin et al. (2005) in a chronic pain population and similar to the second component reported by Hoelzle and Meyer (2009) in an outpatient clinical sample. This component is also conceptually similar to Morey’s (1991) Factor 3, which reportedly represented egocentricity and exploitativeness in interpersonal relationships. The main difference is that the ANT scale did not heavily load on this factor in Morey’s (1991) study, and it instead loaded strongly on Morey’s substance use factor. Hoelzle and Meyer (2009) found that the ANT scale had a similar pattern loading on two of the components found in their study, as ANT loaded heavily with the MAN, DOM, and AGG scales (as it did in the current study), but also loaded heavily with the ALC and DRG scales (as in Morey’s study). The third component, labeled “Social Distancing”, is hypothesized to measure distant and poor quality interpersonal relationships. When considering the underlying scales (NON and WRM) and what they are purported to measure (Morey, 1996), this component may describe individuals who tend to have a cold and rejecting interpersonal style and perceive that they are unsupported by those around them. This factor was not found by Morey (1991) in his clinical sample; however, it is not a unique finding among special populations. For example, while Schinka (1995) only evaluated 20 of the full scales, he also found a Social Distancing factor in his alcohol-dependent sample that was primarily defined by a high positive loading on NON and a high negative loading on WRM. Schinka (1995) proposed that this component represents interpersonal “coolness,” mistrust, and social distancing related to more severe pathology or personality dysfunction. Likewise, Tasca et al. (2002) found evidence for a similar factor in their eating disordered population. The factor was low in WRM and high in NON and PAR. Tasca et al. (2002) suggested that this factor highlights an important interpersonal component related to eating disorders, which might have implications for treatment that is of the interpersonal nature. Both Tasca et al. (2002) and Schinka (1995) concluded that, since this factor was not found in Morey’s (1991) clinical

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

RELIABILITY AND COMPONENT STRUCTURE OF PAI

247

sample, a slightly different interpretation of PAI profiles might be required for special clinical populations. When considering the definition of this component and the makeup of the current study’s sample, it can be hypothesized that this component measures an interpersonal variable found in patients in a neuropsychological sample, which may be related to their chronic mental health and medical disorders. Specifically, it might be that these patients experience extensive emotional distress and social isolation because of their symptoms and poor prognosis. Thus, they might begin to feel hopeless and believe that others cannot or will not help them. Consequently they may distance themselves in their interpersonal relationships due to their perception that these relationships are no longer helpful or supportive. There might also be some degree of suspiciousness involved given the loading of PAR on this component. This in turn has treatment implications because it may be more difficult to develop a therapeutic alliance. The current study’s results suggest that the component structure of the PAI in a neuropsychological sample is in some ways more similar to the eating-disordered (Tasca et al., 2002) and alcohol-dependent (Schinka, 1995) samples than Morey’s (1991) normative study. In terms of commonalities between neuropsychological, substance abuse, and eating disorder populations that would explain the similar findings across studies, one hypothesis is that these three special populations had a similar “Social Distancing” factor due to a manifestation of some type of cognitive rigidity that could adversely affect their interpersonal relationships. Cognitive rigidity may involve rumination, compulsive behaviors, interpersonal inflexibility, and distorted perceptions that lead to interpersonal isolation and withdrawal. Within the current study’s mixed neuropsychological sample, cognitive rigidity may manifest in various psychological and neurological disorders common in a neuropsychological practice. Morey’s (1991) clinical sample did not contain this same make-up of disorders and this might be why he did not find a similar factor. The fourth component, labeled “Substance Use Vulnerability”, is hypothesized to be fairly straightforward as it represents the personality characteristics and behaviors commonly associated with alcohol and substance abuse problems. Of note, the ANT scale also heavily loaded on this component, which is consistent with this conceptualization because behaviors associated with ANT have been associated with substance abuse in the literature (Morey, 1991). This “Substance Use Vulnerability” component is somewhat similar to Morey’s (1991) Factor 2, which he described as representing behavioral acting out related to impulsivity and poor judgment. However, as mentioned, in the current study ANT and AGG did not load on this component, but instead loaded on the “Behavioral Acting Out” component. This could possibly suggest that in a neuropsychological sample, patients with substance use problems do not tend to exhibit the same degree of acting out seen in Morey’s (1991) clinical sample. Karlin et al. (2005) proposed a similar interpretation in their study, as they suggested that the link between substance use and acting act found in a general population and some other clinical populations is not evident when considering patients with chronic pain. Finally the last component, labeled “Random Responding”, is hypothesized to basically be a profile validity component, and it is essentially identical to Factor 4 in Morey’s (1991) clinical sample. Others studies have not reported a clear-cut profile validity component. While the current study’s results and Frazier et al.’s (2006) results provide support for the validity of the 22 PAI full scales in a neuropsychological sample, no studies have been published examining the factor structure of the 11 clinical scales in a mixed

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

248

MICHELLE BUSSE ET AL.

neuropsychological sample. In the current study a two-component solution was found when analyzing the 11 clinical scales. These results replicated the two-factor solution found with Morey’s (1991) normal sample, but differed from Morey’s three-factor solution in the clinical sample. In the current study all clinical scales loaded strongly on the first component, labeled “Internalizing Behaviors”, with the exception of MAN, ANT, ALC, and DRG scales, making it a reasonably straightforward component structure. Thus, the “Internalizing Behaviors” component is hypothesized to generally measure significant subjective distress and affective disruption and is consistent with the general distress component discussed above and found in previous research (Deisinger, 1995; Karlin et al., 2005; Schinka, 1995; Tasca et al., 2002). The “Externalizing Behaviors” component measures acting-out tendencies and is similar to the “Behavioral Acting Out” component discussed above, which has also been found in previous research (Hoelzle & Meyer, 2009; Karlin et al., 2005). This component is also conceptually similar to Morey’s (1991). The difference in the results between the current study and Morey’s (1991) clinical sample focused on the fact that MAN heavily loaded on its own factor in Morey’s sample, but not in the current study. This is a relatively small difference and likely does not impact interpretation. One can hypothesize that since the sample in the current study did not contain patients displaying a wide variety of more severe psychopathologies and personality disorders the scales tended to load together better. When analyzing the research literature on the factor structure of the 11 clinical scales, Morey’s (1991) three-factor solution in the clinical sample was replicated in a TBI sample (Demakis et al., 2007) and a chronic pain sample (Karlin et al., 2005), but the two-component solution observed in the current study has not been previously described in a clinical sample. However, interestingly, when Morey (1991) examined the factor structure of the 11 clinical scales in a normal sample, he found a very similar two-factor solution to the one described in the current study. Thus, Morey’s (1991) normal sample and this study’s mixed neuropsychological sample do not have the additional component representing egocentricity and exploitiveness in relationships that was found in Morey’s clinical population. A possible explanation for this might revolve around the make-up of the samples, with fewer personality disorder traits being present in the current study’s sample and Morey’s normal sample. When examining studies that explored the factor structure of the 22 clinical scales, Morey’s (1991) four-factor solution has been replicated in a chronic pain sample (Karlin et al., 2005) and a neuropsychological sample (Frazier et al., 2006). The fivecomponent solution found in the current study was also found by Tasca et al. (2002) in an eating-disordered sample and by Cashel et al. (2003) in a university counseling sample. Thus, varieties of factor structures have been found in other special populations. Hoelzle and Meyer (2009) observed that different results may be found with different factor retention techniques, and this might account for the variability seen in the factor structure reported in various studies. Overall, the current study’s findings varied only slightly from Morey’s original results, even though the CFA did not find that his original model was a good fit for the current data. Thus, these results generally lend support for the use of the PAI with a neuropsychological population. Moreover, the results suggest clinicians could interpret the PAI as suggested by Morey (1991). The one caveat is that when thinking about treatment recommendations a slightly different interpretation of PAI profiles may be

RELIABILITY AND COMPONENT STRUCTURE OF PAI

249

required for special clinical populations. For example, the “Social Distancing” component can be conceptualized as representing individuals who are limited in their ability to be supportive and empathic in relationships, due to possible cognitive rigidity, as well as individuals who perceive a lack of available quality social support in their lives. As a result, it is possible these patients will initially be resistant to developing and partaking in a therapeutic relationship. Gaining a better understanding of these patients will be important for the treating clinician.

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

Limitations There are several limitations involved in the current study. Given the archival nature of the data, the analyses and inferences made were limited by the data contained within the dataset. For example, the sample was predominately Caucasian; thus, making inferences about other ethnic groups from this data will not be possible. In addition, the sample is a mixed clinical sample; thus, it is not possible to generalize about the psychometric properties of the PAI in specific neurological disorders (e.g., epilepsy or brain tumors). In general, caution must be exercised when applying the results to populations of patients that are not highly represented in the current study. It should also be noted that slightly different methods were used when developing the current sample. Morey only excluded individuals based on excessive item omission rates (more than 33); however, this study also used Morey’s criteria for determining whether a profile is invalid and consequently excluded participants who produced an invalid profile. Due to this difference, the current study’s sample might have slightly restricted ranges of scores relative to Morey’s clinical sample. This difference has the potential to impact the reported component structure. However, other similar studies used the same exclusionary criteria as the current study (Karlin et al., 2005; Schinka, 1995; Tasca et al., 2002). Lastly, there are a number of alternate factor or component retention methods available, such as parallel analysis and Velicer’s MAP procedure (O’Connor, 2000), and depending on the method used, results may differ. In the context of PAI studies, Hoelzle and Meyer (2009) identified this as a potential issue in explaining the variability of results. In the current study, once the CFA failed to provide support for Morey’s (1991) original factor analysis, it was decided to replicate his methodology in the follow-up analysis with PCA, including using the criteria of eigenvalue > 1 (reference) for factor retention. However, MAP and/or parallel analysis might have resulted in a different component solution, as noted by Hoelzle and Meyer (2009). It is possible that this component retention guideline could result in the identification of sample-specific components that are not necessarily generalizable to other samples. Furthermore, Fabrigar, Wegener, MacCallum, and Strahan (1999) suggested that this procedure is somewhat arbitrary and studies have shown it can lead to both over- and under-retention of components. Similarly, the current study initially used CFA in an attempt to confirm the models reported by Morey (1991). However, Hopwood and Donnellan (2010) suggested that there are several limitations to using CFA to evaluate the structure of personality trait inventories, due to the characteristic intricacy of personality, concerns related to its measurement, and concerns related to the way CFA models are applied and interpreted. Given these limitations, Hopwood and Donnellan (2010) recommended

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

250

MICHELLE BUSSE ET AL.

(a) using multiple factor analytic methods, (b) considering test structure in the context of previously reported results when making decisions about overall model fit, and (c) considering the practical implications of model modifications used to improve fit. The current study attempted to address the limitation noted by these authors by using both CFA and PCA, and hypothesizing about the constructs measured by the components in the context of previous literature. The limitations noted by Hopewell and Donnellan were illustrated in the current study where the CFA indicated that Morey’s original factor structure was not a good fit, but the follow-up PCA results were only slightly different from Morey’s original solutions. Thus, in terms of practical implications, the differences in interpretation between the normative sample and the current sample would also be minimal. Regardless, replication of the current results with different factor retention criteria and different neuropsychological samples would be highly beneficial.

CONCLUSION Consistent with previous literature (Frazier et al., 2006) and the PAI manual (Morey, 1991), results generally supported the reliability and validity of the PAI in a mixed neuropsychological sample, although there were also similarities to other special samples like alcohol abuse and eating disorders. The psychometric properties of the PAI used in a neuropsychological sample closely aligned with the original data (Morey, 1991), and previous research in a neuropsychological sample (Frazier et al., 2006). Thus, the current study provides additional support for the PAI in neuropsychological populations, although slight modifications in interpretation may be appropriate.

REFERENCES Boone, D. (1998). Internal consistency reliability of the Personality Assessment Inventory with psychiatric inpatients. Journal of Clinical Psychology, 54, 839–843. Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., Dahlstrom, W. G., & Kaemmer, B. (1989). Minnesota Multiphasic Personality Inventory-2 (MMPI-2): Manual for administration and scoring. Minneapolis, MN: University of Minnesota Press. Cashel, M. L., Dollinger, S. J., & Cunningham, J. (2003, August). Construct and discriminant validity of the PAI for university counseling center clients. Poster presented at the American Psychological Association Convention, Toronto, Canada. Demakis, G. J., Hammond, F., Knotts, A., Cooper, D. B., Clement, P., Kennedy, J., & Sawyer, T. (2007). The Personality Assessment Inventory in individuals with traumatic brain injury. Archives of Clinical Neuropsychology, 22, 123–130. doi:10.1016/j.acn.2006.09.004 Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299. Frazier, T. W., Naugle, R. I., & Haggerty, K. A. (2006). Psychometric adequacy and comparability of the short and full forms of the Personality Assessment Inventory. Psychological Assessment, 18(3), 324–333. doi:10.1037/1040-3590.18.3.324 Greene, R. L., & Weiner, I. B. (2008). History of personality assessment. In R. L. Greene, & I. B. Weiner (Eds.), Handbook of personality assessment (pp. 3–18). Hoboken, NJ: John Wiley & Sons.

Downloaded by [Selcuk Universitesi] at 16:28 23 January 2015

RELIABILITY AND COMPONENT STRUCTURE OF PAI

251

Hoelzle, J. B., & Meyer, G. J. (2009). The invariant component structure of the Personality Assessment Inventory (PAI) full scales. Journal of Personality Assessment, 91(2), 175–186. doi:10.1080/00223890802634316 Hopwood, C. J., & Donnellan, M. B. (2010). How should the internal structure of personality inventories be evaluated? Personality and Social Psychology Review, 14(3), 332–346. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. Karlin, B. E., Creech, S. K., Grimes, J. S., Clark, T. S., Meagher, M. W., & Morey, L. C. (2005). The Personality Assessment Inventory with chronic pain patients: Psychometric properties and clinical utility. Journal of Clinical Psychology, 61(12), 1571–1585. Kurtz, J. E., & Blais, M. A. (2007). Introduction to the special issue on the Personality Assessment Inventory. Journal of Personality Assessment, 88(1), 1–4. Millon, T. (1994). Millon Clinical Multiaxial Inventory-III Manual (2nd ed.). Minneapolis, MN: National Computer Systems. Morey, L. C. (1991). Personality Assessment Inventory: Professional Manual. Odessa, FL: Psychological Assessment Resources. Morey, L. C. (2003). Essentials of PAI assessment. Hoboken, NJ: John Wiley & Sons. Nunnally, J. C. (1967). Psychometric theory. New York, NY: McGraw-Hill. O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavioral Research Methods, Instruments & Computers, 23, 396–402. Piotrowski, C. (2000). How popular is the personality assessment inventory in practice and training? Psychological Reports, 86, 65–66. Schinka, J. A. (1995). Personality Assessment Inventory scale characteristics and factor structure in the assessment of alcohol dependency. Journal of Personality Assessment, 64, 101–111. Tasca, G. A., Wood, J., Demidenko, N., & Bissada, H. (2002). Using the PAI with an eating disordered population: Scale characteristics, factor structure, and differences among diagnostic groups. Journal of Personality Assessment, 79, 337–356. Wise, E. A., Streiner, D. L., & Walfish, S. (2010). A review and comparison of the reliabilities of the MMPI-2, MCMI-III, and PAI presented in their respective test manuals. Measurement and Evaluation in Counseling and Development, 42(4), 246–254. doi:10.1177/ 0748175609354594