This article was downloaded by: [New York University] On: 16 June 2015, At: 07:15 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Multivariate Behavioral Research Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hmbr20
Rotation to Maximize the Construct Validity of Factors in the NEO Personality Inventory Robert R. McCrae & Paul T. Costa Jr. Published online: 10 Jun 2010.
To cite this article: Robert R. McCrae & Paul T. Costa Jr. (1989) Rotation to Maximize the Construct Validity of Factors in the NEO Personality Inventory, Multivariate Behavioral Research, 24:1, 107-124, DOI: 10.1207/s15327906mbr2401_7 To link to this article: http://dx.doi.org/10.1207/s15327906mbr2401_7
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.
Downloaded by [New York University] at 07:15 16 June 2015
This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Multivariate Behavioral Research, 24(1), 107-124
Rotation to Maximize the Construct Validity of Factors in the NEO Personality Inventory Robert R. McCrae and Paul T. Costa, Jr.
Downloaded by [New York University] at 07:15 16 June 2015
Gerontology Research Center National Institute on Aging, National Institutes of Health
The NEO Personality Inventory (NEO-PI) consists of 5 global domain and 18 specific facet scales developed to measure aspects of the five major dimensions of normal personality. To obtain optimal measures of these five dimensions from the NEO-PI scales, a method for the orthogonal rotation of principal components to maximize the convergent and discriminant validity of the rotated factors (validimax rotation) is proposed and applied to NEO-PI factors. Self-report data from 983 men and women were used to obtain the factors, and six alternative operationalizations of the fivefactor model were used as external criteria to guide rotation. The rotation obtained was cross-validated on peer and spouse ratings on the NEO-PI, and in a second sample. NEO-PI domain scales, varimax factors, and validimax factors all showed evidence of construct validity, but validimax factors were somewhat superior, especially as measures of Agreeableness and Conscientiousness.
As Gorsuch (1974) pointed out, researchers often rely on existing factor analytic procedures because they are convenient and familiar, even if they are not fully appropriate for the data a t hand. He suggested that the requirements of the analysis be kept in mind when designing the study; when that is not possible, it may be necessary to develop new procedures. This article concerns such a case. I n addition, it advocates a strategy of factor rotation guided by considerations of external validity rather than internal structure, which may have broader applications.
Five Factors in the NEO Personality Inventory The impetus for this research was the need to find an optimal method of scoring the NEO Personality Inventory (NEO-PI; Costa & McCrae, 1985b) to measure the five-factor model of personality. The NEO-PI began as an inventory to measure aspects of three Requests for reprints should be sent to Robert R. McCrae, Personality, Stress and Coping Section, Gerontology Research Center, 4940 Eastern Avenue, Baltimore, MD 21224. JANUARY 1989
107
Downloaded by [New York University] at 07:15 16 June 2015
R. R. McCrae and P. T. Costa, Jr
broad domains of personality-neuroticism (N), extraversion ( E ) , and openness to experience (0).Six specific facets were assessed within each domain, and domain scores were obtained by summing the facet scores. Conventional factor analyses (McCrae & Costa, 1983a) showed that the three hypothesized N, E , and 0 factors could be recovered from analyses of the 18 facet scales in both self-reports and spouse ratings. But more research (Digman & Inouye, 1986; Goldberg, 1981; McCrae & Costa, 1985, 1987) made it clear that two additional dimensions or domains-agreeableness (A) and conscientiousness (C)-were needed to provide a comprehensive assessment of normal personality, and research began on the development of additional scales (McCrae & Costa, 1987). Ideally, both A and C would also have been represented by several facet scales to parallel the first three domains. Presently, however, only global, 18-item scales have been developed and validated. Because both the inventory and the theoretical model it operationalizes were rooted in factor analysis, it would seem logical to factor the scales of the NEO-PI to test the fit between the instrument and the model. Further, factor scores might imp~*ove on the raw scores as measures of the underlying constructs, par-ticularly in the case of the Agreeableness and Conscientiousness scales, which are shorter and somewhat less reliable than the other domain scales. I t is clear, however, that conventional factoi. rotations are unlikely to yield an optimal solution when three of the five factors have six markers, whereas two factors have only a single marker apiece. One solution to a problem of underrepresentation of the A and C factors would be a Procrustes rotation (Schonemann, 1966) in which the hypothesized factors would be represented in a target matrix, and an optimal fit between the observed and hypothesized factor structure would be sought. Our chief interest, howevela, was not in the internal factor structure of the NEO-PI scales, but in the validity of the factors as measures of the five dimensions of personality. This led us to adopt a conceptually distinct, although mathematically related, approach.
Internal and External Criteria JOT Rotatiotr Virtually all factor rotation techniques commonly used are based on the internal structure of the variables analyzed. Thurstone's principle of simple stmcture has been elaborated in oblique 108
MULTIVARIATE BEHAVIORAL RESEARCH
Downloaded by [New York University] at 07:15 16 June 2015
R. R. McCrae and P. T. Costa, Jr.
and orthogonal variations that simplify rows (quartimax) or columns (varimax) of the factor matrix or maximize such related criteria as hyperplane count (Cattell, 1952). All such techniques assume that the "best" position of the factors can be determined by examining the relations of the factors to the variables factored. Procrustes rotations may employ theoretical considerations or prior empirical results to guide rotation, but they are still essentially internal analyses, in which the focus of attention is on the pattern of factor loadings. Even when replicated, there is a certain circularity inherent in this process: The variables are analyzed in terms of the factors, and the factors are defined by their correlations with the variables. Normally this circularity is ignored, and the obtained factors a r e interpreted in terms of broader constructs that have meaning beyond the specific set of variables involved. Ideally, the interpretation of factors is subjected to construct validation by correlation of factor scores with theoretically relevant convergent and discriminant criteria. There is thus a two-stage process: Factors are first extracted, rotated, and interpreted; the interpretation is then validated against external criteria. If the results are unsatisfactory, the process may be repeated, perhaps trying a different type of rotation. When the factor analyst knows in advance what constructs he or she wishes to measure, this is an awkward and inefficient process. Instead of an iterated two-stage process of trial and error, this article suggests an analytic procedure for directly rotating factors to maximize convergent and discriminant validity, which we will call validimaz rotation. I t must be stated at the outset, however, that this procedure will maximize validity only with respect t o the set of external criteria employed; if these are inappropriate or poorly measured, the validimax factors will probably not provide good measures of the intended construct. In this approach, factors are interpreted by their pattern of external correlates rather than by the loading of variables; the goal is construct validity rather than simple structure. I t may still be worthwhile to examine the factor loadings, but the rotation itself is determined by the correlations of factor scores. This means that validimax rotation is particularly well suited to the measurement of theoretically defined constructs in individuals, and it should have practical utility in psychological assessment. The constructs of interest in this case are the five major dimensions of personality, which have been identified in analyses JANUARY 1989
109
R. R. McCrae and P. T. Costa, Jr
Downloaded by [New York University] at 07:15 16 June 2015
of a number of different methods and instruments. Self-reports, spouse ratings, peer ratings, and interviewer Q-sorts provide a variety of operationalizations of the model, and a strong argument could be made for claiming that the best rotation of factors from the NEO-PI is that which simultaneously maximizes convergent and discriminant validity against such a diverse set of external criteria. Factor scores resulting from this rotation should provide optimal measures of the five-factor model.
Rationale and Method yf Validimax Rotatior/
A number of different approaches to the problem of optimizing external validity might be taken, ranging from hand rotation (e.g., McCrae, Costa, & Busch, 1986) to structural equations modeling (e.g., Joreskog & Sorbom, 1979), in which the effects of a variety of alternative assumptions about the nature of the factors and the measures could be compared. Based on our conceptualization of personality and our experience with personality measures, we are prepared to assume that the five dimensions are orthogonal, and that principal components are an acceptable approximation to the underlying factors. These simplifying assumptions make it possible to use a simpler technique. The mathematic basis for our validimax rotation is Schonemann's (1966) orthogonal Procrustes solution. Procrustes rotation is normally applied to a set of factor loadings to find a leastsquares best fit to a target (e.g., McCrae & Costa, 1983a). Our interest, however, is with the matrix of correlations between the factors and external criterion variables, and we seek a rotation that maximizes the convergent and minimizes the discriminant correlations. Provided that we are dealing with orthogonal components (and with correlations based on the same sample as that in which the factors are derived), correlating rotated factors with criteria is mathematically equivalent to "rotating" correlations between the unrotated factors and the criteria. Thus, Schonemann's technique can be applied directly to the matrix of correlations between the unrotated factors and external variables; the target matrix is composed of 1s and 0s which represent hypothesized convergent and discriminant correlations, respectively. Intuitively, it may be helpful to recall that in the orthogonal case, factor loadings are equivalent to the correlations of the variables with the factors; similarly, the criterion correlations can be viewcvl 110
MULTIVARIATE BEHAVIORAL RESEARCH
Downloaded by [New York University] at 07:15 16 June 2015
R. R. McCrae and P. T. Costa, Jr.
as "loadings" on the factors that can be rotated to the target. The result is a transformation matrix that can be applied directly to the initial unrotated factors to yield a rotated factor pattern and factor scoring weights and scores. Like all Procmstes techniques, this method capitalizes on chance to an unknown degree (but see Acito & Anderson, 1980, for a defense of these procedures). To evaluate the generalizability of the solution, two kinds of cross-validation are examined. First, the factor scoring matrix generated from self-reports on the NEOP I is applied to spouse and peer ratings on the rating form of the NEO-PI to generalize across observers. Second, the factors are scored on a second sample for whom both self-reports and peer ratings are available to generalize across samples.
Method Subjects Subjects for the major analyses were 983 men and women who completed the NEO-PI in 1986. Of these, 434 had participated for the previous 6 years in a series of studies as members of the Augmented Baltimore Longitudinal Study of Aging (BLSA; Shock e t al., 1984). Different subsets of these individuals had provided data on other self-report instruments and had been the targets of ratings by spouses, peers, and trained interviewers. The other 549 subjects in the sample were recruited from participants in the BLSA who were not previously enrolled in the studies of personality, and from the group of peers who provided ratings of the original group. The 502 men and 481 women in the full sample ranged in age from 19 to 93 years. In general, they are well-educated and healthy community-dwelling volunteers; details are given elsewhere (Costa & McCrae, 1988). The second, cross-validation sample consisted of 100 men from a small college (Costa, McCrae, & Dembroski, 1989). They were asked t o nominate two male and two female acquaintances who could provide data on the rating form of the NEO-PI. Mean ratings were calculated from a total of 344 raters.
Measures The NEO-PI is a 181-item questionnaire developed through factor analysis to f i t a five-dimensional model of personality (Costa & McCrae, 1985b). An earlier version, the NEO Inventory (McCrae JANUARY 1989
111
Downloaded by [New York University] at 07:15 16 June 2015
R. R. McCrae and P. T. Costa, Jr
& Costa, 1983a), measured six traits in each of the three domains of N, E , and 0 . Recent modifications (McCrae & Costa, 1987) have added two new scales to measure the domains of A and C. Item scoring in the NEO-PI is balanced to control for acyuiescence, and socially desirable responding does not appear to bias scores (McCrae & Costa, 1983b). Internal consistencies for- the 48-item N, E , and O scales ranged from .87 to .93 in this sample; internal consistencies for the 18-item A and C scales were .76 and .86, respectively (Costa & McCrae, 1988). Uncorrected 6-year stability coefficients for N, E , and 0 scales were .83,.82, and .83, respectively. The NEO-PI has been correlated with other inventories, observer ratings, and sentence completions and has been used in the prediction of somatic complaints, psychological wellbeing, and coping behavior (Costa & McCrae, 1985b). An observer-rating version of the NEO-PI-Form R-parallels the self-report version except that items are phrased in the third person. Details on reliability and validity of Form R are given in the manual (Costa & McCrae, 1985b); in general, they are comparable to those found for self-reports. Form R of the NEO-PI was administered to from one to four peer raters in 1983 (McCrae & Costa, 1987) and to spouses in 1986. Mean peer ratings and spouse ratings on the NEO-PI domain scales form the first two sets of external criteria to serve as targets for factor rotation of the self-report data. Several additional instruments have been used to measure the five-factor model of personality. On the basis of analyses of trait adjectives in several samples of college students, Goldberg (1983) devised a 40-item list of 9-point bipolar adjective scales to measure his conception of the five-factor model. In subsequent research, we added 40 additional items to his list and interpreted five varimax factors extracted from the full 80 items as measures of N, E , 0, A, and C (McCrae & Costa, 1985, 1987). Mean peer ratings and self-reports on these factors are used as the third and fourth sets of external criteria. Self-Q-sorts on Block's (1961) California Q-Set (CQS) were obtained from another subset of individuals between 1981 and 1985; again, five varimax factors were extracted that resembled N, E, 0, A, and C. Further hand rotation was employtd to improve the interpretation of the E and A factors. Factor. scores based on these self-reports form the fifth set of criteria. Based on a short life history interview, two trained raters recorded theil112
MULTIVARIATE BEHAVIORAL RESEARCH
Downloaded by [New York University] at 07:15 16 June 2015
R. R. McCrae and P. T. Costa, Jr.
impressions of some of these individuals using a slightly different form of the CQS, and scales were created from these mean ratings by summing the items that defined the self-report factors (see McCrae, Costa, & Busch, 1986, for details) to provide the last set of criteria. The fact that personality data were collected over a period of 6 years probably has little impact on the outcome, because personality is highly stable in the adult years (Costa & McCrae, 1985a). The fact that different numbers and subsets of subjects provided data on different instruments does present some problems for the analysis, because the mathematical equations become only approximate when correlations are based on different subsets. However, previous analyses have suggested that there is little systematic difference between those who did and did not participate in various aspects of the study, and supplementary analyses suggested that the validimax rotation was not particularly sensitive to changes in the sample. Thus, in this article all available data are used; specific Ns are presented in the tables.
Analyses Three sets of scores are compared in this article: raw domain scale scores from the NEO-PI; varimax-rotated principal component scores from an analysis of the 18 N, E , and 0 facets and the A and C domain scales; and validimax-rotated components from the same set of variables. (N, E , and 0 domain scores were omitted from the factor analyses because they are linear combinations of the facet scales.) The Statistical Analysis System (SAS Institute Inc., 1982) Factor procedure was used to generate varimax-rotated factors and factor scores; the Matrix procedure was used to derive the validimax transformation matrix and other matrices of interest. (A sample SAS program is available from the authors.) For validimax rotation, Schonemann's technique was applied to the 5 x 30 matrix of correlations between the NEOP I factors and the six sets of external criteria; therefore, it simultaneously maximizes 30 convergent correlations and minimizes 120 discriminant correlations. Coefficients of congruence (Wrigley & Neuhaus, 1955) were used to compare solutions calculated separately for men and women and to compare solutions in self-report data with solutions in peer and spouse rating data. Mean correlations a r e calculated using Fisher's x-transformation, weighted by the number of observations. JANUARY 1989
113
R. R. McCrae and P. T. Costa, Jr
Downloaded by [New York University] at 07:15 16 June 2015
Results Principal components analysis of the 20 self-report NEO-PI scales showed that five factors had eigenvalues greater than 1.0: together they accounted for 6'3% of the variance. Results from the varimax rotation are given in Table 1. T h e loadings here show relatively clear N and 0 factors, and the A and C scales define two of the other factors. The Extraversion factor, however, does not cohere quite as hypothesized; Assertiveness and Activity have higher loadings on the C factor than the intended E factor, and Excitement Seeking has a higher (absolute) loading on the A factor. Hostility also shows a higher (absolute) loading on the A factor than on its intended N factor. However, despite the imbalance in the number of definers for the five factors, the varimax rotation places 16 of the 20 scales on the hypothesized factor. The varimax factors also show substantial correlations with the corresponding domain scales, ranging from .75 to .97. Table 1 also includes factor loadings for the validimax solution in the full sample. (Analyses were repeated for men and women separately, with similar results; coefficients of congruence ranged from .81 to .98, suggesting at least a rough replication across sexes.) I t should be recalled that the factor pattern matrix is of secondary concern in validimax rotation, which is based solely on correlations between factor scores and external criteria. Factor loadings are normally viewed as simple-structure representations of the relations among the factored variables, and it is relatively easy to interpret the validimax loadings in this light. But simple structure was not a criterion for rotation here, and the loadings should be seen as representing the position of NEO-PI scales within the theoretical space of the five-factor model. Considered this way, it is encouraging to see that the hypothesized structure of the N, E, and 0 factors is clearly recovered. Openness to Feelings has a somewhat larger loading on the E factor than on the hypothesized 0 factor, and both Hostility and Excitement Seeking have equal loadings on their intended factor and A, but all other variables load chiefly on the expected factor. I t seems probable that the validimax solution approximates the hypothesized structure better than the varimax solution, because the latter is more sensitive to the imbalance in the number of definers per factor. Of particular substantive interest are the loadings of the 18 114
MULTIVARIATE BEHAVIORAL RESEARCH
c
% Downloaded by [New York University] at 07:15 16 June 2015
$
n
Multivariate Behavioral Research Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hmbr20
Rotation to Maximize the Construct Validity of Factors in the NEO Personality Inventory Robert R. McCrae & Paul T. Costa Jr. Published online: 10 Jun 2010.
To cite this article: Robert R. McCrae & Paul T. Costa Jr. (1989) Rotation to Maximize the Construct Validity of Factors in the NEO Personality Inventory, Multivariate Behavioral Research, 24:1, 107-124, DOI: 10.1207/s15327906mbr2401_7 To link to this article: http://dx.doi.org/10.1207/s15327906mbr2401_7
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.
Downloaded by [New York University] at 07:15 16 June 2015
This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Multivariate Behavioral Research, 24(1), 107-124
Rotation to Maximize the Construct Validity of Factors in the NEO Personality Inventory Robert R. McCrae and Paul T. Costa, Jr.
Downloaded by [New York University] at 07:15 16 June 2015
Gerontology Research Center National Institute on Aging, National Institutes of Health
The NEO Personality Inventory (NEO-PI) consists of 5 global domain and 18 specific facet scales developed to measure aspects of the five major dimensions of normal personality. To obtain optimal measures of these five dimensions from the NEO-PI scales, a method for the orthogonal rotation of principal components to maximize the convergent and discriminant validity of the rotated factors (validimax rotation) is proposed and applied to NEO-PI factors. Self-report data from 983 men and women were used to obtain the factors, and six alternative operationalizations of the fivefactor model were used as external criteria to guide rotation. The rotation obtained was cross-validated on peer and spouse ratings on the NEO-PI, and in a second sample. NEO-PI domain scales, varimax factors, and validimax factors all showed evidence of construct validity, but validimax factors were somewhat superior, especially as measures of Agreeableness and Conscientiousness.
As Gorsuch (1974) pointed out, researchers often rely on existing factor analytic procedures because they are convenient and familiar, even if they are not fully appropriate for the data a t hand. He suggested that the requirements of the analysis be kept in mind when designing the study; when that is not possible, it may be necessary to develop new procedures. This article concerns such a case. I n addition, it advocates a strategy of factor rotation guided by considerations of external validity rather than internal structure, which may have broader applications.
Five Factors in the NEO Personality Inventory The impetus for this research was the need to find an optimal method of scoring the NEO Personality Inventory (NEO-PI; Costa & McCrae, 1985b) to measure the five-factor model of personality. The NEO-PI began as an inventory to measure aspects of three Requests for reprints should be sent to Robert R. McCrae, Personality, Stress and Coping Section, Gerontology Research Center, 4940 Eastern Avenue, Baltimore, MD 21224. JANUARY 1989
107
Downloaded by [New York University] at 07:15 16 June 2015
R. R. McCrae and P. T. Costa, Jr
broad domains of personality-neuroticism (N), extraversion ( E ) , and openness to experience (0).Six specific facets were assessed within each domain, and domain scores were obtained by summing the facet scores. Conventional factor analyses (McCrae & Costa, 1983a) showed that the three hypothesized N, E , and 0 factors could be recovered from analyses of the 18 facet scales in both self-reports and spouse ratings. But more research (Digman & Inouye, 1986; Goldberg, 1981; McCrae & Costa, 1985, 1987) made it clear that two additional dimensions or domains-agreeableness (A) and conscientiousness (C)-were needed to provide a comprehensive assessment of normal personality, and research began on the development of additional scales (McCrae & Costa, 1987). Ideally, both A and C would also have been represented by several facet scales to parallel the first three domains. Presently, however, only global, 18-item scales have been developed and validated. Because both the inventory and the theoretical model it operationalizes were rooted in factor analysis, it would seem logical to factor the scales of the NEO-PI to test the fit between the instrument and the model. Further, factor scores might imp~*ove on the raw scores as measures of the underlying constructs, par-ticularly in the case of the Agreeableness and Conscientiousness scales, which are shorter and somewhat less reliable than the other domain scales. I t is clear, however, that conventional factoi. rotations are unlikely to yield an optimal solution when three of the five factors have six markers, whereas two factors have only a single marker apiece. One solution to a problem of underrepresentation of the A and C factors would be a Procrustes rotation (Schonemann, 1966) in which the hypothesized factors would be represented in a target matrix, and an optimal fit between the observed and hypothesized factor structure would be sought. Our chief interest, howevela, was not in the internal factor structure of the NEO-PI scales, but in the validity of the factors as measures of the five dimensions of personality. This led us to adopt a conceptually distinct, although mathematically related, approach.
Internal and External Criteria JOT Rotatiotr Virtually all factor rotation techniques commonly used are based on the internal structure of the variables analyzed. Thurstone's principle of simple stmcture has been elaborated in oblique 108
MULTIVARIATE BEHAVIORAL RESEARCH
Downloaded by [New York University] at 07:15 16 June 2015
R. R. McCrae and P. T. Costa, Jr.
and orthogonal variations that simplify rows (quartimax) or columns (varimax) of the factor matrix or maximize such related criteria as hyperplane count (Cattell, 1952). All such techniques assume that the "best" position of the factors can be determined by examining the relations of the factors to the variables factored. Procrustes rotations may employ theoretical considerations or prior empirical results to guide rotation, but they are still essentially internal analyses, in which the focus of attention is on the pattern of factor loadings. Even when replicated, there is a certain circularity inherent in this process: The variables are analyzed in terms of the factors, and the factors are defined by their correlations with the variables. Normally this circularity is ignored, and the obtained factors a r e interpreted in terms of broader constructs that have meaning beyond the specific set of variables involved. Ideally, the interpretation of factors is subjected to construct validation by correlation of factor scores with theoretically relevant convergent and discriminant criteria. There is thus a two-stage process: Factors are first extracted, rotated, and interpreted; the interpretation is then validated against external criteria. If the results are unsatisfactory, the process may be repeated, perhaps trying a different type of rotation. When the factor analyst knows in advance what constructs he or she wishes to measure, this is an awkward and inefficient process. Instead of an iterated two-stage process of trial and error, this article suggests an analytic procedure for directly rotating factors to maximize convergent and discriminant validity, which we will call validimaz rotation. I t must be stated at the outset, however, that this procedure will maximize validity only with respect t o the set of external criteria employed; if these are inappropriate or poorly measured, the validimax factors will probably not provide good measures of the intended construct. In this approach, factors are interpreted by their pattern of external correlates rather than by the loading of variables; the goal is construct validity rather than simple structure. I t may still be worthwhile to examine the factor loadings, but the rotation itself is determined by the correlations of factor scores. This means that validimax rotation is particularly well suited to the measurement of theoretically defined constructs in individuals, and it should have practical utility in psychological assessment. The constructs of interest in this case are the five major dimensions of personality, which have been identified in analyses JANUARY 1989
109
R. R. McCrae and P. T. Costa, Jr
Downloaded by [New York University] at 07:15 16 June 2015
of a number of different methods and instruments. Self-reports, spouse ratings, peer ratings, and interviewer Q-sorts provide a variety of operationalizations of the model, and a strong argument could be made for claiming that the best rotation of factors from the NEO-PI is that which simultaneously maximizes convergent and discriminant validity against such a diverse set of external criteria. Factor scores resulting from this rotation should provide optimal measures of the five-factor model.
Rationale and Method yf Validimax Rotatior/
A number of different approaches to the problem of optimizing external validity might be taken, ranging from hand rotation (e.g., McCrae, Costa, & Busch, 1986) to structural equations modeling (e.g., Joreskog & Sorbom, 1979), in which the effects of a variety of alternative assumptions about the nature of the factors and the measures could be compared. Based on our conceptualization of personality and our experience with personality measures, we are prepared to assume that the five dimensions are orthogonal, and that principal components are an acceptable approximation to the underlying factors. These simplifying assumptions make it possible to use a simpler technique. The mathematic basis for our validimax rotation is Schonemann's (1966) orthogonal Procrustes solution. Procrustes rotation is normally applied to a set of factor loadings to find a leastsquares best fit to a target (e.g., McCrae & Costa, 1983a). Our interest, however, is with the matrix of correlations between the factors and external criterion variables, and we seek a rotation that maximizes the convergent and minimizes the discriminant correlations. Provided that we are dealing with orthogonal components (and with correlations based on the same sample as that in which the factors are derived), correlating rotated factors with criteria is mathematically equivalent to "rotating" correlations between the unrotated factors and the criteria. Thus, Schonemann's technique can be applied directly to the matrix of correlations between the unrotated factors and external variables; the target matrix is composed of 1s and 0s which represent hypothesized convergent and discriminant correlations, respectively. Intuitively, it may be helpful to recall that in the orthogonal case, factor loadings are equivalent to the correlations of the variables with the factors; similarly, the criterion correlations can be viewcvl 110
MULTIVARIATE BEHAVIORAL RESEARCH
Downloaded by [New York University] at 07:15 16 June 2015
R. R. McCrae and P. T. Costa, Jr.
as "loadings" on the factors that can be rotated to the target. The result is a transformation matrix that can be applied directly to the initial unrotated factors to yield a rotated factor pattern and factor scoring weights and scores. Like all Procmstes techniques, this method capitalizes on chance to an unknown degree (but see Acito & Anderson, 1980, for a defense of these procedures). To evaluate the generalizability of the solution, two kinds of cross-validation are examined. First, the factor scoring matrix generated from self-reports on the NEOP I is applied to spouse and peer ratings on the rating form of the NEO-PI to generalize across observers. Second, the factors are scored on a second sample for whom both self-reports and peer ratings are available to generalize across samples.
Method Subjects Subjects for the major analyses were 983 men and women who completed the NEO-PI in 1986. Of these, 434 had participated for the previous 6 years in a series of studies as members of the Augmented Baltimore Longitudinal Study of Aging (BLSA; Shock e t al., 1984). Different subsets of these individuals had provided data on other self-report instruments and had been the targets of ratings by spouses, peers, and trained interviewers. The other 549 subjects in the sample were recruited from participants in the BLSA who were not previously enrolled in the studies of personality, and from the group of peers who provided ratings of the original group. The 502 men and 481 women in the full sample ranged in age from 19 to 93 years. In general, they are well-educated and healthy community-dwelling volunteers; details are given elsewhere (Costa & McCrae, 1988). The second, cross-validation sample consisted of 100 men from a small college (Costa, McCrae, & Dembroski, 1989). They were asked t o nominate two male and two female acquaintances who could provide data on the rating form of the NEO-PI. Mean ratings were calculated from a total of 344 raters.
Measures The NEO-PI is a 181-item questionnaire developed through factor analysis to f i t a five-dimensional model of personality (Costa & McCrae, 1985b). An earlier version, the NEO Inventory (McCrae JANUARY 1989
111
Downloaded by [New York University] at 07:15 16 June 2015
R. R. McCrae and P. T. Costa, Jr
& Costa, 1983a), measured six traits in each of the three domains of N, E , and 0 . Recent modifications (McCrae & Costa, 1987) have added two new scales to measure the domains of A and C. Item scoring in the NEO-PI is balanced to control for acyuiescence, and socially desirable responding does not appear to bias scores (McCrae & Costa, 1983b). Internal consistencies for- the 48-item N, E , and O scales ranged from .87 to .93 in this sample; internal consistencies for the 18-item A and C scales were .76 and .86, respectively (Costa & McCrae, 1988). Uncorrected 6-year stability coefficients for N, E , and 0 scales were .83,.82, and .83, respectively. The NEO-PI has been correlated with other inventories, observer ratings, and sentence completions and has been used in the prediction of somatic complaints, psychological wellbeing, and coping behavior (Costa & McCrae, 1985b). An observer-rating version of the NEO-PI-Form R-parallels the self-report version except that items are phrased in the third person. Details on reliability and validity of Form R are given in the manual (Costa & McCrae, 1985b); in general, they are comparable to those found for self-reports. Form R of the NEO-PI was administered to from one to four peer raters in 1983 (McCrae & Costa, 1987) and to spouses in 1986. Mean peer ratings and spouse ratings on the NEO-PI domain scales form the first two sets of external criteria to serve as targets for factor rotation of the self-report data. Several additional instruments have been used to measure the five-factor model of personality. On the basis of analyses of trait adjectives in several samples of college students, Goldberg (1983) devised a 40-item list of 9-point bipolar adjective scales to measure his conception of the five-factor model. In subsequent research, we added 40 additional items to his list and interpreted five varimax factors extracted from the full 80 items as measures of N, E , 0, A, and C (McCrae & Costa, 1985, 1987). Mean peer ratings and self-reports on these factors are used as the third and fourth sets of external criteria. Self-Q-sorts on Block's (1961) California Q-Set (CQS) were obtained from another subset of individuals between 1981 and 1985; again, five varimax factors were extracted that resembled N, E, 0, A, and C. Further hand rotation was employtd to improve the interpretation of the E and A factors. Factor. scores based on these self-reports form the fifth set of criteria. Based on a short life history interview, two trained raters recorded theil112
MULTIVARIATE BEHAVIORAL RESEARCH
Downloaded by [New York University] at 07:15 16 June 2015
R. R. McCrae and P. T. Costa, Jr.
impressions of some of these individuals using a slightly different form of the CQS, and scales were created from these mean ratings by summing the items that defined the self-report factors (see McCrae, Costa, & Busch, 1986, for details) to provide the last set of criteria. The fact that personality data were collected over a period of 6 years probably has little impact on the outcome, because personality is highly stable in the adult years (Costa & McCrae, 1985a). The fact that different numbers and subsets of subjects provided data on different instruments does present some problems for the analysis, because the mathematical equations become only approximate when correlations are based on different subsets. However, previous analyses have suggested that there is little systematic difference between those who did and did not participate in various aspects of the study, and supplementary analyses suggested that the validimax rotation was not particularly sensitive to changes in the sample. Thus, in this article all available data are used; specific Ns are presented in the tables.
Analyses Three sets of scores are compared in this article: raw domain scale scores from the NEO-PI; varimax-rotated principal component scores from an analysis of the 18 N, E , and 0 facets and the A and C domain scales; and validimax-rotated components from the same set of variables. (N, E , and 0 domain scores were omitted from the factor analyses because they are linear combinations of the facet scales.) The Statistical Analysis System (SAS Institute Inc., 1982) Factor procedure was used to generate varimax-rotated factors and factor scores; the Matrix procedure was used to derive the validimax transformation matrix and other matrices of interest. (A sample SAS program is available from the authors.) For validimax rotation, Schonemann's technique was applied to the 5 x 30 matrix of correlations between the NEOP I factors and the six sets of external criteria; therefore, it simultaneously maximizes 30 convergent correlations and minimizes 120 discriminant correlations. Coefficients of congruence (Wrigley & Neuhaus, 1955) were used to compare solutions calculated separately for men and women and to compare solutions in self-report data with solutions in peer and spouse rating data. Mean correlations a r e calculated using Fisher's x-transformation, weighted by the number of observations. JANUARY 1989
113
R. R. McCrae and P. T. Costa, Jr
Downloaded by [New York University] at 07:15 16 June 2015
Results Principal components analysis of the 20 self-report NEO-PI scales showed that five factors had eigenvalues greater than 1.0: together they accounted for 6'3% of the variance. Results from the varimax rotation are given in Table 1. T h e loadings here show relatively clear N and 0 factors, and the A and C scales define two of the other factors. The Extraversion factor, however, does not cohere quite as hypothesized; Assertiveness and Activity have higher loadings on the C factor than the intended E factor, and Excitement Seeking has a higher (absolute) loading on the A factor. Hostility also shows a higher (absolute) loading on the A factor than on its intended N factor. However, despite the imbalance in the number of definers for the five factors, the varimax rotation places 16 of the 20 scales on the hypothesized factor. The varimax factors also show substantial correlations with the corresponding domain scales, ranging from .75 to .97. Table 1 also includes factor loadings for the validimax solution in the full sample. (Analyses were repeated for men and women separately, with similar results; coefficients of congruence ranged from .81 to .98, suggesting at least a rough replication across sexes.) I t should be recalled that the factor pattern matrix is of secondary concern in validimax rotation, which is based solely on correlations between factor scores and external criteria. Factor loadings are normally viewed as simple-structure representations of the relations among the factored variables, and it is relatively easy to interpret the validimax loadings in this light. But simple structure was not a criterion for rotation here, and the loadings should be seen as representing the position of NEO-PI scales within the theoretical space of the five-factor model. Considered this way, it is encouraging to see that the hypothesized structure of the N, E, and 0 factors is clearly recovered. Openness to Feelings has a somewhat larger loading on the E factor than on the hypothesized 0 factor, and both Hostility and Excitement Seeking have equal loadings on their intended factor and A, but all other variables load chiefly on the expected factor. I t seems probable that the validimax solution approximates the hypothesized structure better than the varimax solution, because the latter is more sensitive to the imbalance in the number of definers per factor. Of particular substantive interest are the loadings of the 18 114
MULTIVARIATE BEHAVIORAL RESEARCH
c
% Downloaded by [New York University] at 07:15 16 June 2015
$
n
