This article was downloaded by: [134.117.10.200] On: 08 May 2015, At: 21:00 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Personality Assessment Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hjpa20

Criterion Validity and Practical Utility of the Minnesota Multiphasic Personality Inventory–2–Restructured Form (MMPI–2–RF) in Assessments of Police Officer Candidates a

b

c

a

Anthony M. Tarescavage , David M. Corey , Herbert M. Gupton & Yossef S. Ben-Porath a

Department of Psychology, Kent State University

b

Corey & Stewart, Portland, Oregon

c

Honololu Police Department, Honolulu, Hawaii Published online: 14 Jan 2015.

Click for updates To cite this article: Anthony M. Tarescavage, David M. Corey, Herbert M. Gupton & Yossef S. Ben-Porath (2015): Criterion Validity and Practical Utility of the Minnesota Multiphasic Personality Inventory–2–Restructured Form (MMPI–2–RF) in Assessments of Police Officer Candidates, Journal of Personality Assessment, DOI: 10.1080/00223891.2014.995800 To link to this article: http://dx.doi.org/10.1080/00223891.2014.995800

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions

Journal of Personality Assessment, 0(0), 1–13, 2015 Copyright Ó Taylor & Francis Group, LLC ISSN: 0022-3891 print / 1532-7752 online DOI: 10.1080/00223891.2014.995800

Criterion Validity and Practical Utility of the Minnesota Multiphasic Personality Inventory–2–Restructured Form (MMPI–2–RF) in Assessments of Police Officer Candidates ANTHONY M. TARESCAVAGE,1 DAVID M. COREY,2 HERBERT M. GUPTON,3 AND YOSSEF S. BEN-PORATH1 1

Downloaded by [134.117.10.200] at 21:00 08 May 2015

Department of Psychology, Kent State University 2 Corey & Stewart, Portland, Oregon 3 Honololu Police Department, Honolulu, Hawaii

Minnesota Multiphasic Personality Inventory–2–Restructured Form scores for 145 male police officer candidates were compared with supervisor ratings of field performance and problem behaviors during their initial probationary period. Results indicated that the officers produced meaningfully lower and less variant substantive scale scores compared to the general population. After applying a statistical correction for range restriction, substantive scale scores from all domains assessed by the inventory demonstrated moderate to large correlations with performance criteria. The practical significance of these results was assessed with relative risk ratio analyses that examined the utility of specific cutoffs on scales demonstrating associations with performance criteria.

Police misconduct is a relatively infrequent problem, but because of the increased power and responsibility associated with these positions, these incidents affect public opinion and citizen safety (Bradford, Jackson, & Stanko, 2009; Carr, Napolitano, & Keating, 2007; Mastrofski, Reisig, & McCluskey, 2002; Mazerolle, Bennett, Antrobus, & Eggins, 2012; Weitzer, 2002). For these reasons, the screening process for police officers is substantially more involved than in other personnel selection contexts. Indeed, in 1973 the federal government recommended that all police officers undergo a screening process that included a test of mental ability, a background investigation, an oral interview, and a psychological evaluation (National Advisory Commission on Criminal Justice Standards and Goals, 1973). Since the National Advisory Commission’s recommendation, psychological assessment has become a routine component of police officer screening (Cochrane, Tett, & Vandecreek, 2003), and most states have identified standard practices for psychological screening (see Corey & Borum, 2013, for a review). Of these states, California has developed some of the most comprehensive standards (California Commission on Peace Officer Standards and Training [POST], 2005), which are often emulated by other jurisdictions. Psychological testing is mandated according to California POST Commission Regulation 1955, with all police officer candidates administered at least two self-report instruments that measure normal and abnormal behaviors. This practice is common in most police officer evaluations (Serafino, 2010). As reported by Cochrane et al. (2003), a variety of psychological tests are typically used in police officer psychological evaluations. Commonly used measures include the Personality

Assessment Inventory (PAI; Morey, 2007), Inwald Personality Inventory (IPI; Inwald, Knatz, & Shusman, 1982), California Psychological Inventory (CPI; Gough, 1956), NEO Personality Inventory–Revised (Costa & MacCrae, 1992), 16 Personality Factor Questionnaire (Cattell & Eber, 1950), and the Minnesota Multiphasic Personality Inventory–2 (MMPI–2; Butcher et al., 2001). One factor contributing to preference for these instruments is the availability of research demonstrating their validity in police candidate evaluations, which is a recommended characteristic of any test used for this purpose (International Association of Chiefs of Police, 2009). A challenging aspect of establishing the validity of psychological tests in police officer assessments is that outcome data are available only for candidates who are hired based on several considerations, which typically include results of a written entrance test, polygraph examination, and background investigations. In addition to these preselection factors, officers are selected in part based on the findings from their psychological evaluation, which often includes MMPI–2 or MMPI–2– Restructured Form (MMPI–2–RF) scores. Because of these preselection and selection factors, the population of hired police officers has substantially lower and less variant scores on psychological tests than do members of the general population. In addition to the preselection and selection factors just discussed, police officer candidates are typically more defensive than the general population (Carpenter & Raza, 1987; Detrick & Chibnall, 2008; Detrick, Chibnall, & Call, 2010; Hiatt & Hargrave, 1988; P. A. Weiss, Vivian, Weiss, Davis, & Rostow, 2013), which further restricts the range of their psychological test scores. Perhaps for this reason, research has demonstrated that defensive responding on the PAI and MMPI–2 in this setting is associated with reduced estimates of score validity (Caillouet, Boccaccini, Varela, Davis, & Rostow, 2010; Lowmaster & Morey, 2012). Previous studies have addressed the issue of range restriction by applying statistical corrections for this phenomenon

Received June 6, 2014; Revised October 11, 2014. Address correspondence to Anthony M. Tarescavage, Department of Psychology, Kent State University, 144 Kent Hall, Kent, OH 44242; Email: [email protected]

1

Downloaded by [134.117.10.200] at 21:00 08 May 2015

2 (Lowmaster & Morey, 2012; Sellbom, Fischler, & Ben-Porath, 2007). For example, Sellbom et al. (2007) found in a sample of police officers from Minnesota that the prehire scores on the Clinical and Restructured Clinical (RC) scales of the MMPI–2 had moderate to strong disattenuated correlations with supervisor ratings of police officer problem behaviors when applying formulas derived by Hunter and Schmidt (2004). If these correlates were not corrected for range restriction, the small zero-order effect sizes they obtained could have created the mistaken impression that the MMPI–2 scales had limited validity in predicting problematic outcomes in police officer candidates. In fact, follow-up relative risk ratio (RRR) analyses reported by Sellbom et al. (2007) indicated substantially increased risk for dysfunctional behaviors associated with Clinical and RC scales demonstrating small zeroorder correlations with the outcome variables. Moreover, the test’s Clinical scales have a well-documented research base in police officer samples (Boes, Chandler, & Timm, 1997; Brewster & Stoloff, 1999; Ones, Viswesvaran, Cullen, Drees, & Langkamp, 2003; Scogin, Schumacher, Gardner, & Chaplin, 1995; Sellbom et al., 2007; W. U. Weiss, Davis, Rostow, & Kinsman, 2003). With regard to validity, Sellbom et al. (2007) found that the RC scale validity estimates were relatively stronger than the MMPI–2’s original Clinical scales estimates among police officer candidates. These findings converge with other research showing that the RC scales have improved psychometric features when compared to the Clinical scales, especially in terms of discriminant validity (Arbisi, Sellbom, & Ben-Porath, 2008; Ben-Porath & Tellegen, 2008; Handel & Archer, 2008; Sellbom, Ben-Porath, & Graham, 2006; Sellbom, Graham, & Schenk, 2006). The MMPI–2–RF (Ben-Porath & Tellegen, 2008/2011), which is the focus of the current investigation, is an alternative instrument created from the MMPI–2 item pool. It has two new scale sets to assist with and complement interpretation of the RC scales (i.e., the Higher-Order and Specific Problems scales), in addition to revised versions of the MMPI–2 Validity and Personality Psychopathology–5 (PSY–5) scales. The MMPI–2–RF Validity scales were designed to assess for random, acquiescent, counteracquiescent, overreporting, and underreporting response styles. The Higher-Order scales measure three superordinate constructs identified in factor analyses of the RC scales (Emotional/Internalizing Dysfunction, Thought Dysfunction, and Behavioral/Externalizing Dysfunction). The Specific Problems scales measure RC scale subdomains and other more narrowly focused constructs, and the PSY–5 scales assess constructs associated with personality psychopathology. The reliability and validity of MMPI–2–RF scores have been extensively documented in the MMPI–2–RF Technical Manual (Tellegen & Ben-Porath, 2008/2011), and its psychometric performance has been studied in more than 200 peer-reviewed articles (University of Minnesota Press, 2014). The test manual and scoring software include comparison group data (means and standard deviations on the 51 MMPI–2–RF scales) for a variety of samples from settings in which the inventory is used, including police officer candidates. Although the literature on use of the MMPI–2–RF scales in evaluations of police candidates is developing (Sellbom et al., 2007; Tarescavage, Brewster, Corey, & Ben-Porath, 2014; Tarescavage, Corey, & Ben-Porath, 2014), more

TARESCAVAGE, COREY, GUPTON, BEN-PORATH research on the validity of the MMPI–2–RF in this setting is needed to identify the most generalizable associations.

THIS STUDY The purpose of this study was to examine associations between preemployment scores on all of the MMPI–2–RF substantive scales and postemployment problem behaviors in a diverse sample of police officers undergoing their probationary period. To this end, we calculated zero-order correlations between scores on the substantive scales of the MMPI–2–RF and field training officer evaluations of their probationary period performance. These analyses can be used to identify psychological constructs measured by the MMPI–2–RF that are associated with job-related problems and to guide the use of the MMPI–2–RF in the context of preemployment evaluations of police officer candidates, particularly after integrating findings from other studies. Consistent with this goal, we interpreted the findings in the context of discovery as delineated by Reichenbach (1938), who argued for the merits of using scientific findings to generate hypotheses. We also calculated range restriction-corrected correlations between MMPI–2–RF scores and these variables to account for the effect of preselection and selection factors discussed earlier. Finally, we examined the practical utility of these results by calculating RRRs for MMPI–2–RF scales that were correlated with the outcome variables. RRRs indicate increase in risk for a poor outcome when scale elevations occur. We examined the risk associated with the 65T cutoff used traditionally to identify clinically significant MMPI scale elevations, as well as cutoffs of 60T, 55T, 50T, and 45T. We also examined low score cutoffs of 38T and 33T for inverse associations. Past research has indicated that adjusted cutoffs are appropriate in this setting due to selection and underreporting issues discussed earlier and by Sellbom et al. (2007). METHOD Participants The sample included 146 male police officers from the Honolulu Police Department who were administered the MMPI–2 (33.6%) or MMPI–2–RF (66.4%). The MMPI–2–RF scales were scored from both versions of the instrument, which is possible because the MMPI–2 booklet contains all of the MMPI–2–RF items. Past research has documented the equivalence of scale scores when using this approach (Tellegen & Ben-Porath, 2008/2011; Van der Heijden, Egger, & Derksen, 2010). Only one officer evidenced protocol invalidity, as defined by the test authors’ published cutoffs (Cannot Say [CNS] > 18, Variable Response Inconsistency Revised [VRIN-r] > 79, True Response Inconsistency-Revised [TRINr] > 79, Infrequent Responses-Revised [F-r] > 119, or Infrequent Psychopathology Responses-Revised [Fp-r] > 99; Ben-Porath & Tellegen, 2008/2011). The final sample of 145 male police officers ranged in age from 19 to 51 years (M D 28.2, SD D 6.0). Their years of education ranged from 12 to 26 (M D 15.0, SD D 2.9). Race information was available only for the 97 officers administered the MMPI–2–RF, with findings indicating that the subsample was 46.3% Asian, 40.0% Hawaiian or Pacific Islander, 35.0% White, 15.0% Hispanic,

MMPI–2–RF POLICE SCREENING

Downloaded by [134.117.10.200] at 21:00 08 May 2015

and 2.5% African American (i.e., groups are not mutually exclusive). Given that all MMPIs were administered between 2003 and 2008 and the hiring practices of the agency had not been significantly modified during this time, these ethnic breakdowns are likely to similarly characterize the remaining 49 officers administered the MMPI–2. Overall, 85.6% of the sample successfully completed the probationary period, 8.9% left voluntarily, and 5.5% left involuntarily.

Measures MMPI–2–RF. The MMPI–2–RF (Ben-Porath & Tellegen, 2008/2011) is a 338-item measure of personality psychopathology with 51 scales. Nine of these scales are intended to measure the validity of a test taker’s responses in terms of non-content-based invalid responding (e.g., nonresponding, random responding, and acquiescent responding) as well as content-based invalid responding (e.g., overreporting and underreporting). The remaining 42 scales are organized in a hierarchical manner in five substantive domains: (a) Emotional Dysfunction, (b) Thought Dysfunction, (c) Behavioral Dysfunction, (d) Interpersonal Functioning, and (e) Somatic/ Cognitive Problems. The scales measure a broad range of psychological constructs related to personality and psychopathology. Information about reliability, validity, and descriptive findings in a broad range of samples is reported in the MMPI– 2–RF Technical Manual (Tellegen & Ben-Porath, 2008/2011). Selection Validation Survey. The Selection Validation Survey (SVS) is a record review form developed by the second author to measure police officer performance during the initial 18-month probationary period, which includes training academy and a subsequent 16-week field training and evaluation program (FTEP). A copy of the SVS form is available on request. The SVS coding form has three primary sections: Problem Behaviors, Field Performance Dimensions, and General Performance Dimensions. Problem Behavior scores were coded as 0 (no problem), 1 (problem during probation), and 2 (problem leading to separation). Field Performance Dimension scores are subdivided into ratings under normal and stress conditions. A score of 0 on either indicates no problems and a score of 1 indicates some problems. Finally, the General Performance Dimension scores are coded as 0 (no significant problems) or 1 (some problems). The Field Performance and General Performance Dimensions were coded by the third author’s assistant, who was trained by a board-certified police and public safety psychologist using a structured training methodology employed by this psychologist with other research projects using the SVS form (cf. Tarescavage, Corey, et al., 2014). Personnel records and daily observation reports (DORs) collected during the FTEP were used to code the SVS forms. The personnel records included citizen and internal affairs complaints. The DOR data were 4-point daily ratings (1 D poor, 2 D needs improvement, 3 D made improvements, and 4 D meets standard) made by FTEP coaches on all of the Field Performance and General Performance Dimensions represented on the SVS. Problems are noted when these archived records indicated two or more incidents of the same counterproductive behavior (i.e., daily rating of 2 or lower) or when the behavior was of sufficient severity to warrant disciplinary action or termination. The Problem Behaviors were coded by the

3 sergeant assigned to oversee the FTEP at the collection site. This individual was also trained by the police and public safety psychologist previously described. The Problem Behaviors variables were coded using the personnel records and DOR data, as well as the sergeant’s knowledge of the recruits’ performance during their probationary period. Owing to practical limitations, it was not possible to obtain repeated reviews of the same records for the purpose of estimating interrater reliability. To facilitate interpretation of the SVS, variables were a priori mapped onto the California POST Commission psychological screening dimensions by the second author (a boardcertified police and public safety psychologist using these standards). According to POST regulation 1955(d), all police officer candidates “shall be evaluated, at a minimum, against job-related psychological constructs herein incorporated by reference contained and defined in the POST Peace Officer Psychological Screening Dimensions.” In accordance with the just-cited regulation, a preemployment evaluation must assess the following job-related psychological domains: social competence, teamwork, adaptability and flexibility, conscientiousness and dependability, impulse control, integrity and ethics, emotional regulation and stress tolerance, decision making and judgment, assertiveness and persuasiveness, and avoiding substance use and other risk-taking bBehavior (California POST Commission, 2005). The validity evidence underlying the California POST psychological screening dimensions is reported extensively elsewhere (Spilberg & Corey, 2014). The SVS variables mapped well onto the California POST domains but covered slightly different content than the POST dimensions. Three dimensions were not explicitly represented on the SVS or had inadequate frequency of reported problems (as described in the Data Analyses subsection): adaptability and flexibility (the ability to change gears and easily adjust to the many different, sudden, and sometimes competing demands inherent in law enforcement work), impulse control (taking proper precautions and avoiding impulsive or unnecessarily risky behavior to ensure both public and personal safety), and avoiding substance use and other risk-taking behaviors (avoiding participation in behavior that is inappropriate, self-damaging, or can adversely impact organizational functioning; e.g., alcohol and drug abuse, domestic violence, sale of drugs, and problematic gambling). Two SVS variables—routine task performance problems (the ability to apply job-relevant knowledge, skills, and abilities in the context of routine demands) and feedback acceptance problems (the ability to consider one’s own role in or contribution to problems and, in the face of constructive criticism, to integrate feedback into a plan for self-improvement)—were specifically measured during field training but were not clearly represented by the POST dimensions.

Procedure Each officer was administered the MMPI–2 or MMPI–2–RF as part of the mandated preemployment psychological evaluation. Additional components of the postconditional-offer evaluation included the CPI (Gough, 1956), the PAI (Morey, 2007), the State–Trait Anger Expression Inventory (Spielberger, 1988), a structured life history questionnaire, a comprehensive clinical interview, and a review of the hiring agency’s background investigation and polygraph

4

Downloaded by [134.117.10.200] at 21:00 08 May 2015

examination. Determinations of suitability (i.e., qualified vs. unqualified) were made by the hiring agency’s staff or contract psychologist on the basis of the aggregate evidence judged against the previously described California POST dimensions.

Data Analyses We calculated T score means and standard deviations for all MMPI–2–RF substantive scales. As previously mentioned, police officer candidates typically present in a more defensive manner than the general population, so we also calculated means and standard deviations for the test’s underreporting validity scales, L-r and K-r. We compared the descriptive findings for this sample with those of the normative sample (who by definition produced mean T scores of 50 and standard deviations of 10 on all scales) and to the male law enforcement officer comparison group reported in the MMPI–2–RF Technical Manual (Tellegen & Ben-Porath, 2008/2011). Consistent with traditional benchmarks for MMPI–2 and MMPI–2–RF interpretation, we used a difference of 5 T score points to demarcate a practically meaningful finding (Graham, 2012). We calculated Pearson product–moment correlations (point-biserial for dichotomous criteria) between the MMPI– 2–RF substantive scales and SVS criteria to investigate the predictive validity of the instrument because we sought to examine linear associations. Low-frequency criteria in which dysfunctional behavior occurred in less than 2.5% of cases (i.e., n D 3 or less) were excluded to mitigate the influence of outliers on the correlational analyses. As a result, we excluded 18 of the 44 SVS variables that were mapped onto the California POST psychological screening dimensions. These variables included use of excessive force (n D 0), restraint and control problems under normal conditions (n D 1), tactical skills under normal conditions (n D 3), failure to exercise appropriate discretion (n D 1), does not take responsibility for mistakes (n D 0), uncooperative toward peers (n D 0), uncooperative toward supervisors (n D 0), shows bias toward others (n D 0), rude behavior (n D 0), abuses authority (n D 0), deceptiveness (n D 3), nonsexual misconduct (n D 1), sexual misconduct (n D 0), uses position for personal advantage (n D 0), conduct unbecoming (n D 1), unlawful activity (n D 0), excessive tardiness or absenteeism (n D 1), and misses court appearances (n D 0). Although practically meaningful MMPI correlations typically occur at r > j.20j (Graham, Ben-Porath, & McNulty, 1999; Sellbom, Ben-Porath, et al., 2006), following Sellbom et al. (2007), we defined practically meaningful zero-order correlations as statistically significant r values > j.15j in this study because our sample yielded range-restricted scores on the MMPI–2–RF substantive scales, thus attenuating the correlations (cf. Lowmaster & Morey, 2012; Sellbom et al., 2007). Furthermore, in Sellbom et al. (2007), correlations of this magnitude yielded practically meaningful findings. For example, higher scores on RC9 were associated with substantially increased risk for bad attitudes toward public (RRR D 4.20, p < .05), an association yielding a zero-order correlation of .15. Even though we calculated zero-order correlations between all MMPI–2–RF substantive scales and SVS criteria, we did not adjust alpha levels to correct for potential familywise error rates, because, as discussed earlier, this study was conducted in the context of discovery (Reichenbach, 1938). Ellis (2010)

TARESCAVAGE, COREY, GUPTON, BEN-PORATH described alpha adjustment techniques as “an alarming trend” that reduces statistical power and complicates literature synthesis” (p. 79). In this study, field-research data were used, yielding a sample size that was limited to 145 officers. With this sample size, the likelihood of detecting true correlational magnitudes of .15 using an alpha of .05 was .44 (Faul, Erdfelder, Lang, & Buchner, 2007). Adjusting the alpha to .01 would yield a likelihood of .22, and further lowering the alpha to .001 would yield a likelihood of .07. Such alpha-adjustment procedures would have substantially limited the practically meaningful information possibly gained from this study. The importance of replicating our findings, however, is emphasized in the Discussion section of this article. For the same reason, we also calculated range restrictioncorrected correlations using formulas derived from Hunter and Schmidt (2004). Three pieces of information were required to apply the formula: (a) the zero-order correlation between the MMPI–2–RF scale score and SVS criterion, (b) the standard deviation of the MMPI–2–RF scale score in this sample, and (c) the standard deviation of the MMPI–2–RF scale score in the general population (i.e., the unrestricted standard deviation). For example, as seen in Table 1, the zeroorder correlation between MMPI–2–RF Emotional/Internalizing Dysfunction (EID) and SVS Tactical Skills Problems under Stress Conditions is .14. The standard deviation for EID was 5.7 in this sample. The general population standard deviation is 10. Applying the range-restriction correction using these values yields a correlation of .24, which is the unbiased (in terms of range restriction) validity estimate for EID scores as predictors of Tactical Skills Problems in this sample. Other studies have used normative information in this manner to estimate the unrestricted standard deviation (Hoffman, 1995; Ones & Viswesvaran, 2003; Sackett & Ostgaard, 1994). Finally, to investigate the practical utility of MMPI–2–RF scale scores, we calculated RRRs with SVS criteria. RRR values are calculated by dividing the risk of a negative outcome for individuals who score at or above the cutoff by the risk of a negative outcome for individuals who score below the cutoff. We calculated 95% confidence intervals for the RRRs, which indicate nonsignificant findings if the range overlaps with one (meaning one cannot reject the null hypothesis that there is an equal risk of negative outcome in both groups). We only calculated RRRs for scales that were meaningfully correlated with SVS criteria (i.e., statistically significant zero-order correlations of a magnitude r > j.15j). We chose to base the criteria for calculating RRRs on the zero-order correlations because, unlike correlates, RRRs cannot be disattenuated for range restriction. We investigated cutoffs of > 65T, 60T, 55T, 50T, and 45T for scales demonstrating positive associations with criteria (such that high scores on the MMPI–2–RF were associated with dysfunctional outcomes). For negative associations, cutoffs of < 38T and < 33T were used. Similar to the correlational analyses, in which only criteria with base rates (BRs) of 2.5% or higher were used, we limited the RRR analyses to cutoffs yielding selection ratios (SRs) of at least 2.5% to mitigate the influence of outliers. We further limited the RRR analyses to cutoffs producing less than a 20% SR in an effort to decrease the possibility of false positive findings, as the BRs of problem behaviors in this sample ranged from 2.8% to 22.8%. These criteria yielded 86 RRR analyses, but as reported later only the 26 statistically significant findings are

.11 (.21) .04 (.08) .02 (.04) .13 (.24)

.02 (.05) .02 (.04)

¡.06 (¡.08) ¡.03 (¡.05) ¡.06 (¡.09) ¡.04 (¡.05) ¡.03 (¡.04)

.06 (.08) .04 (.06) .16 (.22) .06 (.09)

.00 (.00) ¡.08 (¡.11) ¡.10 (¡.14)

.13 (.24)

.08 (.12)

.06 (.09) .02 (.03) .09 (.15)

.03 (.05)

.14 (.22)

.10 (.19) .10 (.19) .04 (.07)

¡.04 (¡.05) ¡.06 (¡.08) ¡.02 (¡.03)

.15 (.21) .04 (.05) .05 (.07)

.21 (.38) .03 (.05)

*

.16 (.30)

¡.08 (¡.11) ¡.11 (¡.15)

¡.09 (¡.12)

¡.05 (¡.08) ¡.12 (¡.17)

.00 (.00)

.10 (.16)

.12 (.18)

.09 (.14)

*

.05 (.08) .04 (.06) .01 (.02)

.04 (.06) .06 (.10) .04 (.06)

¡.14 (¡.19)

¡.09 (¡.10)

¡.03 (¡.03) ¡.12 (¡.12) ¡.07 (¡.07)

.00 (.01) ¡.05 (¡.06) ¡.01 (¡.01)

¡.10 (¡.13)

¡.02 (¡.03)

¡.16 (¡.17)

.13 (.18)

42.8 5.9 43.1 6.2

MLS

.01 (.01)

.13 (.18)

44.2 5.3 43.4 4.6

HPC

¡.04 (¡.08)

¡.05 (¡.12) ¡.09 (¡.18)

.00 (.00)

¡.01 (¡.02) ¡.02 (¡.05) ¡.01 (¡.04) ¡.03 (¡.06)

46.4 3.9 46.3 3.9

GIC

.02 (.03) ¡.01 (¡.02) .00 (.01) .02 (.03)

.01 (.02) .19* (.31) .04 (.06) .18* (.30)

¡.05 (¡.14)

¡.05 (¡.12) .08 (.20) .06 (.15) .13 (.31)

.00 (.00)

.07 (.14) .05 (.10) .11 (.20) .00 (.00)

.18* (.24)

.07 (.10)

.05 (.09)

¡.01 (¡.02) ¡.06 (¡.08) ¡.05 (¡.07) .13 (.21)

¡.01 (¡.02)

¡.01 (¡.01)

.01 (.01)

¡.08 (¡.12) ¡.01 (¡.02) ¡.10 (¡.15) .12 (.20)

.09 (.12) .04 (.06) .06 (.09)

¡.05 (¡.08) ¡.01 (¡.01) ¡.10 (¡.15)

.15 (.20) .07 (.10) .10 (.14)

¡.01 (¡.01) .01 (.01) ¡.05 (¡.07) .06 (.10) ¡.01 (¡.01) .10 (.16)

¡.10 (¡.15) ¡.14 (¡.22) ¡.06 (¡.08) ¡.13 (¡.18) .14 (.23)

¡.01 (¡.01) ¡.07 (¡.11) ¡.09 (¡.12) ¡.10 (¡.14) .06 (.10)

.11 (.16)

**

¡.02 (¡.02) .19* (.31)

.00 (.00)

.10 (.24)

.00 (.00)

¡.04 (¡.10) ¡.08 (¡.15) .05 (.14) ¡.11 (¡.20) .07 (.17) .03 (.05)

¡.04 (¡.10) ¡.08 (¡.14)

.11 (.28)

¡.08 (¡.12) ¡.09 (¡.12) ¡.13 (¡.18) .25** (.40) ¡.06 (¡.16) ¡.13 (¡.24)

¡.07 (¡.11) ¡.04 (¡.06) ¡.02 (¡.03) .11 (.19)

.05 (.07) .08 (.11) .17* (.23) .04 (.06)

¡.09 (¡.14) ¡.11 (¡.15) ¡.14 (¡.20) .11 (.19) ¡.03 (¡.08) ¡.18* (¡.33) ¡.10 (¡.16) .00 (.00) ¡.04 (¡.05) .23** (.37) ¡.04 (¡.10) .00 (.00)

*

.05 (.09)

.02 (.03) .11 (.19) ¡.03 (¡.04) .03 (.06)

45.8 6.9 43.2 7.5

RC9

¡.01 (¡.01) ¡.10 (¡.15) .04 (.05) ¡.11 (¡.16) .01 (.02) ¡.01 (¡.01) ¡.09 (¡.15) .00 (.00) ¡.05 (¡.07) .13 (.22) .05 (.07) ¡.13 (¡.21) ¡.07 (¡.09) .02 (.02) .16 (.26)

.03 (.04)

.21* (.30)

.00 (.00)

.02 (.02)

.09 (.13) .01 (.01)

46.6 7.3 43.6 6.7

RC8

¡.05 (¡.08) ¡.09 (¡.12) ¡.07 (¡.10) .18 (.30)

¡.08 (¡.12)

¡.05 (¡.07) ¡.02 (¡.03)

39.4 6.2 37.9 5.5

RC7

.14 (.20) .06 (.09) .02 (.03) ¡.01 (¡.01) * .17 (.24) ¡.01 (¡.01) .25** (.35) .00 (.00)

.06 (.08) .20* (.28)

¡.01 (.01)

.12 (.17)

.09 (.13) .08 (.12)

46.5 6.9 45.6 6.0

RC6

¡.05 (¡.09)

¡.02 (¡.05) ¡.02 (¡.03) ¡.07 (¡.18) ¡.10 (¡.19) .04 (.09) .02 (.04)

¡.05 (¡.12) ¡.09 (¡.18)

¡.05 (¡.12) ¡.09 (¡.18)

.02 (.05)

46.0 7.0 45.2 6.7

NUC

¡.01 (¡.03) ¡.03 (¡.05)

43.5 5.3 42.8 5.6

COG

¡.07 (¡.13) .02 (.04)

.03 (.06)

¡.01 (¡.01)

¡.06 (¡.12) ¡.05 (¡.10) .11 (.21)

¡.04 (¡.07)

¡.01 (¡.01)

¡.02 (¡.05)

¡.09 (¡.17)

.03 (.04) .05 (.08) .04 (.05)

.08 (.16) .06 (.11) .11 (.20)

¡.04 (¡.06) ¡.12 (¡.23)

.00 (.00)

.08 (.11)

¡.02 (¡.03) ¡.04 (¡.08) ¡.04 (¡.06) ¡.02 (¡.05)

.01 (.01)

.12 (.18) .02 (.03) .08 (.11)

.00 (.00)

.03 (.04)

¡.05 (¡.07) ¡.02 (¡.04)

.09 (.12)

¡.04 (¡.05) .02 (.04) .08 (.11) .02 (.04) ¡.03 (¡.04) ¡.03 (¡.06) .10 (.14) ¡.01 (¡.03)

.03 (.04) .06 (.08)

¡.04 (¡.06) ¡.07 (¡.14)

¡.01 (¡.01) ¡.05 (¡.09)

.02 (.03) .06 (.08)

Somatic/Cognitive

¡.09 (¡.12) ¡.02 (¡.04) ¡.16 (¡.25) ¡.08 (¡.12) ¡.03 (¡.04) .22 (.36) ¡.05 (¡.13) ¡.10 (¡.19) ¡.21* (¡.28) .09 (.12) ¡.10 (¡.15) ¡.03 (¡.05) ¡.13 (¡.18) .26** (.41) ¡.05 (¡.12) ¡.09 (¡.18)

.01 (.02)

¡.08 (¡.08)

¡.05 (¡.05) ¡.14 (¡.15)

.04 (.05)

¡.07 (¡.10) ¡.09 (¡.12) ¡.04 (¡.06)

.03 (.04)

.24 (.36) ¡.09 (¡.09) .11 (.16) ¡.07 (¡.07) .10 (.16) ¡.06 (¡.06)

.14 (.15)

¡.05 (¡.07)

¡.01 (¡.01) .14 (.22)

.02 (.03)

.06 (.10)

¡.02 (¡.03)

¡.02 (¡.02) ¡.18* (¡.28) ¡.10 (¡.14)

¡.15 (¡.20) ¡.02 (¡.03) ¡.10 (¡.14) ¡.08 (¡.10)

¡.07 (¡.07) ¡.03 (¡.04) .03 (.04) .06 (.07)

¡.03 (¡.05) .00 (.00)

¡.02 (¡.03)

**

.15 (.23)

.01 (.01)

¡.12 (¡.19) .11 (.16)

.10 (.15)

.02 (.02) .18* (.27) .01 (.01) .11 (.17)

¡.01 (¡.01) .15 (.23) .00 (.00) .18* (.27)

¡.06 (¡.06) ¡.07 (¡.07)

¡.07 (¡.09)

¡.05 (¡.06)

¡.01 (¡.01) .18 (.27) *

¡.04 (¡.05)

.01 (.02) ¡.12 (¡.16)

48.3 7.4 46.1 7.0

RC4

¡.12 (¡.12)

¡.01 (¡.01) ¡.03 (¡.03)

45.8 9.7 44.0 8.4

RC3

¡.01 (¡.02) .05 (.08)

.06 (.10) .04 (.07)

40.5 6.4 41.2 7.0

RC2

¡.08 (¡.11) ¡.05 (¡.08) ¡.02 (¡.03) ¡.06 (¡.10) .18 (.27) .00 (.00) ¡.17* (¡.24) .00 (.01) ¡.08 (¡.13) .18* (.27)

.01 (.02)

.02 (.04) .04 (.07) .02 (.04)

.09 (.13) ¡.08 (¡.12) .00 (.00) ¡.08 (¡.11) ¡.04 (¡.06) ¡.02 (¡.02)

.03 (.04)

.22** (.40)

¡.07 (¡.09) ¡.04 (¡.06)

.13 (.24)

.02 (.04) .06 (.11)

¡.09 (¡.13) ¡.07 (¡.10) .03 (.04) ¡.01 (¡.02)

.07 (.10)

.11 (.18)

.16 (.30)

¡.08 (¡.11) ¡.07 (¡.10)

.18* (.25)

.06 (.09) .14 (.23) .11 (.18) .19* (.30)

.08 (.16)

¡.01 (¡.01)

.01 (.02)

.05 (.08) .03 (.05)

.13 (.25) .07 (.13)

¡.03 (¡.04) ¡.11 (¡.16)

.09 (.12) .03 (.04)

42.3 6.2 41.2 6.0

RC1

40.6 5.2 39.8 4.9

RCd

50.1 7.1 47.8 6.7

BXD

45.8 7.2 43.6 6.5

THD

Restructured Clinical

Note. n D 145 male officers. EID D Emotional/Internalizing Dysfunction; THD D Thought Dysfunction; BXD D Behavioral/Externalizing Dysfunction; RCd D Demoralization; RC1 D Somatic Complaints; RC2 D Low Positive Emotions; RC3 D Cynicism; RC4 D Antisocial Behavior; RC6 D Ideas of Persecution; RC7 D Dysfunctional Negative Emotions; RC8 D Aberrant Experiences; RC9 D Hypomanic Activation; MLS D Malaise; GIC D Gastrointestinal Complaints; HPC D Head Pain Complaints; NUC D Neurological Complaints; COG D Cognitive Complaints. * p < .05. **p < .01. Statistically significant correlations reaching an effect size of  j.15j are shown in bold.

Sample M 36.5 Sample SD 5.7 Comparison group M 36.3 Comparison group SD 6.0 Emotional Control and Stress Tolerance Problems Domain Tactical Skills: Stress Conditions .14 (.24) Decision Making Problems: Stress .10 (.18) Conditions Learning Problems: Stress .09 (.16) Conditions Restraint & Control Problems: Stress .14 (.25) Conditions Routine Task Performance Problems Domain Report Writing Problems .05 (.09) Learning Problems: Normal .10 (.17) Conditions Academic/Learning Problems .11 (.19) Driving Problems .12 (.20) Navigational Problems .05 (.09) Radio Problems .16 (.27) Decision Making and Judgment Problems Domain Decision Making Problems: Normal .04 (.07) Conditions Multitasking Problems: Normal .05 (.08) Conditions Multitasking Problems: Stress .10 (.17) Conditions Feedback Acceptance Problems Domain Failure to Accept Feedback .20* (.34) Assertiveness Problems Domain Failure to Control Conflict .09 (.16) Failure to Engage Subjects .08 (.15) Assertiveness/Control Problems: .02 (.04) Normal Conditions Social Competence and Teamwork Problems Domain Assertiveness/Control Problems: .17* (.29) Stress Conditions Interpersonal Problems ¡.03 (¡.05) Interpersonal Problems: Normal .10 (.18) Conditions Interpersonal Problems: Stress .16 (.28) Conditions Teamwork .17* (.29) Integrity Problems Domain Integrity .02 (.04) Conscientious and Commitment Problems Domain Initiative & Drive .11 (.19) Commitment .14 (.24) Conscientiousness .04 (.06)

EID

Higher-Order

TABLE 1.—Minnesota Multiphasic Personality Inventory–2–Restructured Form Higher-Order, Restructured Clinical, and Somatic-Cognitive Specific Problems scales correlations (disattenuated) with Selection Validation Survey criteria.

Downloaded by [134.117.10.200] at 21:00 08 May 2015

6

Downloaded by [134.117.10.200] at 21:00 08 May 2015

presented to save space. To further clarify the predictive utility of MMPI–2–RF cutoffs, we also calculated classification accuracies in terms of sensitivity, specificity, positive predictive power, and negative predictive power. As already mentioned, SVS Problem Behavior scores were coded as 0 (no problem), 1 (problem during probation), and 2 (problem leading to separation). Because RRR analyses require dichotomous data, we collapsed problem during probation and problem leading to separation into one problem category. Owing to the low rate of involuntary departure (5.5%), recoding these variables as 0 versus 2 would yield a set of variables in which all BRs would be deemed unacceptably low ( 1.00) included EID (Cohen’s d D 1.72), Dysfunctional Negative Emotions (Cohen’s d D 1.31), Negative Emotionality/Neuroticism-Revised (Cohen’s d D 1.29), Demoralization (Cohen’s d D 1.24), Stress/Worry (Cohen’s d D 1.16), Low Positive Emotions (Cohen’s d D 1.16), and Anger Proneness (Cohen’s d D 1.11). The following scales demonstrated large effect sizes, as described by Cohen (1992; d D .80–.99): Somatic Complaints, Malaise, Cognitive Complaints, Helplessness, Self-Doubt, Inefficacy, Aggression, Family Problems, and Shyness. MMPI–2–RF Scales Head Pain Complaints, Anxiety, Multiple Specific Fears, and Behavior Restricting Fears demonstrated medium effect sizes (d D .50–.79). The sample in this study evidenced significant range restriction in comparison to the normative sample, as the median standard deviation was 6.6—approximately two thirds of the normative sample standard deviation of 10. Notably, this sample had meaningfully higher underreporting validity scale scores compared to the normative sample (L-r: M D 55.4, SD D 12.1, Cohen’s d D .49; K-r: M D 61.0, SD D 8.1, Cohen’s d D 1.21), which likely contributed to these findings. These findings are slightly lower than the male law enforcement officer comparison group (L-r: M D 58.6, SD D 12.6, Cohen’s d D –.26; K-r: M D 62.9, SD D 8.0, Cohen’s d D ¡.24). However, no clinically meaningful differences (as defined by a difference of > 5T) were observed between the validity and substantive scale scores in this sample versus the male law enforcement officer comparison group. Correlations Zero-order and disattenuated correlations between MMPI– 2–RF substantive scales and SVS criteria are presented in

TARESCAVAGE, COREY, GUPTON, BEN-PORATH Table 1 for the Higher-Order, Restructured Clinical, and Somatic/Cognitive Specific Problems scales, in Table 2 for the Internalizing and Externalizing Specific Problems scales, and in Table 3 for the Interpersonal Specific Problems scales and Pesonality-Psychopathology–5 scales. Please see table notes for scale names and abbreviations. To facilitate interpretation, the findings are summarized in reference to each of the five MMPI–2–RF domains, which include (a) Emotional Dysfunction, (b) Thought Dysfunction, (c) Behavioral Dysfunction, (d) Somatic/Cognitive Complaints, and (e) Interpersonal Functioning. In the Emotional Dysfunction domain, the RC scale Low Positive Emotions (RC2) had the most robust findings, including meaningful correlations with criteria from the Emotional Control and Stress Tolerance, Routine Task Performance, Decision Making and Judgment, Assertiveness, and Social Competence and Teamwork dimensions. Other findings from the Emotional Dysfunction domain included associations with Academic/Learning Problems (ANP), Failure to Accept Feedback (EID, RCd, NFC, & INTR-r), Assertiveness/Control Problems under Stress Conditions (EID & INTR-r), and poor Teamwork (EID, RCd, & INTR-r). Because SUI had no variance, results for this scale are not included in the tables or text. In the Thought Dysfunction domain, RC scale Persecutory Ideation (RC6) demonstrated a number of associations with Routine Task Performance Problems, including Learning Problems under Normal Conditions, Navigational Problems, and Radio Problems. All the scales in this domain were associated with Multitasking Problems under Stress Conditions. Finally, PSYC-r was associated with poor Initiative/Drive and RC8 was correlated with Navigational Problems. Findings in the Behavioral Dysfunction domain were notable for their negative associations with problem behaviors (i.e., such that lower scores predicted poor outcomes). The Specific Problems scale Activation (ACT) had the most robust findings in this domain, as it demonstrated negative associations with many criteria, including variables from the Emotional/Control and Stress Tolerance, Routine Task Performance, Feedback Acceptance, Assertiveness, Social Competence and Teamwork, and Integrity problem domains. Many of the other scales in the Behavioral Dysfunction domain (BXD, RC4, JCP, & DISC-r) were negatively associated with Interpersonal Problems under Normal Conditions. Other findings included negative associations with Decision Making Problems under Stress Conditions (JCP), Failure to Control Conflict (AGGR-r), and Integrity Problems (AGG). The findings for the Somatic/Cognitive scales were primarily limited to associations with Malaise (MLS), which demonstrated positive associations with the following problem domains: Emotional Control and Stress Tolerance, Routine Task Performance, Decision Making and Judgment, and Social Competence and Teamwork. Other Somatic/Cognitive findings included an association with Radio Problems (RC1) and a negative association with Report Writing Problems (NUC). Findings in the Interpersonal Functioning domain were particularly strong for Family Problems (FML) and Social Avoidance (SAV), both of which were positively associated with variables from the Emotional Control/Stress Tolerance Problems and Assertiveness/Control Problems domains. FML was positively correlated with poor Initiative/Drive and

¡.09 (¡.20) .07 (.15) .05 (.11) .03 (.07)

¡.03 (¡.06) ¡.04 (¡.09) ¡.06 (¡.13) .05 (.11) .02 (.03) .05 (.07) .07 (.10)

¡.01 (¡.01)

.04 (.06) ¡.01 (¡.01) .02 (.03) .10 (.15) .12 (.18)

.13 (.28) ¡.03 (¡.07) ¡.02 (¡.04) .12 (.26) .04 (.09)

.07 (.15) ¡.05 (¡.11) .05 (.11) ¡.04 (¡.09) ¡.03 (¡.06)

.10 (.17) .04 (.07) .12 (.21)

¡.04 (¡.07)

.10 (.17) ¡.10 (¡.17) ¡.01 (¡.02) .03 (.05) ¡.04 (¡.07)

¡.02 (¡.06) ¡.06 (¡.17) .03 (.09)

¡.04 (¡.11)

.00 (.00) ¡.04 (¡.11) ¡.04 (¡.11) ¡.07 (¡.20) ¡.04 (¡.11)

¡.04 (¡.11) ¡.07 (¡.20) .06 (.17)

.04 (.06) ¡.01 (¡.01) ¡.06 (¡.09)

¡.02 (¡.04) .05 (.11) .04 (.09)

.12 (.25) .03 (.06) .11 (.23)

¡.01 (¡.02) .07 (.12) .09 (.16)

¡.03 (¡.09)

¡.06 (¡.11)

.19* (.28)

.08 (.18)

.01 (.02)

¡.05 (¡.14) ¡.06 (¡.17) ¡.04 (¡.11)

¡.09 (¡.25) ¡.03 (¡.09) .09 (.25) .00 (.00) ¡.02 (¡.06) ¡.09 (¡.25)

¡.09 (¡.25) ¡.09 (¡.25) ¡.08 (¡.22) ¡.04 (¡.11)

¡.03 (¡.05) .04 (.07) .16 (.28)

.07 (.15) ¡.07 (¡.15) .11 (.24)

.01 (.02) .03 (.06) ¡.01 (¡.02)

.14 (.24) ¡.01 (¡.02) ¡.04 (¡.07) ¡.04 (¡.07)

44.7 3.5 44.6 3.1

AXY

Internalizing

¡.02 (¡.03) ¡.06 (¡.09) .08 (.12)

¡.02 (¡.04) .08 (.18) .04 (.09) ¡.03 (¡.07) .09 (.20) .08 (.18)

¡.06 (¡.13) .11 (.23) ¡.10 (¡.21) .00 (.00) ¡.13 (¡.27) ¡.09 (¡.19)

.07 (.10) .09 (.13) .01 (.01) .12 (.18)

40.9 5.7 41.4 6.3

STW

.04 (.07) .07 (.12) .06 (.11) .13 (.23) .07 (.12) .15 (.26)

.06 (.13) .12 (.26) .12 (.26) .04 (.09)

¡.06 (¡.13) ¡.06 (¡.13) .01 (.02) ¡.03 (¡.06)

42.9 6.7 40.9 5.9

NFC

.03 (.04) .03 (.04) .15 (.22) .08 (.12) .00 (.00) .11 (.16)

43.8 4.5 43.1 4.1

43.0 4.7 41.7 3.9

SFD

.08 (.15) .14 (.26) .09 (.17)

¡.07 (¡.13)

.02 (.04) ¡.08 (¡.15) ¡.11 (¡.21) .03 (.06) ¡.11 (¡.21)

¡.09 (¡.17) .06 (.12) ¡.11 (¡.21)

¡.04 (¡.08)

¡.09 (¡.17) ¡.12 (¡.23) .06 (.12)

¡.04 (¡.08) ¡.09 (¡.17) .22** (.40) .10 (.19) .13 (.24) .08 (.15)

.14 (.26) .15 (.28) ¡.01 (¡.02) ¡.07 (¡.13)

41.6 5.2 40.9 5.4

ANP

¡.08 (¡.12) ¡.02 (¡.03) .01 (.02)

.10 (.15) ¡.03 (¡.05) .10 (.15)

.00 (.00)

.07 (.11) .01 (.02) .06 (.09) .08 (.12) ¡.02 (¡.03)

¡.02 (¡.03) ¡.02 (¡.03) ¡.06 (¡.09) ¡.04 (¡.06) ¡.06 (¡.09) ¡.06 (¡.09)

.07 (.11) .05 (.08) .06 (.09)

.09 (.14)

.07 (.11) .01 (.02) .02 (.03)

.07 (.11) .02 (.03) .14 (.21) ¡.01 (¡.02) .02 (.03) .10 (.15)

.07 (.11) .10 (.15) .03 (.05) .06 (.09)

44.5 6.6 42.7 6.3

MSF

.04 (.06) ¡.05 (¡.08) .01 (.02)

¡.05 (¡.08)

.02 (.03) ¡.04 (¡.06) .03 (.05)

¡.10 (¡.16) .02 (.03) ¡.06 (¡.09) ¡.03 (¡.05) .00 (.00) ¡.03 (¡.05)

¡.01 (¡.02) ¡.04 (¡.06) ¡.09 (¡.14) ¡.06 (¡.09)

44.8 6.4 43.8 3.8

BRF

¡.10 (¡.11) ¡.02 (¡.02) ¡.07 (¡.08)

¡.10 (¡.11)

¡.04 (¡.05) ¡.13 (¡.15) ¡.21* (¡.24) ¡.12 (¡.14) ¡.12 (¡.14)

¡.11 (¡.12) ¡.15 (¡.17) ¡.11 (¡.12)

¡.05 (¡.06)

¡.13 (¡.15) ¡.11 (¡.12) ¡.02 (¡.02)

¡.09 (¡.10) ¡.09 (¡.10) ¡.14 (¡.16) ¡.04 (¡.05) ¡.11 (¡.12) ¡.12 (¡.14)

¡.06 (¡.07) ¡.20* (¡.22) ¡.12 (¡.14) ¡.10 (¡.11)

51.8 8.8 49.6 8.6

JCP

.06 (.09) .00 (.00) .01 (.01)

¡.12 (¡.17)

.02 (.03) ¡.13 (¡.19) ¡.12 (¡.17) .03 (.04) ¡.04 (¡.06)

¡.10 (¡.15) .01 (.01) .06 (.09)

¡.08 (¡.12)

.01 (.01) .04 (.06) ¡.01 (¡.01)

.03 (.04) .01 (.01) ¡.13 (¡.19) ¡.06 (¡.09) ¡.15 (¡.22) ¡.08 (¡.12)

¡.11 (¡.18) ¡.06 (¡.10) ¡.11 (¡.18)

¡.18* (¡.29)

¡.09 (¡.15) ¡.16 (¡.26) ¡.14 (¡.23) ¡.11 (¡.18) ¡.07 (¡.11)

¡.12 (¡.19) ¡.12 (¡.19) ¡.12 (¡.19)

¡.10 (¡.16)

¡.11 (¡.18) ¡.11 (¡.18) ¡.03 (¡.05)

¡.12 (¡.19) ¡.10 (¡.16) ¡.02 (¡.03) ¡.07 (¡.11) ¡.09 (¡.15) ¡.08 (¡.13)

¡.13 (¡.21) ¡.11 (¡.18) ¡.06 (¡.10) ¡.14 (¡.23)

43.1 6.1 42.4 6.7

AGG

Externalizing

¡.02 (¡.03) .01 (.01) ¡.01 (¡.01) ¡.08 (¡.12)

46.2 6.8 45.1 5.8

SUB

ACT

¡.13 (¡.15) ¡.14 (¡.16) ¡.07 (¡.08)

¡.17* (¡.19)

¡.19* (¡.22) ¡.13 (¡.15) ¡.13 (¡.15) ¡.19* (¡.22) ¡.21* (¡.24)

¡.09 (¡.10) ¡.18* (¡.21) ¡.07 (¡.08)

¡.19* (¡.22)

¡.05 (¡.06) ¡.15 (¡.17) ¡.11 (¡.13)

¡.19* (¡.22) ¡.06 (¡.07) ¡.05 (¡.06) ¡.10 (¡.11) ¡.13 (¡.15) ¡.04 (¡.05)

¡.10 (¡.11) ¡.14 (¡.16) ¡.05 (¡.06) ¡.19* (¡.22)

45.3 8.7 43.2 8.1

Note. n D 145 male officers. HLP D Helplessness/Hopelessness; SFD D Self–Doubt; NFC D Inefficacy; STW D Stress/Worry; AXY D Anxiety; ANP D Anger Proneness; BRF D Behavior–Restricting Fears; MSF D Multiple Specific Fears; JCP D Juvenile Conduct Problems; SUB D Substance Abuse; AGG D Aggression; ACT D Activation. * p < .05. **p j.15j and yielded SRs ranging from 2.5% to 20%). Sensitivity, specificity, and predictive powers are also presented. To save space, we only provide the 26 statistically significant findings of the 86 analyses. With the aim of assisting the reader with interpretation, the following is a description of the RRR for EID and Failure to Accept Feedback (i.e., the first row of Table 4). The SR indicates that 10.3% of the sample scored at or above a cutoff of 45T. The BR indicates that 2.8% of the sample was determined by supervisors to have restraint and control problems under stress conditions. The risk of having these problems if EID is > 45T is 13.3%, and if EID is 80T) with paranoid delusional thinking. However, in this sample, which produced scores below the 65T clinical elevation cutoff, scores on this scale likely reflect suspiciousness of and alienation from others rather than severely disordered thinking. Our findings suggest that these characteristics could affect a police officer’s learning and processing of information, as well as acquisition of skills that require multitasking such as navigation and radio operation.

10

TARESCAVAGE, COREY, GUPTON, BEN-PORATH TABLE 5.—Summary table of interpretable correlations. Scale Higher-Order Emotional/Internalizing Dysfunction (EID) Thought Dysfunction (THD) Behavioral/Externalizing Dysfunction (BXD) Restructured Clinical Demoralization (RCd) Somatic Complaints (RC1) Low Positive Emotions (RC2)

Cynicism (RC3) Antisocial Behavior (RC4) Persecutory Ideation (RC6)

Downloaded by [134.117.10.200] at 21:00 08 May 2015

Aberrant Experiences (RC8) Specific Problems Malaise (MLS)

Head Pain Complaints (HPC) Inefficacy (NFC) Anger Proneness (ANP) Juvenile Conduct Problems (JCP) Aggression (AGG) Activation (ACT)

Family Problems (FML)

Social Avoidance (SAV)

Personality Psychopathology–5 Aggressiveness–Revised (AGGR-r) Psychoticism–Revised (PSYC-r) Disconstraint–Revised (DISC-r) Introversion/Low Positive Emotionality–Revised (INTR-r)

Correlates

Failure to Accept Feedback (.20/.34) Assertiveness Problems: Stress Conditions (.17/.29) Multitasking Problems: Stress Conditions (.18/.25) Interpersonal Problems: Normal Conditions (–.17/–.24) Failure to Accept Feedback (.22/.40) Teamwork (.21/.38) Radio Problems (.19/.30) Restraint & Control Problems: Stress Conditions (.18/.27) Learning Problems: Normal Conditions (.18/.27) Driving Problems (.18/.27) Failure to Control Conflict (.24/.36) Interpersonal Problems (.18/.27) Interpersonal Problems: Normal Conditions (.18/.27) Multitasking Problems: Normal Conditions (–.18/–.28) Interpersonal Problems: Normal Conditions (–.21/–.28) Learning Problems: Normal Conditions (.20/.28) Radio Problems (.25/.35) Multitasking Problems: Stress Conditions (.21/.30) Multitasking Problems: Stress Conditions (.18/.24) Restraint & Control Problems: Stress Conditions (.18/.30) Learning Problems: Normal Conditions (.23/.37) Driving Problems (.19/.31) Radio Problems (.18/.30) Multitasking Problems: Normal Conditions (.25/.40) Assertiveness/Control Problems: Stress Conditions (.19/.31) Interpersonal Problems (.22/.36) Interpersonal Problems: Normal Conditions (.26/.41) Report Writing Problems (–.18/–.33) Failure to Accept Feedback (.19/.28) Academic/Learning Problems (.22/.40) Decision Making Problems: Stress Conditions (–.20/–.22) Interpersonal Problems: Normal Conditions (–.21/–.24) Integrity (–.18/–.29) Restraint & Control Problems: Stress Conditions (–.19/–.22) Report Writing Problems (–.19/–.22) Failure to Accept Feedback (–.19/–.22) Failure to Engage Subjects (–.18/–.21) Assertiveness/Control Problems: Stress Conditions (–.19/–.22) Interpersonal Problems: Stress Conditions (–.19/–.22) Teamwork (–.21/–.24) Integrity (–.17/–.19) Tactical Skills: Stress Conditions (.21/.32) Learning Problems: Stress Conditions (.18/.28) Interpersonal Problems: Stress Conditions (.20/.31) Initiative & Drive (.19/.29) Conscientiousness (.22/.33) Restraint & Control Problems: Stress Conditions (.16/.22) Failure to Accept Feedback (.27/.36) Interpersonal Problems: Stress Conditions (.17/.23) Teamwork (.16/.22) Failure to Control Conflict (–.18/–.26) Multitasking Problems: Stress Conditions (.20/.27) Initiative & Drive (.18/.24) Interpersonal Problems: Normal Conditions (–.21/–.27) Failure to Accept Feedback (.24/.32) Assertiveness/Control Problems: Stress Conditions (.16/.22) Interpersonal Problems: Stress Conditions (.19/.25) Teamwork (.16/.22)

Note. All correlates are statistically significant (p < .05). Correlations to left in parentheses are zero–order; correlations to the right are disattenuated for range restriction.

Findings from the Behavioral/Externalizing domain were notable for negative associations with problem behaviors, especially for the Specific Problems Activation scale (ACT). Negative findings with poor performance in this domain are common in other studies as well (Tarescavage, Brewster, et al., 2014; Tarescavage, Corey, et al., 2014). This sample produced normal-range scores on ACT in which lower scores reflect reduced energy levels. Consequently, the negative correlations between ACT and problematic performance criteria might indicate that individuals with lower levels of energy and engagement with their environment are likely to have problems engaging subjects and with restraint and control under stress conditions, as well as problems with assertiveness, interpersonal interactions, and teamwork. Conversely, candidates with moderate levels of activation as measured by

the MMPI–2–RF ACT scale might be more likely to be assertive and engaged with their environment, coworkers, and citizens. An alternative explanation is the negative associations result from a third variable associated with underreporting or the illusion of mental health created by psychological defenses (Shedler, Mayman, & Manis, 1993). Findings in the Somatic/Cognitive domain were limited primarily to the MLS scale. MLS measures an overall sense of physical debilitation and poor health, and scores on this scale are associated with complaints of sleep disturbance, fatigue, and low energy (Tellegen & Ben-Porath, 2008/2011). In this sample, MLS scale scores were associated with a number of Routine Task Performance and Social Competence/Teamwork problems. Police candidates with higher scores on MLS might lack the energy to effectively interact with others and perform

Downloaded by [134.117.10.200] at 21:00 08 May 2015

MMPI–2–RF POLICE SCREENING routine aspects of their position. This interpretation is consistent with the absence of associations between these criteria and the other Somatic/Cognitive scales. The Specific Problems scale FML was the most robust predictor of job performance problems among the MMPI–2–RF Interpersonal Functioning scales. FML assesses past and present conflictual relationships with family members. The FML findings were notable for the relatively broad range of criteria that were associated with higher scores on the scale. These included variables in the Emotional Control and Stress Tolerance, Social Competence and Teamwork, Initiative and Drive, and Conscientiousness problem domains, and a poor overall rating. It could be that officers scoring higher on FML lack social support from family or experience distractions at work due to their family problems, thus affecting their performance overall and across a variety of job domains. The Specific Problems scale SAV, which is associated with emotional restriction, difficulty forming close relationships, and social introversion, was associated with failure to accept feedback, interpersonal problems under stress conditions, and teamwork problems. These findings reflect that officers scoring higher on SAV might have resulting interpersonal problems, perhaps because of their lack of engagement in social interactions. Alternatively, higher scores on the FML and SAV scales could reflect interpersonal problems resulting from personality pathology. Indeed, the PSY–5 scales as a set, and INTR-r in particular, were associated with probation problems. As noted earlier, the cohort in this study produced relatively higher and more variant MMPI–2–RF scale scores than other police officer samples. We found the disattenuated estimates of the statistically significant zero-order correlations were nearly all greater than .20, which is the traditional demarcation for meaningful MMPI findings in clinical settings (Graham, 2012). Further, most of these reflected a medium effect size, as described by Cohen (1992). These findings highlight the benefits of applying corrections for attenuation due to range restriction in police officer and personnel selection research, even if range restriction is relatively modest compared to other samples. Of note in this context, range restriction corrections enable the comparison of validity coefficients across police officer samples with varying levels of psychopathology, such that their strength can be evaluated in relation to a common level of variance. RRR analyses produced a number of meaningful findings. For example, the BR for Multi-Tasking Problems under Stress Conditions was 22.1% in this sample. Of this sample, 10.3% scored at or above 55T on Thought Dysfunction (THD). Among those scoring at or above 55T on this scale, 46.7% were identified as having problems with multitasking under stress conditions, whereas only 19.2% of those who scored below this level on THD were similarly rated. Thus, police officers who scored 55T or above on THD were more than twice as likely to be rated as having problems with multitasking under stress conditions than those who scored below the cutoff. Illustrating the potential utility of even lower cutoffs, individuals who scored at or above 45T on EID were at nearly a nine times greater risk of problems related to Failure to Accept Feedback than were those who scored below 45T on this scale. This finding had a selection ratio of 10.3% and base rate of 2.8%. It is important to consider, however, that this sample produced particularly low and range-restricted scores

11 on some of the MMPI–2–RF substantive scales, such as EID (M D 36.5, SD D 5.7). Consequently these findings might not generalize to other settings with higher, more variant scores than those reported here. Replication with other samples from a variety of law enforcement agencies will assist in the identification of the most generalizable cutoffs. At this juncture, we do not recommend using cutoffs below 55T in this setting. The RRR presented were generally associated with adequate sensitivity and specificity rates, as the median values of these statistics were .28 and .92, respectively. A specificity level approximating .90 (and, alternatively, a false positive rate of .10) is desirable in this setting, where psychological evaluations can meaningfully affect an applicant’s career and livelihood. From an organizational standpoint, documentation of high specificity rates is useful when responding to legal challenges of psychological tests, although this might result in relatively low sensitivity estimates. Relatedly, it is important to note that although positive predictive values, which are the ratio of true positives to the number of scale elevations, were generally low (median D .29), these values are attenuated by low base rates (Altman & Bland, 1994). In the context of this study, preselection and selection factors likely decreased the BRs of problem behaviors; thus, the most useful statistics are the RRRs, sensitivities, and specificities, all of which are not affected by low BR concerns. This study is not without limitations. First, we did not have a sufficient number of female police officers, so we conducted analyses for the male police officers only. Relatedly, the sample was primarily comprised of Asian Americans and Hawaiian and Pacific Islanders. These sample characteristics could limit the generalizability of our results. Although approximately 85% of police officers nationally are male (Langton, 2010), they are also primarily of White ethnic descent (Castaneda & Ridgeway, 2010). This limitation notwithstanding, the racial diversity in the sample relative to other investigations in this setting with the MMPI–2–RF scales (e.g., Sellbom et al., 2007; Tarescavage, Brewster, et al., 2014; Tarescavage, Corey, et al., 2014) highlights the criterion validity of the instrument among different ethnic groups. Moreover, because racial test bias investigations with the MMPI–2 (see Graham, 2012, for a summary) and MMPI–2–RF (Monnot, Quirk, Hoerger, & Brewer, 2009) indicate that the predictive validity of the tests is typically comparable across ethnicities, the results of this study are likely to generalize to settings with other racial compositions. Second, we did not include some problem behavior criteria in the analyses because of low BRs for some of the variables. Because the sample was relatively small, including these variables might have introduced influential outliers with the potential to skew our results. Some of these variables would have been particularly informative, as they typically involved more severe problem behaviors such as sexual misconduct and missing court appearances. Future investigations using larger samples are needed to identify correlates with more severe behaviors like these. As noted earlier, a third limitation was that we were unable to obtain data needed to estimate the interrater reliability of our criterion measures. As first described by Spearman (1904), decreases in reliability attenuate correlational coefficients. Thus, limited reliability would have resulted in failure

Downloaded by [134.117.10.200] at 21:00 08 May 2015

12

TARESCAVAGE, COREY, GUPTON, BEN-PORATH

to identify additional associations between MMPI–2–RF scores and negative outcomes, but it would not have produced spurious correlations. Relatedly, although only one trained coder completed each individual section of the SVS, the coding was based on daily rating information that might have been made by multiple coders not trained on the purposes and protocol of this study. However, here, too, any decrement to reliability would attenuate correlation coefficients but would not produce spurious correlations. Finally, we had a relatively small sample of 145 officers, which limited our power to detect statistically significant associations between test scores and SVS criteria. This sample characteristic is particularly salient to this investigation because we conducted significance testing on correlations that were artificially diminished due to range restriction. This limitation, inherent to field research of the type reported in this study, underscores the need to replicate and integrate our findings with those of other studies. Relatedly, owing to power concerns, we did not implement alpha-adjustment procedures to mitigate the risk of Type I errors, so the findings of this study alone should be interpreted cautiously. These limitations notwithstanding, this study adds to a growing literature on the criterion validity and practical utility of the MMPI–2–RF in preemployment assessments of police officer candidates (Sellbom et al., 2007; Tarescavage, Brewster, et al., 2014; Tarescavage, Corey, et al., 2014). In particular, this investigation illustrates the benefits of using disattenuation procedures to estimate the validity of MMPI– 2–RF-based assessment and the ability to rely on lower (than the traditional 65T) cutoffs in these evaluations. However, replication with other samples from a variety of law enforcement agencies will assist in the identification of the most generalizable cutoffs for use in clinical practice.

ACKNOWLEDGMENTS The authors gratefully acknowledge Casey O. Stewart, PsyD, ABPP; Jennifer Doyle, PhD; and Lyntillis Boyd for their contributions to data collection. FUNDING The research reported in this article was supported by a grant from the MMPI publisher, the University of Minnesota Press. As coauthors of the MMPI–2–RF Police Candidate Interpretive Report, Yossef S. Ben-Porath and David M. Corey receive royalties on sales of the report. Yossef S. BenPorath is a paid consultant to the MMPI–2–RF publisher, the University of Minnesota Press, and distributor, Pearson Assessments. He receives royalties on sales of MMPI–2–RF materials and research grants from the MMPI–2–RF publisher. REFERENCES Altman, D. G., & Bland, J. M. (1994). Statistics notes: Diagnostic tests 2. Predictive values. British Medical Journal, 309, 102. Arbisi, P. A., Sellbom, M., & Ben-Porath, Y. S. (2008). Empirical correlates of the MMPI–2 Restructured Clinical (RC) scales in psychiatric inpatients. Journal of Personality Assessment, 90, 122–128. doi:10.1080/ 00223890701845146

Ben-Porath, Y. S., & Tellegen, A. (2008). Empirical correlates of the MMPI–2 Restructured Clinical (RC) scales in mental health, forensic, and nonclinical settings: An introduction. Journal of Personality Assessment, 90, 119–121. doi:10.1080/00223890701845120 Ben-Porath, Y. S., & Tellegen, A. (2011). MMPI–2–RF: Manual for administration, scoring and interpretation. Minneapolis: University of Minnesota Press. (Original work published 2008) Boes, J. O., Chandler, C. J., & Timm, H. W. (1997). Police integrity: Use of personality measures to identify corruption-prone officers (Pub. No. PERSTR-97-003). Monterey, CA: Defense Personnel Security Research Center. Bradford, B., Jackson, J., & Stanko, E. A. (2009). Contact and confidence: Revisiting the impact of public encounters with the police. Policing & Society, 19(1), 20–46. Brewster, J., & Stoloff, M. L. (1999). Using the good cop/bad cop profile with the MMPI–2. Journal of Police and Criminal Psychology, 14(2), 29–34. Butcher, J., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., Dahlstrom, G. W., & Kaemmer, B. (2001). Minnesota Multiphasic Personality Inventory–2: Manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press. Caillouet, B. A., Boccaccini, M. T., Varela, J. G., Davis, R. D., & Rostow, C. D. (2010). Predictive validity of the MMPI–2 PSY–5 scales and facets for law enforcement officer employment outcomes. Criminal Justice and Behavior, 37, 217–238. California Commission on Peace Officer Standards and Training. (2005). Peace officer psychological screening dimensions. Sacramento, CA: Author. Carpenter, B. N., & Raza, S. M. (1987). Personality characteristics of police applicants: Comparisons across subgroups and with other populations. Journal of Police Science & Administration, 15(1), 10–17. Carr, P. J., Napolitano, L., & Keating, J. (2007). We never call the cops and here is why: A qualitative examination of legal cynicism in three Philadelphia neighborhoods. Criminology, 45, 445–480. Castaneda, L. W., & Ridgeway, G. (2010). Today’s police and sheriff recruits: Insights from the newest members of America’s law enforcement community. Santa Monica, CA: The RAND Corporation. Cattell, R. B., & Eber, H. (1950). The 16 personality factor questionnaire. Champaign, IL: Institute for Personality and Ability Testing. Cochrane, R. E., Tett, R. P., & Vandecreek, L. (2003). Psychological testing and the selection of police officers: A national survey. Criminal Justice and Behavior, 30, 511–537. Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155. Corey, D. M., & Borum, R. (2013). Forensic assessment for high risk occupations. In R. K. Otto & I. B. Weiner (Eds.), Handbook of psychology (Vol. 11, pp. 246–269). Hoboken, NJ: Wiley. Costa, P. T., & MacCrae, R. R. (1992). Revised NEO Personality Inventory (NEO PI–R) and NEO Five-Factor Inventory (NEO FFI): Professional manual. Odessa, FL: Psychological Assessment Resources. Detrick, P., & Chibnall, J. T. (2008). Positive response distortion by police officer applicants: Association of Paulhus Deception Scales with MMPI–2 and Inwald Personality Inventory validity scales. Assessment, 15(1), 87–96. Detrick, P., Chibnall, J. T., & Call, C. (2010). Demand effects on positive response distortion by police officer applicants on the Revised NEO Personality Inventory. Journal of Personality Assessment, 92, 410–415. Ellis, P. D. (2010). The essential guide to effect sizes. Cambridge, England: Cambridge University Press. Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. Gough, H. G. (1956). California Psychological Inventory. Palo Alto, CA: Consulting Psychologists Press. Graham, J. R. (2012). The MMPI–2: Assessing personality and psychopathology (5th ed.). New York, NY: Oxford University Press. Graham, J. R., Ben-Porath, Y. S., & McNulty, J. L. (1999). MMPI–2 correlates for outpatient community mental health settings. Minneapolis: University of Minnesota Press. Handel, R. W., & Archer, R. P. (2008). An investigation of the psychometric properties of the MMPI–2 Restructured Clinical (RC) scales with mental

Downloaded by [134.117.10.200] at 21:00 08 May 2015

MMPI–2–RF POLICE SCREENING health inpatients. Journal of Personality Assessment, 90, 239–249. doi:10.1080/00223890701884954 Hiatt, D., & Hargrave, G. E. (1988). MMPI profiles of problem peace officers. Journal of Personality Assessment, 52, 722–731. Hoffman, C. C. (1995). Applying range restriction corrections using published norms: Three case studies. Personnel Psychology, 48, 913–923. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage. International Association of Chiefs of Police. (2009). Preemployment psychological evaluation guidelines. Alexandria, VA: Author. Retrieved from http://www. theiacp.org/portals/0/documents/pdfs/Psych-PreemploymentPsychEval.pdf Inwald, R., Knatz, H., & Shusman, E. (1982). Inwald personality inventory manual. New York, NY: Hilson Research. Langton, L. (2010). Women in law enforcement, 1987–2008. Washington, DC: U.S. Department of Justice, Office of Justice Programs, Bureau of Justice Statistics. Lowmaster, S. E., & Morey, L. C. (2012). Predicting law enforcement officer job performance with the Personality Assessment Inventory. Journal of Personality Assessment, 94, 254–261. Mastrofski, S. D., Reisig, M. D., & McCluskey, J. D. (2002). Police disrespect toward the public: An encounter-based analysis. Criminology, 40, 519–552. Mazerolle, L., Bennett, S., Antrobus, E., & Eggins, E. (2012). Procedural justice, routine encounters and citizen perceptions of police: Main findings from the Queensland Community Engagement Trial (QCET). Journal of Experimental Criminology, 8, 343–367. Monnot, M. J., Quirk, S. W., Hoerger, M., & Brewer, L. (2009). Racial bias in personality assessment: Using the MMPI–2 to predict psychiatric diagnoses of African American and Caucasian chemical dependency inpatients. Psychological Assessment, 21, 137–151. doi:10.1037/a0015316 Morey, L. C. (2007). Personality Assessment Inventory (PAI). Lutz, FL: Psychological Assessment Resources. National Advisory Commission on Criminal Justice Standards and Goals. (1973). Task force on police. Washington, DC: Author. Ones, D., & Viswesvaran, C. (2003). Job-specific applicant pools and national norms for personality scales: Implications for range-restriction corrections in validation research. Journal of Applied Psychology, 88, 570–577. Ones, D., Viswesvaran, C., Cullen, M., Drees, S., & Langkamp, K. (2003). Personality and police officer behaviors: A comprehensive meta-analysis. Paper presented at the symposium conducted at the 18th annual meeting of the Society for Industrial and Organizational Psychology, Orlando, FL. Reichenbach, H. (1938). Experience and prediction: An analysis of the foundations and the structure of knowledge. Chicago, IL: University of Chicago Press. Sackett, P. R., & Ostgaard, D. J. (1994). Job-specific applicant pools and national norms for cognitive ability tests: Implications for range restriction corrections in validation research. Journal of Applied Psychology, 79, 680–684. Scogin, F., Schumacher, J., Gardner, J., & Chaplin, W. (1995). Predictive validity of psychological testing in law enforcement settings. Professional Psychology: Research and Practice, 26, 68–71.

13 Sellbom, M., Ben-Porath, Y. S., & Graham, J. R. (2006). Correlates of the MMPI–2 Restructured Clinical (RC) scales in a college counseling setting. Journal of Personality Assessment, 86, 88–99. Sellbom, M., Fischler, G. L., & Ben-Porath, Y. S. (2007). Identifying MMPI–2 predictors of police officer integrity and misconduct. Criminal Justice and Behavior, 34, 985–1004. Sellbom, M., Graham, J. R., & Schenk, P. W. (2006). Incremental validity of the MMPI–2 Restructured Clinical (RC) scales in a private practice sample. Journal of Personality Assessment, 86, 196–205. Serafino, G. F. (2010). Fundamental issues in police psychology assessment. In P. A. Weiss (Ed.), Personality assessment in police psychology: A 21st century perspective (pp. 29–55). Springfield, IL: Thomas. Shedler, J., Mayman, M., & Manis, M. (1993). The illusion of mental health. American Psychologist, 48, 1117–1131. doi:10.1037/0003-066X.48.11. 1117 Spearman, C. (1904). The proof and measurement of association between two things. The American Journal of Psychology, 15(1), 72–101. Spielberger, C. D. (1988). Manual for the State–Trait Anger Expression Inventory (STAXI). Odessa, FL: Psychological Assessment Resources. Spilberg, S. W., & Corey, D. M. (2014). POST peace officer psychological screening manual. Sacramento, CA: California Commission on Peace Officer Standards and Training. Tarescavage, A. M., Brewster, J., Corey, D. M., & Ben-Porath, Y. S. (2014). Use of pre-hire MMPI–2–RF police candidate scores to predict supervisor ratings of post-hire performance. Assessment. Advance online publication. doi:10.1177/1073191114548445 Tarescavage, A. M., Corey, D. M., & Ben-Porath, Y. S. (2014). Minnesota Multiphasic Personality Inventory–2–Restructured Form (MMPI–2–RF) predictors of police officer problem behavior. Assessment, 22, 116–132. doi:10.1177/1073191114534885 Tellegen, A., & Ben-Porath, Y. S. (2011). Minnesota Mutliphasic Personality Inventory–2–Restructured Form: Technical manual. Minneapolis: University of Minnesota Press. (Original work published 2008) University of Minnesota Press. (2014). MMPI–2–RF references. Retrieved from http://www.upress.umn.edu/test-division/MMPI-2-RF/mmpi-2-rfreferences Van der Heijden, P. T., Egger, J. I., & Derksen, J. J. (2010). Comparability of scores on the MMPI–2–RF scales generated with the MMPI–2 and MMPI– 2–RF booklets. Journal of Personality Assessment, 92, 254–259. doi:10.1080/00223891003670208 Weiss, P. A., Vivian, J. E., Weiss, W. U., Davis, R. D., & Rostow, C. D. (2013). The MMPI–2 L scale, reporting uncommon virtue, and predicting police performance. Psychological Services, 10, 123. Weiss, W. U., Davis, R., Rostow, C., & Kinsman, S. (2003). The MMPI–2 L scale as a tool in police selection. Journal of Police and Criminal Psychology, 18(1), 57–60. Weitzer, R. (2002). Incidents of police misconduct and public opinion. Journal of Criminal Justice, 30, 397–408.