Psychological Assessment 2015, Vol. 27, No. 2, 733-737
© 2015 American Psychological Association 1040-3590/15/$ 12.00 http://dx.doi.org/10.1037/a0038802
BRIEF REPORT
Analysis of Minnesota Multiphasic Personality Inventory-2-Restmctured Form Response Bias Indicators as Suppressors or Moderators in a Medical Setting Rebecca E. Wershba
Dona E. C. Locke
Arizona State University
Mayo Clinic, Scottsdale, Arizona
Richard I. Lanyon Arizona State University The use of response bias indicators in psychological measurement has been contentious, with debate as to whether they actually suppress or moderate the ability of substantive psychological indicators to identify the construct of interest. Suppression would indicate that predictor variables contain invalid variance that the bias indicators can suppress, while moderation would indicate differential levels of predictive validity at different levels of bias. Response bias indicators on the Minnesota Multiphasic Personality Inventory (MMPI)-2-Restructured Form (MMPI-2-RF) [infrequent re sponses (F-r), infrequent somatic responses (Fs), infrequent psychopathology responses (Fp-r), adjustment validity (K-r), uncommon virtues (L-r), symptom validity (FBS-r), and Response Bias Scale (RBS)] were tested to determine whether they suppressed or moderated the ability of the Restructured Clinical Scale 1 (RC1) and Neurologic Complaints (NUC) scale to discriminate between epileptic seizures (ES) and nonepileptic seizures (NES, a conversion disorder that is often misdiagnosed as ES). The MMPI-2-RF was completed by 399 patients with a confirmed diagnosis of ES or NES via Epilepsy Monitoring Unit evaluation. Moderated logistic regression was used to test for moderation, and logistic regression was used to test for suppression. Most of the response bias variables showed a suppressor effect, but moderator effects were not found. These findings extend the use of bias indicators to a psychomedical context.
Keywords: MMPI-2-RF, response bias, nonepileptic seizures, epilepsy
Response bias, as assessed by what have traditionally been termed validity scales, has long been a source of concern for psychologists and others who create and use psychological assess ment instruments. Response bias has been described as a “consis tent tendency to respond inaccurately to a substantive indicator, resulting in a systematic error in prediction” (McGrath, Mitchell,
Kim, & Hough, 2010). Response bias might create an artificially good impression of a person’s psychological functioning; this is known as positive impression management (PIM). Conversely, negative impression management (NIM ) refers to a response bias that indicates functioning that is w orse than the actual condition. The M innesota Multiphasic Personality Inventory (MMPI)-2Restructured Form (M MPI-2-RF; Ben-Porath & Tellegen, 2008) includes a number of response bias indicators, both for PIM, uncommon virtues (L-r) and adjustment validity (K-r), and for NIM, infrequent responses (F-r); infrequent somatic responses (Fs); and infrequent psychopathology responses (Fp-r). Additional bias indicators include symptom validity (FBS-r) and Response Bias Scale (RBS), both o f which include aspects of PIM and NIM. The M M PI-2-RF and MMPI-2 validity scales have been shown to be effective at detecting NIM (Gervais, Lees-Haley, & Ben-Porath, 2007; Wygant et al„ 2009) and PIM (Baer, W etter, Nichols, Greene, & Berry, 1995; Sellbom & Bagby, 2008) and distinguish ing between malingering and genuine medical conditions (Sell bom, Wygant, & Bagby, 2012). The wealth of current research plus a variety o f bias indicators makes the M M PI-2-RF an excel lent tool for further research in response bias.
This article was published Online First March 2, 2015. Rebecca E. Wershba, Department of Psychology, Arizona State Univer sity; Dona E. C. Locke, Department of Psychiatry and Psychology, Mayo Clinic, Scottsdale, Arizona; Richard I. Lanyon, Department of Psychology, Arizona State University. Rebecca E. Wershba is now at Department of Psychiatry, Cambridge Health Alliance. This research was supported in part by an MMPI-2/MMPI-2-RF rescor ing grant from the University of Minnesota Press. We thank Roger Millsap, PhD, and Joseph G. Hentz, MS, for their statistical input. Correspondence concerning this article should be addressed to Rebecca E. Wershba, Department of Psychiatry, Cambridge Health Alliance, 1493 Cambridge Street, Cambridge, MA 02139-9991. E-mail: werre5@gmail .com 733
734
WERSHBA, LOCKE, AND LANYON
There are two possible ways in which response bias scales can improve prediction of a criterion by a substantive indicator. It could act as a suppressor, which means that it suppresses invalid variance in a companion predictor variable. For example, the K correction was designed for the original MMPI to suppress invalid variance due to the expected effect of “defensiveness” in assessing psychopathology, on the assumption that highly defensive persons are less forthcoming about impaired psychological functioning. A response bias scale can also act as a moderator by changing the predictive ability of the substantive indicators at lower or higher levels. For example, given a low response bias scale score, the substantive indicator may predict a criterion with a high degree of accuracy, but as response bias increases, the substantive indicator may lose predictive validity. Although it is generally accepted that response bias scores reflect a respondent’s general approach to responding to a test, there is debate as to whether high scores affect the substantive scales in a clinically significant way. Indeed, McGrath et al.’s (2010) review article concluded that there is insufficient evi dence that response bias indicators affect the relationship be tween a substantive indicator and a criterion to a practically meaningful extent. The studies reviewed involved personality assessment, workplace variables, emotional disorders, eligibil ity for disability, and forensic assessment. There was insuffi cient evidence for drawing conclusions regarding the latter three populations, but for the first two, evidence indicated only mild support for the utility of bias indicators. However, a major concern regarding this review by prominent researchers in the bias indicator field (e.g., Rohling et al., 2011) involved the overly wide-ranging nature of conclusions reached based on the articles reviewed. The issues for the main populations studied in that review (those in the workplace) primarily in volved P1M, as opposed to the NIM concerns that are prominent in psychopathological, forensic, or litigating populations. An additional concern with the review is that each study addressed either moderation or suppression but not both, raising the possibility that response bias might have affected the substan tive indicators’ accuracy in ways that were not measured. One promising area in which to study the clinical effect of response bias is the psychomedical field. An appropriate popula tion for such a study is patients with a differential diagnosis of epileptic seizures (ES) or nonepileptic seizures (NES), a conver sion disorder that is often misdiagnosed as ES (Cragar, Berry, Fakhoury, Cibula, & Schmitt, 2002). Clinician observation of seizure activity may raise suspicion for NES, as the seizure-like activity in NES is typically physiologically inconsistent with epi leptic seizure semiology. A patient with a possible diagnosis of NES may be sent to an Epilepsy Monitoring Unit (EMU), in which video-electroencephalogram (vEEG) is used as the “gold standard” to differentiate between ES and NES. Here, vEEG detects the presence or absence of epileptiform discharge during observed seizure-like activity, and can be used to accurately confirm a diagnosis that might be suggested by an office exam or psycho logical assessment. The MMPI-2-RF is one such test that has been utilized to discriminate between ES and NES. Recent research findings have shown that a cut score of 65 on the Restructured Clinical Scale 1 (RC1; somatic complaints) scale discriminated between these dis orders at an overall hit rate of 68% (Locke et al., 2010). The
Neurological Complaints (NUC) scale was also a useful discrim inator with an overall hit rate of 67%. It is possible that the utility of the substantive scales could be improved by incorporating the potential suppressor or moderator effect of MMPI-2-RF validity scale scores on the RC1 or NUC scales.
Aim of the Present Study The goal of the present study was to evaluate whether response bias indicators (K-r, L-r, F-r, Fs, Fp-r, FBS-r, RBS) impact sub stantive scales in this unique population and to determine whether any such impact occurs through moderation or suppression.
Method Participants Participants were patients who had been evaluated in the Epi lepsy Monitoring Unit (EMU) at the Mayo Clinic Hospital in Phoenix, Arizona between April 2001 and April 2009. This sample has been previously utilized in three studies with different goals and analyses (Locke et al., 2010; Locke & Thomas, 2011; Thomas & Locke, 2010). A total of 664 patients were admitted during this time period. All were given the MMPI-2 as part of a standard neuropsychological evaluation, which was then rescored to the MMPI-2-RF. Diagnosis was determined by a board certified neu rologist and fellowship-trained epilepsy specialist on the basis of the vEEG findings during admission. Of the 664 patients, 221 were diagnosed with epilepsy, 219 were diagnosed with NES, 24 with both ES and NES, 166 were inde terminate, and 34 patients were diagnosed with other physiological disorders such as sleep, autonomic nervous system, or vascular disorders. Details of the specific criteria for each diagnostic cate gory can be found in Locke et al. (2010). Patients other than pure ES or NES were excluded from the study. We also excluded the readmissions among the NES and ES patients (n = 11) and protocols invalid due to missing items or random responding (True Response Inconsistency Scale (TRIN) or Variable Response In consistency Scale (VRIN) >80, n = 24; cannot say £15, n = 6). After exclusions, the sample included 196 ES and 203 NES pa tients. Demographic and medical history information was collected via record review.
Statistical Analyses To determine whether moderation or suppression existed for these patients, binary logistic regression was utilized. Moderation was tested through moderated logistic regression. Suppression was tested through stepwise logistical regression, and comparing the standardized regression coefficient of the substantive indicator when the bias indicator was included as a covariate to when it was not included. If addition of the bias indicator significantly affected the outcome, it remained in the equation; otherwise, it was re moved. If the coefficient of the substantive scale was higher when the bias indicator was included, this was evidence of suppression.
Results In a previous study examining MMPI-2-RF scale differences between the ES and NES groups in this population, Locke et al.
MMPI-2-RF RESPONSE BIAS INDICATORS AS SUPPRESSORS
735
Table 1 Demographic Data and Psychiatric History for Each Diagnostic Group ES
NES
Variable
n
M
SD
n
M
SD
tlx2
Age (years) Gender (% female) Ethnicity (% White) Handedness (% right) Education (years) Current psychotropic medicines (% yes) Presence of psychiatric history (% yes) Presence of substance use history (% yes) WRAT-4 Reading WAIS-I1I Full Scale IQ
196 196 196 196 196 196 196 196 176 175
41.89 65.8 92.3 88.3 13.94 74.5 53.1 14.8 100.35 100.38
14.82
203 203 203 203 203 203 203 203 174 152
43.46 81.8 94.6 88.2 14.00 50.7 80.3 18.2 100.18 101.23
13.83
-1 .1 2 16.50* .59 .24 -.7 2 30.12* 34.97* .37 .41 -.6 3
-
■
— —
2.26 — — —
9.44 11.71
___ ___. ___
2.28 ___ ___ ___
9.59 13.69
Note. ES = epileptic seizures; NES = nonepileptic seizures. * x2 values, p < .05.
(2010) presented detailed data on group differences for all the MMPI-2-RF scales as well as effect sizes related to those differ ences. It was shown that the largest effect sizes occurred with the RC1 (t|j; = 0.108) and NUC (rip = 0.113) scales. These two scales were therefore selected as the focus of the present analyses. De mographic and other comparisons (see Table 1) showed that the NES group had a higher percentage of females than the ES group, used more psychotropic medications, and had a greater likelihood of a history of psychiatric treatment. Table 2 contains mean differences between the ES and NES groups on MMP1-2-RF scales of interest, using gender and psy chiatric medications as covariates. NES patients scored signifi cantly higher than ES patients on substantive scales RC1 and NUC, and on validity scales Fs, FBS-r, RBS, and K-r. Table 2 also provides the percentage of persons in each group scoring at or above the cut-score for each scale that indicates the greatest degree of validity problems and/or over- or underreporting; as suggested by the MMPI-2-RF manual, this occurs at T = 100, with the exception of scales L-r (T = 80) and K-r (T = 70).
Correlations were calculated between each of the substantive indicators, the bias indicators, and the criterion variables; these are shown in Table 3. Bias indicators were found to be minimally related to the diagnosis of NES and ES (all correlations
© 2015 American Psychological Association 1040-3590/15/$ 12.00 http://dx.doi.org/10.1037/a0038802
BRIEF REPORT
Analysis of Minnesota Multiphasic Personality Inventory-2-Restmctured Form Response Bias Indicators as Suppressors or Moderators in a Medical Setting Rebecca E. Wershba
Dona E. C. Locke
Arizona State University
Mayo Clinic, Scottsdale, Arizona
Richard I. Lanyon Arizona State University The use of response bias indicators in psychological measurement has been contentious, with debate as to whether they actually suppress or moderate the ability of substantive psychological indicators to identify the construct of interest. Suppression would indicate that predictor variables contain invalid variance that the bias indicators can suppress, while moderation would indicate differential levels of predictive validity at different levels of bias. Response bias indicators on the Minnesota Multiphasic Personality Inventory (MMPI)-2-Restructured Form (MMPI-2-RF) [infrequent re sponses (F-r), infrequent somatic responses (Fs), infrequent psychopathology responses (Fp-r), adjustment validity (K-r), uncommon virtues (L-r), symptom validity (FBS-r), and Response Bias Scale (RBS)] were tested to determine whether they suppressed or moderated the ability of the Restructured Clinical Scale 1 (RC1) and Neurologic Complaints (NUC) scale to discriminate between epileptic seizures (ES) and nonepileptic seizures (NES, a conversion disorder that is often misdiagnosed as ES). The MMPI-2-RF was completed by 399 patients with a confirmed diagnosis of ES or NES via Epilepsy Monitoring Unit evaluation. Moderated logistic regression was used to test for moderation, and logistic regression was used to test for suppression. Most of the response bias variables showed a suppressor effect, but moderator effects were not found. These findings extend the use of bias indicators to a psychomedical context.
Keywords: MMPI-2-RF, response bias, nonepileptic seizures, epilepsy
Response bias, as assessed by what have traditionally been termed validity scales, has long been a source of concern for psychologists and others who create and use psychological assess ment instruments. Response bias has been described as a “consis tent tendency to respond inaccurately to a substantive indicator, resulting in a systematic error in prediction” (McGrath, Mitchell,
Kim, & Hough, 2010). Response bias might create an artificially good impression of a person’s psychological functioning; this is known as positive impression management (PIM). Conversely, negative impression management (NIM ) refers to a response bias that indicates functioning that is w orse than the actual condition. The M innesota Multiphasic Personality Inventory (MMPI)-2Restructured Form (M MPI-2-RF; Ben-Porath & Tellegen, 2008) includes a number of response bias indicators, both for PIM, uncommon virtues (L-r) and adjustment validity (K-r), and for NIM, infrequent responses (F-r); infrequent somatic responses (Fs); and infrequent psychopathology responses (Fp-r). Additional bias indicators include symptom validity (FBS-r) and Response Bias Scale (RBS), both o f which include aspects of PIM and NIM. The M M PI-2-RF and MMPI-2 validity scales have been shown to be effective at detecting NIM (Gervais, Lees-Haley, & Ben-Porath, 2007; Wygant et al„ 2009) and PIM (Baer, W etter, Nichols, Greene, & Berry, 1995; Sellbom & Bagby, 2008) and distinguish ing between malingering and genuine medical conditions (Sell bom, Wygant, & Bagby, 2012). The wealth of current research plus a variety o f bias indicators makes the M M PI-2-RF an excel lent tool for further research in response bias.
This article was published Online First March 2, 2015. Rebecca E. Wershba, Department of Psychology, Arizona State Univer sity; Dona E. C. Locke, Department of Psychiatry and Psychology, Mayo Clinic, Scottsdale, Arizona; Richard I. Lanyon, Department of Psychology, Arizona State University. Rebecca E. Wershba is now at Department of Psychiatry, Cambridge Health Alliance. This research was supported in part by an MMPI-2/MMPI-2-RF rescor ing grant from the University of Minnesota Press. We thank Roger Millsap, PhD, and Joseph G. Hentz, MS, for their statistical input. Correspondence concerning this article should be addressed to Rebecca E. Wershba, Department of Psychiatry, Cambridge Health Alliance, 1493 Cambridge Street, Cambridge, MA 02139-9991. E-mail: werre5@gmail .com 733
734
WERSHBA, LOCKE, AND LANYON
There are two possible ways in which response bias scales can improve prediction of a criterion by a substantive indicator. It could act as a suppressor, which means that it suppresses invalid variance in a companion predictor variable. For example, the K correction was designed for the original MMPI to suppress invalid variance due to the expected effect of “defensiveness” in assessing psychopathology, on the assumption that highly defensive persons are less forthcoming about impaired psychological functioning. A response bias scale can also act as a moderator by changing the predictive ability of the substantive indicators at lower or higher levels. For example, given a low response bias scale score, the substantive indicator may predict a criterion with a high degree of accuracy, but as response bias increases, the substantive indicator may lose predictive validity. Although it is generally accepted that response bias scores reflect a respondent’s general approach to responding to a test, there is debate as to whether high scores affect the substantive scales in a clinically significant way. Indeed, McGrath et al.’s (2010) review article concluded that there is insufficient evi dence that response bias indicators affect the relationship be tween a substantive indicator and a criterion to a practically meaningful extent. The studies reviewed involved personality assessment, workplace variables, emotional disorders, eligibil ity for disability, and forensic assessment. There was insuffi cient evidence for drawing conclusions regarding the latter three populations, but for the first two, evidence indicated only mild support for the utility of bias indicators. However, a major concern regarding this review by prominent researchers in the bias indicator field (e.g., Rohling et al., 2011) involved the overly wide-ranging nature of conclusions reached based on the articles reviewed. The issues for the main populations studied in that review (those in the workplace) primarily in volved P1M, as opposed to the NIM concerns that are prominent in psychopathological, forensic, or litigating populations. An additional concern with the review is that each study addressed either moderation or suppression but not both, raising the possibility that response bias might have affected the substan tive indicators’ accuracy in ways that were not measured. One promising area in which to study the clinical effect of response bias is the psychomedical field. An appropriate popula tion for such a study is patients with a differential diagnosis of epileptic seizures (ES) or nonepileptic seizures (NES), a conver sion disorder that is often misdiagnosed as ES (Cragar, Berry, Fakhoury, Cibula, & Schmitt, 2002). Clinician observation of seizure activity may raise suspicion for NES, as the seizure-like activity in NES is typically physiologically inconsistent with epi leptic seizure semiology. A patient with a possible diagnosis of NES may be sent to an Epilepsy Monitoring Unit (EMU), in which video-electroencephalogram (vEEG) is used as the “gold standard” to differentiate between ES and NES. Here, vEEG detects the presence or absence of epileptiform discharge during observed seizure-like activity, and can be used to accurately confirm a diagnosis that might be suggested by an office exam or psycho logical assessment. The MMPI-2-RF is one such test that has been utilized to discriminate between ES and NES. Recent research findings have shown that a cut score of 65 on the Restructured Clinical Scale 1 (RC1; somatic complaints) scale discriminated between these dis orders at an overall hit rate of 68% (Locke et al., 2010). The
Neurological Complaints (NUC) scale was also a useful discrim inator with an overall hit rate of 67%. It is possible that the utility of the substantive scales could be improved by incorporating the potential suppressor or moderator effect of MMPI-2-RF validity scale scores on the RC1 or NUC scales.
Aim of the Present Study The goal of the present study was to evaluate whether response bias indicators (K-r, L-r, F-r, Fs, Fp-r, FBS-r, RBS) impact sub stantive scales in this unique population and to determine whether any such impact occurs through moderation or suppression.
Method Participants Participants were patients who had been evaluated in the Epi lepsy Monitoring Unit (EMU) at the Mayo Clinic Hospital in Phoenix, Arizona between April 2001 and April 2009. This sample has been previously utilized in three studies with different goals and analyses (Locke et al., 2010; Locke & Thomas, 2011; Thomas & Locke, 2010). A total of 664 patients were admitted during this time period. All were given the MMPI-2 as part of a standard neuropsychological evaluation, which was then rescored to the MMPI-2-RF. Diagnosis was determined by a board certified neu rologist and fellowship-trained epilepsy specialist on the basis of the vEEG findings during admission. Of the 664 patients, 221 were diagnosed with epilepsy, 219 were diagnosed with NES, 24 with both ES and NES, 166 were inde terminate, and 34 patients were diagnosed with other physiological disorders such as sleep, autonomic nervous system, or vascular disorders. Details of the specific criteria for each diagnostic cate gory can be found in Locke et al. (2010). Patients other than pure ES or NES were excluded from the study. We also excluded the readmissions among the NES and ES patients (n = 11) and protocols invalid due to missing items or random responding (True Response Inconsistency Scale (TRIN) or Variable Response In consistency Scale (VRIN) >80, n = 24; cannot say £15, n = 6). After exclusions, the sample included 196 ES and 203 NES pa tients. Demographic and medical history information was collected via record review.
Statistical Analyses To determine whether moderation or suppression existed for these patients, binary logistic regression was utilized. Moderation was tested through moderated logistic regression. Suppression was tested through stepwise logistical regression, and comparing the standardized regression coefficient of the substantive indicator when the bias indicator was included as a covariate to when it was not included. If addition of the bias indicator significantly affected the outcome, it remained in the equation; otherwise, it was re moved. If the coefficient of the substantive scale was higher when the bias indicator was included, this was evidence of suppression.
Results In a previous study examining MMPI-2-RF scale differences between the ES and NES groups in this population, Locke et al.
MMPI-2-RF RESPONSE BIAS INDICATORS AS SUPPRESSORS
735
Table 1 Demographic Data and Psychiatric History for Each Diagnostic Group ES
NES
Variable
n
M
SD
n
M
SD
tlx2
Age (years) Gender (% female) Ethnicity (% White) Handedness (% right) Education (years) Current psychotropic medicines (% yes) Presence of psychiatric history (% yes) Presence of substance use history (% yes) WRAT-4 Reading WAIS-I1I Full Scale IQ
196 196 196 196 196 196 196 196 176 175
41.89 65.8 92.3 88.3 13.94 74.5 53.1 14.8 100.35 100.38
14.82
203 203 203 203 203 203 203 203 174 152
43.46 81.8 94.6 88.2 14.00 50.7 80.3 18.2 100.18 101.23
13.83
-1 .1 2 16.50* .59 .24 -.7 2 30.12* 34.97* .37 .41 -.6 3
-
■
— —
2.26 — — —
9.44 11.71
___ ___. ___
2.28 ___ ___ ___
9.59 13.69
Note. ES = epileptic seizures; NES = nonepileptic seizures. * x2 values, p < .05.
(2010) presented detailed data on group differences for all the MMPI-2-RF scales as well as effect sizes related to those differ ences. It was shown that the largest effect sizes occurred with the RC1 (t|j; = 0.108) and NUC (rip = 0.113) scales. These two scales were therefore selected as the focus of the present analyses. De mographic and other comparisons (see Table 1) showed that the NES group had a higher percentage of females than the ES group, used more psychotropic medications, and had a greater likelihood of a history of psychiatric treatment. Table 2 contains mean differences between the ES and NES groups on MMP1-2-RF scales of interest, using gender and psy chiatric medications as covariates. NES patients scored signifi cantly higher than ES patients on substantive scales RC1 and NUC, and on validity scales Fs, FBS-r, RBS, and K-r. Table 2 also provides the percentage of persons in each group scoring at or above the cut-score for each scale that indicates the greatest degree of validity problems and/or over- or underreporting; as suggested by the MMPI-2-RF manual, this occurs at T = 100, with the exception of scales L-r (T = 80) and K-r (T = 70).
Correlations were calculated between each of the substantive indicators, the bias indicators, and the criterion variables; these are shown in Table 3. Bias indicators were found to be minimally related to the diagnosis of NES and ES (all correlations
