Neurop~ychologyReview, Vol. 2, No. 3, 1991

Forensic Neuropsychology: The Art of Practicing a Science that Does Not Yet Exist David F a u s t j

Despite its future promise, neuropsychologicat evidence generally lacks scientifically demonstrated value for resolving legal issues, and thus, if admitted into court, shouM be accorded little or no weight. In support of this contention, examples of problems and limits in forensic neuropsychology are described. These include contrasts between the clinical and forensic context; the base-rate problem; lack of standardized practices; problems assessing credibility or malingering; difficulties determining prior functioning~ limits in the capacity to integrate complex data; and the lack of relation between judgmental accuracy and education, experience, or credentials. Some possible counterarguments are also addressed. KEY WORDS: forensic neuropsychology; clinical assessment; neuropsychological methods; intellectual functioning.

INTRODUCTION

Neuropsychological evidence is generally of negligible value in resolving legal issues. Whether neuropsychological evidence should be admitted into the courts is ultimately a matter for the courts to decide, but if admitted such evidence should be accorded little or no weight. In presenting this argument I cannot be comprehensive as the subject matter is so vast (for more detailed coverage see Faust et al., 1991). Thus, I will offer a sample of problems that illustrate the obstacles and limitations in knowledge that need to be overcome before neuropsychological evidence witl offer true assistance to the trier of fact. 1Psychology Department, Chafee Center, University of Rhode Island, Kingston, Rhode IsJand 02881. To whom correspondence should be addressed. 205 1040~7308/9110900.0205506.50/0 © 1991 Plenum Publishing Corporation

206

Faust

Before proceeding, I wish to make clear what I am not arguing. First, nothing I will present should be viewed as challenging psychologists' character, integrity, intelligence, or motivations. Second, my criticisms are directed toward forensic neuropsychology, and not neuropsychology as a whole. For example, I am not criticizing efforts to provide therapeutic or remedial services to brain-injured individuals (which is not the same as blindly endorsing all such activities), nor am I criticizing research efforts in the field. Rather, the quality and quantity of research within clinical neuropsychology are clearly praiseworthy. In fact, I anticipate that these research efforts will eventually create a foundation for forensic assessment that truly assists the trier of fact. However, I do not believe we are there yet. Along these lines, the value or accuracy of clinical assessment should not be assumed but should be demonstrated or proven through scientific research. With the foregoing in mind, it is possible to consider the scientific status of forensic neuropsycholog3,.

PROBLEMS AND LIMITATIONS IN FORENSIC NEUROPSYCHOLOGY: SOME EXAMPLES Contrasts Between the Clinical and Forensic Contexts

Suppose Neuropsychological Method X achieves some degree of success in answering certain questions within the clinical setting or context. An expert may suppose that this level of success generalizes to the legal context. However, claims or demonstrations of clinical utility often fail to establish that the method or the clinician is able to answer legal questions with reasonable accuracy or validity, or is capable of aiding the trier of fact. There are at least three such potential gaps (Ziskin, 1981) between the clinic and the courtroom. For one, a clinical diagnosis may not relate in any clear way to the alleged consequences of an injury or event. For example, the diagnosis of brain damage may relate minimally to a legal issue, such as the individual's functional capacity or mental competency. As Reitan (1986) stated, It is meaningful to label only if the label has some significance. The label of "brain damage" has little specific significance, considering the great variability among individuals who fall in this category. Thus, the development of a procedure which permits only identification of individuals for whom use of the label is appropriate is equivalent to name-calling, especially when the label has adverse connotations.

(p. 13) Second, the clinician's role may change considerably when he or she moves from the clinical to the forensic setting (Rogers and Cavanaugh, 1983). As Faust and Ziskin (1988) stated,

Forensic Neuropsychology

207

Clinicians who enter the forensic arena also shift from their more familiar role as the patient's helping agent and instead seek to uncover truth, whatever its implications for the person under examination. The clinician thus becomes a potential adversary. The forensic role is often less familiar or practiced, the clinician's engrained tendency to support or empathize may cloud objectivity, and the person being examined may be less inclined to disclose information openly and honestly. (p. 32) One of the reviewers interpreted the previous passage as suggesting that clinicians play lawyer in the courtroom, another reviewer requested data demonstrating behavioral shifts across settings, and another expressed uncertainly about the difference between clinical and forensic roles. My main point, however, is that the role intended for the clinician in the courtroom commonly does differ from that expected in the clinic. In most clinical situations, the patient seeks out the clinician for help, and it is the professional's role to provide that help. In contrast, the courts expect an expert to call it as he or she sees i t w t o present evidence honestly, and not to intentionally slant or distort the evidence in favor of the examinee. Telling the truth may or may not coincide with the examinee's interests, and in the extreme case may even contribute to a sentence of death. In the clinic, one can imagine saying to one's patient, "I don't think I'd better write that letter for you, because I don't want to lie and what I have to say might hurt you," whereas it is hard to imagine taking the stand as an expert and telling the judge, "I can't answer that question, because if I said what I really think, Smith here could lose out on a million dollars." Third, scientific knowledge and research within the field may relate minimally to questions of legal interest, In many cases there will be few, if any, studies on neuropsychological tests or procedures that address specific legal questions. The interests and concerns that have occupied those involved in applied neuropsychological research and diagnosis are often very different from the issues and concerns of greatest relevance to the courts. Thus, the clinician may be required to draw broad generalizations from currently existing (and often limited) procedures and scientific knowledge to situations, individuals, and questions that may show only limited resemblance to typical populations and questions of interest. It is this third type of contrast I will now focus upon and illustrate through a series of examples.

Obvious Versus Subtle Cases A considerable proportion of the applied research in clinical neuropsychology involves the accuracy with which tests and methods identify or localize brain damage. However, most of this research has minimal ap-

208

Faust

plication to the legal context. For one, many of these studies involve individuals for whom brain damage has been established with virtual or near certainly by some method other than neuropsychological tests, such as a CT scan. In cases of this type the courts do not need the neuropsychotogist's assistance in identifying brain damage, for when brain damage can be identified definitively or with virtual certainty by other methods, the trier of fact does not need the neuropsychologist to indicate what is already known. In contrast, when the courts need help in determining the presence of brain damage because other methods do not provide definitive evidence, the neuropsychologist is generally reduced to speculation and guesswork. The field has thus far developed limited procedures for even evaluating or determining the accuracy of neuropsychological assessment techniques when other methods cannot identify brain damage with relative certainty, much less an adequate and scientifically established assessment technology for making such determinations. Further, although many studies demonstrate reasonable accuracy in identifying obvious brain damage, which can usually be detected through other methods anyway, various studies show poor accuracy when neuropsychological methods are applied to less obvious or gross cases. The latter might involve cases in which brain damage can still be detected with a high degree of accuracy through other means, such as advanced imaging techniques. Some examples of such studies can be provided. Sweet et al. (1985) administered the Luria-Nebraska Neuropsychological Battery (LNNB) to groups of patients, one of which was comprised of individuals described as having subtle brain damage. The use of various classification rules resulted in the misidentification of from two-thirds to three-fourths of these subjects. The classification rules had achieved higher levels of accuracy with other samples. Klesges et al's. (1985) research involved the Halstead-Reitan Neuropsychological Battery (H-R) and less extreme cases than those common in many other studies with the battery. Accuracy in identifying brain damage exceeded by only 1% the level that would have been obtained by applying the base rates, i.e., disregarding the testing entirely and merely guessing that every subject was brain damaged. Stated differently, the H-R contributed virtually nothing to accurate identification. Golden (1986) noted that neuropsychological assessment methods were developed around clear cases of brain injury. He stated, These are situationswhere patients sufferedmassivestrokes, dementingillnesses, and other disorderswhich are easilydiagnosed,in manycases, simplyby inspection and a conversationwith a patient. In such clear cases the role of these factors is overwhelmedby the brain injuryitself.These techniquesbecomequestionablewhen they are taken fromsuch evaluationsto the more subtle and difficultforensicevaluations where all this informationis neither so striking nor so clear. (p. 12)

Forensic Neuropsychology

209

If the accuracy of neuropsychologicat assessment is dubious in subtle cases in which it is still possible to obtain independent verification of brain damage through other methods, can it be presumed that such assessment is sufficiently accurate for legal purposes in even more subtle cases, or cases in which it is not possible to obtain independent verification of brain damage? There would seem little or no justification for such a presumption, especially in the absence of supportive research, Rather, the accuracy with which brain damage is identified in such cases is uncertain, at best, and subject to considerable doubt. Earlier studies on mild head injury suggesting that long-lasting and serious deficit is common, in contrast to later and better controlled studies, suggesting that good or excellent recovery is typical (e.g., Levin et al., 1987), show that what might be seen as the sensitivity of neuropsychological tests may instead represent false-positive error, that is, the tendency to identify disorder that is not present. One might further consider that litigation involving mild head injury is quite common.

Description and Prediction of Everyday Functioning In many courtroom cases, the key issue is how the individual is functioning, and will function, in everyday life. The implications of a brain injury are obviously much different if it is associated with negligible versus significant impairment in everyday functioning. However, the neuropsychologist's attempt to link tests or observation data to everyday functioning is greatly hampered by the lack of scientific data on the topic. Studies examining the relation between neuropsychological test performance and everyday functioning have yielded, at best, mixed findings (see chap. 4 of Faust et al., 1991). Further, this research is limited in scope, or addresses only a restricted range of neuropsychological batteries or tests. For example, there is a dearth of research on everyday functioning that involves the flexible battery or process-qualitative approaches, which so many practitioners use. Even if one argues that such approaches are not amenable to studies on everyday functioning, which would seem equivalent to maintaining that their properties in this regard must be decided by something other than scientific data, it is still possible to determine whether adherents of these approaches reach accurate judgments or predictions about everyday functioning. In summarizing the state of research on neuropsychological test performance and everyday functioning in 1984, Dodrill and Clemmons stated, "Most comments offered by neuropsychologists concerning the likelihood of successful life performance are based on clinical judgment rather than on empirical criteria established with neuropsychological tests" (p. 520).

210

Faust

They further stated, "Ethical standards and scientific rigor demand that we more specifically anchor our predictions about functioning in life on empirically based studies" (p. 526). And more recently, Guilmette et al. (1990) commented, "Assessment of capacity to work or to perform other everyday tasks of independent living begs for more research on the 'external validity' of our measures and the development of other means to assess such crucial functioning" (p. 390). Examples of research on this topic can also be provided. Dodritl and Ctemmons (1984) analyzed relations between the H-R and MMPI Minnesota Multiphasic Personality Inventory) scores of adolescents with seizure disorder and the functional status of these subjects 3 t o l l years later. This is one of the only studies that has examined predictive, as opposed to concurrent, validity. No significant relations were obtained between the MMPI scores and functional status. For the neuropsychological test scores, predictive procedures were developed through a derivation sample and then validated on the remaining subjects. Most of the H-R tests did not prove useful in predicting either vocational adjustment or independence in living. Even the few variables or tests that showed a statistically significant relation to functioning were of limited practical utility. Use of these variables to predict vocational adjustment and independence in living produced respective improvements of 19% and 4% over the accuracy level expected by chance. Greater improvement over chance level was attained in predicting results on an index of overall adjustment, although as noted by the authors only gross distinctions were possible. For example, it was not possible to make accurate distinctions between those classified on this index as nonfunctioning versus partially functioning. In discussing the results of their study, Dodrill and Clemmons stated, "Among Halstead's tests, a clear or unique pattern of abilities important to ultimate adjustment does not emerge. This, of course, may be because adjustment to life requires a variety of skills, the full nature of which is difficult or impossible to specify with precision" (p. 526). Shelton and Parsons (1987) examined relations between self-reports of everyday functioning, emotional status, and performance on neuropsychological measures. Measures of emotional status showed a number of significant relations with self-reports of everyday functioning, with individuals who obtained scores suggesting greater distress tending to report greater everyday impairment. However, these researchers obtained "minimal or no relations" between performance on the neuropsychologicat tests and selfreports of everyday functioning. Baird et al. (1985) obtained a correlation of .35 between overall performance on a series of neuropsychological tests and self-assessed quality of life among a group of candidates for cerebral

Forensic Neuropsychology

21t

revascutarization. Level of education showed a higher correlation with the quality of life index than did overall neuropsychologicaI test performance. There are some studies, such as those conducted by McSweeny et al. (1985) and Heaton et al. (1978), that suggest stronger associations between neuropsychological test performance and everyday functioning. However, even if one ignored the other negative literature that exists, these studies do not make a particularly strong case for the predictive power of neuropsychological instruments (for a detailed discussion of this point, see Faust et al., 1991). For example, the McSweeny et al. (1985) study involved older individuals with pulmonary problems, and forensic cases are likely to involve different types of individuals in different types of situations. The authors themselves noted that their results required cross-validation with different types of populations. Additionally, most of the tests showed little relation to indices of functioning, and even the stronger predictive relations that were found were of modest size. When the various tests were combined through multiple correlation analyses, the neuropsychological measures still contributed only modestly, or weakly, to the prediction of outcome. Further, cross-validation of the multiple correlations was not performed. Limitations in the Heaton et al. (1978) study include lack of specification as to why the unemployed subjects were unemployed. Thus, one does not know how many who were not working could not work. These comments are not intended to denigrate the worthwhile efforts of these authors, nor to imply that the studies showing negative outcomes are free from methodological flaws. The point is that the studies that yielded more positive outcomes do not provide evidence of a clearly superior quality to the studies that yielded more negative outcomes. The description and prediction of everyday functioning raises immensely difficult and complicated problems, and neuropsychology is still far from an adequate solution (see Faust, in press). Although progress is being made (see Tupper and Cicerone, 1990), current research does not support the proposition that neuropsychological tests or assessment methods permit accurate description of current functioning, nor prediction of future functioning. For most tests and batteries, no research on everyday functioning has been conducted, and certainly such predictive powers cannot be presumed. Clinicians may claim that they are able to overcome limits in the predictive power of tests through various means, such as experience or expansion of the data base. For example, self-reports and observational data may also be obtained. Clinicians may further claim that they have special abilities to "integrate" diverse sources of information, and that this provides an advantage over jury members and leads to accurate judgments that can assist the trier of fact. However, scientific study does not support the value

212

Faust

of experience or these purported data integration abilities, and rather shows that clinicians cannot even begin to approach or manage such complex mental feats (see below). Lacking an established scientific foundation from which to describe or predict everyday functioning, the neuropsychologist's judgments about such matters are based on little more than conjecture. Where there is a lack of evidence to support the accuracy or validity of neuropsychologists' appraisals regarding a particular legal question, it should not be assumed that these determinations can be made accurately. Rather, the burden of proof should be on the one who offers expert testimony to show through proper scientific means that either the method employed or the particular diagnostician achieves a respectable degree of descriptive or predictive validity with regard to the question being asked, The Base-Rate Problem

The term "base rates" simply refers to the frequency with which something occurs. For example, if a disorder occurs in I of 100 individuals, the base rate is 1%. Base rates are critically important in diagnosis and prediction. Lacking knowledge of base rates, one can rarely, if ever, determine whether a diagnostic sign or indicator is valid, or even whether use of a valid sign increases or decreases the accuracy of diagnosis or prediction. First, in order to judge whether a true relation exists between such things as a symptom and a disorder, one must know the frequency (base rate) with which the symptom occurs among those with and without the disorder. For example, although a 6- or 7-point difference between the highest and lowest subtest score on the Wechsler Adult Intelligence Scale --Revised (WAIS-R) might be viewed as a sign of brain damage, research shows that this level of scatter is common among normal individuals (Matarazzo et aL, 1988). Thus, in order to determine whether a valid relation exists between a possible diagnostic sign and a disorder, it is not enough to know how often the two co-occur. One must also know how often the sign occurs among individuals without the disorder. Unless the sign is more common among those with the disorder than those without it, the sign does not bear a valid relation to the disorder. This point applies whether the sign is a behavioral symptom, a score on a neuropsychological test, or a characteristic of interview behavior. Second, even should a diagnostic sign show a valid or demonstrated relation to a disorder or outcome, another form of base-rate information is necessary to determine whether the sign is truly useful in identifying disorder or predicting outcome. One must also know the frequency of the disorder or outcome. Lacking this information, the clinician does not even

Forensic Neuropsychology

213

know whether use of the valid sign leads to more frequent errors in the identification of the disorder than would result were the sign disregarded entirely. Such principles were outlined years ago by Meehl and Rosen (1955), and a number of researches have addressed their application to neuropsychology (e.g., Russell, 1984; Willis, 1984). Various studies within neuropsychotogy demonstrate the first facet of the base-rate p r o b l e m - that test results or symptoms thought to indicate abnormality may be observed with similar frequency among normal individuals. For example, Gouvier et al. (1988) found that various "symptoms" thought to be specific to head-injured individuals were sometimes reported with similar frequency among normal individuals. Clark and Spreen (1983) found that the absence of an ear advantage on dichotic listening tasks, which had been linked to deviancy, was common among normal subjects. Other literature illustrates the second facet of the base-rate problem, that is, for lack of knowledge about the frequency of a condition, one cannot determine the value of a purported diagnostic sign or test. A further corollary is that the evaluation of a diagnostic sign or test requires comparison of its accuracy to that achieved when one founds judgments on the base rates alone (i.e., assumes the condition is always present if the base rate exceeds 50%, or that it is never present if the base rate falls below 50%). For example, Goldstein and Shelly (1984) report on the accuracy of varying tests in identifying brain damage. Among the measures employed, the WAIS produced about a 66% correct classification rate. However, in this study, the base rate for brain damage among the subjects was 68%. Absent this testing, if one played the base rates and assumed that every subject was brain damaged, one would have achieved a 68% hit rate, which slightly exceeds the accuracy achieved with the WAIS. As such, this study does not provide evidence that the WAIS is useful in the identification of brain damage. The point here is not to argue for or against the use of the WAIS in the diagnosis of brain damage, but rather to illustrate the need to compare diagnostic accuracy rates with those achieved using the base rates alone. Clinicians may not realize that the value of diagnostic signs or testing results varies in relation to the base rates of the conditions or outcomes one wishes to identify. Even should the issue be recognized, the base rates for many disorders are not established. Further, base-rate data about the overall or general frequency of clinical conditions are often of limited use, and rather one needs to know the frequencies applicable to one's work setting or the circumstances under which assessment is conducted. As Russell (1984) and Boyd (1982) observed, base rates may vary across different settings or assessment contexts. The frequency of certain conditions among individuals seen for forensic assessment may differ considerably from the

214

Faust

frequency found in other assessment contexts or settings. However, little is known about the base rates for various disorders or conditions among those seen for forensic neuropsychological assessment. Whether one lacks awareness of the base-rate problem or the base rates are unavailable, the result is the same. One will not know whether the signs or symptoms used to identify a disorder are actually useful, or whether they result in more, or even many more, incorrect than correct identifications of the disorder. Stated differently, one does not know the actual value of the data upon which one has relied. A lawyer who is familiar with this issue can go through the "supportive" data on which the clinician based his or her opinion piece by piece, establishing in each instance that for lack of needed base-rate data (or awareness of the problem), the clinician does not really know if use of the "supportive" evidence results in more erroneous than correct identifications. The base-rate problem creates a major methodological stomachache, one that cannot be casually dismissed, and instead requires recognition of the issues and collection of the needed base-rate data. Lack of Standardized Practices There is considerable diversity in approaches to neuropsychological assessment. According to Wasyliw and Golden (1985), variation in neuropsychological assessment methods makes it is nearly impossible to provide a standard description. Indeed, differences in evaluation techniques can be so broad that two neuropsychologists examining the same person may employ tests with absolutely no overlap in specific content. Neuropsychologists may assert, nonetheless, that although their methods differ, they essentially measure the same thing or lead to the same conclusions. Such assertions are doubtful. There have been limited efforts to examine the consistency of results obtained across varying neuropsychological batteries. For example, there has been virtually no research analyzing agreement rates between clinicians who use the flexible battery approach versus the H-R. Looking at this matter in another way, common arguments in the field about the advantages and disadvantages of one or another assessment approach would be nonsensical if we assumed that all approaches yielded the same data or findings. Among the surveys conducted on neuropsychological assessment practices, Guilmette et al.'s (1990) would seem to be the most detailed and broad based. Guilmette et al. tallied results for their sample as a whole, and also for subgroups of practitioners who varied in their involvement with clinical neuropsychology. Certain items addressed the frequency with

Forensic Neuropsycholog~-

215

which respondents used a variety of standard batteries. Describing results for the full sample first, for the H-R, nearly 60% of the respondents indicated that they never use the entire battery, and less than 30% indicated that they use this battery regularly or more often. For the LNNB, over 60% of the respondents indicated that they never use this battery, and less than 20% indicated that they use it regularly or more often. On a related item addressing orientation or approach to neuropsychological assessment, percentages of endorsement were as follows: Process-Qualitative: 41%; H-R: 30%; Fixed Battery: 12%; LNNB: 6%; and Luria-Christensen: 1%. (These figures do not add up to 100% because there was an additional category for other approaches.) It should be recognized, of course, that the most frequently endorsed orientation--the process-qualitative app r o a c h - d o e s not describe a uniform set of procedures. Different adherents are likely to vary, sometimes considerably, the specific tests they use or the way in which they select tests as they proceed with an assessment. Guilmette et al. 's analysis of subgroups, including those most active in neuropsychological assessment, yielded similar endorsement rates on both of these items, or results suggesting considerable diversity. Guilmette et al.'s findings, combined with such survey work as that conducted by McCaffrey et al. (1985), suggest that even when standard batteries are used, practitioners often add other tests to them in a variable or individualized manner. Thus, it appears that many clinicians "invent" or construct their own combination of tests, and may well alter these combinations across examinees. In the great majority of cases, the practitioner wilt not have performed formal scientific research examining the validity of his or her personal combination, and in particular its accuracy in answering questions of legal relevance. As Guilmette et al. 's work and other surveys indicate, the "standard" in neuropsychology, at least if defined by frequency of use or popularity, is that there is no standard overall approach to neuropsychological assessment. No matter what approach or battery a practitioner may use, whether it is one or another set battery, or a flexible or process-qualitative approach, the practitioner is still within a minority, for there is no specific set of measures or tests used by most practitioners. This tremendous diversity likely reflects the ongoing controversies and disputes within neuropsychology, and the lack of agreement about standards or methods of assessment. Again, because the results obtained on neuropsychological assessment can differ depending on the specific measures employed, this creates considerable uncertainty about the extent to which the findings or ultimate conclusions reflect the examinee's status as opposed to the practitioner's choice of measures or assessment strategy.

216

Faust

Golden et al. (1982) have addressed diversity in theory and method within neuropsychology. Responding to a critical commentary on the LNNB, they explained, "Clearly, much of the controversy that appears to exist is based on a divergence in theoretical orientation" (p. 298). They further indicated, "Since many of these differences rest upon basic philosophical differences, they are not likely to be resolved" (p. 298). Guilmette et aL (1990) have also addressed varying approaches to neuropsychological assessment, such as set versus flexible batteries. They too pointed out the lack of scientific underpinnings for the selection of assessment strategies, stating, "Practitioners require much greater scientific guidance in order to know when and how to optimally utilize each approach. Comparative studies in this area would seem to be of utmost importance" (p. 390). Indeed, lacking the needed scientific knowledge, the selection of one approach over another often must be based on other factors or considerations, such as clinical impression, personal philosophy, and biases.

Assessment of Credibility Research shows that individuals can alter their scores on intellectual, neuropsychological, and personality tests (Rogers, 1984, 1988). Although results have not been entirely consistent, there is positive evidence that numerous measures are susceptible to faking. Mensch and Woods (1986), for example, administered the LNNB to 32 subjects on two occasions. On one administration, subjects were instructed to perform as best they could; on a second administration they were instructed to produce results that would suggest brain damage. Across the subscales of the LNNB, subjects produced higher (more abnormal) scores under the "fake-bad" condition. Mensch and Woods further noted that virtually every subject taking the test under the "fake-bad" instructions produced elevations on a sufficient number of scales to suggest the possibility of brain damage. Given the number of tests shown to be susceptible to faking, it would be dubious to presume that individuals cannot alter their responses on tests that have not yet been subjected to such study. Rogers (1984) noted that his overview of psychological tests "underscores the susceptibility of all psychometric approaches to dissimulation. More specifically, neither the ambiguity of the method or its intended goal (i.e., clinical, personality, research, or vocational) prevents deliberate distortion of responses" (p. 98). Altering test results and getting away with it are different matters. Studies that have directly examined neuropsychologists' capacity to detect faking or malingering on a comprehensive neuropsychologicat battery have generally shown poor rates of detection (Heaton et aL, 1978; Faust, Hart,

Forensic Neuropsychoiogy

217

and Guilmette, 1988; Faust, Hart, Guilmette, and Arkes, 1988). For example, in the Heaton et al. study, practitioners performed at chance level to about 20% above chance level when attempting to distinguish the H-R protocols of individuals with genuine neuropsychological dysfunction versus those faking dysfunction. The clinicians' accuracy was not related to their level of experience nor to their confidence in their judgments. Faust et al. 's studies suggest that children and adolescents can alter their scores on neuropsychological tests, and in so doing fool clinicians into diagnosing brain damage. Methodological objections have been raised to these types of studies on the detection of malingering (e.g., Bigler, 1990), to which Faust and colleagues have responded (see Faust and Guilmette, 1990). In any case, it is not as if there is a body of research of superior methodological quality directly showing that clinicians are facile at this detective task, thereby offsetting the doubts created by these negative outcomes. Whatever the shortcomings of clinicians' attempts to detect malingering, there are objective approaches of proven usefulness. In particular, various validity indices have been developed for the MMPI, a number of which have demonstrated utility in the detection of malingering (Greene, 1988; Ziskin and Faust, 1988). Symptom validity testing also shows promise and has gained some empirical support (Binder and Pankratz, 1987; Pankratz, 1979). There also has been a flurry of research activity on the topic of malingering, and in the coming years neuropsychologists may well add techniques of proven value to the MMPI. In the course of these research efforts, it will be important to ensure that methods apply to forensic populations, and further that clinicians follow validated decision rules rather than accepting the outcomes when they agree with their subjective impressions and rejecting them when they do not, which is essentially the same as relying on subjective judgment.

Problems Determining Prior Functioning In order to determine whether an injury or event has produced a toss or decline in functioning, one needs to know how an individual functioned beforehand. As McMahon (1983) noted, "One cannot assess a loss or deficit without knowing the premorbid level of functioning, without knowing what abilities were present before the alleged loss occurred . . . . Whatever the source, such premorbid data are essential--both for comparative purposes to assess loss and for the purpose of formulating a prognosis" (p. 409). Although there is broad agreement on this need, formal methods for assessing prior functioning, when available, are prone to substantial error. When such formal methods are lacking the clinician often must rely upon

218

Faust

impressionistic or subjective approaches, which have been described as even more problematic (Reynolds and Kaufman, 1985). The most extensive efforts to develop methods for determining prior functioning have focused on intellectual ability. One such approach depends on scatter among the Wechsler subtests, or compares what are presumably " h o l d " subtests to " d o n ' t h o l d " subtests. Research on this approach has been mixed, at best. According to Klesges et al. (1985), the resultant estimations are "notoriously unreliable." Another approach found estimates of prior intellectual functioning on a composite of demographic variables shown to correlate with scores on intellectual testing, such as education and socioeconomic status. Various studies show that substantial error is not uncommon with these methods, particularly when they are used with children (see Faust et al., 1991). Silverstein (1987) reviewed a number of the studies on the topic and concluded that some of them showed accuracy levels only slightly greater than chance, and others accuracy no greater than chance. He described the accuracy of methods that depend on demographic variables as "distressingly Iow." Closely related to the "hold"/"don't hold" approach is the "best performance method," in which one uses one or more of the highest scores the individual obtains on intelligence testing or other cognitive measures to estimate prior functioning. It can be easily shown that such an approach is likely to lead to an overestimation, if not a gross overestimation, of prior functioning, the possible result being either a false conclusion that loss has occurred or an overestimation of loss. For example, normal individuals show about a 7-point difference between their highest and lowest subtest scores on the WAIS-R (Matarazzo et aL, 1988). Given this, assume that an individual has a "true" Full Scale IQ of 100. This hypothetical average individual would typically obtain at least one subtest score that falls about half this 7-point spread, or 3 or 4 points, above the subtest mean of 10. If one then prorates Full Scale IQ on the basis of this subtest score, which falls 1-11/3 SD above the mean, the resultant score of 115-120 overestimates the actual Full Scale IQ by t5-20 points. Similarly, research by Alekoumbides et al. (1987) indicates that considerable scatter can be expected across a neuropsychological battery. If one uses the highest score, or few highest scores, the typical result will again be, a marked overestimation of overall neuropsychological or intellectual ability. This marked tendency to overestimate prior functioning was demonstrated directly in a recent study (Mortensen et al., 1991). Thus, it would appear that the best performance method is seriously flawed, and there is a paucity of research to suggest the opposite conclusion, that is, this approach leads to accurate estimates.

Forensic Neuropsychoiogy

/:19

The relative failure to develop accurate methods for estimating prior functioning in the most intensely researched area---overall intellectual functioning-raises substantial doubt about the accuracy of estimations involving less well-researched areas. Obviously, neuropsychological assessments cover a range of abilities, such as memory and executive functions, that go beyond those covered in intellectual testing. In most cases, there is essentially no scientific information on the success achieved in estimating prior functioning in the areas covered on neuropsychological batteries, assuming there are any standard methods for forming these estimates in the first place. Given all of this, it seems doubtful that prior functioning in many or most pertinent areas can be determined with a reasonable degree of accuracy. If results have been unsatisfactory, in the best researched area, what should be presumed about the capacity to determine prior status in other areas that have been minimally researched, if researched at all? It is also commonly asserted that prior personality and behavioral functioning should be assessed, either to determine change in these spheres, or because personality functioning is said to influence the intensity or form of symptoms. Here again, validated methods for assessing prior personality functioning are lacking. (In fact, there is considerable difficulty assessing current personality functioning--see Ziskin and Faust, 1988.) As Gardner (1975) observed, "Any discussion of brain damage must include, as a vital component in the equation, the personality of the individual in the days prior to his injury" (p. 438). And further, "Barring, however, unusually good communication with the patient's family, we generally learn little about this personality, and so must either speculate about it blindly or deny its relevance altogether. It is thus understandable, however regrettable, that many doctors should take the latter course and insist on talking about, say, 'the personality of the frontal patient' as a fixed entity" (pp. 438-439). Even should the clinician interview the family, the value of the information they provide may be of questionable value given concerns raised by authors such as Heaton and Pendleton (1981) and Schwartz (1986) about the objectivity of such sources. These concerns would seem to be accentuated in forensic cases, in which family members may have much to gain or lose by the impressions the neuropsychologist develops. Various authors have summarized the scientific standing of methods for determining prior status. For example, Barons et al. (1984) indicated that most methods for estimating prior functioning "rely heavily upon clinb caI judgment and intuition and have low levels of interrater reliability" (p. 885). Klesges and Troster (1987) observed that there is no well-validated, highly accurate method for assessing prior functioning. They stated in regard to children, "Given the maturational, situational, and sociocultural influences on children's intellectual levels, reliably estimating premorbid

Faust

220

levels is extremely difficult, perhaps even impossible with current technologies" (p. 10). Limits in methods for determining prior intellectual, neuropsychological, and emotional status would seem to highlight the need to obtain concrete evidence about prior functioning (e.g., past school records) and information from collateral sources, including those who do not have a direct investment in the outcome of the case.

Capacity for Complex Data Integration Many neuropsychologists endorse the importance of data integration and the analysis of configural relations among data. For example, Wasyliw and Golden (1985) noted that "without an integration of relevant data regarding the mechanisms of injury, the neuropsychology of the brain, the effects of personality factors, and clinical experience with given procedures, the data from any test procedures can be misleading" (p. 153). Costa (1988) stated, "Clinical neuropsychological formulation always relies on interpretation of pattern, and pattern in turn, colors the interpretation of any given test result" (p. 5). Lezak (1983) indicated, "For the examination to supply answers to many of the diagnostic questions and most of the treatment and planning questions requires integration of all the data--from tests, observations made in the course of the examination, and the history of the problem" (p. 162). Although the need to "integrate all of the data" is emphasized by these and many other neuropsychologists, and although the ability to do so is often taken for granted, it is extremely doubtful that clinicians can perform such cognitive operations. Elsewhere, I have reviewed literature on clinicians' capacities to integrate complex data (e.g., Faust, 1984, 1990; Faust et aL, 1991, chap. 5). The reader can consult these sources for detailed documentation of the points that follow. Evidence for limitations in the ability to manage complex data comes from various sources. One line of research examines clinicians' judgmental accuracy when they are provided with varying amounts of information. This research suggests that once clinicians are provided with a limited amount of valid information, additional information does not lead to a significant gain in judgmental accuracy (see Faust et al., 1991). In fact, accuracy may even decrease. For example, in Wedding's (1983) study, diagnostic errors were as or more common when judgments were based on a complete, versus a partial, H-R battery. Such a result is likely explained by clinicians' difficulties distinguishing between valid and invalid variables. As more and more data are obtained, there will be more and more potentially "bad" data, or data that are not useful for purposes of prediction. If one alters

Forensic Neuropsychology

221

or adjusts judgments founded on better data in accord with less valid or invalid data, judgmental accuracy will decline. Other research shows that it is possible, using mathematical methods, to build models or decision-making formulae that reproduce clinicians' decisions with a high degree of accuracy. Accurate reproduction can be achieved even with models that disregard configural relations among variables. Modeling studies should not be interpreted as showing that clinicians are unable to perform any type of configural analysis. The models are designed to copy clinicians' decisions, not necessarily their underlying reasoning. What the findings do suggest, however, is that clinicians' decisions, even those they believe depend upon configural analysis, can be largely duplicated by procedures that ignore configural relations. These findings create serious doubt that any configuraI analyses that clinicians might perform contribute anything of importance to judgmental accuracy, which in turn raises serious questions about clinicians' ability to perform sophisticated configural analysis. Further strengthening this conclusion is the finding that simple actuarial formulae, which merely add together a few variables, consistently equal or exceed the accuracy of clinicians given access to more extensive data and who claim to perform complex data analysis (see Dawes et al., 1989). (The one study in neuropsychology [Heaton et aL, 1981] sometimes claimed to demonstrate exception to the actuarial advantage does not carry this burden as it compared clinical and automated judgment, the latter containing only one of the two elements that necessarily define an actuarial procedure, the other being conclusions based on empirically established frequencies.) Research also shows that individuals have difficulties deciphering interactive or configural cues among even two or three variables (see Slovic and Lichtenstein, 1971). Taken together, these various lines of research suggest that individuals, clinicians included, have a restricted capacity to manage or use complex data. Although many studies have now been conducted, virtually none of them have demonstrated or uncovered capacities for complex data integration or configural analysis that even begin to approach the level of complexity described and prescribed by clinicians advocating such strategies. Individuals often have difficulty properly integrating even two or three variables, much less the dozens and dozens of variables produced, for example, by a series of neuropsychologicat and personality tests, combined with a clinical interview. Summarizing the scientific research on complex judgment capacity, Faust (1990) stated, Clinicians' purported capacity to integrate numerous variables is not supported by scientific research . . . . To maintain [an alternative position] . , , when so much research points in this direction and so little research suggests otherwise, would

222

Faust

seem an exercise in wishful thinking. The common belief in the capacity to perform complex configural analysis and data integration might thus be appropriately described as a shared professional myth. (p. 478)

False Markers of Accuracy

Neuropsychologists' courtroom opinions often resist the type of objective verification that may be possible in other scientific areas. For example, in a recent courtroom case, a neuropsychologist stated, "I didn't s e n s e that he was malingering." In addition, the judge or jury is likely to tack objective information on the specific expert's judgmental accuracy. As such, the trier of fact is often left with little recourse but to depend on indirect, seemingly reasonable markers of accuracy, such as experts' background training, experience, and credentials, and the confidence they express in their opinions. However, there is a dearth of research showing a positive relation between such variables and accuracy, and for a number of the variables there is direct scientific evidence that suggests they are not related to accuracy (see Ziskin and Faust, 1988). For example, there is a lack of evidence to suggest that diplomates in forensic psychology or clinical neuropsychology are more accurate diagnosticians than those who lack this credential. Additionally, various studies within clinical psychology in general, and a number of studies within neuropsychology specifically, have analyzed the relation between experience and diagnostic or predictive accuracy. This research does not support the assertion that more experienced clinicians reach more accurate conclusions than less experienced clinicians (see Wedding and Faust, 1989). Some studies show no differences between psychologist's and nonpsychologist's success in identifying brain damage on the basis of screening tests (e.g., Blankenhorn and Cerbus, 1975; Goldberg, 1959). Other studies have involved neuropsychological test batteries. Heaton et al. (1978) found no relation between level of experience and the capacity to distinguish neuropsychological protocols of individuals faking deficit versus those with genuine deficit. Wedding (1983) obtained no statistically significant relations between prior experience with the H-R battery among t5 judges and accuracy in determining presence or type of brain damage. Faust, Guilmette, Hart, Arkes, Fishburne, and Davey (1988) found no relations between neuropsychologists' training, experience, and judgmental accuracy, even when analysis was restricted to highly contrasting groups, such as those with a median level of 0 versus 200 hours of face-to-face supervision in neuropsychology or 500 versus 12,000 hours of clinical experience in neuropsychology.

Forensic Neuropsychoiogy

223

Research shows that subjective beliefs about one's judgmental accuracy and objective measures of accuracy often disagree (e.g., Dawes, 1986; Fischhoff, 1982). Various studies demonstrate little or no relation between clinicians' confidence and their judgment accuracy, or even a negative relation, with the more confident judges sometimes being among the least accurate (e.g., Wedding, 1983). Across various studies, the pervasive finding is overconfidence, that is, individuals are more confident in their judgments than is warranted by their actual level of accuracy (e.g., Dawes, 1986; Lichtenstein et al., 1982). For example, Faust, Hart, and Guilmette (1988) found that on a diagnostic task in which no clinician was correct--the detection of faking on neuropsychological tests--many clinicians were nevertheless quite confident in their judgments. This literature on pseudomarkers of accuracy is important in at least two respects. First, to the extent that the judge and jury rely upon factors that seem to be, but are not, related to accuracy, their judgments may be prejudiced. Second, when confronted with negative scientific literature, experts often fall back on their experience or credentials, claiming or implying that these qualifications can be taken as evidence that their judgments are accurate, or that they are exempt from the research findings. However, research on the very factors that they cite, such as experience and confidence, is also negative, and there is no comparable body of scientific research to suggest an alternative conclusion. Thus, claims for accuracy based on such variables as training, experience, or confidence fly in the face of negative literature and are essentially equivalent to arguing that despite the scientific evidence that does exist, one should take the clinician's word that things are different.

A SMALL SAMPLING OF COUNTERARGUMENTS Selective Citing of Positive Literature or Dismissing Negative Literature

Forensic neuropsychologists may attempt to minimize, dismiss, or explain away limits in scientific knowledge or contrary literature. For example, in some areas, the expert may be able to cite some supportive literature. However, if the expert can cite a number of supportive studies, it is quite likely that contrary studies exist on the same topic. Suppose there is a set of studies, some supporting a position and others conflicting with it, and further suppose that there is no clear overall superiority in the methods employed across the supportive and nonsupportive studies. It would seem fairly clear that in such circumstances the issues are controversial and remain unresolved by the profession. Stated differently, the existence of these

224

Faust

negative studies creates substantial doubt, and doubt erodes credibility. Further, if there exists mixed evidence and the scientific community has not yet been unable to solve the issues, how can the trier of fact be expected to do so during the course of a trial? In some cases there is a tendency to dismiss negative literature that does exist, such as that on clinical versus actuarial judgment, for hypothetical evidence, that is, evidence the clinician speculates would materialize were certain studies undertaken. A common version of this argument is that some set of studies has not involved experts. For example, I have heard it argued that research on clinical versus actuarial judgment might (or would!) come out differently if "real" experts were involved (and this despite the fact that a number of these studies involved highly qualified professionals---see Dawes et al., 1989). One could get the impression that what defines an expert, in the eyes of such a critic, is a study in which the clinicians achieve the results the critic thinks they should. In science, hope or belief is not a substitute for evidence, and one conducts research to see how it will turn out. In other cases there is little or no evidence on a particular topic or method. For example, there is very little research linking children's performance on neuropsychological batteries to their everyday functioning. In the absence of positive literature on a method, there is often little basis for presuming that the method works well or produces positive results. Sincere belief in a method cannot substitute for positive data, nor does it serve as a basis for dismissing negative evidence. Fortune-tellers may believe sincerely in their abilities, but I think few forensic psychologists would think it proper were the judge to invite such soothsayers into the court and ask them to predict the plaintiff's future. As Hyman (1977) has so clearly described and as research also shows (see Lichtenstein et al., 1982), individuals, including mental health professionals, often develop inflated beliefs in their powers of judgment. The need for scientific evidence is further bolstered by the frequency with which positive beliefs or conjectures have been disproven, once subjected to scientific scrutiny. As a result, we no longer swing the mentally disordered from wicker baskets suspended from the ceiling, drill holes in their skulls, or inject them with malaria. The Chapmans' (1967, 1969) seminal research on illusory correlation shows the ease with which false beliefs are formed and, once formed, resist counterevidence. Various topics covered in the more general literature on clinical psychology and clinical judgment have not been adequately studied within neuropsychology, and therefore generalization to neuropsychology has not been established. However, this provides little basis for the proposition that the neuropsychologist is free from the problems or limits demonstrated in other areas. Generalization has in fact been shown in a number of areas

Forensic Neuropsychology

225

(see Wedding and Faust, 1989), and at best, where there is a tack of research examining generalization, the issue is unresolved. Lack of evidence in either direction for a proposition should not be misconstrued as positive evidence for that proposition.

Purported Discontinuity Between Psychology and Clinica~ Neuropsychology The neuropsychologist may attempt to separate his or her specialty area from the flaws and limits of general clinical psychology. It may be claimed or suggested that the neuropsychologist, or neuropsychotogy, is not subject to the same problems and limits as clinical psychology in general and somehow stands above it. Neuropsychology, although possessing distinctive features, is still a subarea or branch of clinical psychology and shares many common features. In many instances, in fact, the neuropsychotogist and general clinical practitioner use the exact same methods. As do other clinical psychologists, the neuropsychologist often conducts interviews. As do other clinical psychologists, neuropsychotogists often administer and interpret personality tests. Such activities may be described as essential components of neuropsychological evaluation. That it happens to be a neuropsychologist who conducts an interview and administers personatity tests does not alter their problematic features. The neuropsychologist is no less restricted by the limits of interview techniques and personality tests than is the general clinical practitioner (for a review of such methods in the context of legal assessment, see Ziskin and Faust, 1988). The neuropsychologist may concede the above points but still assert that the difference lies in the use of neuropsychological tests, or tests specifically designed to assess brain functions or intellectual/cognitive abilities° Reynolds (1982), however, has disabused this notion. When discussing neuropsychological tests, he stated, "There is little that can be said to be psychologically or psychometrically unique about any of these tests. The), are more or less similar to tests that psychologists have been using for the last 60 or so years" (p. 76). Golden (1986) and Russell (1987) also discuss the psychometric limits of neuropsychological tests.

Acceptance Acceptance within the scientific community, or by the court, is often cited to support forensic activities in clinical neuropsychology. Acceptance within the profession should not necessarily be equated with "scientifically sound" or "established." First, groups of individuals can share incorrect

226

Faust

beliefs (again, see the Chapmans' 1967 and 1969 research on illusory correlation). It is easy to cite instances in which practitioners generally accepted some belief that was not scientifically supported and that later turned out to be false, such as belief in the broad curative power of bleeding individuals with leeches. A major function of science is to test belief and uncover error, and the frequency with which science has corrected mistaken or commonsense notions attests to its need and usefulness in this regard. As Reynolds (1986) stated, "Clinical mythology is easily developed and dies a hard death" (p. 127). He noted further that although results on a psychological test may appear to make good sense, "perhaps therein lies its greatest danger. Intuitive appeal, clinical acumen, and perceived utility are just not sufficient; they must be combined with good, sound empirical research" (p. 135). The other argument is that "the courts find psychological (or neuropsychological) evidence useful." Typically, the evidence that the courts find it useful is that they admit such testimony. Such an argument would seem potentially incompatible with efforts to set professional standards for courtroom work. If the court's admission of the evidence is to be taken as proof that it is useful, how can one also argue that according to some other standard the psychologist's work or testimony was substandard or lacked merit? Beyond this, the argument is commonly founded on circular grounds, as it has often been psychologists and psychiatrists who first maintained that their services were helpful to the courts. The courts have sometimes accepted these arguments, with forensic experts then turning around and suggesting that such acceptance proves that the evidence is useful. Neither professional acceptance, or the court's acceptance, should be confused with scientifically demonstrated or proven.

"IF YOU DON'T HAVE ANYTHING BETTER TO SUGGEST . . ." Traveling at perhaps too fast a speed, a motorist approaches a canyon crossing on a foggy night. A pedestrian spots him, waves his arms frantically, and shouts as loudly as he can, "Stop!" The bridge is out!" The driver screeches to a halt, rolls down his window, and says, "If you don't have anything better to suggest, I'd suggest that you not say anything." With that he hits the accelerator, races forward, and then down. One of the reviewers stated that it is an empty exercise and a great disservice to criticize without providing alternative directions. I do not understand the view that one should remain silent unless one can suggest some affirmative action (a position that would go a long way toward placing Sir Karl Popper on the intellectual junk heap), although I fully share the

Forensic Neuropsychoiog7

227

desire that something better be developed. Were one told by an informed source that one's impending brain surgery had zero chance of success, would one respond, "It is wrong to criticize the doctor unless you can think of something better to do," and then lie back down on the table. Some of the problems that plague forensic neuropsychology could be corrected by exercising greater care in assessment activities and by adhering more closely to the scientific literature. For example, problems created by the failure to follow standard assessment methods could often be easily solved if evaluators were not so lax about procedures of administration. I have tried to contribute something to the solution of such problems by writing at length about strategies for improving clinical judgment (e.g., Faust, 1986; Faust and Nurcombe, 1989; Wedding and Faust, 1989), and by presenting to various groups of psychologists and neuropsychologists on approaches for improving legal evaluations. In my" current research, I am also trying to develop new methods for decreasing judgmental biases and increasing judgmental accuracy. In other areas, problems stem primarily from gaps in the scientific knowledge base, and they resist quick or easy solutions--for now the needed bridges may not exist or may require more than a little additional work. Although perceived limitations in the forensic area or in other areas of practice mac differ, all responsible psychologists have refused requests they believed were unreasonable or impossible to fulfill; the choice to not act can be a perfectly honorable one. One's boundaries may be more elastic in the clinical context, particularly when someone is terribly distressed and knowingly accepts the stated odds of improvement and any associated treatment risks. The forensic context is different, for one's obligation is to other than the client and one presumably would not undertake an assessment if one thought there was minimal chance of fulfilling one's task objectives (which, in theory, is to reach accurate conclusions). None of this is to say we should not be proud of the progress we are making, nor that we should happily accept the limitations that remain. However, it is sometimes most sensible and constructive to focus on foundational activities before proceeding further, a task that many neuropsychologists are busily engaged in and that no doubt will create new avenues of travel.

ACKNOWLEDGMENTS In a number of places in the article I address comments to anonymous reviewers, whom I would like to thank for their efforts and thoughtful comments. I would also like to thank Dr. Tony Puente for his patience and

228

Faust

input, and Drs. Robyn Dawes, Lew Goldberg, and Jay Ziskin for reviewing an earlier version of the manuscript and offering many helpful suggestions. REFERENCES Alekoumbides, A., Charter, R. A., Adkins, T. G., and Seaeat, G. F. (1987). The diagnosis of brain damage by the WAIS, WMS, and Reitan Battery utilizing standardized scores corrected for age and education. The International Journal of Clinical Neuropsychology 9: 11-28. Baird, A. D., Adams, K. M., Ausman, J. I., and Diaz, F. G. (1985). Medical, neuropsychological, and quality-of-life correlates of cerebrovascular disease. Rehabilitation Psychology 30: 145-155. Barona, A., Reynolds, C. R., and Chastain, R. (1984). A demographically based index of premorbid intelligence for the WAIS-R. Journal of Consulting and Clinical Psychology 52: 885-887. Bigler, E. D. (1990). Neuropsyehology and malingering: Comment on Faust, Hart, and Guitmette (1988). Journal of Consulting and Clinical Psychology 58: 244-247. Binder, L. M., and Pankratz, L. (1987). Neuropsychologieat evidence of a factitious memory complaint. Journal of Clinical and Experimental Neuropsychology 9: 167-171. Blankenhorn, A., and Cerbus, G. (1975). Clinical and actuarial evaluation of organic brain damage by psychologists and non-psychologists using the Memory for Designs. Perceptual and Motor Skills 40: 99-102. Boyd, J. L. (1982). Reply to Woodward. Journal of Consulting and Clinical Psychology 50: 289-290. Chapman, L. J., and Chapman, J. P. (1967). Genesis of popular but erroneous psychodiagnostic observations. Journal of Abnormal Psychology 72: 193-204. Chapman, L. J., and Chapman, J. P, (1969). Illusory correlation as an obstacle to the use of valid psychodiagnostic signs. Journal of Abnormal Psychology 74: 271-280. Clark, C. M., and Spreen, O. (1983). Psychometric properties of dichotic words tests. Journal of Clinical Neuropsychology 5: 169-179. Costa, L. (1988). Clinical neuropsychology: Prospects and problems. The Clinical Neuropsychologist 2: 3-11. Dawes, R. M. (t986). Representative thinking in clinical judgment. Clinical Psychology Review 6: 425-441. Dawes, R. M., Faust, D., and Meehl, P. E. (1989). Clinical versus actuarial judgment. Science 243: 1668-1674. Dodrill, C. B., and Clemmons, D. (1984). Use of neuropsychological tests to identify high school students with epilepsy who later demonstrate inadequate performances in life. Journal of Consulting and Clinical Psychology 52: 520-527. Faust, D. (1984). The Limits of Scientific Reasoning, University of Minnesota Press, Minneapolis. Faust, D. (1986). Research on human judgment and its application to clinical practice. Professional Psychology: Research and Practice 17: 420-430. Faust, D. (1990). Data integration in legal evaluations: Can clinicians deliver on their premises? Behavioral Sciences and the Law 7: 469-483. Faust, D. (in press). The use of neuropsychological tests to describe and prescribe: Why polishing the crystal ball won't help. In Glueckauf, R., Sechrest, L., Bond, G., and McDonel, B. (eds.), Improving the Quality of Assessment Practices in Rehabilitation Psychology, Sage, London. Faust, D., and Guilmette, T. J. (1990). To say it's not so doesn't prove that it isn't. Research on the detection of malingering. Reply to Bigler. Journal of Consulting and Clinical Psychology 58: 248-250.

Forensic Neuropsychology

229

Faust, D., Guilmette, T. J., Hart, K., Arkes, H. R., Fishburne, F. J., and Davey, L. (1988). Nenropsychologists' training, experience, and judgment accuracy. Archives of Clinicat Neurop~vhology 3: 145-163. Faust, D., Hart, K., and Guilmette, T. J. (1988). Pediatric malingering: The capacity of children to fake believable deficits on neuropsychological testing. Journal of Consulting and Clinical Psychology 56: 578-582. Faust, D., Hart, K., Guilmette, T. J., and Arkes, H. R. (i988). Neuropsychologists' capacity to detect adolescent malingerers. Professional Psychology: Research and Practice 19: 508515. Faust, D., and Nurcombe, B. (1989). Improving the accuracy of clinical judgment. Psychiatry 52: 197-208. Faust, D., and Ziskin, J. (1988). The expert witness in psychology and psychiatry. Science 241: 31-35. Faust, D., Ziskin, J., and Hiers, J. B. (1991). Brain Damage Claims: Coping with Neurop.~ychological Evidence (Vols. 1 and 2), Law and Psychology Press, Los Angeles. Fischhoff, B. (1982). Debiasing. In Kahneman, D., Slovic, P., and Tversky, A. (eds.), Judgment Under Uncertainty."Heur~tics and Biases, Cambridge University Press, New York, pp. 422444. Gardner, H. (1975). The Shattered Mind: The Person After Brain Damage, Knopf, New York. Goldberg, L. R. (1959). The effectiveness of clinicians' judgments: The diagnosis of organic brain damage from the Bender-Gestalt Test. Journal of Consulting Psychology 23: 28-33. Golden, C. J. (1986). Forensic neuropsychology: Introduction and overview, in Golden, C. J., and Strider, M. A. (eds.), Forensic Neuropsychology, Plenum, New York, pp. 1-47. Golden, C. J., Ariel, R. N., McKay, S. E., Wilkening, G. N., Wolf, B. A., and Maclnnes, W. D. (1982). The Luria-Nebraska Neuropsychological Battery: Theoretical orientation and comment. Journal of Consulting and Clinical Psychology 50:291-300. Goldstein, G., and Shelly, C. (1984). Discriminative validity of various intelligence and neuropsychological tests, Journal of Consulting and Clin&al Psychology 52: 383-389. Gouvier, W. D., Uddo-Crane, M., and Brown, L. M. (1988). Base rates for post-concussional symptoms. Archives of Clinical Neuropsychology 3: 273-278. Greene, R. L. (1988). Assessment of malingering and defensiveness by objective personality inventories. In Rogers, R. (ed.), Clinical Assessment of Malingeffng and Deception, Guilford, New York, pp. 123-158. Guilmette, T. J., Faust, D., Hart, K., and Arkes, H. R. (1990). A national survey of psychologists who offer neuropsychological services. Archives of Clinical Neuropsychotogy 5: 373392. Heaton, R. K., Chelune, G. J., and Lehman, R. A. W. (1978). Using neuropsychologicat and personality tests to assess the likelihood of patient employment. The Journal of Nervous and Mental Disease 166: 408-416. Heaton, R. K., Grant, I., Anthony, W. Z., and Lehman, A. W. (1981). A comparison of clinical and automated interpretation of the Halstead-Reitan Battery. JoutTaal of Clinical Neuropsychology 3: 121-141. Heaton, R. K., and Pendleton, M. G. (1981). Use of neuropsyehological tests to predict adult patients' everyday functioning. Journal of Consulting and Clinical Psychology 49: 807-821. Heaton, R. K., Smith, H. H., Jr., Lehman, R. A , & Vogt, A. T. (1978). Prospects for faking believable deficits on neuropsychological testing. Journal of Consulting and Clinical Psychology 46; 892-900. Hyman, R. (1977). "Cold reading": How to convince strangers you know all about them. Tire Zetetic, 18-37. Klesges, R. C., Fisher, L., Vasey, M., and Pheley, A. (1985). Predicting adult premorbid functioning levels: Another look~ The International Journal of Clh~icai Neuropsychology 7: 1-3. Klesges, R. C., and Troster, A. I. (1987). A review of premorbid indices of intellectual and neuropsychological functioning: What have we learned in the past five years? The International Journal of Clin&al Neuropsychology 9: 1-11.

230

Faust

Levin, H. S., Mattis, S., Ruff, R. M., Eisenberg, H. M., Marshall, L. F., Tabaddor, K., High, W. M., Jr., and Frankowski, R. F. (1987). Neurobehavioral outcome following minor head injury: A three-center study. Journal of Neurosurgery 66: 234-243. Lezak, M. D. (1983). Neuropsychological Assessment (2nd ed.). Oxford University Press, New York. Liehtenstein, S., Fischhoff, B., and Phillips, L. D. (1982). Calibration of probabilities: The state of the art to 1980. In Kahneman, D., Slovic, P., and Tversky, A. (eds.), Judgment Under Uncertainty:Heuristicsand Biases, Cambridge University Press, New York, pp. 306-334. Matarazzo, J. D., Daniel, M. H., Prifitera, A., and Herman, D. O. (1988). Inter-subtest scatter in the WAIS-R standardization sample. Journal of Clinical Psychology 44: 940-950. McCaffrey, R. J., Malloy, P. F., and Brief, D. J. (1985). Internship opportunities in clinical neuropsychology emphasizing recent INS training certificate. Professional Psychology: Research and Practice 16: 236-252. McMahon, E. A. (1983). Forensic issues in clinical neuropsychology. In Golden, C. J., and Vicente, P. J. (eds.), Foundations of Clinical Neuropsychology, Plenum Press, New York, pp. 401-427. McSweeny, A. J., Grant, I., Heaton, R. K., Prigatano, G. P., and Adams, K. M. (1985). Relationship of neuropsychological status to everyday functioning in healthy and chronically ill persons. Journal of Clinical and Experimental Neuropsychology 7: 281-291. Meehl, P. E., and Rosen, A. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin 52: 194-216. Mensch, A. J., and Woods, D. J. (1986). Patterns of feigning brain damage on the LNNB. The International Journal of Clinical Neuropsychology 8: 59-63. Mortensen, E. L., Gade, A., and Reinisch, J. M. (1991). A critical note on Lezak's 'best performance method' in clinical neuropsychology. Journal of Clinical and Experimental Neuropsychology 13: 361-371. Pankratz, L. (1979). Symptom validity testing and symptom retraining: Procedures for the assessment and treatment of functional sensory deficits. Journal of Consulting and Clinical Psychology 47: 409-410. Reitan, R. M. (1986). Theoretical and methodological bases of the Halstead-Reitan Neuropsychological Battery. In Grant, G., and Adams, K. M. (eds.), Neuropsychological Assessment of Neuropsychiatric Disorders, Oxford University Press, New York, pp. 3-29. Reynolds, C. F. (1982). Neuropsychological assessment in education: A caution. Journal of Research and Development in Education 15: 76-79. Reynolds, C. R. (1986). Clinical acumen but psychometric naivete in neuropsychological assessment of educational disorders. Archives of Clinical Neuropsychology 1: 121-137. Reynolds, C. R., and Kaufman, A. S. (1985). Clinical assessment of children's intelligence with the Wechsler Scales. In Wolman, B. B. (ed.), Handbook of Intelligence, Wiley, New York, pp. 601-661. Rogers, R. (1984). Towards an empirical model of malingering and deception. Behavioral Sciences and the Law 2: 93-111. Rogers, R. (1988). Current status of clinical methods. In Rogers, R. (ed.), Clinical Assessment of Malingering and Deception, Guilford, New York, pp. 293-308. Rogers, R., and Cavanaugh, J. L. (1983). "Nothing but the truth" . . . A re-examination of malingering. Journal of Law and Psychiatry, 11: 443-460. Russell, E. W. (1984). Theory and development of pattern analysis methods related to the Halstead-Reitan Battery. In Logue, P. E., and Schear, J. M. (eds.), Clinical Neuropsychology. A Multidisciplinary Approach, Thomas, Springfield, IL, pp. 50-98. Russell, E. W. (1987). A reference scale method for constructing neuropsychological test batteries. Journal of Clinical and Experimental Neuropsychology 9: 376-392. Schwartz, M. L. (1986). Book review [Closed Head Injury: Psychological, Social and Family Consequences. Brooks, N. (ed.), Oxford Press, New York, 1984.] Journal of Clinical and Experimental Neuropsychology 8: 617-621. Shelton, M. D., and Parsons, O. A. (1987). Alcoholics' self-assessment of their neuropsychological functioning in everyday life. Journal of Clinical Psychology 43: 395-403.

Forensic Neuropsychology

231

Silverstein, A. B. (1987). Accuracy of estimates of premorbid intelligence based on demographic variables. Journal of Clinical Psychology 43: 493-495. Slovic, P., and Lichtenstein, S. C. (1971). Comparison of Bayesian and regression approaches to the study of information processing in judgment. Organizational Behavior and Human Performance 6: 649-744. Sweet, J. J., Osmon, D. C., Rozensky, R. H., and Tovian, S. M. (t985). Comparison of the decision-tree and standard methods of the Luria-Nebraska Neuropsychological Battery. Journal of Consulting and Clinical Psychology 53: 185-188. Tupper, D. E., and Cicerone, K. D. (eds.). (1990). The Neuropsychotogy of Everyday Life: Assessment and Basic Competencies, Kluwer Academic Publishers, Boston. Wasyliw, O. E., and Golden, C. J. (1985). Neuropsychological evaluation in the assessment of personal injury. Behavioral Sciences and the Law 3: 149-164. Wedding, D. (1983). Clinical and statistical prediction in neuropsychology. Clinical Neuropsychology 5: 49-55. Wedding, D., and Faust, D. (1989). Clinical judgment and decision making in neuropsychotogy. Archives of Clinical Neuropsychology 4: 233-265. Willis, W. G. (1984). Reanalysis of an actuarial approach to neuropsychological diagnosis in consideration of base rates. Journal of Consulting and Clinical Psychology 52: 567-569. Ziskin, J. (1981). Coping with Psychiatric' and Psychological Testimony (Vols. 1 and 2, 3rd ed.), Law and Psychology Press, Los Angeles. Ziskin, J., and Faust, D. (1988). Coping with Psychiatric and Psychological Testimony (Vols. 1-3, 4th ed.). Law and Psychology Press, Los Angeles.