Multiple Sclerosis and Related Disorders 1 (2012) 196–201
Contents lists available at SciVerse ScienceDirect
Multiple Sclerosis and Related Disorders journal homepage: www.elsevier.com/locate/msard
Review
Computerized cognitive testing for patients with multiple sclerosis Helen Lapshin a,c,n, Paul O’Connor b,c, Krista L. Lanctˆot a,c, Anthony Feinstein a,c a b c
Sunnybrook Health Sciences Centre, 2075 Bayview Avenue, Toronto, ON, Canada M4N 3M5 St. Michael’s Hospital, 30 Bond Street, Toronto, ON, Canada M5B 1W8 University of Toronto, 27 King’s College Circle, Toronto, ON, Canada M5S 1A1
a r t i c l e i n f o
a b s t r a c t
Article history: Received 15 February 2012 Received in revised form 27 April 2012 Accepted 1 May 2012
Cognitive dysfunction affects 40–65% of multiple sclerosis (MS) patients. To date, conventional paper and pencil neuropsychological testing has been the method of choice for detecting deficits. However, the expense and lack of access to these tests make it difficult for many patients to obtain an assessment. With the introduction of computerized cognitive testing, certain barriers to assessment can be reduced. This paper critically reviews the currently available computerized batteries, including the Automated Neuropsychology Assessment Matrix (ANAM), the Mindstreams Computerized Cognitive Battery (MCCB), the Amsterdam Neuropsychological Tasks (ANT), the Cognitive Stability Index (CSI), and the Cognitive Drug Research (CDR) battery. Recent developments in this field show promise, although further research is required before this approach can be more widely implemented. & 2012 Elsevier B.V. All rights reserved.
Keywords: Multiple sclerosis Neuropsychological testing Computerized cognitive assessment Cognition Automated Neuropsychology Assessment Matrix Mindstreams Computerized Cognitive Battery
Contents 1. 2.
3. 4. 5.
6.
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 Detecting cognitive dysfunction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 2.1. Multiple Sclerosis Screening Questionnaire (MSNQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 2.2. Brief batteries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 2.3. Minimal Assessment of Cognitive Function in MS (MACFIMS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Limited access to assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 Computerized tests of cognitive function in MS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 5.1. Automated Neuropsychology Assessment Matrix (ANAM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 5.2. Mindstreams Computerized Cognitive Battery (MCCB) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 5.3. Amsterdam Neuropsychological Tasks (ANT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 5.4. Cognitive Stability Index (CSI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 5.5. Cognitive Drug Research (CDR) battery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 5.6. Miscellaneous tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Disclosure statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Role of the funding source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
1. Introduction n Corresponding author at: Sunnybrook Health Sciences Centre, 2075 Bayview Avenue, Toronto, ON, Canada M4N 3M5. Tel.: þ 1 416 480 6100x7626. E-mail address: [email protected] (H. Lapshin).
2211-0348/$ - see front matter & 2012 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.msard.2012.05.001
This review paper addresses the clinically relevant question of barriers to neuropsychological testing in those with multiple sclerosis (MS) and explores whether computerized approaches
H. Lapshin et al. / Multiple Sclerosis and Related Disorders 1 (2012) 196–201
may prove helpful where conventional testing is inaccessible. The literature devoted to computerized testing in MS is critically reviewed following which suggestions are made on how to build on promising early work. Cognitive dysfunction affects approximately 40–65% of patients with multiple sclerosis (MS) (Rao et al., 1991a). The most common types of deficits involve slowed information processing speed and impaired memory (Benedict et al., 2002; Brassington and Marsh, 1998). The latter includes problems related to working, episodic and semantic memory (Beatty et al., 1988; Benedict et al., 2003, 2006; Grafman et al., 1991; Thornton and Raz, 1997). Both acquisition and recall are affected (Thornton and Raz, 1997; Thornton et al., 2002). Processing speed deficits are evident in tests such as the Paced Auditory Serial Addition Test (PASAT) which has an impairment rate of approximately 27%, and the Symbol Digit Modalities Test which approximately 52% of patients fail (Benedict et al., 2006). Executive dysfunction is also common with difficulties in planning, concept formation and abstract reasoning reported (Foong et al., 1997, 1999; Heaton et al., 1985). Data show that impairment on the Delis–Kaplan Executive Function System Sorting Test occurs 15–25% of the time (Benedict et al., 2006). Visuospatial and verbal fluency deficits may also be present although less frequently (Achiron et al., 2005; Basso et al., 1996; Benedict and Zivadinov, 2011; Huber et al., 1987). Results for the Judgement of Line Orientation Test show impairment in approximately 22% of patients, and scores on the Controlled Oral Word Association Test demonstrate an impairment rate of approximately 13% (Benedict et al., 2006). Cognitive dysfunction has an adverse effect on the quality of life of afflicted MS patients. Impaired individuals have greater difficulties with work, social life, and activities of daily living (Rao et al., 1991b). In addition, it can negatively affect driving ability (Schultheis et al., 2001).
197
Anxiety and certain personality attributes such as conscientiousness and neuroticism have also been found to influence patients’ responses in the MSNQ introducing a note of caution when interpreting findings (Akbar et al., 2011b). An attempt at validating an internet version of the measure in order to expand its utility has been unsuccessful (Akbar et al., 2010). Possible reasons for this were the use of a biased sample, the high rates of depression in the sample selected and the unreliability of collecting data in the absence of a test administrator or the supervision of a research assistant. 2.2. Brief batteries Several brief screening batteries have been implemented for use with MS patients, the most often cited being the Neuropsychological Screening Battery for Multiple Sclerosis (NPSBMS) (Rao et al., 1991a). This assessment takes approximately 30 min to administer and consists of four tests: the Selective Reminding Test which is a verbal learning test, 7/24 Spatial Recall Test, a spatial learning test, Paced Auditory Serial Addition Test which measures speed of information processing and working memory, and the Controlled Oral Word Association Test, a word fluency and retrieval task dependent on speed of information processing. A variant of this battery, the Rao Brief Repeatable Battery of Neuropsychological Tests (BRB-N), includes these 4 tests (the 7/24 replaced by the 10/36) and the Symbol Digit Modalities Test added as another measure of information processing speed (Rao, 1990). Fifteen alternate versions of the tests have been developed for serial testing, with the exception of the PASAT, which has 2 alternate forms. Brief batteries are relatively quick to administer but require trained personnel, which limits their use in clinical settings. 2.3. Minimal Assessment of Cognitive Function in MS (MACFIMS)
2. Detecting cognitive dysfunction Given the implications of cognitive dysfunction for patients’ lives, detection is important. Although there is limited evidence to support the efficacy of neuropsychological rehabilitation, promising data show that cognitive training could facilitate memory ¨ ¨ al ¨ ainen, ¨ improvement (Rosti-Otajarvi and Ham 2011; Chiaravalloti et al., 2005). Detecting cognitive impairment not only allows clinicians to address these difficulties in the form of rehabilitation or compensatory techniques (Johnson et al., 2009), but also helps researchers to understand the etiology of cognitive dysfunction which is fundamental to the advancement of new treatments. A number of approaches have been developed for this detecting cognitive dysfunction: (1) the self-report method, for example the Multiple Sclerosis Neuropsychological Screening Questionnaire (MSNQ) (Benedict et al., 2003), (2) screening batteries such as the Brief Repeatable Battery (BRB-N) (Rao, 1990), and (3) a more comprehensive evaluation, such as the Minimal Assessment of Cognitive Function in MS (MACFIMS) (Benedict et al., 2002).
The Minimal Assessment of Cognitive Function in MS (MACFIMS) is a comprehensive cognitive evaluation validated for use in patients with MS (Benedict et al., 2002). The battery takes approximately 90 min to administer and the tests were chosen by consensus by a group of leading neuropsychologists and clinical psychologists who specialize in the field of cognitive dysfunction associated with MS. The battery comprises seven tests which encompass five cognitive domains often impaired in MS: processing speed and working memory, learning and memory, executive function, visuospatial processing and verbal fluency and retrieval. All tests chosen are known to be sensitive to the kinds of cognitive deficits experienced by MS patients. The authors recommend the battery be accompanied by measures of pre-morbid IQ and depression. Although this battery is a valid and reliable measure of cognitive dysfunction in patients with MS, it requires a significant amount of time and expertise to administer, once again placing it beyond the capabilities of most MS clinics. The data also require neuropsychological expertise to score and interpret.
2.1. Multiple Sclerosis Screening Questionnaire (MSNQ) 3. Limited access to assessment The Multiple Sclerosis Neuropsychological Screening Questionnaire (MSNQ) is a five minute, fifteen item, self-administered screening questionnaire that includes two separate forms: one for the patient and another for an informant. Data reveal that the patient form correlates more strongly with depression whereas the informant report more closely reflects patients’ performance on objective neuropsychological tests (Benedict et al., 2003). The informant MSNQ has a reported sensitivity and specificity of 0.83 and 0.97, respectively, for detecting cognitive dysfunction.
There are many obstacles to obtaining a cognitive assessment from a neuropsychologist. Referrals are frequently made by a neurologist but almost one in three MS patients do not see a neurologist on a regular basis (Minden et al., 2008). Data from the United States show that those who do not have regular neurology contact are more likely to have one or more of the following characteristics: be poor, have no health insurance, live in rural areas, be African American, have a disease duration of more than
198
H. Lapshin et al. / Multiple Sclerosis and Related Disorders 1 (2012) 196–201
15 years, have difficulty walking but are not using an assistive device, rely on a wheelchair or scooter, or be bedridden. Conversely, female patients and those who have experienced a relapse in the past year see a neurologist more often (Minden et al., 2008). Adding to the challenge of obtaining an assessment is a paucity of neuropsychologists. Canadian data from the last decade reveal that there were only 230 neuropsychologists in the country with only a small number engaged in MS research. How many neuropsychologists assess MS patients for clinical purposes is not known (Hayman-Abello et al., 2003). The result of these many barriers is that cognitive testing is an important component of MS management that many patients never receive.
reaction time which measures the basic speed of response to a stimulus, choice reaction whereby the time it takes a participant to press one of two keys on the keyboard depending on which stimulus is presented on the screen is measured, and semantic reaction time where a stimulus is presented on the screen and the participant is asked to decide whether it belongs to a certain semantic category (Reicker et al., 2007). Results showed that all three test modalities were successful in distinguishing MS patients from healthy controls. This result has been replicated by another study that also focused on indices of attention and reaction time (De Sonneville et al., 2002). Apart from tests of information processing speed, one computerized index of executive function has been used in MS research. The finding that patients were impaired when compared to controls was difficult to interpret, however, as the test had a significant motor component (Foong et al., 1997).
4. Methods 5.1. Automated Neuropsychology Assessment Matrix (ANAM) The literature review was conducted with Medline, using the following keywords: (1) (multiple sclerosis) & (neuropsychological tests/neuropsychological testing) & (computerized or internet), and (2) (multiple sclerosis) & (cognition/cognitive function) & (computerized or Internet).
5. Computerized tests of cognitive function in MS Computerized cognitive assessment provides a potential solution to some of the problems identified above. Administration and scoring do not require the expertise of a neuropsychologist which saves both time and money. In addition, computerized testing has good reliability and provides exact measures of response times in tests of speed of information processing (Cook et al., 2009; Wilken et al., 2003). There are, however, disadvantages associated with this process. Patients may not be fully computer literate which may deter them from taking these tests. Impairments such as muscle weakness and poor coordination may hinder or prevent testing. Finally, it may prove challenging to assess free recall and other tests requiring verbal responses without the assistance of a test administrator. Several studies have used single computerized tests to supplement conventional paper and pencil testing. In this instance, the term ‘‘paper and pencil’’ testing refers to tests in which the patient is not required to use a computer (tests that are scored on, but are not administered by, the computer are regarded as ‘‘paper and pencil’’ tests for the purposes of this paper). Single computerized tests have included the STROOP (Macniven et al., 2008), a computerized version of the PASAT (Lengenfelder et al., 2006) and the SDMT (Akbar et al., 2011a). The latter study deserves further comment for there is an emerging consensus that the paper and pencil version of the SDMT, may be the most sensitive and specific cognitive test for MS patient. This has led researchers to suggest that it replace the three second PASAT as the single cognitive index in the MS Functional Composite (MSFC) (Brochet et al., 2008; Drake et al., 2010). In the computerized SDMT (SDMT-c) study significant differences were found between 119 MS patient and 38 healthy controls. When the SDMT-c alone was compared to the BRB-N, the gold standard in this study, a sensitivity and a specificity of 71% and 84%, respectively, was found with respect to predicting cognitive dysfunction (Akbar et al., 2011a). This result overlaps with that obtained from another study that focused on computerized indices of motor programming and automatic and controlled processing (Kujala et al., 1994). Another study explored the efficacy of the Computerized Tests of Information Processing (CTIP) which consists of three reaction time tests: simple
Several computerized cognitive batteries have been employed with MS patients. The Automated Neuropsychology Assessment Matrix (ANAM) is a library of computerized tests created by the Department of Defense in the United States. The ANAM has been used as a screening tool for cognitive impairment in a variety of populations including patients with systemic lupus erythematosus, Parkinson’s disease, Alzheimer’s disease, brain injury, migraines, and multiple sclerosis (Kane et al., 2007). The goal of a recent study was to validate a select number of ANAM tests for use with MS patients (Wilken et al., 2003). Fifty relapsing-remitting MS patients were assessed with the ANAM battery, the results of which correlated with a battery of neuropsychological tests and distinguished patients from healthy controls as well as cognitively impaired from cognitively intact patients. The ANAM battery measures reaction time, working memory, speed of information processing, problem solving, recognition memory and fine motor speed and coordination. The battery takes approximately 30 min to administer. There is little research on the reliability of this battery. A study of the stability of scores over repeated measures in 25 healthy college students examined five ANAM tests, four of which were employed by Wilken et al. (2003) in the MS study discussed above: Simple reaction time, Continuous performance test (a concentration measure in which the participant is asked whether the letter displayed is the same as the previous letter displayed), Math processing (a measure of speed of information processing in which the participant is asked to solve a mathematical problem), and Sternberg memory (a measure of recognition memory in which the participant is asked to remember a series of letters and recognize whether any of these letters appear in a subsequent sequence). Intraclass correlation coefficients (ICCs) were used to determine score stability over five trials (four ICCs). Each test showed all four ICCs were Z0.75, which demonstrated excellent agreement between repeated administrations (Kaminski et al., 2009). For normative data please refer to Hanly et al. (2010). Further validation of the ANAM is needed because this study had a modest sample size made up exclusively of relapsing-remitting MS patients. Patients also had little physical disability (i.e., mean EDSS of 2.2) with the authors noting that individuals with poor upper motor limb coordination would have difficulties completing the ANAM because it requires the use of a computer mouse. Another potential drawback of the ANAM is the absence of a measure of executive function. 5.2. Mindstreams Computerized Cognitive Battery (MCCB) The Mindstreams Computerized Cognitive Battery (MCCB) measures speed of information processing, attention, executive function, motor skills, visual spatial skills, verbal function, and memory, and also computes a global cognitive index
H. Lapshin et al. / Multiple Sclerosis and Related Disorders 1 (2012) 196–201
(Achiron et al., 2007). MS patients performed more poorly than healthy controls on most measures. However, the wider applicability of the MCCB is hindered by a paucity of data—of the 51 patients tested, three quarters had relapsing-remitting MS, 17% had probable MS, and, once more, physical disability was low (i.e., mean EDSS of 2.6). The MCCB has also been successful in distinguishing healthy elderly participants from those with mild cognitive impairment (MCI) (Dwolatzky et al., 2004). Norms for the MCCB exist for a healthy elderly sample (Dwolatzky et al., 2004). As with the ANAM there is a motor component involved. Finally a potential drawback of the MCCB is that the tests cannot be interpreted on site but must instead be sent by internet to the NeuroTrax Corporation for interpretation (NeuroTrax Corporation, 2000–2012). While generally reliable, such an approach could introduce potential concerns of confidentiality for patients. 5.3. Amsterdam Neuropsychological Tasks (ANT) The Amsterdam Neuropsychological Tasks (ANT) targets focused, divided, and sustained attention, executive function, and speed of information processing, and requires approximately 90 min of administration time. Results from a study of 53 MS patients and 58 healthy controls showed that patients had deficits with attention and speed of information processing and that these correlated with disease severity (i.e., EDSS) and duration (De Sonneville et al., 2002). Healthy control data can be found in Marchetta et al. (2008). Unlike the previous computer studies discussed, this one examined approximately equal proportions of relapsing-remitting, secondary progressive and primary progressive patients. Patients in this study also had greater physical disability than previously reported (i.e., EDSS of 4.7); but the sample size once again remained modest. The ANT requires the use of a mouse, a potential drawback, and the battery itself is too long to be used on a regular basis in a clinical setting where time constraints are a problem. Adding to the battery’s construct validity, results from the ANT have correlated with neuroimaging findings (Lazeron et al., 2006). 5.4. Cognitive Stability Index (CSI) The Cognitive Stability Index (CSI) is a computerized battery developed for screening cognitive impairment in patients with neurologic illness in general. This battery, reported to take approximately 30 min to administer, was used to explore cognitive difficulties in 40 patients with MS (Younes et al., 2007). No mention was made of disease course or EDSS score. The CSI measures attention, speed of information processing, visual memory, and reaction time. Forty patients completed the CSI, a comprehensive neuropsychological battery, and the PASAT alone. Results showed that the CSI was significantly more sensitive than just the PASAT (i.e., 86% as opposed to 28%) in detecting cognitive dysfunction as determined by the comprehensive neuropsychological battery. Furthermore, a study examining CSI test results in patients with traumatic brain injury, attention deficit/hyperactivity disorder, and Alzheimer’s disease, but not MS, found that all four indices had satisfactory test–retest reliability (attention r ¼0.73, speed of information processing r¼ 0.73, visual memory r ¼0.68, and reaction time r ¼0.80) (Erlanger et al., 2002). For normative data please refer to Erlanger et al. (2002). A major disadvantage to the CSI, however, is that it is administered over the internet. In the absence of supervision, reliability becomes a significant confounding issue, as demonstrated in the failed validation study of the computerized MSNQ (Akbar et al., 2010). There is also a subtle motor component to testing: patients are asked to respond by pushing a number key, which could prove problematic for those with more advanced disability.
199
5.5. Cognitive Drug Research (CDR) battery The Cognitive Drug Research (CDR) battery includes tests that measure five cognitive domains: attention, vigilance, working memory, episodic memory, and speed of information processing, and the average of the tests gives a composite score. The system takes approximately 15–20 min to administer and could potentially replace the BRNB and MACFIMS in research and clinical settings where time and administration expertise are an issue. A study comparing the CDR to the PASAT and the Digit Symbol Substitution Test (DSST) alone showed a significant correlation between a composite score and scores from each of the two tests (Edgar et al., 2011). In the same study, most CDR measures had good test–retest reliability ( 40.7), with the exception of working and episodic memory indices. In addition, unlike the computerized batteries previously discussed, the CDR has a minimal motor component. Instead of using a keyboard or mouse to respond, patients used a response box with two large buttons for ‘‘YES’’ and ‘‘NO’’. There were, however, some limitations to this study: the sample consisted of only relapsing-remitting patients with minimal physical disability (i.e., mean EDSS¼ 2.8), and there was no comparison between the CDR and an existing cognitive battery validated for use with MS patients. An additional drawback of the CDR is that tests requiring verbal responses were recorded by an administrator.
5.6. Miscellaneous tests The computerized attention battery (TAP) is a collection of attention tests that has been used with MS patients. However, the original battery is not in English and is therefore not discussed in this paper (Schulz et al., 2006; Penner et al., 2003).
6. Conclusion Data from a small number of studies suggest that computerized cognitive testing may help offset some of the obstacles to cognitive assessment for MS patients. The few studies published to date all have small to medium sample sizes, are generally limited to relapsing remitting patients with little to moderate disability and, to varying degrees, contain testing that demands intact motor function. Reliability, discriminant validity, ecological validity and longitudinal data are also lacking. Nevertheless, early forays are promising. Without exception all the batteries were able to distinguish MS patients from healthy control subjects. The challenge now is to refine testing, move away from motor confounders, boost sample size, pay greater attention to neurological variables that could influence cognition, such as disease course, and assess test–retest reliability. Future computerized batteries would benefit from including tests that assess all known cognitive deficits associated with multiple sclerosis, namely the five cognitive domains outlined in the MACFIMS battery: information processing speed, memory, executive function, visuospatial processing, and word fluency and retrieval (Benedict et al., 2002). New computerized tests should also ideally be able to generate results in a comprehensible format that can be interpreted according to clearly defined parameters. This should also include clear instructions on how the results of tests should be reported to patients. The likelihood of a catastrophic reaction (Goldstein, 1980) is very rare, but the possibility suggests that testing should be undertaken by a health care professional who has had experience dealing with MS patients and who has the ability to assuage anxiety. Many of these challenges were faced by MS researchers 20 years ago when cognition became a new focus
200
H. Lapshin et al. / Multiple Sclerosis and Related Disorders 1 (2012) 196–201
of inquiry. Cautious optimism suggests that once again these can be met successfully with computerized cognitive assessments.
Disclosure statement H. Lapshin receives support from an Ontario Graduate Scholarship grant. Dr. O’Connor serves on scientific advisory boards for Novartis, Sanofi-Aventis, Bayer Schering Pharma, Genentech, Inc., and Roche; has received speaker honoraria from Biogen Idec, Teva Pharmaceutical Industries Ltd., Novartis, and Sanofi-Aventis; has served as a consultant for Biogen Idec, Actelion Pharmaceuticals Ltd., Bayer Schering Pharma, EMD Serono, Inc., Teva Pharmaceutical Industries Ltd., Genentech Inc., and Warburg Pincus; has received research support from Abbott, Bayer Schering Pharma, Novartis, BioMS Medical, Sanofi-Aventis, CIS Pharma, Genmab A/S, Cognosci, Inc., Wyeth, Daiichi Sankyo, and Roche; and serves as the National Scientific and Clinical Advisor to the MS Society of Canada. Dr. Lanctˆot has received research support and/or speaker’s honoraria from Abbott Laboratories, Lundbeck Canada Inc., Pfizer Canada Inc., Janssen Ortho, MedImmune, and Wyeth. Dr. Feinstein has served on scientific advisory boards for Merck Serono and Avanir Pharmaceuticals; has received speaker honoraria from Merck Serono, Teva Pharmaceutical Industries Ltd., Bayer Schering Pharma, and Biogen Idec; serves on the editorial boards of Multiple Sclerosis and the African Journal of Psychiatry; receives publishing royalties for The Clinical Neuropsychiatry of Multiple Sclerosis (Cambridge University Press, 2007); serves on the Medical Advisory Committee for the Multiple Sclerosis Society of Canada; conducts neuropsychiatric evaluation, cognitive testing, brain imaging in neuropsychiatry in his clinical practice (20% effort); and receives research support from Teva Pharmaceutical Industries Ltd., Merck Serono, Canadian Institute of Health Research, and the Multiple Sclerosis Society of Canada.
Role of the funding source This review was funded by the Multiple Sclerosis Society of Canada.
References Achiron A, Polliak M, Rao SM, Barak Y, Lavie M, Appelboim N, et al. Cognitive patterns and progression in multiple sclerosis: construction and validation of percentile curves. Journal of Neurology, Neurosurgery and Psychiatry 2005;76(5):744–9. Achiron A, Doniger GM, Harel Y, Appleboim-Gavish N, Lavie M, Simon ES. Prolonged response times characterize cognitive performance in multiple sclerosis. European Journal of Neurology 2007;14:1102–8. Akbar N, Honarmand K, Kou N, Levine B, Rector N, Feinstein A. Validity of an Internet version of the Multiple Sclerosis Neuropsychological Questionnaire. Multiple Sclerosis 2010;16(12):1500–6. Akbar N, Honarmand K, Kou N, Feinstein A. Validity of a computerized version of the Symbol Digit Modalities Test in multiple sclerosis. Journal of Neurology 2011;258:373–9. Akbar N, Honarmand K, Feinstein A. Self-assessment of cognition in multiple sclerosis: the role of personality and anxiety. Cognitive and Behavioral Neurology 2011b;24(3):115–21. Basso MR, Beason-Hazen S, Lynn J, Rammohan K, Bornstein RA. Screening for cognitive dysfunction in multiple sclerosis. Archives of Neurology 1996;53(10):980–4. Beatty WW, Goodkin DE, Monson N, Beatty PA, Hertsgaard D. Anterograde and retrograde amnesia in patients with chronic progressive multiple sclerosis. Archives of Neurology 1988;45(6):611–9. Benedict RH, Fischer JS, Archibald CJ, Arnett PA, Beatty WW, Bobholz J, et al. Minimal neuropsychological assessment of MS patients: a consensus approach. Clinical Neuropsychologist 2002;16(3):381–97. Benedict RH, Munschauer F, Linn R, Miller C, Murphy E, Foley F, et al. Screening for multiple sclerosis cognitive impairment using a self-administered 15-item questionnaire. Multiple Sclerosis 2003;9(1):95–101.
Benedict RHB, Cookfair D, Gavett R, Gunther M, Munschauer F, Garg N, et al. Validity of the minimal assessment of cognitive function in multiple sclerosis. Journal of the International Neuropsychological Society 2006;12(4):549–58. Benedict RHB, Zivadinov R. Risk factors for and management of cognitive dysfunction in multiple sclerosis. Nature Reviews Neurology 2011;7(6):332–42. Brassington JC, Marsh NV. Neuropsychological aspects of multiple sclerosis. Neuropsychology Review 1998;8(2):43–77. Brochet B, Deloire MSA, Bonnet M, Salort-Campana E, Ouallet JC, Petry KG, et al. Should SDMT substitute for PASAT in MSFC? A 5-year longitudinal study Multiple Sclerosis 2008;14:1242–9. Chiaravalloti ND, DeLuca J, Moore NB, Ricker JH. Treating learning impairments improves memory performance in multiple sclerosis: a randomized clinical trial. Multiple Sclerosis 2005;11:58–68. Cook TH, et al. Neuropsychological assessment of neuropsychiatric and neuromedical disorders. 3rd ed. New York: Oxford University Press; 2009. De Sonneville LMJ, Boringa JB, Reuling IEW, Lazeron RHC, Ade r HJ, Polman CH. Information processing characteristics in subtypes of multiple sclerosis. Neuropsychologia 2002;40:1751–65. Drake AS, Weinstock-Guttman B, Morrow SA, Hojnacki D, Munschauer FE, Benedict RHB. Psychometrics and normative data for the Multiple Sclerosis Functional Composite: replacing the PASAT with the Symbol Digit Modalities Test. Multiple Sclerosis 2010;16(2):228–37. Dwolatzky T, Whitehead V, Doniger GM, Simon ES, Schweiger A, Jaffe D, et al. Validity of the Mindstreams Computerized Cognitive Battery for mild cognitive impairment. Journal of Molecular Neuroscience 2004;24:33–44. Edgar C, Jongen PJ, Sanders E, Sindic C, Goffette S, Dupuis M, et al. Cognitive performance in relapsing remitting multiple sclerosis: a longitudinal study in daily practice using a brief computerized cognitive battery. BMC Neurology 2011;11:68. Erlanger DM, Kaushik T, Broshek D, Freeman J, Feldman D, Festa J. Development and validation of a web-based screening tool for monitoring cognitive status. Journal of Head Trauma Rehabilitation 2002;17(5):458–76. Foong J, Rozewicz L, Quaghebeur G, Davie CA, Kartsounis LD, Thompson AJ, et al. Executive function in multiple sclerosis: the role of frontal lobe pathology. Brain 1997;120:15–26. Foong J, Rozewicz L, Davie CA, Thompson AJ, Miller DH, Ron MA. Correlates of executive function in multiple sclerosis: the use of magnetic resonance spectroscopy as an index of focal pathology. Journal of Neuropsychiatry & Clinical Neurosciences 1999;11(1):45–50. Goldstein K. Language and language disturbances.New York: Grune & Stratton; 1980. Grafman J, Rao S, Bernardin L, Leo GJ. Automatic memory processes in patients with multiple sclerosis. Archives of Neurology 1991;48(10):1072–5. Hanly JG, Omisade A, Su L, Farewell V, Fisk JD. Assessment of cognitive function in systemic lupus erythematosus, rheumatoid arthritis, and multiple sclerosis by computerized neuropsychological tests. American College of Rheumatology 2010;62(5):1478–86. Hayman-Abello BA, Hayman-Abello SE, Rourke BP. Human Neuropsychology in Canada: the 1990s (A Review of Research by Canadian Neuropsychologists Conducted Over the Past Decade). Canadian Psychology 2003;44(2):100–38. Heaton RK, Nelson LM, Thompson DS, Burks JS, Franklin GM. Neuropsychological findings in relapsing-remitting and chronic progressive multiple sclerosis. Journal of Consulting and Clinical Psychology 1985;53(1):103–10. Huber SJ, Paulson GW, Shuttleworth EC, Chakeres D, Clapp LE, Pakalnis A, et al. Magnetic resonance imaging correlates of dementia in multiple sclerosis. Archives of Neurology 1987;44(7):732–6. Johnson KL, Bamer AM, Yorkston KM, Amtmann D. Use of cognitive aids and other assistive technology by individuals with multiple sclerosis. Disability and Rehabilitation: Assistive Technology 2009;4(1):1–8. Kaminski TW, Groff RM, Glutting JJ. Examining the stability of Automated Neuropsychological Assessment Metric (ANAM) baseline test scores. Journal of Clinical and Experimental Neuropsychology 2009;31(6):689–99. Kane RL, Roebuck-Spencer T, Short P, Kabat M, Wilken J. Identifying and monitoring cognitive deficits in clinical populations using Automated Neuropsychological Assessment Metrics (ANAM) tests. Archives of Clinical Neuropsychology 2007;22S:S115–26. Kujala P, Portin A, Revonsuo A, Ruutiainen J. Automatic and controlled information processing in multiple sclerosis. Brain 1994;117:1115–26. Lazeron RHC, de Sonneville LMJ, Scheltens P, Polman CH, Barkhof F. Cognitive slowing in multiple sclerosis is strongly associated with brain volume reduction. Multiple Sclerosis 2006;12:760–8. Lengenfelder J, Bryant D, Diamond BJ, Kalmar JH, Moore NB, DeLuca J. Processing speed interacts with working memory efficiency in multiple sclerosis. Archives of Clinical Neuropsychology 2006;21:229–38. Macniven JAB, Davis C, Ho MY, Bradshaw CM, Szabadi E, Constantinescu CS. Stroop performance in multiple sclerosis: information processing, selective attention, or executive functioning? Journal of the International Neuropsychological Society 2008;14:805–14. Marchetta NDJ, De Sonneville LMJ, Krabbendam L, Jolles J. Sustained and focused attention in adult ADHD. Journal of Attention Disorders 2008;11(6):664–76. Minden SL, Hoaglin DC, Hadden L, Frankel D, Robbins T, Perloff J. Access to and utilization of neurologists by people with multiple sclerosis. Neurology 2008;70(132):1141–9. Penner IK, Rausch M, Kappos L, Opwis K, Radu¨ WW. Analysis of impairment related functional architechture in MS patients during performance of different attention tasks. Journal of Neurology 2003;250:461–72. Rao S. A manual for the Brief Repeatable Battery of Neuropsychological Tests in Multiple Sclerosis.New York, NY: National Multiple Sclerosis Society; 1990.
H. Lapshin et al. / Multiple Sclerosis and Related Disorders 1 (2012) 196–201
Rao SM, Leo GJ, Bernardin L, Unverzagt F. Cognitive dysfunction in multiple sclerosis, I: frequency, patterns, and prediction. Neurology 1991a;41(5):685–91. Rao SM, Leo GJ, Ellington L, Nauertz T, Bernardin L, Unverzagt F. Cognitive dysfunction in multiple sclerosis, II: impact on employment and social functioning. Neurology 1991b;41(5):692–6. Reicker LI, Tombaugh TN, Walker L, Freedman MS. Reaction time: an alternative method for assessing the effects of multiple sclerosis on information processing speed. Archives of Clinical Neuropsychology 2007;22:655–64. ¨ ¨ al ¨ ainen ¨ Rosti-Otajarvi EM, Ham PI. Neuropsychological rehabilitation for multiple sclerosis (Review). The Cochrane Collaboration, vol. 11. John Wiley & Sons, Ltd.; 2011. Schultheis MT, Garay E, DeLuca J. The influence of cognitive impairment on driving performance in multiple sclerosis. Neurology 2001;56:1089–94.
201
Schulz D, Kopp B, Kunkel A, Faiss JH. Cognition in the early stage of multiple sclerosis. Journal of Neurology 2006;253:1002–10. Thornton AE, Raz N. Memory impairment in multiple sclerosis: a quantitative review. Neuropsychology 1997;11(3):357–66. Thornton AE, Raz N, Tucke KA. Memory in multiple sclerosis: contextual encoding deficits. Journal of the International Neuropsychological Society 2002;8(3): 395–409. Wilken JA, Kane R, Sullivan CL, Wallin M, Usiskin JB, Quig ME, et al. The utility of computerized neuropsychological assessment of cognitive dysfunction in patients with relapsing-remitting multiple sclerosis. Multiple Sclerosis 2003;9:119–27. Younes M, Hill J, Quinless J, Kilduff M, Peng B, Cook SD, et al. Internet-based cognitive testing in multiple sclerosis. Multiple Sclerosis 2007;13:1011–9.
Contents lists available at SciVerse ScienceDirect
Multiple Sclerosis and Related Disorders journal homepage: www.elsevier.com/locate/msard
Review
Computerized cognitive testing for patients with multiple sclerosis Helen Lapshin a,c,n, Paul O’Connor b,c, Krista L. Lanctˆot a,c, Anthony Feinstein a,c a b c
Sunnybrook Health Sciences Centre, 2075 Bayview Avenue, Toronto, ON, Canada M4N 3M5 St. Michael’s Hospital, 30 Bond Street, Toronto, ON, Canada M5B 1W8 University of Toronto, 27 King’s College Circle, Toronto, ON, Canada M5S 1A1
a r t i c l e i n f o
a b s t r a c t
Article history: Received 15 February 2012 Received in revised form 27 April 2012 Accepted 1 May 2012
Cognitive dysfunction affects 40–65% of multiple sclerosis (MS) patients. To date, conventional paper and pencil neuropsychological testing has been the method of choice for detecting deficits. However, the expense and lack of access to these tests make it difficult for many patients to obtain an assessment. With the introduction of computerized cognitive testing, certain barriers to assessment can be reduced. This paper critically reviews the currently available computerized batteries, including the Automated Neuropsychology Assessment Matrix (ANAM), the Mindstreams Computerized Cognitive Battery (MCCB), the Amsterdam Neuropsychological Tasks (ANT), the Cognitive Stability Index (CSI), and the Cognitive Drug Research (CDR) battery. Recent developments in this field show promise, although further research is required before this approach can be more widely implemented. & 2012 Elsevier B.V. All rights reserved.
Keywords: Multiple sclerosis Neuropsychological testing Computerized cognitive assessment Cognition Automated Neuropsychology Assessment Matrix Mindstreams Computerized Cognitive Battery
Contents 1. 2.
3. 4. 5.
6.
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 Detecting cognitive dysfunction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 2.1. Multiple Sclerosis Screening Questionnaire (MSNQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 2.2. Brief batteries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 2.3. Minimal Assessment of Cognitive Function in MS (MACFIMS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Limited access to assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 Computerized tests of cognitive function in MS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 5.1. Automated Neuropsychology Assessment Matrix (ANAM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 5.2. Mindstreams Computerized Cognitive Battery (MCCB) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 5.3. Amsterdam Neuropsychological Tasks (ANT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 5.4. Cognitive Stability Index (CSI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 5.5. Cognitive Drug Research (CDR) battery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 5.6. Miscellaneous tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Disclosure statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Role of the funding source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
1. Introduction n Corresponding author at: Sunnybrook Health Sciences Centre, 2075 Bayview Avenue, Toronto, ON, Canada M4N 3M5. Tel.: þ 1 416 480 6100x7626. E-mail address: [email protected] (H. Lapshin).
2211-0348/$ - see front matter & 2012 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.msard.2012.05.001
This review paper addresses the clinically relevant question of barriers to neuropsychological testing in those with multiple sclerosis (MS) and explores whether computerized approaches
H. Lapshin et al. / Multiple Sclerosis and Related Disorders 1 (2012) 196–201
may prove helpful where conventional testing is inaccessible. The literature devoted to computerized testing in MS is critically reviewed following which suggestions are made on how to build on promising early work. Cognitive dysfunction affects approximately 40–65% of patients with multiple sclerosis (MS) (Rao et al., 1991a). The most common types of deficits involve slowed information processing speed and impaired memory (Benedict et al., 2002; Brassington and Marsh, 1998). The latter includes problems related to working, episodic and semantic memory (Beatty et al., 1988; Benedict et al., 2003, 2006; Grafman et al., 1991; Thornton and Raz, 1997). Both acquisition and recall are affected (Thornton and Raz, 1997; Thornton et al., 2002). Processing speed deficits are evident in tests such as the Paced Auditory Serial Addition Test (PASAT) which has an impairment rate of approximately 27%, and the Symbol Digit Modalities Test which approximately 52% of patients fail (Benedict et al., 2006). Executive dysfunction is also common with difficulties in planning, concept formation and abstract reasoning reported (Foong et al., 1997, 1999; Heaton et al., 1985). Data show that impairment on the Delis–Kaplan Executive Function System Sorting Test occurs 15–25% of the time (Benedict et al., 2006). Visuospatial and verbal fluency deficits may also be present although less frequently (Achiron et al., 2005; Basso et al., 1996; Benedict and Zivadinov, 2011; Huber et al., 1987). Results for the Judgement of Line Orientation Test show impairment in approximately 22% of patients, and scores on the Controlled Oral Word Association Test demonstrate an impairment rate of approximately 13% (Benedict et al., 2006). Cognitive dysfunction has an adverse effect on the quality of life of afflicted MS patients. Impaired individuals have greater difficulties with work, social life, and activities of daily living (Rao et al., 1991b). In addition, it can negatively affect driving ability (Schultheis et al., 2001).
197
Anxiety and certain personality attributes such as conscientiousness and neuroticism have also been found to influence patients’ responses in the MSNQ introducing a note of caution when interpreting findings (Akbar et al., 2011b). An attempt at validating an internet version of the measure in order to expand its utility has been unsuccessful (Akbar et al., 2010). Possible reasons for this were the use of a biased sample, the high rates of depression in the sample selected and the unreliability of collecting data in the absence of a test administrator or the supervision of a research assistant. 2.2. Brief batteries Several brief screening batteries have been implemented for use with MS patients, the most often cited being the Neuropsychological Screening Battery for Multiple Sclerosis (NPSBMS) (Rao et al., 1991a). This assessment takes approximately 30 min to administer and consists of four tests: the Selective Reminding Test which is a verbal learning test, 7/24 Spatial Recall Test, a spatial learning test, Paced Auditory Serial Addition Test which measures speed of information processing and working memory, and the Controlled Oral Word Association Test, a word fluency and retrieval task dependent on speed of information processing. A variant of this battery, the Rao Brief Repeatable Battery of Neuropsychological Tests (BRB-N), includes these 4 tests (the 7/24 replaced by the 10/36) and the Symbol Digit Modalities Test added as another measure of information processing speed (Rao, 1990). Fifteen alternate versions of the tests have been developed for serial testing, with the exception of the PASAT, which has 2 alternate forms. Brief batteries are relatively quick to administer but require trained personnel, which limits their use in clinical settings. 2.3. Minimal Assessment of Cognitive Function in MS (MACFIMS)
2. Detecting cognitive dysfunction Given the implications of cognitive dysfunction for patients’ lives, detection is important. Although there is limited evidence to support the efficacy of neuropsychological rehabilitation, promising data show that cognitive training could facilitate memory ¨ ¨ al ¨ ainen, ¨ improvement (Rosti-Otajarvi and Ham 2011; Chiaravalloti et al., 2005). Detecting cognitive impairment not only allows clinicians to address these difficulties in the form of rehabilitation or compensatory techniques (Johnson et al., 2009), but also helps researchers to understand the etiology of cognitive dysfunction which is fundamental to the advancement of new treatments. A number of approaches have been developed for this detecting cognitive dysfunction: (1) the self-report method, for example the Multiple Sclerosis Neuropsychological Screening Questionnaire (MSNQ) (Benedict et al., 2003), (2) screening batteries such as the Brief Repeatable Battery (BRB-N) (Rao, 1990), and (3) a more comprehensive evaluation, such as the Minimal Assessment of Cognitive Function in MS (MACFIMS) (Benedict et al., 2002).
The Minimal Assessment of Cognitive Function in MS (MACFIMS) is a comprehensive cognitive evaluation validated for use in patients with MS (Benedict et al., 2002). The battery takes approximately 90 min to administer and the tests were chosen by consensus by a group of leading neuropsychologists and clinical psychologists who specialize in the field of cognitive dysfunction associated with MS. The battery comprises seven tests which encompass five cognitive domains often impaired in MS: processing speed and working memory, learning and memory, executive function, visuospatial processing and verbal fluency and retrieval. All tests chosen are known to be sensitive to the kinds of cognitive deficits experienced by MS patients. The authors recommend the battery be accompanied by measures of pre-morbid IQ and depression. Although this battery is a valid and reliable measure of cognitive dysfunction in patients with MS, it requires a significant amount of time and expertise to administer, once again placing it beyond the capabilities of most MS clinics. The data also require neuropsychological expertise to score and interpret.
2.1. Multiple Sclerosis Screening Questionnaire (MSNQ) 3. Limited access to assessment The Multiple Sclerosis Neuropsychological Screening Questionnaire (MSNQ) is a five minute, fifteen item, self-administered screening questionnaire that includes two separate forms: one for the patient and another for an informant. Data reveal that the patient form correlates more strongly with depression whereas the informant report more closely reflects patients’ performance on objective neuropsychological tests (Benedict et al., 2003). The informant MSNQ has a reported sensitivity and specificity of 0.83 and 0.97, respectively, for detecting cognitive dysfunction.
There are many obstacles to obtaining a cognitive assessment from a neuropsychologist. Referrals are frequently made by a neurologist but almost one in three MS patients do not see a neurologist on a regular basis (Minden et al., 2008). Data from the United States show that those who do not have regular neurology contact are more likely to have one or more of the following characteristics: be poor, have no health insurance, live in rural areas, be African American, have a disease duration of more than
198
H. Lapshin et al. / Multiple Sclerosis and Related Disorders 1 (2012) 196–201
15 years, have difficulty walking but are not using an assistive device, rely on a wheelchair or scooter, or be bedridden. Conversely, female patients and those who have experienced a relapse in the past year see a neurologist more often (Minden et al., 2008). Adding to the challenge of obtaining an assessment is a paucity of neuropsychologists. Canadian data from the last decade reveal that there were only 230 neuropsychologists in the country with only a small number engaged in MS research. How many neuropsychologists assess MS patients for clinical purposes is not known (Hayman-Abello et al., 2003). The result of these many barriers is that cognitive testing is an important component of MS management that many patients never receive.
reaction time which measures the basic speed of response to a stimulus, choice reaction whereby the time it takes a participant to press one of two keys on the keyboard depending on which stimulus is presented on the screen is measured, and semantic reaction time where a stimulus is presented on the screen and the participant is asked to decide whether it belongs to a certain semantic category (Reicker et al., 2007). Results showed that all three test modalities were successful in distinguishing MS patients from healthy controls. This result has been replicated by another study that also focused on indices of attention and reaction time (De Sonneville et al., 2002). Apart from tests of information processing speed, one computerized index of executive function has been used in MS research. The finding that patients were impaired when compared to controls was difficult to interpret, however, as the test had a significant motor component (Foong et al., 1997).
4. Methods 5.1. Automated Neuropsychology Assessment Matrix (ANAM) The literature review was conducted with Medline, using the following keywords: (1) (multiple sclerosis) & (neuropsychological tests/neuropsychological testing) & (computerized or internet), and (2) (multiple sclerosis) & (cognition/cognitive function) & (computerized or Internet).
5. Computerized tests of cognitive function in MS Computerized cognitive assessment provides a potential solution to some of the problems identified above. Administration and scoring do not require the expertise of a neuropsychologist which saves both time and money. In addition, computerized testing has good reliability and provides exact measures of response times in tests of speed of information processing (Cook et al., 2009; Wilken et al., 2003). There are, however, disadvantages associated with this process. Patients may not be fully computer literate which may deter them from taking these tests. Impairments such as muscle weakness and poor coordination may hinder or prevent testing. Finally, it may prove challenging to assess free recall and other tests requiring verbal responses without the assistance of a test administrator. Several studies have used single computerized tests to supplement conventional paper and pencil testing. In this instance, the term ‘‘paper and pencil’’ testing refers to tests in which the patient is not required to use a computer (tests that are scored on, but are not administered by, the computer are regarded as ‘‘paper and pencil’’ tests for the purposes of this paper). Single computerized tests have included the STROOP (Macniven et al., 2008), a computerized version of the PASAT (Lengenfelder et al., 2006) and the SDMT (Akbar et al., 2011a). The latter study deserves further comment for there is an emerging consensus that the paper and pencil version of the SDMT, may be the most sensitive and specific cognitive test for MS patient. This has led researchers to suggest that it replace the three second PASAT as the single cognitive index in the MS Functional Composite (MSFC) (Brochet et al., 2008; Drake et al., 2010). In the computerized SDMT (SDMT-c) study significant differences were found between 119 MS patient and 38 healthy controls. When the SDMT-c alone was compared to the BRB-N, the gold standard in this study, a sensitivity and a specificity of 71% and 84%, respectively, was found with respect to predicting cognitive dysfunction (Akbar et al., 2011a). This result overlaps with that obtained from another study that focused on computerized indices of motor programming and automatic and controlled processing (Kujala et al., 1994). Another study explored the efficacy of the Computerized Tests of Information Processing (CTIP) which consists of three reaction time tests: simple
Several computerized cognitive batteries have been employed with MS patients. The Automated Neuropsychology Assessment Matrix (ANAM) is a library of computerized tests created by the Department of Defense in the United States. The ANAM has been used as a screening tool for cognitive impairment in a variety of populations including patients with systemic lupus erythematosus, Parkinson’s disease, Alzheimer’s disease, brain injury, migraines, and multiple sclerosis (Kane et al., 2007). The goal of a recent study was to validate a select number of ANAM tests for use with MS patients (Wilken et al., 2003). Fifty relapsing-remitting MS patients were assessed with the ANAM battery, the results of which correlated with a battery of neuropsychological tests and distinguished patients from healthy controls as well as cognitively impaired from cognitively intact patients. The ANAM battery measures reaction time, working memory, speed of information processing, problem solving, recognition memory and fine motor speed and coordination. The battery takes approximately 30 min to administer. There is little research on the reliability of this battery. A study of the stability of scores over repeated measures in 25 healthy college students examined five ANAM tests, four of which were employed by Wilken et al. (2003) in the MS study discussed above: Simple reaction time, Continuous performance test (a concentration measure in which the participant is asked whether the letter displayed is the same as the previous letter displayed), Math processing (a measure of speed of information processing in which the participant is asked to solve a mathematical problem), and Sternberg memory (a measure of recognition memory in which the participant is asked to remember a series of letters and recognize whether any of these letters appear in a subsequent sequence). Intraclass correlation coefficients (ICCs) were used to determine score stability over five trials (four ICCs). Each test showed all four ICCs were Z0.75, which demonstrated excellent agreement between repeated administrations (Kaminski et al., 2009). For normative data please refer to Hanly et al. (2010). Further validation of the ANAM is needed because this study had a modest sample size made up exclusively of relapsing-remitting MS patients. Patients also had little physical disability (i.e., mean EDSS of 2.2) with the authors noting that individuals with poor upper motor limb coordination would have difficulties completing the ANAM because it requires the use of a computer mouse. Another potential drawback of the ANAM is the absence of a measure of executive function. 5.2. Mindstreams Computerized Cognitive Battery (MCCB) The Mindstreams Computerized Cognitive Battery (MCCB) measures speed of information processing, attention, executive function, motor skills, visual spatial skills, verbal function, and memory, and also computes a global cognitive index
H. Lapshin et al. / Multiple Sclerosis and Related Disorders 1 (2012) 196–201
(Achiron et al., 2007). MS patients performed more poorly than healthy controls on most measures. However, the wider applicability of the MCCB is hindered by a paucity of data—of the 51 patients tested, three quarters had relapsing-remitting MS, 17% had probable MS, and, once more, physical disability was low (i.e., mean EDSS of 2.6). The MCCB has also been successful in distinguishing healthy elderly participants from those with mild cognitive impairment (MCI) (Dwolatzky et al., 2004). Norms for the MCCB exist for a healthy elderly sample (Dwolatzky et al., 2004). As with the ANAM there is a motor component involved. Finally a potential drawback of the MCCB is that the tests cannot be interpreted on site but must instead be sent by internet to the NeuroTrax Corporation for interpretation (NeuroTrax Corporation, 2000–2012). While generally reliable, such an approach could introduce potential concerns of confidentiality for patients. 5.3. Amsterdam Neuropsychological Tasks (ANT) The Amsterdam Neuropsychological Tasks (ANT) targets focused, divided, and sustained attention, executive function, and speed of information processing, and requires approximately 90 min of administration time. Results from a study of 53 MS patients and 58 healthy controls showed that patients had deficits with attention and speed of information processing and that these correlated with disease severity (i.e., EDSS) and duration (De Sonneville et al., 2002). Healthy control data can be found in Marchetta et al. (2008). Unlike the previous computer studies discussed, this one examined approximately equal proportions of relapsing-remitting, secondary progressive and primary progressive patients. Patients in this study also had greater physical disability than previously reported (i.e., EDSS of 4.7); but the sample size once again remained modest. The ANT requires the use of a mouse, a potential drawback, and the battery itself is too long to be used on a regular basis in a clinical setting where time constraints are a problem. Adding to the battery’s construct validity, results from the ANT have correlated with neuroimaging findings (Lazeron et al., 2006). 5.4. Cognitive Stability Index (CSI) The Cognitive Stability Index (CSI) is a computerized battery developed for screening cognitive impairment in patients with neurologic illness in general. This battery, reported to take approximately 30 min to administer, was used to explore cognitive difficulties in 40 patients with MS (Younes et al., 2007). No mention was made of disease course or EDSS score. The CSI measures attention, speed of information processing, visual memory, and reaction time. Forty patients completed the CSI, a comprehensive neuropsychological battery, and the PASAT alone. Results showed that the CSI was significantly more sensitive than just the PASAT (i.e., 86% as opposed to 28%) in detecting cognitive dysfunction as determined by the comprehensive neuropsychological battery. Furthermore, a study examining CSI test results in patients with traumatic brain injury, attention deficit/hyperactivity disorder, and Alzheimer’s disease, but not MS, found that all four indices had satisfactory test–retest reliability (attention r ¼0.73, speed of information processing r¼ 0.73, visual memory r ¼0.68, and reaction time r ¼0.80) (Erlanger et al., 2002). For normative data please refer to Erlanger et al. (2002). A major disadvantage to the CSI, however, is that it is administered over the internet. In the absence of supervision, reliability becomes a significant confounding issue, as demonstrated in the failed validation study of the computerized MSNQ (Akbar et al., 2010). There is also a subtle motor component to testing: patients are asked to respond by pushing a number key, which could prove problematic for those with more advanced disability.
199
5.5. Cognitive Drug Research (CDR) battery The Cognitive Drug Research (CDR) battery includes tests that measure five cognitive domains: attention, vigilance, working memory, episodic memory, and speed of information processing, and the average of the tests gives a composite score. The system takes approximately 15–20 min to administer and could potentially replace the BRNB and MACFIMS in research and clinical settings where time and administration expertise are an issue. A study comparing the CDR to the PASAT and the Digit Symbol Substitution Test (DSST) alone showed a significant correlation between a composite score and scores from each of the two tests (Edgar et al., 2011). In the same study, most CDR measures had good test–retest reliability ( 40.7), with the exception of working and episodic memory indices. In addition, unlike the computerized batteries previously discussed, the CDR has a minimal motor component. Instead of using a keyboard or mouse to respond, patients used a response box with two large buttons for ‘‘YES’’ and ‘‘NO’’. There were, however, some limitations to this study: the sample consisted of only relapsing-remitting patients with minimal physical disability (i.e., mean EDSS¼ 2.8), and there was no comparison between the CDR and an existing cognitive battery validated for use with MS patients. An additional drawback of the CDR is that tests requiring verbal responses were recorded by an administrator.
5.6. Miscellaneous tests The computerized attention battery (TAP) is a collection of attention tests that has been used with MS patients. However, the original battery is not in English and is therefore not discussed in this paper (Schulz et al., 2006; Penner et al., 2003).
6. Conclusion Data from a small number of studies suggest that computerized cognitive testing may help offset some of the obstacles to cognitive assessment for MS patients. The few studies published to date all have small to medium sample sizes, are generally limited to relapsing remitting patients with little to moderate disability and, to varying degrees, contain testing that demands intact motor function. Reliability, discriminant validity, ecological validity and longitudinal data are also lacking. Nevertheless, early forays are promising. Without exception all the batteries were able to distinguish MS patients from healthy control subjects. The challenge now is to refine testing, move away from motor confounders, boost sample size, pay greater attention to neurological variables that could influence cognition, such as disease course, and assess test–retest reliability. Future computerized batteries would benefit from including tests that assess all known cognitive deficits associated with multiple sclerosis, namely the five cognitive domains outlined in the MACFIMS battery: information processing speed, memory, executive function, visuospatial processing, and word fluency and retrieval (Benedict et al., 2002). New computerized tests should also ideally be able to generate results in a comprehensible format that can be interpreted according to clearly defined parameters. This should also include clear instructions on how the results of tests should be reported to patients. The likelihood of a catastrophic reaction (Goldstein, 1980) is very rare, but the possibility suggests that testing should be undertaken by a health care professional who has had experience dealing with MS patients and who has the ability to assuage anxiety. Many of these challenges were faced by MS researchers 20 years ago when cognition became a new focus
200
H. Lapshin et al. / Multiple Sclerosis and Related Disorders 1 (2012) 196–201
of inquiry. Cautious optimism suggests that once again these can be met successfully with computerized cognitive assessments.
Disclosure statement H. Lapshin receives support from an Ontario Graduate Scholarship grant. Dr. O’Connor serves on scientific advisory boards for Novartis, Sanofi-Aventis, Bayer Schering Pharma, Genentech, Inc., and Roche; has received speaker honoraria from Biogen Idec, Teva Pharmaceutical Industries Ltd., Novartis, and Sanofi-Aventis; has served as a consultant for Biogen Idec, Actelion Pharmaceuticals Ltd., Bayer Schering Pharma, EMD Serono, Inc., Teva Pharmaceutical Industries Ltd., Genentech Inc., and Warburg Pincus; has received research support from Abbott, Bayer Schering Pharma, Novartis, BioMS Medical, Sanofi-Aventis, CIS Pharma, Genmab A/S, Cognosci, Inc., Wyeth, Daiichi Sankyo, and Roche; and serves as the National Scientific and Clinical Advisor to the MS Society of Canada. Dr. Lanctˆot has received research support and/or speaker’s honoraria from Abbott Laboratories, Lundbeck Canada Inc., Pfizer Canada Inc., Janssen Ortho, MedImmune, and Wyeth. Dr. Feinstein has served on scientific advisory boards for Merck Serono and Avanir Pharmaceuticals; has received speaker honoraria from Merck Serono, Teva Pharmaceutical Industries Ltd., Bayer Schering Pharma, and Biogen Idec; serves on the editorial boards of Multiple Sclerosis and the African Journal of Psychiatry; receives publishing royalties for The Clinical Neuropsychiatry of Multiple Sclerosis (Cambridge University Press, 2007); serves on the Medical Advisory Committee for the Multiple Sclerosis Society of Canada; conducts neuropsychiatric evaluation, cognitive testing, brain imaging in neuropsychiatry in his clinical practice (20% effort); and receives research support from Teva Pharmaceutical Industries Ltd., Merck Serono, Canadian Institute of Health Research, and the Multiple Sclerosis Society of Canada.
Role of the funding source This review was funded by the Multiple Sclerosis Society of Canada.
References Achiron A, Polliak M, Rao SM, Barak Y, Lavie M, Appelboim N, et al. Cognitive patterns and progression in multiple sclerosis: construction and validation of percentile curves. Journal of Neurology, Neurosurgery and Psychiatry 2005;76(5):744–9. Achiron A, Doniger GM, Harel Y, Appleboim-Gavish N, Lavie M, Simon ES. Prolonged response times characterize cognitive performance in multiple sclerosis. European Journal of Neurology 2007;14:1102–8. Akbar N, Honarmand K, Kou N, Levine B, Rector N, Feinstein A. Validity of an Internet version of the Multiple Sclerosis Neuropsychological Questionnaire. Multiple Sclerosis 2010;16(12):1500–6. Akbar N, Honarmand K, Kou N, Feinstein A. Validity of a computerized version of the Symbol Digit Modalities Test in multiple sclerosis. Journal of Neurology 2011;258:373–9. Akbar N, Honarmand K, Feinstein A. Self-assessment of cognition in multiple sclerosis: the role of personality and anxiety. Cognitive and Behavioral Neurology 2011b;24(3):115–21. Basso MR, Beason-Hazen S, Lynn J, Rammohan K, Bornstein RA. Screening for cognitive dysfunction in multiple sclerosis. Archives of Neurology 1996;53(10):980–4. Beatty WW, Goodkin DE, Monson N, Beatty PA, Hertsgaard D. Anterograde and retrograde amnesia in patients with chronic progressive multiple sclerosis. Archives of Neurology 1988;45(6):611–9. Benedict RH, Fischer JS, Archibald CJ, Arnett PA, Beatty WW, Bobholz J, et al. Minimal neuropsychological assessment of MS patients: a consensus approach. Clinical Neuropsychologist 2002;16(3):381–97. Benedict RH, Munschauer F, Linn R, Miller C, Murphy E, Foley F, et al. Screening for multiple sclerosis cognitive impairment using a self-administered 15-item questionnaire. Multiple Sclerosis 2003;9(1):95–101.
Benedict RHB, Cookfair D, Gavett R, Gunther M, Munschauer F, Garg N, et al. Validity of the minimal assessment of cognitive function in multiple sclerosis. Journal of the International Neuropsychological Society 2006;12(4):549–58. Benedict RHB, Zivadinov R. Risk factors for and management of cognitive dysfunction in multiple sclerosis. Nature Reviews Neurology 2011;7(6):332–42. Brassington JC, Marsh NV. Neuropsychological aspects of multiple sclerosis. Neuropsychology Review 1998;8(2):43–77. Brochet B, Deloire MSA, Bonnet M, Salort-Campana E, Ouallet JC, Petry KG, et al. Should SDMT substitute for PASAT in MSFC? A 5-year longitudinal study Multiple Sclerosis 2008;14:1242–9. Chiaravalloti ND, DeLuca J, Moore NB, Ricker JH. Treating learning impairments improves memory performance in multiple sclerosis: a randomized clinical trial. Multiple Sclerosis 2005;11:58–68. Cook TH, et al. Neuropsychological assessment of neuropsychiatric and neuromedical disorders. 3rd ed. New York: Oxford University Press; 2009. De Sonneville LMJ, Boringa JB, Reuling IEW, Lazeron RHC, Ade r HJ, Polman CH. Information processing characteristics in subtypes of multiple sclerosis. Neuropsychologia 2002;40:1751–65. Drake AS, Weinstock-Guttman B, Morrow SA, Hojnacki D, Munschauer FE, Benedict RHB. Psychometrics and normative data for the Multiple Sclerosis Functional Composite: replacing the PASAT with the Symbol Digit Modalities Test. Multiple Sclerosis 2010;16(2):228–37. Dwolatzky T, Whitehead V, Doniger GM, Simon ES, Schweiger A, Jaffe D, et al. Validity of the Mindstreams Computerized Cognitive Battery for mild cognitive impairment. Journal of Molecular Neuroscience 2004;24:33–44. Edgar C, Jongen PJ, Sanders E, Sindic C, Goffette S, Dupuis M, et al. Cognitive performance in relapsing remitting multiple sclerosis: a longitudinal study in daily practice using a brief computerized cognitive battery. BMC Neurology 2011;11:68. Erlanger DM, Kaushik T, Broshek D, Freeman J, Feldman D, Festa J. Development and validation of a web-based screening tool for monitoring cognitive status. Journal of Head Trauma Rehabilitation 2002;17(5):458–76. Foong J, Rozewicz L, Quaghebeur G, Davie CA, Kartsounis LD, Thompson AJ, et al. Executive function in multiple sclerosis: the role of frontal lobe pathology. Brain 1997;120:15–26. Foong J, Rozewicz L, Davie CA, Thompson AJ, Miller DH, Ron MA. Correlates of executive function in multiple sclerosis: the use of magnetic resonance spectroscopy as an index of focal pathology. Journal of Neuropsychiatry & Clinical Neurosciences 1999;11(1):45–50. Goldstein K. Language and language disturbances.New York: Grune & Stratton; 1980. Grafman J, Rao S, Bernardin L, Leo GJ. Automatic memory processes in patients with multiple sclerosis. Archives of Neurology 1991;48(10):1072–5. Hanly JG, Omisade A, Su L, Farewell V, Fisk JD. Assessment of cognitive function in systemic lupus erythematosus, rheumatoid arthritis, and multiple sclerosis by computerized neuropsychological tests. American College of Rheumatology 2010;62(5):1478–86. Hayman-Abello BA, Hayman-Abello SE, Rourke BP. Human Neuropsychology in Canada: the 1990s (A Review of Research by Canadian Neuropsychologists Conducted Over the Past Decade). Canadian Psychology 2003;44(2):100–38. Heaton RK, Nelson LM, Thompson DS, Burks JS, Franklin GM. Neuropsychological findings in relapsing-remitting and chronic progressive multiple sclerosis. Journal of Consulting and Clinical Psychology 1985;53(1):103–10. Huber SJ, Paulson GW, Shuttleworth EC, Chakeres D, Clapp LE, Pakalnis A, et al. Magnetic resonance imaging correlates of dementia in multiple sclerosis. Archives of Neurology 1987;44(7):732–6. Johnson KL, Bamer AM, Yorkston KM, Amtmann D. Use of cognitive aids and other assistive technology by individuals with multiple sclerosis. Disability and Rehabilitation: Assistive Technology 2009;4(1):1–8. Kaminski TW, Groff RM, Glutting JJ. Examining the stability of Automated Neuropsychological Assessment Metric (ANAM) baseline test scores. Journal of Clinical and Experimental Neuropsychology 2009;31(6):689–99. Kane RL, Roebuck-Spencer T, Short P, Kabat M, Wilken J. Identifying and monitoring cognitive deficits in clinical populations using Automated Neuropsychological Assessment Metrics (ANAM) tests. Archives of Clinical Neuropsychology 2007;22S:S115–26. Kujala P, Portin A, Revonsuo A, Ruutiainen J. Automatic and controlled information processing in multiple sclerosis. Brain 1994;117:1115–26. Lazeron RHC, de Sonneville LMJ, Scheltens P, Polman CH, Barkhof F. Cognitive slowing in multiple sclerosis is strongly associated with brain volume reduction. Multiple Sclerosis 2006;12:760–8. Lengenfelder J, Bryant D, Diamond BJ, Kalmar JH, Moore NB, DeLuca J. Processing speed interacts with working memory efficiency in multiple sclerosis. Archives of Clinical Neuropsychology 2006;21:229–38. Macniven JAB, Davis C, Ho MY, Bradshaw CM, Szabadi E, Constantinescu CS. Stroop performance in multiple sclerosis: information processing, selective attention, or executive functioning? Journal of the International Neuropsychological Society 2008;14:805–14. Marchetta NDJ, De Sonneville LMJ, Krabbendam L, Jolles J. Sustained and focused attention in adult ADHD. Journal of Attention Disorders 2008;11(6):664–76. Minden SL, Hoaglin DC, Hadden L, Frankel D, Robbins T, Perloff J. Access to and utilization of neurologists by people with multiple sclerosis. Neurology 2008;70(132):1141–9. Penner IK, Rausch M, Kappos L, Opwis K, Radu¨ WW. Analysis of impairment related functional architechture in MS patients during performance of different attention tasks. Journal of Neurology 2003;250:461–72. Rao S. A manual for the Brief Repeatable Battery of Neuropsychological Tests in Multiple Sclerosis.New York, NY: National Multiple Sclerosis Society; 1990.
H. Lapshin et al. / Multiple Sclerosis and Related Disorders 1 (2012) 196–201
Rao SM, Leo GJ, Bernardin L, Unverzagt F. Cognitive dysfunction in multiple sclerosis, I: frequency, patterns, and prediction. Neurology 1991a;41(5):685–91. Rao SM, Leo GJ, Ellington L, Nauertz T, Bernardin L, Unverzagt F. Cognitive dysfunction in multiple sclerosis, II: impact on employment and social functioning. Neurology 1991b;41(5):692–6. Reicker LI, Tombaugh TN, Walker L, Freedman MS. Reaction time: an alternative method for assessing the effects of multiple sclerosis on information processing speed. Archives of Clinical Neuropsychology 2007;22:655–64. ¨ ¨ al ¨ ainen ¨ Rosti-Otajarvi EM, Ham PI. Neuropsychological rehabilitation for multiple sclerosis (Review). The Cochrane Collaboration, vol. 11. John Wiley & Sons, Ltd.; 2011. Schultheis MT, Garay E, DeLuca J. The influence of cognitive impairment on driving performance in multiple sclerosis. Neurology 2001;56:1089–94.
201
Schulz D, Kopp B, Kunkel A, Faiss JH. Cognition in the early stage of multiple sclerosis. Journal of Neurology 2006;253:1002–10. Thornton AE, Raz N. Memory impairment in multiple sclerosis: a quantitative review. Neuropsychology 1997;11(3):357–66. Thornton AE, Raz N, Tucke KA. Memory in multiple sclerosis: contextual encoding deficits. Journal of the International Neuropsychological Society 2002;8(3): 395–409. Wilken JA, Kane R, Sullivan CL, Wallin M, Usiskin JB, Quig ME, et al. The utility of computerized neuropsychological assessment of cognitive dysfunction in patients with relapsing-remitting multiple sclerosis. Multiple Sclerosis 2003;9:119–27. Younes M, Hill J, Quinless J, Kilduff M, Peng B, Cook SD, et al. Internet-based cognitive testing in multiple sclerosis. Multiple Sclerosis 2007;13:1011–9.
