Original Paper Neurocpidcmiology 1992; 11:288-295
3 Italian Multicentre Study on Dementia. SMID Centre. Florence: b Istituto Nazionale di Riposo e Cura per Anziani (1NRCA). Istituto di Riccrca a Carattcre Scientifico, Appignano: c Department of Biostatistics and Epidemiology. Fidia, Abano Terme: d Department of Neurology and Psychiatry, University of Florence. Italy
Key Words Abbreviated mental test Screening Dementia Validity Prevalence survey
Validation of the Hodkinson Abbreviated Mental Test as a Screening Instrument for Dementia in an Italian Population
Abstract We investigated the accuracy of the Hodkinson abbreviated mental test (AMT) as a screening instrument for dementia in an Italian population. The AMT was administered by nonme dical personnel to 124 subjects > 5 9 years old. Each subject independently underwent a clinical evaluation for dementia (DSM-III criteria), and scores on the AMT were compared to corresponding clinical diagnoses (standard for comparison). Twenty of the 124 subjects were found to be affected by dementia upon clinical investigation. Although a score of 6 on the AMT showed the best combination of sensitivity (90%) and specificity (89%), only a score of 7 yielded 100% sensitiv ity (71 % specificity). Specificity was higher in men. younger, and more educated subjects.
Introduction An important methodological issue in cur rent morbidity studies of dementia is case ascertainment [ 1]. Because prevalence or inci dence figures obtained by counting subjects who have been diagnosed by the medical ser vices of a given community may underesti mate true morbidity, several recent studies have used the alternative door-to-door ap proach [2], Since the direct examination of each subject included in the sample is ex tremely time-consuming, two-phase surveys
may be more practicable: during phase 1, all subjects in the study are screened by nonme dical personnel using a brief cognitive test; only individuals who screened positive un dergo a clinical examination in phase 2. This survey design generated the need for brief, easy-to-administer, and well-tolerated screening instruments of acceptable accuracy. We investigated the accuracy of the Hodkin son abbreviated mental test (AMT) [3] as a screening test for dementia in an Italian popu lation [4], We addressed the following specific questions: (1) Can the AMT, developed and
Presented in part at the EURODEM workshop ‘Case finding for dementia in epidemiological studies'. Liverpool, UK. March 1989, and at the annual meeting o f the World Federation o f Neurology. Research Group on Ncurocpidcmiology, Chicago. III., USA. April 1989.
Dr. Walter A. Rocca CentroSM ID Via it Prato 58 1-50123 Firenze (Italy)
© 1992 S. Karger AG, Basel 0251-5350/92/ 0 1 16-0288S2.75/0
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
Walter A. Roccaa Salvatore Bonaiutob Andrea Lippia Paolo Lucianib Tiziana Pistarellib Antonio Grandinetlib Fabiano Cavarzeranc Luigi AmaduccF
Subjects and Methods General Study Design and Sample To investigate the validity (external or criterion validity) [5] of the AMT as a screening instrument for dementia, the clinical examination was used as a stan dard for comparison. We administered the screening test to all subjects in the study; at a different time, and regardless of the result of the screening test, we investi gated all subjects to reach or exclude a diagnosis of dementia. Each subject’s score on the AMT was then compared to the corresponding clinical diagnosis. The study sample included the following 2 seg ments. (1) A simple random sample of 122 subjects that was drawn from the complete list of 778 people > 59 years old residing in the municipality of Appignano (Maccrata province, Italy) on January 1, 1987. The resident list was obtained from the registry office of the municipality; both household and institutional ized populations were included. (2) A clinical series of 21 subjects, residing in Appignano, who were affected by dementia at the time of the study. These patients were in contact with one of the investigators (SB, PL. TP, AG) because of their dementia. The study was originally designed to include only the probability sample; however, the clinical series was added at a later date to increase the number of demented patients suitable for the screening test and to be included in the sensitivity estimates. Subjects, or their next of kin, were invited to partic ipate in the study at the time of the screening interview or at first contact; no written consent was needed. From both segments of the sample, we excluded sub jects selected for the study who could not be com pletely investigated because: (1) they refused to be screened, clinically evaluated, or both, or (2) they could not be formally tested because of severe hearing loss, mutism, or advanced dementia. The probability sample was investigated between March 1987 and February 1988; the clinical series between February and March 1990.
In the probability sample, we first administered the AMT and then the examination (time lapse of 1-3 months); in the clinical series we first confirmed the diagnosis of dementia and then administered the screening instrument (time lapse of few days). Subjects belonging to the probability sample were ineligible for inclusion in the clinical series. Screening Instrument The screening instrument was an Italian transla tion of the AMT, that is a 10-itcm version of the Blcssed-Roth information-memory-concentration test [3, 6], Each question scores 1 point, and the total score ranges from 0 (no correct answer) to 10 (10 correct answers). For most items, the Italian version was a direct translation from English; the question ‘Name of present monarch?’ was changed into ‘Name of present president?’. When the subject was interviewed at home or in an institution, the item ‘Name of hospital’ was changed to ‘Name of place’, and for the item ‘Recogni tion of two people’, the subject was asked to recognize two family members, or the interviewer plus one family member. An Italian street name was used for the 5-min address recall. The AMT was part of a broader questionnaire and was administered, at home or at the institution of current residence, by one of seven medically unsophisticated interviewers. The in terviewers were trained for the administration of the questionnaire and specifically for the use of the AMT. In the probability sample, the screening test preceded the clinical evaluation; for the clinical series, the inter viewers were kept uninformed regarding the subjects’ status or the reasons for the interview. Standard for Comparison The standard for comparison was the clinical diag nosis of dementia (DSM-III criteria) [7], The diagnos tic protocol followed a step-by-step algorithm as fol lows. (1) Confirmation and measurement of the cogni tive deficit by the Mini-mental state (MMS) [8] and the Blesscd-Roth information-memory-conccntration test (modified by Hodkinson) [3, 6). (2) Confirmation and measurement of the functional deficit by the BlcsscdRoth dementia scale [6], (3) Exclusion of depression, mental retardation, or psychoses, and diagnosis of dementia (DSM-III criteria) [7] by medical history, the Hamilton rating scale for depression [9], and physical and neurological examinations. Information required to complete the Blesscd-Roth scale and historical data were obtained from an informant; a relative in close and continuous contact with the subject was preferred whenever possible.
289
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
tested in English-speaking populations, be used in Italian populations? (2) What is the optimal cutoff score for screening purposes? (3) What is the influence of the age, sex, and education of the respondent on test perfor mance? (4) Which items better discriminate between demented and nondemented sub jects?
Table 1. Age, sex, and education distributions of the total study sample and of the group of demented patients Women1
Men1
Classification
n Age
60-69 70-79 80-89 Total
Education
Illiterate No formal education -2nd grade2 3rd-4th grade Over 4th grade Total
n
%
Both sexes' %
n
%
28(0) 28(5) 17(8) 73(13)
38.4 38.4 23.3 100.0
50(0) 51(8) 23(12) 124(20)
40.3 41.1 18.5 100.0
5.9
11(9)
15.1
14(11)
11.3
0.0
7(1) 39(3) 16(0) 73(13)
9.6 53.4 21.9 100.0
7(1) 62(8) 41(0) 124(20)
5.6 50.0 33.1 100.0
22(0) 23(3) 6(4) 51(7)
43.1 45.1 11.8 100.0
3(2) 0(0) 23(5) 25(0) 51(7)
45.1 49.0 100.0
1 Numbers and percentages include both demented and nondemented subjects in the study sample; the num bers in parentheses refer only to demented subjects. 2 Able to read and write without formal education, or did not reach the 3rd grade.
Data Analysis The screening validity of the AMT was measured by computing its sensitivity and specificity at various cutoff levels [5, 10]; approximate 95% confidence intervals of the estimates were drawn. In addition. AMT accuracy was described by its receiver-operatorcharacteristic curve [11]. The effect of age, sex, and education on the specificity of the AMT was investi gated through subgroup analyses. Age and education were dichotomized following statistical consider ations, that is, to obtain approximately equal samples in each subgroup. The individual performance of the 10 items of the AMT was also measured.
290
Results From an intended probability sample of 122 subjects, 13 were excluded for the follow ing reasons: 6 subjects were administered the AMT but refused to be clinically investigated, 2 subjects could not be tested because of severe hearing loss (1 was deaf-mute), and 5 because of severe dementia. The final proba bility sample comprised, therefore, 109 sub jects for whom complete information was available. Of these 109 subjects, 5 were found to be affected by dementia; the remaining 104 were nondemented at the time of the study. From an intended clinical series of 21 de mented subjects, 6 patients were excluded be cause they had reached advanced stages of the disease and their cognitive performance could not be tested by nonmedical personnel. The complete sample included, therefore, 104 nondemented and 20 demented subjects (5 from the probability sample and 15 from the clinical series).
Rocca/Bonaiuto/Lippi/Luciani/ Pistarelli/Grandinctti/Cavarzeran/ Amaducci
Validity of Dementia Screening
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
In the probability sample, the diagnostic protocol was first applied by a psychiatry resident (PL), who compiled a specifically designed clinical-data form. He did not have access to the AMT scores obtained by the interviewers. This investigator was trained for the use of the form and for the diagnosis of dementia. Each form was then reviewed by a more experienced neurol ogist (AL), who reexamined several subjects for an independent evaluation. For the clinical series of pre viously diagnosed dementias, one of 2 specialists (TP or AG) confirmed the eligibility of patients using the above-mentioned clinical-data form.
Table 2. Sensitivity and specificity o f the A M T at various screening levels (n = 124)
Screening Test result1 level positive total n
0 1 2 3 4 5 6 7 8 9 10
%
2 1.6 2 1.6 2.4 3 7 5.6 13 10.5 19 15.3 29 23.4 50 40.3 83 66.9 110 88.7 124 100.0
negative true false + +
total n
%
2 0 2 0 3 0 7 0 12 1 15 4 18 11 20 30 20 63 20 90 20 104
122 122 121 117 111 105 95 74 41 14
98.4 98.4 97.6 94.4 89.5 84.7 76.6 59.7 33.1 11.3
0
0.0
Sensitivity2 %
Specificity3 %
10.0 10.0 15.0 35.0 60.0 75.0(56.0-94.0) 90.0(76.9-100.0) lOO.O(-) 100.0(-)
100.0 100.0 100.0 100.0 99.0 96.2(92.5-99.9) 89.4 (83.5-95.3) 71.2(62.5-79.9) 39.4 (30.0-48.8) 13.5
false true
18 18 17 13 8 5 2 0 0 0 0
104 104 104 104 103 100 93 74 41 14 0
100.0 100.0
0.0
Table 1 reports the age and sex, and the education and sex distributions of the com plete study sample, and, separately, of the 20 demented patients. In the overall sample, 18 women and 3 men (16.9%) had no formal education or had not reached the 3rd grade of elementary school; 14 of them were unable to read or write. Six subjects (4.8%) were living in a nursing home at the time of the study; 8 subjects (6.4%) reported moderate visual loss, and 8 (6.4%) moderate hearing loss. Of the 20 demented patients, 8 were aged 70-79 years, 12 were 80-89 years old. Twelve subjects had no formal education or did not reach 3rd grade. Measures of cognitive and behavioral
performance were distributed in the 20 de mented subjects as follows: (1) the MMS me dian score was 12 (range 3-22); (2) the median score on the Blessed-Roth information-memo ry-concentration test was 15 (range 6-31), and (3) the median score on the Blessed-Roth de mentia scale was 13 (range 2-21). Table 2 shows the number of subjects who screened positive or negative at different cut off levels of the AMT and the corresponding sensitivity and specificity. Although a score of 6 (4 or more errors) showed the best combina tion of sensitivity and specificity, only the score of 7 (3 or more errors) gave 100% sensi tivity. The trade-off between sensitivity and
291
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
1 The test result is positive (+) if the score is smaller than or equal to the screening level, negative (-) if otherwise. 2 Sensitivity is the proportion of truly diseased persons in the screened population who are identified as diseased by the screening test. Estimates of sensitivity restricted to the probability sample were as follows: screen ing level 0 = 0.0; 1 - 0.0; 2 = 0.0; 3 = 0.0; 4 = 40.0; 5 - 60.0; 6 = 80.0; 7 = 100.0; 8 = 100.0; 9 = 100.0; 10=100.0. Values in parentheses are approximate 95% confidence intervals, and are reported only for screening levels around the one selected. 3 Specificity is the proportion of truly nondiseased persons who are so identified by the screening test. Estimates of specificity do not change by restricting the sample to the probability sample. Values in parentheses are approx imate 95% confidence intervals, and are reported only for screening levels around the one selected.
SPECIFICITY (%) 100
90
Fig. 1. Receiver-operator-charactcrislic curve describing the ac curacy of the AMT as a screening instrument for dementia.
80
70
60
50
40
30
20
10
0
1-SPECIFICITY (%)
Table 3. Specificity of the AMT at various screening levels by age. sex, and education
Screening Total level12 (n-104) %
3 4 5 6 7 8 9 10
Specificity, %2 by education
by sex
by age 60-69 years (n=50)
70+ years (n=54)
men (n=44)
women (n=60)
over 4th 4 th grade or less grade (n=63) (n=41)
100.0 100.0 98.0 90.0 82.0 46.0 14.0
100.0
100.0
98.2 94.4 88.9 61.1 33.3 13.0
100.0 100.0 100.0 93.2 52.3 20.5
100.0 98.3 93.3 81.7 55.0 30.0 8.3
100.0 100.0 100.0 97.6 90.2 58.5 24.4
100.0
99.0 96.2 89.4 71.2 39.4 13.5
0.0
0.0
0.0
0.0
0.0
0.0
0.0
100.0
98.4 93.6 84.1 58.7 27.0 6.3
292
Rocca/ Bonaiuto/Lippi/Luciani/ Pistarelli/Grandinetti/Cavarzeran/ Amaducci
Validity of Dementia Screening
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
1 The lines for scores 0.1. and 2 are not reported because they were identical to the line for a score of 3. 2 Specificity is the proportion of truly nondiseased persons who are so identified by the screening test.
Discussion In the present study, we investigated the validity of the AMT as a screening test for dementia. Although a two-phase prevalence survey of dementia in the population of Appignano, Italy [4] provided the incentive for this validation, our results may be relevant to future epidemiologic studies of dementia in our country and abroad. The AMT was se lected for the following reasons: (1) it is brief, well tolerated, and easy to administer by non medical personnel; (2) it does not require reading or writing; (3) it can be easily adapted to the Italian culture, and (4) it was shown to correlate well with the Blessed-Roth informa tion-memory-concentration test [6] in both institutionalized [3, 12] and living-at-home populations [13]. One important limitation of
the AMT, which was originally designed for hospital or institutional populations, is the loss of sensitivity of some items when the test is administered at home. Our results suggest that the 2 items ‘Name of place’ and ‘Recogni tion of two people’ did not contribute no ticeably to the screening validity of the AMT. Although the AMT is a subset of the Blessed-Roth information-memory-concen tration test that was used in the clinical evalua tion, the screening test can be considered inde pendent from the standard for comparison selected for our study. The actual standard for comparison was a diagnosis of dementia based on DSM-1II criteria, and this diagnosis was reached using a combination of cognitive, be havioral, and historical evidence. In order to confirm the presence of a cognitive decline, the specialist used two tests, one of which was the MMS. In addition, the degree of behavioral decline was assessed independently using the Blessed-Roth dementia scale. Since a major objective of this study was to validate the AMT on a sample representative of the target population, we first chose an unselected group of subjects (the probability sample). An advantage of this sample was to reproduce on a small scale the target popula tion and, therefore, to include subjects with mild dementia or with other conditions which could interfere with the detection of demen tia. The principal limitation of this represen tative sample was the small number of sub jects found to be simultaneously affected by dementia and able to undergo a cognitive test. Of the 10 demented patients found in the probability sample, 5 could not be formally tested; therefore, our sample size was ade quate to investigate the specificity of the test but insufficient to investigate its sensitivity. Since available resources did not allow us to study a broader probability sample, a series of known cases of dementia living in the same municipality was added at a later time. As a
293
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
specificity is well described by the receiveroperator-characteristic curve (fig. 1); it shows graphically that the best balance between sen sitivity and specificity corresponded to a score of 6. Sensitivity estimates obtained by restricting the sample to the probability seg ment are also reported in table 2; estimates of specificity did not change following such re striction. The influence of age, sex, and education on the specificity of the AMT is shown in table 3. Specificity was higher for all cutoff levels in men compared to women, in younger com pared to older, and in more educated com pared to less educated subjects. Table 4 shows the sensitivity and specific ity of individual items of the AMT. Two items, ‘Name of place’ and ‘Recognition of two people’ had the lowest sensitivity. The best balance between sensitivity and specific ity was reached by the items ‘Year’, ‘Year of World War T, ‘Count backwards 20-1’, and ‘Date of birth’.
Table 4. Sensitivity and specificity o f individual item s o f the AM T (n = 124)
Item
total
Age Time (hour) 5-min address recall Year Name of place (home/institution) Recognition of two people Date of birth Year of World War I Name of present president Count backwards 20-1
Negative1
Positive1
false true
true false + +
total n
%
Sensi Speci tivity2 ficity3 % %
n
%
14 10 84 26
11.3 8.1 67.7 21.0
10 8 20 14
4 2 64 12
110 114 40 98
88.7 91.9 32.3 79.0
10 12 0 6
100 102 40 92
50.0 40.0 100.0 70.0
96.2 98.1 38.5 88.5
3 3 17 56 83 22
2.4 2.4 13.7 45.2 66.9 17.7
2 3 12 18 20 12
1 0 5 38 63 10
121 121 107 68 41 102
97.6 97.6 86.3 54.8 33.1 82.3
18 17 8 2 0 8
103 104 99 66 41 94
10.0 15.0 60.0 90.0 100.0 60.0
99.0 100.0 95.2 63.5 39.4 90.4
consequence of this mixed-sample design, only our estimates of specificity can be con sidered as being population based. The esti mates of sensitivity were based mainly on clinically recognized cases; therefore, we may have overestimated the sensitivity of the AMT compared to its use in the general popu lation [ 11 ]. If we restricted the sample to the 5 cases of dementia included in the probability sample, sensitivity was lower at correspond ing cutoff scores; however, the best balance between sensitivity and specificity remained at a score of 6. Findings from the complete sample were consistent with those suggested by the representative sample. Since we excluded from the study 11 sub jects who could not be formally tested by non medical personnel because of severe demen tia, we investigated the ability of the AMT to detect cases of intitial or moderately ad vanced dementia. This restriction has proba bly lowered our sensitivity estimates. On the
294
other hand, to include subjects with terminal stages of the disease in the validation, one needs to arbitrarily assign them a score of 0 on the AMT; this approach would artificially in crease the estimates of sensitivity. Since cases of advanced dementia can be detected in the population without any formal instrument [14], they may conveniently be excluded when measuring the screening accuracy of a cognitive test. We found the screening level of 6 (4 or more errors) to give the best balance of sensi tivity and specificity. Consistently, Hodkinson, in his original study, found that the AMT score of 6 corresponded to the cutoff score of 24 on the Blessed-Roth test [3], At all cutoff levels, specificity was higher in men, younger, and more educated subjects. The difference between men and women was probably ex plained by the fact that women tended to be both older and less educated than men in our sample.
Rocca/Bonaiulo/LippL/Luciani/ Pistarelli/Gfandinetti/Cavarzeran/ Amaducci
Validity of Dementia Screening
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
1 The item was positive (+) if the subject did not respond correctly, negative (-) if the subject responded correctly. 2 Sensitivity is the proportion of truly diseased persons in the screened population who are identified as diseased by the screening item. 3 Specificity is the proportion of truly nondiseased persons who are so identified by the screening item.
When using the AMT as a screening instru ment for dementia in a two-phase prevalence survey, maximum sensitivity is highly desir able even at the expense of some loss in speci ficity. Therefore, we chose the screening level of 7 (7 or less) for our prevalence survey of dementia [4]. A decision on the screening level may vary according to the size of the target population. We found that a cutoff score of 6 screened positive only 14% of the probability segment of our sample, while a cutoff score of 7 screened positive 32% of the sample. For a relatively small population like that in Appignano (n = 778 people > 59 years old), the use of a 100% sensitive screening level was possi ble since the corresponding loss of specificity was still compatible with available resources for the study. On the other hand, when the target population is much larger, it may be practical to use the best compromise between sensitivity and specificity. In this case, the
risk of losing some of the patients affected by dementia in the population should be weighed against the reduction in time and costs of the survey. Our study suggests that brief cognitive tests like the AMT may be adequate to screen for dementia in Italy, although maximum sensi tivity can only be reached at the expense of some loss in specificity. Acknowledgements This study was supported in part by the Istituto Nazionalc di Riposo e Cura per Anziani (INRCA), Istituto di Ricerca a Carattere Scientifico, Grant No. 1033 (June 16. 1988), and was facilitated by collabora tion with the Amministrazione comunale di Appignano and the Unita’ Sanitaria Locale N. 15 della Rcgione Marche. The authors thank Dr. Lucia Galluzzo for the general study coordination, Emanuela Manzella for administering the screening instrument to a series of demented subjects. Dr. Francesco Grigoletto for help in data analysis, and Adriana Frasson for technical assistance in preparing the illustration.
1 Henderson AS, Jorm AF: Is caseascertainment of Alzheimer’s dis ease in field surveys practicable? Psychol Med 1987;17:549-555. 2 Rocca WA, Amaducci L; Epidemi ology of Alzheimer’s disease: in An derson DW (ed): Neuroepidemiolo gy: A Tribute to Bruce Schoenberg. Boca Raton, CRC, 1991, pp 55-96. 3 Hodkinson HM: Evaluation of a mental test score for assessment of mental impairment in the elderly. Age Ageing 1972;1:233-238. 4 Rocca WA, BonaiuloS, Lippi A, Luciani P, Turtù F, Cavarzeran F. Amaducci L: Prevalence of clini cally diagnosed Alzheimer’s disease and other dementing disorders: A door-to-door survey in Appignano, Maccrata Province, Italy. Neurol ogy 1990;40:626-631.
5 Last JM: A Dictionary of Epidemi ology. ed 2. New York, Oxford Uni versity Press, 1988. 6 Blessed G, Tomlinson BE. Roth M: The association between quantita tive measures of dementia and of senile change in the cerebral grey matter of elderly subjects. Br J Psy chiatry 1968; 114:797-811. 7 American Psychiatry Association: Diagnostic and Statistical Manual of Mental Disorders, ed 3. Washing ton. American Psychiatric Associa tion, 1980. 8 Folstein MF, Folstein SE, McHugh PR: ‘Mini-mental state’. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975;12:189-198. 9 Hamilton M: Development of a rat ing scale for primary depression ill ness. Br J Soc Clin Psychol 1967:6: 278-296.
10 Ritchie K: The screening of cogni tive impairment in the elderly: A critical review of current methods. J Clin Epidemiol 1988:41:635-643. 11 Fletcher RH. Fletcher SW. Wagner EH: Clinical Epidemiology: The Es sentials, ed 2. Baltimore, Williams and Wilkins, 1988. 12 Qureshi K.N. Hodkinson HM: Eval uation of a ten question mental test in the institutionalized elderly. Age Ageing 1974;3:152-157. 13 Thompson P. Blessed G: Correla tion between the 37-item mental test score and abbreviated 10-item men tal test score by psychogeriatric day patients. Br J Psychiatry 1987:151: 206-209. 14 Schoenberg BS, Anderson DW, Haercr AF: Severe dementia. Preva lence and clinical features in a biracial US population. Arch Neurol 1985;42:740-743.
295
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
References
3 Italian Multicentre Study on Dementia. SMID Centre. Florence: b Istituto Nazionale di Riposo e Cura per Anziani (1NRCA). Istituto di Riccrca a Carattcre Scientifico, Appignano: c Department of Biostatistics and Epidemiology. Fidia, Abano Terme: d Department of Neurology and Psychiatry, University of Florence. Italy
Key Words Abbreviated mental test Screening Dementia Validity Prevalence survey
Validation of the Hodkinson Abbreviated Mental Test as a Screening Instrument for Dementia in an Italian Population
Abstract We investigated the accuracy of the Hodkinson abbreviated mental test (AMT) as a screening instrument for dementia in an Italian population. The AMT was administered by nonme dical personnel to 124 subjects > 5 9 years old. Each subject independently underwent a clinical evaluation for dementia (DSM-III criteria), and scores on the AMT were compared to corresponding clinical diagnoses (standard for comparison). Twenty of the 124 subjects were found to be affected by dementia upon clinical investigation. Although a score of 6 on the AMT showed the best combination of sensitivity (90%) and specificity (89%), only a score of 7 yielded 100% sensitiv ity (71 % specificity). Specificity was higher in men. younger, and more educated subjects.
Introduction An important methodological issue in cur rent morbidity studies of dementia is case ascertainment [ 1]. Because prevalence or inci dence figures obtained by counting subjects who have been diagnosed by the medical ser vices of a given community may underesti mate true morbidity, several recent studies have used the alternative door-to-door ap proach [2], Since the direct examination of each subject included in the sample is ex tremely time-consuming, two-phase surveys
may be more practicable: during phase 1, all subjects in the study are screened by nonme dical personnel using a brief cognitive test; only individuals who screened positive un dergo a clinical examination in phase 2. This survey design generated the need for brief, easy-to-administer, and well-tolerated screening instruments of acceptable accuracy. We investigated the accuracy of the Hodkin son abbreviated mental test (AMT) [3] as a screening test for dementia in an Italian popu lation [4], We addressed the following specific questions: (1) Can the AMT, developed and
Presented in part at the EURODEM workshop ‘Case finding for dementia in epidemiological studies'. Liverpool, UK. March 1989, and at the annual meeting o f the World Federation o f Neurology. Research Group on Ncurocpidcmiology, Chicago. III., USA. April 1989.
Dr. Walter A. Rocca CentroSM ID Via it Prato 58 1-50123 Firenze (Italy)
© 1992 S. Karger AG, Basel 0251-5350/92/ 0 1 16-0288S2.75/0
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
Walter A. Roccaa Salvatore Bonaiutob Andrea Lippia Paolo Lucianib Tiziana Pistarellib Antonio Grandinetlib Fabiano Cavarzeranc Luigi AmaduccF
Subjects and Methods General Study Design and Sample To investigate the validity (external or criterion validity) [5] of the AMT as a screening instrument for dementia, the clinical examination was used as a stan dard for comparison. We administered the screening test to all subjects in the study; at a different time, and regardless of the result of the screening test, we investi gated all subjects to reach or exclude a diagnosis of dementia. Each subject’s score on the AMT was then compared to the corresponding clinical diagnosis. The study sample included the following 2 seg ments. (1) A simple random sample of 122 subjects that was drawn from the complete list of 778 people > 59 years old residing in the municipality of Appignano (Maccrata province, Italy) on January 1, 1987. The resident list was obtained from the registry office of the municipality; both household and institutional ized populations were included. (2) A clinical series of 21 subjects, residing in Appignano, who were affected by dementia at the time of the study. These patients were in contact with one of the investigators (SB, PL. TP, AG) because of their dementia. The study was originally designed to include only the probability sample; however, the clinical series was added at a later date to increase the number of demented patients suitable for the screening test and to be included in the sensitivity estimates. Subjects, or their next of kin, were invited to partic ipate in the study at the time of the screening interview or at first contact; no written consent was needed. From both segments of the sample, we excluded sub jects selected for the study who could not be com pletely investigated because: (1) they refused to be screened, clinically evaluated, or both, or (2) they could not be formally tested because of severe hearing loss, mutism, or advanced dementia. The probability sample was investigated between March 1987 and February 1988; the clinical series between February and March 1990.
In the probability sample, we first administered the AMT and then the examination (time lapse of 1-3 months); in the clinical series we first confirmed the diagnosis of dementia and then administered the screening instrument (time lapse of few days). Subjects belonging to the probability sample were ineligible for inclusion in the clinical series. Screening Instrument The screening instrument was an Italian transla tion of the AMT, that is a 10-itcm version of the Blcssed-Roth information-memory-concentration test [3, 6], Each question scores 1 point, and the total score ranges from 0 (no correct answer) to 10 (10 correct answers). For most items, the Italian version was a direct translation from English; the question ‘Name of present monarch?’ was changed into ‘Name of present president?’. When the subject was interviewed at home or in an institution, the item ‘Name of hospital’ was changed to ‘Name of place’, and for the item ‘Recogni tion of two people’, the subject was asked to recognize two family members, or the interviewer plus one family member. An Italian street name was used for the 5-min address recall. The AMT was part of a broader questionnaire and was administered, at home or at the institution of current residence, by one of seven medically unsophisticated interviewers. The in terviewers were trained for the administration of the questionnaire and specifically for the use of the AMT. In the probability sample, the screening test preceded the clinical evaluation; for the clinical series, the inter viewers were kept uninformed regarding the subjects’ status or the reasons for the interview. Standard for Comparison The standard for comparison was the clinical diag nosis of dementia (DSM-III criteria) [7], The diagnos tic protocol followed a step-by-step algorithm as fol lows. (1) Confirmation and measurement of the cogni tive deficit by the Mini-mental state (MMS) [8] and the Blesscd-Roth information-memory-conccntration test (modified by Hodkinson) [3, 6). (2) Confirmation and measurement of the functional deficit by the BlcsscdRoth dementia scale [6], (3) Exclusion of depression, mental retardation, or psychoses, and diagnosis of dementia (DSM-III criteria) [7] by medical history, the Hamilton rating scale for depression [9], and physical and neurological examinations. Information required to complete the Blesscd-Roth scale and historical data were obtained from an informant; a relative in close and continuous contact with the subject was preferred whenever possible.
289
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
tested in English-speaking populations, be used in Italian populations? (2) What is the optimal cutoff score for screening purposes? (3) What is the influence of the age, sex, and education of the respondent on test perfor mance? (4) Which items better discriminate between demented and nondemented sub jects?
Table 1. Age, sex, and education distributions of the total study sample and of the group of demented patients Women1
Men1
Classification
n Age
60-69 70-79 80-89 Total
Education
Illiterate No formal education -2nd grade2 3rd-4th grade Over 4th grade Total
n
%
Both sexes' %
n
%
28(0) 28(5) 17(8) 73(13)
38.4 38.4 23.3 100.0
50(0) 51(8) 23(12) 124(20)
40.3 41.1 18.5 100.0
5.9
11(9)
15.1
14(11)
11.3
0.0
7(1) 39(3) 16(0) 73(13)
9.6 53.4 21.9 100.0
7(1) 62(8) 41(0) 124(20)
5.6 50.0 33.1 100.0
22(0) 23(3) 6(4) 51(7)
43.1 45.1 11.8 100.0
3(2) 0(0) 23(5) 25(0) 51(7)
45.1 49.0 100.0
1 Numbers and percentages include both demented and nondemented subjects in the study sample; the num bers in parentheses refer only to demented subjects. 2 Able to read and write without formal education, or did not reach the 3rd grade.
Data Analysis The screening validity of the AMT was measured by computing its sensitivity and specificity at various cutoff levels [5, 10]; approximate 95% confidence intervals of the estimates were drawn. In addition. AMT accuracy was described by its receiver-operatorcharacteristic curve [11]. The effect of age, sex, and education on the specificity of the AMT was investi gated through subgroup analyses. Age and education were dichotomized following statistical consider ations, that is, to obtain approximately equal samples in each subgroup. The individual performance of the 10 items of the AMT was also measured.
290
Results From an intended probability sample of 122 subjects, 13 were excluded for the follow ing reasons: 6 subjects were administered the AMT but refused to be clinically investigated, 2 subjects could not be tested because of severe hearing loss (1 was deaf-mute), and 5 because of severe dementia. The final proba bility sample comprised, therefore, 109 sub jects for whom complete information was available. Of these 109 subjects, 5 were found to be affected by dementia; the remaining 104 were nondemented at the time of the study. From an intended clinical series of 21 de mented subjects, 6 patients were excluded be cause they had reached advanced stages of the disease and their cognitive performance could not be tested by nonmedical personnel. The complete sample included, therefore, 104 nondemented and 20 demented subjects (5 from the probability sample and 15 from the clinical series).
Rocca/Bonaiuto/Lippi/Luciani/ Pistarelli/Grandinctti/Cavarzeran/ Amaducci
Validity of Dementia Screening
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
In the probability sample, the diagnostic protocol was first applied by a psychiatry resident (PL), who compiled a specifically designed clinical-data form. He did not have access to the AMT scores obtained by the interviewers. This investigator was trained for the use of the form and for the diagnosis of dementia. Each form was then reviewed by a more experienced neurol ogist (AL), who reexamined several subjects for an independent evaluation. For the clinical series of pre viously diagnosed dementias, one of 2 specialists (TP or AG) confirmed the eligibility of patients using the above-mentioned clinical-data form.
Table 2. Sensitivity and specificity o f the A M T at various screening levels (n = 124)
Screening Test result1 level positive total n
0 1 2 3 4 5 6 7 8 9 10
%
2 1.6 2 1.6 2.4 3 7 5.6 13 10.5 19 15.3 29 23.4 50 40.3 83 66.9 110 88.7 124 100.0
negative true false + +
total n
%
2 0 2 0 3 0 7 0 12 1 15 4 18 11 20 30 20 63 20 90 20 104
122 122 121 117 111 105 95 74 41 14
98.4 98.4 97.6 94.4 89.5 84.7 76.6 59.7 33.1 11.3
0
0.0
Sensitivity2 %
Specificity3 %
10.0 10.0 15.0 35.0 60.0 75.0(56.0-94.0) 90.0(76.9-100.0) lOO.O(-) 100.0(-)
100.0 100.0 100.0 100.0 99.0 96.2(92.5-99.9) 89.4 (83.5-95.3) 71.2(62.5-79.9) 39.4 (30.0-48.8) 13.5
false true
18 18 17 13 8 5 2 0 0 0 0
104 104 104 104 103 100 93 74 41 14 0
100.0 100.0
0.0
Table 1 reports the age and sex, and the education and sex distributions of the com plete study sample, and, separately, of the 20 demented patients. In the overall sample, 18 women and 3 men (16.9%) had no formal education or had not reached the 3rd grade of elementary school; 14 of them were unable to read or write. Six subjects (4.8%) were living in a nursing home at the time of the study; 8 subjects (6.4%) reported moderate visual loss, and 8 (6.4%) moderate hearing loss. Of the 20 demented patients, 8 were aged 70-79 years, 12 were 80-89 years old. Twelve subjects had no formal education or did not reach 3rd grade. Measures of cognitive and behavioral
performance were distributed in the 20 de mented subjects as follows: (1) the MMS me dian score was 12 (range 3-22); (2) the median score on the Blessed-Roth information-memo ry-concentration test was 15 (range 6-31), and (3) the median score on the Blessed-Roth de mentia scale was 13 (range 2-21). Table 2 shows the number of subjects who screened positive or negative at different cut off levels of the AMT and the corresponding sensitivity and specificity. Although a score of 6 (4 or more errors) showed the best combina tion of sensitivity and specificity, only the score of 7 (3 or more errors) gave 100% sensi tivity. The trade-off between sensitivity and
291
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
1 The test result is positive (+) if the score is smaller than or equal to the screening level, negative (-) if otherwise. 2 Sensitivity is the proportion of truly diseased persons in the screened population who are identified as diseased by the screening test. Estimates of sensitivity restricted to the probability sample were as follows: screen ing level 0 = 0.0; 1 - 0.0; 2 = 0.0; 3 = 0.0; 4 = 40.0; 5 - 60.0; 6 = 80.0; 7 = 100.0; 8 = 100.0; 9 = 100.0; 10=100.0. Values in parentheses are approximate 95% confidence intervals, and are reported only for screening levels around the one selected. 3 Specificity is the proportion of truly nondiseased persons who are so identified by the screening test. Estimates of specificity do not change by restricting the sample to the probability sample. Values in parentheses are approx imate 95% confidence intervals, and are reported only for screening levels around the one selected.
SPECIFICITY (%) 100
90
Fig. 1. Receiver-operator-charactcrislic curve describing the ac curacy of the AMT as a screening instrument for dementia.
80
70
60
50
40
30
20
10
0
1-SPECIFICITY (%)
Table 3. Specificity of the AMT at various screening levels by age. sex, and education
Screening Total level12 (n-104) %
3 4 5 6 7 8 9 10
Specificity, %2 by education
by sex
by age 60-69 years (n=50)
70+ years (n=54)
men (n=44)
women (n=60)
over 4th 4 th grade or less grade (n=63) (n=41)
100.0 100.0 98.0 90.0 82.0 46.0 14.0
100.0
100.0
98.2 94.4 88.9 61.1 33.3 13.0
100.0 100.0 100.0 93.2 52.3 20.5
100.0 98.3 93.3 81.7 55.0 30.0 8.3
100.0 100.0 100.0 97.6 90.2 58.5 24.4
100.0
99.0 96.2 89.4 71.2 39.4 13.5
0.0
0.0
0.0
0.0
0.0
0.0
0.0
100.0
98.4 93.6 84.1 58.7 27.0 6.3
292
Rocca/ Bonaiuto/Lippi/Luciani/ Pistarelli/Grandinetti/Cavarzeran/ Amaducci
Validity of Dementia Screening
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
1 The lines for scores 0.1. and 2 are not reported because they were identical to the line for a score of 3. 2 Specificity is the proportion of truly nondiseased persons who are so identified by the screening test.
Discussion In the present study, we investigated the validity of the AMT as a screening test for dementia. Although a two-phase prevalence survey of dementia in the population of Appignano, Italy [4] provided the incentive for this validation, our results may be relevant to future epidemiologic studies of dementia in our country and abroad. The AMT was se lected for the following reasons: (1) it is brief, well tolerated, and easy to administer by non medical personnel; (2) it does not require reading or writing; (3) it can be easily adapted to the Italian culture, and (4) it was shown to correlate well with the Blessed-Roth informa tion-memory-concentration test [6] in both institutionalized [3, 12] and living-at-home populations [13]. One important limitation of
the AMT, which was originally designed for hospital or institutional populations, is the loss of sensitivity of some items when the test is administered at home. Our results suggest that the 2 items ‘Name of place’ and ‘Recogni tion of two people’ did not contribute no ticeably to the screening validity of the AMT. Although the AMT is a subset of the Blessed-Roth information-memory-concen tration test that was used in the clinical evalua tion, the screening test can be considered inde pendent from the standard for comparison selected for our study. The actual standard for comparison was a diagnosis of dementia based on DSM-1II criteria, and this diagnosis was reached using a combination of cognitive, be havioral, and historical evidence. In order to confirm the presence of a cognitive decline, the specialist used two tests, one of which was the MMS. In addition, the degree of behavioral decline was assessed independently using the Blessed-Roth dementia scale. Since a major objective of this study was to validate the AMT on a sample representative of the target population, we first chose an unselected group of subjects (the probability sample). An advantage of this sample was to reproduce on a small scale the target popula tion and, therefore, to include subjects with mild dementia or with other conditions which could interfere with the detection of demen tia. The principal limitation of this represen tative sample was the small number of sub jects found to be simultaneously affected by dementia and able to undergo a cognitive test. Of the 10 demented patients found in the probability sample, 5 could not be formally tested; therefore, our sample size was ade quate to investigate the specificity of the test but insufficient to investigate its sensitivity. Since available resources did not allow us to study a broader probability sample, a series of known cases of dementia living in the same municipality was added at a later time. As a
293
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
specificity is well described by the receiveroperator-characteristic curve (fig. 1); it shows graphically that the best balance between sen sitivity and specificity corresponded to a score of 6. Sensitivity estimates obtained by restricting the sample to the probability seg ment are also reported in table 2; estimates of specificity did not change following such re striction. The influence of age, sex, and education on the specificity of the AMT is shown in table 3. Specificity was higher for all cutoff levels in men compared to women, in younger com pared to older, and in more educated com pared to less educated subjects. Table 4 shows the sensitivity and specific ity of individual items of the AMT. Two items, ‘Name of place’ and ‘Recognition of two people’ had the lowest sensitivity. The best balance between sensitivity and specific ity was reached by the items ‘Year’, ‘Year of World War T, ‘Count backwards 20-1’, and ‘Date of birth’.
Table 4. Sensitivity and specificity o f individual item s o f the AM T (n = 124)
Item
total
Age Time (hour) 5-min address recall Year Name of place (home/institution) Recognition of two people Date of birth Year of World War I Name of present president Count backwards 20-1
Negative1
Positive1
false true
true false + +
total n
%
Sensi Speci tivity2 ficity3 % %
n
%
14 10 84 26
11.3 8.1 67.7 21.0
10 8 20 14
4 2 64 12
110 114 40 98
88.7 91.9 32.3 79.0
10 12 0 6
100 102 40 92
50.0 40.0 100.0 70.0
96.2 98.1 38.5 88.5
3 3 17 56 83 22
2.4 2.4 13.7 45.2 66.9 17.7
2 3 12 18 20 12
1 0 5 38 63 10
121 121 107 68 41 102
97.6 97.6 86.3 54.8 33.1 82.3
18 17 8 2 0 8
103 104 99 66 41 94
10.0 15.0 60.0 90.0 100.0 60.0
99.0 100.0 95.2 63.5 39.4 90.4
consequence of this mixed-sample design, only our estimates of specificity can be con sidered as being population based. The esti mates of sensitivity were based mainly on clinically recognized cases; therefore, we may have overestimated the sensitivity of the AMT compared to its use in the general popu lation [ 11 ]. If we restricted the sample to the 5 cases of dementia included in the probability sample, sensitivity was lower at correspond ing cutoff scores; however, the best balance between sensitivity and specificity remained at a score of 6. Findings from the complete sample were consistent with those suggested by the representative sample. Since we excluded from the study 11 sub jects who could not be formally tested by non medical personnel because of severe demen tia, we investigated the ability of the AMT to detect cases of intitial or moderately ad vanced dementia. This restriction has proba bly lowered our sensitivity estimates. On the
294
other hand, to include subjects with terminal stages of the disease in the validation, one needs to arbitrarily assign them a score of 0 on the AMT; this approach would artificially in crease the estimates of sensitivity. Since cases of advanced dementia can be detected in the population without any formal instrument [14], they may conveniently be excluded when measuring the screening accuracy of a cognitive test. We found the screening level of 6 (4 or more errors) to give the best balance of sensi tivity and specificity. Consistently, Hodkinson, in his original study, found that the AMT score of 6 corresponded to the cutoff score of 24 on the Blessed-Roth test [3], At all cutoff levels, specificity was higher in men, younger, and more educated subjects. The difference between men and women was probably ex plained by the fact that women tended to be both older and less educated than men in our sample.
Rocca/Bonaiulo/LippL/Luciani/ Pistarelli/Gfandinetti/Cavarzeran/ Amaducci
Validity of Dementia Screening
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
1 The item was positive (+) if the subject did not respond correctly, negative (-) if the subject responded correctly. 2 Sensitivity is the proportion of truly diseased persons in the screened population who are identified as diseased by the screening item. 3 Specificity is the proportion of truly nondiseased persons who are so identified by the screening item.
When using the AMT as a screening instru ment for dementia in a two-phase prevalence survey, maximum sensitivity is highly desir able even at the expense of some loss in speci ficity. Therefore, we chose the screening level of 7 (7 or less) for our prevalence survey of dementia [4]. A decision on the screening level may vary according to the size of the target population. We found that a cutoff score of 6 screened positive only 14% of the probability segment of our sample, while a cutoff score of 7 screened positive 32% of the sample. For a relatively small population like that in Appignano (n = 778 people > 59 years old), the use of a 100% sensitive screening level was possi ble since the corresponding loss of specificity was still compatible with available resources for the study. On the other hand, when the target population is much larger, it may be practical to use the best compromise between sensitivity and specificity. In this case, the
risk of losing some of the patients affected by dementia in the population should be weighed against the reduction in time and costs of the survey. Our study suggests that brief cognitive tests like the AMT may be adequate to screen for dementia in Italy, although maximum sensi tivity can only be reached at the expense of some loss in specificity. Acknowledgements This study was supported in part by the Istituto Nazionalc di Riposo e Cura per Anziani (INRCA), Istituto di Ricerca a Carattere Scientifico, Grant No. 1033 (June 16. 1988), and was facilitated by collabora tion with the Amministrazione comunale di Appignano and the Unita’ Sanitaria Locale N. 15 della Rcgione Marche. The authors thank Dr. Lucia Galluzzo for the general study coordination, Emanuela Manzella for administering the screening instrument to a series of demented subjects. Dr. Francesco Grigoletto for help in data analysis, and Adriana Frasson for technical assistance in preparing the illustration.
1 Henderson AS, Jorm AF: Is caseascertainment of Alzheimer’s dis ease in field surveys practicable? Psychol Med 1987;17:549-555. 2 Rocca WA, Amaducci L; Epidemi ology of Alzheimer’s disease: in An derson DW (ed): Neuroepidemiolo gy: A Tribute to Bruce Schoenberg. Boca Raton, CRC, 1991, pp 55-96. 3 Hodkinson HM: Evaluation of a mental test score for assessment of mental impairment in the elderly. Age Ageing 1972;1:233-238. 4 Rocca WA, BonaiuloS, Lippi A, Luciani P, Turtù F, Cavarzeran F. Amaducci L: Prevalence of clini cally diagnosed Alzheimer’s disease and other dementing disorders: A door-to-door survey in Appignano, Maccrata Province, Italy. Neurol ogy 1990;40:626-631.
5 Last JM: A Dictionary of Epidemi ology. ed 2. New York, Oxford Uni versity Press, 1988. 6 Blessed G, Tomlinson BE. Roth M: The association between quantita tive measures of dementia and of senile change in the cerebral grey matter of elderly subjects. Br J Psy chiatry 1968; 114:797-811. 7 American Psychiatry Association: Diagnostic and Statistical Manual of Mental Disorders, ed 3. Washing ton. American Psychiatric Associa tion, 1980. 8 Folstein MF, Folstein SE, McHugh PR: ‘Mini-mental state’. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975;12:189-198. 9 Hamilton M: Development of a rat ing scale for primary depression ill ness. Br J Soc Clin Psychol 1967:6: 278-296.
10 Ritchie K: The screening of cogni tive impairment in the elderly: A critical review of current methods. J Clin Epidemiol 1988:41:635-643. 11 Fletcher RH. Fletcher SW. Wagner EH: Clinical Epidemiology: The Es sentials, ed 2. Baltimore, Williams and Wilkins, 1988. 12 Qureshi K.N. Hodkinson HM: Eval uation of a ten question mental test in the institutionalized elderly. Age Ageing 1974;3:152-157. 13 Thompson P. Blessed G: Correla tion between the 37-item mental test score and abbreviated 10-item men tal test score by psychogeriatric day patients. Br J Psychiatry 1987:151: 206-209. 14 Schoenberg BS, Anderson DW, Haercr AF: Severe dementia. Preva lence and clinical features in a biracial US population. Arch Neurol 1985;42:740-743.
295
Downloaded by: Univ. of California Santa Barbara 128.111.121.42 - 5/19/2018 8:39:06 AM
References
