Clinical Neurology and Neurosurgery 123 (2014) 55–60
Contents lists available at ScienceDirect
Clinical Neurology and Neurosurgery journal homepage: www.elsevier.com/locate/clineuro
The squares test as a measure of hand function in multiple sclerosis Jeroen Gielen a, *, Jorne Laton a , J. Van Schependom a,f , P.P. De Deyn b,c,d,e, Guy Nagels f,g a
Center for Neurosciences, C4N, Faculty of Medicine and Pharmaceutical Sciences UZ Brussel, Laarbeeklaan, 101, Brussels 1090, Belgium Alzheimer Research Center Groningen, University Medical Center Groningen, the Netherlands Institute Born-Bunge, University of Antwerp, Belgium d Department of Neurology ZNA Middelheim & Memory Clinic, Antwerp, Belgium e IBB, UA, Campus Drie Eiken, Universiteitsplein 1, Building T, 6th floor, Wilrijk 2610, Belgium f Faculty of Medicine and Pharmaceutical Sciences, Service d'Orthopédagogie Clinique, Faculté de Psychologie et des Sciences de l'Education, Université de Mons, Place du Parc 20, Mons 7000, Belgium g National MS Center, Melsbroek, Belgium b c
A R T I C L E I N F O
A B S T R A C T
Article history: Received 23 December 2013 Received in revised form 31 March 2014 Accepted 8 May 2014 Available online 20 May 2014
Deterioration of hand function can be important in multiple sclerosis (MS). The standard way of assessing hand function in MS is the 9-hole peg test (9HPT), one of the three components of the MS functional composite measure. In this study we examine the squares test (ST), a test of hand function that is used extensively in handedness research. We evaluated reproducibility of the ST in 49 healthy controls, and both discriminatory power and concurrent validity of the ST in 38 MS patients and 18 age and gender matched controls. The ST proved to be a reliable and easy to administrate paper-and-pencil test of hand function. The ST showed a high and highly significant correlation with the standard 9HPT over a broad range of Expanded Disability Status Scale (EDSS) scores, and had high discriminatory power, also comparable to the 9HPT. Therefore, the ST is a candidate test for use in composite measures of MS related functional deficits for clinical practice and in clinical trials. ã 2014 Elsevier B.V. All rights reserved.
Keywords: Multiple sclerosis Functional composite Hand function Nine-hole peg test Squares test
1. Introduction Hand function in multiple sclerosis (MS) patients is commonly impaired due to cerebellar dysfunction. The chance of symptom remission is low and treatment has not been effective [1 ]. Hand function is commonly assessed using the 9 hole peg test [2]. This is one of three components in the multiple sclerosis functional composite (MSFC) [3] which has already been used as an outcome measure in large clinical trials in MS [4]. Follow-up of MS patients in clinical trials and in clinical practice often involves the use of several scales and tests. Using simpler and shorter tests therefore decreases the time and energy which both patients and staff have to use for these assessments. The 9-hole peg test (9HPT) has a time limit of 5 min per trial. Both hands are tested
twice, which means that the test can last for 20 min plus setup time [5]. The squares test (ST) was described as a paper-and-pencil test of hand function, which correlates with other performance measures in the sidedness field, such as pegboard tests [6]. The ST is simpler and shorter than the 9HPT, but has not yet been validated in MS. The goal of this study was to test whether the ST is a reproducible test, whether the discriminatory power of the ST and 9HPT are comparable between MS patients and matched controls, and whether 9HPT and ST are significantly correlated in an MS population. If the ST complies with these standards, it would do well as a part of the MSFC. 2. Materials and methods 2.1. Subjects
* Corresponding author at: UZ Jette, Laarbeeklaan, 101, Brussels 1090, Belgium. Tel.: +32474301789. E-mail addresses: [email protected] (J. Gielen), [email protected] (J. Laton), [email protected] (J. Van Schependom), [email protected] (P.P. De Deyn), [email protected] (G. Nagels). http://dx.doi.org/10.1016/j.clineuro.2014.05.008 0303-8467/ ã 2014 Elsevier B.V. All rights reserved.
The hospital ethics committee approved the study protocol. Patients were included after having given their written informed consent. They were selected from the population that attended the rehabilitation program in the National Multiple Sclerosis Center (NMSC) in Melsbroek, Belgium. Patients were only included if they
56
J. Gielen et al. / Clinical Neurology and Neurosurgery 123 (2014) 55–60
had a clinically definite diagnosis of multiple sclerosis according to the revised McDonald criteria [7], and were enrolled in the NMSC rehabilitation program. Normal controls were recruited among center personnel and college students. In total, 105 subjects were included in three panels: MS patients, matched controls and test-retest control subjects. The MS panel (“MS patients”) included 37 MS patients (20 men, 17 women), and the concurrent matched control panel (“matched control group”) numbered 18 healthy volunteers (9 men, 9 women). The test-retest panel (“TRT control”) consisted of 49 healthy controls (14 men, 35 women). One MS patient was removed due to being an outlier. Average MS patient age was 54 11 year (range 23–75). Average age in the matched control group was 50 14 years (range 22–75) and 21 2 years (range 18–24) in the TRT control group. Median score on the Expanded Disability Status Scale (EDSS) in the patient group was 7 (range 3–8). Eight patients had relapsing-remitting MS, 18 had secondary progressive MS and 12 suffered from primary progressive MS. The MS panel in turn was divided into four groups based on their EDSS score. A summary of the patient group demography is show in Table 1. 2.2. Testing Subjects underwent the 9HPT, and the ST [6]. In the 9HPT [8], arm- and hand function is measured using a pegboard. Subjects are requested to pick up nine pegs one at a time, and place them in the holes of the pegboard as quickly as possible, in any order. Then, without pausing, the subject has to remove all pegs and place them in the container, one at a time. Subjects can use only one hand in each test session, and may use the other hand to stabilize the pegboard. Timing starts as soon as the subject touches the first peg, and stops when the last peg hits the container. Each hand is tested twice [5]. The squares test is slightly modified from Annett, who described it as a measure of handedness [6]. It consists of a white sheet of paper, on which four grids are printed. The top and bottom of the page show a 20 (horizontal) by 3 (vertical) grid of 6 6 mm squares. The middle of the page shows two 20 (horizontal) by 10 grids of 6 6 mm squares. Subjects are instructed to first use their right hand to fill out the upper 20 3 grid, and place a dot inside as many squares as possible during a 10 s practice trial. They are requested to start in the upper row of the grid, and complete each row from left to right. Then they are given 30 s to complete the second grid, which has 20 10 squares. In the next step the examiner turns the page over 180 , so that the unused grids are now on top of the page. Subjects then complete the practice grid and the second 20 10 grid with their left hand, starting with the top row and working from right to left for each row. An example of the ST can be seen in Fig. 1. In addition, patients were evaluated using the 25-foot walk test (25-FWT) and the Paced Auditory Serial Addition Test (PASAT). These tests were administered according to the instructions in the MSFC manual [5].
The Edinburgh Inventory (EI) was used to determine handedness [9]. Subjects were also asked to name their dominant hand, according to the MSFC manual [5]. All patients were assessed with the Expanded Disability Status Scale [10]. Visual acuity was measured using a Snellen chart [11]. 2.3. Statistics Statistical analyses were performed in R [12] and MATLAB R2012b(r). All numbers are listed as mean standard deviation, unless otherwise specified. Pearson correlation coefficients were used to examine associations between measurements. Variance comparisons were done with the two-sample F-test for equal variances. P-values are given as returned by the respective correlation and variance MATLAB function to indicate the significance of the results. Receiver operating characteristic (ROC) analyses were performed to compare the performance of both tests as binary classifier (between patient and normal control). This analysis compares the sensitivity and specificity of both tests: the fractions of true and false positives compared to the total actual positives and negatives respectively. A high value for both measures indicates a capable classifier. Additionally, the ROC area under the curve (AUC) was calculated as a measure to compare both tests. A high AUC value corresponds with a high test accuracy. 3. Results ST scores showed normal but different distributions for MS patients and controls on the Lilliefors test. In the patient group, mean scores for the ST were 37.40 11.56 dots and mean z-scores for the 9HPT were 0.031 0.0082 s 1. Correlation between both tests was good and highly significant (r = 0.63, p < 0.0001). Fig. 2 shows the test scores for the patients, including EDSS values. In the matched control group, mean scores for the ST were 61.06 13.76 dots and mean z-scores for the 9HPT were 0.049 0.0068 s 1. Correlation between both tests was good and significant (r = 0.50, p = 0.034). Fig. 3 shows the above results for both the patients and the matched controls. In the (younger) TRT group, mean scores for the ST were 76.29 12.91 dots and mean z-scores for the 9HPT were 0.056 0.0048 s 1. Correlation between both tests was low (r = 0.28, p = 0.05). In the total group of 104 subjects, the two test scores correlated highly (r = 0.84, p < 0.0001). The control groups combined also showed a good, highly significant correlation (r = 0.51, p < 0.0001). Variance was also examined separately for the dominant and non-dominant hands, for both 9HPT and ST with two-sample F-tests for equal variances. There were no significant differences in variance between dominant and non-dominant hand for 9HPT and ST in any of the three subject groups (all p values > 0.2). Visual acuity of the best eye ranged from 20/15 to 20/200 (median 20/40). All patients were able to complete both 9HPT and ST, even patients with poor vision. Fig. 4 shows the testing scores
Table 1 EDSS range
Age
Female (%)
Right hand dominant (%)
Right visual acuity
Left visual acuity
Disease duration
Relapse remitting (%)
Secondary progressive (%)
Primary progressive (%)
3–6 6–6.5 6.5–7 7–8
53 16 54 13 54 9 55 8
43 57 20 62
100 86 100 100
6.3 1.6 5.6 1.8 6 2.1 5.9 1.6
6 1.8 5.6 2.4 6.9 2.2 6.3 2.2
19 13 30 16 14 7.7 17 9.3
29 14 20 15
43 71 40 46
29 14 40 38
J. Gielen et al. / Clinical Neurology and Neurosurgery 123 (2014) 55–60
57
Fig. 1. Squares test examples. Two examples of the ST made by patients outside the subject group. Patients have one practise table and one testing table for each hand. Patient (a) filled in 40 squares with the right hand, 27 with the left and spent 65 s on the 9HPT. Patient (b) filled in 10 squares with the right hand, barely any correctly with the left and spent 101 s on the 9HPT. These figures were trimmed down to remove unfilled squares.
Fig. 2. Correlation of the 9HPT and ST scores of the MS patients. The x axis shows the inverse of the 9HPT score because we expect it to rise with increasing disability, while we expect the ST score on the y axis to decrease. The scores show a high correlation. EDSS values are colour coded on the data points.
Fig. 3. Correlation of the 9HPT and ST scores of the MS patients and controls. Analogue to Fig. 2 for both patients and matched controls. EDSS values have been removed. The straight blue line depicts a linear regression line where the shaded area is its confidence interval. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
correlation between ST and visus
correlation between 9hpt and visus
9
●● ●
● ●
6
●
● ● ●
●
● ●● ●
●
●
●
3
25
type
50 Squares test (number of dots) ●
Controls
●
Visual acuity in best eye
Visual acuity in best eye
●
9
●
●
●
●●
●
●
● ●●
6
●
● ●
●
● ●
●
3
75
Patients
0.02
0.03 0.04 0.05 1 / 9hpt (1/s)
type
●
Controls
0.06
Patients
Fig. 4. Visual acuity. In this figure the visual acuity scores, based on a Snellen chart, are compared with the ST and 9HPT scores on the left and right respectively. Visual acuity shows low correlation with the test scores for both tests and both subject groups.
58
J. Gielen et al. / Clinical Neurology and Neurosurgery 123 (2014) 55–60
Test − retest evaluation of the squares test ● ●
110 ● ●
second measurement (number of dots)
compared to the visual acuity score of the best eye, based on a Snellen chart. Table 2 gives the correlation values for patients and controls. Correlations are low and not significantly different from zero. The forty-nine TRT controls were re-evaluated with the ST, within four weeks of the first assessment. The correlation between these two measures was good and highly significant (r = 0.67, p < 0.0001, Fig. 5). In 40 out of 49 subjects, the difference between the two measurements was smaller than 20% of the baseline value. The average difference between the two measurements was 11.5% 10.2 of the baseline value. Both ST and 9HPT showed excellent discriminatory power between MS patients and the matched control group, as shown in Fig. 6. The AUC was 0.9145 for ST and 0.9532 for 9HPT. Fig. 7 shows box plots for the ST and 9HPT scores over four different EDSS ranges. There is no statistically significant way to differentiate between the EDSS ranges based on the testing scores, most box plots have a significant overlap. Neither is there a correlation between the tests and the EDSS score. Especially the 6– 6.5 range shows a high variability in both hand function scores.
100
● ● ● ●
● ●
90
● ●
80
● ●
● ●
● ●
70
● ● ● ● ● ● ● ●
● ● ● ● ● ●● ● ● ● ● ●
● ● ● ● ● ●
● ● ● ●
● ●
● ● ● ●
● ●
● ●
● ● ● ●● ● ● ● ● ●
● ●
60
80
100
first measurement (number of dots) Fig. 5. Test retest evaluation. Test and retest evaluation of the ST scores of the TRT controls (n = 49). The scores on the y axis were achieved four weeks after those on the x axis. They have a good correlation. The straight blue line depicts a linear regression line where the shaded area is its confidence interval. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
ROC differentiation between MS patients and matched controls
0.2
0.4
0.6
0.8
1.0
wanted to create an easy-to-use test of hand function that depended on aiming at targets, which is also necessary in a pegboard test. In the design of the test, care was taken to avoid using an apparatus that was cumbersome to produce or move. Also, the
9HPT versus visual acuity in the best eye
Correlation
Correlation p-value
Patients Controls Both
0.1354 0.3698 0.4634
0.4244 0.1310 0.0004
Correlation
Correlation p-value
0.1777 0.2510 0.4800
0.2927 0.3150 0.0002
SQUARES 9HPT
0.0
Table 2
ST versus visual acuity in the best eye Patients Controls Both
● ●
● ●
● ●
Sensitivity
For the measurement of functional disability in MS, the expanded disability status score [10], 1983) is still widely used, despite its limitations [13]. An attempt to improve upon the EDSS as a measure of disease progression in MS was made by the construction of the multiple sclerosis functional composite [14]. This is a composite measure which includes a test of walking function (25 foot walk test), but also tests of cognitive (Paced Auditory Serial Addition Test, PASAT) and hand function. In the construction of the MSFC, three candidate measures for upper limb function were assessed: the Arm Box and blocks test, the 9HPT and the Purdue pegboard test. The existing data that was available for retrospective review by the international panel constructing the MSFC favoured inclusion of the 9HPT [14], but the consensus panel clearly recommended that new measures should be tested as potential replacement tests in the MSFC: “An attractive characteristic of the functional composite outcome measure is that measures within a clinical dimension are interchangeable; for example, if a performance test superior to the nine hole peg test or the box and block test is developed it may be substituted for the arm dimension.” [14]. Replacement tests have been proposed for the ambulation component of the MSFC [15] and for the cognitive component [16]. As a candidate replacement for the 9HPT, Bonzano et al. [13] quantitatively measured finger opposition movements as a test for disability impact on fine hand motor functions. In our study, we examined whether a test of hand function that has been used for several decades in handedness research, the ST, also has potential as a test of hand function in MS. Annett [6] designed the squares test as a group test for hand skill assessment that would resemble a peg moving task. She
● ● ● ●
● ●
● ●
● ● ● ●
● ●
4. Discussion
● ●
● ●
● ●
1.0
0.8
0.6
0.4
0.2
0.0
Specificity Fig. 6. ST and 9HPT receiver operating characteristics. Receiver operating characteristic curves for the ST and 9HPT showing a measure of the rate of true positives versus true negatives at varying thresholds. Both tests show a high discriminatory value and AUC.
J. Gielen et al. / Clinical Neurology and Neurosurgery 123 (2014) 55–60
A) Squares test
60
B) 9 hole peg board test
50
0.04 9HPT score
ST score
59
40 30 20
0.03
0.02
10 3−6
6−6.5 6.5−7 EDSS range
7−8
3−6
6−6.5 6.5−7 EDSS range
7−8
Fig. 7. Relation between EDSS hand function scoring. Box plots of the (A) ST and (B) 9HPT scores for varying ranges of EDSS scores. If the EDSS was truly a measure for general disability in MS, one would expect both the ST and the inverse of the 9HPT to decrease with increasing EDSS score. Yet, no significant correlation can be detected.
test had to be easy to score [6]. A potential advantage is that the ST leaves a record of performance, namely the filled-out test form, which can be a plus in the documentation of source data in clinical trials. The first element of validity we examined was the reliability of the test results. In the analysis of test-retest reliability (Fig. 2), no floor or ceiling effects (defined as more than 20% of tested patients have minimal or maximal score [17]) were observed. Correlations between a ST at baseline, and a second ST within four weeks in a healthy control group were high and highly significant (Fig. 2). Since the ST involves marking targets on paper, the dominant hand might be more practiced than the non-dominant hand. In children, the variance in ST score for the non-dominant hand was therefore higher than for the dominant hand [6]. This was not the case in our three adult subject groups. Although the ST resembles a pegboard task in some ways, there are also differences: in the ST, the manipulated object (pencil) is held in a normal grip for writing [6], while pegs in the 9HPT are held in a pincer grip. Annett [6] also remarked that in the ST, the pen is continuously held during the test, while in the 9HPT, pegs are grasped and released several times. Also, the degrees of freedom in the ST might be lower than in the 9HPT if the subject’s forearm rests on the table during the test. Because of these differences in both tasks, it was necessary to examine whether both tests measure the behaviour of interest, hand function, in a comparable way. Since age has a significant effect on hand strength, precision and speed [18], we used an age matched control group for this part of the study. In this age group, one might expect some to perform poorer than others due to worse visual acuity. This was not the case in our study group: correlation of visual acuity with test results was low for both patients and controls. Correlation was higher when both groups were combined. This can be explained by a general trend of both test scores and visual acuity being higher in the control group. In the examination of correlations between tests, we labelled correlations up to 0.3 as low, from 0.31 to 0.69 as moderate and from 0.7 as high [17]. Both ST and 9HPT perform very well in the discrimination between MS patients and matched healthy controls, as is evident from the ROC analysis in Fig. 6. This supports the validity of the ST in our application. A higher score on the 9HPT (seconds needed to complete the test) means a worse performance. In the MSFC, the 9HPT is inverted for this reason [14], but an inversion of the ST score is not necessary, because a higher score on the ST (number of dots
correctly placed) means a better performance on the test. Since we expect an inverse correlation between ST and 9HPT raw scores, 1/ 9HPT was plotted against ST score in Fig. 3. This plot shows a high correlation, confirming the concurrent validity of both tests. Divergent validity is supported by the lack of correlation between ST and EDSS (Fig. 7 panel A). If the EDSS was in reality a test of general functional loss in MS, the lack of correlation and of a downward trend in the data would be disappointing. In reality however, the EDSS is heavily loaded on ambulatory function, especially in the score range of our subjects. This is supported by the lack of correlation between the 9HPT and EDSS as shown in Fig. 7, panel B. Our study also showed that the ST is feasible and well supported in a broad age and EDSS range in MS patients. It requires almost no setup time since only a pencil and the test form are needed. 5. Conclusions The squares test as presented in this paper can by no means replace a rehabilitation evaluation of upper limb function as it would be done by a physical or occupational therapist in a rehabilitation context. However, the ST is a reliable test of hand function. It is a brief and easy to administer paper-and pencil-test that can be used in a broad EDSS range. It has a high test–retest correlation, and correlates highly with an accepted measure of hand function in MS, the 9HPT. The ST is simpler and shorter to perform than the 9HPT, and it is more closely related to daily activities like writing. In addition, no material apart from pen and paper is needed, and a record of the test is easily kept. Therefore the ST is a candidate to be used in composite measures of MS related functional deficits for clinical practice and in clinical trials. Future work The lowest EDSS score in our patient group was 3, which is still relatively severe. It would be interesting to look at groups of patients that are more moderately affected, and see if the two tests show the same correlation as they do in our current group. Secondly, MS patients do show variability in functional performance during the day. An interesting approach would be to look at ST and 9HPT results at different times of the day. Lastly, we hope that our research stimulates the rehabilitation community to do further work in evaluating the ST as a tool to detect impairment at an earlier stage, to see if rehabilitation can start sooner, in a more focused and personalized way.
60
J. Gielen et al. / Clinical Neurology and Neurosurgery 123 (2014) 55–60
Acknowledgements GN holds the “BiogenIdec-National MS Center Melsbroek” named chair, and the “Merck–Novartis” named chair, both at the Center for Neurosciences at the Vrije Universiteit Brussel, Belgium. JVS is a pre-doctoral scholar (aspirant) of the FWO. We gratefully acknowledge the assistance of Mrs Ann Van Remoortel and the other MS and research nurses from the National MS Center Melsbroek. References [1] Shields SD, Cheng X, Gasser A, Saab CY, Tyrrell L, Eastman EM, et al. A channelopathy contributes to cerebellar dysfunction in a model of multiple sclerosis. Ann Neurol 2012;71:186–94. [2] Goodkin DE, Hertsgaard D, Seminary J. Upper extremity function in multiple sclerosis: improving assessment sensitivity with box-and-block and nine-hole peg tests. Arch Phys Med Rehabil 1988;69:850–4. [3] Fischer JS, Rudick RA, Cutter GR, Reingold SC. The multiple sclerosis functional composite measure (MSFC): an integrated approach to MS clinical outcome assessment. National MS Society Clinical Outcomes Assessment Task Force. Mult Scler 1999;5:244–50. [4] Ontaneda D, LaRocca N, Coetzee T, Rudick R. Revisiting the multiple sclerosis functional composite: proceedings from the national multiple sclerosis society (NMSS) task force on clinical disability measures. Mult Scler 2012;18:1074–80. [5] Fischer J, Jak A, Kniker J, Rudick R, Cutter G. Multiple sclerosis functional composite (MSFC): administration and scoring manual. National MS Society 2001. [6] Annett M. Five tests of hand skill. Cortex 1992;28:583–600.
[7] Polman CH, Reingold SC, Banwell B, Clanet M, Cohen a J, Filippi M, et al. Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann Neurol 2011;69:292–302. [8] Goodkin D, Priore R, Wende K. Comparing the ability of various compositive outcomes to discriminate treatment effects in MS clinical trials. Mult Scler 1998;4:480–6. [9] Oldfield R. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 1971;9:99–113. [10] Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an Expanded Disability Status Scale (EDSS). Neurology 1983;33:1444–52. [11] Kniestedt C, Stamper RL. Visual acuity and its measurement. Ophthalmol Clin North Am 2003;16:155–70. [12] R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2013. [13] Bonzano L, Sormani MP, Tacchino A, Abate L, Lapucci C, Mancardi GL, et al. Quantitative assessment of finger motor impairment in multiple sclerosis. PLoS One 2013;8:e65225. [14] Cutter GR, Baier ML, Rudick a R, Cookfair DL, Fischer JS, Petkau J, et al. Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain 1999;122(5):871–82. [15] Goldman MD, Motl RW, Rudick R. A possible clinical outcome measures for clinical trials in patients with multiple sclerosis. Ther Adv Neurol Disord 2010;3:229–39. [16] Drake AS, Weinstock-Guttman B, Morrow SA, Hojnacki D, Munschauer FE, Benedict RHB. Psychometrics and normative data for the multiple sclerosis functional composite: replacing the PASAT with the symbol digit modalities test. Mult Scler 2010;16:228–37. [17] Schäffler N, Schönberg P, Stephan J, Stellmann J-P, Gold SM, Heesen C. Comparison of patient-reported outcome measures in multiple sclerosis. Acta Neurol Scand 2013;128:114–21. [18] Ranganathan VK, Siemionow V, Sahgal V, Yue GH. Effects of aging on hand function. J Am Geriatr Soc 2001;49:1478–84.
Contents lists available at ScienceDirect
Clinical Neurology and Neurosurgery journal homepage: www.elsevier.com/locate/clineuro
The squares test as a measure of hand function in multiple sclerosis Jeroen Gielen a, *, Jorne Laton a , J. Van Schependom a,f , P.P. De Deyn b,c,d,e, Guy Nagels f,g a
Center for Neurosciences, C4N, Faculty of Medicine and Pharmaceutical Sciences UZ Brussel, Laarbeeklaan, 101, Brussels 1090, Belgium Alzheimer Research Center Groningen, University Medical Center Groningen, the Netherlands Institute Born-Bunge, University of Antwerp, Belgium d Department of Neurology ZNA Middelheim & Memory Clinic, Antwerp, Belgium e IBB, UA, Campus Drie Eiken, Universiteitsplein 1, Building T, 6th floor, Wilrijk 2610, Belgium f Faculty of Medicine and Pharmaceutical Sciences, Service d'Orthopédagogie Clinique, Faculté de Psychologie et des Sciences de l'Education, Université de Mons, Place du Parc 20, Mons 7000, Belgium g National MS Center, Melsbroek, Belgium b c
A R T I C L E I N F O
A B S T R A C T
Article history: Received 23 December 2013 Received in revised form 31 March 2014 Accepted 8 May 2014 Available online 20 May 2014
Deterioration of hand function can be important in multiple sclerosis (MS). The standard way of assessing hand function in MS is the 9-hole peg test (9HPT), one of the three components of the MS functional composite measure. In this study we examine the squares test (ST), a test of hand function that is used extensively in handedness research. We evaluated reproducibility of the ST in 49 healthy controls, and both discriminatory power and concurrent validity of the ST in 38 MS patients and 18 age and gender matched controls. The ST proved to be a reliable and easy to administrate paper-and-pencil test of hand function. The ST showed a high and highly significant correlation with the standard 9HPT over a broad range of Expanded Disability Status Scale (EDSS) scores, and had high discriminatory power, also comparable to the 9HPT. Therefore, the ST is a candidate test for use in composite measures of MS related functional deficits for clinical practice and in clinical trials. ã 2014 Elsevier B.V. All rights reserved.
Keywords: Multiple sclerosis Functional composite Hand function Nine-hole peg test Squares test
1. Introduction Hand function in multiple sclerosis (MS) patients is commonly impaired due to cerebellar dysfunction. The chance of symptom remission is low and treatment has not been effective [1 ]. Hand function is commonly assessed using the 9 hole peg test [2]. This is one of three components in the multiple sclerosis functional composite (MSFC) [3] which has already been used as an outcome measure in large clinical trials in MS [4]. Follow-up of MS patients in clinical trials and in clinical practice often involves the use of several scales and tests. Using simpler and shorter tests therefore decreases the time and energy which both patients and staff have to use for these assessments. The 9-hole peg test (9HPT) has a time limit of 5 min per trial. Both hands are tested
twice, which means that the test can last for 20 min plus setup time [5]. The squares test (ST) was described as a paper-and-pencil test of hand function, which correlates with other performance measures in the sidedness field, such as pegboard tests [6]. The ST is simpler and shorter than the 9HPT, but has not yet been validated in MS. The goal of this study was to test whether the ST is a reproducible test, whether the discriminatory power of the ST and 9HPT are comparable between MS patients and matched controls, and whether 9HPT and ST are significantly correlated in an MS population. If the ST complies with these standards, it would do well as a part of the MSFC. 2. Materials and methods 2.1. Subjects
* Corresponding author at: UZ Jette, Laarbeeklaan, 101, Brussels 1090, Belgium. Tel.: +32474301789. E-mail addresses: [email protected] (J. Gielen), [email protected] (J. Laton), [email protected] (J. Van Schependom), [email protected] (P.P. De Deyn), [email protected] (G. Nagels). http://dx.doi.org/10.1016/j.clineuro.2014.05.008 0303-8467/ ã 2014 Elsevier B.V. All rights reserved.
The hospital ethics committee approved the study protocol. Patients were included after having given their written informed consent. They were selected from the population that attended the rehabilitation program in the National Multiple Sclerosis Center (NMSC) in Melsbroek, Belgium. Patients were only included if they
56
J. Gielen et al. / Clinical Neurology and Neurosurgery 123 (2014) 55–60
had a clinically definite diagnosis of multiple sclerosis according to the revised McDonald criteria [7], and were enrolled in the NMSC rehabilitation program. Normal controls were recruited among center personnel and college students. In total, 105 subjects were included in three panels: MS patients, matched controls and test-retest control subjects. The MS panel (“MS patients”) included 37 MS patients (20 men, 17 women), and the concurrent matched control panel (“matched control group”) numbered 18 healthy volunteers (9 men, 9 women). The test-retest panel (“TRT control”) consisted of 49 healthy controls (14 men, 35 women). One MS patient was removed due to being an outlier. Average MS patient age was 54 11 year (range 23–75). Average age in the matched control group was 50 14 years (range 22–75) and 21 2 years (range 18–24) in the TRT control group. Median score on the Expanded Disability Status Scale (EDSS) in the patient group was 7 (range 3–8). Eight patients had relapsing-remitting MS, 18 had secondary progressive MS and 12 suffered from primary progressive MS. The MS panel in turn was divided into four groups based on their EDSS score. A summary of the patient group demography is show in Table 1. 2.2. Testing Subjects underwent the 9HPT, and the ST [6]. In the 9HPT [8], arm- and hand function is measured using a pegboard. Subjects are requested to pick up nine pegs one at a time, and place them in the holes of the pegboard as quickly as possible, in any order. Then, without pausing, the subject has to remove all pegs and place them in the container, one at a time. Subjects can use only one hand in each test session, and may use the other hand to stabilize the pegboard. Timing starts as soon as the subject touches the first peg, and stops when the last peg hits the container. Each hand is tested twice [5]. The squares test is slightly modified from Annett, who described it as a measure of handedness [6]. It consists of a white sheet of paper, on which four grids are printed. The top and bottom of the page show a 20 (horizontal) by 3 (vertical) grid of 6 6 mm squares. The middle of the page shows two 20 (horizontal) by 10 grids of 6 6 mm squares. Subjects are instructed to first use their right hand to fill out the upper 20 3 grid, and place a dot inside as many squares as possible during a 10 s practice trial. They are requested to start in the upper row of the grid, and complete each row from left to right. Then they are given 30 s to complete the second grid, which has 20 10 squares. In the next step the examiner turns the page over 180 , so that the unused grids are now on top of the page. Subjects then complete the practice grid and the second 20 10 grid with their left hand, starting with the top row and working from right to left for each row. An example of the ST can be seen in Fig. 1. In addition, patients were evaluated using the 25-foot walk test (25-FWT) and the Paced Auditory Serial Addition Test (PASAT). These tests were administered according to the instructions in the MSFC manual [5].
The Edinburgh Inventory (EI) was used to determine handedness [9]. Subjects were also asked to name their dominant hand, according to the MSFC manual [5]. All patients were assessed with the Expanded Disability Status Scale [10]. Visual acuity was measured using a Snellen chart [11]. 2.3. Statistics Statistical analyses were performed in R [12] and MATLAB R2012b(r). All numbers are listed as mean standard deviation, unless otherwise specified. Pearson correlation coefficients were used to examine associations between measurements. Variance comparisons were done with the two-sample F-test for equal variances. P-values are given as returned by the respective correlation and variance MATLAB function to indicate the significance of the results. Receiver operating characteristic (ROC) analyses were performed to compare the performance of both tests as binary classifier (between patient and normal control). This analysis compares the sensitivity and specificity of both tests: the fractions of true and false positives compared to the total actual positives and negatives respectively. A high value for both measures indicates a capable classifier. Additionally, the ROC area under the curve (AUC) was calculated as a measure to compare both tests. A high AUC value corresponds with a high test accuracy. 3. Results ST scores showed normal but different distributions for MS patients and controls on the Lilliefors test. In the patient group, mean scores for the ST were 37.40 11.56 dots and mean z-scores for the 9HPT were 0.031 0.0082 s 1. Correlation between both tests was good and highly significant (r = 0.63, p < 0.0001). Fig. 2 shows the test scores for the patients, including EDSS values. In the matched control group, mean scores for the ST were 61.06 13.76 dots and mean z-scores for the 9HPT were 0.049 0.0068 s 1. Correlation between both tests was good and significant (r = 0.50, p = 0.034). Fig. 3 shows the above results for both the patients and the matched controls. In the (younger) TRT group, mean scores for the ST were 76.29 12.91 dots and mean z-scores for the 9HPT were 0.056 0.0048 s 1. Correlation between both tests was low (r = 0.28, p = 0.05). In the total group of 104 subjects, the two test scores correlated highly (r = 0.84, p < 0.0001). The control groups combined also showed a good, highly significant correlation (r = 0.51, p < 0.0001). Variance was also examined separately for the dominant and non-dominant hands, for both 9HPT and ST with two-sample F-tests for equal variances. There were no significant differences in variance between dominant and non-dominant hand for 9HPT and ST in any of the three subject groups (all p values > 0.2). Visual acuity of the best eye ranged from 20/15 to 20/200 (median 20/40). All patients were able to complete both 9HPT and ST, even patients with poor vision. Fig. 4 shows the testing scores
Table 1 EDSS range
Age
Female (%)
Right hand dominant (%)
Right visual acuity
Left visual acuity
Disease duration
Relapse remitting (%)
Secondary progressive (%)
Primary progressive (%)
3–6 6–6.5 6.5–7 7–8
53 16 54 13 54 9 55 8
43 57 20 62
100 86 100 100
6.3 1.6 5.6 1.8 6 2.1 5.9 1.6
6 1.8 5.6 2.4 6.9 2.2 6.3 2.2
19 13 30 16 14 7.7 17 9.3
29 14 20 15
43 71 40 46
29 14 40 38
J. Gielen et al. / Clinical Neurology and Neurosurgery 123 (2014) 55–60
57
Fig. 1. Squares test examples. Two examples of the ST made by patients outside the subject group. Patients have one practise table and one testing table for each hand. Patient (a) filled in 40 squares with the right hand, 27 with the left and spent 65 s on the 9HPT. Patient (b) filled in 10 squares with the right hand, barely any correctly with the left and spent 101 s on the 9HPT. These figures were trimmed down to remove unfilled squares.
Fig. 2. Correlation of the 9HPT and ST scores of the MS patients. The x axis shows the inverse of the 9HPT score because we expect it to rise with increasing disability, while we expect the ST score on the y axis to decrease. The scores show a high correlation. EDSS values are colour coded on the data points.
Fig. 3. Correlation of the 9HPT and ST scores of the MS patients and controls. Analogue to Fig. 2 for both patients and matched controls. EDSS values have been removed. The straight blue line depicts a linear regression line where the shaded area is its confidence interval. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
correlation between ST and visus
correlation between 9hpt and visus
9
●● ●
● ●
6
●
● ● ●
●
● ●● ●
●
●
●
3
25
type
50 Squares test (number of dots) ●
Controls
●
Visual acuity in best eye
Visual acuity in best eye
●
9
●
●
●
●●
●
●
● ●●
6
●
● ●
●
● ●
●
3
75
Patients
0.02
0.03 0.04 0.05 1 / 9hpt (1/s)
type
●
Controls
0.06
Patients
Fig. 4. Visual acuity. In this figure the visual acuity scores, based on a Snellen chart, are compared with the ST and 9HPT scores on the left and right respectively. Visual acuity shows low correlation with the test scores for both tests and both subject groups.
58
J. Gielen et al. / Clinical Neurology and Neurosurgery 123 (2014) 55–60
Test − retest evaluation of the squares test ● ●
110 ● ●
second measurement (number of dots)
compared to the visual acuity score of the best eye, based on a Snellen chart. Table 2 gives the correlation values for patients and controls. Correlations are low and not significantly different from zero. The forty-nine TRT controls were re-evaluated with the ST, within four weeks of the first assessment. The correlation between these two measures was good and highly significant (r = 0.67, p < 0.0001, Fig. 5). In 40 out of 49 subjects, the difference between the two measurements was smaller than 20% of the baseline value. The average difference between the two measurements was 11.5% 10.2 of the baseline value. Both ST and 9HPT showed excellent discriminatory power between MS patients and the matched control group, as shown in Fig. 6. The AUC was 0.9145 for ST and 0.9532 for 9HPT. Fig. 7 shows box plots for the ST and 9HPT scores over four different EDSS ranges. There is no statistically significant way to differentiate between the EDSS ranges based on the testing scores, most box plots have a significant overlap. Neither is there a correlation between the tests and the EDSS score. Especially the 6– 6.5 range shows a high variability in both hand function scores.
100
● ● ● ●
● ●
90
● ●
80
● ●
● ●
● ●
70
● ● ● ● ● ● ● ●
● ● ● ● ● ●● ● ● ● ● ●
● ● ● ● ● ●
● ● ● ●
● ●
● ● ● ●
● ●
● ●
● ● ● ●● ● ● ● ● ●
● ●
60
80
100
first measurement (number of dots) Fig. 5. Test retest evaluation. Test and retest evaluation of the ST scores of the TRT controls (n = 49). The scores on the y axis were achieved four weeks after those on the x axis. They have a good correlation. The straight blue line depicts a linear regression line where the shaded area is its confidence interval. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
ROC differentiation between MS patients and matched controls
0.2
0.4
0.6
0.8
1.0
wanted to create an easy-to-use test of hand function that depended on aiming at targets, which is also necessary in a pegboard test. In the design of the test, care was taken to avoid using an apparatus that was cumbersome to produce or move. Also, the
9HPT versus visual acuity in the best eye
Correlation
Correlation p-value
Patients Controls Both
0.1354 0.3698 0.4634
0.4244 0.1310 0.0004
Correlation
Correlation p-value
0.1777 0.2510 0.4800
0.2927 0.3150 0.0002
SQUARES 9HPT
0.0
Table 2
ST versus visual acuity in the best eye Patients Controls Both
● ●
● ●
● ●
Sensitivity
For the measurement of functional disability in MS, the expanded disability status score [10], 1983) is still widely used, despite its limitations [13]. An attempt to improve upon the EDSS as a measure of disease progression in MS was made by the construction of the multiple sclerosis functional composite [14]. This is a composite measure which includes a test of walking function (25 foot walk test), but also tests of cognitive (Paced Auditory Serial Addition Test, PASAT) and hand function. In the construction of the MSFC, three candidate measures for upper limb function were assessed: the Arm Box and blocks test, the 9HPT and the Purdue pegboard test. The existing data that was available for retrospective review by the international panel constructing the MSFC favoured inclusion of the 9HPT [14], but the consensus panel clearly recommended that new measures should be tested as potential replacement tests in the MSFC: “An attractive characteristic of the functional composite outcome measure is that measures within a clinical dimension are interchangeable; for example, if a performance test superior to the nine hole peg test or the box and block test is developed it may be substituted for the arm dimension.” [14]. Replacement tests have been proposed for the ambulation component of the MSFC [15] and for the cognitive component [16]. As a candidate replacement for the 9HPT, Bonzano et al. [13] quantitatively measured finger opposition movements as a test for disability impact on fine hand motor functions. In our study, we examined whether a test of hand function that has been used for several decades in handedness research, the ST, also has potential as a test of hand function in MS. Annett [6] designed the squares test as a group test for hand skill assessment that would resemble a peg moving task. She
● ● ● ●
● ●
● ●
● ● ● ●
● ●
4. Discussion
● ●
● ●
● ●
1.0
0.8
0.6
0.4
0.2
0.0
Specificity Fig. 6. ST and 9HPT receiver operating characteristics. Receiver operating characteristic curves for the ST and 9HPT showing a measure of the rate of true positives versus true negatives at varying thresholds. Both tests show a high discriminatory value and AUC.
J. Gielen et al. / Clinical Neurology and Neurosurgery 123 (2014) 55–60
A) Squares test
60
B) 9 hole peg board test
50
0.04 9HPT score
ST score
59
40 30 20
0.03
0.02
10 3−6
6−6.5 6.5−7 EDSS range
7−8
3−6
6−6.5 6.5−7 EDSS range
7−8
Fig. 7. Relation between EDSS hand function scoring. Box plots of the (A) ST and (B) 9HPT scores for varying ranges of EDSS scores. If the EDSS was truly a measure for general disability in MS, one would expect both the ST and the inverse of the 9HPT to decrease with increasing EDSS score. Yet, no significant correlation can be detected.
test had to be easy to score [6]. A potential advantage is that the ST leaves a record of performance, namely the filled-out test form, which can be a plus in the documentation of source data in clinical trials. The first element of validity we examined was the reliability of the test results. In the analysis of test-retest reliability (Fig. 2), no floor or ceiling effects (defined as more than 20% of tested patients have minimal or maximal score [17]) were observed. Correlations between a ST at baseline, and a second ST within four weeks in a healthy control group were high and highly significant (Fig. 2). Since the ST involves marking targets on paper, the dominant hand might be more practiced than the non-dominant hand. In children, the variance in ST score for the non-dominant hand was therefore higher than for the dominant hand [6]. This was not the case in our three adult subject groups. Although the ST resembles a pegboard task in some ways, there are also differences: in the ST, the manipulated object (pencil) is held in a normal grip for writing [6], while pegs in the 9HPT are held in a pincer grip. Annett [6] also remarked that in the ST, the pen is continuously held during the test, while in the 9HPT, pegs are grasped and released several times. Also, the degrees of freedom in the ST might be lower than in the 9HPT if the subject’s forearm rests on the table during the test. Because of these differences in both tasks, it was necessary to examine whether both tests measure the behaviour of interest, hand function, in a comparable way. Since age has a significant effect on hand strength, precision and speed [18], we used an age matched control group for this part of the study. In this age group, one might expect some to perform poorer than others due to worse visual acuity. This was not the case in our study group: correlation of visual acuity with test results was low for both patients and controls. Correlation was higher when both groups were combined. This can be explained by a general trend of both test scores and visual acuity being higher in the control group. In the examination of correlations between tests, we labelled correlations up to 0.3 as low, from 0.31 to 0.69 as moderate and from 0.7 as high [17]. Both ST and 9HPT perform very well in the discrimination between MS patients and matched healthy controls, as is evident from the ROC analysis in Fig. 6. This supports the validity of the ST in our application. A higher score on the 9HPT (seconds needed to complete the test) means a worse performance. In the MSFC, the 9HPT is inverted for this reason [14], but an inversion of the ST score is not necessary, because a higher score on the ST (number of dots
correctly placed) means a better performance on the test. Since we expect an inverse correlation between ST and 9HPT raw scores, 1/ 9HPT was plotted against ST score in Fig. 3. This plot shows a high correlation, confirming the concurrent validity of both tests. Divergent validity is supported by the lack of correlation between ST and EDSS (Fig. 7 panel A). If the EDSS was in reality a test of general functional loss in MS, the lack of correlation and of a downward trend in the data would be disappointing. In reality however, the EDSS is heavily loaded on ambulatory function, especially in the score range of our subjects. This is supported by the lack of correlation between the 9HPT and EDSS as shown in Fig. 7, panel B. Our study also showed that the ST is feasible and well supported in a broad age and EDSS range in MS patients. It requires almost no setup time since only a pencil and the test form are needed. 5. Conclusions The squares test as presented in this paper can by no means replace a rehabilitation evaluation of upper limb function as it would be done by a physical or occupational therapist in a rehabilitation context. However, the ST is a reliable test of hand function. It is a brief and easy to administer paper-and pencil-test that can be used in a broad EDSS range. It has a high test–retest correlation, and correlates highly with an accepted measure of hand function in MS, the 9HPT. The ST is simpler and shorter to perform than the 9HPT, and it is more closely related to daily activities like writing. In addition, no material apart from pen and paper is needed, and a record of the test is easily kept. Therefore the ST is a candidate to be used in composite measures of MS related functional deficits for clinical practice and in clinical trials. Future work The lowest EDSS score in our patient group was 3, which is still relatively severe. It would be interesting to look at groups of patients that are more moderately affected, and see if the two tests show the same correlation as they do in our current group. Secondly, MS patients do show variability in functional performance during the day. An interesting approach would be to look at ST and 9HPT results at different times of the day. Lastly, we hope that our research stimulates the rehabilitation community to do further work in evaluating the ST as a tool to detect impairment at an earlier stage, to see if rehabilitation can start sooner, in a more focused and personalized way.
60
J. Gielen et al. / Clinical Neurology and Neurosurgery 123 (2014) 55–60
Acknowledgements GN holds the “BiogenIdec-National MS Center Melsbroek” named chair, and the “Merck–Novartis” named chair, both at the Center for Neurosciences at the Vrije Universiteit Brussel, Belgium. JVS is a pre-doctoral scholar (aspirant) of the FWO. We gratefully acknowledge the assistance of Mrs Ann Van Remoortel and the other MS and research nurses from the National MS Center Melsbroek. References [1] Shields SD, Cheng X, Gasser A, Saab CY, Tyrrell L, Eastman EM, et al. A channelopathy contributes to cerebellar dysfunction in a model of multiple sclerosis. Ann Neurol 2012;71:186–94. [2] Goodkin DE, Hertsgaard D, Seminary J. Upper extremity function in multiple sclerosis: improving assessment sensitivity with box-and-block and nine-hole peg tests. Arch Phys Med Rehabil 1988;69:850–4. [3] Fischer JS, Rudick RA, Cutter GR, Reingold SC. The multiple sclerosis functional composite measure (MSFC): an integrated approach to MS clinical outcome assessment. National MS Society Clinical Outcomes Assessment Task Force. Mult Scler 1999;5:244–50. [4] Ontaneda D, LaRocca N, Coetzee T, Rudick R. Revisiting the multiple sclerosis functional composite: proceedings from the national multiple sclerosis society (NMSS) task force on clinical disability measures. Mult Scler 2012;18:1074–80. [5] Fischer J, Jak A, Kniker J, Rudick R, Cutter G. Multiple sclerosis functional composite (MSFC): administration and scoring manual. National MS Society 2001. [6] Annett M. Five tests of hand skill. Cortex 1992;28:583–600.
[7] Polman CH, Reingold SC, Banwell B, Clanet M, Cohen a J, Filippi M, et al. Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann Neurol 2011;69:292–302. [8] Goodkin D, Priore R, Wende K. Comparing the ability of various compositive outcomes to discriminate treatment effects in MS clinical trials. Mult Scler 1998;4:480–6. [9] Oldfield R. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 1971;9:99–113. [10] Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an Expanded Disability Status Scale (EDSS). Neurology 1983;33:1444–52. [11] Kniestedt C, Stamper RL. Visual acuity and its measurement. Ophthalmol Clin North Am 2003;16:155–70. [12] R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2013. [13] Bonzano L, Sormani MP, Tacchino A, Abate L, Lapucci C, Mancardi GL, et al. Quantitative assessment of finger motor impairment in multiple sclerosis. PLoS One 2013;8:e65225. [14] Cutter GR, Baier ML, Rudick a R, Cookfair DL, Fischer JS, Petkau J, et al. Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain 1999;122(5):871–82. [15] Goldman MD, Motl RW, Rudick R. A possible clinical outcome measures for clinical trials in patients with multiple sclerosis. Ther Adv Neurol Disord 2010;3:229–39. [16] Drake AS, Weinstock-Guttman B, Morrow SA, Hojnacki D, Munschauer FE, Benedict RHB. Psychometrics and normative data for the multiple sclerosis functional composite: replacing the PASAT with the symbol digit modalities test. Mult Scler 2010;16:228–37. [17] Schäffler N, Schönberg P, Stephan J, Stellmann J-P, Gold SM, Heesen C. Comparison of patient-reported outcome measures in multiple sclerosis. Acta Neurol Scand 2013;128:114–21. [18] Ranganathan VK, Siemionow V, Sahgal V, Yue GH. Effects of aging on hand function. J Am Geriatr Soc 2001;49:1478–84.
