The Spine Journal
-
(2014)
-
Clinical Study
Longitudinal construct validity and responsiveness of measures of walking capacity in individuals with lumbar spinal stenosis Christy C. Tomkins-Lane, PhDa,*, Michele C. Battie, PhDb, Luciana G. Macedo, PhDb a Mount Royal University, U243E, 4825 Mount Royal Gate SW, Calgary, Alberta, Canada T3E 6K6 Department of Physical Therapy, Faculty of Rehabilitation Medicine, University of Alberta, 3-48 Corbett Hall, Edmonton, Alberta, Canada T6G 2G4
b
Received 16 August 2012; revised 6 September 2013; accepted 7 November 2013
Abstract
BACKGROUND CONTEXT: Walking capacity is a primary outcome indicator for individuals with lumbar spinal stenosis (LSS). Therefore, there is a demand for psychometrically sound measures of walking that are responsive to change. PURPOSE: The primary objective of this study was to examine longitudinal construct validity of the Physical Function Scale of the Swiss Spinal Stenosis Questionnaire (PF Scale), the Oswestry Disability Index (ODI), and the walking capacity items from these scales specifically for the assessment of walking capacity in LSS using the objective Self-Paced Walking Test (SPWT) as the external standard. A secondary objective was to examine responsiveness of measures of walking using a self-reported walking capacity change scale as the external criterion standard. STUDY DESIGN: Patients were prospectively enrolled. PATIENT SAMPLE: Twenty-six patients were included in this study (17 women and 9 men), with an average age of 68.5 years (SD, 9.2). All participants had LSS diagnosed by a spine specialist surgeon based on both clinical examination and imaging, as well as self-reported walking limitations (neurogenic claudication). OUTCOME MEASURES: The self-reported outcome measures included in this study were PF Scale, ODI, and self-reported walking capacity change score. FUNCTIONAL MEASURES: The functional measure used in the study was SPWT. METHODS: Longitudinal construct validity was assessed using the correlational method. Internal responsiveness was examined using Guyatt responsiveness index and external responsiveness using receiver operating characteristic analysis. Change in the SPWT and the self-reported walking capacity change score were used as external criteria for the analysis. RESULTS: The highest correlations with change in the SPWT were 0.78 for the ODI walking item and 0.78 for the walking capacity change score. Changes in the PF Scale and ODI score were correlated with change in the criterion SPWT at r50.56 and r50.70, respectively. There were no differences observed between the PF Scale and ODI for any of the responsiveness indices. CONCLUSIONS: The PF Scale, ODI, and walking capacity change score are able to detect changes in walking capacity in people with LSS. The individual walking capacity item from the ODI appears to be the most valid and responsive to changes in measured walking and may be a reasonable alternative for measuring walking when an objective test such as the SPWT is not feasible. Ó 2014 Elsevier Inc. All rights reserved.
Keywords:
Validity; Responsiveness; Spinal stenosis; Walking; Outcomes; Neurogenic claudication
FDA device/drug status: Not applicable. Author disclosures: CCT-L: Nothing to disclose. MCB: Grants: Alberta Innovates Health Solutions (C). LGM: Nothing to disclose. The disclosure key can be found on the Table of Contents and at www. TheSpineJournalOnline.com. The study was supported by the Health Research Fund administered by Alberta Innovates Health Solutions, as well as the EFF Support for the Advancement of Scholarship, University of Alberta. CCT-L received support for her doctoral research from the Alberta Heritage Foundation for Medical 1529-9430/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.spinee.2013.11.030
Research and the Alberta Provincial CIHR Training Program in Bone and Joint Health. MCB receives support from the Canada Research Chairs Program. LGM receives postdoctoral support from Canadian Institutes for Health Research and Alberta Innovates Health Solutions. There are no conflicts of interest. * Corresponding author. Mount Royal University, U243E, 4825 Mount Royal Gate SW, Calgary, Alberta, Canada T3E 6K6. Tel.: (403) 440-8671; fax: (403) 440-6744. E-mail address: [email protected] (C.C. Tomkins-Lane)
2
C.C. Tomkins-Lane et al. / The Spine Journal
Introduction People with lumbar spinal stenosis (LSS) often have limited walking capacities because of pain or neurologic symptoms associated with neurogenic claudication. Walking capacity is measured to assess functional status, treatment outcomes, and the natural progression of the condition. It has been suggested that improved walking capacity is the primary goal of most treatments for LSS [1]. Therefore, there is a demand for psychometrically sound measures of walking that are valid, reproducible, and responsive to change [2,3]. The Self-Paced Walking Test (SPWT) has been established as a criterion measure of walking capacity in people with LSS [4]. However, although observational tests are more accurate for measurement of walking, treatment outcomes in patients with LSS are often determined using self-reported questionnaires [5]. Self-reported instruments are generally more practical and easy to use. Of the selfreported instruments used to measure walking capacity in this population, only the Physical Function Scale of the Swiss Spinal Stenosis Questionnaire (PF Scale) [6] and the Oswestry Disability Index (ODI) [7,8] have been the focus of psychometric research. Both scales have been shown to be valid and reproducible for use in measurement of walking in patients with LSS [6,9–13]. One study also showed that the individual items addressing walking capacity from both the PF Scale (Item 1) and the ODI (Item 4) are valid and reproducible for this purpose [11]. Previous studies have also shown the PF Scale and ODI to be sufficiently responsive to detect clinically significant changes in patient satisfaction and global clinical status [6,9,14–16]. However, the ability of these questionnaires to detect actual changes in measured walking capacity has not been examined. If these questionnaires are being used to measure walking capacity, it is very important to know how sensitive they are to changes in walking capacity, as measured objectively. There are two ways to determine the ability of a measure to detect change, and both are components of validity. The ability of a measure to detect clinically meaningful change is known as responsiveness. This implies that there exists an external measure that is valid for assessment of clinically meaningful change. Generally, a global change score of patient-perceived change or satisfaction is used. However, if available, observable physical findings have a longstanding preference over self-reported measures for assessing patient outcomes [17]. When an objective gold standard measure of a construct is available, sensitivity to change can be examined as an element of construct validity. The correlation between change in a given measure and change in the gold standard represents the ability of the chosen measure to detect change in the construct over time. This type of analysis has been termed longitudinal construct validity [17]. If longitudinal data include an objective gold standard measure of walking capacity and a related global change score, both responsiveness to change, using the global change score as
-
(2014)
-
the external criterion, and sensitivity to change (longitudinal construct validity), using the objective criterion measure, can be determined. To date, no studies of which we are aware have examined the responsiveness or longitudinal construct validity of measures of walking in LSS using an objective test as the external criterion. Purpose The primary objective of this study was to examine longitudinal construct validity of the PF Scale, the ODI, and the walking capacity items from these scales, specifically for the assessment of walking capacity in LSS using the objective SPWT as the external standard. The aim was to determine whether these self-reported measures of walking capacity are acceptable substitutes for the reference standard SPWT. A secondary objective was to examine responsiveness of measures of walking using a self-reported walking capacity change scale as the external criterion standard.
Methods Design Subjects were prospectively enrolled and retested 2 years after baseline testing. Patient sample Subjects were recruited through the clinical practices of three spine specialist surgeons. Inclusion criteria were 45 years of age or older, LSS diagnosed by a spine specialist surgeon, central or combined LSS confirmed on imaging (magnetic resonance imaging/computed tomography), and self-reported LSS-associated walking limitations or symptoms exacerbated by walking (neurogenic claudication). Combined LSS was defined as a combination of central LSS plus lateral recess or foraminal stenosis. Exclusion criteria included surgery for LSS within the prior 12 months or any comorbid condition that would limit walking capacity or make a SPWT medically inadvisable, as judged by the subjects’ physician (eg, severe cardiopulmonary, lower extremity musculoskeletal conditions, or peripheral vascular disease). Subjects may or may not have subsequently elected surgery for LSS. Ethical approval for the study was obtained through the University of Alberta Health Research Ethics Board. Outcome measures Self-reported measures The self-reported measures included the PF Scale and the ODI. The PF Scale was designed to specifically assess walking capacity in LSS and is calculated as the unweighted mean of the five items in the scale [6]. The resulting possible scores of 1 to 4 represent a range from mild to
C.C. Tomkins-Lane et al. / The Spine Journal
severe limitation in physical function/walking. The published value for minimal clinically important difference for the PF Scale is 0.5 [6]. The ODI was calculated as a percentage of the total possible score of 50, with a greater score representing greater back-related disability. Published values for minimal clinically important difference on the ODI in LSS populations include 8.2 [18], 11 [19], and 12.8 [20]. In addition to the aforementioned questionnaires, two items that address walking distance specifically were isolated for analysis. These included the first item from the PF Scale and the fourth item from the ODI (#4). PF Scale Item 1 is scored from 1 to 4 and includes the following options: ability to walk ‘‘over 3.2 km (2 mi),’’ ‘‘over 2 blocks but less than 3.2 km,’’ ‘‘over 15.24 m (50 ft) but less than 2 blocks,’’ and ‘‘less than 15.24 m.’’ A higher score indicates a lower walking capacity. ODI Item 4 is scored from 1 to 6 and includes the following options: ‘‘pain does not prevent me from walking any distance,’’ ‘‘pain prevents me from walking 2 km,’’ ‘‘pain prevents me from walking more than 1 km,’’ ‘‘pain prevents me from walking more than 500 m,’’ ‘‘I can only walk using a stick or crutches,’’ and ‘‘I am in bed most of the time.’’ A higher score on this item indicates a lower walking capacity. At 2-year follow-up, a seven-point walking capacity change score was included that asked: ‘‘How would you say your walking capacity is today compared to your last visit approximately 2 years ago?’’ The options ranged from 7 (a great deal better) to 1 (a great deal worse). Criterion measure: SPWT protocol Immediately after completion of the study questionnaire at both time points, subjects completed the SPWT. The SPWT and its development have been described in detail previously [4]. The SPWT is based on the operational definition for walking capacity, which is the distance a person with LSS is able to walk without support on a level surface at a self-selected speed before being forced to stop because of symptoms of LSS. The test was found to have high testretest reliability (intraclass correlation coefficient50.98) for group-level analysis [4]. According to the SPWT protocol, subjects walked continuously at their own pace around the outer lane of a track until they felt they had to stop because of symptoms of LSS or until a time limit of 30 minutes had been reached. All measurements were completed at two time points, with approximately 2 years between testing occasions. Data analysis methods Longitudinal construct validity Longitudinal construct validity was assessed using the correlational method [17,21–24], which examined the relationship between change in the SPWT and change in the self-reported variables using Spearman rank correlation coefficients. When a criterion standard of change is quantified
-
(2014)
-
3
on a continuous scale (such as the SPWT), correlation coefficients can be used as indices of sensitivity to change (longitudinal validity) [22]. The measure of longitudinal validity is the relative strength of correlation between change in the external criterion (SPWT) and change in the instrument score [17]. The magnitude of a relationship is judged to be low if correlation coefficients vary between 0 and 0.29, moderate between 0.3 and 0.59, and a strong relationship is judged if coefficients are above 0.6 [25]. A correlation of 0.8 or higher is recommended before a surrogate measure is acceptable to be used in place of the gold standard [25].
Responsiveness analysis The responsiveness analysis was conducted using the guidelines suggested by Terwee et al. [26]. To examine the ability of the self-reported measures to detect perceived change in walking capacity, we looked at responsiveness using a self-reported walking capacity change score to dichotomize patients into changed or not. A clinically meaningful improved status of walking capacity was measured at follow-up using a perceived change scale ranging from 1 to 7 points, in which a score of 1 represented a great deal worse and a score of 7 a great deal better. Patients were considered to have clearly improved their walking ability if they reported a score of 6 or higher (moderately better or a great deal better) on the self-reported walking capacity change scale. We also conducted a responsiveness analysis using the SPWT as the external criterion. Because there are no established values for clinically meaningful change in walking distance, we defined change as the minimal detectable change for the SPWT. Reliability coefficients for the SPWT defined in a previous study [4] were used to calculate standard error of measurement (SEM) and consequently minimal detectable change (MDC). The SEM was calculated using the formula SEM5SDO1R (SD represents the standard deviation of the measure [927.2 m] and R represents the reliability coefficient [intraclass correlation coefficient50.98]). The MDC was calculated using the formula MDC5SEM1.96O2 [26,27]. Patients were defined as having changed if the difference between baseline and follow-up SPWT tests was greater than the MDC for the test. This change is defined as a true change, above that of the measurement error. The SEM of the SPWT was 131.13 m and the MDC was 363.47 m. Internal responsiveness was assessed using Guyatt responsiveness index (GRI). This index takes into consideration true changes that are assessed by an external criterion [28]. It is calculated by dividing the mean change of patients who have improved by the SD of change of patients reporting no improvement. Higher GRI values indicate greater levels of responsiveness [28]. We calculated 84% confidence intervals because nonoverlapping 84% confidence intervals are equivalent to a Z test of mean at the 0.05 level [29].
4
C.C. Tomkins-Lane et al. / The Spine Journal
External responsiveness was assessed using receiver operating characteristic (ROC) curve analyses to compare the ability of the PF Scale, the ODI, PF Scale Item 1, and ODI Item 4 to discriminate those people who had and had not clearly improved on the self-reported walking capacity change score [30]. A secondary ROC analysis was conducted using the MDC of the SPWT to define patients as improved or not. The larger the area under the curve (AUC), the greater a measure’s ability to distinguish between patients who did and did not undergo an important change [22]. The DeLong statistic was used to evaluate whether AUC values were statistically significantly different between measures [31]. Data analyses were performed using PASW 18.0 (IBM SPSS Statistics 18 for Windows, Armonk, NY, USA) and the Hong Kong ROC program to compare ROC curves. Descriptive statistics were used to report on the sample characteristics.
-
(2014)
-
had changed walking at the follow-up. Two of these patients were classified as changed on the self-reported walking change question but as not on the SPWT. GRI Guyatt responsiveness index and 84% confidence intervals are presented in Table 3. Using the MDC of the SPWT to dichotomize patients into improved and not improved, the PF Scale had the highest GRI followed by the walking item of the ODI, but overlapping confidence intervals demonstrated that none of the measures examined were significantly different. When the walking capacity change score was used to dichotomize patients as clinically improved or not, again the GRI was highest (1.67) for the PF Scale followed by the walking item of the ODI; however, all with overlapping 84% confidence interval differences in GRIs were not statistically significant. ROC curve analysis
Results Twenty-six patients were included in this study (17 women and 9 men), with an average age of 68.5 years (SD, 9.2), back pain duration of 13.2 years (SD, 13.0), and leg pain duration of 9.0 years (SD, 11.1). Baseline and follow-up scores for the primary outcomes are presented in Table 1. The mean follow-up time was 23.8562.6 months (range, 19–30 months). Of the 26 participants, 11 (42%) had surgery for LSS during the 2 years between baseline and follow-up, and of these 11 patients, 10 also had some form of nonsurgical therapy. Of the 15 patients who did not have surgery, eight underwent conservative treatment and seven had no treatment over the 2-year period. Longitudinal construct validity: correlational analysis Results of the correlational analysis can be found in Table 2. The highest correlations with change in the SPWT were r50.78 (p5.000) for the ODI walking item and r50.78 (p5.000) for the walking capacity change score. Using the MDC of the SPWT to dichotomize into changed or not, five patients (19.2%) exhibited true change in walking capacity. The change question used to evaluate perceived change in walking capacity demonstrated that when using a score of 6 or higher, nine patients (34.6%) Table 1 Mean values 6 SD for the Self-Paced Walking Test, Physical Function Scale, and Oswestry Disability Index at baseline and follow-up Outcome measure
Baseline
Follow-up
Self-Paced Walking Test Physical Function Scale Oswestry Disability Index Physical Function Scale Item 1 Oswestry Disability Index Item 4
1337.366943.46 2.1260.56 44.63612.44 2.1260.71 2.8861.2
1239.046918.65 1.8160.58 40.42611.73 1.8860.65 2.6961.4
SD, standard deviation.
Receiver operating characteristic curves are presented in Figs. 1 and 2. Using the MDC of the SPWT to dichotomize patients as improved or not, the AUCs were 0.77 (SD, 0.14) for the PF Scale, 0.73 (SD, 0.12) for the ODI, 0.64 (SD, 0.17) for Item 1 of the PF Scale, and 0.76 (SD, 0.15) for Item 4 from the ODI (Fig. 1). The DeLong statistic comparing the AUC of the PF scale and ODI was z50.47 (p5.32) and z51.47 (p5.87) when comparing Item 1 of the PF Scale with Item 4 from the ODI. Using the walking capacity change score to dichotomize patients as improved or not, the AUC values were 0.92 (0.05) for the SPWT, 0.94 (0.04) for the PF Scale, 0.96 (0.03) for the ODI, 0.71 (0.10) for Item 1 of the PF Scale, and 0.86 (0.07) for Item 4 of the ODI (Fig. 2). The DeLong test statistic comparing the AUC for the PF scale and ODI was z50.29 (p5.39). Comparing the AUC for the PF scale and SPWT, the statistic was z50.34 (p5.37) and z50.57 (p5.29) when comparing the ODI and SPWT. Additionally, the DeLong test comparing the AUC for Item 1 of the PF Scale and Item 4 of the ODI was z51.45 (p5.07).
Discussion This is the first study of which we are aware to use an objective gold standard measure of the construct of walking capacity to examine the validity and responsiveness of selfreported measurements of change in walking capacity. None of the self-reported measures met the standard required to be considered an acceptable surrogate for actual change in walking capacity as measured with the SPWT. Of the self-reported measures investigated, both the walking capacity change score and the walking item from the ODI were most highly correlated with the SPWT (r50.78). Although the high correlation between the walking capacity change score and the SPWT (r50.78)
C.C. Tomkins-Lane et al. / The Spine Journal
-
(2014)
-
5
Table 2 Spearman rank correlation coefficients between change scores for measures of walking capacity SPWT change SPWT change PF change ODI change PF Item 1 change ODI Item 4 change Self-reported change
PF change
1.00 0.56** (0.90 to 0.19) 1.00 0.70** (0.93 to 0.25) 0.72** (0.35 to 0.98) 0.50** (0.90 to 0.20) 0.50** (0.22 to 0.91) 0.78** (1.04 to 0.50) 0.78** (0.46 to 1.02)
0.50** (0.23 to 0.92)
ODI change
PF Item 1 change
1.00 0.50** (0.15 to 0.87)
1.00
0.78** (0.44 to 1.02)
ODI Item 4 change
0.61** (0.39 to 1.00)
Self-reported change
1.00
0.70** (1.02 to 0.43) 0.80** (1.02 to 0.46) 0.46* (0.85 to 0.11) 0.73** 1.00 (1.01 to 0.43)
SPWT, Self-Paced Walking Test; PF, Physical Function Scale of Swiss Spinal Stenosis Questionnaire; ODI, Oswestry Disability Index. *p!.05; **p!.01 (95% confidence interval).
indicates that individuals are capable of detecting change in their walking over time, the change score does not provide any information about walking distance or degree of limitation. Therefore, its use would be limited to studies of change. The walking item of the ODI does ask about pain-limited walking distance and could therefore be used to estimate walking distance, as well as to assess change. The validity of this item is supported by our earlier crosssectional study of baseline measurements where this item was found to be most highly correlated with the SPWT (r50.83) [11]. This item was also shown to correlate strongly (r50.62) with objectively measured walking in a study comparing self-reported measures with the Shuttle Walking Test in people with chronic low back pain [32]. Thus, the use of this one item may be preferable to other self-reported instruments as a surrogate for the SPWT when an objective walking test is not practical. The ODI showed a stronger relationship to change in walking capacity (SPWT; r50.70) than the PF Scale (r50.56), yet no statistically significant difference in their responsiveness was found when using the SPWT as the dichotomizing variable in GRI and ROC analysis. Similarly, when using the self-reported walking capacity change variable as the external criterion, there was no significant
difference in the responsiveness indices between the PF Scale and the ODI. These results suggest that the PF Scale and ODI are similarly sensitive to change in both measured and reported walking capacities, with the ODI performing slightly better in correlational analysis. Other studies have examined responsiveness of the Physical Function (PF) Scale and ODI for use in LSS populations but have used either patient satisfaction or a global change scale as the external criterion variable, rather than an objective walking test [9,14–16]. Stucki et al. [9] found the PF Scale to be more responsive in identifying change in satisfaction than the nondisease-specific Sickness Impact Profile and Roland Morris Questionnaire. In another study, Stucki et al. [6] found that a change in the PF Scale correlated (0.72) with patient satisfaction after surgery. Cleland et al. [15] reported on the responsiveness of both the PF Scale and ODI using a global change scale as the external criterion. This study found the ODI to have a higher AUC and correlation with the change score compared with the PF Scale, whereas the PF scale demonstrated a higher GRI. These results differ from the present study where the PF Scale was found to have a higher AUC and GRI compared with the ODI (although not statistically different). The difference in results likely stems from the difference in choice
Table 3 Mean change 6 SD for measures of walking capacity in subjects who did and did not change according to the SPWT and self-report as well as Guyatt responsiveness index values Outcome measure Physical Function Scale Oswestry Disability Index Physical Function Scale Item 1 Oswestry Disability Index Item 4
SPWT Physical Function Scale Oswestry Disability Index Physical Function Scale Item 1 Oswestry Disability Index Item 4
Did not change (SPWT)*
Changed (SPWT)
Guyatt responsiveness index (84% CI)
0.8460.67 12.83611.03 0.8060.84 1.4061.14
1.45 0.85 1.04 1.23
Did not change (self-reported)y
Changed (self-reported)
Guyatt responsiveness index (84% CI)
547.966784.53 0.0460.45 3.33610.52 0.060.79 0.3561.06
751.006871.95 0.9660.38 18.45610.99 0.6760.71 1.2260.97
0.96 1.67 0.85 0.84 1.16
0.1860.58 2.16615.13 0.1060.77 0.1061.34
(0.74–2.16) (0.37–1.33) (0.36–1.73) (0.60–1.86)
(0.44–1.48) (0.91–2.44) (0.39–1.31) (0.42–1.26) (0.73–1.59)
SD, standard deviation; SPWT, Self-Paced Walking Test; CI, confidence interval. * Change on the SPWT was defined as a difference between baseline and follow-up SPWT tests greater than 363.47 m. y Self-reported change was defined as a score of 6 or 7 on the self-reported walking change scale.
6
C.C. Tomkins-Lane et al. / The Spine Journal
Fig. 1. (Top, A) Receiver operating characteristic curve of the Physical Function Scale, (Top, B) the Oswestry Disability Index, (Bottom, C) Item 1 of the Physical Function Scale, and (Bottom, D) Item 4 of the Oswestry Disability Index with improvement defined through the use of the SelfPaced Walking Test.
of external criterion (global change vs. objective walking test). Only one other study to date has specifically examined responsiveness of measures of walking in people with LSS. In their study, Rainville et al. [16] studied patients who were undergoing treatment for LSS and neurogenic claudication. Before and after treatment, patients completed a questionnaire, as well as an SPWT and treadmill test. ‘‘Global change in clinical status’’ was chosen as the
-
(2014)
-
Fig. 2. (Top, A) Receiver operating characteristic curve of the Physical Function Scale, (Top, B) the Oswestry Disability Index, (Bottom, C) Item 1 of the Physical Function Scale, and (Bottom, D) Item 4 of the Oswestry Disability Index with improvement defined through the self-reported walking capacity change score.
external criterion for responsiveness analysis and was assessed with the following question: ‘‘Since treatment, how have your symptoms with walking changed?’’ This study concluded that neither the SPWT nor the treadmill test showed meaningful ability to detect change, given that objective changes in walking showed little concordance with patients’ perception of change in clinical status. Yet, the question we need to consider is what are we trying to measure? Rainville et al. [16] suggested that limited
C.C. Tomkins-Lane et al. / The Spine Journal
walking is the most relevant area of impaired function for patients with neurogenic claudication, and therefore, assessment of patients’ ability to walk is a standard part of evaluation for LSS. Therefore, the external criterion in such studies should logically be a measure of walking capacity. This is not to say that perceived changes in walking capacity are not important, but these perceptions seem to be tapping into a slightly different construct. Such measurements may compliment objective measures of walking capacity in future studies of patients with LSS. Results of the present study also highlight the potential danger in using retrospective change scores to assess responsiveness when an objective measure of the construct is available. Norman et al. [33] suggested that retrospective analysis of responsiveness using self-reported change scales is problematic given that often the psychometric properties of the global change score have not been established, and judgment of change is psychologically difficult. They also suggested that this type of analyses may be misleading because the responsiveness coefficient will always be greater than zero even when there is no detectable difference in the patient cohort, as long as there is a positive correlation between the self-reported instruments and the global change score [33]. This theory applies to the present study in that the PF Scale, ODI, and walking capacity change score rely on patient perception and are therefore highly related. This could lead to validity and responsiveness results that are misleading. For example, the correlations between the self-reported walking change score and changes in the PF Scale (r50.70) and ODI (r50.80) were much higher than the correlation between these measures and the SPWT (r50.56 and 0.70, respectively). If the self-reported change scale was used as the sole criterion, this would lead readers to believe that the PF Scale and ODI are more sensitive to changes in walking than they actually are. Strengths and limitations Strengths of the present study include the use of an objective criterion standard measure of walking capacity, as well as the use of multiple responsiveness analyses. Limitations include the relatively small sample size, precluding the analysis of responsiveness by subgroups (eg, surgery vs. no surgery).
Conclusions None of the self-reported measures examined in this study met the standard required to be considered an acceptable surrogate for the actual change in walking capacity as measured with the objective SPWT. It is therefore recommended that when feasible, an objective test of walking capacity be conducted to assess change. Results suggest that both the PF Scale and the ODI are able to detect changes in
-
(2014)
-
7
both measured and reported walking capacity in people with LSS. It also appears that individuals with LSS are able to detect changes in walking capacity over time using a single-item walking change score. However, this change score does not inform care providers of the degree of walking limitation and, therefore, may be of limited use outside studies of change. The individual walking capacity item from the ODI appears to be the most valid and responsive to changes in measured walking and may be a reasonable alternative for measuring walking when an objective test such as the SPWT is not feasible. References [1] Katz JN, Harris MB. Clinical practice. Lumbar spinal stenosis. N Engl J Med 2008;358:818–25. [2] Deen HG Jr, Zimmerman RS, Lyons MK, et al. Test-retest reproducibility of the exercise treadmill examination in lumbar spinal stenosis. Mayo Clin Proc 2000;75:1002–7. [3] Yamashita K, Hayashi J, Ohzono K, et al. Correlation of patient satisfaction with symptom severity and walking ability after surgical treatment for degenerative lumbar spinal stenosis. Spine 2003;28: 2477–81. [4] Tomkins CC, Battie MC, Rogers T, et al. A criterion measure of walking capacity in lumbar spinal stenosis and its comparison with a treadmill protocol. Spine 2009;34:2449. [5] Deen HG Jr, Zimmerman RS, Lyons MK, et al. Measurement of exercise tolerance on the treadmill in patients with symptomatic lumbar spinal stenosis: a useful indicator of functional status and surgical outcome. J Neurosurg 1995;83:27–30. [6] Stucki G, Daltroy L, Liang MH, et al. Measurement properties of a self-administered outcome measure in lumbar spinal stenosis. Spine 1996;21:796–803. [7] Fairbank JC, Couper J, Davies JB, et al. The Oswestry low back pain disability questionnaire. Physiotherapy 1980;66:271–3. [8] Fairbank JC, Pynsent PB. The Oswestry disability index. Spine 2000;25:2940–52. [9] Stucki G, Liang MH, Fossel AH, et al. Relative responsiveness of condition-specific and generic health status measures in degenerative lumbar spinal stenosis. J Clin Epidemiol 1995;48:1369–78. [10] Comer CM, Conaghan PG, Tennant A. Internal construct validity of the Swiss Spinal Stenosis questionnaire: Rasch analysis of a diseasespecific outcome measure for lumbar spinal stenosis. Spine 2011;1: 1969–76. [11] Tomkins-Lane CC, Battie MC. Validity and reproducibility of selfreport measures of walking capacity in lumbar spinal stenosis. Spine 2010;35:2097–102. [12] Tomkins CC, Battie MC, Hu R. Construct validity of the physical function scale of the Swiss Spinal Stenosis Questionnaire for the measurement of walking capacity. Spine 2007;32:1896–901. [13] Pratt RK, Fairbank JC, Virr A. The reliability of the Shuttle Walking Test, the Swiss Spinal Stenosis Questionnaire, the Oxford Spinal Stenosis Score, and the Oswestry Disability Index in the assessment of patients with lumbar spinal stenosis. Spine 2002;27:84–91. [14] Walsh TL, Hanscom B, Lurie JD, et al. Is a condition-specific instrument for patients with low back pain/leg symptoms really necessary? The responsiveness of the Oswestry Disability Index, MODEMS, and the SF-36. Spine 2003;28:607–15. [15] Cleland JA, Whitman JM, Houser JL, et al. Psychometric properties of selected tests in patients with lumbar spinal stenosis. Spine J 2012;12:921–31. [16] Rainville J, Childs LA, Pena EB, et al. Quantification of walking ability in subjects with neurogenic claudication from lumbar spinal stenosis—a comparative study. Spine J 2012;12:109.
8
C.C. Tomkins-Lane et al. / The Spine Journal
[17] Deyo RA. Measuring the functional status of patients with low back pain. Arch Phys Med Rehabil 1988;69:1044–53. [18] Parker SL, Mendenhall SK, Shau DN, et al. Minimum clinically important difference in pain, disability, and quality of life after neural decompression and fusion for same-level recurrent lumbar stenosis: understanding clinical versus statistical significance. J NeurosurgSpine 2012;16:471–8. [19] Lauridsen HH, Hartvigsen J, Manniche C, et al. Responsiveness and minimal clinically important difference for pain and disability instruments in low back pain patients. BMC Musculoskelet Disord 2006;7:82. [20] Copay AG, Glassman SD, Subach BR, et al. Minimum clinically important difference in lumbar spine surgery patients: a choice of methods using the Oswestry Disability Index, Medical Outcomes Study questionnaire Short Form 36, and pain scales. Spine J 2008;8: 968–74. [21] Stratford PW, Binkley J, Solomon P, et al. Assessing change over time in patients with low back pain. Phys Ther 1994;74:528–33. [22] Stratford PW, Binkley FM, Riddle DL. Health status measures: strategies and analytic methods for assessing change scores. Phys Ther 1996;76:1109–23. [23] Stratford PW, Riddle DL. Assessing sensitivity to change: choosing the appropriate change coefficient. Health Qual Life Outcomes 2005;3:23. [24] Husted JA, Cook RJ, Farewell VT, et al. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol 2000;53:459–68.
-
(2014)
-
[25] Domholdt E. Physical therapy research: principles and applications. 2nd ed. Philadelphia, PA: Saunders, 2000. [26] Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007;60:34–42. [27] Haley SM, Fragala-Pinkham MA. Interpreting change scores of tests and measures used in physical therapy. Phys Ther 2006;86:735–43. [28] Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Dis 1987;40: 171–8. [29] Tryon WW. Evaluating statistical difference, equivalence, and indeterminacy using inferential confidence intervals: an integrated alternative method of conducting null hypothesis statistical tests. Psychol Methods 2001;6:371–86. [30] Deyo RA, Centor RM. Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance. J Chronic Dis 1986;39:897–906. [31] DeLong E, DeLong D, Clarke-Pearson D. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–45. [32] Campbell H, Rivero-Arias O, Johnston K, et al. Responsiveness of objective, disease-specific, and generic outcome measures in patients with chronic low back pain: an assessment for improving, stable, and deteriorating patients. Spine 2006;31:815–22. [33] Norman GR, Stratford P, Regehr G. Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol 1997;50:869–79.
-
(2014)
-
Clinical Study
Longitudinal construct validity and responsiveness of measures of walking capacity in individuals with lumbar spinal stenosis Christy C. Tomkins-Lane, PhDa,*, Michele C. Battie, PhDb, Luciana G. Macedo, PhDb a Mount Royal University, U243E, 4825 Mount Royal Gate SW, Calgary, Alberta, Canada T3E 6K6 Department of Physical Therapy, Faculty of Rehabilitation Medicine, University of Alberta, 3-48 Corbett Hall, Edmonton, Alberta, Canada T6G 2G4
b
Received 16 August 2012; revised 6 September 2013; accepted 7 November 2013
Abstract
BACKGROUND CONTEXT: Walking capacity is a primary outcome indicator for individuals with lumbar spinal stenosis (LSS). Therefore, there is a demand for psychometrically sound measures of walking that are responsive to change. PURPOSE: The primary objective of this study was to examine longitudinal construct validity of the Physical Function Scale of the Swiss Spinal Stenosis Questionnaire (PF Scale), the Oswestry Disability Index (ODI), and the walking capacity items from these scales specifically for the assessment of walking capacity in LSS using the objective Self-Paced Walking Test (SPWT) as the external standard. A secondary objective was to examine responsiveness of measures of walking using a self-reported walking capacity change scale as the external criterion standard. STUDY DESIGN: Patients were prospectively enrolled. PATIENT SAMPLE: Twenty-six patients were included in this study (17 women and 9 men), with an average age of 68.5 years (SD, 9.2). All participants had LSS diagnosed by a spine specialist surgeon based on both clinical examination and imaging, as well as self-reported walking limitations (neurogenic claudication). OUTCOME MEASURES: The self-reported outcome measures included in this study were PF Scale, ODI, and self-reported walking capacity change score. FUNCTIONAL MEASURES: The functional measure used in the study was SPWT. METHODS: Longitudinal construct validity was assessed using the correlational method. Internal responsiveness was examined using Guyatt responsiveness index and external responsiveness using receiver operating characteristic analysis. Change in the SPWT and the self-reported walking capacity change score were used as external criteria for the analysis. RESULTS: The highest correlations with change in the SPWT were 0.78 for the ODI walking item and 0.78 for the walking capacity change score. Changes in the PF Scale and ODI score were correlated with change in the criterion SPWT at r50.56 and r50.70, respectively. There were no differences observed between the PF Scale and ODI for any of the responsiveness indices. CONCLUSIONS: The PF Scale, ODI, and walking capacity change score are able to detect changes in walking capacity in people with LSS. The individual walking capacity item from the ODI appears to be the most valid and responsive to changes in measured walking and may be a reasonable alternative for measuring walking when an objective test such as the SPWT is not feasible. Ó 2014 Elsevier Inc. All rights reserved.
Keywords:
Validity; Responsiveness; Spinal stenosis; Walking; Outcomes; Neurogenic claudication
FDA device/drug status: Not applicable. Author disclosures: CCT-L: Nothing to disclose. MCB: Grants: Alberta Innovates Health Solutions (C). LGM: Nothing to disclose. The disclosure key can be found on the Table of Contents and at www. TheSpineJournalOnline.com. The study was supported by the Health Research Fund administered by Alberta Innovates Health Solutions, as well as the EFF Support for the Advancement of Scholarship, University of Alberta. CCT-L received support for her doctoral research from the Alberta Heritage Foundation for Medical 1529-9430/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.spinee.2013.11.030
Research and the Alberta Provincial CIHR Training Program in Bone and Joint Health. MCB receives support from the Canada Research Chairs Program. LGM receives postdoctoral support from Canadian Institutes for Health Research and Alberta Innovates Health Solutions. There are no conflicts of interest. * Corresponding author. Mount Royal University, U243E, 4825 Mount Royal Gate SW, Calgary, Alberta, Canada T3E 6K6. Tel.: (403) 440-8671; fax: (403) 440-6744. E-mail address: [email protected] (C.C. Tomkins-Lane)
2
C.C. Tomkins-Lane et al. / The Spine Journal
Introduction People with lumbar spinal stenosis (LSS) often have limited walking capacities because of pain or neurologic symptoms associated with neurogenic claudication. Walking capacity is measured to assess functional status, treatment outcomes, and the natural progression of the condition. It has been suggested that improved walking capacity is the primary goal of most treatments for LSS [1]. Therefore, there is a demand for psychometrically sound measures of walking that are valid, reproducible, and responsive to change [2,3]. The Self-Paced Walking Test (SPWT) has been established as a criterion measure of walking capacity in people with LSS [4]. However, although observational tests are more accurate for measurement of walking, treatment outcomes in patients with LSS are often determined using self-reported questionnaires [5]. Self-reported instruments are generally more practical and easy to use. Of the selfreported instruments used to measure walking capacity in this population, only the Physical Function Scale of the Swiss Spinal Stenosis Questionnaire (PF Scale) [6] and the Oswestry Disability Index (ODI) [7,8] have been the focus of psychometric research. Both scales have been shown to be valid and reproducible for use in measurement of walking in patients with LSS [6,9–13]. One study also showed that the individual items addressing walking capacity from both the PF Scale (Item 1) and the ODI (Item 4) are valid and reproducible for this purpose [11]. Previous studies have also shown the PF Scale and ODI to be sufficiently responsive to detect clinically significant changes in patient satisfaction and global clinical status [6,9,14–16]. However, the ability of these questionnaires to detect actual changes in measured walking capacity has not been examined. If these questionnaires are being used to measure walking capacity, it is very important to know how sensitive they are to changes in walking capacity, as measured objectively. There are two ways to determine the ability of a measure to detect change, and both are components of validity. The ability of a measure to detect clinically meaningful change is known as responsiveness. This implies that there exists an external measure that is valid for assessment of clinically meaningful change. Generally, a global change score of patient-perceived change or satisfaction is used. However, if available, observable physical findings have a longstanding preference over self-reported measures for assessing patient outcomes [17]. When an objective gold standard measure of a construct is available, sensitivity to change can be examined as an element of construct validity. The correlation between change in a given measure and change in the gold standard represents the ability of the chosen measure to detect change in the construct over time. This type of analysis has been termed longitudinal construct validity [17]. If longitudinal data include an objective gold standard measure of walking capacity and a related global change score, both responsiveness to change, using the global change score as
-
(2014)
-
the external criterion, and sensitivity to change (longitudinal construct validity), using the objective criterion measure, can be determined. To date, no studies of which we are aware have examined the responsiveness or longitudinal construct validity of measures of walking in LSS using an objective test as the external criterion. Purpose The primary objective of this study was to examine longitudinal construct validity of the PF Scale, the ODI, and the walking capacity items from these scales, specifically for the assessment of walking capacity in LSS using the objective SPWT as the external standard. The aim was to determine whether these self-reported measures of walking capacity are acceptable substitutes for the reference standard SPWT. A secondary objective was to examine responsiveness of measures of walking using a self-reported walking capacity change scale as the external criterion standard.
Methods Design Subjects were prospectively enrolled and retested 2 years after baseline testing. Patient sample Subjects were recruited through the clinical practices of three spine specialist surgeons. Inclusion criteria were 45 years of age or older, LSS diagnosed by a spine specialist surgeon, central or combined LSS confirmed on imaging (magnetic resonance imaging/computed tomography), and self-reported LSS-associated walking limitations or symptoms exacerbated by walking (neurogenic claudication). Combined LSS was defined as a combination of central LSS plus lateral recess or foraminal stenosis. Exclusion criteria included surgery for LSS within the prior 12 months or any comorbid condition that would limit walking capacity or make a SPWT medically inadvisable, as judged by the subjects’ physician (eg, severe cardiopulmonary, lower extremity musculoskeletal conditions, or peripheral vascular disease). Subjects may or may not have subsequently elected surgery for LSS. Ethical approval for the study was obtained through the University of Alberta Health Research Ethics Board. Outcome measures Self-reported measures The self-reported measures included the PF Scale and the ODI. The PF Scale was designed to specifically assess walking capacity in LSS and is calculated as the unweighted mean of the five items in the scale [6]. The resulting possible scores of 1 to 4 represent a range from mild to
C.C. Tomkins-Lane et al. / The Spine Journal
severe limitation in physical function/walking. The published value for minimal clinically important difference for the PF Scale is 0.5 [6]. The ODI was calculated as a percentage of the total possible score of 50, with a greater score representing greater back-related disability. Published values for minimal clinically important difference on the ODI in LSS populations include 8.2 [18], 11 [19], and 12.8 [20]. In addition to the aforementioned questionnaires, two items that address walking distance specifically were isolated for analysis. These included the first item from the PF Scale and the fourth item from the ODI (#4). PF Scale Item 1 is scored from 1 to 4 and includes the following options: ability to walk ‘‘over 3.2 km (2 mi),’’ ‘‘over 2 blocks but less than 3.2 km,’’ ‘‘over 15.24 m (50 ft) but less than 2 blocks,’’ and ‘‘less than 15.24 m.’’ A higher score indicates a lower walking capacity. ODI Item 4 is scored from 1 to 6 and includes the following options: ‘‘pain does not prevent me from walking any distance,’’ ‘‘pain prevents me from walking 2 km,’’ ‘‘pain prevents me from walking more than 1 km,’’ ‘‘pain prevents me from walking more than 500 m,’’ ‘‘I can only walk using a stick or crutches,’’ and ‘‘I am in bed most of the time.’’ A higher score on this item indicates a lower walking capacity. At 2-year follow-up, a seven-point walking capacity change score was included that asked: ‘‘How would you say your walking capacity is today compared to your last visit approximately 2 years ago?’’ The options ranged from 7 (a great deal better) to 1 (a great deal worse). Criterion measure: SPWT protocol Immediately after completion of the study questionnaire at both time points, subjects completed the SPWT. The SPWT and its development have been described in detail previously [4]. The SPWT is based on the operational definition for walking capacity, which is the distance a person with LSS is able to walk without support on a level surface at a self-selected speed before being forced to stop because of symptoms of LSS. The test was found to have high testretest reliability (intraclass correlation coefficient50.98) for group-level analysis [4]. According to the SPWT protocol, subjects walked continuously at their own pace around the outer lane of a track until they felt they had to stop because of symptoms of LSS or until a time limit of 30 minutes had been reached. All measurements were completed at two time points, with approximately 2 years between testing occasions. Data analysis methods Longitudinal construct validity Longitudinal construct validity was assessed using the correlational method [17,21–24], which examined the relationship between change in the SPWT and change in the self-reported variables using Spearman rank correlation coefficients. When a criterion standard of change is quantified
-
(2014)
-
3
on a continuous scale (such as the SPWT), correlation coefficients can be used as indices of sensitivity to change (longitudinal validity) [22]. The measure of longitudinal validity is the relative strength of correlation between change in the external criterion (SPWT) and change in the instrument score [17]. The magnitude of a relationship is judged to be low if correlation coefficients vary between 0 and 0.29, moderate between 0.3 and 0.59, and a strong relationship is judged if coefficients are above 0.6 [25]. A correlation of 0.8 or higher is recommended before a surrogate measure is acceptable to be used in place of the gold standard [25].
Responsiveness analysis The responsiveness analysis was conducted using the guidelines suggested by Terwee et al. [26]. To examine the ability of the self-reported measures to detect perceived change in walking capacity, we looked at responsiveness using a self-reported walking capacity change score to dichotomize patients into changed or not. A clinically meaningful improved status of walking capacity was measured at follow-up using a perceived change scale ranging from 1 to 7 points, in which a score of 1 represented a great deal worse and a score of 7 a great deal better. Patients were considered to have clearly improved their walking ability if they reported a score of 6 or higher (moderately better or a great deal better) on the self-reported walking capacity change scale. We also conducted a responsiveness analysis using the SPWT as the external criterion. Because there are no established values for clinically meaningful change in walking distance, we defined change as the minimal detectable change for the SPWT. Reliability coefficients for the SPWT defined in a previous study [4] were used to calculate standard error of measurement (SEM) and consequently minimal detectable change (MDC). The SEM was calculated using the formula SEM5SDO1R (SD represents the standard deviation of the measure [927.2 m] and R represents the reliability coefficient [intraclass correlation coefficient50.98]). The MDC was calculated using the formula MDC5SEM1.96O2 [26,27]. Patients were defined as having changed if the difference between baseline and follow-up SPWT tests was greater than the MDC for the test. This change is defined as a true change, above that of the measurement error. The SEM of the SPWT was 131.13 m and the MDC was 363.47 m. Internal responsiveness was assessed using Guyatt responsiveness index (GRI). This index takes into consideration true changes that are assessed by an external criterion [28]. It is calculated by dividing the mean change of patients who have improved by the SD of change of patients reporting no improvement. Higher GRI values indicate greater levels of responsiveness [28]. We calculated 84% confidence intervals because nonoverlapping 84% confidence intervals are equivalent to a Z test of mean at the 0.05 level [29].
4
C.C. Tomkins-Lane et al. / The Spine Journal
External responsiveness was assessed using receiver operating characteristic (ROC) curve analyses to compare the ability of the PF Scale, the ODI, PF Scale Item 1, and ODI Item 4 to discriminate those people who had and had not clearly improved on the self-reported walking capacity change score [30]. A secondary ROC analysis was conducted using the MDC of the SPWT to define patients as improved or not. The larger the area under the curve (AUC), the greater a measure’s ability to distinguish between patients who did and did not undergo an important change [22]. The DeLong statistic was used to evaluate whether AUC values were statistically significantly different between measures [31]. Data analyses were performed using PASW 18.0 (IBM SPSS Statistics 18 for Windows, Armonk, NY, USA) and the Hong Kong ROC program to compare ROC curves. Descriptive statistics were used to report on the sample characteristics.
-
(2014)
-
had changed walking at the follow-up. Two of these patients were classified as changed on the self-reported walking change question but as not on the SPWT. GRI Guyatt responsiveness index and 84% confidence intervals are presented in Table 3. Using the MDC of the SPWT to dichotomize patients into improved and not improved, the PF Scale had the highest GRI followed by the walking item of the ODI, but overlapping confidence intervals demonstrated that none of the measures examined were significantly different. When the walking capacity change score was used to dichotomize patients as clinically improved or not, again the GRI was highest (1.67) for the PF Scale followed by the walking item of the ODI; however, all with overlapping 84% confidence interval differences in GRIs were not statistically significant. ROC curve analysis
Results Twenty-six patients were included in this study (17 women and 9 men), with an average age of 68.5 years (SD, 9.2), back pain duration of 13.2 years (SD, 13.0), and leg pain duration of 9.0 years (SD, 11.1). Baseline and follow-up scores for the primary outcomes are presented in Table 1. The mean follow-up time was 23.8562.6 months (range, 19–30 months). Of the 26 participants, 11 (42%) had surgery for LSS during the 2 years between baseline and follow-up, and of these 11 patients, 10 also had some form of nonsurgical therapy. Of the 15 patients who did not have surgery, eight underwent conservative treatment and seven had no treatment over the 2-year period. Longitudinal construct validity: correlational analysis Results of the correlational analysis can be found in Table 2. The highest correlations with change in the SPWT were r50.78 (p5.000) for the ODI walking item and r50.78 (p5.000) for the walking capacity change score. Using the MDC of the SPWT to dichotomize into changed or not, five patients (19.2%) exhibited true change in walking capacity. The change question used to evaluate perceived change in walking capacity demonstrated that when using a score of 6 or higher, nine patients (34.6%) Table 1 Mean values 6 SD for the Self-Paced Walking Test, Physical Function Scale, and Oswestry Disability Index at baseline and follow-up Outcome measure
Baseline
Follow-up
Self-Paced Walking Test Physical Function Scale Oswestry Disability Index Physical Function Scale Item 1 Oswestry Disability Index Item 4
1337.366943.46 2.1260.56 44.63612.44 2.1260.71 2.8861.2
1239.046918.65 1.8160.58 40.42611.73 1.8860.65 2.6961.4
SD, standard deviation.
Receiver operating characteristic curves are presented in Figs. 1 and 2. Using the MDC of the SPWT to dichotomize patients as improved or not, the AUCs were 0.77 (SD, 0.14) for the PF Scale, 0.73 (SD, 0.12) for the ODI, 0.64 (SD, 0.17) for Item 1 of the PF Scale, and 0.76 (SD, 0.15) for Item 4 from the ODI (Fig. 1). The DeLong statistic comparing the AUC of the PF scale and ODI was z50.47 (p5.32) and z51.47 (p5.87) when comparing Item 1 of the PF Scale with Item 4 from the ODI. Using the walking capacity change score to dichotomize patients as improved or not, the AUC values were 0.92 (0.05) for the SPWT, 0.94 (0.04) for the PF Scale, 0.96 (0.03) for the ODI, 0.71 (0.10) for Item 1 of the PF Scale, and 0.86 (0.07) for Item 4 of the ODI (Fig. 2). The DeLong test statistic comparing the AUC for the PF scale and ODI was z50.29 (p5.39). Comparing the AUC for the PF scale and SPWT, the statistic was z50.34 (p5.37) and z50.57 (p5.29) when comparing the ODI and SPWT. Additionally, the DeLong test comparing the AUC for Item 1 of the PF Scale and Item 4 of the ODI was z51.45 (p5.07).
Discussion This is the first study of which we are aware to use an objective gold standard measure of the construct of walking capacity to examine the validity and responsiveness of selfreported measurements of change in walking capacity. None of the self-reported measures met the standard required to be considered an acceptable surrogate for actual change in walking capacity as measured with the SPWT. Of the self-reported measures investigated, both the walking capacity change score and the walking item from the ODI were most highly correlated with the SPWT (r50.78). Although the high correlation between the walking capacity change score and the SPWT (r50.78)
C.C. Tomkins-Lane et al. / The Spine Journal
-
(2014)
-
5
Table 2 Spearman rank correlation coefficients between change scores for measures of walking capacity SPWT change SPWT change PF change ODI change PF Item 1 change ODI Item 4 change Self-reported change
PF change
1.00 0.56** (0.90 to 0.19) 1.00 0.70** (0.93 to 0.25) 0.72** (0.35 to 0.98) 0.50** (0.90 to 0.20) 0.50** (0.22 to 0.91) 0.78** (1.04 to 0.50) 0.78** (0.46 to 1.02)
0.50** (0.23 to 0.92)
ODI change
PF Item 1 change
1.00 0.50** (0.15 to 0.87)
1.00
0.78** (0.44 to 1.02)
ODI Item 4 change
0.61** (0.39 to 1.00)
Self-reported change
1.00
0.70** (1.02 to 0.43) 0.80** (1.02 to 0.46) 0.46* (0.85 to 0.11) 0.73** 1.00 (1.01 to 0.43)
SPWT, Self-Paced Walking Test; PF, Physical Function Scale of Swiss Spinal Stenosis Questionnaire; ODI, Oswestry Disability Index. *p!.05; **p!.01 (95% confidence interval).
indicates that individuals are capable of detecting change in their walking over time, the change score does not provide any information about walking distance or degree of limitation. Therefore, its use would be limited to studies of change. The walking item of the ODI does ask about pain-limited walking distance and could therefore be used to estimate walking distance, as well as to assess change. The validity of this item is supported by our earlier crosssectional study of baseline measurements where this item was found to be most highly correlated with the SPWT (r50.83) [11]. This item was also shown to correlate strongly (r50.62) with objectively measured walking in a study comparing self-reported measures with the Shuttle Walking Test in people with chronic low back pain [32]. Thus, the use of this one item may be preferable to other self-reported instruments as a surrogate for the SPWT when an objective walking test is not practical. The ODI showed a stronger relationship to change in walking capacity (SPWT; r50.70) than the PF Scale (r50.56), yet no statistically significant difference in their responsiveness was found when using the SPWT as the dichotomizing variable in GRI and ROC analysis. Similarly, when using the self-reported walking capacity change variable as the external criterion, there was no significant
difference in the responsiveness indices between the PF Scale and the ODI. These results suggest that the PF Scale and ODI are similarly sensitive to change in both measured and reported walking capacities, with the ODI performing slightly better in correlational analysis. Other studies have examined responsiveness of the Physical Function (PF) Scale and ODI for use in LSS populations but have used either patient satisfaction or a global change scale as the external criterion variable, rather than an objective walking test [9,14–16]. Stucki et al. [9] found the PF Scale to be more responsive in identifying change in satisfaction than the nondisease-specific Sickness Impact Profile and Roland Morris Questionnaire. In another study, Stucki et al. [6] found that a change in the PF Scale correlated (0.72) with patient satisfaction after surgery. Cleland et al. [15] reported on the responsiveness of both the PF Scale and ODI using a global change scale as the external criterion. This study found the ODI to have a higher AUC and correlation with the change score compared with the PF Scale, whereas the PF scale demonstrated a higher GRI. These results differ from the present study where the PF Scale was found to have a higher AUC and GRI compared with the ODI (although not statistically different). The difference in results likely stems from the difference in choice
Table 3 Mean change 6 SD for measures of walking capacity in subjects who did and did not change according to the SPWT and self-report as well as Guyatt responsiveness index values Outcome measure Physical Function Scale Oswestry Disability Index Physical Function Scale Item 1 Oswestry Disability Index Item 4
SPWT Physical Function Scale Oswestry Disability Index Physical Function Scale Item 1 Oswestry Disability Index Item 4
Did not change (SPWT)*
Changed (SPWT)
Guyatt responsiveness index (84% CI)
0.8460.67 12.83611.03 0.8060.84 1.4061.14
1.45 0.85 1.04 1.23
Did not change (self-reported)y
Changed (self-reported)
Guyatt responsiveness index (84% CI)
547.966784.53 0.0460.45 3.33610.52 0.060.79 0.3561.06
751.006871.95 0.9660.38 18.45610.99 0.6760.71 1.2260.97
0.96 1.67 0.85 0.84 1.16
0.1860.58 2.16615.13 0.1060.77 0.1061.34
(0.74–2.16) (0.37–1.33) (0.36–1.73) (0.60–1.86)
(0.44–1.48) (0.91–2.44) (0.39–1.31) (0.42–1.26) (0.73–1.59)
SD, standard deviation; SPWT, Self-Paced Walking Test; CI, confidence interval. * Change on the SPWT was defined as a difference between baseline and follow-up SPWT tests greater than 363.47 m. y Self-reported change was defined as a score of 6 or 7 on the self-reported walking change scale.
6
C.C. Tomkins-Lane et al. / The Spine Journal
Fig. 1. (Top, A) Receiver operating characteristic curve of the Physical Function Scale, (Top, B) the Oswestry Disability Index, (Bottom, C) Item 1 of the Physical Function Scale, and (Bottom, D) Item 4 of the Oswestry Disability Index with improvement defined through the use of the SelfPaced Walking Test.
of external criterion (global change vs. objective walking test). Only one other study to date has specifically examined responsiveness of measures of walking in people with LSS. In their study, Rainville et al. [16] studied patients who were undergoing treatment for LSS and neurogenic claudication. Before and after treatment, patients completed a questionnaire, as well as an SPWT and treadmill test. ‘‘Global change in clinical status’’ was chosen as the
-
(2014)
-
Fig. 2. (Top, A) Receiver operating characteristic curve of the Physical Function Scale, (Top, B) the Oswestry Disability Index, (Bottom, C) Item 1 of the Physical Function Scale, and (Bottom, D) Item 4 of the Oswestry Disability Index with improvement defined through the self-reported walking capacity change score.
external criterion for responsiveness analysis and was assessed with the following question: ‘‘Since treatment, how have your symptoms with walking changed?’’ This study concluded that neither the SPWT nor the treadmill test showed meaningful ability to detect change, given that objective changes in walking showed little concordance with patients’ perception of change in clinical status. Yet, the question we need to consider is what are we trying to measure? Rainville et al. [16] suggested that limited
C.C. Tomkins-Lane et al. / The Spine Journal
walking is the most relevant area of impaired function for patients with neurogenic claudication, and therefore, assessment of patients’ ability to walk is a standard part of evaluation for LSS. Therefore, the external criterion in such studies should logically be a measure of walking capacity. This is not to say that perceived changes in walking capacity are not important, but these perceptions seem to be tapping into a slightly different construct. Such measurements may compliment objective measures of walking capacity in future studies of patients with LSS. Results of the present study also highlight the potential danger in using retrospective change scores to assess responsiveness when an objective measure of the construct is available. Norman et al. [33] suggested that retrospective analysis of responsiveness using self-reported change scales is problematic given that often the psychometric properties of the global change score have not been established, and judgment of change is psychologically difficult. They also suggested that this type of analyses may be misleading because the responsiveness coefficient will always be greater than zero even when there is no detectable difference in the patient cohort, as long as there is a positive correlation between the self-reported instruments and the global change score [33]. This theory applies to the present study in that the PF Scale, ODI, and walking capacity change score rely on patient perception and are therefore highly related. This could lead to validity and responsiveness results that are misleading. For example, the correlations between the self-reported walking change score and changes in the PF Scale (r50.70) and ODI (r50.80) were much higher than the correlation between these measures and the SPWT (r50.56 and 0.70, respectively). If the self-reported change scale was used as the sole criterion, this would lead readers to believe that the PF Scale and ODI are more sensitive to changes in walking than they actually are. Strengths and limitations Strengths of the present study include the use of an objective criterion standard measure of walking capacity, as well as the use of multiple responsiveness analyses. Limitations include the relatively small sample size, precluding the analysis of responsiveness by subgroups (eg, surgery vs. no surgery).
Conclusions None of the self-reported measures examined in this study met the standard required to be considered an acceptable surrogate for the actual change in walking capacity as measured with the objective SPWT. It is therefore recommended that when feasible, an objective test of walking capacity be conducted to assess change. Results suggest that both the PF Scale and the ODI are able to detect changes in
-
(2014)
-
7
both measured and reported walking capacity in people with LSS. It also appears that individuals with LSS are able to detect changes in walking capacity over time using a single-item walking change score. However, this change score does not inform care providers of the degree of walking limitation and, therefore, may be of limited use outside studies of change. The individual walking capacity item from the ODI appears to be the most valid and responsive to changes in measured walking and may be a reasonable alternative for measuring walking when an objective test such as the SPWT is not feasible. References [1] Katz JN, Harris MB. Clinical practice. Lumbar spinal stenosis. N Engl J Med 2008;358:818–25. [2] Deen HG Jr, Zimmerman RS, Lyons MK, et al. Test-retest reproducibility of the exercise treadmill examination in lumbar spinal stenosis. Mayo Clin Proc 2000;75:1002–7. [3] Yamashita K, Hayashi J, Ohzono K, et al. Correlation of patient satisfaction with symptom severity and walking ability after surgical treatment for degenerative lumbar spinal stenosis. Spine 2003;28: 2477–81. [4] Tomkins CC, Battie MC, Rogers T, et al. A criterion measure of walking capacity in lumbar spinal stenosis and its comparison with a treadmill protocol. Spine 2009;34:2449. [5] Deen HG Jr, Zimmerman RS, Lyons MK, et al. Measurement of exercise tolerance on the treadmill in patients with symptomatic lumbar spinal stenosis: a useful indicator of functional status and surgical outcome. J Neurosurg 1995;83:27–30. [6] Stucki G, Daltroy L, Liang MH, et al. Measurement properties of a self-administered outcome measure in lumbar spinal stenosis. Spine 1996;21:796–803. [7] Fairbank JC, Couper J, Davies JB, et al. The Oswestry low back pain disability questionnaire. Physiotherapy 1980;66:271–3. [8] Fairbank JC, Pynsent PB. The Oswestry disability index. Spine 2000;25:2940–52. [9] Stucki G, Liang MH, Fossel AH, et al. Relative responsiveness of condition-specific and generic health status measures in degenerative lumbar spinal stenosis. J Clin Epidemiol 1995;48:1369–78. [10] Comer CM, Conaghan PG, Tennant A. Internal construct validity of the Swiss Spinal Stenosis questionnaire: Rasch analysis of a diseasespecific outcome measure for lumbar spinal stenosis. Spine 2011;1: 1969–76. [11] Tomkins-Lane CC, Battie MC. Validity and reproducibility of selfreport measures of walking capacity in lumbar spinal stenosis. Spine 2010;35:2097–102. [12] Tomkins CC, Battie MC, Hu R. Construct validity of the physical function scale of the Swiss Spinal Stenosis Questionnaire for the measurement of walking capacity. Spine 2007;32:1896–901. [13] Pratt RK, Fairbank JC, Virr A. The reliability of the Shuttle Walking Test, the Swiss Spinal Stenosis Questionnaire, the Oxford Spinal Stenosis Score, and the Oswestry Disability Index in the assessment of patients with lumbar spinal stenosis. Spine 2002;27:84–91. [14] Walsh TL, Hanscom B, Lurie JD, et al. Is a condition-specific instrument for patients with low back pain/leg symptoms really necessary? The responsiveness of the Oswestry Disability Index, MODEMS, and the SF-36. Spine 2003;28:607–15. [15] Cleland JA, Whitman JM, Houser JL, et al. Psychometric properties of selected tests in patients with lumbar spinal stenosis. Spine J 2012;12:921–31. [16] Rainville J, Childs LA, Pena EB, et al. Quantification of walking ability in subjects with neurogenic claudication from lumbar spinal stenosis—a comparative study. Spine J 2012;12:109.
8
C.C. Tomkins-Lane et al. / The Spine Journal
[17] Deyo RA. Measuring the functional status of patients with low back pain. Arch Phys Med Rehabil 1988;69:1044–53. [18] Parker SL, Mendenhall SK, Shau DN, et al. Minimum clinically important difference in pain, disability, and quality of life after neural decompression and fusion for same-level recurrent lumbar stenosis: understanding clinical versus statistical significance. J NeurosurgSpine 2012;16:471–8. [19] Lauridsen HH, Hartvigsen J, Manniche C, et al. Responsiveness and minimal clinically important difference for pain and disability instruments in low back pain patients. BMC Musculoskelet Disord 2006;7:82. [20] Copay AG, Glassman SD, Subach BR, et al. Minimum clinically important difference in lumbar spine surgery patients: a choice of methods using the Oswestry Disability Index, Medical Outcomes Study questionnaire Short Form 36, and pain scales. Spine J 2008;8: 968–74. [21] Stratford PW, Binkley J, Solomon P, et al. Assessing change over time in patients with low back pain. Phys Ther 1994;74:528–33. [22] Stratford PW, Binkley FM, Riddle DL. Health status measures: strategies and analytic methods for assessing change scores. Phys Ther 1996;76:1109–23. [23] Stratford PW, Riddle DL. Assessing sensitivity to change: choosing the appropriate change coefficient. Health Qual Life Outcomes 2005;3:23. [24] Husted JA, Cook RJ, Farewell VT, et al. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol 2000;53:459–68.
-
(2014)
-
[25] Domholdt E. Physical therapy research: principles and applications. 2nd ed. Philadelphia, PA: Saunders, 2000. [26] Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007;60:34–42. [27] Haley SM, Fragala-Pinkham MA. Interpreting change scores of tests and measures used in physical therapy. Phys Ther 2006;86:735–43. [28] Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Dis 1987;40: 171–8. [29] Tryon WW. Evaluating statistical difference, equivalence, and indeterminacy using inferential confidence intervals: an integrated alternative method of conducting null hypothesis statistical tests. Psychol Methods 2001;6:371–86. [30] Deyo RA, Centor RM. Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance. J Chronic Dis 1986;39:897–906. [31] DeLong E, DeLong D, Clarke-Pearson D. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–45. [32] Campbell H, Rivero-Arias O, Johnston K, et al. Responsiveness of objective, disease-specific, and generic outcome measures in patients with chronic low back pain: an assessment for improving, stable, and deteriorating patients. Spine 2006;31:815–22. [33] Norman GR, Stratford P, Regehr G. Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol 1997;50:869–79.
