Archives of Physical Medicine and Rehabilitation journal homepage: www.archives-pmr.org Archives of Physical Medicine and Rehabilitation 2015;96:1385-96
ORIGINAL RESEARCH
Validity of the Neuromuscular Recovery Scale: A Measurement Model Approach Craig Velozo, PhD, OTR,a Michael Moorhouse, PhD, CRC,b Elizabeth Ardolino, PhD, PT,c Doug Lorenz, PhD,d Sarah Suter, MS, PT,e D. Michele Basso, EdD, PT,f Andrea L. Behrman, PhD, PTg,h From the aDivision of Occupational Therapy, Medical University of South Carolina, Charleston, SC; bDepartment of Behavioral Science and Community Health, University of Florida, Gainesville, FL; cDoctor of Physical Therapy ProgrameAustin, University of St. Augustine for Health Sciences, Austin, TX; dDepartment of Biostatistics and Informatics, University of Louisville, Louisville, KY; eDepartment of Occupational Therapy, University of Florida, Gainesville, FL; fSchool of Health and Rehabilitation Sciences, The Ohio State University, Columbus, OH; and g Department of Neurological Surgery, University of Louisville, Louisville, KY; and hKentucky Spinal Cord Injury Research Center, University of Louisville, Louisville, KY.
Abstract Objective: To determine how well the Neuromuscular Recovery Scale (NRS) items fit the Rasch, 1-parameter, partial-credit measurement model. Design: Confirmatory factor analysis (CFA) and principal components analysis (PCA) of residuals were used to determine dimensionality. The Rasch, 1-parameter, partial-credit rating scale model was used to determine rating scale structure, person/item fit, point-measure item correlations, item discrimination, and measurement precision. Setting: Seven NeuroRecovery Network clinical sites. Participants: Outpatients (NZ188) with spinal cord injury. Interventions: Not applicable. Main Outcome Measure: NRS. Results: While the NRS met 1 of 3 CFA criteria, the PCA revealed that the Rasch measurement dimension explained 76.9% of the variance. Ten of 11 items and 91% of the patients fit the Rasch model, with 9 of 11 items showing high discrimination. Sixty-nine percent of the ratings met criteria. The items showed a logical item-difficulty order, with Stand retraining as the easiest item and Walking as the most challenging item. The NRS showed no ceiling or floor effects and separated the sample into almost 5 statistically distinct strata; individuals with an American Spinal Injury Association Impairment Scale (AIS) D classification showed the most ability, and those with an AIS A classification showed the least ability. Items not meeting the rating scale criteria appear to be related to the low frequency counts. Conclusions: The NRS met many of the Rasch model criteria for construct validity. Archives of Physical Medicine and Rehabilitation 2015;96:1385-96 ª 2015 by the American Congress of Rehabilitation Medicine
Spinal cord injury (SCI) results in a range of functional limitations, but few outcome measures evaluate a return of normal function. Recently, a new outcome measure specifically designed to assess recovery after SCI, the Neuromuscular Recovery Scale (NRS), was developed within the Christopher and Dana Reeve Foundation NeuroRecovery Network (NRN) over an 8-year period by scientists and physical therapists. The network has used the Supported by the Craig H. Neilsen Foundation (grant no. 164521). Disclosures: none.
NRS as an outcome measure since 2008. The NRS is an 11-item scale that compares sitting, standing, walking, and transfers relative to typical performance (appendixes 1 and 2). The extent of recovery is classified into 4 phases, with higher phases indicating a greater return of normal movement. Up to 3 subphases per phase capture incremental changes within each task (appendixes 1 and 2). Validity testing is needed for the NRS to establish this important psychometric property of the new scale. While there are numerous forms of validity testing, construct validity is important in that it includes a wide range of theory
0003-9993/15/$36 - see front matter ª 2015 by the American Congress of Rehabilitation Medicine http://dx.doi.org/10.1016/j.apmr.2015.04.004
1386
C. Velozo et al
testing.1 Psychometricians clearly distinguish between weak and strong forms of construct validity testing.2 Cronbach3 describes any correlation of the test score with external criterion (eg, a criterion standard) as a weak determination of construct validity, and defines an explicit theoretical explanation as a strong determination of validity. One approach to explicating the theoretical basis of an instrument is to demonstrate that the instrument measures the intended construct in a logical manner. For example, the rating scale associated with each NRS item should measure increments of more ability. For example, lower ratings on the Walking item suggest that more weight support is required to perform the movement, whereas higher ratings suggest that less weight support is required. Similarly, the NRS items should reflect different amounts of ability; that is, the Sit item should be easier for the patient to perform than the Walking item. Furthermore, patients measured by the NRS should conform to an expected ordering of patients from lower ability to higher ability. The Rasch measurement model or the 1-parameter logistic item response theory model offers an empirical way to verify the content validity of clinical measures. This measurement model has been applied to a variety of rehabilitation and health care measures.4 For example, using Rasch analysis, Avery et al5 show that the Gross Motor Function Measure-88 demonstrates a logical developmental motor sequence for children with cerebral palsy. Catz et al6 found that the Spinal Cord Independence Measure III showed that the item rating scales were properly ordered, and the item difficulty hierarchy was logical and stable across most of the clinical subgroups. The purpose of this study was to investigate the construct validity of the NRS using the Rasch, 1-parameter, item response theory partial-credit model. The following construct structures were investigated for the NRS: (1) dimensionality; (2) item rating structure; (3) fit of items and patients to the Rasch model; (4) item difficulty hierarchy; and (5) person abilityeitem difficulty match. In addition to investigating the above item and person psychometrics, the ability of the NRS to differentiate persons from different levels based on the American Spinal Injury Association Impairment Scale (AIS) was explored.
Methods Participants We examined 188 patients with motor complete and incomplete SCI across 7 outpatient clinical sites in the Christopher and Dana Reeve Foundation NRN: Boston Medical Center, Boston, Massachusetts; Frazier Rehab Institute, Louisville, Kentucky; Kessler Institute for Rehabilitation, West Orange, New Jersey; Magee Rehabilitation Hospital, Philadelphia, Pennsylvania; Ohio State University Medical Center, Columbus, Ohio; Shepherd Center, Atlanta, Georgia; and The Institute for Rehabilitation and Research
List of abbreviations: AIS CFA NRN NRS PCA SCI
American Spinal Injury Association Impairment Scale confirmatory factor analysis NeuroRecovery Network Neuromuscular Recovery Scale principal components analysis spinal cord injury
(TIRR) Memorial Hermann, Houston, Texas. Exclusion criteria included patients who (1) were ventilator dependent; (2) used a phrenic or diaphragmatic pacer; (3) had undergone tendon transfer procedures; (4) were pregnant; (5) used a cardiac demand pacemaker; (6) used oral or intrathecal delivery of antispasticity medications or morphine (could become eligible if complied with weaning from these medications); (7) had uncontrolled autonomic dysreflexia; (8) were currently using illegal drugs; (9) had active, untreated deep vein thrombosis; or (10) noncompliant participants. Enrollment criteria for patient entry in the NRN locomotor training program included (1) the presence of a nonprogressive spinal cord lesion above T11; (2) no current participation in an inpatient rehabilitation program; and (3) a medical referral by an NRN physician. Enrollment characteristics of the entire population are provided in table 1. Patients with motor incomplete SCI considered in this article were enrolled in the NRN between February 2008 and April 2011. Patients with injuries classified as AIS A and B were enrolled in this study between August 2011 and February 2013.
Neuromuscular Recovery Scale While recovery involves internal and external mechanisms, the NRS is designed to provide a functional recovery measure that focuses on noncompensatory recovery. The NRS includes 11 items focused on the capacity of the trunk and lower extremity musculature to perform set tasks (Sit, Reverse Sit-up, Sit-up, Trunk extension, Sit-to-stand, Stand, Walking, Stand retraining, Stand adaptability, Step retraining, Step adaptability) (see appendixes 1 and 2). Across
Table 1
Demographics (NZ188)
Demographic and Clinical Characteristics Sex Male Female Missing data Age (y) AIS level A B C D Missing data Injury level (missing data) Cervical Thoracic Lumbar Mechanism of injury Motor vehicle collision Fall Sporting accident Nontrauma Medical/surgical Violence Other Time since SCI (y) 3
Values 141 41 6 39.3
(75) (22) (3) (18e79)
20 19 49 98 2 2 132 53 3
(11) (10) (26) (52) (1) (1) (70) (28) (2)
67 30 36 16 14 16 9 1.2 81 56 51
(36) (16) (19) (9) (7) (9) (5) (0.1e53.1) (43) (30) (27)
NOTE. Values are n (%) or mean (range).
www.archives-pmr.org
Neuromuscular Recovery Scale validity items, the items represent a hierarchy of performance with the expectation that sit and trunk-related items will be the “easiest” items and standing and stepping items will be the “hardest” items. Within items, the rating scale represents a hierarchy of performance from the lowest level (scored phase 1A) to a high level of capacity (scored phase 4). Each item has a phase rating (1e4) and subphase ratings (AeC). Counting phases and subphases, the rating scales span from 4 to 10 ratings (see appendixes 1 and 2). Phase 1 represents the greatest impairment relative to normal movement patterns, with most people being nonambulatory and sitting being the goal. At phase 2, people begin to stand and weight support independently with associated proper kinematics. The terms normal, appropriate, and proper refer to kinematics typically used by adults when performing NRS items. Whether an item is performed with or without proper kinematics is based on observations and judgments of the clinician (eg, evidence of hip hike or foot drop would be considered abnormal kinematics). Prior medical history that might alter preinjury movement, such as arthritis, is taken into account by the clinician when rating the performance. At phase 3, walking begins with several steps to continuous stepping. Phase 4 reflects normal locomotor performance with marked adaptability to varying conditions and a return to recreational activities (eg, running). The lowest level of capacity for the item Sit, for example, is unable to sit, whereas the highest level of capacity is the ability to sit and reach forward and laterally >10in and return to the midline with typical, appropriate kinematics. In this study, the initial or baseline evaluation incorporating the NRS occurred before the initiation of the intervention, and the time to complete the NRS ranged from approximately 30 to 50 minutes depending on the degree of neuromuscular capacity. A reevaluation was conducted with the NRS every 20 intervention sessions; for example, if the intervention was provided daily (5 times/wk), then a reevaluation occurred after 4 weeks (ie, 20 sessions).
Raters At its simplest, the data were derived from 6 centers with 3 to 6 therapists at any one time, but personnel turnover makes it difficult to identify the level of education or amount of job experience of the therapists. However, the NRN uses standardization procedures to overcome these challenges. Each therapist attended standardized training on the use of the NRS provided at their clinical site, at the annual national meeting of the NeuroRecovery Network, or both. The training consisted of a review of each item, a demonstration of each task with persons without SCI and with persons with SCI of varying injury severities, and practice by the therapist in performing and scoring the NRS. Comparison of scores and rationale for scoring were discussed for each patient assessed to resolve questions and inconsistencies. Ongoing standardization of therapists occurs with site reviews of patient videos and monthly discussions on conference calls with each clinical supervisor.
Data analysis We used Mplus software7,a confirmatory factor analysis (CFA) to assess the dimensionality of the NRS. Unidimensionality is an assumption of the Rasch model. The criteria for unidimensionality were comparative fit index >.90, root mean square error of approximation .95.8 To provide a full test of dimensionality, we followed the CFA with a principal components analysis (PCA) of residuals using Winsteps software.9,b www.archives-pmr.org
1387 We applied a partial-credit, Rasch, 1-parameter, logistic rating scale model using Winsteps software.9 In contrast to traditional classical test theory analyses that focus on test-level psychometrics, Rasch analysis is a probabilistic model that focuses on itemlevel psychometrics. For example, fit statistics are used to identify items and persons fitting or misfitting the measurement model. In addition, Rasch analysis places person ability and item difficulty on the same linear continuum, allowing the identification of ceiling and floor effects. We used Linacre’s 3 essential rating scale criteria10: (1) 10 responses for each rating category; (2) rating scale average calibrations advancing monotonically with each category; and (3) outfit mean-squares for each rating scale category .95). The PCA of Rasch residuals revealed that the Rasch measurement dimension (person variance and item variance) explained 76.9% of the variance. The first component explained 5.3% of the variance (expectation of Rasch model is 4%). The variance explained by the items, 43.4%, was over 8.2 times the variance explained by the first component, suggesting the dominance of the measurement construct. An eigenvalue of 2.6 for the first component was >2.0, the smallest amount that can be considered a dimension. Eight of the 11 NRS items had .50 loadings: 4 standing/stepping items had negative loadings .50 (Stand retraining, Step adaptability, Step retraining, Stand adaptability), and 4 sitting/trunk items had positive loadings .54 (Reverse sit-up, Trunk extension in sitting, Sit, Sit-up).
Rating scale analysis Only 1 item, Trunk extension in sitting, and 69% of the ratings (59/ 85) met Linacre’s 3 essential rating scale criteria. Table 2 presents the rating categories for each item that did not meet Linacre’s 3 essential criteria. The most common limitation was items showing
1388 Table 2
C. Velozo et al Ratings not meeting rating scale essential criteria
Item (No. of Ratings)
Low Frequency
NonHigh monotonic Outfit
Stand retraining (4) 1A Stand adaptability (9) 1A 3B Step retraining (10) 3A 4A Step adaptability (10) 3A 3B 3C 4A Sit (9) 2C 2C* Sit-up (7) 1B 3C 4A 3C* 4A* Reverse sit-up (5) 3C 4A Trunk extension in sitting (8) Sit-to-stand (8) 3A 3B 3C Stand (8) 2B 3A 4A Walking (7) 3B 3C 4A
1A 1C 2A 3C* 4A* 3C* 4A*
NOTE. Phase (number) subphases (letter) represent ratings for each item (see appendixes 1 and 2). Only ratings that did not meet criterion are listed. * Items have low frequency counts.
.57. Ninety-one percent (172/188) of the patients fit the measurement model. Eight patients misfit both in infit and outfit statistics, and 5 additional patients showed only misfit of outfit statistics; 1 additional patient showed only misfit of infit statistics.
Precision Person separation reliability, which is analogous to Cronbach alpha, was .92, and person separation, an indicator of how well Table 3
the sample was separated by the instrument items, was 3.48. Item separation reliability was .90. The NRS separated the sample into 4.97 strata and items into 17.0 strata that were 3 SEs apart. Figure 1 presents the information function for the 11 items of the NRS. The ability of items to discriminate is represented by the item slope. Nine of the 11 items show high discrimination, with Sit-up and Reverse Sit-up showing low item discrimination.
Person abilityeitem difficulty match Figure 2 presents the map of item difficulty and person ability. Items are presented to the left of the dashed vertical line, with easy items at the bottom of the map and difficult items at the top. Stand retraining, Trunk extension in sitting, Sit, and Stand adaptability represent the easiest items, and Walking, Reverse sit-up, Sit-up, and Step adaptability represent the most challenging items. Persons, listed according to their AIS classification, are presented from low ability on the NRS at the bottom of the map, and persons of high ability at the top. The NRS showed no ceiling or floor effects (ie, no groupings of individuals at the top and bottom of the scale, respectively). In general, the distribution of patients according to AIS classifications appears as expected, with individuals with D classifications at the top half of the distribution and individuals with A classifications at the bottom half of the distribution. Individuals with C and B classifications were mixed in the middle to lower half of the distribution.
Discussion Overall, the NRS showed many measurement characteristics that are consistent with the theoretical framework for measuring neuromuscular task-specific function in individuals with SCI. While the NRS did not meet all CFA criterions for unidimensionality, the PCA of Rasch residuals showed that the items tended to load on a single factor. Ten of 11 items and 91% of the patients fit the Rasch measurement model. The NRS item difficulty distribution was well matched to the sample, with the instrument separating the sample into almost 5 statistically distinct strata, with 9 of 11 of the items showing high
Item statistics: misfit order Infit
Item Step retraining Reverse sit-up Sit Stand retraining Sit-up Trunk extension in sitting Step adaptability Stand adaptability Sit-to-stand Stand Walking
Measure 0.06 0.87 0.75 2.57 0.82 0.98 0.85 0.78 0.40 0.66 1.41
Model SE
MNSQ
.06 .10 .06 .12 .11 .06 .07 .06 .07 .07 .09
1.50 1.39 1.26 1.20 1.16 0.95 0.98 0.86 0.73 0.62 0.47
Outfit ZSTD 4.00 3.20 2.00 1.30 1.40 0.40 0.10 1.40 2.00 3.00 4.70
MNSQ 1.40 1.43 1.41 1.18 1.19 1.01 0.98 0.83 0.73 0.59 0.47
ZSTD 3.20 3.20 2.70 0.60 1.60 0.10 0.20 1.60 2.00 2.90 4.30
PTMEAS CORR .76 .57 .79 .52 .65 .79 .80 .85 .82 .82 .82
Abbreviations: MNSQ, mean square; PTMEAS CORR, point-measure correlation; ZSTD, standardized Z value.
www.archives-pmr.org
Neuromuscular Recovery Scale validity
Fig 1
Item characteristic curves for each of the items of the NRS.
discrimination. Items reflecting positioning (eg, Sit, Stand) were easier than items reflecting trunk motion and gait (eg, Step adaptability, Reverse sit-up, Walking). Furthermore, NRS measures in general separated patients according to the AIS classifications. The NRS showed limitations in the rating scale and discrimination ability for several items. Uncertainty in the rating scale psychometrics appears to be a function of low frequency of several rating categories. Of note, since a partial-credit model evaluates each item as having its own unique scale (vs collapsing across the rating scale for all items of the instrument), it was difficult to meet the criterion of 10 observations per rating, especially when items have a large range of rating categories. In addition, the low rating frequencies may be at least a partial explanation for items failing to meet other rating scale criteria (ie, monotonicity and model fit). The NRS has the ability to discriminate among individuals, not based on injury severity (ie, AIS classification), but on one’s recovered ability to perform a functional task. As expected, these functional abilities relate to the severity of injury, thus providing the clinician another tool for classifying patients. These results are congruent with those found by Behrman et al13 in which the NRS was found to discriminate patients on the basis of function within single AIS classifications. The easiest item was Stand retraining, most likely because of the method of administration. The patient is supported in the body weight support systemc and must maintain a kinematically appropriate standing posture with assistance from trainers. The trainers are allowed to provide the patient with whatever assistance is needed to maintain this appropriate standing posture. Therefore, the item may actually test the training team’s ability to hold a person in the appropriate posture, instead of the person’s true ability to maintain this posture. Furthermore, in other studies,14,15 test-retest and interrater reliabilities were shown to be low for Stand retraining, and most subjects had a ceiling effect for this item. The most difficult items were Walking, Reverse Sit-up, Sit-up, and Step adaptability. All of these items require
www.archives-pmr.org
1389 increased trunk control through a larger range of motion or in dynamic positions. As expected, the Walking item was the most difficult item for this sample. The scoring of the item involves the person’s ability to control the trunk while weight shifting in standing, and progresses to being able to maintain appropriate kinematics at the trunk, pelvis, and bilateral lower extremities during walking and negotiating obstacles, independently. The wide range of difficulty of the NRS items and the lack of floor and ceiling effects make this tool a valuable outcome measure for the SCI population. A person with an SCI, theoretically, can be evaluated on the NRS in the acute care setting, and continue to be assessed by this same instrument through outpatient rehabilitation. The NRS is the only outcome measure specifically designed for the SCI population that has the ability to test a variety of functional activities throughout a wide spectrum of recovery while disallowing compensation. In addition to our findings supporting the validity of the NRS, these findings suggest that the NRS supports task-specific training and can be useful for clinical decision-making. First, the functional items phase scoring of the NRS allows the identification of patient impairments and functional deficits that can be the focus of treatment. For example, for the item Stand Adaptability, a rating of 1A indicates the patient is unable to maintain proper trunk kinematics, suggesting that interventions to support an upright trunk would be important. Second, the item difficulty order (consistent across 91% of study participants), which shows Trunk extension in sitting and Sit as the easiest items, supports the theory that posture is foundational to more challenging functional tasks such as Walking and Step adaptability. This finding is consistent with studies showing that atypical, compensatory trunk movements promote maladaptive, nonoptimal recovery of the extremities16,17 and that postural training improves motor deficits in other neurologic conditions.18 Finally, the rating scale hierarchical progression (consistent across 96% of the subphases [ratings]) supports a framework for a graded treatment approach such as a graded reduction in physical assistance and a graded improvement in kinematics. Graded treatment approaches are common in clinical practice and are conceptually and empirically supported in the literature.19,20
Study limitations To date, all testing has been conducted using highly skilled therapists who had a significant amount of experience with the tool. We are uncertain what degree of training is necessary to achieve competency in administering and scoring this instrument. Future studies will examine this and further establish the psychometric properties of the NRS. Further studies are also needed to determine whether a subscale of only the overground items would be a valid instrument. The use of only overground items would extend the utility of the NRS into clinics that lack the necessary equipment to assess the treadmill items. Since the NRS is intended to measure change, the psychometric stability of the item calibrations need to be investigated in future studies. Also, expanding the NRS to include upper extremity items is also recommended to broaden the utility of the instrument as an assessment and progression tool for neuromuscular recovery.
1390
C. Velozo et al
Fig 2 Map of item difficulty measures relative to AIS classification. This plot presents the item difficulties (left of dashed vertical line) against the AIS classifications (right of dashed vertical line). Lower (more negative) values represent “easy” items and individuals of lower ability, while higher (more positive) values represent more challenging items and individuals of higher ability. Abbreviations: M, mean; REV SIT, Reverse sit-up; S, 1 SD; STAND ADAPT, Stand adaptability; STAND RETRAIN, Stand retraining; STEP ADAPT, Step adaptability; STEP RETRAIN, Step retraining; T, 2 SDs; TRUNK EXT SIT, Trunk extension in sitting.
Conclusions The NRS meets many of the Rasch model criteria for construct validity for use in individuals with SCI for a wide range of injury severities, AIS A through D.
Suppliers a. Mplus software; Muthe´n & Muthe´n. b. Winsteps software; Available at: http://www.winsteps.com. c. Therastride; Innoventor. Currently manufactured by Power NeuroRecovery (www.powerneurecovery.weebly.com).
Keywords Outcome assessment (health care); Recovery of function; Rehabilitation; Spinal cord injuries
Corresponding author Craig Velozo, PhD, OTR, Medical University of South Carolina, Division of Occupational Therapy, 151-B Rutledge Ave, MSC 962, Charleston, SC 29425. E-mail address: [email protected].
Acknowledgments We thank the members of the Christopher and Dana Reeve Foundation NRN for their previous efforts in NRS development and current work on data collection; and Sarah Morrison, PT (Shepherd Center, Atlanta, GA) and Mary Schmidt, PT, DPT, MS (Magee Rehab Hospital, Philadelphia, PA) for their reviews of drafts and suggestions. We also acknowledge the current or past NRN Center Directors: Steve Ahr and Douglas Stevens, MD (Frazier Rehab Institute); Daniel E. Graves, PhD (Memorial Hermann/The Institute of Rehabilitation and Research); Sarah Morrison, PT, and Keith Tansey, MD, PhD (Shepherd Center); Gail F. Forrest, PhD (Kessler Medical Rehabilitation Research and Education Corp); and D. Michele Basso, PT, EdD (The Ohio State University Medical Center); Mary Schmidt Read, PT, DPT, MS (Magee Rehabilitation Hospital); plus all other current and previous network members; the NRN Reeve Foundation’s Susan Howley and Joseph Canose; and the leadership, foresight, and support of V. Reggie Edgerton, PhD, Moses Chao, PhD, Michael Fehlings, MD, PhD, Andrei Krassioukov, MD, PhD, and Shelly Sorani, MA. For a list of all NRN team members, please visit: http://www. christopherreeve.org/site/c.ddJFKRNoFiG/b.5399929/k.6F37/Neuro Recovery_Network.htm. www.archives-pmr.org
Neuromuscular Recovery Scale validity
Appendix 1 Administration and Scoring of the Phases Administering the NRS The administration of the phasing tool should take place during 1 evaluation period. The testing should begin with the first of the overground items, sit, continue through the overground items, and then follow with the items in the body weight support treadmill environment. Each item is tested and scored separately. The score of the item is the highest phase in which the patient can complete the item according to the description without any compensation. The examiner then calculates the overall phase based on the combined phase scores of each of the individual items. The continuum of scores proceeds from unable to complete the task to achieving full recovery by executing the tasks as done preinjury. Assess the patient beginning with phase 1A abilities and then move up the phase scoring, because the lower phases must be achieved before higher scoring can be considered. Assistance cannot be provided to the body segment being assessed. The lower of 2 debatable scores should be selected. There is no advantage to scoring higher, and all assessments should be directly compared with the ability preinjury to complete the specific task. This is important because goal setting and progression will be based on the current phase scoring. Scoring lower, if unsure of 2 possible scores, guarantees that the correct task and neuromuscular skill will be sufficiently targeted with therapy aimed at this skill deficit. Subphases that are not scored are designated as not applicable and account for the progression of recovery within a task and across tasks.
1391 2C); able to sit and complete forward and lateral reaches that extend to 12.7cm (5in) (phase 3A); able to sit and complete forward and lateral reaches that extends >12.7cm and 25.4cm (10in) beyond the tips of the fingers when the arm is raised to 90 of shoulder flexion or abduction (phase 3C); and not applicable (phase 4).
Reverse sit-up The patient is positioned in an unsupported sitting position at the edge of the mat, with the feet flat on the floor. The patient is asked to slowly lower his/her trunk down to the mat until the back and head are flat on the mat, without using the upper extremities for support and while maintaining a 90 angle in the knees. The examiner scores the patient based on how controlled the patient can lower the trunk, the position of the upper extremities, the compensatory use of the lower extremities, and the amount of effort required to complete the task. Manual assistance is not provided unless needed to avoid striking the mat with excessive force. Scores are categorized as follows: not applicable (phase 1Ae1C); unable (phase 2A); patient shows control through the first 45 but loses control during the lower half of the movement (phase 2B); not applicable (phase 2C); patient is able to complete task with arms elevated in order to provide a counterbalance (phase 3A); not applicable (phase 3B); patient is able to complete task with arms held across chest, but displaying considerable effort (phase 3C); and patient is able to complete task with arms held across chest without considerable effort (phase 4).
Abilities during overground assessment
Sit-up
Overground assessment defines the capacity of the neuromuscular system without the benefit of the body weight support treadmill or manual facilitation to execute specific motor tasks.
The patient is positioned in a supine position on the mat, with the lower extremities off the edge of the mat. The patient is asked to return to a sitting position without the use of the upper extremities, manual assistance, or momentum, while maintaining the knees at a 90 position. The examiner scores the patient based on how much of the trunk is able to be raised off the mat, the position of the upper extremities, the compensatory use of the lower extremities, and the amount of effort required to complete the task. Scores are categorized as follows: not applicable (phase 1A); unable (phase 1B); patient is able to raise head (phase 1C); patient is able to raise head and initiate shoulder lift (phase 2A); patient is able to raise head, shoulders, and scapulae (phase 2B); not applicable (phase 2C); patient is able to sit up, but with inappropriate position (arms elevated and/or hip/knee flexion/extension) (phase 3A); not applicable (phase 3B); patient is able to sit up properly, but displaying considerable effort (phase 3C); and patient is able to sit up properly without considerable effort (phase 4).
Sit The patient is positioned in an unsupported sitting position at the edge of a firm mat. The patient’s feet should be flat on the floor. The patient is then asked to sit without upper extremity support in a proper posture position (ie, head and shoulders in alignment, neutral pelvis position). If the patient is unable to attain this position, the examiner may assist the patient into this position and then ask the patient to maintain the posture. The patient is then assigned a phase based on the patient’s ability to maintain the proper posture in an unsupported position. If the patient is able to attain and maintain the proper posture (ie, phase 2), then the patient is asked to elevate the upper extremities, as described in the chart, and reach forward and laterally. The patient is assigned a phase based on these abilities. If the patient is incapable of elevating the upper extremities because of complications of the neuromuscular system, the patient may be graded on the ability to lean the trunk forward and laterally. Scores are categorized as follows: unable (phase 1A); unable to attain, able to sit with inappropriate posture (phase 1B); unable to attain, able to sit with appropriate posture (phase 1C); able to attain and maintain appropriate position for at least 1 minute (phase 2A); able to attain and maintain appropriate position indefinitely (phase 2B); able to sit and hold arms parallel to legs for at least 30 seconds (phase www.archives-pmr.org
Trunk extension in sitting The patient is positioned in sitting with the feet flat on the floor, leans forward at the waist (or is assisted forward) to rest the chest fully on the lap. The patient is asked to return to an upright sitting position without the use of the upper extremities or manual assistance. The examiner grades the patient based on how much of the trunk is raised into sitting, the position of the upper extremities, and the amount of effort required to complete the task.
1392 Scores are categorized as follows: unable (phase 1A); patient is able to initiate thoracic spine extension (phase 1B); patient is able to initiate and maintain thoracic spine extension (phase 1C); patient is able to initiate and maintain thoracic and initiate lumbar spine extension (phase 2A); patient is able to initiate and maintain thoracic and lumbar spine extension but displays considerable effort (phase 2B); not applicable (phase 2C); patient is able to initiate and maintain thoracic and lumbar spine extension without considerable effort (phase 3A); not applicable (phase 3B); patient is able to complete task with arms behind head, but displays considerable effort (phase 3C); and patient is able to complete task with arms behind head without considerable effort (phase 4).
Sit-to-stand The patient begins seated at the edge of the mat with feet flat on the floor. The patient is asked to stand up from the mat without the use of the upper extremities. The examiner first assesses how high the patient can raise his/her body off the mat without any assistance from the examiner. If the patient is able to raise the body approximately 50% off the mat, then the examiner may provide assistance to the patient’s pelvis and/or lower extremities, and assess the patient’s control at the trunk and pelvis, as appropriate. If the patient is able to transition from sit to stand without physical assistance from the examiner, the patient is scored according to his/her kinematics during the task and the position of the upper extremities during the task. Scores are categorized as follows: not applicable (phase 1Ae1B); unable (phase 1C); patient is able to initiate weight bearing (phase 2A); the patient is able to raise himself up off the mat towards standing and achieves 50% of the trajectory towards standing (phase 2B); patient is able to complete the task while maintaining proper kinematics of the head, shoulders, and trunk, and manual facilitation may be provided at the pelvis, knees, or ankles if needed (phase 2C); patient is able to complete the task with proper kinematics of the head, shoulders, and trunk, and manual facilitation may be provided at the knees or ankles if needed (phase 3A); patient is able to complete the task with proper kinematics of the head, shoulders, trunk, and pelvis, inappropriate kinematics at the knees and ankles, and no manual facilitation (phase 3B); patient is able to complete the task with proper kinematics, no manual facilitation, and uses the arms as a counterbalance (phase 3C); and patient is able to complete task properly without using the arms as a counterbalance (phase 4).
Stand The patient begins in a standing position, with physical assistance provided by the examiner as needed. The patient is asked to stand with a proper posture (ie, head, shoulders, and trunk extended, pelvis properly positioned under the head and shoulders, with the knees extended to adequately maintain body weight without hyperextension). The patient is assigned a phase based on the ability to maintain appropriate kinematics at each body segment. Please note that when the trunk is assessed (ie, phases 1Ce2A), assistance may be provided to the lower extremities and pelvis by the examiner and other trainers. Likewise, assistance may be provided to the lower extremities in phase 2 to assess the kinematics of the pelvis. To be assigned phase 3, the patient must be able to maintain standing balance without physical assistance. Scores are categorized as follows: not applicable (phase 1Ae1B); unable (phase 1C); patient is able to stand with inappropriate posture and manual facilitation at the pelvis, knees, or ankles provided as
C. Velozo et al needed (phase 2A); patient is able to stand with appropriate posture for at least 1 minute, and manual facilitation provided at the knees and ankles as needed (phase 2B); patient is able to stand with appropriate posture for at least 1 minute, and manual facilitation provided at the knees and ankles as needed (phase 2C); patient is able to stand with appropriate posture and without manual facilitation for at least 1 minute (phase 3A); patient is able to stand with appropriate posture without manual facilitation for at least 1 minute (phase 3B); and patient is able to stand with appropriate posture and without manual facilitation indefinitely (phase 3C).
Walking The patient begins in standing position, with physical assistance as needed by the examiner at pelvis and lower extremities. The patient is asked to shift weight laterally. If the patient is able to shift weight without assistance at the trunk, the patient is then asked to shift weight forward and back in a stride position (left and right). The patient is assigned a phase based on the ability to assume stride right and stride left positions while maintaining appropriate kinematics at each body segment. If the patient is able to independently assume stride positions and perform weight shifts independently while maintaining appropriate kinematics at all body segments, the patient is assigned a phase based on the ability to perform repetitive steps with appropriate kinematics. Scores are categorized as follows: not applicable (phase 1Ae1C); unable to shift weight (phase 2A); patient is able to shift weight laterally with inappropriate kinematics at the head, shoulders, and trunk (phase 2B); patient is able to shift weight laterally with appropriate kinematics at the head, shoulders, and trunk, but is unable to initiate the stride position (phase 2C); patient is able to shift weight laterally with appropriate kinematics of the head, shoulders, and trunk, but inappropriate kinematics of the legs (phase 3A); patient is able to shift weight laterally with appropriate kinematics, but not in stride (phase 3B); patient is able to shift weight laterally and in stride with appropriate kinematics (phase 3C); and patient is able to walk with appropriate kinematics (phase 4).
Abilities during treadmill assessment Once the overground assessment has been concluded, the patient is positioned in a body weight support treadmill system. The steptraining environment uses the body weight support treadmill and manual assistance to assess the capacity and independence of the nervous system to stand and generate steps in a safe environment. The capacity of the nervous system is assessed by identifying the treadmill speed, body weight support, and facilitation needed to generate the optimal stepping pattern. This capacity of the nervous system is referred to as retraining. The independence of the nervous system is referred to as adaptability, and is assessed by identifying the treadmill speed and body weight support where independence from manual assistance occurs. There are 4 areas of assessment during step training: (1) Stand retraining, (2) Stand adaptability, (3) Step retraining, and (4) Step adaptability.
Stand retraining In this area, the capacity of the nervous system to bear weight in static standing is assessed, recorded as body weight support. The www.archives-pmr.org
Neuromuscular Recovery Scale validity patient is positioned in the body weight support treadmill environment in an upright posture, with head, shoulders, and trunk extended, the pelvis properly positioned under the head and shoulders, with the knees extended to adequately maintain body weight without compensation. The patient should initially be supported with 75% body weight support (or the maximum amount of body weight support with the patient’s feet remaining flat on the treadmill). The body weight support is then lowered as far as possible for the patient to maintain proper posture and positioning of the head, trunk, pelvis, and legs for at least 5 minutes, with the trainers providing as much physical assistance as needed to maintain the patient in an upright standing position. The lowest body weight support that is achieved is then noted, and the patient is assigned a phase level based on this amount of body weight support. Scores are categorized as a percent of total body weight support which the patient can maintain with trainers providing assistance: body weight support at 40% (phase 1A); body weight support 20% to 39% (phase 1B); not applicable (phase 1Ce2A); body weight support 10% to 19% (phase 2B); not applicable (phase 2Ce3B); body weight support at 10% (phase 3C); and not applicable (phase 4).
1393 associated with walking. Leg loading is maximized, as are normative walking speeds (.89e1.34m/s; 2.0e3.0mph), resulting in scores of body weight support and speed. The body weight support level is adjusted to the body weight support where most of the body segments were independent during stand adaptability. The patient then begins stepping at a speed of .89m/s (2.0mph), with physical assistance from the trainers to ensure the best posture and kinematics while walking. The body weight support and speed are adjusted up or down to assess the lowest body weight support and highest speed at which the patient can generate the best stepping pattern with assistance from the trainers. The patient is assigned a phase level based on these 2 variables. Scores are categorized as follows: body weight support is set at 60% (phase 1A); body weight support 55% to 59% (phase 1B); body weight support 50% to 54% (phase 1C); body weight support 45% to 49% (phase 2A); body weight support 40% to 44% (phase 2B); body weight support 35% to 39% (phase 2C); body weight support 30% to 34% (phase 3A); body weight support 20% to 29% (phase 3B); and body weight support is at 20% (phase 3C). Patients who can maintain proper kinematics with 50% body weight support while running at speeds of 1.52m/s (3.4mph) are scored as phase 4.
Stand adaptability In this area, the patient’s ability to independently maintain appropriate kinematics at each body segment without physical assistance is assessed, reflected by body weight support required. The patient begins at the lowest body weight support achieved during Stand retraining, in an upright posture with head, shoulders, and trunk extended, the pelvis properly positioned under the head and shoulders, and the knees and ankles positioned at neutral to adequately maintain body weight. The body weight support is then adjusted until the patient is able to independently maintain an upright posture at the trunk, with assistance provided at the pelvis and lower extremities. If the patient is able to maintain an upright posture at the trunk with
ORIGINAL RESEARCH
Validity of the Neuromuscular Recovery Scale: A Measurement Model Approach Craig Velozo, PhD, OTR,a Michael Moorhouse, PhD, CRC,b Elizabeth Ardolino, PhD, PT,c Doug Lorenz, PhD,d Sarah Suter, MS, PT,e D. Michele Basso, EdD, PT,f Andrea L. Behrman, PhD, PTg,h From the aDivision of Occupational Therapy, Medical University of South Carolina, Charleston, SC; bDepartment of Behavioral Science and Community Health, University of Florida, Gainesville, FL; cDoctor of Physical Therapy ProgrameAustin, University of St. Augustine for Health Sciences, Austin, TX; dDepartment of Biostatistics and Informatics, University of Louisville, Louisville, KY; eDepartment of Occupational Therapy, University of Florida, Gainesville, FL; fSchool of Health and Rehabilitation Sciences, The Ohio State University, Columbus, OH; and g Department of Neurological Surgery, University of Louisville, Louisville, KY; and hKentucky Spinal Cord Injury Research Center, University of Louisville, Louisville, KY.
Abstract Objective: To determine how well the Neuromuscular Recovery Scale (NRS) items fit the Rasch, 1-parameter, partial-credit measurement model. Design: Confirmatory factor analysis (CFA) and principal components analysis (PCA) of residuals were used to determine dimensionality. The Rasch, 1-parameter, partial-credit rating scale model was used to determine rating scale structure, person/item fit, point-measure item correlations, item discrimination, and measurement precision. Setting: Seven NeuroRecovery Network clinical sites. Participants: Outpatients (NZ188) with spinal cord injury. Interventions: Not applicable. Main Outcome Measure: NRS. Results: While the NRS met 1 of 3 CFA criteria, the PCA revealed that the Rasch measurement dimension explained 76.9% of the variance. Ten of 11 items and 91% of the patients fit the Rasch model, with 9 of 11 items showing high discrimination. Sixty-nine percent of the ratings met criteria. The items showed a logical item-difficulty order, with Stand retraining as the easiest item and Walking as the most challenging item. The NRS showed no ceiling or floor effects and separated the sample into almost 5 statistically distinct strata; individuals with an American Spinal Injury Association Impairment Scale (AIS) D classification showed the most ability, and those with an AIS A classification showed the least ability. Items not meeting the rating scale criteria appear to be related to the low frequency counts. Conclusions: The NRS met many of the Rasch model criteria for construct validity. Archives of Physical Medicine and Rehabilitation 2015;96:1385-96 ª 2015 by the American Congress of Rehabilitation Medicine
Spinal cord injury (SCI) results in a range of functional limitations, but few outcome measures evaluate a return of normal function. Recently, a new outcome measure specifically designed to assess recovery after SCI, the Neuromuscular Recovery Scale (NRS), was developed within the Christopher and Dana Reeve Foundation NeuroRecovery Network (NRN) over an 8-year period by scientists and physical therapists. The network has used the Supported by the Craig H. Neilsen Foundation (grant no. 164521). Disclosures: none.
NRS as an outcome measure since 2008. The NRS is an 11-item scale that compares sitting, standing, walking, and transfers relative to typical performance (appendixes 1 and 2). The extent of recovery is classified into 4 phases, with higher phases indicating a greater return of normal movement. Up to 3 subphases per phase capture incremental changes within each task (appendixes 1 and 2). Validity testing is needed for the NRS to establish this important psychometric property of the new scale. While there are numerous forms of validity testing, construct validity is important in that it includes a wide range of theory
0003-9993/15/$36 - see front matter ª 2015 by the American Congress of Rehabilitation Medicine http://dx.doi.org/10.1016/j.apmr.2015.04.004
1386
C. Velozo et al
testing.1 Psychometricians clearly distinguish between weak and strong forms of construct validity testing.2 Cronbach3 describes any correlation of the test score with external criterion (eg, a criterion standard) as a weak determination of construct validity, and defines an explicit theoretical explanation as a strong determination of validity. One approach to explicating the theoretical basis of an instrument is to demonstrate that the instrument measures the intended construct in a logical manner. For example, the rating scale associated with each NRS item should measure increments of more ability. For example, lower ratings on the Walking item suggest that more weight support is required to perform the movement, whereas higher ratings suggest that less weight support is required. Similarly, the NRS items should reflect different amounts of ability; that is, the Sit item should be easier for the patient to perform than the Walking item. Furthermore, patients measured by the NRS should conform to an expected ordering of patients from lower ability to higher ability. The Rasch measurement model or the 1-parameter logistic item response theory model offers an empirical way to verify the content validity of clinical measures. This measurement model has been applied to a variety of rehabilitation and health care measures.4 For example, using Rasch analysis, Avery et al5 show that the Gross Motor Function Measure-88 demonstrates a logical developmental motor sequence for children with cerebral palsy. Catz et al6 found that the Spinal Cord Independence Measure III showed that the item rating scales were properly ordered, and the item difficulty hierarchy was logical and stable across most of the clinical subgroups. The purpose of this study was to investigate the construct validity of the NRS using the Rasch, 1-parameter, item response theory partial-credit model. The following construct structures were investigated for the NRS: (1) dimensionality; (2) item rating structure; (3) fit of items and patients to the Rasch model; (4) item difficulty hierarchy; and (5) person abilityeitem difficulty match. In addition to investigating the above item and person psychometrics, the ability of the NRS to differentiate persons from different levels based on the American Spinal Injury Association Impairment Scale (AIS) was explored.
Methods Participants We examined 188 patients with motor complete and incomplete SCI across 7 outpatient clinical sites in the Christopher and Dana Reeve Foundation NRN: Boston Medical Center, Boston, Massachusetts; Frazier Rehab Institute, Louisville, Kentucky; Kessler Institute for Rehabilitation, West Orange, New Jersey; Magee Rehabilitation Hospital, Philadelphia, Pennsylvania; Ohio State University Medical Center, Columbus, Ohio; Shepherd Center, Atlanta, Georgia; and The Institute for Rehabilitation and Research
List of abbreviations: AIS CFA NRN NRS PCA SCI
American Spinal Injury Association Impairment Scale confirmatory factor analysis NeuroRecovery Network Neuromuscular Recovery Scale principal components analysis spinal cord injury
(TIRR) Memorial Hermann, Houston, Texas. Exclusion criteria included patients who (1) were ventilator dependent; (2) used a phrenic or diaphragmatic pacer; (3) had undergone tendon transfer procedures; (4) were pregnant; (5) used a cardiac demand pacemaker; (6) used oral or intrathecal delivery of antispasticity medications or morphine (could become eligible if complied with weaning from these medications); (7) had uncontrolled autonomic dysreflexia; (8) were currently using illegal drugs; (9) had active, untreated deep vein thrombosis; or (10) noncompliant participants. Enrollment criteria for patient entry in the NRN locomotor training program included (1) the presence of a nonprogressive spinal cord lesion above T11; (2) no current participation in an inpatient rehabilitation program; and (3) a medical referral by an NRN physician. Enrollment characteristics of the entire population are provided in table 1. Patients with motor incomplete SCI considered in this article were enrolled in the NRN between February 2008 and April 2011. Patients with injuries classified as AIS A and B were enrolled in this study between August 2011 and February 2013.
Neuromuscular Recovery Scale While recovery involves internal and external mechanisms, the NRS is designed to provide a functional recovery measure that focuses on noncompensatory recovery. The NRS includes 11 items focused on the capacity of the trunk and lower extremity musculature to perform set tasks (Sit, Reverse Sit-up, Sit-up, Trunk extension, Sit-to-stand, Stand, Walking, Stand retraining, Stand adaptability, Step retraining, Step adaptability) (see appendixes 1 and 2). Across
Table 1
Demographics (NZ188)
Demographic and Clinical Characteristics Sex Male Female Missing data Age (y) AIS level A B C D Missing data Injury level (missing data) Cervical Thoracic Lumbar Mechanism of injury Motor vehicle collision Fall Sporting accident Nontrauma Medical/surgical Violence Other Time since SCI (y) 3
Values 141 41 6 39.3
(75) (22) (3) (18e79)
20 19 49 98 2 2 132 53 3
(11) (10) (26) (52) (1) (1) (70) (28) (2)
67 30 36 16 14 16 9 1.2 81 56 51
(36) (16) (19) (9) (7) (9) (5) (0.1e53.1) (43) (30) (27)
NOTE. Values are n (%) or mean (range).
www.archives-pmr.org
Neuromuscular Recovery Scale validity items, the items represent a hierarchy of performance with the expectation that sit and trunk-related items will be the “easiest” items and standing and stepping items will be the “hardest” items. Within items, the rating scale represents a hierarchy of performance from the lowest level (scored phase 1A) to a high level of capacity (scored phase 4). Each item has a phase rating (1e4) and subphase ratings (AeC). Counting phases and subphases, the rating scales span from 4 to 10 ratings (see appendixes 1 and 2). Phase 1 represents the greatest impairment relative to normal movement patterns, with most people being nonambulatory and sitting being the goal. At phase 2, people begin to stand and weight support independently with associated proper kinematics. The terms normal, appropriate, and proper refer to kinematics typically used by adults when performing NRS items. Whether an item is performed with or without proper kinematics is based on observations and judgments of the clinician (eg, evidence of hip hike or foot drop would be considered abnormal kinematics). Prior medical history that might alter preinjury movement, such as arthritis, is taken into account by the clinician when rating the performance. At phase 3, walking begins with several steps to continuous stepping. Phase 4 reflects normal locomotor performance with marked adaptability to varying conditions and a return to recreational activities (eg, running). The lowest level of capacity for the item Sit, for example, is unable to sit, whereas the highest level of capacity is the ability to sit and reach forward and laterally >10in and return to the midline with typical, appropriate kinematics. In this study, the initial or baseline evaluation incorporating the NRS occurred before the initiation of the intervention, and the time to complete the NRS ranged from approximately 30 to 50 minutes depending on the degree of neuromuscular capacity. A reevaluation was conducted with the NRS every 20 intervention sessions; for example, if the intervention was provided daily (5 times/wk), then a reevaluation occurred after 4 weeks (ie, 20 sessions).
Raters At its simplest, the data were derived from 6 centers with 3 to 6 therapists at any one time, but personnel turnover makes it difficult to identify the level of education or amount of job experience of the therapists. However, the NRN uses standardization procedures to overcome these challenges. Each therapist attended standardized training on the use of the NRS provided at their clinical site, at the annual national meeting of the NeuroRecovery Network, or both. The training consisted of a review of each item, a demonstration of each task with persons without SCI and with persons with SCI of varying injury severities, and practice by the therapist in performing and scoring the NRS. Comparison of scores and rationale for scoring were discussed for each patient assessed to resolve questions and inconsistencies. Ongoing standardization of therapists occurs with site reviews of patient videos and monthly discussions on conference calls with each clinical supervisor.
Data analysis We used Mplus software7,a confirmatory factor analysis (CFA) to assess the dimensionality of the NRS. Unidimensionality is an assumption of the Rasch model. The criteria for unidimensionality were comparative fit index >.90, root mean square error of approximation .95.8 To provide a full test of dimensionality, we followed the CFA with a principal components analysis (PCA) of residuals using Winsteps software.9,b www.archives-pmr.org
1387 We applied a partial-credit, Rasch, 1-parameter, logistic rating scale model using Winsteps software.9 In contrast to traditional classical test theory analyses that focus on test-level psychometrics, Rasch analysis is a probabilistic model that focuses on itemlevel psychometrics. For example, fit statistics are used to identify items and persons fitting or misfitting the measurement model. In addition, Rasch analysis places person ability and item difficulty on the same linear continuum, allowing the identification of ceiling and floor effects. We used Linacre’s 3 essential rating scale criteria10: (1) 10 responses for each rating category; (2) rating scale average calibrations advancing monotonically with each category; and (3) outfit mean-squares for each rating scale category .95). The PCA of Rasch residuals revealed that the Rasch measurement dimension (person variance and item variance) explained 76.9% of the variance. The first component explained 5.3% of the variance (expectation of Rasch model is 4%). The variance explained by the items, 43.4%, was over 8.2 times the variance explained by the first component, suggesting the dominance of the measurement construct. An eigenvalue of 2.6 for the first component was >2.0, the smallest amount that can be considered a dimension. Eight of the 11 NRS items had .50 loadings: 4 standing/stepping items had negative loadings .50 (Stand retraining, Step adaptability, Step retraining, Stand adaptability), and 4 sitting/trunk items had positive loadings .54 (Reverse sit-up, Trunk extension in sitting, Sit, Sit-up).
Rating scale analysis Only 1 item, Trunk extension in sitting, and 69% of the ratings (59/ 85) met Linacre’s 3 essential rating scale criteria. Table 2 presents the rating categories for each item that did not meet Linacre’s 3 essential criteria. The most common limitation was items showing
1388 Table 2
C. Velozo et al Ratings not meeting rating scale essential criteria
Item (No. of Ratings)
Low Frequency
NonHigh monotonic Outfit
Stand retraining (4) 1A Stand adaptability (9) 1A 3B Step retraining (10) 3A 4A Step adaptability (10) 3A 3B 3C 4A Sit (9) 2C 2C* Sit-up (7) 1B 3C 4A 3C* 4A* Reverse sit-up (5) 3C 4A Trunk extension in sitting (8) Sit-to-stand (8) 3A 3B 3C Stand (8) 2B 3A 4A Walking (7) 3B 3C 4A
1A 1C 2A 3C* 4A* 3C* 4A*
NOTE. Phase (number) subphases (letter) represent ratings for each item (see appendixes 1 and 2). Only ratings that did not meet criterion are listed. * Items have low frequency counts.
.57. Ninety-one percent (172/188) of the patients fit the measurement model. Eight patients misfit both in infit and outfit statistics, and 5 additional patients showed only misfit of outfit statistics; 1 additional patient showed only misfit of infit statistics.
Precision Person separation reliability, which is analogous to Cronbach alpha, was .92, and person separation, an indicator of how well Table 3
the sample was separated by the instrument items, was 3.48. Item separation reliability was .90. The NRS separated the sample into 4.97 strata and items into 17.0 strata that were 3 SEs apart. Figure 1 presents the information function for the 11 items of the NRS. The ability of items to discriminate is represented by the item slope. Nine of the 11 items show high discrimination, with Sit-up and Reverse Sit-up showing low item discrimination.
Person abilityeitem difficulty match Figure 2 presents the map of item difficulty and person ability. Items are presented to the left of the dashed vertical line, with easy items at the bottom of the map and difficult items at the top. Stand retraining, Trunk extension in sitting, Sit, and Stand adaptability represent the easiest items, and Walking, Reverse sit-up, Sit-up, and Step adaptability represent the most challenging items. Persons, listed according to their AIS classification, are presented from low ability on the NRS at the bottom of the map, and persons of high ability at the top. The NRS showed no ceiling or floor effects (ie, no groupings of individuals at the top and bottom of the scale, respectively). In general, the distribution of patients according to AIS classifications appears as expected, with individuals with D classifications at the top half of the distribution and individuals with A classifications at the bottom half of the distribution. Individuals with C and B classifications were mixed in the middle to lower half of the distribution.
Discussion Overall, the NRS showed many measurement characteristics that are consistent with the theoretical framework for measuring neuromuscular task-specific function in individuals with SCI. While the NRS did not meet all CFA criterions for unidimensionality, the PCA of Rasch residuals showed that the items tended to load on a single factor. Ten of 11 items and 91% of the patients fit the Rasch measurement model. The NRS item difficulty distribution was well matched to the sample, with the instrument separating the sample into almost 5 statistically distinct strata, with 9 of 11 of the items showing high
Item statistics: misfit order Infit
Item Step retraining Reverse sit-up Sit Stand retraining Sit-up Trunk extension in sitting Step adaptability Stand adaptability Sit-to-stand Stand Walking
Measure 0.06 0.87 0.75 2.57 0.82 0.98 0.85 0.78 0.40 0.66 1.41
Model SE
MNSQ
.06 .10 .06 .12 .11 .06 .07 .06 .07 .07 .09
1.50 1.39 1.26 1.20 1.16 0.95 0.98 0.86 0.73 0.62 0.47
Outfit ZSTD 4.00 3.20 2.00 1.30 1.40 0.40 0.10 1.40 2.00 3.00 4.70
MNSQ 1.40 1.43 1.41 1.18 1.19 1.01 0.98 0.83 0.73 0.59 0.47
ZSTD 3.20 3.20 2.70 0.60 1.60 0.10 0.20 1.60 2.00 2.90 4.30
PTMEAS CORR .76 .57 .79 .52 .65 .79 .80 .85 .82 .82 .82
Abbreviations: MNSQ, mean square; PTMEAS CORR, point-measure correlation; ZSTD, standardized Z value.
www.archives-pmr.org
Neuromuscular Recovery Scale validity
Fig 1
Item characteristic curves for each of the items of the NRS.
discrimination. Items reflecting positioning (eg, Sit, Stand) were easier than items reflecting trunk motion and gait (eg, Step adaptability, Reverse sit-up, Walking). Furthermore, NRS measures in general separated patients according to the AIS classifications. The NRS showed limitations in the rating scale and discrimination ability for several items. Uncertainty in the rating scale psychometrics appears to be a function of low frequency of several rating categories. Of note, since a partial-credit model evaluates each item as having its own unique scale (vs collapsing across the rating scale for all items of the instrument), it was difficult to meet the criterion of 10 observations per rating, especially when items have a large range of rating categories. In addition, the low rating frequencies may be at least a partial explanation for items failing to meet other rating scale criteria (ie, monotonicity and model fit). The NRS has the ability to discriminate among individuals, not based on injury severity (ie, AIS classification), but on one’s recovered ability to perform a functional task. As expected, these functional abilities relate to the severity of injury, thus providing the clinician another tool for classifying patients. These results are congruent with those found by Behrman et al13 in which the NRS was found to discriminate patients on the basis of function within single AIS classifications. The easiest item was Stand retraining, most likely because of the method of administration. The patient is supported in the body weight support systemc and must maintain a kinematically appropriate standing posture with assistance from trainers. The trainers are allowed to provide the patient with whatever assistance is needed to maintain this appropriate standing posture. Therefore, the item may actually test the training team’s ability to hold a person in the appropriate posture, instead of the person’s true ability to maintain this posture. Furthermore, in other studies,14,15 test-retest and interrater reliabilities were shown to be low for Stand retraining, and most subjects had a ceiling effect for this item. The most difficult items were Walking, Reverse Sit-up, Sit-up, and Step adaptability. All of these items require
www.archives-pmr.org
1389 increased trunk control through a larger range of motion or in dynamic positions. As expected, the Walking item was the most difficult item for this sample. The scoring of the item involves the person’s ability to control the trunk while weight shifting in standing, and progresses to being able to maintain appropriate kinematics at the trunk, pelvis, and bilateral lower extremities during walking and negotiating obstacles, independently. The wide range of difficulty of the NRS items and the lack of floor and ceiling effects make this tool a valuable outcome measure for the SCI population. A person with an SCI, theoretically, can be evaluated on the NRS in the acute care setting, and continue to be assessed by this same instrument through outpatient rehabilitation. The NRS is the only outcome measure specifically designed for the SCI population that has the ability to test a variety of functional activities throughout a wide spectrum of recovery while disallowing compensation. In addition to our findings supporting the validity of the NRS, these findings suggest that the NRS supports task-specific training and can be useful for clinical decision-making. First, the functional items phase scoring of the NRS allows the identification of patient impairments and functional deficits that can be the focus of treatment. For example, for the item Stand Adaptability, a rating of 1A indicates the patient is unable to maintain proper trunk kinematics, suggesting that interventions to support an upright trunk would be important. Second, the item difficulty order (consistent across 91% of study participants), which shows Trunk extension in sitting and Sit as the easiest items, supports the theory that posture is foundational to more challenging functional tasks such as Walking and Step adaptability. This finding is consistent with studies showing that atypical, compensatory trunk movements promote maladaptive, nonoptimal recovery of the extremities16,17 and that postural training improves motor deficits in other neurologic conditions.18 Finally, the rating scale hierarchical progression (consistent across 96% of the subphases [ratings]) supports a framework for a graded treatment approach such as a graded reduction in physical assistance and a graded improvement in kinematics. Graded treatment approaches are common in clinical practice and are conceptually and empirically supported in the literature.19,20
Study limitations To date, all testing has been conducted using highly skilled therapists who had a significant amount of experience with the tool. We are uncertain what degree of training is necessary to achieve competency in administering and scoring this instrument. Future studies will examine this and further establish the psychometric properties of the NRS. Further studies are also needed to determine whether a subscale of only the overground items would be a valid instrument. The use of only overground items would extend the utility of the NRS into clinics that lack the necessary equipment to assess the treadmill items. Since the NRS is intended to measure change, the psychometric stability of the item calibrations need to be investigated in future studies. Also, expanding the NRS to include upper extremity items is also recommended to broaden the utility of the instrument as an assessment and progression tool for neuromuscular recovery.
1390
C. Velozo et al
Fig 2 Map of item difficulty measures relative to AIS classification. This plot presents the item difficulties (left of dashed vertical line) against the AIS classifications (right of dashed vertical line). Lower (more negative) values represent “easy” items and individuals of lower ability, while higher (more positive) values represent more challenging items and individuals of higher ability. Abbreviations: M, mean; REV SIT, Reverse sit-up; S, 1 SD; STAND ADAPT, Stand adaptability; STAND RETRAIN, Stand retraining; STEP ADAPT, Step adaptability; STEP RETRAIN, Step retraining; T, 2 SDs; TRUNK EXT SIT, Trunk extension in sitting.
Conclusions The NRS meets many of the Rasch model criteria for construct validity for use in individuals with SCI for a wide range of injury severities, AIS A through D.
Suppliers a. Mplus software; Muthe´n & Muthe´n. b. Winsteps software; Available at: http://www.winsteps.com. c. Therastride; Innoventor. Currently manufactured by Power NeuroRecovery (www.powerneurecovery.weebly.com).
Keywords Outcome assessment (health care); Recovery of function; Rehabilitation; Spinal cord injuries
Corresponding author Craig Velozo, PhD, OTR, Medical University of South Carolina, Division of Occupational Therapy, 151-B Rutledge Ave, MSC 962, Charleston, SC 29425. E-mail address: [email protected].
Acknowledgments We thank the members of the Christopher and Dana Reeve Foundation NRN for their previous efforts in NRS development and current work on data collection; and Sarah Morrison, PT (Shepherd Center, Atlanta, GA) and Mary Schmidt, PT, DPT, MS (Magee Rehab Hospital, Philadelphia, PA) for their reviews of drafts and suggestions. We also acknowledge the current or past NRN Center Directors: Steve Ahr and Douglas Stevens, MD (Frazier Rehab Institute); Daniel E. Graves, PhD (Memorial Hermann/The Institute of Rehabilitation and Research); Sarah Morrison, PT, and Keith Tansey, MD, PhD (Shepherd Center); Gail F. Forrest, PhD (Kessler Medical Rehabilitation Research and Education Corp); and D. Michele Basso, PT, EdD (The Ohio State University Medical Center); Mary Schmidt Read, PT, DPT, MS (Magee Rehabilitation Hospital); plus all other current and previous network members; the NRN Reeve Foundation’s Susan Howley and Joseph Canose; and the leadership, foresight, and support of V. Reggie Edgerton, PhD, Moses Chao, PhD, Michael Fehlings, MD, PhD, Andrei Krassioukov, MD, PhD, and Shelly Sorani, MA. For a list of all NRN team members, please visit: http://www. christopherreeve.org/site/c.ddJFKRNoFiG/b.5399929/k.6F37/Neuro Recovery_Network.htm. www.archives-pmr.org
Neuromuscular Recovery Scale validity
Appendix 1 Administration and Scoring of the Phases Administering the NRS The administration of the phasing tool should take place during 1 evaluation period. The testing should begin with the first of the overground items, sit, continue through the overground items, and then follow with the items in the body weight support treadmill environment. Each item is tested and scored separately. The score of the item is the highest phase in which the patient can complete the item according to the description without any compensation. The examiner then calculates the overall phase based on the combined phase scores of each of the individual items. The continuum of scores proceeds from unable to complete the task to achieving full recovery by executing the tasks as done preinjury. Assess the patient beginning with phase 1A abilities and then move up the phase scoring, because the lower phases must be achieved before higher scoring can be considered. Assistance cannot be provided to the body segment being assessed. The lower of 2 debatable scores should be selected. There is no advantage to scoring higher, and all assessments should be directly compared with the ability preinjury to complete the specific task. This is important because goal setting and progression will be based on the current phase scoring. Scoring lower, if unsure of 2 possible scores, guarantees that the correct task and neuromuscular skill will be sufficiently targeted with therapy aimed at this skill deficit. Subphases that are not scored are designated as not applicable and account for the progression of recovery within a task and across tasks.
1391 2C); able to sit and complete forward and lateral reaches that extend to 12.7cm (5in) (phase 3A); able to sit and complete forward and lateral reaches that extends >12.7cm and 25.4cm (10in) beyond the tips of the fingers when the arm is raised to 90 of shoulder flexion or abduction (phase 3C); and not applicable (phase 4).
Reverse sit-up The patient is positioned in an unsupported sitting position at the edge of the mat, with the feet flat on the floor. The patient is asked to slowly lower his/her trunk down to the mat until the back and head are flat on the mat, without using the upper extremities for support and while maintaining a 90 angle in the knees. The examiner scores the patient based on how controlled the patient can lower the trunk, the position of the upper extremities, the compensatory use of the lower extremities, and the amount of effort required to complete the task. Manual assistance is not provided unless needed to avoid striking the mat with excessive force. Scores are categorized as follows: not applicable (phase 1Ae1C); unable (phase 2A); patient shows control through the first 45 but loses control during the lower half of the movement (phase 2B); not applicable (phase 2C); patient is able to complete task with arms elevated in order to provide a counterbalance (phase 3A); not applicable (phase 3B); patient is able to complete task with arms held across chest, but displaying considerable effort (phase 3C); and patient is able to complete task with arms held across chest without considerable effort (phase 4).
Abilities during overground assessment
Sit-up
Overground assessment defines the capacity of the neuromuscular system without the benefit of the body weight support treadmill or manual facilitation to execute specific motor tasks.
The patient is positioned in a supine position on the mat, with the lower extremities off the edge of the mat. The patient is asked to return to a sitting position without the use of the upper extremities, manual assistance, or momentum, while maintaining the knees at a 90 position. The examiner scores the patient based on how much of the trunk is able to be raised off the mat, the position of the upper extremities, the compensatory use of the lower extremities, and the amount of effort required to complete the task. Scores are categorized as follows: not applicable (phase 1A); unable (phase 1B); patient is able to raise head (phase 1C); patient is able to raise head and initiate shoulder lift (phase 2A); patient is able to raise head, shoulders, and scapulae (phase 2B); not applicable (phase 2C); patient is able to sit up, but with inappropriate position (arms elevated and/or hip/knee flexion/extension) (phase 3A); not applicable (phase 3B); patient is able to sit up properly, but displaying considerable effort (phase 3C); and patient is able to sit up properly without considerable effort (phase 4).
Sit The patient is positioned in an unsupported sitting position at the edge of a firm mat. The patient’s feet should be flat on the floor. The patient is then asked to sit without upper extremity support in a proper posture position (ie, head and shoulders in alignment, neutral pelvis position). If the patient is unable to attain this position, the examiner may assist the patient into this position and then ask the patient to maintain the posture. The patient is then assigned a phase based on the patient’s ability to maintain the proper posture in an unsupported position. If the patient is able to attain and maintain the proper posture (ie, phase 2), then the patient is asked to elevate the upper extremities, as described in the chart, and reach forward and laterally. The patient is assigned a phase based on these abilities. If the patient is incapable of elevating the upper extremities because of complications of the neuromuscular system, the patient may be graded on the ability to lean the trunk forward and laterally. Scores are categorized as follows: unable (phase 1A); unable to attain, able to sit with inappropriate posture (phase 1B); unable to attain, able to sit with appropriate posture (phase 1C); able to attain and maintain appropriate position for at least 1 minute (phase 2A); able to attain and maintain appropriate position indefinitely (phase 2B); able to sit and hold arms parallel to legs for at least 30 seconds (phase www.archives-pmr.org
Trunk extension in sitting The patient is positioned in sitting with the feet flat on the floor, leans forward at the waist (or is assisted forward) to rest the chest fully on the lap. The patient is asked to return to an upright sitting position without the use of the upper extremities or manual assistance. The examiner grades the patient based on how much of the trunk is raised into sitting, the position of the upper extremities, and the amount of effort required to complete the task.
1392 Scores are categorized as follows: unable (phase 1A); patient is able to initiate thoracic spine extension (phase 1B); patient is able to initiate and maintain thoracic spine extension (phase 1C); patient is able to initiate and maintain thoracic and initiate lumbar spine extension (phase 2A); patient is able to initiate and maintain thoracic and lumbar spine extension but displays considerable effort (phase 2B); not applicable (phase 2C); patient is able to initiate and maintain thoracic and lumbar spine extension without considerable effort (phase 3A); not applicable (phase 3B); patient is able to complete task with arms behind head, but displays considerable effort (phase 3C); and patient is able to complete task with arms behind head without considerable effort (phase 4).
Sit-to-stand The patient begins seated at the edge of the mat with feet flat on the floor. The patient is asked to stand up from the mat without the use of the upper extremities. The examiner first assesses how high the patient can raise his/her body off the mat without any assistance from the examiner. If the patient is able to raise the body approximately 50% off the mat, then the examiner may provide assistance to the patient’s pelvis and/or lower extremities, and assess the patient’s control at the trunk and pelvis, as appropriate. If the patient is able to transition from sit to stand without physical assistance from the examiner, the patient is scored according to his/her kinematics during the task and the position of the upper extremities during the task. Scores are categorized as follows: not applicable (phase 1Ae1B); unable (phase 1C); patient is able to initiate weight bearing (phase 2A); the patient is able to raise himself up off the mat towards standing and achieves 50% of the trajectory towards standing (phase 2B); patient is able to complete the task while maintaining proper kinematics of the head, shoulders, and trunk, and manual facilitation may be provided at the pelvis, knees, or ankles if needed (phase 2C); patient is able to complete the task with proper kinematics of the head, shoulders, and trunk, and manual facilitation may be provided at the knees or ankles if needed (phase 3A); patient is able to complete the task with proper kinematics of the head, shoulders, trunk, and pelvis, inappropriate kinematics at the knees and ankles, and no manual facilitation (phase 3B); patient is able to complete the task with proper kinematics, no manual facilitation, and uses the arms as a counterbalance (phase 3C); and patient is able to complete task properly without using the arms as a counterbalance (phase 4).
Stand The patient begins in a standing position, with physical assistance provided by the examiner as needed. The patient is asked to stand with a proper posture (ie, head, shoulders, and trunk extended, pelvis properly positioned under the head and shoulders, with the knees extended to adequately maintain body weight without hyperextension). The patient is assigned a phase based on the ability to maintain appropriate kinematics at each body segment. Please note that when the trunk is assessed (ie, phases 1Ce2A), assistance may be provided to the lower extremities and pelvis by the examiner and other trainers. Likewise, assistance may be provided to the lower extremities in phase 2 to assess the kinematics of the pelvis. To be assigned phase 3, the patient must be able to maintain standing balance without physical assistance. Scores are categorized as follows: not applicable (phase 1Ae1B); unable (phase 1C); patient is able to stand with inappropriate posture and manual facilitation at the pelvis, knees, or ankles provided as
C. Velozo et al needed (phase 2A); patient is able to stand with appropriate posture for at least 1 minute, and manual facilitation provided at the knees and ankles as needed (phase 2B); patient is able to stand with appropriate posture for at least 1 minute, and manual facilitation provided at the knees and ankles as needed (phase 2C); patient is able to stand with appropriate posture and without manual facilitation for at least 1 minute (phase 3A); patient is able to stand with appropriate posture without manual facilitation for at least 1 minute (phase 3B); and patient is able to stand with appropriate posture and without manual facilitation indefinitely (phase 3C).
Walking The patient begins in standing position, with physical assistance as needed by the examiner at pelvis and lower extremities. The patient is asked to shift weight laterally. If the patient is able to shift weight without assistance at the trunk, the patient is then asked to shift weight forward and back in a stride position (left and right). The patient is assigned a phase based on the ability to assume stride right and stride left positions while maintaining appropriate kinematics at each body segment. If the patient is able to independently assume stride positions and perform weight shifts independently while maintaining appropriate kinematics at all body segments, the patient is assigned a phase based on the ability to perform repetitive steps with appropriate kinematics. Scores are categorized as follows: not applicable (phase 1Ae1C); unable to shift weight (phase 2A); patient is able to shift weight laterally with inappropriate kinematics at the head, shoulders, and trunk (phase 2B); patient is able to shift weight laterally with appropriate kinematics at the head, shoulders, and trunk, but is unable to initiate the stride position (phase 2C); patient is able to shift weight laterally with appropriate kinematics of the head, shoulders, and trunk, but inappropriate kinematics of the legs (phase 3A); patient is able to shift weight laterally with appropriate kinematics, but not in stride (phase 3B); patient is able to shift weight laterally and in stride with appropriate kinematics (phase 3C); and patient is able to walk with appropriate kinematics (phase 4).
Abilities during treadmill assessment Once the overground assessment has been concluded, the patient is positioned in a body weight support treadmill system. The steptraining environment uses the body weight support treadmill and manual assistance to assess the capacity and independence of the nervous system to stand and generate steps in a safe environment. The capacity of the nervous system is assessed by identifying the treadmill speed, body weight support, and facilitation needed to generate the optimal stepping pattern. This capacity of the nervous system is referred to as retraining. The independence of the nervous system is referred to as adaptability, and is assessed by identifying the treadmill speed and body weight support where independence from manual assistance occurs. There are 4 areas of assessment during step training: (1) Stand retraining, (2) Stand adaptability, (3) Step retraining, and (4) Step adaptability.
Stand retraining In this area, the capacity of the nervous system to bear weight in static standing is assessed, recorded as body weight support. The www.archives-pmr.org
Neuromuscular Recovery Scale validity patient is positioned in the body weight support treadmill environment in an upright posture, with head, shoulders, and trunk extended, the pelvis properly positioned under the head and shoulders, with the knees extended to adequately maintain body weight without compensation. The patient should initially be supported with 75% body weight support (or the maximum amount of body weight support with the patient’s feet remaining flat on the treadmill). The body weight support is then lowered as far as possible for the patient to maintain proper posture and positioning of the head, trunk, pelvis, and legs for at least 5 minutes, with the trainers providing as much physical assistance as needed to maintain the patient in an upright standing position. The lowest body weight support that is achieved is then noted, and the patient is assigned a phase level based on this amount of body weight support. Scores are categorized as a percent of total body weight support which the patient can maintain with trainers providing assistance: body weight support at 40% (phase 1A); body weight support 20% to 39% (phase 1B); not applicable (phase 1Ce2A); body weight support 10% to 19% (phase 2B); not applicable (phase 2Ce3B); body weight support at 10% (phase 3C); and not applicable (phase 4).
1393 associated with walking. Leg loading is maximized, as are normative walking speeds (.89e1.34m/s; 2.0e3.0mph), resulting in scores of body weight support and speed. The body weight support level is adjusted to the body weight support where most of the body segments were independent during stand adaptability. The patient then begins stepping at a speed of .89m/s (2.0mph), with physical assistance from the trainers to ensure the best posture and kinematics while walking. The body weight support and speed are adjusted up or down to assess the lowest body weight support and highest speed at which the patient can generate the best stepping pattern with assistance from the trainers. The patient is assigned a phase level based on these 2 variables. Scores are categorized as follows: body weight support is set at 60% (phase 1A); body weight support 55% to 59% (phase 1B); body weight support 50% to 54% (phase 1C); body weight support 45% to 49% (phase 2A); body weight support 40% to 44% (phase 2B); body weight support 35% to 39% (phase 2C); body weight support 30% to 34% (phase 3A); body weight support 20% to 29% (phase 3B); and body weight support is at 20% (phase 3C). Patients who can maintain proper kinematics with 50% body weight support while running at speeds of 1.52m/s (3.4mph) are scored as phase 4.
Stand adaptability In this area, the patient’s ability to independently maintain appropriate kinematics at each body segment without physical assistance is assessed, reflected by body weight support required. The patient begins at the lowest body weight support achieved during Stand retraining, in an upright posture with head, shoulders, and trunk extended, the pelvis properly positioned under the head and shoulders, and the knees and ankles positioned at neutral to adequately maintain body weight. The body weight support is then adjusted until the patient is able to independently maintain an upright posture at the trunk, with assistance provided at the pelvis and lower extremities. If the patient is able to maintain an upright posture at the trunk with
