Poor Performance of Stroke Prognostication Using Age and National Institutes of Health Stroke Scale-100 to Predict 3- and 12-month Outcomes of Ischemic Stroke in China National Stroke Registry Yuesong Pan, MD,*† Jing Jing, MD, PhD,*† Runhua Zhang, MD,*† Xingquan Zhao, MD, PhD,*† Liping Liu, MD, PhD,*† Haichen Wang, MD,‡ Gaifen Liu, PhD,*† Chunxue Wang, MD, PhD,*† Yilong Wang, MD, PhD,*† and Yongjun Wang, MD*†
Background: Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale (SPAN)-100 is a simple and easy-to-use tool for assessing the outcomes of ischemic stroke after thrombolysis. To explore its application, we evaluated SPAN-100’s prognostic value in predicting 3- and 12-month outcomes in general ischemic stroke patients. Methods: We applied the SPAN-100 to ischemic stroke patients from the China National Stroke Registry. Poor outcome was defined as a modified Rankin Scale of 2-6. Discrimination of SPAN-100 was assessed by the area under the receiver–operator curves (AUC) and 95% confidence intervals (CI). We also performed an exploratory post hoc analysis of the performance of the SPAN index score using 80 as the cutoff point. Results: Among 11,894 ischemic stroke patients, 479 (4.0%) patients were SPAN-100 positive. The AUC of SPAN100 for poor outcome was .54 (95% CI, .54-.54) at 3 months and .54 (95% CI, .54.55) at 12 months, respectively. In the exploratory analysis, when 80 was used as the cutoff point of SPAN index score, the AUC for poor outcome was .66 (95% CI, .66-.67) at 3 months and .68 (95% CI, .67-.68) at 12 months, respectively. Conclusions: SPAN-100 suffered from low prediction power for 3- and 12-month outcomes of ischemic stroke in Chinese population. A cutoff point of 80 may improve the performance, but none of them had an AUC above the threshold of .8 required for use in individuals. Key Words: SPAN-100—ischemic stroke—outcome—prognosis. Ó 2014 by National Stroke Association
From the *Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China; †China National Clinical Research Center for Neurological Diseases (NCRC-ND), Beijing, China; and ‡Brain Injury Translational Research Center, Duke University Medical Center, Durham, North Carolina. Received April 13, 2014; accepted April 22, 2014. The CNSR is funded by grants (2006BA101A11, 2009CB521905 and 2013BAI09B03) from the Ministry of Science and Technology and the Ministry of Health of the People’s Republic of China, and grants (BIBD-PXM2013_014226_07_000084) from Beijing Institute for Brain Disorders. Address correspondences to Yongjun Wang, MD, No 6 Tiantanxili, Dongcheng District, Beijing 100050, China. E-mail: yongjunwang1962@ gmail.com; Yilong Wang, MD, PhD. E-mail: [email protected]. 1052-3057/$ - see front matter Ó 2014 by National Stroke Association http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2014.04.031
Recently, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale (SPAN) index was developed by combining age in years plus National Institutes of Health Stroke Scale (NIHSS) score $100 to predict clinical outcome and the risk of intracerebral hemorrhage after thrombolysis.1 SPAN-100 is a simple and easy-to-use tool to assess the risk of hemorrhage and outcome after thrombolysis. However, only 2%-15% of ischemic stroke patients were treated with thrombolysis.2,3 It will be clinically important if the easy-to-use tool of SPAN-100 performed well to predict the prognosis of general ischemic stroke. Although its application in ischemic stroke patients receiving endovascular stroke
Journal of Stroke and Cerebrovascular Diseases, Vol. 23, No. 9 (October), 2014: pp 2335-2340
2335
Y. PAN ET AL.
2336
Table 1. Differences of baseline characteristics between patients with NIHSS and 12-month mRS recorded and patients without data of NIHSS or 12-month mRS recorded Characteristics
NIHSS or mRS not recorded, N 5 521
NIHSS and mRS recorded, N 5 11,894
P value*
Female, n (%) Age (y), mean (SD) Current smoking, n (%) Heavy drinking, n (%) Medical history, n (%) Diabetes mellitus Hypertension Dyslipidemia Cardiovascular disease Atrial fibrillation Stroke History of mRS 2-5, n (%)
196 (37.6) 65.4 6 12.6 120 (23.0) 127 (24.4)
4561 (38.3) 65.5 6 12.3 3190 (26.8) 3215 (27.0)
.74 .16 .06 .18
117 (22.5) 339 (65.1) 74 (14.2) 77 (14.8) 37 (7.1) 189 (36.3) 49 (9.4)
2560 (21.5) 7570 (63.6) 1316 (11.1) 1715 (14.4) 881 (7.4) 4045 (34.0) 1116 (9.4)
.61 .51 .03 .82 .79 .29 .99
Abbreviations: mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale score; SD, standard deviation. *P values indicate comparison using chi-square test for categorical variables and t test or Mann–Whitney U test for continuous variables.
treatment was tested,4 the performance of SPAN-100 to predict outcomes of general ischemic stroke is unclear. Using the data from the China National Stroke Registry (CNSR),5 we aimed to evaluate the prognostic value of SPAN-100 to predict 3- and 12-month outcomes of ischemic stroke and perform an exploratory analysis to improve the performance of the score.
Methods Data were derived from CNSR,5 a nationwide prospective registry of 21,902 consecutive patients with a diagnosis of ischemic stroke, intracerebral hemorrhage, or transient
ischemic attack from 132 hospitals in China between September 2007 and August 2008. Data on demographics, clinical characteristics, and outcomes at the 3, 6, 12 months follow-up visits were collected. The collection of data for the CNSR study was approved by ethics committees at all participating hospitals. Written informed consent was obtained from all patients or their legal representatives. At 3 and 12 months after onset of stroke, the outcomes of all patients were assessed through telephone follow-up interview. The telephone follow-up was centralized for all included patients and based on a shared standardized interview protocol. Favorable outcome was defined as a modified Rankin Scale (mRS) of 0-1 and poor outcome
Table 2. Baseline characteristics between SPAN-100 positive and SPAN-100 negative patients in China National Stroke Registry Characteristics
SPAN-100 positive, N 5 479
SPAN-100 negative, N 5 11,415
P value*
Female, n (%) Age (y), mean (SD) Age categories (y), n (%) ,65 66-79 $80 NIHSS on admission, median (IQR) Current smoking, n (%) Heavy drinking, n (%) Medical history, n (%) Diabetes mellitus Hypertension Dyslipidemia Cardiovascular disease Atrial fibrillation Stroke History of mRS 2-5, n (%)
289 (60.3) 82.0 6 6.8
4272 (37.4) 64.8 6 12.0
,.001 ,.001
4 (.8) 163 (34.0) 312 (65.1) 25 (19-33) 38 (7.9) 48 (10.0)
5547 (48.6) 4739 (41.5) 1129 (9.9) 4 (2-9) 3152 (27.6) 3167 (27.7)
,.001
69 (14.4) 297 (62.0) 29 (6.1) 116 (24.2) 132 (27.6) 209 (43.6) 129 (26.9)
2491 (21.8) 7273 (63.7) 1287 (11.3) 1599 (14.0) 749 (6.6) 3836 (33.6) 987 (8.6)
,.001 .45 ,.001 ,.001 ,.001 ,.001 ,.001
,.001 ,.001 ,.001
Abbreviations: IQR, interquartile range; mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale score; SD, standard deviation; SPAN, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale. *P values indicate comparison using chi-square test for categorical variables and t test or Mann–Whitney U test for continuous variables.
PERFORMANCE OF SPAN-100 IN CNSR
2337
Table 3. 3- and 12-month proportions of poor outcome and mortality by SPAN categories according to different cutoff points SPAN cutoff
Time
100
Number of patients (%) 3 mo 12 mo
80
Number of patients (%) 3 mo 12 mo
Outcome SPAN-x* negative SPAN-x* positive Odds ratioy (95% CI) P value
mRS 2-6 Death mRS 2-6 Death mRS 2-6 Death mRS 2-6 Death
11,415 (96.0) 5143 (45.1) 730 (6.4) 4865 (42.6) 1240 (10.9) 8126 (68.3) 2859 (35.2) 277 (3.4) 2600 (32.0) 451 (5.6)
479 (4.0) 456 (95.2) 256 (53.4) 459 (95.8) 350 (73.1) 3768 (31.7) 2740 (72.7) 709 (18.8) 2724 (72.3) 1139 (30.2)
18.22 (11.92-27.85) 12.17 (9.90-14.96) 22.99 (14.61-36.17) 16.63 (13.35-20.70) 4.28 (3.92-4.68) 5.36 (4.60-6.25) 4.75 (4.35-5.19) 6.10 (5.38-6.88)
,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001
Abbreviations: CI, confidence interval; mRS, modified Rankin Scale; SPAN, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale. *X 5 100 when SPAN cutoff point is 100, X 5 80 when SPAN cutoff point is 80. yAdjusted for sex, smoking, drinking, history of diabetes mellitus, dyslipidemia, coronary heart disease, atrial fibrillation, stroke, and prestroke mRS.
was defined as mRS of 2-6 according to the original derivation study of SPAN-100 score.1 SPAN index was calculated as age in years plus NIHSS on admission.1 According to the derivation study, patients with a SPAN index score $100 were defined as SPAN-100 positive, whereas those with a score ,100 were defined as SPAN-100 negative.1 We also performed an exploratory post hoc analysis using 80 as the cutoff point. Patients with a SPAN index score $80 were designated as SPAN-80 positive and those with a score ,80 were designated as SPAN-80 negative. Patients with missing data of SPAN index score or 12month mRS were excluded from this analysis.
Statistical Analysis Categorical variables were presented as percentages and continuous variables as mean 6 standard deviation or median (interquartile range). Differences between categorical variables were compared using chi-square test, whereas continuous variables were compared using a t test or Mann–Whitney U test. Odds ratios (ORs) with its 95% confidence intervals (CIs) were calculated using multivariable logistic regression adjusting all significant baseline variables in Table 2. We tested the performance of the SPAN score by estimating their discrimination and calibration. Discrimination of the scores was assessed by the area under the receiver–operator curves (AUC) and 95% CIs, sensitivity, and specificity. An AUC of .5 indicates no discrimination, and an AUC of 1.0 indicates perfect discrimination. Calibration was assessed by comparing predicted and observed probability of poor outcome using Pearson correlation coefficient. We tested the performance of the continuous SPAN index score, SPAN-100, and SPAN-80, respectively. The a level of significance was P , .05 on 2 sides. All analyses were performed with SAS software version 9.3 (SAS Institute Inc, Cary, NC).
Results Among 12,415 ischemic stroke patients enrolled in CNSR who consented for follow-up, 494 (4.0%) and 27 patients (.2%) were excluded for missing data of NIHSS or 12-month mRS, respectively. There was no difference between patients with and without data of NIHSS and mRS recorded (Table 1). A total of 11,894 ischemic stroke patients were included in this analysis, of whom 479 (4.0%) were SPAN-100 positive. The baseline characteristics between SPAN-100 positive and SPAN-100 negative patients were shown in Table 2. SPAN-100 positive patients were older, had higher NIHSS score, and more risk factors such as cardiovascular disease, atrial fibrillation, and history of stroke than SPAN-100 negative patients. SPAN-100 positive patients had a higher proportion of poor outcomes at 3 months (95.2% vs 45.1%; adjusted OR, 18.22; 95% CI, 11.92-27.85) and 12 months (95.8% vs. 42.6%; adjusted OR, 22.99; 95% CI, 14.61-36.17) than SPAN-100 negative patients (Table 3). The distributions of mRS at 3 and 12 months according to SPAN categories are shown in Figure 1. The AUC, sensitivity, and specificity of SPAN-100 score are shown in Table 4. The AUC of SPAN-100 for poor outcome was .54 (95% CI, .54-.54) at 3 months and .54 (95% CI, .54-.55) at 12 months. The AUC (c-statistic) of continuous SPAN index score for discrimination between patients with poor and good outcome was .73 (95% CI, .72-.74) at 3 months and .75 (95% CI, .74-.76) at 12 months (Fig 2). Calibration analysis of continuous SPAN index score showed a high correlation between predicted and observed probability of poor outcome at 3 months (r 5 .956, P , .001) and 12 months (r 5 .974, P , .001). The calibration plot of the continuous SPAN index score for poor outcome at 12 months is shown in Figure 3. In the exploratory post hoc analysis, the performance of the score was tested using 80 as the cutoff point. The
Y. PAN ET AL.
2338
Figure 1. Distribution of mRS scores at 3 and 12 months according to SPAN categories. The size of the bars indicates the percentage of patients with a particular mRS score in each group. (A) mRS distribution according to SPAN-100. (B) mRS distribution according to SPAN-80. Abbreviations: mRS, modified Rankin Scale; SPAN, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale.
proportions of poor outcome and mortality at 3 and 12 months according to SPAN-80 categories are shown in Figure 1 and Table 3. The AUC, sensitivity, and specificity of SPAN-80 are shown in Table 4. The AUC of SPAN-80 for poor outcome was .66 (95% CI, .66-.67) at 3 months and .68 (95% CI, .67-.68) at 12 months, which is significantly superior to SPAN-100 (P , .001).
Discussion Our study showed modest performance of the continuous SPAN index score but a low predictive power of SPAN-100 score for 3- and 12-month outcomes of general ischemic stroke in Chinese population. The c-statistic was very close to validation of SPAN-100 in Randomized Trial
Table 4. Predictive value of SPAN score for 3- and 12-month poor outcome and mortality according to different cutoff points SPAN cutoff
Time
Outcome
AUC (95% CI)
Sensitivity, %
Specificity, %
100
3 mo
mRS 2-6 Death mRS 2-6 Death mRS 2-6 Death mRS 2-6 Death
.54 (.54-.54) .62 (.61-.63) .54 (.54-.55) .60 (.59-.61) .66 (.66-.67)* .72 (.70-.73)* .68 (.67-.68)* .73 (.72-.74)*
8.1 26.0 8.6 22.0 48.9 71.9 51.2 71.6
99.6 98.0 99.7 98.7 83.7 72.0 84.1 74.5
12 mo 80
3 mo 12 mo
Abbreviations: AUC, area under the receiver–operator curve; CI, confidence interval; mRS, modified Rankin Scale; SPAN, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale. *P , .001 comparing with AUC of SPAN-100 for corresponding outcome.
PERFORMANCE OF SPAN-100 IN CNSR
2339
Figure 3. Calibration plot of the continuous SPAN index score for poor outcome at 12 months. The continuous line indicates the predicted probability of poor outcome. The dots indicate the observed rates of poor outcome, and the vertical lines indicate the 95% confidence intervals. Abbreviations: mRS, modified Rankin Scale; SPAN, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale.
Figure 2. Receiver–operator curve of the SPAN index score for poor outcome at 3 months (A) and 12 months (B). Abbreviations: SPAN, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale.
Evaluating Performance of the Trevo Retriever Versus the Merci Retriever in Acute Ischemic Stroke (TREVO-2) trial predicting poor 3-month outcome (mRS 3-6) in ischemic stroke patients receiving endovascular stroke treatment (c-statistic 5 .560).4 The SPAN-100 was reported to have high specificity but low sensitivity, and most SPAN-100 positive patients had very high severity of stroke.4 Our study presented similar results. In the exploratory analysis using 80 instead of 100 as the cutoff point, SPAN-80 improved the sensitivity and had a discriminatory power superior to SPAN-100.
Our study has several strengths. First, our study examined the performance of SPAN-100 to predict 3- and 12-month outcomes in a large Chinese ischemic stroke population. Second, we proposed a new cutoff point of SPAN score (80) to improve the discriminatory power to predict 3- and 12-month outcomes. The original application of SPAN-100 was to predict the prognosis of ischemic stroke patients after thrombolysis, and 1 rationale for the creation of SPAN-100 score was that patients 80 years of age and older and with high NIHSS scores (eg, $20) had a higher risk of hemorrhage and poorer outcome for ischemic stroke after thrombolysis.1 However, in general ischemic stroke patients, 60 years of age or older or 65 years of age or older were identified as an independent prognostic factor.6-11 This may explain why SPAN-100 had a low sensitivity in predicting 3- and 12-month outcomes in general ischemic stroke population but the performance improved using 80 as the new cutoff point. Our study has limitations. The study sites of CNSR were mainly selected from urban areas of China representing sites with better stroke care quality than those in rural areas.5 Second, follow-up mRS assessment was performed by telephone interviews rather than faceto-face clinical examination, which may have biased the outcome. However, the telephone assessment of the mRS with a structured interview was shown to have a good agreement with face-to-face assessment.12 Third, although in the exploratory analysis using 80 as the cutoff point, SPAN-80 had a discriminatory power superior to SPAN-100, none of them had an AUC above the threshold of .8 required for use in individuals. The possible explanation is that, although age and NIHSS score are 2 strong predictors of ischemic stroke,7,8,13-15 there are other independent prognosis factors such as acute glucose, hypertension, onset-to-treatment time,
2340
previous transient ischemic attack, ischemic stroke or myocardial infarction.7,13,14 However, instead of developing a new complex score, the aim of our study was to assess and perform the post hoc analysis for the performance of the easy-to-use SPAN-100 score in general ischemic stroke patients. In conclusion, SPAN-100 suffered from low prediction power for 3- and 12-month outcomes of ischemic stroke in Chinese population. A cutoff point of 80 may improve the performance, but none of them had an AUC above the threshold of .8 required for use in individuals.
References 1. Saposnik G, Guzik AK, Reeves M, et al. Stroke Prognostication using Age and NIH Stroke Scale: SPAN-100. Neurology 2013;80:21-28. 2. Grotta JC, Burgin WS, El-Mitwalli A, et al. Intravenous tissue-type plasminogen activator therapy for ischemic stroke: Houston experience 1996 to 2000. Arch Neurol 2001;58:2009-2013. 3. Wang Y, Liao X, Zhao X, et al. Using recombinant tissue plasminogen activator to treat acute ischemic stroke in China: analysis of the results from the Chinese National Stroke Registry (CNSR). Stroke 2011;42:1658-1664. 4. Flint AC, Xiang B, Gupta R, et al. THRIVE score predicts outcomes with a third-generation endovascular stroke treatment device in the TREVO-2 trial. Stroke 2013; 44:3370-3375. 5. Wang Y, Cui L, Ji X, et al. The China National Stroke Registry for patients with acute cerebrovascular events: design, rationale, and baseline patient characteristics. Int J Stroke 2011;6:355-361.
Y. PAN ET AL. 6. Diener HC, Ringleb PA, Savi P. Clopidogrel for the secondary prevention of stroke. Expert Opin Pharmacother 2005;6:755-764. 7. Strbian D, Meretoja A, Ahlhelm FJ, et al. Predicting outcome of IV thrombolysis-treated ischemic stroke patients: the DRAGON score. Neurology 2012;78:427-432. 8. Smith EE, Shobha N, Dai D, et al. Risk score for inhospital ischemic stroke mortality derived and validated within the Get With the Guidelines-Stroke Program. Circulation 2010;122:1496-1504. 9. Singer DE, Chang Y, Borowsky LH, et al. A new risk scheme to predict ischemic stroke and other thromboembolism in atrial fibrillation: the ATRIA study stroke risk score. J Am Heart Assoc 2013;2:e000250. 10. Kamouchi M, Kumagai N, Okada Y, et al. Risk score for predicting recurrence in patients with ischemic stroke: the Fukuoka stroke risk score for Japanese. Cerebrovasc Dis 2012;34:351-357. 11. Johnston SC, Rothwell PM, Nguyen-Huynh MN, et al. Validation and refinement of scores to predict very early stroke risk after transient ischaemic attack. Lancet 2007; 369:283-292. 12. Janssen PM, Visser NA, Dorhout Mees SM, et al. Comparison of telephone and face-to-face assessment of the modified Rankin Scale. Cerebrovasc Dis 2010;29:137-139. 13. Saposnik G, Kapral MK, Liu Y, et al. IScore: a risk score to predict death early after hospitalization for an acute ischemic stroke. Circulation 2011;123:739-749. 14. Ntaios G, Faouzi M, Ferrari J, et al. An integer-based score to predict functional outcome in acute ischemic stroke: the ASTRAL score. Neurology 2012;78:1916-1922. 15. Weimar C, K€ onig IR, Kraywinkel K, et al. Age and National Institutes of Health Stroke Scale Score within 6 hours after onset are accurate predictors of outcome after cerebral ischemia: development and external validation of prognostic models. Stroke 2004;35:158-162.
Background: Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale (SPAN)-100 is a simple and easy-to-use tool for assessing the outcomes of ischemic stroke after thrombolysis. To explore its application, we evaluated SPAN-100’s prognostic value in predicting 3- and 12-month outcomes in general ischemic stroke patients. Methods: We applied the SPAN-100 to ischemic stroke patients from the China National Stroke Registry. Poor outcome was defined as a modified Rankin Scale of 2-6. Discrimination of SPAN-100 was assessed by the area under the receiver–operator curves (AUC) and 95% confidence intervals (CI). We also performed an exploratory post hoc analysis of the performance of the SPAN index score using 80 as the cutoff point. Results: Among 11,894 ischemic stroke patients, 479 (4.0%) patients were SPAN-100 positive. The AUC of SPAN100 for poor outcome was .54 (95% CI, .54-.54) at 3 months and .54 (95% CI, .54.55) at 12 months, respectively. In the exploratory analysis, when 80 was used as the cutoff point of SPAN index score, the AUC for poor outcome was .66 (95% CI, .66-.67) at 3 months and .68 (95% CI, .67-.68) at 12 months, respectively. Conclusions: SPAN-100 suffered from low prediction power for 3- and 12-month outcomes of ischemic stroke in Chinese population. A cutoff point of 80 may improve the performance, but none of them had an AUC above the threshold of .8 required for use in individuals. Key Words: SPAN-100—ischemic stroke—outcome—prognosis. Ó 2014 by National Stroke Association
From the *Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China; †China National Clinical Research Center for Neurological Diseases (NCRC-ND), Beijing, China; and ‡Brain Injury Translational Research Center, Duke University Medical Center, Durham, North Carolina. Received April 13, 2014; accepted April 22, 2014. The CNSR is funded by grants (2006BA101A11, 2009CB521905 and 2013BAI09B03) from the Ministry of Science and Technology and the Ministry of Health of the People’s Republic of China, and grants (BIBD-PXM2013_014226_07_000084) from Beijing Institute for Brain Disorders. Address correspondences to Yongjun Wang, MD, No 6 Tiantanxili, Dongcheng District, Beijing 100050, China. E-mail: yongjunwang1962@ gmail.com; Yilong Wang, MD, PhD. E-mail: [email protected]. 1052-3057/$ - see front matter Ó 2014 by National Stroke Association http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2014.04.031
Recently, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale (SPAN) index was developed by combining age in years plus National Institutes of Health Stroke Scale (NIHSS) score $100 to predict clinical outcome and the risk of intracerebral hemorrhage after thrombolysis.1 SPAN-100 is a simple and easy-to-use tool to assess the risk of hemorrhage and outcome after thrombolysis. However, only 2%-15% of ischemic stroke patients were treated with thrombolysis.2,3 It will be clinically important if the easy-to-use tool of SPAN-100 performed well to predict the prognosis of general ischemic stroke. Although its application in ischemic stroke patients receiving endovascular stroke
Journal of Stroke and Cerebrovascular Diseases, Vol. 23, No. 9 (October), 2014: pp 2335-2340
2335
Y. PAN ET AL.
2336
Table 1. Differences of baseline characteristics between patients with NIHSS and 12-month mRS recorded and patients without data of NIHSS or 12-month mRS recorded Characteristics
NIHSS or mRS not recorded, N 5 521
NIHSS and mRS recorded, N 5 11,894
P value*
Female, n (%) Age (y), mean (SD) Current smoking, n (%) Heavy drinking, n (%) Medical history, n (%) Diabetes mellitus Hypertension Dyslipidemia Cardiovascular disease Atrial fibrillation Stroke History of mRS 2-5, n (%)
196 (37.6) 65.4 6 12.6 120 (23.0) 127 (24.4)
4561 (38.3) 65.5 6 12.3 3190 (26.8) 3215 (27.0)
.74 .16 .06 .18
117 (22.5) 339 (65.1) 74 (14.2) 77 (14.8) 37 (7.1) 189 (36.3) 49 (9.4)
2560 (21.5) 7570 (63.6) 1316 (11.1) 1715 (14.4) 881 (7.4) 4045 (34.0) 1116 (9.4)
.61 .51 .03 .82 .79 .29 .99
Abbreviations: mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale score; SD, standard deviation. *P values indicate comparison using chi-square test for categorical variables and t test or Mann–Whitney U test for continuous variables.
treatment was tested,4 the performance of SPAN-100 to predict outcomes of general ischemic stroke is unclear. Using the data from the China National Stroke Registry (CNSR),5 we aimed to evaluate the prognostic value of SPAN-100 to predict 3- and 12-month outcomes of ischemic stroke and perform an exploratory analysis to improve the performance of the score.
Methods Data were derived from CNSR,5 a nationwide prospective registry of 21,902 consecutive patients with a diagnosis of ischemic stroke, intracerebral hemorrhage, or transient
ischemic attack from 132 hospitals in China between September 2007 and August 2008. Data on demographics, clinical characteristics, and outcomes at the 3, 6, 12 months follow-up visits were collected. The collection of data for the CNSR study was approved by ethics committees at all participating hospitals. Written informed consent was obtained from all patients or their legal representatives. At 3 and 12 months after onset of stroke, the outcomes of all patients were assessed through telephone follow-up interview. The telephone follow-up was centralized for all included patients and based on a shared standardized interview protocol. Favorable outcome was defined as a modified Rankin Scale (mRS) of 0-1 and poor outcome
Table 2. Baseline characteristics between SPAN-100 positive and SPAN-100 negative patients in China National Stroke Registry Characteristics
SPAN-100 positive, N 5 479
SPAN-100 negative, N 5 11,415
P value*
Female, n (%) Age (y), mean (SD) Age categories (y), n (%) ,65 66-79 $80 NIHSS on admission, median (IQR) Current smoking, n (%) Heavy drinking, n (%) Medical history, n (%) Diabetes mellitus Hypertension Dyslipidemia Cardiovascular disease Atrial fibrillation Stroke History of mRS 2-5, n (%)
289 (60.3) 82.0 6 6.8
4272 (37.4) 64.8 6 12.0
,.001 ,.001
4 (.8) 163 (34.0) 312 (65.1) 25 (19-33) 38 (7.9) 48 (10.0)
5547 (48.6) 4739 (41.5) 1129 (9.9) 4 (2-9) 3152 (27.6) 3167 (27.7)
,.001
69 (14.4) 297 (62.0) 29 (6.1) 116 (24.2) 132 (27.6) 209 (43.6) 129 (26.9)
2491 (21.8) 7273 (63.7) 1287 (11.3) 1599 (14.0) 749 (6.6) 3836 (33.6) 987 (8.6)
,.001 .45 ,.001 ,.001 ,.001 ,.001 ,.001
,.001 ,.001 ,.001
Abbreviations: IQR, interquartile range; mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale score; SD, standard deviation; SPAN, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale. *P values indicate comparison using chi-square test for categorical variables and t test or Mann–Whitney U test for continuous variables.
PERFORMANCE OF SPAN-100 IN CNSR
2337
Table 3. 3- and 12-month proportions of poor outcome and mortality by SPAN categories according to different cutoff points SPAN cutoff
Time
100
Number of patients (%) 3 mo 12 mo
80
Number of patients (%) 3 mo 12 mo
Outcome SPAN-x* negative SPAN-x* positive Odds ratioy (95% CI) P value
mRS 2-6 Death mRS 2-6 Death mRS 2-6 Death mRS 2-6 Death
11,415 (96.0) 5143 (45.1) 730 (6.4) 4865 (42.6) 1240 (10.9) 8126 (68.3) 2859 (35.2) 277 (3.4) 2600 (32.0) 451 (5.6)
479 (4.0) 456 (95.2) 256 (53.4) 459 (95.8) 350 (73.1) 3768 (31.7) 2740 (72.7) 709 (18.8) 2724 (72.3) 1139 (30.2)
18.22 (11.92-27.85) 12.17 (9.90-14.96) 22.99 (14.61-36.17) 16.63 (13.35-20.70) 4.28 (3.92-4.68) 5.36 (4.60-6.25) 4.75 (4.35-5.19) 6.10 (5.38-6.88)
,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001
Abbreviations: CI, confidence interval; mRS, modified Rankin Scale; SPAN, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale. *X 5 100 when SPAN cutoff point is 100, X 5 80 when SPAN cutoff point is 80. yAdjusted for sex, smoking, drinking, history of diabetes mellitus, dyslipidemia, coronary heart disease, atrial fibrillation, stroke, and prestroke mRS.
was defined as mRS of 2-6 according to the original derivation study of SPAN-100 score.1 SPAN index was calculated as age in years plus NIHSS on admission.1 According to the derivation study, patients with a SPAN index score $100 were defined as SPAN-100 positive, whereas those with a score ,100 were defined as SPAN-100 negative.1 We also performed an exploratory post hoc analysis using 80 as the cutoff point. Patients with a SPAN index score $80 were designated as SPAN-80 positive and those with a score ,80 were designated as SPAN-80 negative. Patients with missing data of SPAN index score or 12month mRS were excluded from this analysis.
Statistical Analysis Categorical variables were presented as percentages and continuous variables as mean 6 standard deviation or median (interquartile range). Differences between categorical variables were compared using chi-square test, whereas continuous variables were compared using a t test or Mann–Whitney U test. Odds ratios (ORs) with its 95% confidence intervals (CIs) were calculated using multivariable logistic regression adjusting all significant baseline variables in Table 2. We tested the performance of the SPAN score by estimating their discrimination and calibration. Discrimination of the scores was assessed by the area under the receiver–operator curves (AUC) and 95% CIs, sensitivity, and specificity. An AUC of .5 indicates no discrimination, and an AUC of 1.0 indicates perfect discrimination. Calibration was assessed by comparing predicted and observed probability of poor outcome using Pearson correlation coefficient. We tested the performance of the continuous SPAN index score, SPAN-100, and SPAN-80, respectively. The a level of significance was P , .05 on 2 sides. All analyses were performed with SAS software version 9.3 (SAS Institute Inc, Cary, NC).
Results Among 12,415 ischemic stroke patients enrolled in CNSR who consented for follow-up, 494 (4.0%) and 27 patients (.2%) were excluded for missing data of NIHSS or 12-month mRS, respectively. There was no difference between patients with and without data of NIHSS and mRS recorded (Table 1). A total of 11,894 ischemic stroke patients were included in this analysis, of whom 479 (4.0%) were SPAN-100 positive. The baseline characteristics between SPAN-100 positive and SPAN-100 negative patients were shown in Table 2. SPAN-100 positive patients were older, had higher NIHSS score, and more risk factors such as cardiovascular disease, atrial fibrillation, and history of stroke than SPAN-100 negative patients. SPAN-100 positive patients had a higher proportion of poor outcomes at 3 months (95.2% vs 45.1%; adjusted OR, 18.22; 95% CI, 11.92-27.85) and 12 months (95.8% vs. 42.6%; adjusted OR, 22.99; 95% CI, 14.61-36.17) than SPAN-100 negative patients (Table 3). The distributions of mRS at 3 and 12 months according to SPAN categories are shown in Figure 1. The AUC, sensitivity, and specificity of SPAN-100 score are shown in Table 4. The AUC of SPAN-100 for poor outcome was .54 (95% CI, .54-.54) at 3 months and .54 (95% CI, .54-.55) at 12 months. The AUC (c-statistic) of continuous SPAN index score for discrimination between patients with poor and good outcome was .73 (95% CI, .72-.74) at 3 months and .75 (95% CI, .74-.76) at 12 months (Fig 2). Calibration analysis of continuous SPAN index score showed a high correlation between predicted and observed probability of poor outcome at 3 months (r 5 .956, P , .001) and 12 months (r 5 .974, P , .001). The calibration plot of the continuous SPAN index score for poor outcome at 12 months is shown in Figure 3. In the exploratory post hoc analysis, the performance of the score was tested using 80 as the cutoff point. The
Y. PAN ET AL.
2338
Figure 1. Distribution of mRS scores at 3 and 12 months according to SPAN categories. The size of the bars indicates the percentage of patients with a particular mRS score in each group. (A) mRS distribution according to SPAN-100. (B) mRS distribution according to SPAN-80. Abbreviations: mRS, modified Rankin Scale; SPAN, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale.
proportions of poor outcome and mortality at 3 and 12 months according to SPAN-80 categories are shown in Figure 1 and Table 3. The AUC, sensitivity, and specificity of SPAN-80 are shown in Table 4. The AUC of SPAN-80 for poor outcome was .66 (95% CI, .66-.67) at 3 months and .68 (95% CI, .67-.68) at 12 months, which is significantly superior to SPAN-100 (P , .001).
Discussion Our study showed modest performance of the continuous SPAN index score but a low predictive power of SPAN-100 score for 3- and 12-month outcomes of general ischemic stroke in Chinese population. The c-statistic was very close to validation of SPAN-100 in Randomized Trial
Table 4. Predictive value of SPAN score for 3- and 12-month poor outcome and mortality according to different cutoff points SPAN cutoff
Time
Outcome
AUC (95% CI)
Sensitivity, %
Specificity, %
100
3 mo
mRS 2-6 Death mRS 2-6 Death mRS 2-6 Death mRS 2-6 Death
.54 (.54-.54) .62 (.61-.63) .54 (.54-.55) .60 (.59-.61) .66 (.66-.67)* .72 (.70-.73)* .68 (.67-.68)* .73 (.72-.74)*
8.1 26.0 8.6 22.0 48.9 71.9 51.2 71.6
99.6 98.0 99.7 98.7 83.7 72.0 84.1 74.5
12 mo 80
3 mo 12 mo
Abbreviations: AUC, area under the receiver–operator curve; CI, confidence interval; mRS, modified Rankin Scale; SPAN, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale. *P , .001 comparing with AUC of SPAN-100 for corresponding outcome.
PERFORMANCE OF SPAN-100 IN CNSR
2339
Figure 3. Calibration plot of the continuous SPAN index score for poor outcome at 12 months. The continuous line indicates the predicted probability of poor outcome. The dots indicate the observed rates of poor outcome, and the vertical lines indicate the 95% confidence intervals. Abbreviations: mRS, modified Rankin Scale; SPAN, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale.
Figure 2. Receiver–operator curve of the SPAN index score for poor outcome at 3 months (A) and 12 months (B). Abbreviations: SPAN, Stroke Prognostication using Age and NIH (National Institutes of Health) Stroke Scale.
Evaluating Performance of the Trevo Retriever Versus the Merci Retriever in Acute Ischemic Stroke (TREVO-2) trial predicting poor 3-month outcome (mRS 3-6) in ischemic stroke patients receiving endovascular stroke treatment (c-statistic 5 .560).4 The SPAN-100 was reported to have high specificity but low sensitivity, and most SPAN-100 positive patients had very high severity of stroke.4 Our study presented similar results. In the exploratory analysis using 80 instead of 100 as the cutoff point, SPAN-80 improved the sensitivity and had a discriminatory power superior to SPAN-100.
Our study has several strengths. First, our study examined the performance of SPAN-100 to predict 3- and 12-month outcomes in a large Chinese ischemic stroke population. Second, we proposed a new cutoff point of SPAN score (80) to improve the discriminatory power to predict 3- and 12-month outcomes. The original application of SPAN-100 was to predict the prognosis of ischemic stroke patients after thrombolysis, and 1 rationale for the creation of SPAN-100 score was that patients 80 years of age and older and with high NIHSS scores (eg, $20) had a higher risk of hemorrhage and poorer outcome for ischemic stroke after thrombolysis.1 However, in general ischemic stroke patients, 60 years of age or older or 65 years of age or older were identified as an independent prognostic factor.6-11 This may explain why SPAN-100 had a low sensitivity in predicting 3- and 12-month outcomes in general ischemic stroke population but the performance improved using 80 as the new cutoff point. Our study has limitations. The study sites of CNSR were mainly selected from urban areas of China representing sites with better stroke care quality than those in rural areas.5 Second, follow-up mRS assessment was performed by telephone interviews rather than faceto-face clinical examination, which may have biased the outcome. However, the telephone assessment of the mRS with a structured interview was shown to have a good agreement with face-to-face assessment.12 Third, although in the exploratory analysis using 80 as the cutoff point, SPAN-80 had a discriminatory power superior to SPAN-100, none of them had an AUC above the threshold of .8 required for use in individuals. The possible explanation is that, although age and NIHSS score are 2 strong predictors of ischemic stroke,7,8,13-15 there are other independent prognosis factors such as acute glucose, hypertension, onset-to-treatment time,
2340
previous transient ischemic attack, ischemic stroke or myocardial infarction.7,13,14 However, instead of developing a new complex score, the aim of our study was to assess and perform the post hoc analysis for the performance of the easy-to-use SPAN-100 score in general ischemic stroke patients. In conclusion, SPAN-100 suffered from low prediction power for 3- and 12-month outcomes of ischemic stroke in Chinese population. A cutoff point of 80 may improve the performance, but none of them had an AUC above the threshold of .8 required for use in individuals.
References 1. Saposnik G, Guzik AK, Reeves M, et al. Stroke Prognostication using Age and NIH Stroke Scale: SPAN-100. Neurology 2013;80:21-28. 2. Grotta JC, Burgin WS, El-Mitwalli A, et al. Intravenous tissue-type plasminogen activator therapy for ischemic stroke: Houston experience 1996 to 2000. Arch Neurol 2001;58:2009-2013. 3. Wang Y, Liao X, Zhao X, et al. Using recombinant tissue plasminogen activator to treat acute ischemic stroke in China: analysis of the results from the Chinese National Stroke Registry (CNSR). Stroke 2011;42:1658-1664. 4. Flint AC, Xiang B, Gupta R, et al. THRIVE score predicts outcomes with a third-generation endovascular stroke treatment device in the TREVO-2 trial. Stroke 2013; 44:3370-3375. 5. Wang Y, Cui L, Ji X, et al. The China National Stroke Registry for patients with acute cerebrovascular events: design, rationale, and baseline patient characteristics. Int J Stroke 2011;6:355-361.
Y. PAN ET AL. 6. Diener HC, Ringleb PA, Savi P. Clopidogrel for the secondary prevention of stroke. Expert Opin Pharmacother 2005;6:755-764. 7. Strbian D, Meretoja A, Ahlhelm FJ, et al. Predicting outcome of IV thrombolysis-treated ischemic stroke patients: the DRAGON score. Neurology 2012;78:427-432. 8. Smith EE, Shobha N, Dai D, et al. Risk score for inhospital ischemic stroke mortality derived and validated within the Get With the Guidelines-Stroke Program. Circulation 2010;122:1496-1504. 9. Singer DE, Chang Y, Borowsky LH, et al. A new risk scheme to predict ischemic stroke and other thromboembolism in atrial fibrillation: the ATRIA study stroke risk score. J Am Heart Assoc 2013;2:e000250. 10. Kamouchi M, Kumagai N, Okada Y, et al. Risk score for predicting recurrence in patients with ischemic stroke: the Fukuoka stroke risk score for Japanese. Cerebrovasc Dis 2012;34:351-357. 11. Johnston SC, Rothwell PM, Nguyen-Huynh MN, et al. Validation and refinement of scores to predict very early stroke risk after transient ischaemic attack. Lancet 2007; 369:283-292. 12. Janssen PM, Visser NA, Dorhout Mees SM, et al. Comparison of telephone and face-to-face assessment of the modified Rankin Scale. Cerebrovasc Dis 2010;29:137-139. 13. Saposnik G, Kapral MK, Liu Y, et al. IScore: a risk score to predict death early after hospitalization for an acute ischemic stroke. Circulation 2011;123:739-749. 14. Ntaios G, Faouzi M, Ferrari J, et al. An integer-based score to predict functional outcome in acute ischemic stroke: the ASTRAL score. Neurology 2012;78:1916-1922. 15. Weimar C, K€ onig IR, Kraywinkel K, et al. Age and National Institutes of Health Stroke Scale Score within 6 hours after onset are accurate predictors of outcome after cerebral ischemia: development and external validation of prognostic models. Stroke 2004;35:158-162.
