Predicting 10-day Mortality in Patients with Strokes Using Neural Networks and Multivariate Statistical Methods € € ner C € lya Tireli, MDx Gu K. Baykan, PhD,† Yakup Kara, PhD,‡ and Hu ¸ elik, MD,* Omer
Background: The aim of the present study was to evaluate the performance of 2 different multivariate statistical methods and artificial neural networks (ANNs) in predicting the mortality of hemorrhagic and ischemic patients within the first 10 days after stroke. Methods: The multilayer perceptron (MLP) ANN model and multivariate statistical methods (multivariate discriminant analysis [MDA] and logistic regression analysis [LRA]) have been used to predict acute stroke mortality. The data of total 570 patients (230 hemorrhagic and 340 ischemic stroke), who were admitted to the hospital within the first 24 hours after stroke onset, have been used to develop prediction models. The factors affecting the prognosis were used as inputs for prediction models. Survival or death status of the patients was taken as output of the models. Results: For the MLP method, the accuracies were 99.9% in a training data set and 80.9% in a testing data set for the hemorrhagic group, whereas 97.8% and 75.9% for the ischemic group, respectively. For the MDA method, the training and testing performances were 89.8%, 87.8% and 80.6%, 79.7% for hemorrhagic and ischemic groups, respectively. For the LRA method, the training and testing performances for the hemorrhagic group were 89.7% and 86.1%, and for the ischemic group were 81.7% and 80.9%, respectively. Conclusions: Training and test performances yielded different results for ischemic and hemorrhagic groups. MLP method was most successful for the training phase, whereas LRA and MDA methods were successful for the test phase. In the hemorrhagic group, higher prediction performances were achieved for both training and testing phases. Key Words: Predicting outcome—ischemic stroke—hemorrhagic stroke— models—statistical. Ó 2014 by National Stroke Association
Introduction Human life is extending and the population of elderly people is increasing in developed countries. It is expected that longevity of the human life will increase in the
From the *Department of Neurology, Faculty of Medicine, Baskent University, Konya; †Department of Computer Engineering, Selcuk University, Konya; ‡Department of Industrial Engineering, Selcuk University, Konya; and xDepartment of Neurology, Haydarpasa Numune Training and Research Hospital, Istanbul, Turkey. Received November 12, 2013; revision received December 5, 2013; accepted December 16, 2013. Address correspondence to G€ uner C ¸ elik, MD, Department of Neurology, Faculty of Medicine, Baskent University, Konya, Turkey, Hocacihan Mh. Saray Cd. No. 1 Selc¸uklu, Konya, Turkey. E-mail: [email protected]. 1052-3057/$ - see front matter Ó 2014 by National Stroke Association http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2013.12.018
future.1 The incidence of stroke increases with age, and the prognosis is more severe in the elderly people than the younger.1,2 More recently, the addition of more promising and new treatment options has not yet pulled down high mortality and morbidity rates. It is expected that the incidence of stroke and the rate of dependent people will increase with the increase in longevity of human life. To decrease the rates of mortality and morbidity, it is of utmost importance to determine the appropriate treatment, care, and rehabilitation programs for each patient in early stage of stroke. For these purposes, a number of models are present in the literature for prediction of mortality and independent survival rates after stoke. It has been aimed to use these models for randomization of patients in clinical studies and provide case–controls in nonrandomized studies, compare patients in different groups, and analyze the results of randomized trials
Journal of Stroke and Cerebrovascular Diseases, Vol. -, No. - (---), 2014: pp 1-7
1
G. C¸ELIK ET AL.
2
and meta-analysis. For this purpose, various models and methods have been developed up-to-date.3-15 Most of these models are not widely accepted and routinely used in practice. In addition, the rates of the accuracy and reliability have generally not reached the satisfying level for these models. Clinical applicability of these models depends on its easy feasibility/applicability, reliability, and its success in predicting the clinical outcome. The aim of the present study was to determine the performance of 2 different multivariate statistical methods (logistic regression analysis [LRA] and multivariate discriminant analysis [MDA]) and artificial neural networks (ANNs) particularly in early (first 10 days) prognosis of ischemic and hemorrhagic strokes, and to compare them with each other. By this way, it was aimed to provide contribution to the practice with the results of a rigorous experiment conducted based on a wide data set.
Materials and Methods Research Data The data of 1725 patients with stroke, admitted to the emergency service in the first 24 hours of stroke and hospitalized in the Neurology Department in Haydarpasa Numune Training and Research Hospital, have been used in the study. The patients diagnosed of transient ischemic attack, sinus thrombosis, subarachnoid hemorrhage, and brain tumor were excluded from the data set after computed tomography or magnetic resonance imaging. The remaining 968 patients were included in the study. Of these patients 293 (30.3%) died and 675 (69.7%) survived. On the basis of the results of our preliminary experiments, it was decided to include equal number of dead and survived patients in a group to improve the training performance of proposed models. Finally, 2 groups, hemorrhagic with 230 patients and ischemic with 340 patients, were obtained and used in the experiments. To increase the reliability of the results, a 5-fold cross-validation technique was used. The number of patients in the entire data set and modeling data set are given in Table 1. The factors affecting the prognosis of hemorrhagic and ischemic strokes have been determined by analyzing similar studies in the literature and were used as inputs in prediction models. Although age, gender, hypertension, diabetes mellitus, smoking, mean blood pressure on admission, and Scandinavian Stroke Scale score16 were used as common inputs for both groups, inputs for ischemic stroke included blood glucose levels, lacunar infarct, nonlacunar infarct, ischemic stroke with undetermined etiology,12 congestive heart failure, coronary heart disease, myocardial infarction, atrial fibrillation, history of cerebrovascular disease, and transient ischemic attack; inputs for the hemorrhagic group included pulse pressure, localization of hemorrhage (putaminal, thalamic,
Table 1. Number of cases in data sets Stroke type
Data set
Hemorrhagic Entire Modeling Ischemic Entire Modeling Total Entire Modeling
Dead Survived All subjects 119 115 174 170 293 285
138 115 537 170 675 285
257 230 711 340 968 570
cerebellar, pontine, lobar, and others [caudate, thalamocapsular, intraventricular]), volume of hemorrhage, ventricular drainage, and the presence or absence of a midline shift. In the determination of the early-stage outcomes of patients with acute stroke, the findings of average 10-day hospitalization were used. Return and normalization of the cerebral function, however, is variable according to the age, risk factor, and site and size of the lesion 10 days is the average duration of hospitalization in our clinic. Output for both groups was survival or death within the first 10 days after the stroke. Tables 2 and 3 show summary statistics for hemorrhagic and ischemic strokes, respectively.
ANN Model A multilayer perceptron (MLP) neural network is a feed-forward neural network model proposed by Rumelhart et al17 and consists of 1 input layer, 1 or more hidden layers, and 1 output layer.18 The number of neurons in input and output layers is determined by the number of input and output vectors used in the data set. The number of neurons in the hidden layer can be determined experimentally or based on experience. A neuron on a layer is connected to all neurons of the adjacent layer with its weights. Usually, the initial values of weights are randomly selected. Outputs of hidden and output layer neurons are produced depending on selected transfer function and weighted neuron inputs. The MLP model uses a supervised neural network and is trained by a gradient descent method to minimize an error function.19 Generalized delta learning rule based on least square can be used to train an MLP network, so these weights are adjusted for a given set of input–output pairs.20 A typical MLP architecture adopted in this study is shown in Figure 1.
Multivariate Statistical Models In this group of prediction techniques, 2 multivariate statistical methods MDA and LRA were used to predict the stroke mortality. MDA is concerned with the classification of distinct sets of observations, and it tries to find the combination of variables that predicts the group to which an observation belongs. The combination of predictor variables is
PREDICTING MORTALITY OF STROKES
3
Table 2. Summary statistics of the modeling data set (hemorrhagic) Variable Gender N Male Female Total Age Mean 6 SD Range Pulse pressure Mean 6 SD Range Mean blood pressure Mean 6 SD Range Stroke severity (Scandinavian Stroke Scale score) Mean 6 SD Range Hematoma location, n (%) Putaminal Thalamic Pontine Cerebellar Lobar Intraventricular Hematoma size Mean 6 SD Range Risk factors, n (%) Hypertension Diabetes Smoking Hematoma status, n (%) Intraventricular hematoma Shift Hematoma in fourth ventricular
called a linear discriminant function, and this function can then be used to classify new observations whose group membership is unknown. The linear discriminant function is as follows: D5B0 1B1 X1 1B2 X2 1/1Bn Xn ; where D is a discriminant score, B0 is an estimated constant, Bn values are the estimated coefficients, and Xn values are the variables. On the basis of this discriminant function score, an observation is classified into the appropriate group. LRA is a form of regression, which is used when the dependent is a dichotomy and the independents are of any type. In LRA models, the dependent variable is usually binary that can take the value 1 with a probability of success P(Zi) or the value 0 with a probability of failure 1 2 P(Zi). The relationship between
Dead
Survived
All subjects
62 53 115
59 56 115
121 109 230
64.5 6 13.3 29-94
63.8 6 11.8 38-92
64.2 6 12.5 29-94
93.7 6 28.5 20-200
85.9 6 26.7 30-160
89.8 6 27.9 20-200
143.8 6 30.1 80-233.3
136.6 6 27.3 70-203.3
140.2 6 28.9 70-233.3
5.9 6 8.9 1-40
26.1 6 16.1 1-58
16.0 6 16.4 1-58
59 (51.3) 20 (17.4) 22 (19.1) 2 (.2) 14 (12.2) 0 (.0)
45 (39.1) 53 (46.1) 2 (.2) 6 (5.2) 6 (5.2) 4 (3.5)
104 (45.2) 73 (31.7) 24 (10.4) 8 (3.5) 20 (8.7) 4 (1.7)
120.3 6 104.7 4-525
31.6 6 35.9 3-196
75.9 6 89.9 3-525
81 (70.4) 18 (15.6) 5 (4.3)
77 (66.9) 13 (11.3) 12 (10.4)
158 (68.7) 31 (13.5) 17 (7.4)
62 (53.9) 34 (29.6) 38 (33.0)
24 (20.9) 5 (4.3) 7 (6.1)
86 (37.4) 39 (16.9) 45 (19.6)
predictor independent variables and binary dependent variables is expressed with the following nonlinear function: PðZi Þ5
eZi 1 5 ; 11euZi 11eu2Zi
where P(Zi) is a cumulative probability function that takes values between 0 and 1. Zi 5b0 1b1 F1 1b2 F2 1/1bm Fm ; where b0 is the constant of the equation and bm values are the coefficients of the predictor variables. LRA aims to correctly predict the group of outcome for individual observations using the most parsimonious model. A model is created that includes all predictor variables that are useful in predicting the response variable.21
G. C¸ELIK ET AL.
4
Table 3. Summary statistics of the modeling data set (ischemic) Variable Gender N Male Female Total Age Mean 6 SD Range Mean blood pressure Mean 6 SD Range Stroke severity (Scandinavian Stroke Scale score) Mean 6 SD Range Risk factors, n (%) Hypertension Diabetes mellitus Congestive heart failure Coronary artery disease Myocardial infarction Atrial fibrillation Smoke Transient ischemic attack Cerebrovascular disease Stroke subtype, n (%) Lacunar Nonlacunar Undetermined etiology Glucose on admission Mean 6 SD Range
Experimental Results Because the effectiveness of neural network techniques can be affected by their parameters, a number of experiments are carried with different levels of their parameters and different activation functions. In this study, for hemorrhagic and ischemic groups, different experiments were implemented by using MATLAB software package (version 7.04 with neural network toolbox, MathWorks, Inc., Natick, MA). The input data sets of both hemorrhagic and ischemic groups were normalized in the range of 21 to 1. A hyperbolic tangent sigmoid transfer function was selected on the hidden layer and a sigmoid transfer function was used on the output layer. Traingdx was used as a training function in MATLAB, which updates weights and bias values according to the gradient descent momentum and an adaptive learning rate. For training process, the maximum training cycles, learning rate, and the momentum coefficient were selected as 1000, .10, .70, respectively. Mean squared error is used as a performance measure of training processes, which can be calculated as follows: n 2 1X Mean squared error5 Yi 2Y^i ; n i51
Dead
Survived
All subjects
72 98 170
73 97 170
145 195 340
72.5 6 12.2 20-97
68.3 6 13.2 29-98
70.4 6 12.9 20-98
121.7 6 28.5 50-200
114.9 6 25.0 46.6-223.3
118.3 6 27.0 46.6-223.3
9.7 6 10.8 1-55
30.7 6 15.9 1-58
20.2 6 17.2 1-58
97 (57.0) 44 (25.9) 49 (28.8) 41 (24.1) 13 (7.6) 55 (32.4) 10 (5.9) 1 (.6) 49 (28.8)
106 (62.4) 33 (19.4) 22 (12.9) 31 (18.2) 9 (5.3) 38 (22.4) 21 (12.4) 11 (6.5) 37 (21.8)
203 (59.7) 77 (22.6) 71 (20.9) 72 (21.2) 22 (6.5) 93 (27.4) 31 (9.1) 12 (3.5) 86 (25.3)
0 (.0) 120 (70.6) 51 (.3)
29 (17.1) 71 (41.8) 70 (41.2)
29 (8.5) 191 (56.2) 121 (35.6)
174.0 6 72.33 23-562
153.5 6 70.9 45-438
163.7 6 72.4 23-562
where n is the number of sample in training or testing data. Yi is the desired output, and Y^i is the output of the neural networks. The architecture of ANN model consisted of 17 input neurons and 1 output neuron. The number of neurons in the hidden layer varied from 10-100 and the best numbers of neurons for both hemorrhagic and ischemic groups were selected. The best numbers of neurons in hidden layers were found to be 90 and 60 for hemorrhagic and ischemic groups, respectively. To make the testing results more reliable, a 5-fold cross-validation technique was used. Each parameter combination was applied to the training and testing data sets, and prediction accuracy, sensitivity, and specificity of the models were evaluated. The results for training and testing data sets are given in Tables 4 and 5, respectively. In the MLP method, the accuracy values were 99.9% in a training data set and 80.9% in a testing data set for the hemorrhagic group. The same values were 97.8% and 75.9% for the ischemic group, respectively. According to these results, it is observed that MLP has produced better training and testing results for the hemorrhagic group. The sensitivity and specificity values gathered in the
PREDICTING MORTALITY OF STROKES
5
Figure 1. The architecture of 3-layered feedforward artificial neural network.
training phase of hemorrhagic and ischemic groups were more than 97%, and this rate shows that the performance of MLP training was high. In MLP testing processes, sensitivity value (86.9%) was higher than specificity value (77.3%) for the hemorrhagic group and its performance was observed to be much lower for survived patients. In MLP testing processes, sensitivity value (73.4%) was lower than specificity value (77.4%) for the ischemic group, and its performance was observed to be much higher for survived patients. MDA and LRA processes were carried out using SPSS 20 software produced by IBM (Armonk, NY). In the LRA method, the classification process was realized by choosing a cutoff probability value as .50. In the MDA method, the training and testing performances were 89.8% and 87.8%, respectively, for the hemorrhagic group, whereas these values were 80.6% and 79.7% for the ischemic group, respectively. For the hemorrhagic group, sensitivity and specificity values were found to be 88.1% and 91.5% in the training phase, whereas it was 87.8% for the testing phase. For the hemorrhagic group, it was observed from the experimental results that the MDA method shows similar performance for dead and survived patients in training and testing phases. For the ischemic group, sensitivity and specificity values were found to be 85.9% and 75.3% in the training phase, whereas these values were 85.3% and 74.1% for the testing phase. For the ischemic group, similar performance was observed in training and testing phases. In this group, it is observed that the success performance for survived patients is lower. MDA tech-
nique provides much higher classification, specificity, and sensitivity performances for the hemorrhagic group than the ischemic group. In the LRA method, meanwhile the training and testing performances for the hemorrhagic group were observed as 89.7% and 86.1%, respectively, whereas these values were 88.9% and 80.9% for the ischemic group, respectively. For the hemorrhagic group, sensitivity and specificity values were found to be 88.1% and 90.4% in the training phase, whereas these values were 86.9% and 85.2% for the testing phase. For the hemorrhagic group, LRA method shows similar performance values for dead and survived patients in training and testing phases. For the ischemic group, sensitivity and specificity values were 85.3% and 78.1% in the training phase, whereas 85.9% and 75.9% for the testing phase. In this group, it was observed that the success performance for survived patients was lower. In an LRA technique, much higher classification, specificity, and sensitivity performances were gathered for the hemorrhagic group than the ischemic group.
Discussion Comparing the performances of the methods, it was observed that MLP has a 100% classification, specificity, and sensitivity values for both hemorrhagic and ischemic groups in the training phase. LRA and MDA methods were observed to show successful results, although these results are close to each other and lower than MLP for both hemorrhagic and ischemic groups in the training
G. C¸ELIK ET AL.
6
Table 4. Performance of the models for training data set Logistic regression
Discriminant analysis
Artificial neural network
Stroke type
Fold
Sns.
Spc.
Acc.
Sns.
Spc.
Acc.
Sns.
Spc.
Acc.
Hemorrhagic
1 2 3 4 5 Average 1 2 3 4 5 Average
88.0 91.3 88.0 88.0 89.1 88.9 84.6 86.0 84.6 86.8 84.6 85.3
89.1 92.4 90.2 88.0 92.4 90.4 80.1 80.1 76.5 77.2 76.5 78.1
88.6 91.8 89.1 88.0 90.8 89.7 82.4 83.1 80.5 82.0 80.5 81.7
87.0 87.0 87.0 92.4 87.0 88.1 83.8 87.5 86.0 86.8 85.3 85.9
90.2 94.6 91.3 90.2 91.3 91.5 77.2 77.9 73.5 72.8 75.0 75.3
88.6 90.8 89.1 91.3 89.1 89.8 80.5 82.7 79.8 79.8 80.1 80.6
100 100 100 100 100 100.0 97.8 97.8 97.8 99.3 99.3 98.4
98.9 100 100 100 100 99.8 97.1 97.1 97.1 97.8 97.1 97.2
99.5 100 100 100 100 99.9 97.4 97.4 97.4 98.5 98.2 97.8
Ischemic
Abbreviations: Acc, accuracy; Sns, sensitivity; Spc, specifity.
methodology and statistics, it is confirmed that the classification performance of our study is similar to other studies more or less and is even higher than some of them when the results that we acquired with the help of 3 different methods were compared with these studies in the literature. It can be seen that the experimental results we concluded were good enough for physician to make decisions in treatment process. The performance in the hemorrhagic group is especially convincing. The reason of lower performance in the ischemic group may not be methodological or related to the analysis method. Taking the equal number of dead and survived patients in training and testing phases, using sufficient number of patients for training and testing for all 3 methods, and constituting 2 different cohorts with hemorrhagic and ischemic stroke patients are important aspects in our study that increase its reliability. In addition, this work differs from present studies in the literature in terms of comparison of the performances of ANNs and multivariate statistical methods on the same data set. As a result,
phase. However, when the performances of the methods in the testing phase were compared, the best classification, specificity, and sensitivity success (87.8%) for the hemorrhagic group was observed with MDA technique. The best classification, specificity, and sensitivity success (sen. 85.9%, spc. 75.9%, and acc. 80.9%) for the ischemic group was gained with LRA technique. In testing processes, the lowest performance was observed with MLP technique, whereas MDA and LRA results were close to each other. When the training phases were taken into account, however, it was concluded that MLP results were more reliable. The classification, specificity, and sensitivity performances that were gained from all the MLP, MDA, and LRA methods were found to be higher for the hemorrhagic group and lower for the ischemic group. There are studies in the literature that focus on predicting the outcome of stroke after 6 hours,10 5 days,8 30 days,3,7 3 months,6,11 100 days,2,9 6 months,7 or 15 years.13,14 Although these studies differ not only from each other but also from our study in terms of
Table 5. Performance of the models for testing data set LR
DA
Artificial neural network
Stroke type
Fold
Sns.
Spc.
Acc.
Sns.
Spc.
Acc.
Sns.
Spc.
Acc.
Hemorrhagic
1 2 3 4 5 Average 1 2 3 4 5 Average
87.0 95.6 82.6 82.6 86.9 86.9 88.2 76.5 88.2 88.2 88.2 85.9
87.0 73.9 86.9 100.0 78.3 85.2 64.7 70.6 94.1 79.4 70.6 75.9
87.0 84.8 84.8 91.3 82.6 86.1 76.5 73.5 91.2 83.8 79.4 80.9
91.3 95.7 82.6 78.3 91.3 87.8 88.2 76.5 88.2 88.2 85.3 85.3
87.0 82.6 86.9 100.0 82.6 87.8 64.7 64.7 91.2 79.4 70.6 74.1
89.1 89.1 84.8 89.1 87.0 87.8 76.5 70.6 89.7 83.8 77.9 79.7
88.2 78.3 90.0 100.0 78.3 86.9 70.7 71.4 89.7 70.6 74.4 73.4
72.4 78.3 80.8 76.7 78.3 77.3 81.5 72.8 79.5 70.6 82.8 77.4
78.3 78.3 84.8 84.8 78.3 80.9 75.0 72.1 83.8 70.6 77.9 75.9
Ischemic
PREDICTING MORTALITY OF STROKES
we conclude that different statistical methods used on different groups are more promising. We observed that the group constituted the number of patients used, model type used, the number of data sets created in this modeling, and using the data that has an effect on prognosis in the system affects the performance of the models.
Conclusions This study analyzes the performance of 2 different multivariate statistical methods (MDA and LRA) and the MLP ANN in predicting the prognosis of the hemorrhagic and ischemic stroke patients especially in the early period (first 10 days) and compares them. According to the experimental results, the qualifications used in the study, the data set, and the classifier affect the prediction performance directly. The training and testing performances of the methods used show variation for ischemic and hemorrhagic groups. MLP provides higher success in the training phase, whereas other statistical methods provide better results is the testing phase. In the hemorrhagic group, higher prediction performance is obtained in training and testing phases. Finally, there are a limited number of treatment alternatives for stroke, and there is a limited time to use the selected alternative. Therefore, the proposed models with high sensitivity and specificity of ischemic and hemorrhagic strokes can be useful to select the best treatment alternative in an individual patient, and so will increase the success rate of performed treatments.
References 1. Kammersgaard LP, Jørgensen HS, Reith J, et al, Copenhagen Stroke Study. Short- and long-term prognosis for very old stroke patients. The Copenhagen Stroke Study. Age Ageing 2004;33:149-154. 2. Weimar C, Ziegler A, K€ onig IR, et al. Predicting functional outcome and survival after acute ischemic stroke. J Neurol 2002;249:888-895. 3. Tuhrim S, Horowitz DR, Sacher M, et al. Validation and comparison of models predicting survival following intracerebral hemorrhage. Crit Care Med 1995;23:950-954. 4. Edwards DF, Hollingsworth H, Zazulia AR, et al. Artificial neural networks improve the prediction of mortality in intracerebral hemorrhage. Neurology 1999;53:351-357. 5. Inouye M. Predicting outcomes of patients in Japan after first acute stroke using a simple model. Am J Phys Med Rehabil 2001;80:645-649. 6. Johnston KC, Connors AF Jr, Wagner DP, et al. A predictive risk model for outcomes of ischemic stroke. Stroke 2000;31:448-455.
7 7. Counsell C, Dennis M, McDowall M, et al. Predicting outcome after acute and subacute stroke: development and validation of new prognostic models. Stroke 2002; 33:1041-1047. 8. Counsel C, Dennis MS, Lewis S, et al, FOOD Trial Collaboration. Performance of a statistical model to predict stroke outcome in the context of a large, simple, randomized, controlled trial of feeding. Stroke 2003;34:127-133. 9. German Stroke Study Collaboration. Predicting outcome after acute ischemic stroke: an external validation of prognostic models. Neurology 2004;62:581-585. 10. Lewis SC, Sandercock PA, Dennis MS, SCOPE (Stroke Complications and Outcomes Prediction Engine) Collaborations, IST. Predicting outcome in hyper-acute stroke: validation of a prognostic model in the Third International Stroke Trial (IST3). J Neurol Neurosurg Psychiatry 2008;79:397-400. Epub 2007 Aug 31. 11. K€ onig IR, Ziegler A, Bluhmki E, et al, Virtual International Stroke Trials Archive (VISTA) Investigators. Predicting long-term outcome after acute ischemic stroke: a simple index works in patients from controlled clinical trials. Stroke 2008;39:1821-1826. 12. Saposnik G, Kapral MK, Liu Y, et al, Investigators of the Registry of the Canadian Stroke Network, Stroke Outcomes Research Canada (SORCan) Working Group. IScore: a risk score to predict death early after hospitalization for an acute ischemic stroke. Circulation 2011;123:739-749. 13. Li WJ, Gao ZY, He Y, et al. Application and performance of two stroke outcome prediction models in a Chinese population. PM R 2012;4:123-128. 14. Hankey GJ, Jamrozik K, Broadhurst RJ, et al. Five-year survival after first-ever stroke and related prognostic factors in the Perth Community Stroke Study. Stroke 2000; 31:2080-2086. 15. Weir NU, Counsell CE, McDowall M, et al. Reliability of the variables in a new set of models that predict outcome after stroke. J Neurol Neurosurg Psychiatry 2003;74:447-451. 16. Jørgensen HS, Reith J, Nakayama H, et al. What determines good recovery in patients with the most severe strokes? The Copenhagen Stroke Study. Stroke 1999;30:2008-2012. 17. Rumelhart DE, Hinton GE, Williams RJ. Learning representation by back-propagating errors. Nature 1986; 323:533-536. 18. Fu L. Neural networks in computer intelligence. USA: McGraw-Hill 1994:31. 19. Bose NK, Liang P. Multilayer networks. Neural network fundamentals with graphs, algorithms and applications. USA: McGraw-Hill 1996:155-212. € E. Yapay sinir a gı modeli c¸ok katmanlıalgılayıcı. 20. Oztemel _ Yapay Sinir A gları. Istanbul, T€ urkiye: Papatya Yayıncılık 2003:75-114. 21. Boyacıo glu MA, Kara Y, Baykan OK. Predicting bank financial failures using neural networks, support vector machines and multivariate statistical methods: a comparative analysis in the sample of savings deposit insurance fund (SDIF) transferred banks in Turkey. Expert Syst Appl 2009;36:3355-3366.
Background: The aim of the present study was to evaluate the performance of 2 different multivariate statistical methods and artificial neural networks (ANNs) in predicting the mortality of hemorrhagic and ischemic patients within the first 10 days after stroke. Methods: The multilayer perceptron (MLP) ANN model and multivariate statistical methods (multivariate discriminant analysis [MDA] and logistic regression analysis [LRA]) have been used to predict acute stroke mortality. The data of total 570 patients (230 hemorrhagic and 340 ischemic stroke), who were admitted to the hospital within the first 24 hours after stroke onset, have been used to develop prediction models. The factors affecting the prognosis were used as inputs for prediction models. Survival or death status of the patients was taken as output of the models. Results: For the MLP method, the accuracies were 99.9% in a training data set and 80.9% in a testing data set for the hemorrhagic group, whereas 97.8% and 75.9% for the ischemic group, respectively. For the MDA method, the training and testing performances were 89.8%, 87.8% and 80.6%, 79.7% for hemorrhagic and ischemic groups, respectively. For the LRA method, the training and testing performances for the hemorrhagic group were 89.7% and 86.1%, and for the ischemic group were 81.7% and 80.9%, respectively. Conclusions: Training and test performances yielded different results for ischemic and hemorrhagic groups. MLP method was most successful for the training phase, whereas LRA and MDA methods were successful for the test phase. In the hemorrhagic group, higher prediction performances were achieved for both training and testing phases. Key Words: Predicting outcome—ischemic stroke—hemorrhagic stroke— models—statistical. Ó 2014 by National Stroke Association
Introduction Human life is extending and the population of elderly people is increasing in developed countries. It is expected that longevity of the human life will increase in the
From the *Department of Neurology, Faculty of Medicine, Baskent University, Konya; †Department of Computer Engineering, Selcuk University, Konya; ‡Department of Industrial Engineering, Selcuk University, Konya; and xDepartment of Neurology, Haydarpasa Numune Training and Research Hospital, Istanbul, Turkey. Received November 12, 2013; revision received December 5, 2013; accepted December 16, 2013. Address correspondence to G€ uner C ¸ elik, MD, Department of Neurology, Faculty of Medicine, Baskent University, Konya, Turkey, Hocacihan Mh. Saray Cd. No. 1 Selc¸uklu, Konya, Turkey. E-mail: [email protected]. 1052-3057/$ - see front matter Ó 2014 by National Stroke Association http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2013.12.018
future.1 The incidence of stroke increases with age, and the prognosis is more severe in the elderly people than the younger.1,2 More recently, the addition of more promising and new treatment options has not yet pulled down high mortality and morbidity rates. It is expected that the incidence of stroke and the rate of dependent people will increase with the increase in longevity of human life. To decrease the rates of mortality and morbidity, it is of utmost importance to determine the appropriate treatment, care, and rehabilitation programs for each patient in early stage of stroke. For these purposes, a number of models are present in the literature for prediction of mortality and independent survival rates after stoke. It has been aimed to use these models for randomization of patients in clinical studies and provide case–controls in nonrandomized studies, compare patients in different groups, and analyze the results of randomized trials
Journal of Stroke and Cerebrovascular Diseases, Vol. -, No. - (---), 2014: pp 1-7
1
G. C¸ELIK ET AL.
2
and meta-analysis. For this purpose, various models and methods have been developed up-to-date.3-15 Most of these models are not widely accepted and routinely used in practice. In addition, the rates of the accuracy and reliability have generally not reached the satisfying level for these models. Clinical applicability of these models depends on its easy feasibility/applicability, reliability, and its success in predicting the clinical outcome. The aim of the present study was to determine the performance of 2 different multivariate statistical methods (logistic regression analysis [LRA] and multivariate discriminant analysis [MDA]) and artificial neural networks (ANNs) particularly in early (first 10 days) prognosis of ischemic and hemorrhagic strokes, and to compare them with each other. By this way, it was aimed to provide contribution to the practice with the results of a rigorous experiment conducted based on a wide data set.
Materials and Methods Research Data The data of 1725 patients with stroke, admitted to the emergency service in the first 24 hours of stroke and hospitalized in the Neurology Department in Haydarpasa Numune Training and Research Hospital, have been used in the study. The patients diagnosed of transient ischemic attack, sinus thrombosis, subarachnoid hemorrhage, and brain tumor were excluded from the data set after computed tomography or magnetic resonance imaging. The remaining 968 patients were included in the study. Of these patients 293 (30.3%) died and 675 (69.7%) survived. On the basis of the results of our preliminary experiments, it was decided to include equal number of dead and survived patients in a group to improve the training performance of proposed models. Finally, 2 groups, hemorrhagic with 230 patients and ischemic with 340 patients, were obtained and used in the experiments. To increase the reliability of the results, a 5-fold cross-validation technique was used. The number of patients in the entire data set and modeling data set are given in Table 1. The factors affecting the prognosis of hemorrhagic and ischemic strokes have been determined by analyzing similar studies in the literature and were used as inputs in prediction models. Although age, gender, hypertension, diabetes mellitus, smoking, mean blood pressure on admission, and Scandinavian Stroke Scale score16 were used as common inputs for both groups, inputs for ischemic stroke included blood glucose levels, lacunar infarct, nonlacunar infarct, ischemic stroke with undetermined etiology,12 congestive heart failure, coronary heart disease, myocardial infarction, atrial fibrillation, history of cerebrovascular disease, and transient ischemic attack; inputs for the hemorrhagic group included pulse pressure, localization of hemorrhage (putaminal, thalamic,
Table 1. Number of cases in data sets Stroke type
Data set
Hemorrhagic Entire Modeling Ischemic Entire Modeling Total Entire Modeling
Dead Survived All subjects 119 115 174 170 293 285
138 115 537 170 675 285
257 230 711 340 968 570
cerebellar, pontine, lobar, and others [caudate, thalamocapsular, intraventricular]), volume of hemorrhage, ventricular drainage, and the presence or absence of a midline shift. In the determination of the early-stage outcomes of patients with acute stroke, the findings of average 10-day hospitalization were used. Return and normalization of the cerebral function, however, is variable according to the age, risk factor, and site and size of the lesion 10 days is the average duration of hospitalization in our clinic. Output for both groups was survival or death within the first 10 days after the stroke. Tables 2 and 3 show summary statistics for hemorrhagic and ischemic strokes, respectively.
ANN Model A multilayer perceptron (MLP) neural network is a feed-forward neural network model proposed by Rumelhart et al17 and consists of 1 input layer, 1 or more hidden layers, and 1 output layer.18 The number of neurons in input and output layers is determined by the number of input and output vectors used in the data set. The number of neurons in the hidden layer can be determined experimentally or based on experience. A neuron on a layer is connected to all neurons of the adjacent layer with its weights. Usually, the initial values of weights are randomly selected. Outputs of hidden and output layer neurons are produced depending on selected transfer function and weighted neuron inputs. The MLP model uses a supervised neural network and is trained by a gradient descent method to minimize an error function.19 Generalized delta learning rule based on least square can be used to train an MLP network, so these weights are adjusted for a given set of input–output pairs.20 A typical MLP architecture adopted in this study is shown in Figure 1.
Multivariate Statistical Models In this group of prediction techniques, 2 multivariate statistical methods MDA and LRA were used to predict the stroke mortality. MDA is concerned with the classification of distinct sets of observations, and it tries to find the combination of variables that predicts the group to which an observation belongs. The combination of predictor variables is
PREDICTING MORTALITY OF STROKES
3
Table 2. Summary statistics of the modeling data set (hemorrhagic) Variable Gender N Male Female Total Age Mean 6 SD Range Pulse pressure Mean 6 SD Range Mean blood pressure Mean 6 SD Range Stroke severity (Scandinavian Stroke Scale score) Mean 6 SD Range Hematoma location, n (%) Putaminal Thalamic Pontine Cerebellar Lobar Intraventricular Hematoma size Mean 6 SD Range Risk factors, n (%) Hypertension Diabetes Smoking Hematoma status, n (%) Intraventricular hematoma Shift Hematoma in fourth ventricular
called a linear discriminant function, and this function can then be used to classify new observations whose group membership is unknown. The linear discriminant function is as follows: D5B0 1B1 X1 1B2 X2 1/1Bn Xn ; where D is a discriminant score, B0 is an estimated constant, Bn values are the estimated coefficients, and Xn values are the variables. On the basis of this discriminant function score, an observation is classified into the appropriate group. LRA is a form of regression, which is used when the dependent is a dichotomy and the independents are of any type. In LRA models, the dependent variable is usually binary that can take the value 1 with a probability of success P(Zi) or the value 0 with a probability of failure 1 2 P(Zi). The relationship between
Dead
Survived
All subjects
62 53 115
59 56 115
121 109 230
64.5 6 13.3 29-94
63.8 6 11.8 38-92
64.2 6 12.5 29-94
93.7 6 28.5 20-200
85.9 6 26.7 30-160
89.8 6 27.9 20-200
143.8 6 30.1 80-233.3
136.6 6 27.3 70-203.3
140.2 6 28.9 70-233.3
5.9 6 8.9 1-40
26.1 6 16.1 1-58
16.0 6 16.4 1-58
59 (51.3) 20 (17.4) 22 (19.1) 2 (.2) 14 (12.2) 0 (.0)
45 (39.1) 53 (46.1) 2 (.2) 6 (5.2) 6 (5.2) 4 (3.5)
104 (45.2) 73 (31.7) 24 (10.4) 8 (3.5) 20 (8.7) 4 (1.7)
120.3 6 104.7 4-525
31.6 6 35.9 3-196
75.9 6 89.9 3-525
81 (70.4) 18 (15.6) 5 (4.3)
77 (66.9) 13 (11.3) 12 (10.4)
158 (68.7) 31 (13.5) 17 (7.4)
62 (53.9) 34 (29.6) 38 (33.0)
24 (20.9) 5 (4.3) 7 (6.1)
86 (37.4) 39 (16.9) 45 (19.6)
predictor independent variables and binary dependent variables is expressed with the following nonlinear function: PðZi Þ5
eZi 1 5 ; 11euZi 11eu2Zi
where P(Zi) is a cumulative probability function that takes values between 0 and 1. Zi 5b0 1b1 F1 1b2 F2 1/1bm Fm ; where b0 is the constant of the equation and bm values are the coefficients of the predictor variables. LRA aims to correctly predict the group of outcome for individual observations using the most parsimonious model. A model is created that includes all predictor variables that are useful in predicting the response variable.21
G. C¸ELIK ET AL.
4
Table 3. Summary statistics of the modeling data set (ischemic) Variable Gender N Male Female Total Age Mean 6 SD Range Mean blood pressure Mean 6 SD Range Stroke severity (Scandinavian Stroke Scale score) Mean 6 SD Range Risk factors, n (%) Hypertension Diabetes mellitus Congestive heart failure Coronary artery disease Myocardial infarction Atrial fibrillation Smoke Transient ischemic attack Cerebrovascular disease Stroke subtype, n (%) Lacunar Nonlacunar Undetermined etiology Glucose on admission Mean 6 SD Range
Experimental Results Because the effectiveness of neural network techniques can be affected by their parameters, a number of experiments are carried with different levels of their parameters and different activation functions. In this study, for hemorrhagic and ischemic groups, different experiments were implemented by using MATLAB software package (version 7.04 with neural network toolbox, MathWorks, Inc., Natick, MA). The input data sets of both hemorrhagic and ischemic groups were normalized in the range of 21 to 1. A hyperbolic tangent sigmoid transfer function was selected on the hidden layer and a sigmoid transfer function was used on the output layer. Traingdx was used as a training function in MATLAB, which updates weights and bias values according to the gradient descent momentum and an adaptive learning rate. For training process, the maximum training cycles, learning rate, and the momentum coefficient were selected as 1000, .10, .70, respectively. Mean squared error is used as a performance measure of training processes, which can be calculated as follows: n 2 1X Mean squared error5 Yi 2Y^i ; n i51
Dead
Survived
All subjects
72 98 170
73 97 170
145 195 340
72.5 6 12.2 20-97
68.3 6 13.2 29-98
70.4 6 12.9 20-98
121.7 6 28.5 50-200
114.9 6 25.0 46.6-223.3
118.3 6 27.0 46.6-223.3
9.7 6 10.8 1-55
30.7 6 15.9 1-58
20.2 6 17.2 1-58
97 (57.0) 44 (25.9) 49 (28.8) 41 (24.1) 13 (7.6) 55 (32.4) 10 (5.9) 1 (.6) 49 (28.8)
106 (62.4) 33 (19.4) 22 (12.9) 31 (18.2) 9 (5.3) 38 (22.4) 21 (12.4) 11 (6.5) 37 (21.8)
203 (59.7) 77 (22.6) 71 (20.9) 72 (21.2) 22 (6.5) 93 (27.4) 31 (9.1) 12 (3.5) 86 (25.3)
0 (.0) 120 (70.6) 51 (.3)
29 (17.1) 71 (41.8) 70 (41.2)
29 (8.5) 191 (56.2) 121 (35.6)
174.0 6 72.33 23-562
153.5 6 70.9 45-438
163.7 6 72.4 23-562
where n is the number of sample in training or testing data. Yi is the desired output, and Y^i is the output of the neural networks. The architecture of ANN model consisted of 17 input neurons and 1 output neuron. The number of neurons in the hidden layer varied from 10-100 and the best numbers of neurons for both hemorrhagic and ischemic groups were selected. The best numbers of neurons in hidden layers were found to be 90 and 60 for hemorrhagic and ischemic groups, respectively. To make the testing results more reliable, a 5-fold cross-validation technique was used. Each parameter combination was applied to the training and testing data sets, and prediction accuracy, sensitivity, and specificity of the models were evaluated. The results for training and testing data sets are given in Tables 4 and 5, respectively. In the MLP method, the accuracy values were 99.9% in a training data set and 80.9% in a testing data set for the hemorrhagic group. The same values were 97.8% and 75.9% for the ischemic group, respectively. According to these results, it is observed that MLP has produced better training and testing results for the hemorrhagic group. The sensitivity and specificity values gathered in the
PREDICTING MORTALITY OF STROKES
5
Figure 1. The architecture of 3-layered feedforward artificial neural network.
training phase of hemorrhagic and ischemic groups were more than 97%, and this rate shows that the performance of MLP training was high. In MLP testing processes, sensitivity value (86.9%) was higher than specificity value (77.3%) for the hemorrhagic group and its performance was observed to be much lower for survived patients. In MLP testing processes, sensitivity value (73.4%) was lower than specificity value (77.4%) for the ischemic group, and its performance was observed to be much higher for survived patients. MDA and LRA processes were carried out using SPSS 20 software produced by IBM (Armonk, NY). In the LRA method, the classification process was realized by choosing a cutoff probability value as .50. In the MDA method, the training and testing performances were 89.8% and 87.8%, respectively, for the hemorrhagic group, whereas these values were 80.6% and 79.7% for the ischemic group, respectively. For the hemorrhagic group, sensitivity and specificity values were found to be 88.1% and 91.5% in the training phase, whereas it was 87.8% for the testing phase. For the hemorrhagic group, it was observed from the experimental results that the MDA method shows similar performance for dead and survived patients in training and testing phases. For the ischemic group, sensitivity and specificity values were found to be 85.9% and 75.3% in the training phase, whereas these values were 85.3% and 74.1% for the testing phase. For the ischemic group, similar performance was observed in training and testing phases. In this group, it is observed that the success performance for survived patients is lower. MDA tech-
nique provides much higher classification, specificity, and sensitivity performances for the hemorrhagic group than the ischemic group. In the LRA method, meanwhile the training and testing performances for the hemorrhagic group were observed as 89.7% and 86.1%, respectively, whereas these values were 88.9% and 80.9% for the ischemic group, respectively. For the hemorrhagic group, sensitivity and specificity values were found to be 88.1% and 90.4% in the training phase, whereas these values were 86.9% and 85.2% for the testing phase. For the hemorrhagic group, LRA method shows similar performance values for dead and survived patients in training and testing phases. For the ischemic group, sensitivity and specificity values were 85.3% and 78.1% in the training phase, whereas 85.9% and 75.9% for the testing phase. In this group, it was observed that the success performance for survived patients was lower. In an LRA technique, much higher classification, specificity, and sensitivity performances were gathered for the hemorrhagic group than the ischemic group.
Discussion Comparing the performances of the methods, it was observed that MLP has a 100% classification, specificity, and sensitivity values for both hemorrhagic and ischemic groups in the training phase. LRA and MDA methods were observed to show successful results, although these results are close to each other and lower than MLP for both hemorrhagic and ischemic groups in the training
G. C¸ELIK ET AL.
6
Table 4. Performance of the models for training data set Logistic regression
Discriminant analysis
Artificial neural network
Stroke type
Fold
Sns.
Spc.
Acc.
Sns.
Spc.
Acc.
Sns.
Spc.
Acc.
Hemorrhagic
1 2 3 4 5 Average 1 2 3 4 5 Average
88.0 91.3 88.0 88.0 89.1 88.9 84.6 86.0 84.6 86.8 84.6 85.3
89.1 92.4 90.2 88.0 92.4 90.4 80.1 80.1 76.5 77.2 76.5 78.1
88.6 91.8 89.1 88.0 90.8 89.7 82.4 83.1 80.5 82.0 80.5 81.7
87.0 87.0 87.0 92.4 87.0 88.1 83.8 87.5 86.0 86.8 85.3 85.9
90.2 94.6 91.3 90.2 91.3 91.5 77.2 77.9 73.5 72.8 75.0 75.3
88.6 90.8 89.1 91.3 89.1 89.8 80.5 82.7 79.8 79.8 80.1 80.6
100 100 100 100 100 100.0 97.8 97.8 97.8 99.3 99.3 98.4
98.9 100 100 100 100 99.8 97.1 97.1 97.1 97.8 97.1 97.2
99.5 100 100 100 100 99.9 97.4 97.4 97.4 98.5 98.2 97.8
Ischemic
Abbreviations: Acc, accuracy; Sns, sensitivity; Spc, specifity.
methodology and statistics, it is confirmed that the classification performance of our study is similar to other studies more or less and is even higher than some of them when the results that we acquired with the help of 3 different methods were compared with these studies in the literature. It can be seen that the experimental results we concluded were good enough for physician to make decisions in treatment process. The performance in the hemorrhagic group is especially convincing. The reason of lower performance in the ischemic group may not be methodological or related to the analysis method. Taking the equal number of dead and survived patients in training and testing phases, using sufficient number of patients for training and testing for all 3 methods, and constituting 2 different cohorts with hemorrhagic and ischemic stroke patients are important aspects in our study that increase its reliability. In addition, this work differs from present studies in the literature in terms of comparison of the performances of ANNs and multivariate statistical methods on the same data set. As a result,
phase. However, when the performances of the methods in the testing phase were compared, the best classification, specificity, and sensitivity success (87.8%) for the hemorrhagic group was observed with MDA technique. The best classification, specificity, and sensitivity success (sen. 85.9%, spc. 75.9%, and acc. 80.9%) for the ischemic group was gained with LRA technique. In testing processes, the lowest performance was observed with MLP technique, whereas MDA and LRA results were close to each other. When the training phases were taken into account, however, it was concluded that MLP results were more reliable. The classification, specificity, and sensitivity performances that were gained from all the MLP, MDA, and LRA methods were found to be higher for the hemorrhagic group and lower for the ischemic group. There are studies in the literature that focus on predicting the outcome of stroke after 6 hours,10 5 days,8 30 days,3,7 3 months,6,11 100 days,2,9 6 months,7 or 15 years.13,14 Although these studies differ not only from each other but also from our study in terms of
Table 5. Performance of the models for testing data set LR
DA
Artificial neural network
Stroke type
Fold
Sns.
Spc.
Acc.
Sns.
Spc.
Acc.
Sns.
Spc.
Acc.
Hemorrhagic
1 2 3 4 5 Average 1 2 3 4 5 Average
87.0 95.6 82.6 82.6 86.9 86.9 88.2 76.5 88.2 88.2 88.2 85.9
87.0 73.9 86.9 100.0 78.3 85.2 64.7 70.6 94.1 79.4 70.6 75.9
87.0 84.8 84.8 91.3 82.6 86.1 76.5 73.5 91.2 83.8 79.4 80.9
91.3 95.7 82.6 78.3 91.3 87.8 88.2 76.5 88.2 88.2 85.3 85.3
87.0 82.6 86.9 100.0 82.6 87.8 64.7 64.7 91.2 79.4 70.6 74.1
89.1 89.1 84.8 89.1 87.0 87.8 76.5 70.6 89.7 83.8 77.9 79.7
88.2 78.3 90.0 100.0 78.3 86.9 70.7 71.4 89.7 70.6 74.4 73.4
72.4 78.3 80.8 76.7 78.3 77.3 81.5 72.8 79.5 70.6 82.8 77.4
78.3 78.3 84.8 84.8 78.3 80.9 75.0 72.1 83.8 70.6 77.9 75.9
Ischemic
PREDICTING MORTALITY OF STROKES
we conclude that different statistical methods used on different groups are more promising. We observed that the group constituted the number of patients used, model type used, the number of data sets created in this modeling, and using the data that has an effect on prognosis in the system affects the performance of the models.
Conclusions This study analyzes the performance of 2 different multivariate statistical methods (MDA and LRA) and the MLP ANN in predicting the prognosis of the hemorrhagic and ischemic stroke patients especially in the early period (first 10 days) and compares them. According to the experimental results, the qualifications used in the study, the data set, and the classifier affect the prediction performance directly. The training and testing performances of the methods used show variation for ischemic and hemorrhagic groups. MLP provides higher success in the training phase, whereas other statistical methods provide better results is the testing phase. In the hemorrhagic group, higher prediction performance is obtained in training and testing phases. Finally, there are a limited number of treatment alternatives for stroke, and there is a limited time to use the selected alternative. Therefore, the proposed models with high sensitivity and specificity of ischemic and hemorrhagic strokes can be useful to select the best treatment alternative in an individual patient, and so will increase the success rate of performed treatments.
References 1. Kammersgaard LP, Jørgensen HS, Reith J, et al, Copenhagen Stroke Study. Short- and long-term prognosis for very old stroke patients. The Copenhagen Stroke Study. Age Ageing 2004;33:149-154. 2. Weimar C, Ziegler A, K€ onig IR, et al. Predicting functional outcome and survival after acute ischemic stroke. J Neurol 2002;249:888-895. 3. Tuhrim S, Horowitz DR, Sacher M, et al. Validation and comparison of models predicting survival following intracerebral hemorrhage. Crit Care Med 1995;23:950-954. 4. Edwards DF, Hollingsworth H, Zazulia AR, et al. Artificial neural networks improve the prediction of mortality in intracerebral hemorrhage. Neurology 1999;53:351-357. 5. Inouye M. Predicting outcomes of patients in Japan after first acute stroke using a simple model. Am J Phys Med Rehabil 2001;80:645-649. 6. Johnston KC, Connors AF Jr, Wagner DP, et al. A predictive risk model for outcomes of ischemic stroke. Stroke 2000;31:448-455.
7 7. Counsell C, Dennis M, McDowall M, et al. Predicting outcome after acute and subacute stroke: development and validation of new prognostic models. Stroke 2002; 33:1041-1047. 8. Counsel C, Dennis MS, Lewis S, et al, FOOD Trial Collaboration. Performance of a statistical model to predict stroke outcome in the context of a large, simple, randomized, controlled trial of feeding. Stroke 2003;34:127-133. 9. German Stroke Study Collaboration. Predicting outcome after acute ischemic stroke: an external validation of prognostic models. Neurology 2004;62:581-585. 10. Lewis SC, Sandercock PA, Dennis MS, SCOPE (Stroke Complications and Outcomes Prediction Engine) Collaborations, IST. Predicting outcome in hyper-acute stroke: validation of a prognostic model in the Third International Stroke Trial (IST3). J Neurol Neurosurg Psychiatry 2008;79:397-400. Epub 2007 Aug 31. 11. K€ onig IR, Ziegler A, Bluhmki E, et al, Virtual International Stroke Trials Archive (VISTA) Investigators. Predicting long-term outcome after acute ischemic stroke: a simple index works in patients from controlled clinical trials. Stroke 2008;39:1821-1826. 12. Saposnik G, Kapral MK, Liu Y, et al, Investigators of the Registry of the Canadian Stroke Network, Stroke Outcomes Research Canada (SORCan) Working Group. IScore: a risk score to predict death early after hospitalization for an acute ischemic stroke. Circulation 2011;123:739-749. 13. Li WJ, Gao ZY, He Y, et al. Application and performance of two stroke outcome prediction models in a Chinese population. PM R 2012;4:123-128. 14. Hankey GJ, Jamrozik K, Broadhurst RJ, et al. Five-year survival after first-ever stroke and related prognostic factors in the Perth Community Stroke Study. Stroke 2000; 31:2080-2086. 15. Weir NU, Counsell CE, McDowall M, et al. Reliability of the variables in a new set of models that predict outcome after stroke. J Neurol Neurosurg Psychiatry 2003;74:447-451. 16. Jørgensen HS, Reith J, Nakayama H, et al. What determines good recovery in patients with the most severe strokes? The Copenhagen Stroke Study. Stroke 1999;30:2008-2012. 17. Rumelhart DE, Hinton GE, Williams RJ. Learning representation by back-propagating errors. Nature 1986; 323:533-536. 18. Fu L. Neural networks in computer intelligence. USA: McGraw-Hill 1994:31. 19. Bose NK, Liang P. Multilayer networks. Neural network fundamentals with graphs, algorithms and applications. USA: McGraw-Hill 1996:155-212. € E. Yapay sinir a gı modeli c¸ok katmanlıalgılayıcı. 20. Oztemel _ Yapay Sinir A gları. Istanbul, T€ urkiye: Papatya Yayıncılık 2003:75-114. 21. Boyacıo glu MA, Kara Y, Baykan OK. Predicting bank financial failures using neural networks, support vector machines and multivariate statistical methods: a comparative analysis in the sample of savings deposit insurance fund (SDIF) transferred banks in Turkey. Expert Syst Appl 2009;36:3355-3366.
