Injury, Int. J. Care Injured 46 (2015) 86–93
Contents lists available at ScienceDirect
Injury journal homepage: www.elsevier.com/locate/injury
Validation of the revised injury severity classification score in patients with moderate-to-severe traumatic brain injury Rahul Raj a,*, Tuomas Brinck b, Markus B. Skrifvars c, Riku Kivisaari a, Jari Siironen a, Rolf Lefering d, Lauri Handolin b a
Department of Neurosurgery, To¨o¨lo¨ Hospital, Helsinki University Hospital, Topeliuksenkatu 5, PB 266, FI-00029 HUS, Helsinki, Finland Department of Orthopedics and Traumatology, To¨o¨lo¨ Hospital, Helsinki University Hospital, Topeliuksenkatu 5, PB 266, FI-00029 HUS, Helsinki, Finland c Department of Intensive Care, Meilahti Hospital, Helsinki University Hospital, Haartmaninkatu 4, PB 340, FI-00029 HUS, Helsinki, Finland d Institute for Research in Operative Medicine (IFOM), Faculty of Health, University of Witten/Herdecke, Cologne Merheim Medical Centre, Ostmerheimer Straße 200, Cologne 51109, Germany b
A R T I C L E I N F O
A B S T R A C T
Article history: Accepted 8 August 2014
Introduction: By analysing risk-adjusted mortality ratios, weaknesses in the process of care might be identified. Traumatic brain injury (TBI) is the main cause of death in trauma, and thus it is crucial that trauma prediction models are valid for TBI patients. Accordingly, we assessed the validity of the RISC score in TBI patients by internal and external validation analyses. Methods: Patients with moderate-to-severe TBI admitted to the TraumaRegister DGU1 (TR-DGU) and the trauma registry of Helsinki University Hospital (TR-THEL) in 2006–2011 were included in this retrospective open cohort study. Definition of moderate-to-severe TBI was head abbreviated injury scale of 3 or higher. Subgroup analysis for patients with isolated and polytrauma TBI was performed. The performance of the RISC score was evaluated by assessing its discrimination (area under the curve, AUC) and calibration (Hosmer–Lemeshow [H–L] test). Results: Among the 9106 and 809 patients with moderate-to-severe TBI admitted to TR-DGU and TRTHEL, unadjusted mortality was 26% and 23%, respectively. Internal and external validation of the RISC score showed good discrimination (TR-DGU AUC 0.89, 95% confidence interval [CI] 0.88–0.90 and TRTHEL AUC 0.84, 95% CI 0.81–0.87), but poor calibration (p < 0.001) in patients with moderate-to-severe TBI. Subgroup analysis found the discrimination only to be modest in isolated TBI (AUC 0.76) and calibration to be particularly poor in polytrauma TBI (TR-DGU H–L = 4356, p < 0.001; TR-THEL H–L 112, p < 0.001). Conclusion: The RISC score was found to be of limited predictive value in patients with moderate-tosevere TBI. A new general trauma scoring system that includes TBI specific prognostic factors is warranted. ß 2014 Elsevier Ltd. All rights reserved.
Keywords: Traumatic brain injury Prognostic model External validation Revised injury severity classification score TR-DGU TR-THEL Trauma register
Abbreviations: AIS, abbreviated injury scale; APACHE, acute physiology and chronic health evaluation; AUC, area under the curve; CI, confidence internal; DGU, German ˆ -statistic; IQR, interquartile range; iso-sTBI, isolated severe traumatic brain Society of Trauma Surgery or Deutsche Gesellschaft fu¨r Unfallchirurgie; H–L, Hosmer–Lemeshow C injury; ISS, injury severity score; mTBI, mild traumatic brain injury; NISS, new injury severity score; noTBI, no traumatic brain injury; poly-sTBI, polytrauma severe traumatic brain injury; RISC, revised injury severity classification; RTS, revised trauma score; SD, standard deviation; Sektion NIS, Committee on Emergency Medicine, Intensive Care and Trauma Management of the German Trauma Society; SMR, standardized mortality ratio; sTBI, severe traumatic brain injury; TBI, traumatic brain injury; TR-DGU, TraumaRegister DGU1; TRISS, trauma and injury severity score; TR-THEL, Trauma Register of Helsinki University Hospital. * Corresponding author at: Topeliuksenkatu 5, PB-266, 00029 HUS, Finland. Tel.: +358 443191190. E-mail addresses: [email protected] (R. Raj), tuomas.brinck@hus.fi (T. Brinck), markus.skrifvars@hus.fi (M.B. Skrifvars), riku.kivisaari@hus.fi (R. Kivisaari), jari.siironen@hus.fi (J. Siironen), [email protected] (R. Lefering), lauri.handolin@hus.fi (L. Handolin). http://dx.doi.org/10.1016/j.injury.2014.08.026 0020–1383/ß 2014 Elsevier Ltd. All rights reserved.
R. Raj et al. / Injury, Int. J. Care Injured 46 (2015) 86–93
Introduction The failure of many randomized controlled trials in traumatic brain injury (TBI) research has lead to a re-evaluation of the research approach to improve TBI outcomes. Comparative effectiveness research, that is analysis of differences in outcome and relate them to differences in treatment practises in real clinical scenarios with broad patient populations, is becoming increasingly important in TBI research [1]. By analysing and comparing riskadjusted mortality ratios (defined as the ratio of observed to predicted mortality) between trauma registries weaknesses of current process of care might be identified. This is a crucial step for any improvement to care processes which may ultimately improve patient outcomes [2,3]. To calculate accurate risk-adjusted mortality ratios, a prediction model with good statistical performance is essential. The heterogeneous spectrum of TBI has made it challenging to create a prediction model accurately predicting risks for both non-TBI trauma patients and trauma patients with TBI [4]. However, as TBI is the leading cause of death in trauma, it is crucial that any prediction model used in trauma research displays good performance in patients with TBI [5]. Since its introduction in 1987, the trauma - injury severity score (TRISS) has been the gold standard for the performance evaluation of trauma centers and for identifying areas for critical review [6]. However, the TRISS have been criticized, and in 2009, the TraumaRegister DGU1 (TR-DGU) from the German Trauma Society (Deutsche Gesellschaft fu¨r Unfallchirurgie or DGU) released its own trauma score (the revised injury severity classification or RISC) based on data from 1993 to 2000 [7,8]. The RISC score is routinely used in TR-DGU, which is one of the largest trauma registries in Europe. Several prediction models specifically targeted for TBI patients have been developed, although few enjoy as widespread use as the RISC score in TR-DGU [9–11]. The aim of this study was to assess the validity of the RISC score for case-mix analysis in patients with TBI by conducting internal and external validation analyses. Methods Trauma registries The TraumaRegister DGU1 (TR-DGU) of the German Trauma Society (Deutsche Gesellschaft fu¨r Unfallchirurgie, DGU) was founded in 1993. The aim of this multicentre database is an anonymous and standardized documentation of severely injured patients. Currently, over 30,000 cases from more than 600 hospitals are annually entered into the database. The participating hospitals are primarily located in Germany (90%), but an increasing number of hospitals from other countries also contribute data. Data are collected prospectively from the prehospital phase, emergency room and initial surgery, intensive care unit and discharge. Inclusion criteria is admission to hospital via emergency room (ER) with subsequent ICU care or reach the hospital with vital signs and die before admission to ICU. The infrastructure for documentation, data management and data analysis is provided by the Academy for Trauma Surgery (Akademie der Unfallchirurgie GmbH). As a compulsory tool for quality assessment, no informed consent is necessary for data collection. Scientific data analysis is approved according to a peer review procedure established by the Committee on Emergency Medicine, Intensive Care and Trauma Management of the German Trauma Society (Sektion NIS). The present study is in line with the guidelines of the TraumaRegister DGU1 and registered as TR-DGU Project ID: 2012-053 II. The Trauma Register of Helsinki University Hospital (TR-THEL) is a local trauma database from To¨o¨lo¨ Hospital (Trauma Unit of
87
Helsinki University Hospital), which is one of the largest trauma centers in Scandinavia, serving nearly 2 million inhabitants. The TR-THEL was founded in 2006 as a benchmarking project aimed at improving the quality of trauma care in the region. The ethics committee of Helsinki University Hospital waived the need for informed consent for entering patients into the database (approval nr. 19.10.2009/30). During 2006–2011, approximately 400–450 patients with major trauma (defined as having an injury severity score [ISS] > 15) were annually admitted into the database. Data are collected in accordance with the standard documentation from the TR-DGU guidelines and the Utstein template [12]. The TR-THEL board approved of the present study and gave us access to the database. Study population The TR-DGU and TR-THEL were screened for primary transferred patients with major trauma (defined as ISS > 15) entered into one or the other registry during 2006 to 2011. From TR-DGU, only German level-1 trauma centers annually treating more than 50 major trauma cases were included (n = 85). In addition, as children under the age of 16 and patients with penetrating nonhead injuries are treated at a different location in the Helsinki University Hospital complex, these patients were excluded for better inter-register comparison. Definition of moderate-to-severe TBI was an abbreviated injury scale [AIS]-head of 3 or higher [13]. Patients were further categorized into polytrauma TBI (at least one other body part AIS 2) and isolated TBI (defined as no other body part AIS 2). For comparison reasons, we also defined patients with no TBI (AIS-head = 0) and patients with mild TBI (AIS-head 1-2). The revised injury severity classification score The RISC score was developed in 2009, based upon severely injured patients admitted to TR-DGU in 1993–2000 [8]. The RISC score includes 11 variables: age at injury, new injury severity score (NISS), AIS-head, AIS-extremities, GCS (first pre-clinical assessment), partial thromboplastin time, base excess, preclinical cardiac arrest with reanimation/defibrillation, preclinical systolic blood pressure
Contents lists available at ScienceDirect
Injury journal homepage: www.elsevier.com/locate/injury
Validation of the revised injury severity classification score in patients with moderate-to-severe traumatic brain injury Rahul Raj a,*, Tuomas Brinck b, Markus B. Skrifvars c, Riku Kivisaari a, Jari Siironen a, Rolf Lefering d, Lauri Handolin b a
Department of Neurosurgery, To¨o¨lo¨ Hospital, Helsinki University Hospital, Topeliuksenkatu 5, PB 266, FI-00029 HUS, Helsinki, Finland Department of Orthopedics and Traumatology, To¨o¨lo¨ Hospital, Helsinki University Hospital, Topeliuksenkatu 5, PB 266, FI-00029 HUS, Helsinki, Finland c Department of Intensive Care, Meilahti Hospital, Helsinki University Hospital, Haartmaninkatu 4, PB 340, FI-00029 HUS, Helsinki, Finland d Institute for Research in Operative Medicine (IFOM), Faculty of Health, University of Witten/Herdecke, Cologne Merheim Medical Centre, Ostmerheimer Straße 200, Cologne 51109, Germany b
A R T I C L E I N F O
A B S T R A C T
Article history: Accepted 8 August 2014
Introduction: By analysing risk-adjusted mortality ratios, weaknesses in the process of care might be identified. Traumatic brain injury (TBI) is the main cause of death in trauma, and thus it is crucial that trauma prediction models are valid for TBI patients. Accordingly, we assessed the validity of the RISC score in TBI patients by internal and external validation analyses. Methods: Patients with moderate-to-severe TBI admitted to the TraumaRegister DGU1 (TR-DGU) and the trauma registry of Helsinki University Hospital (TR-THEL) in 2006–2011 were included in this retrospective open cohort study. Definition of moderate-to-severe TBI was head abbreviated injury scale of 3 or higher. Subgroup analysis for patients with isolated and polytrauma TBI was performed. The performance of the RISC score was evaluated by assessing its discrimination (area under the curve, AUC) and calibration (Hosmer–Lemeshow [H–L] test). Results: Among the 9106 and 809 patients with moderate-to-severe TBI admitted to TR-DGU and TRTHEL, unadjusted mortality was 26% and 23%, respectively. Internal and external validation of the RISC score showed good discrimination (TR-DGU AUC 0.89, 95% confidence interval [CI] 0.88–0.90 and TRTHEL AUC 0.84, 95% CI 0.81–0.87), but poor calibration (p < 0.001) in patients with moderate-to-severe TBI. Subgroup analysis found the discrimination only to be modest in isolated TBI (AUC 0.76) and calibration to be particularly poor in polytrauma TBI (TR-DGU H–L = 4356, p < 0.001; TR-THEL H–L 112, p < 0.001). Conclusion: The RISC score was found to be of limited predictive value in patients with moderate-tosevere TBI. A new general trauma scoring system that includes TBI specific prognostic factors is warranted. ß 2014 Elsevier Ltd. All rights reserved.
Keywords: Traumatic brain injury Prognostic model External validation Revised injury severity classification score TR-DGU TR-THEL Trauma register
Abbreviations: AIS, abbreviated injury scale; APACHE, acute physiology and chronic health evaluation; AUC, area under the curve; CI, confidence internal; DGU, German ˆ -statistic; IQR, interquartile range; iso-sTBI, isolated severe traumatic brain Society of Trauma Surgery or Deutsche Gesellschaft fu¨r Unfallchirurgie; H–L, Hosmer–Lemeshow C injury; ISS, injury severity score; mTBI, mild traumatic brain injury; NISS, new injury severity score; noTBI, no traumatic brain injury; poly-sTBI, polytrauma severe traumatic brain injury; RISC, revised injury severity classification; RTS, revised trauma score; SD, standard deviation; Sektion NIS, Committee on Emergency Medicine, Intensive Care and Trauma Management of the German Trauma Society; SMR, standardized mortality ratio; sTBI, severe traumatic brain injury; TBI, traumatic brain injury; TR-DGU, TraumaRegister DGU1; TRISS, trauma and injury severity score; TR-THEL, Trauma Register of Helsinki University Hospital. * Corresponding author at: Topeliuksenkatu 5, PB-266, 00029 HUS, Finland. Tel.: +358 443191190. E-mail addresses: [email protected] (R. Raj), tuomas.brinck@hus.fi (T. Brinck), markus.skrifvars@hus.fi (M.B. Skrifvars), riku.kivisaari@hus.fi (R. Kivisaari), jari.siironen@hus.fi (J. Siironen), [email protected] (R. Lefering), lauri.handolin@hus.fi (L. Handolin). http://dx.doi.org/10.1016/j.injury.2014.08.026 0020–1383/ß 2014 Elsevier Ltd. All rights reserved.
R. Raj et al. / Injury, Int. J. Care Injured 46 (2015) 86–93
Introduction The failure of many randomized controlled trials in traumatic brain injury (TBI) research has lead to a re-evaluation of the research approach to improve TBI outcomes. Comparative effectiveness research, that is analysis of differences in outcome and relate them to differences in treatment practises in real clinical scenarios with broad patient populations, is becoming increasingly important in TBI research [1]. By analysing and comparing riskadjusted mortality ratios (defined as the ratio of observed to predicted mortality) between trauma registries weaknesses of current process of care might be identified. This is a crucial step for any improvement to care processes which may ultimately improve patient outcomes [2,3]. To calculate accurate risk-adjusted mortality ratios, a prediction model with good statistical performance is essential. The heterogeneous spectrum of TBI has made it challenging to create a prediction model accurately predicting risks for both non-TBI trauma patients and trauma patients with TBI [4]. However, as TBI is the leading cause of death in trauma, it is crucial that any prediction model used in trauma research displays good performance in patients with TBI [5]. Since its introduction in 1987, the trauma - injury severity score (TRISS) has been the gold standard for the performance evaluation of trauma centers and for identifying areas for critical review [6]. However, the TRISS have been criticized, and in 2009, the TraumaRegister DGU1 (TR-DGU) from the German Trauma Society (Deutsche Gesellschaft fu¨r Unfallchirurgie or DGU) released its own trauma score (the revised injury severity classification or RISC) based on data from 1993 to 2000 [7,8]. The RISC score is routinely used in TR-DGU, which is one of the largest trauma registries in Europe. Several prediction models specifically targeted for TBI patients have been developed, although few enjoy as widespread use as the RISC score in TR-DGU [9–11]. The aim of this study was to assess the validity of the RISC score for case-mix analysis in patients with TBI by conducting internal and external validation analyses. Methods Trauma registries The TraumaRegister DGU1 (TR-DGU) of the German Trauma Society (Deutsche Gesellschaft fu¨r Unfallchirurgie, DGU) was founded in 1993. The aim of this multicentre database is an anonymous and standardized documentation of severely injured patients. Currently, over 30,000 cases from more than 600 hospitals are annually entered into the database. The participating hospitals are primarily located in Germany (90%), but an increasing number of hospitals from other countries also contribute data. Data are collected prospectively from the prehospital phase, emergency room and initial surgery, intensive care unit and discharge. Inclusion criteria is admission to hospital via emergency room (ER) with subsequent ICU care or reach the hospital with vital signs and die before admission to ICU. The infrastructure for documentation, data management and data analysis is provided by the Academy for Trauma Surgery (Akademie der Unfallchirurgie GmbH). As a compulsory tool for quality assessment, no informed consent is necessary for data collection. Scientific data analysis is approved according to a peer review procedure established by the Committee on Emergency Medicine, Intensive Care and Trauma Management of the German Trauma Society (Sektion NIS). The present study is in line with the guidelines of the TraumaRegister DGU1 and registered as TR-DGU Project ID: 2012-053 II. The Trauma Register of Helsinki University Hospital (TR-THEL) is a local trauma database from To¨o¨lo¨ Hospital (Trauma Unit of
87
Helsinki University Hospital), which is one of the largest trauma centers in Scandinavia, serving nearly 2 million inhabitants. The TR-THEL was founded in 2006 as a benchmarking project aimed at improving the quality of trauma care in the region. The ethics committee of Helsinki University Hospital waived the need for informed consent for entering patients into the database (approval nr. 19.10.2009/30). During 2006–2011, approximately 400–450 patients with major trauma (defined as having an injury severity score [ISS] > 15) were annually admitted into the database. Data are collected in accordance with the standard documentation from the TR-DGU guidelines and the Utstein template [12]. The TR-THEL board approved of the present study and gave us access to the database. Study population The TR-DGU and TR-THEL were screened for primary transferred patients with major trauma (defined as ISS > 15) entered into one or the other registry during 2006 to 2011. From TR-DGU, only German level-1 trauma centers annually treating more than 50 major trauma cases were included (n = 85). In addition, as children under the age of 16 and patients with penetrating nonhead injuries are treated at a different location in the Helsinki University Hospital complex, these patients were excluded for better inter-register comparison. Definition of moderate-to-severe TBI was an abbreviated injury scale [AIS]-head of 3 or higher [13]. Patients were further categorized into polytrauma TBI (at least one other body part AIS 2) and isolated TBI (defined as no other body part AIS 2). For comparison reasons, we also defined patients with no TBI (AIS-head = 0) and patients with mild TBI (AIS-head 1-2). The revised injury severity classification score The RISC score was developed in 2009, based upon severely injured patients admitted to TR-DGU in 1993–2000 [8]. The RISC score includes 11 variables: age at injury, new injury severity score (NISS), AIS-head, AIS-extremities, GCS (first pre-clinical assessment), partial thromboplastin time, base excess, preclinical cardiac arrest with reanimation/defibrillation, preclinical systolic blood pressure
