Internationally comparable diagnosis-specific survival probabilities for calculation of the ICD-10-based Injury Severity Score.

ORIGINAL ARTICLE

Internationally comparable diagnosis-specific survival probabilities for calculation of the ICD-10Ybased Injury Severity Score Rolf Gedeborg, MD, PhD, Margaret Warner, PhD, Li-Hui Chen, PhD, Pauline Gulliver, PhD, Colin Cryer, PhD, Yvonne Robitaille, PhD, Robert Bauer, PhD, Clotilde Ubeda, MD, MSc, Jens Lauritsen, MD, PhD, James Harrison, MBBS, MPH, Geoff Henley, and John Langley, PhD The International Statistical Classification of Diseases, 10th Revision (ICD-10)Ybased Injury Severity Score (ICISS) performs well but requires diagnosis-specific survival probabilities (DSPs), which are empirically derived, for its calculation. The objective was to examine if DSPs based on data pooled from several countries could increase accuracy, precision, utility, and international comparability of DSPs and ICISS. METHODS: Australia, Argentina, Austria, Canada, Denmark, New Zealand, and Sweden provided ICD-10Ycoded injury hospital discharge data, including in-hospital mortality status. Data from the seven countries were pooled using four different methods to create an international collaborative effort ICISS (ICE-ICISS). The ability of the ICISS to predict mortality using the country-specific DSPs and the pooled DSPs was estimated and compared. RESULTS: The pooled DSPs were based on a total of 3,966,550 observations of injury diagnoses from the seven countries. The proportion of injury diagnoses having at least 100 discharges to calculate the DSP varied from 12% to 48% in the country-specific data set and was 66% in the pooled data set. When compared with using a country’s own DSPs for ICISS calculation, the pooled DSPs resulted in somewhat reduced discrimination in predicting mortality (difference in c statistic varied from 0.006 to 0.04). Calibration was generally good when the predicted mortality risk was less than 20%. When Danish and Swedish data were used, ICISS was combined with age and sex in a logistic regression model to predict in-hospital mortality. Including age and sex improved both discrimination and calibration substantially, and the differences from using country-specific or pooled DSPs were minor. CONCLUSION: Pooling data from seven countries generated empirically derived DSPs. These pooled DSPs facilitate international comparisons and enables the use of ICISS in all settings where ICD-10 hospital discharge diagnoses are available. The modest reduction in performance of the ICE-ICISS compared with the country-specific scores is unlikely to outweigh the benefit of internationally comparable Injury Severity Scores possible with pooled data. (J Trauma Acute Care Surg. 2014;76: 358Y365. Copyright * 2014 by Lippincott Williams & Wilkins) LEVEL OF EVIDENCE: Prognostic and epidemiological study, level III. KEY WORDS: Trauma; injury; trauma severity indices; population surveillance; hospital mortality. BACKGROUND:

M

ost studies on injury benefit from or require an accurate injury severity measure. The International Statistical Classification of Diseases, 10th Revision (ICD-10)Ybased Injury Severity Score (ICISS) performs well for the prediction of hospital mortality and needs only ICD-coded hospital

Submitted: April 10, 2013, Revised: August 9, 2013, Accepted: August 12, 2013, Published online: January 6, 2014. From the Department of Surgical Sciences, Anesthesiology and Intensive Care (R.G.), and Uppsala Clinical Research Center (R.G.), Uppsala University, Uppsala, Sweden; National Center for Health Statistics (M.W., L.-H.C.), Centers for Disease Control and Prevention, Hyattsville, Maryland; Injury Prevention Research Unit (P.G., C.C., J.L.), University of Otago, Dunedin, New Zealand; Institut national de sante´ publique du Que´bec (INSPQ) (Y.R.), Montreál, Que´bec, Canada; Austrian Road Safety Board (R.B.), Research and Knowledge Management, Vienna, Austria; Programa de Prevencioń de Lesiones por Causas Externas (C.U.), Instituto Nacional de Epidemiologıá, Dr ‘‘J.H.Jara,’’ ANLIS Malbran, Argentina; Accident Analysis Group (J.L.), Odense University Hospital and IST/Biostatistics University of Southern Denmark, Odense C, Denmark; Research Centre for Injury Studies (J.H., G.H.), Flinders University, Adelaide, Australia. Supplemental digital content is available for this article. Direct URL citations appear in the printed text, and links to the digital files are provided in the HTML text of this article on the journal’s Web site (www.jtrauma.com). Address for reprints: Rolf Gedeborg, MD, PhD, Uppsala Clinical Research Center, Uppsala University. UCR/Scheele, Dag Hammarskjo¨lds va¨g 50A, 3tr. Science Park, SE-751 85 Uppsala, Sweden; email: [email protected]. DOI: 10.1097/TA.0b013e3182a9cd31

358

discharge diagnoses to be calculated.1,2 ICISS is calculated from empirically estimated diagnosis-specific survival probabilities (DSPs) for individual injury ICD codes (International Statistical Classification of Diseases and Health Related Problems).3 The DSP (previously improperly termed survival risk ratio) is the proportion of patients with a specific injury code who survived to discharge after admission to a hospital or trauma center. The calculation of DSPs must be made from a large reference data set, such as all hospital discharges for a country. This hampers the utility of ICISS. Even when large databases are used, many injury diagnoses are infrequent, and the DSPs calculated for these injury diagnoses will be determined with poor precision. The extent to which the DSPs are generalizable is not yet known. Preliminary comparisons of country-specific DSPs from New Zealand, Australia, Denmark, and Sweden conducted by the International Collaborative Effort on Injury Statistics (ICE)4 investigator group, indicated that the majority of DSPs are of comparable magnitude across the countries (unpublished data).5 However, DSPs for some diagnoses vary widely. The objectives of this study were to generate DSPs from ICD-10Ycoded hospital injury diagnoses, based on pooling of J Trauma Acute Care Surg Volume 76, Number 2

Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

J Trauma Acute Care Surg Volume 76, Number 2

Gedeborg et al.

data from several countries and to determine if these pooled DSPs could increase accuracy, precision, utility, and international comparability of ICE-ICISS at least among ‘‘highincome’’ countries.

PATIENTS AND METHODS Data Seven countriesVAustralia, Argentina, Austria, Canada, Denmark, New Zealand, and SwedenVprovided hospital discharge data to calculate pooled injury DSPs. The study population was defined as hospitalized patients with a principal hospital discharge diagnosis in the range ICD-10 S00-T79 (excluding T78).6 No restrictions were imposed on years included or age of patients. Survivors were defined as patients discharged from hospital alive. DSPs were

calculated based on in-hospital deaths. When possible, patients admitted and discharged alive on the same date and readmissions for the same injury were excluded. Duplicate injury ICD codes in a hospital discharge record were excluded from calculation of DSPs. The resulting individual country data profile is summarized in Table 1. The original version of ICD-10 or a clinical modification that did not affect coding up to the fourth position in the injury chapter was used in four countries (Table 1). ICD-10-CA was used in Que´bec, Canada, and ICD-10-AM was used in Australia and New Zealand and increased the detail to some injury diagnoses. For ICD-10-AM, this mainly concerns codes for burns and functional level of spinal cord injury. Occurrences of codes not found in the original ICD-10 version were found to be infrequent (as indicated in Supplemental Digital Content, Appendix Table 1, http://links.lww.com/TA/A340).

TABLE 1. Characteristics of the Datasets Supplied by the Seven Countries Argentina Data source

Australia

Austria

Public hospital discharge Public and private hospital discharge Hospital discharge data from data from DEIS (Statistics data from the National Hospital the national statistical office Department of Health) Morbidity Database (NHMD) (Statistics Austria) compiled by the Australian Institute of Health and Welfare (AIHW). 38% of all hospitals in Approximately 2Y5% of records for Complete coverage of all the country private hospitals were not included. inpatient care Coverage information is given in the annual report Australian Hospital Statistics, available at www.aihw.gov.au

Estimated coverage

Period covered by the data included in the study 2006 Geographical region Entire country Population in covered region during the last year 38,970,261 of the covered period ICD-10 version Original ICD-10 Modifications of the original ICD-10 affecting the None injury chapter to the fourth position of the code Maximum no. discharge diagnoses allowed 2 in registry Total no. observations* 111,104 Distribution of no. observations per diagnosis 0 (not found) 228 (20%) 1Y19 591 (51%) 20Y49 141 (12%) 50Y99 78 (7%) 100Y199 53 (5%) 200Y499 41 (4%) 500Y 36 (3%) No. unique injury diagnoses per hospital record (proportion of all records) 1 NA 2 NA 3 NA Q4 NA No. injury ICD-10 diagnoses covered 940

July 1, 1999, to June 30, 2008 Entire country 21,498,540 ICD-10-AM, versions 1Y5 Increased level of detail that may affect fourth position of code (see text) Variable over time and between regions but at least 20 1,149,708

2002Y2008 Entire country 8,210,000 ICD-10 BMSG 2001 None 1 (at the level of this data source; 9 at the hospital level) 1,069,769

132 (11%) 286 (24%) 151 (13%) 100 (9%) 104 (9%) 135 (12%) 260 (22%)

95 (8%) 429 (37%) 148 (13%) 106 (9%) 94 (8%) 99 (8%) 197(17%)

NA NA NA NA 1,036

NA NA NA NA 1,073

*This does not equal the number of patients since a proportion of the patients have multiple injury diagnoses, thus generating multiple observations per patient. NA, information not available.

* 2014 Lippincott Williams & Wilkins


359


Gedeborg et al.

Three-digit ICD-10 codes (e.g., T71), which appeared in different forms (e.g., T71, T71X, T71x, T719), were standardized (e.g., T719). Most countries provided aggregated data, but three countriesVDenmark, New Zealand, and SwedenVprovided patient-level data that could be used to assess the predictive ability of the pooled DSPs. The study was approved by the regional Human Ethics Committee in Uppsala, Sweden.

APPROACHES TO POOLING DATA TO CALCULATE POOLED DSPS Four different methods for pooling the data from the seven countries were used as follows. (a) Summation (sum) of numerators (number of deaths for the injury diagnosis) and denominators (number of discharges for the injury diagnosis) across the country-specific data sets for each injury diagnosis. This was considered the simplest option. However, if one country’s data set was large relative to the others, it could

Canada

Denmark

dominate the result, and the pooled DSPs would not be representative across countries. We therefore also calculated (b) simple arithmetic means (mean) of DSPs from the seven countries, thereby providing equal weight to each country irrespective of the underlying number of observations. The concern with this approach was that if a DSP for a particular country was based on very few observations, it would have a marked impact on the final pooled estimate. A possible solution to this problem was to use (c) trimmed means (trimmean) of countryspecific DSPs. Trimmed means were calculated as the arithmetic mean of country-specific DSPs after the removal of the lowest and the highest country-specific DSPs. This method is more robust when outliers are present but only applicable when there are data from at least three countries, allowing the detection of outliers. Because Methods a to c had limitations, we also tried a (d) combined approach (combo), using the summation method as described earlier if the number of observations for ICD-10 injury diagnosis for any country was less than 50, otherwise the trimmed mean method. The cutoff (50) was selected based on inspection of a plot of deviation of

New Zealand

Sweden

Que´bec inpatient hospital discharge database from Ministry of Health and Social Services

Odense University Hospital inpatient hospital discharge database

Public hospital discharge data from Ministry of Health

National Patient Registry maintained by the Swedish Board of Health and Welfare

Complete coverage of all injury inpatient care in the province

Complete coverage of all inpatient care since approximately 1975

It has been estimated that 998% of acute hospital inpatient injury admissions were publicly funded in New Zealand.17 2002Y2008 Entire country

Complete coverage of all inpatient care since 1987

April 1, 2006, to March 31, 2009 2002Y2007 Province of Que´bec, one of the 10th Province of Fyn (10% of Denmark) provinces of Canada. 7,500,000 500,000 ICD-10-CA Original ICD-10 Increased level of detail that may affect None fourth position of code (see text) One principal and up to 26 others 7 diagnosis 189,237 33,434 Distribution of no. observations per diagnosis 181 (15%) 612 (52%) 485 (42%) 389 (33%) 154 (13%) 55 (5%) 105 (9%) 42 (4%) 78 (7%) 29 (2%) 74 (6%) 28 (2%) 91 (8%) 13 (1%) No. unique injury diagnoses per hospital record (proportion of all records) 66.0% 46.4% 16.8% 24.6% 6.5% 14.1% 10.6% 14.9% 987 556

360

1998Y2004 Entire country

4,268,900 ICD-10-AM Increased level of detail that may affect fourth position of code (see text) 99 external cause, diagnosis and operation codes 485,722

9,011,392 ICD-10-SE None

180 (15%) 365 (31%) 132 (11%) 103 (9%) 87 (7%) 111 (10%) 190 (16%)

53 (5%) 351 (30%) 183 (16%) 130 (11%) 116 (10%) 124 (11%) 211 (18%)

32.7% 22.9% 14.9% 29.5% 988

57.4% 24.0% 9.0% 9.6% 1,115

8 927,576




Gedeborg et al.

DSPs from the diagnosis-specific median in relation to the underlying number of observations.

Validation of ICE-ICISS Based on Pooled DSPs Patient-level data sets were used to assess the predictive abilities of the ICE-ICISSs based on the pooled DSPs. DSPs were calculated using the four pooling methods described earlier and excluding one of the three countries (Denmark, New Zealand, or Sweden) at a time. ICE-ICISS were then calculated on the patient-level data for the excluded country based on pooled DSPs from the other six countries. This process was repeated for each of these three countries. For comparison, countryspecific ICISSs were also calculated for the three countries using DSPs calculated from the country’s own data. Two countries, Denmark and Sweden, provided data on age and sex.

Statistical Analysis Variability between the seven country-specific DSPs was investigated for each injury diagnosis by calculating the difference from the median DSP and the range of DSPs. Association between two continuous variables was assessed using the Pearson correlation coefficient (r). The predictive ability of ICE-ICISS calculated in the validation process was assessed by entering the logit of the ICEICISS in a logistic regression model for hospital mortality.7 Discrimination (model concordance) was estimated by the c statistic (c index, concordance index), which equals 1 for a perfectly discriminating model.8 Calibration plots using nonparametric smoothers were used to assess calibration, where a curve with intercept 0 and slope 1 would represent perfect calibration.9 Nagelkerke’s R2 was used as a measure of overall fit.10 When comparing different logistic regression models using the same data and predicting the same outcome, a higher pseudo R2 can help indicate which model better predicts the outcome. The scaled Brier score is a model performance measure dependent on the distance between observed and predicted outcome. The score can vary between 0% and 100%, and the higher the score, the better the model performance, reflecting aspects of both calibration and discrimination.9,11 The statistical packages SAS version 9.2 (SAS Institute Inc., Cary, NC) and R version 2.11.1 (R Foundation for Statistical Computing, Vienna, Austria) were used for data management and statistical analyses.

the pooled data set are listed in Supplemental Digital Content 1 (Appendix Table 1, http://links.lww.com/TA/A340). Differences in the country-specific DSP from the median DSP were most pronounced when the DSP was based on few observations and did not seem to be related to a particular country (Fig. 2). However, the variability increased as mortality risk increased (Fig. 3). Country-specific DSPs for some injury diagnoses varied substantially (Table 2). The diagnoses with the largest degree of variability had lower estimated survival probability compared with those with the smallest variability. There was some degree of correlation (r = j0.414) between the range of country-specific DSPs and the mortality risk associated with the injury diagnosis, while there was no apparent correlation (r = 0.065) with number of observations (Supplemental Digital Content 2, Fig. 4, http://links.lww.com/TA/A341). Pooled DSPs were generated using four different methods and were validated (Table 3). Using simple summation of numerators and denominators to generate the DSPs seemed to be the best option for pooling the Swedish and Danish data. For the New Zealand data, the trimmed mean or combined approach seemed to perform slightly better, but the differences in discrimination and calibration were small. For ICISS calculation, countries’ own DSPs performed better than the pooled multinational DSPs. The difference in discrimination varied between the three countries used for validation from 0.006 to 0.044 (Table 3). For all three countries, calibration was good in the part of the population where the predicted mortality risk was less than 20% (Supplemental Digital Content 3Y5, Figs. 5Y7, http://links.lww.com/TA/A342,

RESULTS The number of ICD-10 injury codes represented by at least one observation was 1,168 overall, ranging from 556 to 1,115 among the seven countries. Diagnoses with few observations were present in all countries but were more common in the smaller data sets (Fig. 1). Across data sets, the proportion of diagnoses having at least 20 observations ranged from 30% to 72%. Combining data from all seven countries generated a data set containing 3,966,550 observations. The proportion of injury diagnoses having at least 20 observations increased to 88%. The 150 ICD-10 codes with less than 20 observations in

Figure 1. The proportion of country-specific DSPs (y-axis) as related to the minimum number of occurrences of each ICD-10 code. The cumulative proportion of diagnoses in the pooled data set is indicated by the black curve. Dashed lines indicate proportions of DSPs in the pooled data set based on at least 20 and 100 observations, respectively.



361


Gedeborg et al.

mortality risk in Swedish data (Supplemental Digital Content 4, Fig. 6, http://links.lww.com/TA/A343). These results should be interpreted with caution since there are much fewer data for high mortality risk patients. In the Danish data, the low number of patients with very high mortality risk did not allow a stable estimate of calibration (Supplemental Digital Content 5, Fig. 7, http://links.lww.com/TA/A344). The predictive ability of the ICE-ICISS varied among the three countries used for validation. The Danish data set also seemed to contain more severely injured patients, as indicated by the distribution of predicted injury severity (rug on x-axes in Supplemental Digital Content 5 and 6, Figs. 7 and 8, http://links.lww.com/TA/A344, http://links.lww.com/TA/A345). When age and sex were combined with ICE-ICISS, both discrimination and calibration improved substantially (Table 3). The differences in c statistic when using country-specific or pooled DSPs (ICE-ICISS) were minor, and calibration of these models was generally good for the part of the population with higher predicted mortality (Supplemental Digital Content 6, Fig. 8, http://links.lww.com/TA/A345). Estimated calibration at very high predicted mortality was imprecise for these models. Figure 2. The difference in country-specific DSPs from the median, in relation to the number of underlying observations (discharges) for that code in that country. Each country has a separate color code.

http://links.lww.com/TA/A343, http://links.lww.com/TA/A344). For patients with higher mortality risk, ICISS based on the pooled DPSs tended to underestimate the mortality risk in New Zealand data (Supplemental Digital Content 3, Fig. 5, http://links.lww.com/TA/A342), while it overestimated the

DISCUSSION In this study initiated by ICE,4 data were obtained from seven countries to generate DSPs based on more than 3.9 million hospitalized injury cases. Most DSPs were reasonably consistent across countries. The large pooled set allowed DSPs to be estimated for uncommon diagnoses. When modeled with age and sex, predictive ability differed little between pooled DSPs and country-specific DSPs.

Figure 3. The difference in country-specific DSPs from the median, in relation to the number of underlying observations and mortality risk associated with each diagnosis as estimated by the median of country-specific DSPs. Each country has a separate color code (see legend in Fig. 2). 362




Gedeborg et al.

TABLE 2. Variability Between Countries of DSPs* ICD-10 Code

Diagnosis

Range of Mean of Country-Specific DSPs Country-Specific DSPs

Diagnoses with the lowest degree of variability between countries S807 Multiple superficial injuries of the lower leg S799 Unspecified injury of the hip and thigh S211 Open wound of front wall of thorax T141 Open wound of unspecified body region S122 Fracture of other specified cervical vertebra S829 Fracture of the lower leg, part unspecified T149 Injury, unspecified T589 Toxic effect of carbon monoxide S212 Open wound of the back wall of the thorax S141 Other and unspecified injuries of the cervical spinal cord Diagnoses with the highest degree of variability between countries S271 Traumatic hemothorax S368 Injury of other intra-abdominal organs S027 Multiple fractures involving the skull and facial bones T689 Hypothermia S066 Traumatic subarachnoid hemorrhage S361 Injury of the liver or gallbladder S064 Epidural hemorrhage T175 Foreign body in the bronchus S272 Traumatic hemopneumothorax S065 Traumatic subdural hemorrhage

No. Countries Providing Data

Mean No. Country-Specific Observations

0.033 0.035 0.04 0.041 0.044 0.049 0.05 0.052 0.056 0.064

0.974 0.979 0.963 0.976 0.954 0.97 0.968 0.97 0.976 0.917

6 6 6 6 6 6 6 6 6 6

91 1,741 265 1,703 1,686 1,296 704 531 160 450

0.263 0.264 0.294

0.94 0.925 0.893

7 6 6

737 581 580

0.294 0.379 0.386 0.391 0.408 0.411 0.539

0.852 0.816 0.932 0.92 0.971 0.944 0.826

7 7 7 7 6 7 7

460 1,961 1,097 936 297 967 4,835

*The range of country-specific DSPs calculated for diagnoses where at least six countries provided data and after exclusion of diagnoses with very low mortality risk (mean DSP 9 0.98). The first 10 diagnoses are those with the smallest variability as determined by the range of the DSPs, while the last 10 diagnoses are those with the highest degree of variability.

Injury severity estimation is a cornerstone of injury epidemiology. A previous study cross-validated the predictive ability of ICISS between two countries and found an overall good performance of ICISS.2 We are not aware of any other major attempt to compile data from several countries to generate DSPs that are comparable between countries.

Predictive Ability of the ICE-ICISS The most important performance measure for the DSPs is the ability to generate an ICISS that accurately predicts mortality risk. We assessed this by using a country’s own data to estimate DSPs and calculate ICISS scores. We then compared the predictive ability of this country-specific score to the ICE-ICISS based on DSPs calculated using four different methods of pooling of data that excluded the DSPs from the country being validated. These comparisons indicated that there were only minor differences in prediction owing to the different methods of pooling. Pooling by simply adding numerators and denominators from country-specific DSPs generated the most predictive ICE-ICISS for two of the three countries. We recommend this simple method for pooling data because it uses all available information when estimating the DSPs. The differences seen in predictive ability may reflect variations in the size of the data sets, country-specific coding practices, and different distribution of injury severity in the data sets. ICE-ICISS had less discriminative ability in the Danish

data set but was better calibrated, compared with the Swedish data set, when combined with age and sex. The Danish data set was notably smaller and seemed to contain more severely injured patients when compared with the other two countries. The difference in the distribution of injury severity between the data sets could reflect actual differences in the patterns of injury but could also reflect differences in hospital admission criteria or administrative procedures between the countries.

Potential Considerations When Using the ICE-ICISS The ICISS has previously proven to perform well in comparison with other ISSs, but there are still issues with ICISS that need to be considered.1,2,12Y14 While empirical estimation of the severity score may be seen as a strength, it also presents a problem when estimating survival probabilities for uncommon diagnoses. The pooling of data in this study partly addresses this problem, but there remain a number of diagnoses where survival probability may be imprecise owing to too few observations. However, this lack of precision of DSPs may not be a major problem for population-based research. At the population level, these uncommon diagnoses will have little impact on the validity of the method. When studying small selected subsets of injured patients, such as uncommon poisonings, this may, however, be an issue. Therefore, we recommend that the DSPs be used for population-level research, and only applied



363


Gedeborg et al.

TABLE 3. Validation of Predictive Ability of ICISS on Danish, New Zealand, and Swedish Data Predictors Denmark Danish DSPs Pooled (excluding Denmark) DSPs (sum) Pooled (excluding Denmark) DSPs (mean) Pooled (excluding Denmark) DSPs (trimmean) Pooled (excluding Denmark) DSPs (combo) Age + sex + Danish DSPs Age + sex + pooled (excluding Denmark) DSPs (sum) New Zealand New Zealand DSPs Pooled (excluding New Zealand) DSPs (sum) Pooled (excluding New Zealand) DSPs (mean) Pooled (excluding New Zealand) DSPs (trimmean) Pooled (excluding New Zealand) DSPs (combo) Sweden Swedish DSPs Pooled (excluding Swedish) DSPs (sum) Pooled (excluding Swedish) DSPs (mean) Pooled (excluding Swedish) DSPs (trimmean) Pooled (excluding Swedish) DSPs (combo) Age + sex + Swedish DSPs Age + sex + pooled (excluding Swedish) DSPs (sum)

No. Hospital Admissions

No. Hospital Deaths

23,449

2,529

264,348

707,968

c Statistic

Brier score Scaled, %

Nagelkerke’s R2

0.725 0.681 0.635 0.640 0.676 0.822 0.816

6.06 3.33 2.11 2.23 3.35 15.30 14.40

0.1311 0.0756 0.0525 0.0539 0.0739 0.2613 0.2490

0.876 0.868 0.856 0.870 0.869

6.53 5.65 5.19 6.06 6.00

0.2263 0.2088 0.1940 0.2150 0.2150

0.829 0.815 0.809 0.810 0.800 0.877 0.871

4.02 3.60 3.25 3.28 3.34 6.73 6.06

0.1678 0.1489 0.1423 0.1384 0.1348 0.2385 0.2232

3,156

11,427

Four different methods for pooling of national data sets to generate multinational DSPs are compared with each country’s own DSPs. For Danish and Swedish data, comparisons are also made with models containing age and sex.

with caution for individual or small-group analyses. The number of observations on which the DSP is based is provided for reference when applying the DSPs (see Table, Supplemental Digital Content 7, http://links.lww.com/TA/A346). A major purpose of the ICE-ICISS is to facilitate reliable comparisons of injury severity estimates between countries. If, however, the purpose is to handle injury severity as a confounder, internal DSPs will usually be a better alternative if the study population is large enough to generate them. For studies based on smaller data sets and when country-specific DSPs are not at hand, the ICE-ICISS is available for injury severity estimation.

Performance of ICE-ICISS in Relation to Other Studies In a previous study comparing the performance of ICISS using data from Australian and New Zealand hospitalizations, estimated c statistics using internally derived DSPs (then called survival risk ratios) were 0.85 and 0.86, respectively.2 The corresponding c statistic found in our study for New Zealand data was 0.88. Looking at calibration, Stephenson et al.2 found good calibration in the subpopulation with an estimated mortality of less than 30% and, greater than that, a tendency for overestimation of the mortality risk. This compares well with the calibration curves found in the present study. An apparent aberration is the calibration curves for New Zealand data, where ICISS generally seem to underestimate mortality in patients with high mortality risk. It should 364

be noted that in patients with lower mortality risk, the opposite tendency seem to exist. It is also important to recognize that considerably fewer data are available to estimate the calibration in the high mortality risk range of the curves and the notable effect of adding age and sex. The findings for New Zealand data in the previous study by Stephenson et al. were based on a substantially smaller data set collected during a phase of transition from ICD-9 to ICD-10.2 Adding age and sex to the logistic regression models substantially improved the ability to accurately predict mortality. The difference in predictive ability from using countryspecific or pooled DSPs was minor in these models. It seems that the performance of the ICE-ICISS is reasonably accurate compared with country-specific DSPs.

Limitations of This Study A possible limitation of the present study is that the countries used for validation are all high-income countries. The performance of the score needs to be tested also in a low- and middle-income settings. Major differences in injury epidemiology and types of trauma encountered in different countries must also be considered. Whether differences in injury epidemiology between countries affects the performance of the ICE-ICISS needs to be addressed in further studies. Another limitation is that prehospital deaths were not included in the calculation of the pooled DSPs because these data were not available for six of the seven countries. The addition of out-of-hospital mortality data has been shown to * 2014 Lippincott Williams & Wilkins



Gedeborg et al.

improve predictive ability of ICISS,13,15 and studies on injury should include prehospital deaths to accurately reflect the burden of injury.16 A related issue is the limitation of follow-up to hospital discharge. This is in line with the original definition of the ICISS and a practical limitation in many countries and settings. It should, however, be recognized that transfers (e.g., of head injured patients and elderly patients to nursing homes) may affect in-hospital mortality in a way that is not directly related to injury severity. Differences in the ability to exclude patients admitted and discharged alive on the same date or patients readmitted for a previous injury could also contribute to variability between country-specific DSPs. Also differing between the data sets are the number of diagnosis codes allowed per patient and the actual number of diagnoses recorded per patient. Numbers of codes can reflect differences in the population of injured patients, the level of trauma care provided, or coding practices. A related issue is that we included all relevant codes for injury, even those for very minor or superficial injuries. The combined coding of such injuries with other more severe injuries may reflect a truly higher injury severity, that is, the combined coding of a fracture and a superficial injury might suggest an open fracture. Any general exclusion of injury codes would require a detailed analysis well beyond the scope of this study.

CONCLUSION Pooling of data from seven countries generated empirically derived diagnosis-specific survival estimates that in themselves are informative of severity of individual injuries and allow calculation of the ICE-ICISS. The diagnosis-specific survival estimates and the ICE-ICISS facilitate international comparisons and enable injury severity estimation in all settings where ICD-10 hospital discharge diagnoses are available. The pooled data (ICE-ICISS) DSPs are provided as Supplemental Digital Content (http://links.lww.com/TA/A346). Please cite this article when they are used. AUTHORSHIP J.L. conceived the study. All authors contributed to the assembly of the national data sets and transformation of the data to the format used in the analyses. R.G. performed the analyses and drafted the manuscript. All authors contributed to critical revision of the manuscript. ACKNOWLEDGMENT We thank the International Collaborative Effort on Injury Statistics (ICE), an important initiative from the National Center for Health Statistics, US Centers for Disease Control, for facilitating this international research collaboration. We also acknowledge the assistance of Mathieu Gagne´ for putting the Que´bec (Canada) data set together. Sections of this article are based on data made available by the Australian Institute of Health

and Welfare (AIHW). The authors are responsible for the use of the data in this article.

DISCLOSURE R.G. is also employed at the Swedish Medical Products Agency. The findings and conclusions expressed in this article do not necessarily represent the views of the Medical Products Agency.

REFERENCES 1. Clark DE, Winchell RJ. Risk adjustment for injured patients using administrative data. J Trauma. 2004;57(1):130Y140; discussion 140. 2. Stephenson S, Henley G, Harrison JE, Langley JD. Diagnosis based injury severity scaling: investigation of a method using Australian and New Zealand hospitalisations. Inj Prev. 2004;10(6):379Y383. 3. Osler T, Rutledge R, Deis J, Bedrick E. ICISS: an International Classification of Disease-9 based Injury Severity Score. J Trauma. 1996;41(3): 380Y386; discussion 386Y388. 4. Fingerhut LAE. Proceedings of the International Collaborative Effort on Injury Statistics. US Department of Health and Human Services; 2003. (PHS) 2003-1026. Available at: http://www.cdc.gov/nchs/data/ ice/iceproceedings.pdf 5. National Center for Health Statistics. International Collaborative Effort (ICE) on Injury Statistics: Overview and implications. 2007. Available at: http://www.cdc.gov/nchs/about/otheract/ice/matrix10impl.htm. Date of access: Jan 21, 2013. 6. World Health Organization. International Statistical Classification of Diseases and Health Related Problems ICD-10. 2008 Edition. Geneva, Switzerland: World Health Organization; 2005. 7. Moore L, Lavoie A, Bergeron E, Emond M. Modeling probability-based Injury Severity Scores in logistic regression models: the logit transformation should be used. J Trauma. 2007;62(3):601Y605. 8. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29Y36. 9. Harrell FE Jr. Regression Modeling Strategies. New York, NY: Springer; 2001. 10. Nagelkerke NJD. A note on the general definition of the coefficient of determination. Biometrika. 1991;78:691Y692. 11. Brier GW. Verification of forecasts expressed in terms of probability. Mon Weather Rev. 1950;75:1Y3. 12. Leaphart CL, Graham D, Pieper P, Celso BG, Tepas JJ 3rd. Surgical quality improvement: a simplified method to apply national standards to pediatric trauma care. J Pediatr Surg. 2009;44(1):156Y159. 13. Davie G, Cryer C, Langley J. Improving the predictive ability of the ICDbased Injury Severity Score. Inj Prev. 2008;14(4):250Y255. 14. Hannan EL, Farrell LS. Predicting trauma inpatient mortality in an administrative database: an investigation of survival risk ratios using New York data. J Trauma. 2007;62(4):964Y968. 15. Gedeborg R, Thiblin I, Byberg L, Wernroth L, Michaelsson K. The impact of clinically undiagnosed injuries on survival estimates. Crit Care Med. 2009;37(2):449Y455. 16. Gedeborg R, Chen LH, Thiblin I, et al. Prehospital injury deathsV strengthening the case for prevention: nationwide cohort study. J Trauma Acute Care Surg. 2012;Mar;72(3):765Y72. 17. Langley J, Stephenson S, Thorpe C, Davie G, et al. Accuracy of injury coding under ICD-9 for New Zealand public hospital discharges. Injury Prevention. 2006;12:58Y61.



365

Calculation of the probability of survival for trauma patients based on trauma score and the injury severity score model in fatemi hospital in ardabil.

Sepsis severity score: an internationally derived scoring system from the surviving sepsis campaign database*.

Thoracolumbar injury classification and severity score.

Utility of the injury severity score: a confirmation.

Survival time estimation using Injury Severity Score (ISS) in homicide cases.

Validating the Thoracolumbar Injury Classification and Severity Score.

Letter to the Editor: Thoracolumbar Injury Classification and Severity Score.

Consensus on Severity for Ocular Emergency: The BAsic SEverity Score for Common OculaR Emergencies [BaSe SCOrE].

Comparison of the Ability to Predict Mortality between the Injury Severity Score and the New Injury Severity Score: A Meta-Analysis.

Predictive value of the Status Epilepticus Severity Score (STESS) and its components for long-term survival.

A nomogram for calculation of the Revised Baux Score.

Calculation of the underprivileged area score.

A New Injury Severity Score for Predicting the Length of Hospital Stay in Multiple Trauma Patients.

Developing a burn injury severity score (BISS): adding age and total body surface area burned to the injury severity score (ISS) improves mortality concordance.

Application of abbreviated injury scale and injury severity score in fatal cases with abdominopelvic injuries.

Thoracolumbar Injury Classification and Injury Severity Score System: A Literature Review of Its Safety.

Exact calculation of probabilities of false positives and false negatives for low background counting.

Severity score for hereditary hemorrhagic telangiectasia.

A comparison of the Injury Severity Score and the Trauma Mortality Prediction Model.

A severity score for acute necrotizing encephalopathy.

A genetic score for the prediction of beta-thalassemia severity.

Characteristics of Patients Injured in Road Traffic Accidents According to the New Injury Severity Score.

Application of the Mandible Injury Severity Score to Pediatric Mandibular Fractures.

Validation of the revised injury severity classification score in patients with moderate-to-severe traumatic brain injury.