549
New Technologies, Diagnostic Tools and Drugs
Composite risk scores and composite endpoints in the risk prediction of outcomes in anticoagulated patients with atrial fibrillation The Loire Valley Atrial Fibrillation Project Amitava Banerjee1; Laurent Fauchier2; Anne Bernard-Brunet2; Nicolas Clementy2; Gregory Y. H. Lip1 1University
of Birmingham Centre for Cardiovascular Sciences, City Hospital, Birmingham, UK; 2Service de Cardiologie, Pôle Coeur Thorax Vasculaire, Centre Hospitalier, Universitaire Trousseau et Faculté de Médecine, Université François Rabelais, Tours, France
Summary Several validated risk stratification schemes for prediction of ischaemic stroke (IS)/thromboembolism (TE) and major bleeding are available for patients with non-valvular atrial fibrillation (NVAF). On the basis for multiple common risk factors for IS/TE and bleeding, it has been suggested that composite risk prediction scores may be more practical and user-friendly than separate scores for bleeding and IS/TE. In a long-term prospective hospital registry of anticoagulated patients with newly diagnosed AF, we compared the predictive value of existing risk prediction scores as well as composite risk scores, and also compared these risk scoring systems using composite endpoints. Endpoint 1 was the simple composite of IS and major bleeds. Endpoint 2 was based on a composite of IS plus intracerebral haemorrhage (ICH). Endpoint 3 was based on weighted coefficients for IS/TE and ICH. Endpoint 4 was a composite of stroke, cardiovascular death, TE and major bleeding. The incremental predictive value of these scores over CHADS2 (as reference) for composite endpoints was assessed
Correspondence to: Prof. G. Y. H. Lip University of Birmingham Centre for Cardiovascular Sciences City Hospital, Birmingham B18 7QH, UK E-mail: [email protected]
using c-statistic, net reclassification improvement (NRI) and integrated discrimination improvement (IDI). Of 8,962 eligible individuals, 3,607 (40.2%) had NVAF and were on OAC at baseline. There were no statistically significant differences between the c-statistics of the various risk scores, compared with the CHADS2 score, regardless of the endpoint. For the various risk scores and various endpoints, NRI and IDI did not show significant improvement (≥1%), compared with the CHADS2 score. In conclusion, composite risk scores did not significantly improve risk prediction of endpoints in patients with NVAF, regardless of how endpoints were defined. This would support individualised prediction of IS/TE and bleeding separately using different separate risk prediction tools, and not the use of composite scores or endpoints for everyday ‘real world’ clinical practice, to guide decisions on thromboprophylaxis.
Keywords Cardiology, stroke prevention, thrombosis
Received: December 19, 2013 Accepted after minor revision: January 19, 2014 Prepublished online: January 23, 2014 doi:10.1160/TH13-12-1033 Thromb Haemost 2014; 111: 549–556
Note: The review process for this paper was fully handled by Christian Weber, Editor in Chief.
Introduction The epidemiologic burden of atrial fibrillation (AF) is increasing and the risk of ischaemic stroke (IS)/thromboembolism (TE) is also projected to increase due to the rising burden of comorbidities in patients with AF (1–6). Oral anticoagulation (OAC), whether with vitamin K antagonists (VKA) or novel oral anticoagulants, is a proven therapy for the reduction of risk of IS/TE in the setting of AF (7–10). Validated risk prediction tools are available for IS/TE in patients with non-valvular AF (NVAF) in order to guide the initiation of OAC (11–13). In clinical practice, the bleeding risk of OAC use is weighed against the risk of IS/TE prior to starting OAC, and validated tools are also available for risk prediction of © Schattauer 2014
bleeding in NVAF (13–15). The use of these risk prediction tools for IS/TE and bleeding are recommended in guidelines (16). Given that several risk factors are common to both IS/TE and bleeding in patients with NVAF, a composite risk score for IS/TE and bleeding may offer practical utility by simplifying the process of risk prediction (17, 18). However, such a risk score has not been previously evaluated in a “real-world”, prospective cohort of NVAF patients. The net clinical benefit (NCB) concept was originally suggested by Singer et al. as a means of quantifying the balance between risk of IS/TE and bleeding in individual patients with NVAF (19). Different composite endpoints and different methods of measuring NCB have also been used in studies of AF (20, 21). The role of risk prediction tools in prediction of composite outcomes and NCB has not been previously investigated. Thrombosis and Haemostasis 111.3/2014
Downloaded from www.thrombosis-online.com on 2014-06-15 | ID: 1000465826 | IP: 138.37.211.113 For personal or educational use only. No other uses without permission. All rights reserved.
550
Banerjee et al. Composite risk scores and composite endpoints in AF
In a long-term, prospective study of NVAF, we evaluated the incremental predictive value of different risk scoring systems, including composite risk scores, in the prediction of a range of composite outcomes, to establish whether composite risk prediction tools may offer clinical utility for ‘real world’ patients with NVAF.
Methods Study population The methods of the Loire Valley Atrial Fibrillation Project have been previously reported (22). At the Centre Hospitalier Régional et Universitaire in Tours (France), all patients diagnosed with NVAF or atrial flutter by the Cardiology Department between 2000–2010 were identified, excluding patients with valvular AF. The institution includes four hospitals covering all medical and surgical specialties; the only public institution in an area of around 4,000 km2, serving approximately 400,000 inhabitants. Patients were followed from the first record of AF after January 1, 2000 (i.e. index date) up to the latest data collection at the time of study (December 2010). Treatment at discharge was obtained by screening hospitalisation reports, and information on comorbidities was obtained from the computerised coding system. Patients were included in this study if they were receiving OAC at time of first diagnosis of AF. The study was approved by the Review Board of the Pole Coeur Thorax Vaisseaux from the Trousseau University Hospital in 2010 (December 7, 2010).
Models of risk prediction For each patient, the CHADS2 (11) and CHA2DS2-VASc (12) scores were calculated. The CHADS2 score was the sum of points obtained after adding one point for congestive heart failure, hypertension, age ≥75, and diabetes, and two points for previous stroke or TE (11). The CHA2DS2-VASc score was the sum of points after adding one point for congestive heart failure, hypertension, diabetes, vascular disease (including history of coronary, cerebrovascular or peripheral vascular disease), age 65–74, and female gender, and two points for previous stroke or TE and age ≥75 (12). According to the two risk scores, patients with a score of 0 on either schema were considered as ‘low risk’, 1 as ‘intermediate risk’, and ≥2 as ‘high risk’ of stroke and TE. The HAS-BLED (Hypertension, Abnormal renal and/or liver function, Stroke, Bleeding history or predisposition, Labile International Normalised Ratio [INR], Elderly [> 65 years], Drugs [antiplatelet drugs or NSAIDS]/alcohol excess concomitantly) score is a validated scoring system for bleeding risk stratification in AF patients (14). Patients with HAS-BLED score of 0–2 were deemed to have ‘lowmoderate’ bleeding risk and those with HAS-BLED score of ≥3 were classified as ‘high’ bleeding risk. The risk factors for HAS-BLED were combined with CHADS2 and CHA2DS2-VASc, respectively, to give a composite score. Finally, a multivariable score was derived by performing a multivariable regression analysis to identify predictors of the study outcomes. The factors included in the score were: previous HF, age
>75, age >65, diabetes mellitus, stroke, vascular disease, liver-renal impairment, previous bleeding and labile INR.
Outcomes During follow-up, information on outcomes of TE (including peripheral artery embolism and transient ischaemic attack [TIA]), stroke (ischaemic or haemorrhagic), bleeding, and all-cause mortality were recorded. Major bleeding was defined as bleeding with a reduction in the haemoglobin level of at least 2 g/l, or with transfusion of at least 1 unit of blood, or symptomatic bleeding in a critical area or organ (e.g. intracranial, intraspinal, intraocular, retroperitoneal, intra-articular or pericardial, or intramuscular with compartment syndrome) or bleeding that causes death. All bleeding and thromboembolic events were identified with the diagnosis coded in a subsequent hospitalisation during follow-up.
Models of composite endpoints For each individual, four different models were used to quantify composite outcomes. • Endpoint 1 was the crude sum of IS and major bleeding events (including ICH). • Endpoint 2 was based on the net clinical benefit model originally proposed by Singer et al. (19): Net Clinical Benefit (NCB) = (TE rate off warfarin – TE rate on warfarin) – 1.5x (ICH rate on warfarin-ICH rate off warfarin). The NCB was derived and defined for whole populations from population event rates, but in our analysis, NCB was used in individuals incorporating numbers of events. However, rates of events off warfarin cannot be calculated in our population (underlined sections). Therefore, the actual formula used for Model 2 was: Endpoint 2 = IS/TE +1.5(ICH). Endpoint 2 represents the sum of events in an individual but with weighting based on the Singer NCB model. • Endpoint 3 was based on a model for NCB with weighted coefficients for different categories of ischaemic and haemorrhagic events was first proposed by Connolly et al. (21): NCB= [IR(IS)plac +w1IR(HS)plac- w2IR(SDH)plac + w3IR(ECH)plac + w4IR(MI)plac] – [IR(IS)treat+ w1IR(HS)treat + w2IR(SDH)treat + w3IR(ECH)treat + w4IR(MI)treat] (where IR=incidence rate; IS=ischaemic stroke; HS=haemorrhagic stroke; SDH=subdural haemorrhage; ECH=extracranial haemorrhage; MI=myocardial infarction; plac=placebo; treat=treatment). Again, the underlined sections were not available for our patient population. However, the weightings for the other categories of event were derived from our patient population using the methodology outlined by Connolly et al. (21) to give the following formula for Endpoint 3: Endpoint 3 = 1.701(IS/TE)+2.855(ICH). Therefore, Endpoint 3 represents the sum of events in an individual with weightings based on the Connolly model. • Endpoint 4 was based a composite of all vascular events based on the definition by Hohnloser et al. (20), which included stroke, MI, cardiovascular death, pulmonary embolism systemic embolism or major bleeding. There are several event types unavailable for our patient population, which led to the
Thrombosis and Haemostasis 111.3/2014 Downloaded from www.thrombosis-online.com on 2014-06-15 | ID: 1000465826 | IP: 138.37.211.113 For personal or educational use only. No other uses without permission. All rights reserved.
© Schattauer 2014
Banerjee et al. Composite risk scores and composite endpoints in AF
modified formula for Endpoint 4: Endpoint 4 = stroke +systemic embolism + pulmonary embolism + all-cause death + major haemorrhage
Statistical analysis The study population was stratified on the basis of presence of ≥1 of each of the composite outcomes (Endpoints 1–4). Baseline characteristics were determined separately by presence or absence of each composite outcome, and differences were investigated using chi-squared test for categorical covariates. The Kruskal-Wallis test was used for continuous covariates since the distributions of variables do not have to be normal and the variances do not have to be equal. The predictive value of each scoring system was evaluated for each of the composite outcomes in comparison to the CHADS2 score (as the reference) using c-statistics, Net Reclassification Improvement (NRI) and Integrated Discrimination Improvement (IDI), as previously described (22, 23). C-statistics were compared using the non-parametric method proposed by DeLong et al. (24). The NRI focuses on reclassification tables constructed separately for patients with and without event, and quantifies the correct movement within a particular scoring system in comparison to the CHADS2 score (i.e. increased risk score for events counts positively, and increased risk score for non-events counts negatively). The IDI does not require risk groups and focuses on differences between integrated sensitivity and specificity for different risk prediction models in comparison to the CHADS2 score (23). The analyses aimed to compare the predictive value of different composite scoring systems for different composite outcomes, particularly the incremental predictive value in comparison to the CHADS2 score. C-statistics, NRI and IDI were given to three decimal places. A two-sided p-value < 0.05 was considered statistically significant. All analyses were performed with SPSS statistical software version 21.0 (IBM, Armonk, NY, USA) and NRI and IDI calculations were performed using R statistical software version 2.12.2 (University of Auckland, New Zealand).
Results Of 8,962 eligible individuals, 3607 (40.2%) had NVAF and were on OAC at baseline. Of these, 455 patients had ≥1 ischaemic stroke or major bleeding event (Endpoint 1), whilst 272 patients had a score ≥1 as defined by Endpoint 2 [IS/TE +1.5(ICH)] and for Endpoint 3 [1.701(IS/TE)+2.855(ICH)], 703 patients had a score ≥1 for Endpoint 4 during the study period. After a mean follow-up of 1.65 years (SD 2.44), there were 175 IS, 272 IS/TE, 60 ICH and 327 major bleeding events. ▶ Table 1 illustrates the baseline characteristics of all included individuals by presence or absence of composite event. For each event type, patients with events were older, had higher prevalence of comorbidities and higher risk prediction scores (▶ Table 1). Endpoint 2 and Endpoint 3 identified the same patient population. © Schattauer 2014
Comparisons of models
▶Table 2 shows the comparisons of the different risk prediction
models for the different endpoints. For Endpoint 1, the 14-factor HAS-BLED/CHA2DS2-VASc model compared to the CHADS2 model demonstrated a numerically higher c-statistic, but this was not statistically different (DeLong test p=0.894). Overall, the numerically highest c-statistic was found with the multivariate risk score and Endpoint 4 (0.668, 0.646–0.690;p=0.062), although again this did not reach statistical significance. For all four endpoints, there were no statistically significant differences between the c-statistics for the risk prediction scores and the CHADS2 score, as compared by the DeLong test. For Endpoint 1, there was no statistically significant net reclassification with any scores when compared with CHADS2. For Endpoint 2 and Endpoint 3, there was a positive NRI for the combined CHADS2/HAS-BLED score (0.047, 0.005–0.088; p=0.028), but no other scores, when compared with CHADS2. For Endpoint 4, there was actually a negative NRI for CHA2DS2-VASc, CHADS2/HASBLED, CHA2DS2-VASc/HAS-BLED (all with p
New Technologies, Diagnostic Tools and Drugs
Composite risk scores and composite endpoints in the risk prediction of outcomes in anticoagulated patients with atrial fibrillation The Loire Valley Atrial Fibrillation Project Amitava Banerjee1; Laurent Fauchier2; Anne Bernard-Brunet2; Nicolas Clementy2; Gregory Y. H. Lip1 1University
of Birmingham Centre for Cardiovascular Sciences, City Hospital, Birmingham, UK; 2Service de Cardiologie, Pôle Coeur Thorax Vasculaire, Centre Hospitalier, Universitaire Trousseau et Faculté de Médecine, Université François Rabelais, Tours, France
Summary Several validated risk stratification schemes for prediction of ischaemic stroke (IS)/thromboembolism (TE) and major bleeding are available for patients with non-valvular atrial fibrillation (NVAF). On the basis for multiple common risk factors for IS/TE and bleeding, it has been suggested that composite risk prediction scores may be more practical and user-friendly than separate scores for bleeding and IS/TE. In a long-term prospective hospital registry of anticoagulated patients with newly diagnosed AF, we compared the predictive value of existing risk prediction scores as well as composite risk scores, and also compared these risk scoring systems using composite endpoints. Endpoint 1 was the simple composite of IS and major bleeds. Endpoint 2 was based on a composite of IS plus intracerebral haemorrhage (ICH). Endpoint 3 was based on weighted coefficients for IS/TE and ICH. Endpoint 4 was a composite of stroke, cardiovascular death, TE and major bleeding. The incremental predictive value of these scores over CHADS2 (as reference) for composite endpoints was assessed
Correspondence to: Prof. G. Y. H. Lip University of Birmingham Centre for Cardiovascular Sciences City Hospital, Birmingham B18 7QH, UK E-mail: [email protected]
using c-statistic, net reclassification improvement (NRI) and integrated discrimination improvement (IDI). Of 8,962 eligible individuals, 3,607 (40.2%) had NVAF and were on OAC at baseline. There were no statistically significant differences between the c-statistics of the various risk scores, compared with the CHADS2 score, regardless of the endpoint. For the various risk scores and various endpoints, NRI and IDI did not show significant improvement (≥1%), compared with the CHADS2 score. In conclusion, composite risk scores did not significantly improve risk prediction of endpoints in patients with NVAF, regardless of how endpoints were defined. This would support individualised prediction of IS/TE and bleeding separately using different separate risk prediction tools, and not the use of composite scores or endpoints for everyday ‘real world’ clinical practice, to guide decisions on thromboprophylaxis.
Keywords Cardiology, stroke prevention, thrombosis
Received: December 19, 2013 Accepted after minor revision: January 19, 2014 Prepublished online: January 23, 2014 doi:10.1160/TH13-12-1033 Thromb Haemost 2014; 111: 549–556
Note: The review process for this paper was fully handled by Christian Weber, Editor in Chief.
Introduction The epidemiologic burden of atrial fibrillation (AF) is increasing and the risk of ischaemic stroke (IS)/thromboembolism (TE) is also projected to increase due to the rising burden of comorbidities in patients with AF (1–6). Oral anticoagulation (OAC), whether with vitamin K antagonists (VKA) or novel oral anticoagulants, is a proven therapy for the reduction of risk of IS/TE in the setting of AF (7–10). Validated risk prediction tools are available for IS/TE in patients with non-valvular AF (NVAF) in order to guide the initiation of OAC (11–13). In clinical practice, the bleeding risk of OAC use is weighed against the risk of IS/TE prior to starting OAC, and validated tools are also available for risk prediction of © Schattauer 2014
bleeding in NVAF (13–15). The use of these risk prediction tools for IS/TE and bleeding are recommended in guidelines (16). Given that several risk factors are common to both IS/TE and bleeding in patients with NVAF, a composite risk score for IS/TE and bleeding may offer practical utility by simplifying the process of risk prediction (17, 18). However, such a risk score has not been previously evaluated in a “real-world”, prospective cohort of NVAF patients. The net clinical benefit (NCB) concept was originally suggested by Singer et al. as a means of quantifying the balance between risk of IS/TE and bleeding in individual patients with NVAF (19). Different composite endpoints and different methods of measuring NCB have also been used in studies of AF (20, 21). The role of risk prediction tools in prediction of composite outcomes and NCB has not been previously investigated. Thrombosis and Haemostasis 111.3/2014
Downloaded from www.thrombosis-online.com on 2014-06-15 | ID: 1000465826 | IP: 138.37.211.113 For personal or educational use only. No other uses without permission. All rights reserved.
550
Banerjee et al. Composite risk scores and composite endpoints in AF
In a long-term, prospective study of NVAF, we evaluated the incremental predictive value of different risk scoring systems, including composite risk scores, in the prediction of a range of composite outcomes, to establish whether composite risk prediction tools may offer clinical utility for ‘real world’ patients with NVAF.
Methods Study population The methods of the Loire Valley Atrial Fibrillation Project have been previously reported (22). At the Centre Hospitalier Régional et Universitaire in Tours (France), all patients diagnosed with NVAF or atrial flutter by the Cardiology Department between 2000–2010 were identified, excluding patients with valvular AF. The institution includes four hospitals covering all medical and surgical specialties; the only public institution in an area of around 4,000 km2, serving approximately 400,000 inhabitants. Patients were followed from the first record of AF after January 1, 2000 (i.e. index date) up to the latest data collection at the time of study (December 2010). Treatment at discharge was obtained by screening hospitalisation reports, and information on comorbidities was obtained from the computerised coding system. Patients were included in this study if they were receiving OAC at time of first diagnosis of AF. The study was approved by the Review Board of the Pole Coeur Thorax Vaisseaux from the Trousseau University Hospital in 2010 (December 7, 2010).
Models of risk prediction For each patient, the CHADS2 (11) and CHA2DS2-VASc (12) scores were calculated. The CHADS2 score was the sum of points obtained after adding one point for congestive heart failure, hypertension, age ≥75, and diabetes, and two points for previous stroke or TE (11). The CHA2DS2-VASc score was the sum of points after adding one point for congestive heart failure, hypertension, diabetes, vascular disease (including history of coronary, cerebrovascular or peripheral vascular disease), age 65–74, and female gender, and two points for previous stroke or TE and age ≥75 (12). According to the two risk scores, patients with a score of 0 on either schema were considered as ‘low risk’, 1 as ‘intermediate risk’, and ≥2 as ‘high risk’ of stroke and TE. The HAS-BLED (Hypertension, Abnormal renal and/or liver function, Stroke, Bleeding history or predisposition, Labile International Normalised Ratio [INR], Elderly [> 65 years], Drugs [antiplatelet drugs or NSAIDS]/alcohol excess concomitantly) score is a validated scoring system for bleeding risk stratification in AF patients (14). Patients with HAS-BLED score of 0–2 were deemed to have ‘lowmoderate’ bleeding risk and those with HAS-BLED score of ≥3 were classified as ‘high’ bleeding risk. The risk factors for HAS-BLED were combined with CHADS2 and CHA2DS2-VASc, respectively, to give a composite score. Finally, a multivariable score was derived by performing a multivariable regression analysis to identify predictors of the study outcomes. The factors included in the score were: previous HF, age
>75, age >65, diabetes mellitus, stroke, vascular disease, liver-renal impairment, previous bleeding and labile INR.
Outcomes During follow-up, information on outcomes of TE (including peripheral artery embolism and transient ischaemic attack [TIA]), stroke (ischaemic or haemorrhagic), bleeding, and all-cause mortality were recorded. Major bleeding was defined as bleeding with a reduction in the haemoglobin level of at least 2 g/l, or with transfusion of at least 1 unit of blood, or symptomatic bleeding in a critical area or organ (e.g. intracranial, intraspinal, intraocular, retroperitoneal, intra-articular or pericardial, or intramuscular with compartment syndrome) or bleeding that causes death. All bleeding and thromboembolic events were identified with the diagnosis coded in a subsequent hospitalisation during follow-up.
Models of composite endpoints For each individual, four different models were used to quantify composite outcomes. • Endpoint 1 was the crude sum of IS and major bleeding events (including ICH). • Endpoint 2 was based on the net clinical benefit model originally proposed by Singer et al. (19): Net Clinical Benefit (NCB) = (TE rate off warfarin – TE rate on warfarin) – 1.5x (ICH rate on warfarin-ICH rate off warfarin). The NCB was derived and defined for whole populations from population event rates, but in our analysis, NCB was used in individuals incorporating numbers of events. However, rates of events off warfarin cannot be calculated in our population (underlined sections). Therefore, the actual formula used for Model 2 was: Endpoint 2 = IS/TE +1.5(ICH). Endpoint 2 represents the sum of events in an individual but with weighting based on the Singer NCB model. • Endpoint 3 was based on a model for NCB with weighted coefficients for different categories of ischaemic and haemorrhagic events was first proposed by Connolly et al. (21): NCB= [IR(IS)plac +w1IR(HS)plac- w2IR(SDH)plac + w3IR(ECH)plac + w4IR(MI)plac] – [IR(IS)treat+ w1IR(HS)treat + w2IR(SDH)treat + w3IR(ECH)treat + w4IR(MI)treat] (where IR=incidence rate; IS=ischaemic stroke; HS=haemorrhagic stroke; SDH=subdural haemorrhage; ECH=extracranial haemorrhage; MI=myocardial infarction; plac=placebo; treat=treatment). Again, the underlined sections were not available for our patient population. However, the weightings for the other categories of event were derived from our patient population using the methodology outlined by Connolly et al. (21) to give the following formula for Endpoint 3: Endpoint 3 = 1.701(IS/TE)+2.855(ICH). Therefore, Endpoint 3 represents the sum of events in an individual with weightings based on the Connolly model. • Endpoint 4 was based a composite of all vascular events based on the definition by Hohnloser et al. (20), which included stroke, MI, cardiovascular death, pulmonary embolism systemic embolism or major bleeding. There are several event types unavailable for our patient population, which led to the
Thrombosis and Haemostasis 111.3/2014 Downloaded from www.thrombosis-online.com on 2014-06-15 | ID: 1000465826 | IP: 138.37.211.113 For personal or educational use only. No other uses without permission. All rights reserved.
© Schattauer 2014
Banerjee et al. Composite risk scores and composite endpoints in AF
modified formula for Endpoint 4: Endpoint 4 = stroke +systemic embolism + pulmonary embolism + all-cause death + major haemorrhage
Statistical analysis The study population was stratified on the basis of presence of ≥1 of each of the composite outcomes (Endpoints 1–4). Baseline characteristics were determined separately by presence or absence of each composite outcome, and differences were investigated using chi-squared test for categorical covariates. The Kruskal-Wallis test was used for continuous covariates since the distributions of variables do not have to be normal and the variances do not have to be equal. The predictive value of each scoring system was evaluated for each of the composite outcomes in comparison to the CHADS2 score (as the reference) using c-statistics, Net Reclassification Improvement (NRI) and Integrated Discrimination Improvement (IDI), as previously described (22, 23). C-statistics were compared using the non-parametric method proposed by DeLong et al. (24). The NRI focuses on reclassification tables constructed separately for patients with and without event, and quantifies the correct movement within a particular scoring system in comparison to the CHADS2 score (i.e. increased risk score for events counts positively, and increased risk score for non-events counts negatively). The IDI does not require risk groups and focuses on differences between integrated sensitivity and specificity for different risk prediction models in comparison to the CHADS2 score (23). The analyses aimed to compare the predictive value of different composite scoring systems for different composite outcomes, particularly the incremental predictive value in comparison to the CHADS2 score. C-statistics, NRI and IDI were given to three decimal places. A two-sided p-value < 0.05 was considered statistically significant. All analyses were performed with SPSS statistical software version 21.0 (IBM, Armonk, NY, USA) and NRI and IDI calculations were performed using R statistical software version 2.12.2 (University of Auckland, New Zealand).
Results Of 8,962 eligible individuals, 3607 (40.2%) had NVAF and were on OAC at baseline. Of these, 455 patients had ≥1 ischaemic stroke or major bleeding event (Endpoint 1), whilst 272 patients had a score ≥1 as defined by Endpoint 2 [IS/TE +1.5(ICH)] and for Endpoint 3 [1.701(IS/TE)+2.855(ICH)], 703 patients had a score ≥1 for Endpoint 4 during the study period. After a mean follow-up of 1.65 years (SD 2.44), there were 175 IS, 272 IS/TE, 60 ICH and 327 major bleeding events. ▶ Table 1 illustrates the baseline characteristics of all included individuals by presence or absence of composite event. For each event type, patients with events were older, had higher prevalence of comorbidities and higher risk prediction scores (▶ Table 1). Endpoint 2 and Endpoint 3 identified the same patient population. © Schattauer 2014
Comparisons of models
▶Table 2 shows the comparisons of the different risk prediction
models for the different endpoints. For Endpoint 1, the 14-factor HAS-BLED/CHA2DS2-VASc model compared to the CHADS2 model demonstrated a numerically higher c-statistic, but this was not statistically different (DeLong test p=0.894). Overall, the numerically highest c-statistic was found with the multivariate risk score and Endpoint 4 (0.668, 0.646–0.690;p=0.062), although again this did not reach statistical significance. For all four endpoints, there were no statistically significant differences between the c-statistics for the risk prediction scores and the CHADS2 score, as compared by the DeLong test. For Endpoint 1, there was no statistically significant net reclassification with any scores when compared with CHADS2. For Endpoint 2 and Endpoint 3, there was a positive NRI for the combined CHADS2/HAS-BLED score (0.047, 0.005–0.088; p=0.028), but no other scores, when compared with CHADS2. For Endpoint 4, there was actually a negative NRI for CHA2DS2-VASc, CHADS2/HASBLED, CHA2DS2-VASc/HAS-BLED (all with p
