DOI: 10.1161/CIRCULATIONAHA.115.018844
Prediction of First Cardiovascular Disease Event in Type 1 Diabetes: The Steno T1 Risk Engine
Running title: Vistisen et al.; Prediction of first CVD event in type 1 diabetes Dorte Vistisen, PhD1; Gregers Stig Andersen, PhD1; Christian Stevns Hansen, MD1; Adam Hulman, PhD2,3; Jan Erik Henriksen, PhD4,5; Henning Bech-Nielsen, Professor3,4,5; Marit Eika Jørgensen, MD, PhD1
1
Steno Sten no Diabetes Diab Di abet ab e ess Center, Center, Gentofte, Denmark; 2De Dept ept of Public Health, A Aarhus arh r us University, Aarhus,
D ennmark; 3Da Danish D nish ni shh D Diabetes iabe ia bete be tess Ac te Acad Academy, adem ad emy, em y O Odense, dens de nse, D ns Denmark; enma mark ma rk;; 4Od rk Odense Oden ense en se University Uni nivversit itty Hospital, Hosp Ho spittal sp al, Odense, Oden Od ense en s Denmark; Denmark; De enmarrk; 5Un University niv ver e sity ty ooff S Southern outtheern D Denmark, enma mark,, C ma Copenhagen, ope peenhageen,, De Denmark enm nmarkk
Address for Correspondence: Dorte Vistisen, PhD Steno Diabetes Center Niels Steensens Vej 6 DK-2820 Gentofte, Denmark Tel: +4530753119 Fax: +454443070 E-mail: [email protected] Journal Subject Terms: Chronic Ischemic Heart Disease; Epidemiology; Heart Failure; Diabetes, Type 1
1 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
Abstract
Background—Patients with type 1 diabetes are at increased risk of developing cardiovascular disease (CVD), but are currently undertreated. There are no risk scores used on a regular basis in clinical practice for assessing risk of CVD in type 1 diabetes. Methods and Results—From 4,306 clinically diagnosed adult type 1 diabetes patients, we developed a prediction model for estimating risk of first fatal or non-fatal CVD event (ischemic heart disease, ischemic stroke, heart failure and peripheral artery disease). Detailed clinical data including lifestyle factors were linked to event data from validated national registers. The risk non-parametric, data-driven prediction model was developed using a two-stage approach. First, a non-parametri riic, dat ataat a dr adriv iven iv approach was used to identify potentially informative risk factors and interactions ((random randdom fforest ra orestt results and survival tree an aanalysis). alysis). Secondly, based on resu sults from the first step, p, Poisson regression analysis final model. The CVD an nallysis was uused seed to dderive eriv er ivee thee fi iv fina nal mo na mode d l. T h ffinal he in nal C VD pprediction reedi dicttio ion mo model wa wass ex eexternally tern rnal rn a ly al validated different During median follow-up va aliida d ted in a dif ffeerennt ppopulation opuulat ation of 22,119 at ,1 119 ttype yp pe 1 ddiabetes iab bet etees ppatients. atient ntss. Du nt uring a me m diian n fo ollo ow-up up developed model was 6.8 years y ars (IQR: ye ( QR (I Q : 2.9-10.9)) a total of 793 ((18.4%) 18.4%)) develop ped CVD. The final pprediction rediction mode included age, sex, diabetes duration, systolic blood pressure, LDL cholesterol, HbA1c, albuminuria, glomerular filtration rate, smoking and exercise. Discrimination was excellent for a 5-year CVD event with a C-statistic of 0.826 (95%-CI: 0.807-0.845) in the derivation data and a C-statistic of 0.803 (0.767-0.839) in the validation data. The Hosmer-Lemeshow test showed good calibration (p>0.05) in both cohorts. Conclusions—This high performing CVD risk model allows the implementation of decision rules in a clinical setting.
Key words: Cardiovascular disease, type 1 diabetes, risk factors, survival analysis, prediction model
2 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
Introduction Patients with type 1 diabetes (T1D) have an almost three-fold increased mortality risk compared with the general population.1, 2 Cardiovascular disease (CVD) is the main cause of this excess mortality3 with CVD events happening more than a decade earlier.4 As T1D diabetes is generally diagnosed at a younger ages compares to type 2 diabetes, the time of exposure to diabetes-related CVD risk factors is longer. Also, known risk factors for CVD seem to operate differently in T1D, including a more profound effect of hyperglycemia5 and with no increased CVD risk by male sex6, 7, suggesting a difference in the pathophysiology of CVD in T1D and type 2 diabetes.6 There are several CVD risk models for both the general population and fo for or ty type pe 2 diabetes patients with the ‘UKPDS risk engine’8 as the best known. However, the UKPDS risk engi gine ne has has been bee e n shown sh V in T1D,9 and riskk scores VD sccores developed for type engine to underestimate risk of C CVD 2 diabetes diiabetes will wil ill probably proobab pr ably y not not apply app pplly to T1D T1D D patients pat a ientss in in general. gen enerral al.. Except T1D, Exce ept p for foor a small sma mall study ma stu tu udy onn coronary corronnary heart heartt disease diseease as pprediction red edicction ed onn inn cchildhood hilldh dhoo o d on onsett T 1D,,100 her eree is is, to oour urr kknowledge, nowl no wled led edge ge, on ge only ly oone ne rrisk iskk sc is scor oree fo or forr CV CVD D in T 1D.11 T 1D This hiss score hi scor sc oree was or waas based base ba sedd on the se the there score T1D. Swedish national diabetes register and included patients with a previous CVD event, a group of patients already fulfilling the criteria for secondary prevention of CVD. T1D was pragmatically defined as treatment with insulin only and with age of diabetes onset at 30 years or less, hereby excluding the significant proportion of T1D population diagnosed after age 30.12 None of these risk scores are used on a regular basis in clinical practice. The overall objective of this study was to develop and validate a model for predicting CVD in a large cohort of T1D patients without a previous CVD event.
3 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
Methods Study design and participants The prediction model was developed using 4,996 adult patients with T1D attending the outpatient clinic at Steno Diabetes Center in Gentofte, Denmark, from January 1st 2001 to 30th September 2013. We excluded 682 (13.7%) patients with previous CVD and 8 ( 20 units/week). Also, because urine albumin was measured from a 24-hour urine collectio on in ssome omee om collection patients and by spot urine samples in others, we classified patients into having norm moa oalb lbum lb umin um i urria ia, microalbuminuria or macroalbuminuria macroaalb lbuuminuria rather thann uusing s ng the albumin si normoalbuminuria, m eaasurement di dire r ctly ly y.19 Furthermore, Fur urth th her erm morre, re ha hhaving aviing m macroalbuminuria accro oalbu bumi bu minu mi nuria nu ia iss an iindication ndicattio nd ionn for for in init initiating itia it iati ia ting ti ng measurement directly. further fu urt r he her investigation inveest stig i atio att on aand ndd possibly posssib ibly ib y treatment tre reattmentt for foor nephropathy neephhroopa path hy and and ma mayy th therefore hereffor oree have ve add additional ditionnall predictive pred pr edic ed icti ic tive ti vee iinformation. nfor nf orma or mati ma tion ti on. on All Danish residents have a unique personal identification number recorded in the Danish Civil Registration System.20 The clinical data were linked to mortality data from the Cause of Death Register21 and to morbidity data from the Danish National Patient Register22 using this personal identification number. Data on ethnicity was obtained from the Central Person Register.23 The registers in Denmark are nationwide and covering all residents. We defined CVD as a composite outcome of fatal- and non-fatal events of ischemic heart disease, ischemic stroke, heart failure and peripheral artery disease. Arrhythmia diagnosed before or at baseline was tested as a possible predictor for CVD. See Supplemental Table 1 for the
6 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
specific International Classification of Diseases codes used. According to Danish law, ethics approval and patient consent is not required for registrybased studies. Access and use of the described data are approved by the Danish Data Protection Agency (j-No: 2007-58-0015 and 2012-58-0009). Statistical analysis Statistical analyses were performed in R version 3.1.3 (The R Foundation for Statistical Computing, www.R-project.org) and SAS version 9.2 (SAS Institute, Cary, NC, USA). All patients were followed until first CVD occurrence or death or until censor date 30th September 2013 (date of register extraction) (29,114 person-years). Median follow-up was 6.8 years (IQR: 2.9-10.9), during which 793 (18.4%) developed CVD. For most determinants around five percent or less of the values were missing. Eight perc percent cen entt of the the patients pat atie at ients had no blood pressure measurements ie mea easu ea surements at baseline. su e.. IIn n order to avoid exclusion excl ex clusion of patients cl pat atiien at ientss with with missing mis issi sing si ng g values vallues which whi h ch may may inf infer nfeer bbiased nf iase ia sedd re resu results, sult su lts,,24 m lt missing issi sing ing ddata ataa on at determinants de ete term rminants rm ts were were erre imputed impu puted using pu usin ingg the in the Multivariate Multiv Mu varriatee Imputations Imp mput utatio ut io ons ns by by Chained Chhaine ned Equations Equa Eq u tion onss (MICE) on (M MICE)) method meth me thod th od25 ((mice mice mi ce package pac acka kage ka ge in in R software) soft so ftwa ft ware are re)) with wiith missing-at-random mis issi sing si ng-at ng at-ran ran ando dom do m assumptions. assu as sump mpti mp tion ti onss. Twenty-five on Twent weent nty-fi five fi vee ccopies opie op iess ie of the data, each with missing values suitably imputed, were independently assessed in the analyses described below. Estimates of parameters of interest were averaged across the copies according to Rubin’s rules.26 The risk prediction model was developed using a two-stage approach. First a nonparametric, data-driven approach was used to identify potentially informative risk factors and interactions based on random forest and survival tree analysis. Secondly, Poisson regression analysis was used to derive the final prediction model based on the identified risk factors and interactions identified in the first step.
7 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
Details of the modelling process are as follows. In step 1, a random survival forest analysis was applied to rank the risk factor according to their permutation importance, and predictors with a negative importance score were excluded from further analysis (randomForestSRC package in R). Then, a survival tree model was fitted using the risk factors with a positive importance score (party package in R).27 A survival tree is constructed using recursive partitioning. At each node of the survival tree, the recursive partitioning algorithm identifies the risk factor and ‘‘split’’ in this factor which gives the maximal difference in cardiovascular disease (CVD) rates between the two resulting subgroups. This procedure is applied recursively until the tree has been grown to an optimal number of terminal ‘‘leaves’’. The survival tree approach is a flexible method which can take into account high le leve vell ve level nteractions between predictors. interactions In sstep tepp 2,, a Poisson regression analysis aapproach te pprroach to model yearl pp rlly incidence rates of CVD yearly nT 1D was app plie lied, uusing sin ingg lo in logg-pe gpers pe r on rs o -t - im me aass offs sett var aria ar iabl ia blee (s bl sta tatss ppackage accka kage iin n R) R). Be B caus ca usee it iiss us in T1D applied, log-person-time offset variable (stats Because we elll kknown nownn tthat h t risk ha s ooff CVD D of a pe pers son is no not co onsstaant ove veer ti ime,,2, 3 the he fol follow-up olllo oww up p pperiod eriiodd off well person constant over time, ea ch pparticipant arti ar tici ti cipa ci pant pa nt was as split spl plit it into int ntoo on one e-ye year ear aage ge bands ban ands ds ((Epi Epii pa Ep pack ckag ck agee in R ag ).28 H Hence, ence en ce, on ce onee pa pati patient tien ti entt ma en may each one-year package R). contribute with several records, one per follow-up interval. The informative risk factors and interactions identified in step 1 were tested. Also, because treated and untreated levels of BP and lipids may have different meaning, we further included interactions with medication (antihypertensive treatment or lipid-lowering treatment) in the model. The ratio total:HDL cholesterol was also considered. Backward elimination was used to identify significant predictors of CVD, using a level of significance of 5%. In addition, we used clinical judgement, as is recommended, to choose between highly correlated predictors of equal predictive power.29 Prior to analysis, predictors with a highly skewed distribution (triglycerides, eGFR and TSH) were
8 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
log-transformed to improve model calibration. The accuracy of the predicted CVD risk from the model was evaluated graphically by comparing means of estimated risk with the observed five- and 10-year cumulative CVD incidence in deciles of estimated risk. The calibration of the prediction model was determined using Hosmer-Lemeshow test of goodness of fit (PredictABEL package in R).30 The discriminative ability of the models was evaluated using the C-statistic which is an overall measure of the probability that the predicted CVD risk from the model is higher in a patient developing CVD than in a patient not developing CVD.31 Confidence intervals were computed with 2000 stratified bootstrap replicates (pROC package in R).32 Risk prediction models developed in one population are known to over- oorr un unde underestimate dere de rest re stim st imaate im the he risk in other populations.33 In the variable selection step, predictors are selected if they are estima mate ma tedd to bbee hi te high g ly associated with the CVD ou outc t ome in the derivat tio ionn data, irrespective of estimated highly outcome derivation their heiir true assoc association cia iation nw with ithh th it thee outc ooutcome. utc tcom omee. T om This hiss m hi means ean ns that at a ppredictor reediict c or w with ithh an ooverestimated it vereest ve stim imat im ated d regression egr g ession es coefficient coe oeffiicient iss more re likely lik i elly too be be included in ncllud u ed than thaan a predictor pred edic ed i to ic torr with w th wi h an underestimated unde un d reest stiimaateed regression egr gres essi es sion si on ccoefficient. oeff oe ffic ff icie ic ient ie nt.34 H nt Hence, ence en ce, fo ce forr th thee se sele selected lect le cted ct ed ppredictors, redi re dict di ctoors ct rs, wee would wou ould ld expect expec xp pec ectt the the estimated esti es tima ti mate ma tedd te regression coefficients on average to be smaller in a different dataset. In order to correct for the overestimation of the regression coefficients due to the variable selection, we applied postestimation parameter wise shrinkage of the regression coefficients (shrink package in R), based on calibrating the linear predictor using leave-one-out cross-validation (Jacknife).35 Using this approach, weak predictors less correlated with the CVD outcome are shrunken relatively more than the strong predictors in the model. The shrunken risk prediction model was externally validated in the T1D population of the Funen Diabetes Database in Denmark.36 As a sensitivity analysis, the model without shrinkage
9 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
was tested as well. The validation population consisted of 2,118 adult clinically diagnosed T1D patients from the same criteria as in the derivation cohort and without a previous CVD (13,635 person-years). Baseline was from 2003 and end of follow-up was end of 2014. Median follow-up was 6.6 years (IQR: 4.0-9.1) during which 243 (11.5%) developed CVD (see Table 1 for further characteristics of the validation population). Because relatively few (
Prediction of First Cardiovascular Disease Event in Type 1 Diabetes: The Steno T1 Risk Engine
Running title: Vistisen et al.; Prediction of first CVD event in type 1 diabetes Dorte Vistisen, PhD1; Gregers Stig Andersen, PhD1; Christian Stevns Hansen, MD1; Adam Hulman, PhD2,3; Jan Erik Henriksen, PhD4,5; Henning Bech-Nielsen, Professor3,4,5; Marit Eika Jørgensen, MD, PhD1
1
Steno Sten no Diabetes Diab Di abet ab e ess Center, Center, Gentofte, Denmark; 2De Dept ept of Public Health, A Aarhus arh r us University, Aarhus,
D ennmark; 3Da Danish D nish ni shh D Diabetes iabe ia bete be tess Ac te Acad Academy, adem ad emy, em y O Odense, dens de nse, D ns Denmark; enma mark ma rk;; 4Od rk Odense Oden ense en se University Uni nivversit itty Hospital, Hosp Ho spittal sp al, Odense, Oden Od ense en s Denmark; Denmark; De enmarrk; 5Un University niv ver e sity ty ooff S Southern outtheern D Denmark, enma mark,, C ma Copenhagen, ope peenhageen,, De Denmark enm nmarkk
Address for Correspondence: Dorte Vistisen, PhD Steno Diabetes Center Niels Steensens Vej 6 DK-2820 Gentofte, Denmark Tel: +4530753119 Fax: +454443070 E-mail: [email protected] Journal Subject Terms: Chronic Ischemic Heart Disease; Epidemiology; Heart Failure; Diabetes, Type 1
1 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
Abstract
Background—Patients with type 1 diabetes are at increased risk of developing cardiovascular disease (CVD), but are currently undertreated. There are no risk scores used on a regular basis in clinical practice for assessing risk of CVD in type 1 diabetes. Methods and Results—From 4,306 clinically diagnosed adult type 1 diabetes patients, we developed a prediction model for estimating risk of first fatal or non-fatal CVD event (ischemic heart disease, ischemic stroke, heart failure and peripheral artery disease). Detailed clinical data including lifestyle factors were linked to event data from validated national registers. The risk non-parametric, data-driven prediction model was developed using a two-stage approach. First, a non-parametri riic, dat ataat a dr adriv iven iv approach was used to identify potentially informative risk factors and interactions ((random randdom fforest ra orestt results and survival tree an aanalysis). alysis). Secondly, based on resu sults from the first step, p, Poisson regression analysis final model. The CVD an nallysis was uused seed to dderive eriv er ivee thee fi iv fina nal mo na mode d l. T h ffinal he in nal C VD pprediction reedi dicttio ion mo model wa wass ex eexternally tern rnal rn a ly al validated different During median follow-up va aliida d ted in a dif ffeerennt ppopulation opuulat ation of 22,119 at ,1 119 ttype yp pe 1 ddiabetes iab bet etees ppatients. atient ntss. Du nt uring a me m diian n fo ollo ow-up up developed model was 6.8 years y ars (IQR: ye ( QR (I Q : 2.9-10.9)) a total of 793 ((18.4%) 18.4%)) develop ped CVD. The final pprediction rediction mode included age, sex, diabetes duration, systolic blood pressure, LDL cholesterol, HbA1c, albuminuria, glomerular filtration rate, smoking and exercise. Discrimination was excellent for a 5-year CVD event with a C-statistic of 0.826 (95%-CI: 0.807-0.845) in the derivation data and a C-statistic of 0.803 (0.767-0.839) in the validation data. The Hosmer-Lemeshow test showed good calibration (p>0.05) in both cohorts. Conclusions—This high performing CVD risk model allows the implementation of decision rules in a clinical setting.
Key words: Cardiovascular disease, type 1 diabetes, risk factors, survival analysis, prediction model
2 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
Introduction Patients with type 1 diabetes (T1D) have an almost three-fold increased mortality risk compared with the general population.1, 2 Cardiovascular disease (CVD) is the main cause of this excess mortality3 with CVD events happening more than a decade earlier.4 As T1D diabetes is generally diagnosed at a younger ages compares to type 2 diabetes, the time of exposure to diabetes-related CVD risk factors is longer. Also, known risk factors for CVD seem to operate differently in T1D, including a more profound effect of hyperglycemia5 and with no increased CVD risk by male sex6, 7, suggesting a difference in the pathophysiology of CVD in T1D and type 2 diabetes.6 There are several CVD risk models for both the general population and fo for or ty type pe 2 diabetes patients with the ‘UKPDS risk engine’8 as the best known. However, the UKPDS risk engi gine ne has has been bee e n shown sh V in T1D,9 and riskk scores VD sccores developed for type engine to underestimate risk of C CVD 2 diabetes diiabetes will wil ill probably proobab pr ably y not not apply app pplly to T1D T1D D patients pat a ientss in in general. gen enerral al.. Except T1D, Exce ept p for foor a small sma mall study ma stu tu udy onn coronary corronnary heart heartt disease diseease as pprediction red edicction ed onn inn cchildhood hilldh dhoo o d on onsett T 1D,,100 her eree is is, to oour urr kknowledge, nowl no wled led edge ge, on ge only ly oone ne rrisk iskk sc is scor oree fo or forr CV CVD D in T 1D.11 T 1D This hiss score hi scor sc oree was or waas based base ba sedd on the se the there score T1D. Swedish national diabetes register and included patients with a previous CVD event, a group of patients already fulfilling the criteria for secondary prevention of CVD. T1D was pragmatically defined as treatment with insulin only and with age of diabetes onset at 30 years or less, hereby excluding the significant proportion of T1D population diagnosed after age 30.12 None of these risk scores are used on a regular basis in clinical practice. The overall objective of this study was to develop and validate a model for predicting CVD in a large cohort of T1D patients without a previous CVD event.
3 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
Methods Study design and participants The prediction model was developed using 4,996 adult patients with T1D attending the outpatient clinic at Steno Diabetes Center in Gentofte, Denmark, from January 1st 2001 to 30th September 2013. We excluded 682 (13.7%) patients with previous CVD and 8 ( 20 units/week). Also, because urine albumin was measured from a 24-hour urine collectio on in ssome omee om collection patients and by spot urine samples in others, we classified patients into having norm moa oalb lbum lb umin um i urria ia, microalbuminuria or macroalbuminuria macroaalb lbuuminuria rather thann uusing s ng the albumin si normoalbuminuria, m eaasurement di dire r ctly ly y.19 Furthermore, Fur urth th her erm morre, re ha hhaving aviing m macroalbuminuria accro oalbu bumi bu minu mi nuria nu ia iss an iindication ndicattio nd ionn for for in init initiating itia it iati ia ting ti ng measurement directly. further fu urt r he her investigation inveest stig i atio att on aand ndd possibly posssib ibly ib y treatment tre reattmentt for foor nephropathy neephhroopa path hy and and ma mayy th therefore hereffor oree have ve add additional ditionnall predictive pred pr edic ed icti ic tive ti vee iinformation. nfor nf orma or mati ma tion ti on. on All Danish residents have a unique personal identification number recorded in the Danish Civil Registration System.20 The clinical data were linked to mortality data from the Cause of Death Register21 and to morbidity data from the Danish National Patient Register22 using this personal identification number. Data on ethnicity was obtained from the Central Person Register.23 The registers in Denmark are nationwide and covering all residents. We defined CVD as a composite outcome of fatal- and non-fatal events of ischemic heart disease, ischemic stroke, heart failure and peripheral artery disease. Arrhythmia diagnosed before or at baseline was tested as a possible predictor for CVD. See Supplemental Table 1 for the
6 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
specific International Classification of Diseases codes used. According to Danish law, ethics approval and patient consent is not required for registrybased studies. Access and use of the described data are approved by the Danish Data Protection Agency (j-No: 2007-58-0015 and 2012-58-0009). Statistical analysis Statistical analyses were performed in R version 3.1.3 (The R Foundation for Statistical Computing, www.R-project.org) and SAS version 9.2 (SAS Institute, Cary, NC, USA). All patients were followed until first CVD occurrence or death or until censor date 30th September 2013 (date of register extraction) (29,114 person-years). Median follow-up was 6.8 years (IQR: 2.9-10.9), during which 793 (18.4%) developed CVD. For most determinants around five percent or less of the values were missing. Eight perc percent cen entt of the the patients pat atie at ients had no blood pressure measurements ie mea easu ea surements at baseline. su e.. IIn n order to avoid exclusion excl ex clusion of patients cl pat atiien at ientss with with missing mis issi sing si ng g values vallues which whi h ch may may inf infer nfeer bbiased nf iase ia sedd re resu results, sult su lts,,24 m lt missing issi sing ing ddata ataa on at determinants de ete term rminants rm ts were were erre imputed impu puted using pu usin ingg the in the Multivariate Multiv Mu varriatee Imputations Imp mput utatio ut io ons ns by by Chained Chhaine ned Equations Equa Eq u tion onss (MICE) on (M MICE)) method meth me thod th od25 ((mice mice mi ce package pac acka kage ka ge in in R software) soft so ftwa ft ware are re)) with wiith missing-at-random mis issi sing si ng-at ng at-ran ran ando dom do m assumptions. assu as sump mpti mp tion ti onss. Twenty-five on Twent weent nty-fi five fi vee ccopies opie op iess ie of the data, each with missing values suitably imputed, were independently assessed in the analyses described below. Estimates of parameters of interest were averaged across the copies according to Rubin’s rules.26 The risk prediction model was developed using a two-stage approach. First a nonparametric, data-driven approach was used to identify potentially informative risk factors and interactions based on random forest and survival tree analysis. Secondly, Poisson regression analysis was used to derive the final prediction model based on the identified risk factors and interactions identified in the first step.
7 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
Details of the modelling process are as follows. In step 1, a random survival forest analysis was applied to rank the risk factor according to their permutation importance, and predictors with a negative importance score were excluded from further analysis (randomForestSRC package in R). Then, a survival tree model was fitted using the risk factors with a positive importance score (party package in R).27 A survival tree is constructed using recursive partitioning. At each node of the survival tree, the recursive partitioning algorithm identifies the risk factor and ‘‘split’’ in this factor which gives the maximal difference in cardiovascular disease (CVD) rates between the two resulting subgroups. This procedure is applied recursively until the tree has been grown to an optimal number of terminal ‘‘leaves’’. The survival tree approach is a flexible method which can take into account high le leve vell ve level nteractions between predictors. interactions In sstep tepp 2,, a Poisson regression analysis aapproach te pprroach to model yearl pp rlly incidence rates of CVD yearly nT 1D was app plie lied, uusing sin ingg lo in logg-pe gpers pe r on rs o -t - im me aass offs sett var aria ar iabl ia blee (s bl sta tatss ppackage accka kage iin n R) R). Be B caus ca usee it iiss us in T1D applied, log-person-time offset variable (stats Because we elll kknown nownn tthat h t risk ha s ooff CVD D of a pe pers son is no not co onsstaant ove veer ti ime,,2, 3 the he fol follow-up olllo oww up p pperiod eriiodd off well person constant over time, ea ch pparticipant arti ar tici ti cipa ci pant pa nt was as split spl plit it into int ntoo on one e-ye year ear aage ge bands ban ands ds ((Epi Epii pa Ep pack ckag ck agee in R ag ).28 H Hence, ence en ce, on ce onee pa pati patient tien ti entt ma en may each one-year package R). contribute with several records, one per follow-up interval. The informative risk factors and interactions identified in step 1 were tested. Also, because treated and untreated levels of BP and lipids may have different meaning, we further included interactions with medication (antihypertensive treatment or lipid-lowering treatment) in the model. The ratio total:HDL cholesterol was also considered. Backward elimination was used to identify significant predictors of CVD, using a level of significance of 5%. In addition, we used clinical judgement, as is recommended, to choose between highly correlated predictors of equal predictive power.29 Prior to analysis, predictors with a highly skewed distribution (triglycerides, eGFR and TSH) were
8 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
log-transformed to improve model calibration. The accuracy of the predicted CVD risk from the model was evaluated graphically by comparing means of estimated risk with the observed five- and 10-year cumulative CVD incidence in deciles of estimated risk. The calibration of the prediction model was determined using Hosmer-Lemeshow test of goodness of fit (PredictABEL package in R).30 The discriminative ability of the models was evaluated using the C-statistic which is an overall measure of the probability that the predicted CVD risk from the model is higher in a patient developing CVD than in a patient not developing CVD.31 Confidence intervals were computed with 2000 stratified bootstrap replicates (pROC package in R).32 Risk prediction models developed in one population are known to over- oorr un unde underestimate dere de rest re stim st imaate im the he risk in other populations.33 In the variable selection step, predictors are selected if they are estima mate ma tedd to bbee hi te high g ly associated with the CVD ou outc t ome in the derivat tio ionn data, irrespective of estimated highly outcome derivation their heiir true assoc association cia iation nw with ithh th it thee outc ooutcome. utc tcom omee. T om This hiss m hi means ean ns that at a ppredictor reediict c or w with ithh an ooverestimated it vereest ve stim imat im ated d regression egr g ession es coefficient coe oeffiicient iss more re likely lik i elly too be be included in ncllud u ed than thaan a predictor pred edic ed i to ic torr with w th wi h an underestimated unde un d reest stiimaateed regression egr gres essi es sion si on ccoefficient. oeff oe ffic ff icie ic ient ie nt.34 H nt Hence, ence en ce, fo ce forr th thee se sele selected lect le cted ct ed ppredictors, redi re dict di ctoors ct rs, wee would wou ould ld expect expec xp pec ectt the the estimated esti es tima ti mate ma tedd te regression coefficients on average to be smaller in a different dataset. In order to correct for the overestimation of the regression coefficients due to the variable selection, we applied postestimation parameter wise shrinkage of the regression coefficients (shrink package in R), based on calibrating the linear predictor using leave-one-out cross-validation (Jacknife).35 Using this approach, weak predictors less correlated with the CVD outcome are shrunken relatively more than the strong predictors in the model. The shrunken risk prediction model was externally validated in the T1D population of the Funen Diabetes Database in Denmark.36 As a sensitivity analysis, the model without shrinkage
9 Downloaded from http://circ.ahajournals.org/ at Queen's University--Kingston (CANADA) on February 23, 2016
DOI: 10.1161/CIRCULATIONAHA.115.018844
was tested as well. The validation population consisted of 2,118 adult clinically diagnosed T1D patients from the same criteria as in the derivation cohort and without a previous CVD (13,635 person-years). Baseline was from 2003 and end of follow-up was end of 2014. Median follow-up was 6.6 years (IQR: 4.0-9.1) during which 243 (11.5%) developed CVD (see Table 1 for further characteristics of the validation population). Because relatively few (
