Original Paper
Urologia Internationalis
Urol Int 2014;93:262–268 DOI: 10.1159/000354430
Received: May 2, 2013 Accepted after revision: July 17, 2013 Published online: December 14, 2013
External Validation of Nomogram Predicting the Probability of Specimen-Confined Disease (pT2–3a, R0N0) in Patients Undergoing Radical Prostatectomy and Pelvic Lymph Node Dissection Luigi Mearini a Mauro Gacci b Omar Saleh b Cosimo De Nunzio c Riccardo Schiavina d Alchiede Simonato e Andrea Tubaro c Giorgio Carmignani e Vincenzo Mirone f Marco Carini b Vittorio Bini a Massimo Porena a Members of the Multicenter Italian Report on Radical Prostatectomy: Outcome and Research (MIRROR) Project – Leading Urological No-Profit Foundation for Advanced Research (LUNA) Foundation a Department of Urology, University of Perugia, Perugia, b Department of Urology, University of Florence, Careggi Hospital, Florence, c Department of Urology, Sant’Andrea Hospital, University ‘La Sapienza’ Rome, d Department of Urology, University of Bologna, Alma Mater, Bologna, e Department of Urology, University of Genova, Genova, and f Department of Urology, University of Naples, Naples, Italy
Key Words Radical prostatectomy · High-risk prostate cancer · Nomogram
Abstract Introduction: Before radical prostatectomy (RP), a nomogram [Briganti et al., Eur Urol 2012; 61: 584–592] permits to measure the probability of specimen-confined (SC) disease (pT2–pT3a, node negative with negative margins) in highrisk prostate cancer (PCa). The aim of our study was to perform an external validation of this nomogram. Materials and Methods: Between 2007 and 2011, 623 patients with highrisk PCa (prostate-specific antigen (PSA) >20 ng/ml and/or biopsy Gleason score ≥8 and/or clinical stage T3) underwent RP and pelvic lymph node dissection at tertiary referral cen-
© 2013 S. Karger AG, Basel 0042–1138/13/0933–0262$38.00/0 E-Mail [email protected] www.karger.com/uin
ters. Multivariable logistic regression models predicting the presence of SC disease were built in; we then used the area under curve of the receiver operating characteristic analysis to quantify accuracy of the nomogram to predict SC disease. The extent of over- or underestimation was evaluated within calibration plots. Results: 29% (181/623) of men had SC disease at RP. Preoperative PSA, biopsy Gleason score and stage differed significantly (all p < 0.001) between men with SC disease and those without. External validation of the nomogram showed an acceptable accuracy (area under curve: 66.3, 95% CI 62.4–70%) and a perfect calibration plot. Conclusions: The external cohort validates the original nomogram, with perfect calibration characteristics. The adequate although reduced accuracy may reflect the wide spectrum and behavior of the so-called high-risk PCa. © 2013 S. Karger AG, Basel
Luigi Mearini, MD Urology Department, University of Perugia Policlinico Santa Maria della Misericordia IT–06122 Perugia (Italy) E-Mail luigi.mearini @ tin.it
Introduction
clinical settings, as suggested by the authors. External validation of a predictive (nomogram) or prognostic tool should be mandatory to establish whether the tool works satisfactorily in different patient populations or in different surgical settings or volumes. Moreover, predictive tools need to be simple enough to ensure that physicians are capable of using them in a busy clinical practice [10]. The aim of the present study was to perform an external validation of the nomogram (by discrimination and calibration) after analyzing predictive factors of SC disease at RP in a population of high-risk PC subjects.
The advent of prostate-specific antigen (PSA) has modified clinical presentation of prostate cancer (PCa) [1], with a stage migration and an increasing percentage of patients suitable for radical prostatectomy (RP) [2, 3] or radiotherapy. Despite this, a small percentage of patients continue to present with clinical stage T3 disease: according to Mayo experience [4], the incidence of men undergoing RP for clinical T3 PCa was 3% in 2001. Whereas for some stages therapeutic options are standardized [5], the optimum treatment for locally advanced or high-risk disease (PSA >20 ng/ml, clinical T3N0M0 disease or high Gleason score) is still under debate. High morbidity rates of RP in the so-called high-risk PCa must be balanced against the possibility that surgery really will be curative, i.e. more beneficial when a complete removal of the disease is possible, which is in case of tumor confined to the surgical specimen (negative margin and negative lymph nodes). This thesis is supported by previous data reporting more favorable survival rates in high-risk PCa showing organ-confined disease at RP [6–8]. Clinical overstaging or pathological downstaging are significative, implying that in some subjects surgery alone is sufficient to cure cancer, while in such cases external beam radiotherapy (EBRT) plus long-term hormone therapy (HT) represent overtreatment. According to this, a tool able to predict the presence of specimenconfined (SC) disease after surgery might be useful for selecting the best candidate for the surgical procedure, thus decreasing the rate of patients potentially exposed to unnecessary treatment-related complications. To address this, in 2012 Briganti et al. [9] developed and internally validated a novel predictive model for a favorable pathologic outcome in high-risk patients treated with surgery. In this large experience, 37% of patients had SC disease at RP, i.e. margins and lymph nodes were negative. All baseline variables, including age, PSA, clinical stage and biopsy Gleason score resulted as independent predictors of SC, and, interestingly, patients with SC disease had higher 10-year biochemical-free survival and cancer-specific survival rates than patients with positive margins and/or positive lymph node disease (66 vs. 47 and 98 vs. 88%, respectively). The authors constructed an internally validated nomogram based on age, PSA, clinical stage, and biopsy Gleason score, which demonstrated 72% accuracy in the prediction of SC disease. Despite the excellent performance characteristics reported in the internal validation, the nomogram needs an external confirmation to test its applicability in other
Statistical Analysis Descriptive statistics focused on frequencies and proportions for categorical variables. Means, medians and ranges were recorded as continuously coded variables. Due to their asymmetric distribution, we used Mann-Whitney to compare continuous variables and χ2 to compare proportions. Uni- and multivariate logistic regression models predicting the presence of SC disease at RP were built in. Covariates consisted of age, preoperative PSA, clinical stage and biopsy Gleason grade. Each patient’s data was scored using the original nomogram and each individual probability of SC disease and relative score were calculated. Calibration of the nomogram was assessed by
External Validation of Nomogram Predicting Organ-Confined Disease at RP
Urol Int 2014;93:262–268 DOI: 10.1159/000354430
Materials and Methods Population of Study and Design In 2006, the LUNA (Leading Urological No-Profit Foundation Advanced Research) foundation planned the MIRROR (Multicenter Italian Report on Radical Prostatectomy: Outcome and Research) study. Urologic departments all over Italy were involved in evaluating the oncological, surgical, and functional characteristics of PCa and RP outcomes. All patients treated with a RP for PCa in each urology center were consecutively included: all data were stored and E-mailed in a web data manager (CLICON). The study involved 135 Italian centers enrolling 2,408 consecutive patients. Patients with incomplete data or treated by neoadjuvant therapy were excluded from the current study. Overall, 623 men with non-metastatic, clinical high-risk PCa (defined as patients with at least one of the following risk factors according to the European Association of Urology (EAU) guidelines: preoperative PSA value >20 ng/ml and/or cT3 clinical stage, and/or biopsy Gleason score ≥8) were evaluated. All patients were treated with open RP and pelvic lymph node dissection at six urologic tertiary referral centers for treatment of PCa. At each center, clinical staging was defined according to the 2002 TNM staging system; prostate biopsy cores were obtained under transrectal ultrasound guidance. Dedicated genitourinary pathologists assessed biopsy and pathologic grading according to the Gleason grading system. Before surgery, all patients were staged by means of computed tomography and bone scan. According to the endpoint, the presence of a SC disease was defined as pT2–pT3a PCa with negative surgical margins (R0) and no lymph node invasion (pN0). When required, institutional review board at each contributing institution approved the study.
263
comparing its predicted probability of SC with actual presence of SC disease by using Hosmer-Lemeshow goodness-of-fit statistics. We also built a calibration plot by drawing predicted probability by deciles in the x-axis and the observed outcome in the y-axis, which makes the plot a graphical illustration of the Hosmer-Lemeshow goodness-of-fit test. A perfectly accurate nomogram prediction model would result in a plot where the observed and predicted probabilities for given groups fall along the 45° line. The distance between the pairs and the 45° line is a measure of the absolute error of the nomogram. The performance of the nomogram was also evaluated by drawing a receiver operating characteristic (ROC) curve and calculating the area under the curve (AUC). The AUC reflects a test’s ability to discriminate between diseased and non-diseased patients. An AUC of 1 indicates perfect concordance (100% sensitivity and 100% specificity), whereas an AUC of 0.5 indicates a result equal to chance. IBM-SPSS version 20.0 statistical software was used for statistical analysis. All tests were two-sided with a significance level set at 0.05.
Results
Discussion
Descriptive characteristics of patients are listed in table 1, which included patients of the development cohort. At RP, the overall rate of pT2–pT3a was 69.3% (34.8% pT2a–c, 34.5% pT3a), while the overall negative margin rate was 52.1% and the rate of pathological Gleason score 20 ng/ml, cT3, biopsy Gleason score 8–10), remains controversial, even if many authors [11–19] have shown good results using surgery at least as first-line treatment. This is true also considering that pathological downstaging is sometimes significative (5% for Gontero et al. [11], up to 27% for Ward et al. [4], and 11.1% Briganti et al. [9]); in our experience, the incidence of pathological downstaging in cT3 disease is 5.1%, which should mean that in a discrete number of subjects surgery alone is sufficient to cure cancer [20]. Moreover, if we consider patients with SC disease (29.0% in our experience, 37% for Briganti et al.), these subjects will not need adjuvant EBRT and/or HT, while for example all patients with high-risk disease treated with EBRT need at least 3 years of adjuvant hormone manipulation [21]. The debulking effect of surgery permits superior effects of adjuvant therapy [22]; this concept seems to be supported by the only one randomized study between RP plus adjuvant HT versus EBRT plus adjuvant HT by Akakura et al. [23]: survival rates in the surgery group were better than the radiation group, although with a not statistically significant Mearini et al.
Table 1. Descriptive statistics of original development data (Briganti et al. nomogram) and 623 patients with clinical high-risk PCa
treated with RP and pelvic lymph node dissection at Italian institutions; current data were stratified and compared according to tumor extent (i.e. non-specimen confined vs. SC disease)
Number (%) Age, years Mean Median Range PSA, ng/ml Mean Median Range PSA, ng/ml ≤10 10–20 20–50 50–100 Clinical stage cT1 cT2 cT3 Biopsy GS ≤6 7 ≥8 NCCN/EAU risk One Two Three Pathological stage pT2 pT3a pT3b pT4 PSM Overall pT2 disease Pathologic GS ≤6 7 ≥8 Downstaging (cT3 to pT2) Upstaging (cT1–T2 to ≥pT3) Upgrading (cGS 6 to >, cGS 7 to >) LNI
Briganti et al. overall
Current series
p
overall
non-SC
SC
1,366
623
442 (71.0)
181 (29.0)
65.4 66.0 41–88.9
65.2 66.0 30–85
65.1 66.0 45–85
65.3 67.0 30–76
24.9 21.3 0.5–100.0
20.4 11.5 0.6–100
23.1 13.0 0.6–100
14.0 8.4 2.4–96.6
Urologia Internationalis
Urol Int 2014;93:262–268 DOI: 10.1159/000354430
Received: May 2, 2013 Accepted after revision: July 17, 2013 Published online: December 14, 2013
External Validation of Nomogram Predicting the Probability of Specimen-Confined Disease (pT2–3a, R0N0) in Patients Undergoing Radical Prostatectomy and Pelvic Lymph Node Dissection Luigi Mearini a Mauro Gacci b Omar Saleh b Cosimo De Nunzio c Riccardo Schiavina d Alchiede Simonato e Andrea Tubaro c Giorgio Carmignani e Vincenzo Mirone f Marco Carini b Vittorio Bini a Massimo Porena a Members of the Multicenter Italian Report on Radical Prostatectomy: Outcome and Research (MIRROR) Project – Leading Urological No-Profit Foundation for Advanced Research (LUNA) Foundation a Department of Urology, University of Perugia, Perugia, b Department of Urology, University of Florence, Careggi Hospital, Florence, c Department of Urology, Sant’Andrea Hospital, University ‘La Sapienza’ Rome, d Department of Urology, University of Bologna, Alma Mater, Bologna, e Department of Urology, University of Genova, Genova, and f Department of Urology, University of Naples, Naples, Italy
Key Words Radical prostatectomy · High-risk prostate cancer · Nomogram
Abstract Introduction: Before radical prostatectomy (RP), a nomogram [Briganti et al., Eur Urol 2012; 61: 584–592] permits to measure the probability of specimen-confined (SC) disease (pT2–pT3a, node negative with negative margins) in highrisk prostate cancer (PCa). The aim of our study was to perform an external validation of this nomogram. Materials and Methods: Between 2007 and 2011, 623 patients with highrisk PCa (prostate-specific antigen (PSA) >20 ng/ml and/or biopsy Gleason score ≥8 and/or clinical stage T3) underwent RP and pelvic lymph node dissection at tertiary referral cen-
© 2013 S. Karger AG, Basel 0042–1138/13/0933–0262$38.00/0 E-Mail [email protected] www.karger.com/uin
ters. Multivariable logistic regression models predicting the presence of SC disease were built in; we then used the area under curve of the receiver operating characteristic analysis to quantify accuracy of the nomogram to predict SC disease. The extent of over- or underestimation was evaluated within calibration plots. Results: 29% (181/623) of men had SC disease at RP. Preoperative PSA, biopsy Gleason score and stage differed significantly (all p < 0.001) between men with SC disease and those without. External validation of the nomogram showed an acceptable accuracy (area under curve: 66.3, 95% CI 62.4–70%) and a perfect calibration plot. Conclusions: The external cohort validates the original nomogram, with perfect calibration characteristics. The adequate although reduced accuracy may reflect the wide spectrum and behavior of the so-called high-risk PCa. © 2013 S. Karger AG, Basel
Luigi Mearini, MD Urology Department, University of Perugia Policlinico Santa Maria della Misericordia IT–06122 Perugia (Italy) E-Mail luigi.mearini @ tin.it
Introduction
clinical settings, as suggested by the authors. External validation of a predictive (nomogram) or prognostic tool should be mandatory to establish whether the tool works satisfactorily in different patient populations or in different surgical settings or volumes. Moreover, predictive tools need to be simple enough to ensure that physicians are capable of using them in a busy clinical practice [10]. The aim of the present study was to perform an external validation of the nomogram (by discrimination and calibration) after analyzing predictive factors of SC disease at RP in a population of high-risk PC subjects.
The advent of prostate-specific antigen (PSA) has modified clinical presentation of prostate cancer (PCa) [1], with a stage migration and an increasing percentage of patients suitable for radical prostatectomy (RP) [2, 3] or radiotherapy. Despite this, a small percentage of patients continue to present with clinical stage T3 disease: according to Mayo experience [4], the incidence of men undergoing RP for clinical T3 PCa was 3% in 2001. Whereas for some stages therapeutic options are standardized [5], the optimum treatment for locally advanced or high-risk disease (PSA >20 ng/ml, clinical T3N0M0 disease or high Gleason score) is still under debate. High morbidity rates of RP in the so-called high-risk PCa must be balanced against the possibility that surgery really will be curative, i.e. more beneficial when a complete removal of the disease is possible, which is in case of tumor confined to the surgical specimen (negative margin and negative lymph nodes). This thesis is supported by previous data reporting more favorable survival rates in high-risk PCa showing organ-confined disease at RP [6–8]. Clinical overstaging or pathological downstaging are significative, implying that in some subjects surgery alone is sufficient to cure cancer, while in such cases external beam radiotherapy (EBRT) plus long-term hormone therapy (HT) represent overtreatment. According to this, a tool able to predict the presence of specimenconfined (SC) disease after surgery might be useful for selecting the best candidate for the surgical procedure, thus decreasing the rate of patients potentially exposed to unnecessary treatment-related complications. To address this, in 2012 Briganti et al. [9] developed and internally validated a novel predictive model for a favorable pathologic outcome in high-risk patients treated with surgery. In this large experience, 37% of patients had SC disease at RP, i.e. margins and lymph nodes were negative. All baseline variables, including age, PSA, clinical stage and biopsy Gleason score resulted as independent predictors of SC, and, interestingly, patients with SC disease had higher 10-year biochemical-free survival and cancer-specific survival rates than patients with positive margins and/or positive lymph node disease (66 vs. 47 and 98 vs. 88%, respectively). The authors constructed an internally validated nomogram based on age, PSA, clinical stage, and biopsy Gleason score, which demonstrated 72% accuracy in the prediction of SC disease. Despite the excellent performance characteristics reported in the internal validation, the nomogram needs an external confirmation to test its applicability in other
Statistical Analysis Descriptive statistics focused on frequencies and proportions for categorical variables. Means, medians and ranges were recorded as continuously coded variables. Due to their asymmetric distribution, we used Mann-Whitney to compare continuous variables and χ2 to compare proportions. Uni- and multivariate logistic regression models predicting the presence of SC disease at RP were built in. Covariates consisted of age, preoperative PSA, clinical stage and biopsy Gleason grade. Each patient’s data was scored using the original nomogram and each individual probability of SC disease and relative score were calculated. Calibration of the nomogram was assessed by
External Validation of Nomogram Predicting Organ-Confined Disease at RP
Urol Int 2014;93:262–268 DOI: 10.1159/000354430
Materials and Methods Population of Study and Design In 2006, the LUNA (Leading Urological No-Profit Foundation Advanced Research) foundation planned the MIRROR (Multicenter Italian Report on Radical Prostatectomy: Outcome and Research) study. Urologic departments all over Italy were involved in evaluating the oncological, surgical, and functional characteristics of PCa and RP outcomes. All patients treated with a RP for PCa in each urology center were consecutively included: all data were stored and E-mailed in a web data manager (CLICON). The study involved 135 Italian centers enrolling 2,408 consecutive patients. Patients with incomplete data or treated by neoadjuvant therapy were excluded from the current study. Overall, 623 men with non-metastatic, clinical high-risk PCa (defined as patients with at least one of the following risk factors according to the European Association of Urology (EAU) guidelines: preoperative PSA value >20 ng/ml and/or cT3 clinical stage, and/or biopsy Gleason score ≥8) were evaluated. All patients were treated with open RP and pelvic lymph node dissection at six urologic tertiary referral centers for treatment of PCa. At each center, clinical staging was defined according to the 2002 TNM staging system; prostate biopsy cores were obtained under transrectal ultrasound guidance. Dedicated genitourinary pathologists assessed biopsy and pathologic grading according to the Gleason grading system. Before surgery, all patients were staged by means of computed tomography and bone scan. According to the endpoint, the presence of a SC disease was defined as pT2–pT3a PCa with negative surgical margins (R0) and no lymph node invasion (pN0). When required, institutional review board at each contributing institution approved the study.
263
comparing its predicted probability of SC with actual presence of SC disease by using Hosmer-Lemeshow goodness-of-fit statistics. We also built a calibration plot by drawing predicted probability by deciles in the x-axis and the observed outcome in the y-axis, which makes the plot a graphical illustration of the Hosmer-Lemeshow goodness-of-fit test. A perfectly accurate nomogram prediction model would result in a plot where the observed and predicted probabilities for given groups fall along the 45° line. The distance between the pairs and the 45° line is a measure of the absolute error of the nomogram. The performance of the nomogram was also evaluated by drawing a receiver operating characteristic (ROC) curve and calculating the area under the curve (AUC). The AUC reflects a test’s ability to discriminate between diseased and non-diseased patients. An AUC of 1 indicates perfect concordance (100% sensitivity and 100% specificity), whereas an AUC of 0.5 indicates a result equal to chance. IBM-SPSS version 20.0 statistical software was used for statistical analysis. All tests were two-sided with a significance level set at 0.05.
Results
Discussion
Descriptive characteristics of patients are listed in table 1, which included patients of the development cohort. At RP, the overall rate of pT2–pT3a was 69.3% (34.8% pT2a–c, 34.5% pT3a), while the overall negative margin rate was 52.1% and the rate of pathological Gleason score 20 ng/ml, cT3, biopsy Gleason score 8–10), remains controversial, even if many authors [11–19] have shown good results using surgery at least as first-line treatment. This is true also considering that pathological downstaging is sometimes significative (5% for Gontero et al. [11], up to 27% for Ward et al. [4], and 11.1% Briganti et al. [9]); in our experience, the incidence of pathological downstaging in cT3 disease is 5.1%, which should mean that in a discrete number of subjects surgery alone is sufficient to cure cancer [20]. Moreover, if we consider patients with SC disease (29.0% in our experience, 37% for Briganti et al.), these subjects will not need adjuvant EBRT and/or HT, while for example all patients with high-risk disease treated with EBRT need at least 3 years of adjuvant hormone manipulation [21]. The debulking effect of surgery permits superior effects of adjuvant therapy [22]; this concept seems to be supported by the only one randomized study between RP plus adjuvant HT versus EBRT plus adjuvant HT by Akakura et al. [23]: survival rates in the surgery group were better than the radiation group, although with a not statistically significant Mearini et al.
Table 1. Descriptive statistics of original development data (Briganti et al. nomogram) and 623 patients with clinical high-risk PCa
treated with RP and pelvic lymph node dissection at Italian institutions; current data were stratified and compared according to tumor extent (i.e. non-specimen confined vs. SC disease)
Number (%) Age, years Mean Median Range PSA, ng/ml Mean Median Range PSA, ng/ml ≤10 10–20 20–50 50–100 Clinical stage cT1 cT2 cT3 Biopsy GS ≤6 7 ≥8 NCCN/EAU risk One Two Three Pathological stage pT2 pT3a pT3b pT4 PSM Overall pT2 disease Pathologic GS ≤6 7 ≥8 Downstaging (cT3 to pT2) Upstaging (cT1–T2 to ≥pT3) Upgrading (cGS 6 to >, cGS 7 to >) LNI
Briganti et al. overall
Current series
p
overall
non-SC
SC
1,366
623
442 (71.0)
181 (29.0)
65.4 66.0 41–88.9
65.2 66.0 30–85
65.1 66.0 45–85
65.3 67.0 30–76
24.9 21.3 0.5–100.0
20.4 11.5 0.6–100
23.1 13.0 0.6–100
14.0 8.4 2.4–96.6
