Bone-related Parameters are the Main Prognostic Factors for Overall Survival in Men with Bone Metastases from Castration-resistant Prostate Cancer

2.1. Study design and patients

The trial was a randomized, double-blind, phase 3 study comparing denosumab and zoledronic acid in 1901 men (clinicaltrials.gov identifier NCT00321620 ) [21] . Patients had CRPC (defined as rising PSA levels despite circulating testosterone levels of <0.50 ng/ml), radiological evidence of bone metastasis, but no known brain metastases, and an ECOG score ≤ 2. Baseline demographic and disease characteristics were similar between treatment groups and no differences in overall survival were observed [21] ; therefore, the present analyses included all trial participants ( Table 1 ).

2.2. Variables

Fifteen baseline study variables were examined for prognostic value.

2.3. Statistical methods

As prespecified in the statistical analysis plan, missing baseline values for the following variables were imputed using the mean of the pooled observed data: pain, time from cancer diagnosis to bone metastasis, time from bone metastasis to randomization, Gleason score, and time from cancer diagnosis to first evidence of metastatic disease. Missing baseline values for uNTx, BSAP, alkaline phosphatase, and hemoglobin were not imputed and were excluded. There were no missing values for previous SRE, PSA, current chemotherapy, ECOG score, visceral metastases, or age. Out of 1901 patients in the study, 1745 (92%) were included in this analysis.

Survival time was defined as the time interval (in months) from randomization to death. If a patient was alive by the primary analysis cutoff date or was lost to follow-up by the analysis cutoff date, then survival time was censored at the last contact date or the analysis cutoff date, whichever was first. Kaplan-Meier estimates were presented graphically for survival distribution by each baseline categorical covariate. A Cox proportional hazards model was used to assess the prognostic significance of baseline covariates in univariate and multivariate analyses. In addition, a Cox proportional hazards model with backward selection based on the full dataset was used to select the potential prognostic factors included in the final model. The baseline covariates withp < 0.05 from those selection strategies were included in the final model. Other model selection strategies (such as best subset, forward, and stepwise selections) based on the full dataset were also used to verify the selected prognostic factors in the final model. A concordance index (C-index; range 0–1) [22] was used to assess model predictive discrimination. This index estimates the probability of concordance between predicted and observed responses, with higher index values associated with better discrimination. Internal validation of the final model was done using nonparametric bootstrap resampling with 500 datasets. The same variable selection procedure was repeated for each of the bootstrapped datasets and the selected variables were retained in the final model if the covariates were included in more than half the final models in the 500 bootstrapped samples.

To evaluate the robustness of the process in constructing the prediction model and validating the model based on the full data set, the following steps were implemented [23] :

• 1. 1000 random learning datasets (based on 70% of the full dataset) and 1000 random validation datasets (based on 30% of the full dataset) were generated with 1000 different random seeds.
• 2. The prediction model (with variable selection) was constructed based on the learning dataset using a Cox proportional hazards model with backward selection as described above. Then the final prediction model was fitted into the validation dataset.
• 3. The same model-building and validation process as above was applied to the 1000 learning datasets and 1000 validation datasets.
• 4. C-index values were generated from each dataset. If the distributions of the two sets of C-index values were similar, the robustness of the model selection and validation process was confirmed and the process was then applied to the full dataset.

The final Cox proportional hazards model after validation was used to create nomograms (with and without uNTx and BSAP). The statistical analyses for model development, as well as model validation and nomogram creation, were done using SAS version 9.2 software and the R 3.0.1 rms package, respectively.

3.1. Patients

Baseline demographics and disease characteristics for the overall study population are shown in Table 1 . Median (95% confidence interval [CI]) overall survival time was 20 (18, 21) mo.

3.2. Univariate analyses

Point estimates for each variable are shown in Table 2 . The bone-specific markers alkaline phosphatase, BSAP, and uNTx were each found to be predictors of survival (allp < 0.0001; Fig. 1 ). Two other bone parameters, pain and history of SRE, were also significant in the univariate analysis (bothp < 0.0001). PSA <10 ng/ml, absence of visceral metastases, ECOG ≤1, and hemoglobin >128 g/l were each significant prognosticators of longer survival (allp < 0.0001; Fig. 2 ). PSA with a 20-ng/ml cutoff or median cutoff also yielded highly significant results (data not shown); however, we chose the 10-ng/ml cutoff as it matched the stratification factor prespecified in the protocol, which was considered the best clinically meaningful cutoff at the time the trial was designed. Longer time from diagnosis of primary cancer to first evidence of metastatic disease, longer time from diagnosis of primary cancer to first bone metastasis, and longer time from first bone metastasis to randomization were also significant predictors of longer survival in the univariate analysis (p≤ 0.001). Younger age was an additional significant predictor of longer survival. Although patients with lower Gleason scores (2–6 and 7) had numerically longer survival than patients with higher Gleason scores (8–10), the difference was not statistically significant (p = 0.44 and 0.33, respectively).

Fig. 1 Univariate analyses of bone-specific factors. (A) Alkaline phosphatase (ALP ≤ median [143 U/l] vs > median). (B) Bone-specific alkaline phosphatase (BSAP <146 U/l vs ≥146 U/l). (C) Urinary N-telopeptide (uNTx ≤50 nmol/mmol vs >50 nmol/mmol). (D) Previous skeletal related event (SRE; no vs yes). (E) Worst pain absent or mild (Brief Pain Inventory—Short Form [BPI-SF] score ≤4 vs >4).

Fig. 2 Univariate analyses of disease characteristics and laboratory values. (A) Prostate-specific antigen (PSA; <10 ng/ml vs ≥10 ng/ml). (B) Presence of visceral metastases (no vs yes). (C) Eastern Co-operative Oncology Group (ECOG) performance status (≤1 vs 2). (D) Hemoglobin (≤median [128 g/l] vs >median).

3.3. Multivariate analysis

The process used to construct and validate the prediction model was found to be robust and was, therefore, applied to the full dataset: mean (standard deviation [SD]) C-index values were 0.7171 (0.007) for the learning datasets and 0.7128 (0.013) for the validation datasets. Among the 15 parameters tested in the multivariate analysis, Gleason score, current chemotherapy, and time from diagnosis of primary cancer to metastases were not significant and were excluded from the final model. Bone-related parameters were all highly significantly correlated to overall survival (allp < 0.001). The variables associated with longer survival included uNTx ≤50 nmol/mmol, BSAP <146 U/l, alkaline phosphatase ≤143 U/l, no prior SRE, mild or no pain, longer time from cancer diagnosis to diagnosis of bone metastases, longer time from diagnosis of bone metastases to randomization, PSA <10 ng/ml, absence of visceral metastases, ECOG score ≤1, younger age, and hemoglobin >128 g/l. Results of the 12 significant variables selected for the final model are shown in Table 3 . For the continuous variables (age, time from cancer diagnosis to diagnosis of bone metastases, and time from bone metastasis to randomization), the hazard ratios were close to one, yet highly significant, because they reflect the change in the hazard for any increase of one unit. For example, the hazard ratio for age was 1.012, indicating that a 63-yr-old patient had a 1.2% lower risk of death compared to a 64-yr-old patient. For an age difference of 15 yr (eg, age 60 yr vs age 75 yr), the hazard ratio increases to 1.191, indicating that the younger patient has a 19% lower risk of death. The corrected C-index value of the final model was 0.72, indicating robust discrimination.

As BSAP and uNTx are not commonly measured in clinical practice, we generated an additional model excluding these variables. The same model selection process was implemented (ie, univariable and multivariable analyses with various selection strategies) and all 10 variables were included in the final model. In the 10-variable model, the correlations of each parameter with overall survival were in the same direction and of similar magnitude as in the 12-variable model. The corrected C-index value of the final 10-variable model was approximately 0.71, indicating slightly less discrimination compared with the 12-variable model.

3.4. Nomograms

A clinical tool, or nomogram, for the estimation of survival for an individual patient is shown in Supplementary Figure 1 (12-variable nomogram). A second 10-variable nomogram excluding BSAP and uNTx is shown in Figure 3 . The precision of the tool is dependent on the full set of parameters; thus, data from all variables should be included to achieve the most accurate prediction.

Fig. 3 Ten variable nomograms for the prediction of 12-mo and 24-mo overall survival probability and median overall survival in months. To use this nomogram, complete the following steps: 1. Identify the scale for the first variable and draw a vertical line at the value until it intersects the “Points” axis; read off the corresponding risk score (points) for that variable. 2. Repeat this process for each variable. 3. Calculate the total risk score, that is, the sum of the points for each variable. 4. Locate the total risk score on the “Total points” axis and draw a vertical line downwards to identify the predicted survival probabilities. ^aIn one patient, the diagnosis of bone metastases occurred months before diagnosis of the primary tumor.

4.1. Limitations

Since clinical trial participation is an independent positive predictor of survival [27] , our nomograms may overestimate survival in patients not participating in a clinical trial. We acknowledge that the utility of nomograms to predict survival in routine patient management is as yet untested, and the complexity of these prognostic tools may be a barrier to their application. Use of nonlinear terms for covariates could have increased the magnitude of prediction in the model; however, we chose to use a less complicated model more easily used by clinicians. As the dataset contains only patients with an ECOG score of 1 or 2, the model may not be applicable for scores >2. Quantitation of disease burden such as number of bone metastases could not be taken into consideration, because these data were not recorded. LDH, a known prognostic factor[9] and [12], was not collected in this study. The cutoff points for the variables used in our model were based on the literature; other cutoff points may have yielded different results. In addition, the categorization of continuous variables may have caused information loss and may have introduced error through the assumption that all values of a variable within a category are equivalent. Our dataset is more contemporary than those previously published; however, the relevance of our model for patients receiving the new generation of treatments requires further investigation.

2.1. Study design and patients

The trial was a randomized, double-blind, phase 3 study comparing denosumab and zoledronic acid in 1901 men (clinicaltrials.gov identifier NCT00321620 ) [21] . Patients had CRPC (defined as rising PSA levels despite circulating testosterone levels of <0.50 ng/ml), radiological evidence of bone metastasis, but no known brain metastases, and an ECOG score ≤ 2. Baseline demographic and disease characteristics were similar between treatment groups and no differences in overall survival were observed [21] ; therefore, the present analyses included all trial participants ( Table 1 ).

2.2. Variables

Fifteen baseline study variables were examined for prognostic value.

2.3. Statistical methods

As prespecified in the statistical analysis plan, missing baseline values for the following variables were imputed using the mean of the pooled observed data: pain, time from cancer diagnosis to bone metastasis, time from bone metastasis to randomization, Gleason score, and time from cancer diagnosis to first evidence of metastatic disease. Missing baseline values for uNTx, BSAP, alkaline phosphatase, and hemoglobin were not imputed and were excluded. There were no missing values for previous SRE, PSA, current chemotherapy, ECOG score, visceral metastases, or age. Out of 1901 patients in the study, 1745 (92%) were included in this analysis.

Survival time was defined as the time interval (in months) from randomization to death. If a patient was alive by the primary analysis cutoff date or was lost to follow-up by the analysis cutoff date, then survival time was censored at the last contact date or the analysis cutoff date, whichever was first. Kaplan-Meier estimates were presented graphically for survival distribution by each baseline categorical covariate. A Cox proportional hazards model was used to assess the prognostic significance of baseline covariates in univariate and multivariate analyses. In addition, a Cox proportional hazards model with backward selection based on the full dataset was used to select the potential prognostic factors included in the final model. The baseline covariates withp < 0.05 from those selection strategies were included in the final model. Other model selection strategies (such as best subset, forward, and stepwise selections) based on the full dataset were also used to verify the selected prognostic factors in the final model. A concordance index (C-index; range 0–1) [22] was used to assess model predictive discrimination. This index estimates the probability of concordance between predicted and observed responses, with higher index values associated with better discrimination. Internal validation of the final model was done using nonparametric bootstrap resampling with 500 datasets. The same variable selection procedure was repeated for each of the bootstrapped datasets and the selected variables were retained in the final model if the covariates were included in more than half the final models in the 500 bootstrapped samples.

To evaluate the robustness of the process in constructing the prediction model and validating the model based on the full data set, the following steps were implemented [23] :

• 1. 1000 random learning datasets (based on 70% of the full dataset) and 1000 random validation datasets (based on 30% of the full dataset) were generated with 1000 different random seeds.
• 2. The prediction model (with variable selection) was constructed based on the learning dataset using a Cox proportional hazards model with backward selection as described above. Then the final prediction model was fitted into the validation dataset.
• 3. The same model-building and validation process as above was applied to the 1000 learning datasets and 1000 validation datasets.
• 4. C-index values were generated from each dataset. If the distributions of the two sets of C-index values were similar, the robustness of the model selection and validation process was confirmed and the process was then applied to the full dataset.

The final Cox proportional hazards model after validation was used to create nomograms (with and without uNTx and BSAP). The statistical analyses for model development, as well as model validation and nomogram creation, were done using SAS version 9.2 software and the R 3.0.1 rms package, respectively.

3.1. Patients

Baseline demographics and disease characteristics for the overall study population are shown in Table 1 . Median (95% confidence interval [CI]) overall survival time was 20 (18, 21) mo.

3.2. Univariate analyses

Point estimates for each variable are shown in Table 2 . The bone-specific markers alkaline phosphatase, BSAP, and uNTx were each found to be predictors of survival (allp < 0.0001; Fig. 1 ). Two other bone parameters, pain and history of SRE, were also significant in the univariate analysis (bothp < 0.0001). PSA <10 ng/ml, absence of visceral metastases, ECOG ≤1, and hemoglobin >128 g/l were each significant prognosticators of longer survival (allp < 0.0001; Fig. 2 ). PSA with a 20-ng/ml cutoff or median cutoff also yielded highly significant results (data not shown); however, we chose the 10-ng/ml cutoff as it matched the stratification factor prespecified in the protocol, which was considered the best clinically meaningful cutoff at the time the trial was designed. Longer time from diagnosis of primary cancer to first evidence of metastatic disease, longer time from diagnosis of primary cancer to first bone metastasis, and longer time from first bone metastasis to randomization were also significant predictors of longer survival in the univariate analysis (p≤ 0.001). Younger age was an additional significant predictor of longer survival. Although patients with lower Gleason scores (2–6 and 7) had numerically longer survival than patients with higher Gleason scores (8–10), the difference was not statistically significant (p = 0.44 and 0.33, respectively).

Fig. 1 Univariate analyses of bone-specific factors. (A) Alkaline phosphatase (ALP ≤ median [143 U/l] vs > median). (B) Bone-specific alkaline phosphatase (BSAP <146 U/l vs ≥146 U/l). (C) Urinary N-telopeptide (uNTx ≤50 nmol/mmol vs >50 nmol/mmol). (D) Previous skeletal related event (SRE; no vs yes). (E) Worst pain absent or mild (Brief Pain Inventory—Short Form [BPI-SF] score ≤4 vs >4).

Fig. 2 Univariate analyses of disease characteristics and laboratory values. (A) Prostate-specific antigen (PSA; <10 ng/ml vs ≥10 ng/ml). (B) Presence of visceral metastases (no vs yes). (C) Eastern Co-operative Oncology Group (ECOG) performance status (≤1 vs 2). (D) Hemoglobin (≤median [128 g/l] vs >median).

3.3. Multivariate analysis

The process used to construct and validate the prediction model was found to be robust and was, therefore, applied to the full dataset: mean (standard deviation [SD]) C-index values were 0.7171 (0.007) for the learning datasets and 0.7128 (0.013) for the validation datasets. Among the 15 parameters tested in the multivariate analysis, Gleason score, current chemotherapy, and time from diagnosis of primary cancer to metastases were not significant and were excluded from the final model. Bone-related parameters were all highly significantly correlated to overall survival (allp < 0.001). The variables associated with longer survival included uNTx ≤50 nmol/mmol, BSAP <146 U/l, alkaline phosphatase ≤143 U/l, no prior SRE, mild or no pain, longer time from cancer diagnosis to diagnosis of bone metastases, longer time from diagnosis of bone metastases to randomization, PSA <10 ng/ml, absence of visceral metastases, ECOG score ≤1, younger age, and hemoglobin >128 g/l. Results of the 12 significant variables selected for the final model are shown in Table 3 . For the continuous variables (age, time from cancer diagnosis to diagnosis of bone metastases, and time from bone metastasis to randomization), the hazard ratios were close to one, yet highly significant, because they reflect the change in the hazard for any increase of one unit. For example, the hazard ratio for age was 1.012, indicating that a 63-yr-old patient had a 1.2% lower risk of death compared to a 64-yr-old patient. For an age difference of 15 yr (eg, age 60 yr vs age 75 yr), the hazard ratio increases to 1.191, indicating that the younger patient has a 19% lower risk of death. The corrected C-index value of the final model was 0.72, indicating robust discrimination.

As BSAP and uNTx are not commonly measured in clinical practice, we generated an additional model excluding these variables. The same model selection process was implemented (ie, univariable and multivariable analyses with various selection strategies) and all 10 variables were included in the final model. In the 10-variable model, the correlations of each parameter with overall survival were in the same direction and of similar magnitude as in the 12-variable model. The corrected C-index value of the final 10-variable model was approximately 0.71, indicating slightly less discrimination compared with the 12-variable model.

3.4. Nomograms

A clinical tool, or nomogram, for the estimation of survival for an individual patient is shown in Supplementary Figure 1 (12-variable nomogram). A second 10-variable nomogram excluding BSAP and uNTx is shown in Figure 3 . The precision of the tool is dependent on the full set of parameters; thus, data from all variables should be included to achieve the most accurate prediction.

Fig. 3 Ten variable nomograms for the prediction of 12-mo and 24-mo overall survival probability and median overall survival in months. To use this nomogram, complete the following steps: 1. Identify the scale for the first variable and draw a vertical line at the value until it intersects the “Points” axis; read off the corresponding risk score (points) for that variable. 2. Repeat this process for each variable. 3. Calculate the total risk score, that is, the sum of the points for each variable. 4. Locate the total risk score on the “Total points” axis and draw a vertical line downwards to identify the predicted survival probabilities. ^aIn one patient, the diagnosis of bone metastases occurred months before diagnosis of the primary tumor.

4.1. Limitations

Since clinical trial participation is an independent positive predictor of survival [27] , our nomograms may overestimate survival in patients not participating in a clinical trial. We acknowledge that the utility of nomograms to predict survival in routine patient management is as yet untested, and the complexity of these prognostic tools may be a barrier to their application. Use of nonlinear terms for covariates could have increased the magnitude of prediction in the model; however, we chose to use a less complicated model more easily used by clinicians. As the dataset contains only patients with an ECOG score of 1 or 2, the model may not be applicable for scores >2. Quantitation of disease burden such as number of bone metastases could not be taken into consideration, because these data were not recorded. LDH, a known prognostic factor[9] and [12], was not collected in this study. The cutoff points for the variables used in our model were based on the literature; other cutoff points may have yielded different results. In addition, the categorization of continuous variables may have caused information loss and may have introduced error through the assumption that all values of a variable within a category are equivalent. Our dataset is more contemporary than those previously published; however, the relevance of our model for patients receiving the new generation of treatments requires further investigation.

2.1. Study design and patients

The trial was a randomized, double-blind, phase 3 study comparing denosumab and zoledronic acid in 1901 men (clinicaltrials.gov identifier NCT00321620 ) [21] . Patients had CRPC (defined as rising PSA levels despite circulating testosterone levels of <0.50 ng/ml), radiological evidence of bone metastasis, but no known brain metastases, and an ECOG score ≤ 2. Baseline demographic and disease characteristics were similar between treatment groups and no differences in overall survival were observed [21] ; therefore, the present analyses included all trial participants ( Table 1 ).

2.2. Variables

Fifteen baseline study variables were examined for prognostic value.

2.3. Statistical methods

As prespecified in the statistical analysis plan, missing baseline values for the following variables were imputed using the mean of the pooled observed data: pain, time from cancer diagnosis to bone metastasis, time from bone metastasis to randomization, Gleason score, and time from cancer diagnosis to first evidence of metastatic disease. Missing baseline values for uNTx, BSAP, alkaline phosphatase, and hemoglobin were not imputed and were excluded. There were no missing values for previous SRE, PSA, current chemotherapy, ECOG score, visceral metastases, or age. Out of 1901 patients in the study, 1745 (92%) were included in this analysis.

Survival time was defined as the time interval (in months) from randomization to death. If a patient was alive by the primary analysis cutoff date or was lost to follow-up by the analysis cutoff date, then survival time was censored at the last contact date or the analysis cutoff date, whichever was first. Kaplan-Meier estimates were presented graphically for survival distribution by each baseline categorical covariate. A Cox proportional hazards model was used to assess the prognostic significance of baseline covariates in univariate and multivariate analyses. In addition, a Cox proportional hazards model with backward selection based on the full dataset was used to select the potential prognostic factors included in the final model. The baseline covariates withp < 0.05 from those selection strategies were included in the final model. Other model selection strategies (such as best subset, forward, and stepwise selections) based on the full dataset were also used to verify the selected prognostic factors in the final model. A concordance index (C-index; range 0–1) [22] was used to assess model predictive discrimination. This index estimates the probability of concordance between predicted and observed responses, with higher index values associated with better discrimination. Internal validation of the final model was done using nonparametric bootstrap resampling with 500 datasets. The same variable selection procedure was repeated for each of the bootstrapped datasets and the selected variables were retained in the final model if the covariates were included in more than half the final models in the 500 bootstrapped samples.

To evaluate the robustness of the process in constructing the prediction model and validating the model based on the full data set, the following steps were implemented [23] :

• 1. 1000 random learning datasets (based on 70% of the full dataset) and 1000 random validation datasets (based on 30% of the full dataset) were generated with 1000 different random seeds.
• 2. The prediction model (with variable selection) was constructed based on the learning dataset using a Cox proportional hazards model with backward selection as described above. Then the final prediction model was fitted into the validation dataset.
• 3. The same model-building and validation process as above was applied to the 1000 learning datasets and 1000 validation datasets.
• 4. C-index values were generated from each dataset. If the distributions of the two sets of C-index values were similar, the robustness of the model selection and validation process was confirmed and the process was then applied to the full dataset.

The final Cox proportional hazards model after validation was used to create nomograms (with and without uNTx and BSAP). The statistical analyses for model development, as well as model validation and nomogram creation, were done using SAS version 9.2 software and the R 3.0.1 rms package, respectively.

3.1. Patients

Baseline demographics and disease characteristics for the overall study population are shown in Table 1 . Median (95% confidence interval [CI]) overall survival time was 20 (18, 21) mo.

3.2. Univariate analyses

Point estimates for each variable are shown in Table 2 . The bone-specific markers alkaline phosphatase, BSAP, and uNTx were each found to be predictors of survival (allp < 0.0001; Fig. 1 ). Two other bone parameters, pain and history of SRE, were also significant in the univariate analysis (bothp < 0.0001). PSA <10 ng/ml, absence of visceral metastases, ECOG ≤1, and hemoglobin >128 g/l were each significant prognosticators of longer survival (allp < 0.0001; Fig. 2 ). PSA with a 20-ng/ml cutoff or median cutoff also yielded highly significant results (data not shown); however, we chose the 10-ng/ml cutoff as it matched the stratification factor prespecified in the protocol, which was considered the best clinically meaningful cutoff at the time the trial was designed. Longer time from diagnosis of primary cancer to first evidence of metastatic disease, longer time from diagnosis of primary cancer to first bone metastasis, and longer time from first bone metastasis to randomization were also significant predictors of longer survival in the univariate analysis (p≤ 0.001). Younger age was an additional significant predictor of longer survival. Although patients with lower Gleason scores (2–6 and 7) had numerically longer survival than patients with higher Gleason scores (8–10), the difference was not statistically significant (p = 0.44 and 0.33, respectively).

Fig. 1 Univariate analyses of bone-specific factors. (A) Alkaline phosphatase (ALP ≤ median [143 U/l] vs > median). (B) Bone-specific alkaline phosphatase (BSAP <146 U/l vs ≥146 U/l). (C) Urinary N-telopeptide (uNTx ≤50 nmol/mmol vs >50 nmol/mmol). (D) Previous skeletal related event (SRE; no vs yes). (E) Worst pain absent or mild (Brief Pain Inventory—Short Form [BPI-SF] score ≤4 vs >4).

Fig. 2 Univariate analyses of disease characteristics and laboratory values. (A) Prostate-specific antigen (PSA; <10 ng/ml vs ≥10 ng/ml). (B) Presence of visceral metastases (no vs yes). (C) Eastern Co-operative Oncology Group (ECOG) performance status (≤1 vs 2). (D) Hemoglobin (≤median [128 g/l] vs >median).

3.3. Multivariate analysis

The process used to construct and validate the prediction model was found to be robust and was, therefore, applied to the full dataset: mean (standard deviation [SD]) C-index values were 0.7171 (0.007) for the learning datasets and 0.7128 (0.013) for the validation datasets. Among the 15 parameters tested in the multivariate analysis, Gleason score, current chemotherapy, and time from diagnosis of primary cancer to metastases were not significant and were excluded from the final model. Bone-related parameters were all highly significantly correlated to overall survival (allp < 0.001). The variables associated with longer survival included uNTx ≤50 nmol/mmol, BSAP <146 U/l, alkaline phosphatase ≤143 U/l, no prior SRE, mild or no pain, longer time from cancer diagnosis to diagnosis of bone metastases, longer time from diagnosis of bone metastases to randomization, PSA <10 ng/ml, absence of visceral metastases, ECOG score ≤1, younger age, and hemoglobin >128 g/l. Results of the 12 significant variables selected for the final model are shown in Table 3 . For the continuous variables (age, time from cancer diagnosis to diagnosis of bone metastases, and time from bone metastasis to randomization), the hazard ratios were close to one, yet highly significant, because they reflect the change in the hazard for any increase of one unit. For example, the hazard ratio for age was 1.012, indicating that a 63-yr-old patient had a 1.2% lower risk of death compared to a 64-yr-old patient. For an age difference of 15 yr (eg, age 60 yr vs age 75 yr), the hazard ratio increases to 1.191, indicating that the younger patient has a 19% lower risk of death. The corrected C-index value of the final model was 0.72, indicating robust discrimination.

As BSAP and uNTx are not commonly measured in clinical practice, we generated an additional model excluding these variables. The same model selection process was implemented (ie, univariable and multivariable analyses with various selection strategies) and all 10 variables were included in the final model. In the 10-variable model, the correlations of each parameter with overall survival were in the same direction and of similar magnitude as in the 12-variable model. The corrected C-index value of the final 10-variable model was approximately 0.71, indicating slightly less discrimination compared with the 12-variable model.

3.4. Nomograms

A clinical tool, or nomogram, for the estimation of survival for an individual patient is shown in Supplementary Figure 1 (12-variable nomogram). A second 10-variable nomogram excluding BSAP and uNTx is shown in Figure 3 . The precision of the tool is dependent on the full set of parameters; thus, data from all variables should be included to achieve the most accurate prediction.

Fig. 3 Ten variable nomograms for the prediction of 12-mo and 24-mo overall survival probability and median overall survival in months. To use this nomogram, complete the following steps: 1. Identify the scale for the first variable and draw a vertical line at the value until it intersects the “Points” axis; read off the corresponding risk score (points) for that variable. 2. Repeat this process for each variable. 3. Calculate the total risk score, that is, the sum of the points for each variable. 4. Locate the total risk score on the “Total points” axis and draw a vertical line downwards to identify the predicted survival probabilities. ^aIn one patient, the diagnosis of bone metastases occurred months before diagnosis of the primary tumor.

4.1. Limitations

Since clinical trial participation is an independent positive predictor of survival [27] , our nomograms may overestimate survival in patients not participating in a clinical trial. We acknowledge that the utility of nomograms to predict survival in routine patient management is as yet untested, and the complexity of these prognostic tools may be a barrier to their application. Use of nonlinear terms for covariates could have increased the magnitude of prediction in the model; however, we chose to use a less complicated model more easily used by clinicians. As the dataset contains only patients with an ECOG score of 1 or 2, the model may not be applicable for scores >2. Quantitation of disease burden such as number of bone metastases could not be taken into consideration, because these data were not recorded. LDH, a known prognostic factor[9] and [12], was not collected in this study. The cutoff points for the variables used in our model were based on the literature; other cutoff points may have yielded different results. In addition, the categorization of continuous variables may have caused information loss and may have introduced error through the assumption that all values of a variable within a category are equivalent. Our dataset is more contemporary than those previously published; however, the relevance of our model for patients receiving the new generation of treatments requires further investigation.

2.1. Study design and patients

The trial was a randomized, double-blind, phase 3 study comparing denosumab and zoledronic acid in 1901 men (clinicaltrials.gov identifier NCT00321620 ) [21] . Patients had CRPC (defined as rising PSA levels despite circulating testosterone levels of <0.50 ng/ml), radiological evidence of bone metastasis, but no known brain metastases, and an ECOG score ≤ 2. Baseline demographic and disease characteristics were similar between treatment groups and no differences in overall survival were observed [21] ; therefore, the present analyses included all trial participants ( Table 1 ).

2.2. Variables

Fifteen baseline study variables were examined for prognostic value.

2.3. Statistical methods

As prespecified in the statistical analysis plan, missing baseline values for the following variables were imputed using the mean of the pooled observed data: pain, time from cancer diagnosis to bone metastasis, time from bone metastasis to randomization, Gleason score, and time from cancer diagnosis to first evidence of metastatic disease. Missing baseline values for uNTx, BSAP, alkaline phosphatase, and hemoglobin were not imputed and were excluded. There were no missing values for previous SRE, PSA, current chemotherapy, ECOG score, visceral metastases, or age. Out of 1901 patients in the study, 1745 (92%) were included in this analysis.

Survival time was defined as the time interval (in months) from randomization to death. If a patient was alive by the primary analysis cutoff date or was lost to follow-up by the analysis cutoff date, then survival time was censored at the last contact date or the analysis cutoff date, whichever was first. Kaplan-Meier estimates were presented graphically for survival distribution by each baseline categorical covariate. A Cox proportional hazards model was used to assess the prognostic significance of baseline covariates in univariate and multivariate analyses. In addition, a Cox proportional hazards model with backward selection based on the full dataset was used to select the potential prognostic factors included in the final model. The baseline covariates withp < 0.05 from those selection strategies were included in the final model. Other model selection strategies (such as best subset, forward, and stepwise selections) based on the full dataset were also used to verify the selected prognostic factors in the final model. A concordance index (C-index; range 0–1) [22] was used to assess model predictive discrimination. This index estimates the probability of concordance between predicted and observed responses, with higher index values associated with better discrimination. Internal validation of the final model was done using nonparametric bootstrap resampling with 500 datasets. The same variable selection procedure was repeated for each of the bootstrapped datasets and the selected variables were retained in the final model if the covariates were included in more than half the final models in the 500 bootstrapped samples.

To evaluate the robustness of the process in constructing the prediction model and validating the model based on the full data set, the following steps were implemented [23] :

• 1. 1000 random learning datasets (based on 70% of the full dataset) and 1000 random validation datasets (based on 30% of the full dataset) were generated with 1000 different random seeds.
• 2. The prediction model (with variable selection) was constructed based on the learning dataset using a Cox proportional hazards model with backward selection as described above. Then the final prediction model was fitted into the validation dataset.
• 3. The same model-building and validation process as above was applied to the 1000 learning datasets and 1000 validation datasets.
• 4. C-index values were generated from each dataset. If the distributions of the two sets of C-index values were similar, the robustness of the model selection and validation process was confirmed and the process was then applied to the full dataset.

The final Cox proportional hazards model after validation was used to create nomograms (with and without uNTx and BSAP). The statistical analyses for model development, as well as model validation and nomogram creation, were done using SAS version 9.2 software and the R 3.0.1 rms package, respectively.

3.1. Patients

Baseline demographics and disease characteristics for the overall study population are shown in Table 1 . Median (95% confidence interval [CI]) overall survival time was 20 (18, 21) mo.

3.2. Univariate analyses

Point estimates for each variable are shown in Table 2 . The bone-specific markers alkaline phosphatase, BSAP, and uNTx were each found to be predictors of survival (allp < 0.0001; Fig. 1 ). Two other bone parameters, pain and history of SRE, were also significant in the univariate analysis (bothp < 0.0001). PSA <10 ng/ml, absence of visceral metastases, ECOG ≤1, and hemoglobin >128 g/l were each significant prognosticators of longer survival (allp < 0.0001; Fig. 2 ). PSA with a 20-ng/ml cutoff or median cutoff also yielded highly significant results (data not shown); however, we chose the 10-ng/ml cutoff as it matched the stratification factor prespecified in the protocol, which was considered the best clinically meaningful cutoff at the time the trial was designed. Longer time from diagnosis of primary cancer to first evidence of metastatic disease, longer time from diagnosis of primary cancer to first bone metastasis, and longer time from first bone metastasis to randomization were also significant predictors of longer survival in the univariate analysis (p≤ 0.001). Younger age was an additional significant predictor of longer survival. Although patients with lower Gleason scores (2–6 and 7) had numerically longer survival than patients with higher Gleason scores (8–10), the difference was not statistically significant (p = 0.44 and 0.33, respectively).

Fig. 1 Univariate analyses of bone-specific factors. (A) Alkaline phosphatase (ALP ≤ median [143 U/l] vs > median). (B) Bone-specific alkaline phosphatase (BSAP <146 U/l vs ≥146 U/l). (C) Urinary N-telopeptide (uNTx ≤50 nmol/mmol vs >50 nmol/mmol). (D) Previous skeletal related event (SRE; no vs yes). (E) Worst pain absent or mild (Brief Pain Inventory—Short Form [BPI-SF] score ≤4 vs >4).

Fig. 2 Univariate analyses of disease characteristics and laboratory values. (A) Prostate-specific antigen (PSA; <10 ng/ml vs ≥10 ng/ml). (B) Presence of visceral metastases (no vs yes). (C) Eastern Co-operative Oncology Group (ECOG) performance status (≤1 vs 2). (D) Hemoglobin (≤median [128 g/l] vs >median).

3.3. Multivariate analysis

The process used to construct and validate the prediction model was found to be robust and was, therefore, applied to the full dataset: mean (standard deviation [SD]) C-index values were 0.7171 (0.007) for the learning datasets and 0.7128 (0.013) for the validation datasets. Among the 15 parameters tested in the multivariate analysis, Gleason score, current chemotherapy, and time from diagnosis of primary cancer to metastases were not significant and were excluded from the final model. Bone-related parameters were all highly significantly correlated to overall survival (allp < 0.001). The variables associated with longer survival included uNTx ≤50 nmol/mmol, BSAP <146 U/l, alkaline phosphatase ≤143 U/l, no prior SRE, mild or no pain, longer time from cancer diagnosis to diagnosis of bone metastases, longer time from diagnosis of bone metastases to randomization, PSA <10 ng/ml, absence of visceral metastases, ECOG score ≤1, younger age, and hemoglobin >128 g/l. Results of the 12 significant variables selected for the final model are shown in Table 3 . For the continuous variables (age, time from cancer diagnosis to diagnosis of bone metastases, and time from bone metastasis to randomization), the hazard ratios were close to one, yet highly significant, because they reflect the change in the hazard for any increase of one unit. For example, the hazard ratio for age was 1.012, indicating that a 63-yr-old patient had a 1.2% lower risk of death compared to a 64-yr-old patient. For an age difference of 15 yr (eg, age 60 yr vs age 75 yr), the hazard ratio increases to 1.191, indicating that the younger patient has a 19% lower risk of death. The corrected C-index value of the final model was 0.72, indicating robust discrimination.

As BSAP and uNTx are not commonly measured in clinical practice, we generated an additional model excluding these variables. The same model selection process was implemented (ie, univariable and multivariable analyses with various selection strategies) and all 10 variables were included in the final model. In the 10-variable model, the correlations of each parameter with overall survival were in the same direction and of similar magnitude as in the 12-variable model. The corrected C-index value of the final 10-variable model was approximately 0.71, indicating slightly less discrimination compared with the 12-variable model.

3.4. Nomograms

A clinical tool, or nomogram, for the estimation of survival for an individual patient is shown in Supplementary Figure 1 (12-variable nomogram). A second 10-variable nomogram excluding BSAP and uNTx is shown in Figure 3 . The precision of the tool is dependent on the full set of parameters; thus, data from all variables should be included to achieve the most accurate prediction.

Fig. 3 Ten variable nomograms for the prediction of 12-mo and 24-mo overall survival probability and median overall survival in months. To use this nomogram, complete the following steps: 1. Identify the scale for the first variable and draw a vertical line at the value until it intersects the “Points” axis; read off the corresponding risk score (points) for that variable. 2. Repeat this process for each variable. 3. Calculate the total risk score, that is, the sum of the points for each variable. 4. Locate the total risk score on the “Total points” axis and draw a vertical line downwards to identify the predicted survival probabilities. ^aIn one patient, the diagnosis of bone metastases occurred months before diagnosis of the primary tumor.

4.1. Limitations

Since clinical trial participation is an independent positive predictor of survival [27] , our nomograms may overestimate survival in patients not participating in a clinical trial. We acknowledge that the utility of nomograms to predict survival in routine patient management is as yet untested, and the complexity of these prognostic tools may be a barrier to their application. Use of nonlinear terms for covariates could have increased the magnitude of prediction in the model; however, we chose to use a less complicated model more easily used by clinicians. As the dataset contains only patients with an ECOG score of 1 or 2, the model may not be applicable for scores >2. Quantitation of disease burden such as number of bone metastases could not be taken into consideration, because these data were not recorded. LDH, a known prognostic factor[9] and [12], was not collected in this study. The cutoff points for the variables used in our model were based on the literature; other cutoff points may have yielded different results. In addition, the categorization of continuous variables may have caused information loss and may have introduced error through the assumption that all values of a variable within a category are equivalent. Our dataset is more contemporary than those previously published; however, the relevance of our model for patients receiving the new generation of treatments requires further investigation.

2.1. Study design and patients

The trial was a randomized, double-blind, phase 3 study comparing denosumab and zoledronic acid in 1901 men (clinicaltrials.gov identifier NCT00321620 ) [21] . Patients had CRPC (defined as rising PSA levels despite circulating testosterone levels of <0.50 ng/ml), radiological evidence of bone metastasis, but no known brain metastases, and an ECOG score ≤ 2. Baseline demographic and disease characteristics were similar between treatment groups and no differences in overall survival were observed [21] ; therefore, the present analyses included all trial participants ( Table 1 ).

2.2. Variables

Fifteen baseline study variables were examined for prognostic value.

2.3. Statistical methods

As prespecified in the statistical analysis plan, missing baseline values for the following variables were imputed using the mean of the pooled observed data: pain, time from cancer diagnosis to bone metastasis, time from bone metastasis to randomization, Gleason score, and time from cancer diagnosis to first evidence of metastatic disease. Missing baseline values for uNTx, BSAP, alkaline phosphatase, and hemoglobin were not imputed and were excluded. There were no missing values for previous SRE, PSA, current chemotherapy, ECOG score, visceral metastases, or age. Out of 1901 patients in the study, 1745 (92%) were included in this analysis.

Survival time was defined as the time interval (in months) from randomization to death. If a patient was alive by the primary analysis cutoff date or was lost to follow-up by the analysis cutoff date, then survival time was censored at the last contact date or the analysis cutoff date, whichever was first. Kaplan-Meier estimates were presented graphically for survival distribution by each baseline categorical covariate. A Cox proportional hazards model was used to assess the prognostic significance of baseline covariates in univariate and multivariate analyses. In addition, a Cox proportional hazards model with backward selection based on the full dataset was used to select the potential prognostic factors included in the final model. The baseline covariates withp < 0.05 from those selection strategies were included in the final model. Other model selection strategies (such as best subset, forward, and stepwise selections) based on the full dataset were also used to verify the selected prognostic factors in the final model. A concordance index (C-index; range 0–1) [22] was used to assess model predictive discrimination. This index estimates the probability of concordance between predicted and observed responses, with higher index values associated with better discrimination. Internal validation of the final model was done using nonparametric bootstrap resampling with 500 datasets. The same variable selection procedure was repeated for each of the bootstrapped datasets and the selected variables were retained in the final model if the covariates were included in more than half the final models in the 500 bootstrapped samples.

To evaluate the robustness of the process in constructing the prediction model and validating the model based on the full data set, the following steps were implemented [23] :

• 1. 1000 random learning datasets (based on 70% of the full dataset) and 1000 random validation datasets (based on 30% of the full dataset) were generated with 1000 different random seeds.
• 2. The prediction model (with variable selection) was constructed based on the learning dataset using a Cox proportional hazards model with backward selection as described above. Then the final prediction model was fitted into the validation dataset.
• 3. The same model-building and validation process as above was applied to the 1000 learning datasets and 1000 validation datasets.
• 4. C-index values were generated from each dataset. If the distributions of the two sets of C-index values were similar, the robustness of the model selection and validation process was confirmed and the process was then applied to the full dataset.

The final Cox proportional hazards model after validation was used to create nomograms (with and without uNTx and BSAP). The statistical analyses for model development, as well as model validation and nomogram creation, were done using SAS version 9.2 software and the R 3.0.1 rms package, respectively.

3.1. Patients

Baseline demographics and disease characteristics for the overall study population are shown in Table 1 . Median (95% confidence interval [CI]) overall survival time was 20 (18, 21) mo.

3.2. Univariate analyses

Point estimates for each variable are shown in Table 2 . The bone-specific markers alkaline phosphatase, BSAP, and uNTx were each found to be predictors of survival (allp < 0.0001; Fig. 1 ). Two other bone parameters, pain and history of SRE, were also significant in the univariate analysis (bothp < 0.0001). PSA <10 ng/ml, absence of visceral metastases, ECOG ≤1, and hemoglobin >128 g/l were each significant prognosticators of longer survival (allp < 0.0001; Fig. 2 ). PSA with a 20-ng/ml cutoff or median cutoff also yielded highly significant results (data not shown); however, we chose the 10-ng/ml cutoff as it matched the stratification factor prespecified in the protocol, which was considered the best clinically meaningful cutoff at the time the trial was designed. Longer time from diagnosis of primary cancer to first evidence of metastatic disease, longer time from diagnosis of primary cancer to first bone metastasis, and longer time from first bone metastasis to randomization were also significant predictors of longer survival in the univariate analysis (p≤ 0.001). Younger age was an additional significant predictor of longer survival. Although patients with lower Gleason scores (2–6 and 7) had numerically longer survival than patients with higher Gleason scores (8–10), the difference was not statistically significant (p = 0.44 and 0.33, respectively).

Fig. 1 Univariate analyses of bone-specific factors. (A) Alkaline phosphatase (ALP ≤ median [143 U/l] vs > median). (B) Bone-specific alkaline phosphatase (BSAP <146 U/l vs ≥146 U/l). (C) Urinary N-telopeptide (uNTx ≤50 nmol/mmol vs >50 nmol/mmol). (D) Previous skeletal related event (SRE; no vs yes). (E) Worst pain absent or mild (Brief Pain Inventory—Short Form [BPI-SF] score ≤4 vs >4).

Fig. 2 Univariate analyses of disease characteristics and laboratory values. (A) Prostate-specific antigen (PSA; <10 ng/ml vs ≥10 ng/ml). (B) Presence of visceral metastases (no vs yes). (C) Eastern Co-operative Oncology Group (ECOG) performance status (≤1 vs 2). (D) Hemoglobin (≤median [128 g/l] vs >median).

3.3. Multivariate analysis

The process used to construct and validate the prediction model was found to be robust and was, therefore, applied to the full dataset: mean (standard deviation [SD]) C-index values were 0.7171 (0.007) for the learning datasets and 0.7128 (0.013) for the validation datasets. Among the 15 parameters tested in the multivariate analysis, Gleason score, current chemotherapy, and time from diagnosis of primary cancer to metastases were not significant and were excluded from the final model. Bone-related parameters were all highly significantly correlated to overall survival (allp < 0.001). The variables associated with longer survival included uNTx ≤50 nmol/mmol, BSAP <146 U/l, alkaline phosphatase ≤143 U/l, no prior SRE, mild or no pain, longer time from cancer diagnosis to diagnosis of bone metastases, longer time from diagnosis of bone metastases to randomization, PSA <10 ng/ml, absence of visceral metastases, ECOG score ≤1, younger age, and hemoglobin >128 g/l. Results of the 12 significant variables selected for the final model are shown in Table 3 . For the continuous variables (age, time from cancer diagnosis to diagnosis of bone metastases, and time from bone metastasis to randomization), the hazard ratios were close to one, yet highly significant, because they reflect the change in the hazard for any increase of one unit. For example, the hazard ratio for age was 1.012, indicating that a 63-yr-old patient had a 1.2% lower risk of death compared to a 64-yr-old patient. For an age difference of 15 yr (eg, age 60 yr vs age 75 yr), the hazard ratio increases to 1.191, indicating that the younger patient has a 19% lower risk of death. The corrected C-index value of the final model was 0.72, indicating robust discrimination.

As BSAP and uNTx are not commonly measured in clinical practice, we generated an additional model excluding these variables. The same model selection process was implemented (ie, univariable and multivariable analyses with various selection strategies) and all 10 variables were included in the final model. In the 10-variable model, the correlations of each parameter with overall survival were in the same direction and of similar magnitude as in the 12-variable model. The corrected C-index value of the final 10-variable model was approximately 0.71, indicating slightly less discrimination compared with the 12-variable model.

3.4. Nomograms

A clinical tool, or nomogram, for the estimation of survival for an individual patient is shown in Supplementary Figure 1 (12-variable nomogram). A second 10-variable nomogram excluding BSAP and uNTx is shown in Figure 3 . The precision of the tool is dependent on the full set of parameters; thus, data from all variables should be included to achieve the most accurate prediction.

Fig. 3 Ten variable nomograms for the prediction of 12-mo and 24-mo overall survival probability and median overall survival in months. To use this nomogram, complete the following steps: 1. Identify the scale for the first variable and draw a vertical line at the value until it intersects the “Points” axis; read off the corresponding risk score (points) for that variable. 2. Repeat this process for each variable. 3. Calculate the total risk score, that is, the sum of the points for each variable. 4. Locate the total risk score on the “Total points” axis and draw a vertical line downwards to identify the predicted survival probabilities. ^aIn one patient, the diagnosis of bone metastases occurred months before diagnosis of the primary tumor.

4.1. Limitations

Since clinical trial participation is an independent positive predictor of survival [27] , our nomograms may overestimate survival in patients not participating in a clinical trial. We acknowledge that the utility of nomograms to predict survival in routine patient management is as yet untested, and the complexity of these prognostic tools may be a barrier to their application. Use of nonlinear terms for covariates could have increased the magnitude of prediction in the model; however, we chose to use a less complicated model more easily used by clinicians. As the dataset contains only patients with an ECOG score of 1 or 2, the model may not be applicable for scores >2. Quantitation of disease burden such as number of bone metastases could not be taken into consideration, because these data were not recorded. LDH, a known prognostic factor[9] and [12], was not collected in this study. The cutoff points for the variables used in our model were based on the literature; other cutoff points may have yielded different results. In addition, the categorization of continuous variables may have caused information loss and may have introduced error through the assumption that all values of a variable within a category are equivalent. Our dataset is more contemporary than those previously published; however, the relevance of our model for patients receiving the new generation of treatments requires further investigation.

2.1. Study design and patients

The trial was a randomized, double-blind, phase 3 study comparing denosumab and zoledronic acid in 1901 men (clinicaltrials.gov identifier NCT00321620 ) [21] . Patients had CRPC (defined as rising PSA levels despite circulating testosterone levels of <0.50 ng/ml), radiological evidence of bone metastasis, but no known brain metastases, and an ECOG score ≤ 2. Baseline demographic and disease characteristics were similar between treatment groups and no differences in overall survival were observed [21] ; therefore, the present analyses included all trial participants ( Table 1 ).

2.2. Variables

Fifteen baseline study variables were examined for prognostic value.

2.3. Statistical methods

As prespecified in the statistical analysis plan, missing baseline values for the following variables were imputed using the mean of the pooled observed data: pain, time from cancer diagnosis to bone metastasis, time from bone metastasis to randomization, Gleason score, and time from cancer diagnosis to first evidence of metastatic disease. Missing baseline values for uNTx, BSAP, alkaline phosphatase, and hemoglobin were not imputed and were excluded. There were no missing values for previous SRE, PSA, current chemotherapy, ECOG score, visceral metastases, or age. Out of 1901 patients in the study, 1745 (92%) were included in this analysis.

Survival time was defined as the time interval (in months) from randomization to death. If a patient was alive by the primary analysis cutoff date or was lost to follow-up by the analysis cutoff date, then survival time was censored at the last contact date or the analysis cutoff date, whichever was first. Kaplan-Meier estimates were presented graphically for survival distribution by each baseline categorical covariate. A Cox proportional hazards model was used to assess the prognostic significance of baseline covariates in univariate and multivariate analyses. In addition, a Cox proportional hazards model with backward selection based on the full dataset was used to select the potential prognostic factors included in the final model. The baseline covariates withp < 0.05 from those selection strategies were included in the final model. Other model selection strategies (such as best subset, forward, and stepwise selections) based on the full dataset were also used to verify the selected prognostic factors in the final model. A concordance index (C-index; range 0–1) [22] was used to assess model predictive discrimination. This index estimates the probability of concordance between predicted and observed responses, with higher index values associated with better discrimination. Internal validation of the final model was done using nonparametric bootstrap resampling with 500 datasets. The same variable selection procedure was repeated for each of the bootstrapped datasets and the selected variables were retained in the final model if the covariates were included in more than half the final models in the 500 bootstrapped samples.

To evaluate the robustness of the process in constructing the prediction model and validating the model based on the full data set, the following steps were implemented [23] :

• 1. 1000 random learning datasets (based on 70% of the full dataset) and 1000 random validation datasets (based on 30% of the full dataset) were generated with 1000 different random seeds.
• 2. The prediction model (with variable selection) was constructed based on the learning dataset using a Cox proportional hazards model with backward selection as described above. Then the final prediction model was fitted into the validation dataset.
• 3. The same model-building and validation process as above was applied to the 1000 learning datasets and 1000 validation datasets.
• 4. C-index values were generated from each dataset. If the distributions of the two sets of C-index values were similar, the robustness of the model selection and validation process was confirmed and the process was then applied to the full dataset.

The final Cox proportional hazards model after validation was used to create nomograms (with and without uNTx and BSAP). The statistical analyses for model development, as well as model validation and nomogram creation, were done using SAS version 9.2 software and the R 3.0.1 rms package, respectively.

3.1. Patients

Baseline demographics and disease characteristics for the overall study population are shown in Table 1 . Median (95% confidence interval [CI]) overall survival time was 20 (18, 21) mo.

3.2. Univariate analyses

Point estimates for each variable are shown in Table 2 . The bone-specific markers alkaline phosphatase, BSAP, and uNTx were each found to be predictors of survival (allp < 0.0001; Fig. 1 ). Two other bone parameters, pain and history of SRE, were also significant in the univariate analysis (bothp < 0.0001). PSA <10 ng/ml, absence of visceral metastases, ECOG ≤1, and hemoglobin >128 g/l were each significant prognosticators of longer survival (allp < 0.0001; Fig. 2 ). PSA with a 20-ng/ml cutoff or median cutoff also yielded highly significant results (data not shown); however, we chose the 10-ng/ml cutoff as it matched the stratification factor prespecified in the protocol, which was considered the best clinically meaningful cutoff at the time the trial was designed. Longer time from diagnosis of primary cancer to first evidence of metastatic disease, longer time from diagnosis of primary cancer to first bone metastasis, and longer time from first bone metastasis to randomization were also significant predictors of longer survival in the univariate analysis (p≤ 0.001). Younger age was an additional significant predictor of longer survival. Although patients with lower Gleason scores (2–6 and 7) had numerically longer survival than patients with higher Gleason scores (8–10), the difference was not statistically significant (p = 0.44 and 0.33, respectively).

Fig. 1 Univariate analyses of bone-specific factors. (A) Alkaline phosphatase (ALP ≤ median [143 U/l] vs > median). (B) Bone-specific alkaline phosphatase (BSAP <146 U/l vs ≥146 U/l). (C) Urinary N-telopeptide (uNTx ≤50 nmol/mmol vs >50 nmol/mmol). (D) Previous skeletal related event (SRE; no vs yes). (E) Worst pain absent or mild (Brief Pain Inventory—Short Form [BPI-SF] score ≤4 vs >4).

Fig. 2 Univariate analyses of disease characteristics and laboratory values. (A) Prostate-specific antigen (PSA; <10 ng/ml vs ≥10 ng/ml). (B) Presence of visceral metastases (no vs yes). (C) Eastern Co-operative Oncology Group (ECOG) performance status (≤1 vs 2). (D) Hemoglobin (≤median [128 g/l] vs >median).

3.3. Multivariate analysis

The process used to construct and validate the prediction model was found to be robust and was, therefore, applied to the full dataset: mean (standard deviation [SD]) C-index values were 0.7171 (0.007) for the learning datasets and 0.7128 (0.013) for the validation datasets. Among the 15 parameters tested in the multivariate analysis, Gleason score, current chemotherapy, and time from diagnosis of primary cancer to metastases were not significant and were excluded from the final model. Bone-related parameters were all highly significantly correlated to overall survival (allp < 0.001). The variables associated with longer survival included uNTx ≤50 nmol/mmol, BSAP <146 U/l, alkaline phosphatase ≤143 U/l, no prior SRE, mild or no pain, longer time from cancer diagnosis to diagnosis of bone metastases, longer time from diagnosis of bone metastases to randomization, PSA <10 ng/ml, absence of visceral metastases, ECOG score ≤1, younger age, and hemoglobin >128 g/l. Results of the 12 significant variables selected for the final model are shown in Table 3 . For the continuous variables (age, time from cancer diagnosis to diagnosis of bone metastases, and time from bone metastasis to randomization), the hazard ratios were close to one, yet highly significant, because they reflect the change in the hazard for any increase of one unit. For example, the hazard ratio for age was 1.012, indicating that a 63-yr-old patient had a 1.2% lower risk of death compared to a 64-yr-old patient. For an age difference of 15 yr (eg, age 60 yr vs age 75 yr), the hazard ratio increases to 1.191, indicating that the younger patient has a 19% lower risk of death. The corrected C-index value of the final model was 0.72, indicating robust discrimination.

As BSAP and uNTx are not commonly measured in clinical practice, we generated an additional model excluding these variables. The same model selection process was implemented (ie, univariable and multivariable analyses with various selection strategies) and all 10 variables were included in the final model. In the 10-variable model, the correlations of each parameter with overall survival were in the same direction and of similar magnitude as in the 12-variable model. The corrected C-index value of the final 10-variable model was approximately 0.71, indicating slightly less discrimination compared with the 12-variable model.

3.4. Nomograms

A clinical tool, or nomogram, for the estimation of survival for an individual patient is shown in Supplementary Figure 1 (12-variable nomogram). A second 10-variable nomogram excluding BSAP and uNTx is shown in Figure 3 . The precision of the tool is dependent on the full set of parameters; thus, data from all variables should be included to achieve the most accurate prediction.

Fig. 3 Ten variable nomograms for the prediction of 12-mo and 24-mo overall survival probability and median overall survival in months. To use this nomogram, complete the following steps: 1. Identify the scale for the first variable and draw a vertical line at the value until it intersects the “Points” axis; read off the corresponding risk score (points) for that variable. 2. Repeat this process for each variable. 3. Calculate the total risk score, that is, the sum of the points for each variable. 4. Locate the total risk score on the “Total points” axis and draw a vertical line downwards to identify the predicted survival probabilities. ^aIn one patient, the diagnosis of bone metastases occurred months before diagnosis of the primary tumor.

4.1. Limitations

Since clinical trial participation is an independent positive predictor of survival [27] , our nomograms may overestimate survival in patients not participating in a clinical trial. We acknowledge that the utility of nomograms to predict survival in routine patient management is as yet untested, and the complexity of these prognostic tools may be a barrier to their application. Use of nonlinear terms for covariates could have increased the magnitude of prediction in the model; however, we chose to use a less complicated model more easily used by clinicians. As the dataset contains only patients with an ECOG score of 1 or 2, the model may not be applicable for scores >2. Quantitation of disease burden such as number of bone metastases could not be taken into consideration, because these data were not recorded. LDH, a known prognostic factor[9] and [12], was not collected in this study. The cutoff points for the variables used in our model were based on the literature; other cutoff points may have yielded different results. In addition, the categorization of continuous variables may have caused information loss and may have introduced error through the assumption that all values of a variable within a category are equivalent. Our dataset is more contemporary than those previously published; however, the relevance of our model for patients receiving the new generation of treatments requires further investigation.

2.1. Study design and patients

The trial was a randomized, double-blind, phase 3 study comparing denosumab and zoledronic acid in 1901 men (clinicaltrials.gov identifier NCT00321620 ) [21] . Patients had CRPC (defined as rising PSA levels despite circulating testosterone levels of <0.50 ng/ml), radiological evidence of bone metastasis, but no known brain metastases, and an ECOG score ≤ 2. Baseline demographic and disease characteristics were similar between treatment groups and no differences in overall survival were observed [21] ; therefore, the present analyses included all trial participants ( Table 1 ).

2.2. Variables

Fifteen baseline study variables were examined for prognostic value.

2.3. Statistical methods

As prespecified in the statistical analysis plan, missing baseline values for the following variables were imputed using the mean of the pooled observed data: pain, time from cancer diagnosis to bone metastasis, time from bone metastasis to randomization, Gleason score, and time from cancer diagnosis to first evidence of metastatic disease. Missing baseline values for uNTx, BSAP, alkaline phosphatase, and hemoglobin were not imputed and were excluded. There were no missing values for previous SRE, PSA, current chemotherapy, ECOG score, visceral metastases, or age. Out of 1901 patients in the study, 1745 (92%) were included in this analysis.

Survival time was defined as the time interval (in months) from randomization to death. If a patient was alive by the primary analysis cutoff date or was lost to follow-up by the analysis cutoff date, then survival time was censored at the last contact date or the analysis cutoff date, whichever was first. Kaplan-Meier estimates were presented graphically for survival distribution by each baseline categorical covariate. A Cox proportional hazards model was used to assess the prognostic significance of baseline covariates in univariate and multivariate analyses. In addition, a Cox proportional hazards model with backward selection based on the full dataset was used to select the potential prognostic factors included in the final model. The baseline covariates withp < 0.05 from those selection strategies were included in the final model. Other model selection strategies (such as best subset, forward, and stepwise selections) based on the full dataset were also used to verify the selected prognostic factors in the final model. A concordance index (C-index; range 0–1) [22] was used to assess model predictive discrimination. This index estimates the probability of concordance between predicted and observed responses, with higher index values associated with better discrimination. Internal validation of the final model was done using nonparametric bootstrap resampling with 500 datasets. The same variable selection procedure was repeated for each of the bootstrapped datasets and the selected variables were retained in the final model if the covariates were included in more than half the final models in the 500 bootstrapped samples.

To evaluate the robustness of the process in constructing the prediction model and validating the model based on the full data set, the following steps were implemented [23] :

• 1. 1000 random learning datasets (based on 70% of the full dataset) and 1000 random validation datasets (based on 30% of the full dataset) were generated with 1000 different random seeds.
• 2. The prediction model (with variable selection) was constructed based on the learning dataset using a Cox proportional hazards model with backward selection as described above. Then the final prediction model was fitted into the validation dataset.
• 3. The same model-building and validation process as above was applied to the 1000 learning datasets and 1000 validation datasets.
• 4. C-index values were generated from each dataset. If the distributions of the two sets of C-index values were similar, the robustness of the model selection and validation process was confirmed and the process was then applied to the full dataset.

The final Cox proportional hazards model after validation was used to create nomograms (with and without uNTx and BSAP). The statistical analyses for model development, as well as model validation and nomogram creation, were done using SAS version 9.2 software and the R 3.0.1 rms package, respectively.

3.1. Patients

Baseline demographics and disease characteristics for the overall study population are shown in Table 1 . Median (95% confidence interval [CI]) overall survival time was 20 (18, 21) mo.

3.2. Univariate analyses

Point estimates for each variable are shown in Table 2 . The bone-specific markers alkaline phosphatase, BSAP, and uNTx were each found to be predictors of survival (allp < 0.0001; Fig. 1 ). Two other bone parameters, pain and history of SRE, were also significant in the univariate analysis (bothp < 0.0001). PSA <10 ng/ml, absence of visceral metastases, ECOG ≤1, and hemoglobin >128 g/l were each significant prognosticators of longer survival (allp < 0.0001; Fig. 2 ). PSA with a 20-ng/ml cutoff or median cutoff also yielded highly significant results (data not shown); however, we chose the 10-ng/ml cutoff as it matched the stratification factor prespecified in the protocol, which was considered the best clinically meaningful cutoff at the time the trial was designed. Longer time from diagnosis of primary cancer to first evidence of metastatic disease, longer time from diagnosis of primary cancer to first bone metastasis, and longer time from first bone metastasis to randomization were also significant predictors of longer survival in the univariate analysis (p≤ 0.001). Younger age was an additional significant predictor of longer survival. Although patients with lower Gleason scores (2–6 and 7) had numerically longer survival than patients with higher Gleason scores (8–10), the difference was not statistically significant (p = 0.44 and 0.33, respectively).

Fig. 1 Univariate analyses of bone-specific factors. (A) Alkaline phosphatase (ALP ≤ median [143 U/l] vs > median). (B) Bone-specific alkaline phosphatase (BSAP <146 U/l vs ≥146 U/l). (C) Urinary N-telopeptide (uNTx ≤50 nmol/mmol vs >50 nmol/mmol). (D) Previous skeletal related event (SRE; no vs yes). (E) Worst pain absent or mild (Brief Pain Inventory—Short Form [BPI-SF] score ≤4 vs >4).

Fig. 2 Univariate analyses of disease characteristics and laboratory values. (A) Prostate-specific antigen (PSA; <10 ng/ml vs ≥10 ng/ml). (B) Presence of visceral metastases (no vs yes). (C) Eastern Co-operative Oncology Group (ECOG) performance status (≤1 vs 2). (D) Hemoglobin (≤median [128 g/l] vs >median).

3.3. Multivariate analysis

The process used to construct and validate the prediction model was found to be robust and was, therefore, applied to the full dataset: mean (standard deviation [SD]) C-index values were 0.7171 (0.007) for the learning datasets and 0.7128 (0.013) for the validation datasets. Among the 15 parameters tested in the multivariate analysis, Gleason score, current chemotherapy, and time from diagnosis of primary cancer to metastases were not significant and were excluded from the final model. Bone-related parameters were all highly significantly correlated to overall survival (allp < 0.001). The variables associated with longer survival included uNTx ≤50 nmol/mmol, BSAP <146 U/l, alkaline phosphatase ≤143 U/l, no prior SRE, mild or no pain, longer time from cancer diagnosis to diagnosis of bone metastases, longer time from diagnosis of bone metastases to randomization, PSA <10 ng/ml, absence of visceral metastases, ECOG score ≤1, younger age, and hemoglobin >128 g/l. Results of the 12 significant variables selected for the final model are shown in Table 3 . For the continuous variables (age, time from cancer diagnosis to diagnosis of bone metastases, and time from bone metastasis to randomization), the hazard ratios were close to one, yet highly significant, because they reflect the change in the hazard for any increase of one unit. For example, the hazard ratio for age was 1.012, indicating that a 63-yr-old patient had a 1.2% lower risk of death compared to a 64-yr-old patient. For an age difference of 15 yr (eg, age 60 yr vs age 75 yr), the hazard ratio increases to 1.191, indicating that the younger patient has a 19% lower risk of death. The corrected C-index value of the final model was 0.72, indicating robust discrimination.

As BSAP and uNTx are not commonly measured in clinical practice, we generated an additional model excluding these variables. The same model selection process was implemented (ie, univariable and multivariable analyses with various selection strategies) and all 10 variables were included in the final model. In the 10-variable model, the correlations of each parameter with overall survival were in the same direction and of similar magnitude as in the 12-variable model. The corrected C-index value of the final 10-variable model was approximately 0.71, indicating slightly less discrimination compared with the 12-variable model.

3.4. Nomograms

A clinical tool, or nomogram, for the estimation of survival for an individual patient is shown in Supplementary Figure 1 (12-variable nomogram). A second 10-variable nomogram excluding BSAP and uNTx is shown in Figure 3 . The precision of the tool is dependent on the full set of parameters; thus, data from all variables should be included to achieve the most accurate prediction.

Fig. 3 Ten variable nomograms for the prediction of 12-mo and 24-mo overall survival probability and median overall survival in months. To use this nomogram, complete the following steps: 1. Identify the scale for the first variable and draw a vertical line at the value until it intersects the “Points” axis; read off the corresponding risk score (points) for that variable. 2. Repeat this process for each variable. 3. Calculate the total risk score, that is, the sum of the points for each variable. 4. Locate the total risk score on the “Total points” axis and draw a vertical line downwards to identify the predicted survival probabilities. ^aIn one patient, the diagnosis of bone metastases occurred months before diagnosis of the primary tumor.

4.1. Limitations

Since clinical trial participation is an independent positive predictor of survival [27] , our nomograms may overestimate survival in patients not participating in a clinical trial. We acknowledge that the utility of nomograms to predict survival in routine patient management is as yet untested, and the complexity of these prognostic tools may be a barrier to their application. Use of nonlinear terms for covariates could have increased the magnitude of prediction in the model; however, we chose to use a less complicated model more easily used by clinicians. As the dataset contains only patients with an ECOG score of 1 or 2, the model may not be applicable for scores >2. Quantitation of disease burden such as number of bone metastases could not be taken into consideration, because these data were not recorded. LDH, a known prognostic factor[9] and [12], was not collected in this study. The cutoff points for the variables used in our model were based on the literature; other cutoff points may have yielded different results. In addition, the categorization of continuous variables may have caused information loss and may have introduced error through the assumption that all values of a variable within a category are equivalent. Our dataset is more contemporary than those previously published; however, the relevance of our model for patients receiving the new generation of treatments requires further investigation.

2.1. Study design and patients

The trial was a randomized, double-blind, phase 3 study comparing denosumab and zoledronic acid in 1901 men (clinicaltrials.gov identifier NCT00321620 ) [21] . Patients had CRPC (defined as rising PSA levels despite circulating testosterone levels of <0.50 ng/ml), radiological evidence of bone metastasis, but no known brain metastases, and an ECOG score ≤ 2. Baseline demographic and disease characteristics were similar between treatment groups and no differences in overall survival were observed [21] ; therefore, the present analyses included all trial participants ( Table 1 ).

2.2. Variables

Fifteen baseline study variables were examined for prognostic value.

2.3. Statistical methods

As prespecified in the statistical analysis plan, missing baseline values for the following variables were imputed using the mean of the pooled observed data: pain, time from cancer diagnosis to bone metastasis, time from bone metastasis to randomization, Gleason score, and time from cancer diagnosis to first evidence of metastatic disease. Missing baseline values for uNTx, BSAP, alkaline phosphatase, and hemoglobin were not imputed and were excluded. There were no missing values for previous SRE, PSA, current chemotherapy, ECOG score, visceral metastases, or age. Out of 1901 patients in the study, 1745 (92%) were included in this analysis.

Survival time was defined as the time interval (in months) from randomization to death. If a patient was alive by the primary analysis cutoff date or was lost to follow-up by the analysis cutoff date, then survival time was censored at the last contact date or the analysis cutoff date, whichever was first. Kaplan-Meier estimates were presented graphically for survival distribution by each baseline categorical covariate. A Cox proportional hazards model was used to assess the prognostic significance of baseline covariates in univariate and multivariate analyses. In addition, a Cox proportional hazards model with backward selection based on the full dataset was used to select the potential prognostic factors included in the final model. The baseline covariates withp < 0.05 from those selection strategies were included in the final model. Other model selection strategies (such as best subset, forward, and stepwise selections) based on the full dataset were also used to verify the selected prognostic factors in the final model. A concordance index (C-index; range 0–1) [22] was used to assess model predictive discrimination. This index estimates the probability of concordance between predicted and observed responses, with higher index values associated with better discrimination. Internal validation of the final model was done using nonparametric bootstrap resampling with 500 datasets. The same variable selection procedure was repeated for each of the bootstrapped datasets and the selected variables were retained in the final model if the covariates were included in more than half the final models in the 500 bootstrapped samples.

To evaluate the robustness of the process in constructing the prediction model and validating the model based on the full data set, the following steps were implemented [23] :

• 1. 1000 random learning datasets (based on 70% of the full dataset) and 1000 random validation datasets (based on 30% of the full dataset) were generated with 1000 different random seeds.
• 2. The prediction model (with variable selection) was constructed based on the learning dataset using a Cox proportional hazards model with backward selection as described above. Then the final prediction model was fitted into the validation dataset.
• 3. The same model-building and validation process as above was applied to the 1000 learning datasets and 1000 validation datasets.
• 4. C-index values were generated from each dataset. If the distributions of the two sets of C-index values were similar, the robustness of the model selection and validation process was confirmed and the process was then applied to the full dataset.

The final Cox proportional hazards model after validation was used to create nomograms (with and without uNTx and BSAP). The statistical analyses for model development, as well as model validation and nomogram creation, were done using SAS version 9.2 software and the R 3.0.1 rms package, respectively.

3.1. Patients

Baseline demographics and disease characteristics for the overall study population are shown in Table 1 . Median (95% confidence interval [CI]) overall survival time was 20 (18, 21) mo.

3.2. Univariate analyses

Point estimates for each variable are shown in Table 2 . The bone-specific markers alkaline phosphatase, BSAP, and uNTx were each found to be predictors of survival (allp < 0.0001; Fig. 1 ). Two other bone parameters, pain and history of SRE, were also significant in the univariate analysis (bothp < 0.0001). PSA <10 ng/ml, absence of visceral metastases, ECOG ≤1, and hemoglobin >128 g/l were each significant prognosticators of longer survival (allp < 0.0001; Fig. 2 ). PSA with a 20-ng/ml cutoff or median cutoff also yielded highly significant results (data not shown); however, we chose the 10-ng/ml cutoff as it matched the stratification factor prespecified in the protocol, which was considered the best clinically meaningful cutoff at the time the trial was designed. Longer time from diagnosis of primary cancer to first evidence of metastatic disease, longer time from diagnosis of primary cancer to first bone metastasis, and longer time from first bone metastasis to randomization were also significant predictors of longer survival in the univariate analysis (p≤ 0.001). Younger age was an additional significant predictor of longer survival. Although patients with lower Gleason scores (2–6 and 7) had numerically longer survival than patients with higher Gleason scores (8–10), the difference was not statistically significant (p = 0.44 and 0.33, respectively).

Fig. 1 Univariate analyses of bone-specific factors. (A) Alkaline phosphatase (ALP ≤ median [143 U/l] vs > median). (B) Bone-specific alkaline phosphatase (BSAP <146 U/l vs ≥146 U/l). (C) Urinary N-telopeptide (uNTx ≤50 nmol/mmol vs >50 nmol/mmol). (D) Previous skeletal related event (SRE; no vs yes). (E) Worst pain absent or mild (Brief Pain Inventory—Short Form [BPI-SF] score ≤4 vs >4).

Fig. 2 Univariate analyses of disease characteristics and laboratory values. (A) Prostate-specific antigen (PSA; <10 ng/ml vs ≥10 ng/ml). (B) Presence of visceral metastases (no vs yes). (C) Eastern Co-operative Oncology Group (ECOG) performance status (≤1 vs 2). (D) Hemoglobin (≤median [128 g/l] vs >median).

3.3. Multivariate analysis

The process used to construct and validate the prediction model was found to be robust and was, therefore, applied to the full dataset: mean (standard deviation [SD]) C-index values were 0.7171 (0.007) for the learning datasets and 0.7128 (0.013) for the validation datasets. Among the 15 parameters tested in the multivariate analysis, Gleason score, current chemotherapy, and time from diagnosis of primary cancer to metastases were not significant and were excluded from the final model. Bone-related parameters were all highly significantly correlated to overall survival (allp < 0.001). The variables associated with longer survival included uNTx ≤50 nmol/mmol, BSAP <146 U/l, alkaline phosphatase ≤143 U/l, no prior SRE, mild or no pain, longer time from cancer diagnosis to diagnosis of bone metastases, longer time from diagnosis of bone metastases to randomization, PSA <10 ng/ml, absence of visceral metastases, ECOG score ≤1, younger age, and hemoglobin >128 g/l. Results of the 12 significant variables selected for the final model are shown in Table 3 . For the continuous variables (age, time from cancer diagnosis to diagnosis of bone metastases, and time from bone metastasis to randomization), the hazard ratios were close to one, yet highly significant, because they reflect the change in the hazard for any increase of one unit. For example, the hazard ratio for age was 1.012, indicating that a 63-yr-old patient had a 1.2% lower risk of death compared to a 64-yr-old patient. For an age difference of 15 yr (eg, age 60 yr vs age 75 yr), the hazard ratio increases to 1.191, indicating that the younger patient has a 19% lower risk of death. The corrected C-index value of the final model was 0.72, indicating robust discrimination.

As BSAP and uNTx are not commonly measured in clinical practice, we generated an additional model excluding these variables. The same model selection process was implemented (ie, univariable and multivariable analyses with various selection strategies) and all 10 variables were included in the final model. In the 10-variable model, the correlations of each parameter with overall survival were in the same direction and of similar magnitude as in the 12-variable model. The corrected C-index value of the final 10-variable model was approximately 0.71, indicating slightly less discrimination compared with the 12-variable model.

3.4. Nomograms

A clinical tool, or nomogram, for the estimation of survival for an individual patient is shown in Supplementary Figure 1 (12-variable nomogram). A second 10-variable nomogram excluding BSAP and uNTx is shown in Figure 3 . The precision of the tool is dependent on the full set of parameters; thus, data from all variables should be included to achieve the most accurate prediction.

Fig. 3 Ten variable nomograms for the prediction of 12-mo and 24-mo overall survival probability and median overall survival in months. To use this nomogram, complete the following steps: 1. Identify the scale for the first variable and draw a vertical line at the value until it intersects the “Points” axis; read off the corresponding risk score (points) for that variable. 2. Repeat this process for each variable. 3. Calculate the total risk score, that is, the sum of the points for each variable. 4. Locate the total risk score on the “Total points” axis and draw a vertical line downwards to identify the predicted survival probabilities. ^aIn one patient, the diagnosis of bone metastases occurred months before diagnosis of the primary tumor.

4.1. Limitations

Since clinical trial participation is an independent positive predictor of survival [27] , our nomograms may overestimate survival in patients not participating in a clinical trial. We acknowledge that the utility of nomograms to predict survival in routine patient management is as yet untested, and the complexity of these prognostic tools may be a barrier to their application. Use of nonlinear terms for covariates could have increased the magnitude of prediction in the model; however, we chose to use a less complicated model more easily used by clinicians. As the dataset contains only patients with an ECOG score of 1 or 2, the model may not be applicable for scores >2. Quantitation of disease burden such as number of bone metastases could not be taken into consideration, because these data were not recorded. LDH, a known prognostic factor[9] and [12], was not collected in this study. The cutoff points for the variables used in our model were based on the literature; other cutoff points may have yielded different results. In addition, the categorization of continuous variables may have caused information loss and may have introduced error through the assumption that all values of a variable within a category are equivalent. Our dataset is more contemporary than those previously published; however, the relevance of our model for patients receiving the new generation of treatments requires further investigation.