A Hybrid Extended Cox-frailty and Random Forest Framework for Prostate Cancer Survival: Integrating Clinical Interpretability with Machine Learning Performance

Emmanuel Asare Ayim *

Department of Statistics and Actuarial Science, Takoradi Technical University, Sekondi-Takoradi, Ghana.

Benjamin Odoi

Department of Mathematical Sciences, University of Mines and Technology (UMaT), Tarkwa, Ghana.

Henry Otoo

Department of Mathematical Sciences, University of Mines and Technology (UMaT), Tarkwa, Ghana.

*Author to whom correspondence should be addressed.


Abstract

Prostate cancer risk stratification in low-resource clinical settings requires survival-prediction tools that address violations of the proportional hazards assumption, unobserved patient heterogeneity, and nonlinear covariate interactions that classical regression frameworks cannot represent. This study develops a Hybrid Extended Cox–Frailty and Random Forest (Hybrid Cox-RF) model using a retrospective cohort of 200 patients with prostate cancer from a Ghanaian tertiary hospital (161 deaths, 80.5%, over the full follow-up period; 3 competing other-cause deaths, 1.5%; and 36 censored observations, 18.0%). A sequential pipeline was applied: (1) Aalen–Johansen competing-risks quantification; (2) Schoenfeld residual-based testing of the proportional hazards (PH) assumption; (3) extended Cox modelling with time-varying covariates for four confirmed PH violators; (4) gamma frailty correction; (5) six machine-learning classifiers benchmarked by 10-fold stratified cross-validation at a fixed 24-month mortality horizon (112 of 200 patients, 56.0%, died by 24 months); and (6) construction of the Hybrid Cox-RF model by embedding out-of-bag Random Forest logit risk scores derived from the 24-month classifier into the full-follow-up gamma frailty Cox partial likelihood with sandwich variance correction. Random Forest was the best-performing classifier at the 24-month horizon (AUC = 0.750; sensitivity = 0.782; F1 = 0.810); PSA was the dominant predictor (14.2% mean Gini decrease), followed by BMI (11.8%), Stage T4 (9.6%), age (8.3%), and Gleason score (7.1%). For the full-follow-up survival endpoint, the Hybrid Cox-RF achieved a 10-fold cross-validated C-index of 0.743, compared with 0.671 for the standard Cox model (ΔC = +0.072; a 10.7% relative improvement), an AIC of 1364.21 (ΔAIC = 57.7 versus the next-best Extended Cox Frailty model), a Hosmer–Lemeshow p-value of 0.794, and a calibration slope of 0.97. Because classifier tuning and hybrid-model fitting were not incorporated into a fully nested cross-validation loop, these hybrid-model estimates should be interpreted as an internally cross-validated upper bound pending confirmation through external validation or nested cross-validation. The RF risk score was strongly associated with mortality (HR = 10.91 per 0.5-unit logit increase; 95% CI: 5.47–21.77); most clinical covariate hazard ratios remained stable (within ±15%) after its inclusion, the RF score absorbed the linear age effect, and rural residence remained independently significant (HR = 1.597, p = 0.008), indicating structural barriers to healthcare access beyond clinical severity. Pending external validation and nested cross-validation, the Hybrid Cox-RF framework is proposed as a candidate analytical approach for prostate cancer survival prediction in Ghanaian and comparable sub-Saharan African clinical settings, offering an interpretable artificial-intelligence approach that can support precision oncology and clinical decision-making in low-resource healthcare systems.

Keywords: Prostate cancer survival, cox proportional hazards model, random forest, hybrid survival modelling, machine learning classifiers


How to Cite

Ayim, Emmanuel Asare, Benjamin Odoi, and Henry Otoo. 2026. “A Hybrid Extended Cox-Frailty and Random Forest Framework for Prostate Cancer Survival: Integrating Clinical Interpretability With Machine Learning Performance”. Journal of Cancer and Tumor International 16 (3):143-58. https://doi.org/10.9734/jcti/2026/v16i3367.

Downloads

Download data is not yet available.