Use the original KORCC database9two recent studies reported28,29First, Byun et al.28 A deep learning-based survival prediction model was used to assess the prognosis of non-metastatic clear-cell RCC. DeepSurv’s Harrel’s C indices for recurrence and cancer-specific survival were 0.802 and 0.834, respectively. More recently, Kim et al.29 developed ML-based algorithms to predict the likelihood of recurrence at 5 and 10 years after surgery. The maximum area under the receiver operating characteristic curve (AUROC) was obtained from the Naive Bayes (NB) model with values of 0.836 and 0.784 at 5 and 10 years, respectively.
The current study used the updated KORCC database. It currently contains clinical data for over 10,000 patients. To our knowledge, this is the largest dataset of Asian populations with RCC. Using this dataset, we can develop a more accurate model with very high precision (range 0.77 to 0.94) and F1 score (range 0.77 to 0.97, Table 3).Accuracy values were relatively high compared to previous models, including Kattan nomograms, Leibovich models, and GRANT scores, which were around 0.75,6,7,8Among them, the Kattan nomogram was developed using a cohort of 601 patients with clinically localized RCC, with an overall C-index of 74%.FiveSubsequent analysis of the same group of patients using additional prognostic variables such as tumor necrosis, vascular invasion, and tumor grade showed a C-index as high as 82%.30Their prediction accuracy was not yet as high as ours.
Additionally, short-term (3-year) recurrence and survival data can be included to help develop more sophisticated surveillance strategies. Another strength of the current study is that most of the algorithms introduced so far have been applied.18,19,20,21,22,23,24,25,26, showing relatively consistent performance with high accuracy. Finally, we also performed external validation using another (SNUBH) cohort and achieved well-maintained high precision and F1 scores in both recurrence and survival (Fig. 2). External validation of predictive models is essential to ensure and correct for inter-institutional differences, especially when using multi-institutional datasets.
AUROC has been primarily used as a standard to evaluate the performance of predictive models5,6,7,8,29However, AUROC equally weights changes in sensitivity and specificity without considering clinically meaningful information.6Moreover, the lack of ability to compare the performance of different ML models is another limitation of the AUROC method.31Therefore, we adopted accuracy and F1 score instead of AUROC as evaluation metrics. SMOTE plus F1 Score17which is used as a measure of better accuracy to solve the problem of imbalanced data27.
RCC is not a single disease, but multiple histologically defined cancers with distinct genetic characteristics, clinical course, and response to treatment.32Regarding metastatic RCC, the International Metastatic Renal Cell Carcinoma Database Consortium and Memorial Sloan Kettering Cancer Center risk models have been extensively validated and are widely used to predict survival outcomes in patients receiving systemic therapy.33,34However, both risk models were developed without considering histologic subtype. Therefore, it was presumed that predictive performance was strongly influenced by clear cell type (the predominant histological subtype) RCC.Interestingly, a previous study using the Korean metastatic RCC registry found that both risk models reliably predicted progression and survival even in non-clear cell RCC35In the current study, we also found very high precision and F1 scores for all indicators tested after performing subgroup analysis according to tissue type (clear and non-clear cell type RCC) (Supplementary Table 3). and 4). Taken together, these findings suggest that prognostic differences between clear and non-clear cell types of RCC appear to be offset in both metastatic and non-metastatic RCC. Further efforts are needed to develop and validate advanced predictive models for individual subtypes of non-clear cell RCC.
The current study had several limitations. First, due to the small number of long-term 10-year follow-up cases, the problem of data imbalance was inevitable. Subsequently, the 10-year recurrence-free rate was reported to be 45.3%. The majority of patients had no further long-term follow-up if there was no evidence of disease at 5 years. However, we employed both SMOTE and F1 scores to resolve these imbalanced data issues. The retrospective design of this study was also an inherent limitation. Another limitation is that the forecast model developed only included the South Korean population. Model validation using data from other countries and races is also required. Regarding non-clear cell RCC, the current study cohort is still relatively small due to the rarity of the disease, and it was inevitable to integrate and analyze each subtype together. Further research is needed to develop and validate predictive models. Moreover, the lack of more accurate classifiers such as cross-validation and bootstrapping is another limitation of current research. Finally, you should follow her web-embedded deployment of the model for better accessibility and portability.
