All methods were carried out in accordance with relevant guidelines and regulations. The study, referenced by IRAS project ID 304542, was approved by the UK Health Research Authority and Health and Care Research Wales (UK Research Ethics Committee). All transplant patients in the UK consent to the use of their data in a mandatory national register at the time they are added to the transplant waiting list. This project uses anonymised data from the national register so individual patient consent was not required.
data
Our study is based on the analysis of the UK Transplant Registry data set provided by NHSBT. The data describes 36,653 approved kidney transplants performed in 24 transplant centers in the UK between 2000 and 2020. All transplants are from deceased donors. The follow-up period is approximately 22 years in total. Each transplant is initially described by seven entries describing three identifiers, 12 immunosuppression tracking indicators, 143 donor, recipient and transplant characteristics and the targeted outcome. We consider transplants as independent and exclude transplant, donor and recipient identifiers. Information on post-transplant immunosuppression is discarded as it is not available at the time of the offer decision. Donor, recipient and transplant characteristics serve as input features for modelling. Of these, 24 refer to the recipient, 109 to the donor and 10 to the transplant overall. Characteristics of both recipients and donors include general information such as gender, ethnicity, age, blood type, height, weight, or body mass index (BMI). More specific information is also available such as transplant center, number of previous transplants, wait time, ease of matching, and dialysis status. Donor data includes cause of death, past medical history, and blood test results including renal function (estimated glomerular filtration rate, eGFR). Transplant data includes immunological match of donor and recipient.
Duplicate rows are removed and values outside reasonable clinical ranges are removed. Categorical values are reviewed by clinicians and simplified (or removed) if necessary. BMI is recalculated based on weight and height. Both weight and height are discarded to limit redundant information. Blood measurements are harmonized across data sets by selecting the first measurement available (usually at donor registration) and the maximum value during the donation process. As the calculation of eGFR varies from hospital to hospital, this indicator is recalculated across data sets using a consistent definition (see section A.1 in the appendix). Recipient dialysis status is also simplified to dialysis duration and dialysis method (pre-dialysis, hemodialysis, or peritoneal dialysis) at the time of transplant. Notably, dialysis time for pre-dialysis recipients is set to 0 by default. Transplant offers that do not meet the inclusion criteria, such as double organ transplants and multi-organ transplants, are discarded.
The outcomes present in the dataset include information on graft failure, patient death, and graft failure. Graft failure excludes death with a functioning graft, whereas graft failure indicates either graft failure or death. In this study, we focus on predicting graft failure and patient death. Each outcome is represented as a pair that includes the event time and a right-censoring indicator. Right-censoring is a common type of censoring in survival analysis and represents loss to follow-up for the event of interest. This can occur for a variety of reasons, including end of study, competing events, etc. Thus, right-censoring information provides partial information on survival time, which is only known to be greater than the censoring time. Transplant outcomes are recalculated for consistency.
After removing features that exhibit more than 50% missing values across the entire dataset, the data is described through 50 input variables. At this stage, the data contains 8% missingness. An overview of this data cleaning process is shown in Figure 1. Furthermore, the complete list of features and targets considered at the final stage of this process is provided in the Appendix (Section A.2).
End-to-end data processing pipeline from raw data to model testing. Data cleaning details are shown on the left. Cross-validation is performed before and after feature selection.
Model training and validation
In this article, we compare the Cox PH model, DeepHit, and Random Survival Forest in a single risk setting. We provide a posteriori interpretation of the different models and discuss their performance.
The following methodology is applied: First, the data is stratified with respect to the censoring indicator, reserving 80% of the data for training and the remaining 20% for testing. Due to the change in the matching policy and the difference in follow-up times for older and more recent offers, we do not split the data based on implantation date. After this first step, the numerical values are standardized and the categorical values are one-hot encoded. The mean and variance are calculated on the training data only. Due to the presence of outliers in the data, standardization seems more relevant than normalization. MissForest, an iterative imputation method that relies on random forests, is then used to impute missing features.17MissForest is first trained on the training data and then applied to all data. This solution was selected among several imputation techniques, including MIDAS, a variational autoencoder based imputation technique.18; MICE, an iterative method for multi-column imputation19; MissForest itself (a variant of MICE), and a simple imputer that just returns the mean value. These methods are compared on a sample of data where missingness is introduced by randomly masking known values. To simplify the end-to-end data processing pipeline and reduce the burden of data requirements, we use the same training dataset for both preprocessing and model training. Previous tests have not shown any particular difference with more partitioned data management. Therefore, after the imputation step, the survival analysis model is trained by 5-fold cross-validation. This process is performed for the first time for feature selection. This is achieved by inputting Gaussian noise as features, i.e., selecting features with a higher importance than the importance attached to the noise. Based on this subset of features, 5-fold cross-validation is repeated for the final model training. Model calibration is then performed. Predictions are post-hoc adjusted to match the observed outcome ratios by training a logistic regression model. Model evaluation is done by calculating the agreement and AUROC scores over 100 bootstraps of the test data. The survival model is interpreted clinically using SHAP. To do so, we fix a specific time point (1, 5 or 10 years) and consider how the model predicts the occurrence of the event up to that point. The coefficients of the Cox PH model are also provided. The choice to consider the coefficients of the Cox model rather than using SHAP for interpretability is due to the fact that the inherent interpretability of the Cox model is an important factor in model selection, especially as this model is typically interpreted in this way. Figure 1 shows the overall methodology and the code used for the experiments can be found at https://github.com/AchilleSalaun/Xamelot.
Following this processing pipeline, we compare the Cox PH model, random survival forests, and neural networks. Both DeepHit and survival forests require time discretization, so we restrict the prediction of transplant outcomes to 1, 5, and 10 years. This step is8We use a grid search to tune the hyperparameters, and then use the Breslow estimator to derive a baseline for the Cox PH model.20In addition, a regularization parameter is introduced, \(1e^{-4}\) To address collinearity in the data, the survival random forest is fed with 300 trees. Finally, we train DeepHit in a single-risk manner. While predicting transplant failure, the model is instantiated with two hidden layers of 100 neurons with 10% dropout. The neural network used to predict patient death shows one hidden layer of 200 neurons followed by two layers of 100 neurons. For both transplant failure and patient death prediction, training is performed for 50 epochs with a batch size of 64 and a learning rate of 100. \(1e^{-2}\).
