We built machine learning algorithms that incorporate both variability and trends of dynamic parameters from daily data within 72 hours of dialysis initiation to predict the likelihood of early dialysis release in critically ill patients with AKI. These models were developed, cross-validated, and tested in time, and demonstrated to have good discriminatory power in predicting renal function recovery at discharge. Furthermore, we applied class weighting to address data imbalance, used LASSO to develop models with fewer variables, and predicted short time windows (first 24 or 48 hours), all of which showed good discriminatory power. These results suggest the potential clinical utility of integrating into EMR for clinical decision support systems. Finally, we used SHAP values and PDPs to identify key features that influence the model predictions. Early vital signs and input/output domains were key drivers of the model. Explainable machine learning-based predictions of AKI-D recovery using existing EMR data have the potential to improve risk stratification and gain insights into patient outcomes.
Poor recovery of renal function after AKI-D is associated with increased morbidity and mortality, as well as high healthcare costs.25,26This is consistent with the results of previous epidemiological studies.1,26In this study, dialysis-dependent patients were found to have higher short-term (3 months) and long-term (1 year) mortality rates after discharge compared with patients released from acute dialysis (Supplementary Fig. S3). Early prediction of recovery from AKI-D in critically ill patients has important implications for patient-centered care.27Currently, prediction relies solely on clinical experience. The most commonly used indicator for discontinuing dialysis is an increase in urinary output.FiveHowever, the accuracy of urine volume in predicting successful RRT cessation is controversial, with reported AUROCs ranging from 0.63 to 0.91 and various cut-off values.5,28Furthermore, urine volume is commonly used as an indicator of renal function recovery after RRT discontinuation, rather than as an early-stage marker.Traditional functional biomarkers (serum/urinary Cr or cystatin C) and new biomarkers (such as kidney injury molecule-1, neutrophil gelatinase-associated lipocalin, osteopontin, tissue inhibitor of metalloproteinase-2/insulin-like growth factor binding protein-7, and proenkephalin A 119-159) have been investigated as predictors of AKI-D recovery.5,8,11,27,29Current biomarkers for renal function recovery after AKI-D are not widely used to identify patients with a high likelihood of early renal function recovery because additional samples are required and conclusive evidence is limited. The urgent need for precise guides for liberation from RRT was also recognized at the recent Acute Dialysis Quality Initiative (ADQI) consensus conference.30Experts emphasized the importance of integrating big data analytics with single-case EMR evaluation to enable personalized RRT for every individual. To fill this gap, there is an unmet clinical need to integrate EMRs to evaluate the predictive value of RRT discontinuation and prognosis in AKI-D.
Machine learning models developed in critical nephrology can leverage data collected in the EMR to predict important renal outcomes.13,31,32As data accumulate, these models will likely provide additional benefits such as earlier prediction and improved accuracy. However, validated machine learning models for predicting acute dialysis discontinuation in critical situations have not been widely studied. To our knowledge, one prior study used a machine learning approach to predict freedom from RRT in patients with AKI-D. Pattharanitima et al. used the Medical Information Mart for Intensive Care (MIMIC-III) database to predict RRT-free survival in critically ill patients with AKI requiring continuous renal replacement therapy (CRRT).16Of the 684 patients, 30% successfully discontinued RRT. Models using 81 features extracted from admission to CRRT initiation yielded AUROC values ranging from 0.43 to 0.7. In the present study of 1,381 AKI-D patients, we used 90 variables from the first 3 days after dialysis, including all vital signs and input/output recordings. Thus, the variability and trends across multiple time points in these data were incorporated into the model. The prediction models in the present study performed well, with AUROCs of 0.77–0.81 in the development cohort and 0.82–0.85 in the temporal test cohort. Apart from the candidate predictors, the difference in model performance between the present study and previous studies may also be due to differences in the characteristics of the study population, the number of participants, and different feature windows. The first 3 days are considered the acute phase for ICU patients, and recovery from shock, as exemplified by septic shock, often occurs within the first 3 days.33Importantly, providing additional prognostic information after the initial intensive care period may inform subsequent medical decisions, such as consideration of clinical trials for high-risk groups or potential withdrawal of life-sustaining medical treatment. Furthermore, in addition to the 72-hour model, we trained models at 24 and 48 hours, both of which maintained good predictive performance.
As shown in Table 3 and Table S6, the proposed threshold of 0.5 for predicting renal function recovery provided good specificity, while a threshold of 0.3 increased sensitivity. Decision curve analysis revealed the net benefit of using these models in clinical decision making by considering the trade-off between sensitivity and specificity at different threshold probabilities. Using the models results in more benefits than harms at both thresholds of 0.5 and 0.3. Thus, a lower threshold, such as 0.3, could identify a wider subset of patients who may recover renal function. On the other hand, a threshold of 0.5 would result in fewer false positives and reduce alarm fatigue, which is a major concern in ICU alarm systems.34,35Therefore, the choice of threshold in practical applications should be based on whether healthcare providers need assistance to accurately identify patients who can and cannot recover after AKI-D while effectively managing resources.
Using an interpretable machine learning algorithm, we found that, not surprisingly, urinary volume was the variable most influential on renal recovery after initiation of dialysis for AKI. Patients successfully liberated from RRT had significantly higher urinary volumes. According to the PDP (Figure 4), patients with a urinary volume >1570 ml 72 hours after dialysis were more likely to achieve dialysis independence at discharge. Figure 4 shows the PDP of top predictors by SHAP value and cutoff values favorable for renal recovery. In our study, the top 20 variables include previously well-studied factors of renal recovery, such as urinary volume and BUN.5,36,37and less-explored variables (e.g., enteral nutrition intake during the first 3 days after dialysis). In addition, we categorized the top 20 variables identified by the XGBoost model by clinical domains, including comorbidities, vital signs, laboratory data, and input/output domains. In addition to urine output, most of the early predictors were related to the vital signs domain (Fig. 3B). There is a general consensus that hemodynamic instability caused by excessive fluid removal during dialysis impedes renal recovery.38However, traditional predictive models of renal recovery often overlook vital signs due to their complexity and dynamic nature.Bellomo et al. conducted a retrospective study in critically ill patients in shock and found that higher levels of relative hypotension during the first hours of vasopressor support were associated with a significantly higher risk of adverse kidney-related outcomes.39Consistent with current evidence, our data suggest that early vital signs such as SpO2, respiratory rate trends as well as systolic blood pressure variability are significantly associated with renal prognosis in critically ill patients. Furthermore, the use of LASSO models with more limited variables and the incorporation of routinely collected laboratory data provide a practical means of rapid integration into the EMR (Supplementary Table S8). An illustration showing the interpretability of the models and the evolution of key features over time using two separate individuals is presented in Supplementary Fig. S4. Overall, deconvolving the explainable machine learning model reveals new insights into how early stage ICU patient features interact with the patient's future events.
The strength of this study lies in its size, including 1,381 patients with AKI-D among 26,593 ICU admissions. We also had complete data on vital signs and inputs and outputs, with a very low missingness rate (< 1%). Furthermore, we linked cause of death data from the NHIRD to reduce the risk of attrition bias. This is particularly important in ICU studies, as 40%–60% of critically ill patients with AKI-D will discontinue treatment due to withdrawal of life support or death.Four.
This study has several limitations. First, recovery status was determined at the time of discharge, although we recognized that dialysis weaning may occur further. However, the mean length of hospital stay for critically ill patients with AKI-D is longer (25.0 days for the entire cohort and 30.5 days for the recovery group), and non-recovered patients with AKI-D have a nephrologist's confirmation of dialysis-dependent critical illness certification before discharge. Thus, renal prognosis is clinically important. Second, our model made a one-time early prediction of renal recovery based on data obtained within 3 days after the start of dialysis. Subsequent events may cause patient outcomes to deviate from the prediction. Although we developed additional models with different time ranges (1 and 2 days), continuously updated predictions would be more appropriate in these cases. Third, limitations of this retrospective database include the lack of other important predictors, such as the degree of urinary proteinuria, timed creatinine clearance, and novel renal biomarkers that may affect kidney and patient recovery. Finally, we used a temporal test, which can be considered an intermediate validation between internal and external validation.40Recovery and mortality rates in this cohort were comparable to those reported in the literature;4,26The results need to be further validated in other settings.
