DESIGN, POPULATION, RESULTS
This article follows the Transparency Report for Multivariable Predictive Models (TRIPOD) checklist for individual prognosis or diagnosis.19 and guidance on developing and reporting predictive models20. This study was approved by the Research Ethics Committee of the University Hospital Center of Montreal (CHUM) and waived the requirement for individual informed consent given the low-risk and retrospective nature of the study. All experiments and data processing were performed in accordance with retrospective personal identification guidelines and regulations.
This study was based on a cohort of patients admitted to CHUM’s ICU from 1 January 2012 to 31 December 2019 with a minimum hospital stay of 16 hours. For patients who experienced multiple her ICU admissions, we included only the data from her most recent ICU stay to avoid processing correlated data. Data were collected retrospectively from her EHR data.
The predicted outcome was to become a potential organ donor. Potential organ donors were defined as patients belonging to one of her four subgroups: (1) a locally identified actual organ donor (he was admitted to CHUM for symptoms that ultimately led to death and organ retrieval); (2) an actual organ donor who was diagnosed with neurological death at another hospital and he was transferred to CHUM for organ retrieval; (3) a potential organ donor referred her ODO for donation but was deemed ineligible for donation (surrogate decision maker refusal, workup detected medical contraindications, etc.); ). (4) Potential organ donors who have not been referred to ODO. Patients in this last category were identified by ODO through ongoing local mortality audits, and patients with recent cancer, active cancer, metastatic cancer, disseminated infection, or multiple organ failure were excluded. was Mortality audits screened potential organ donors as patients with critical neurological conditions, on mechanical ventilation, who died within 24 hours of the end of treatment, or who had critical neurological conditions. defined as patients who did not, were on a ventilator, and died within 3 hours after the end of treatment. end of care.
predictor
Variable selection
Predictors primarily included time series from laboratory analyzes and static clinical variables routinely collected as part of ICU care. We used only his two static variables: patient’s medical specialty (neurosurgery, internal medicine, cardiology, etc.) and presence or absence of cranial radiography.
Patients identified as potential organ donors receive different care than other patients (e.g., certain types of tests are performed more frequently, others are not performed) , took steps to avoid learning from differences in medical practice.20 Not a true clinical pattern, but an outcome of organ donation evaluation. First, demographic and anthropomorphic variables (such as age, biological sex, height, and weight) are not a priori exclusion criteria for potential organ donors and are not included as predictors. did not. Second, we excluded rare laboratory analyzes (defined as being requested in her <10% of all ICU patients) from the dataset. Finally, to mitigate the impact of increased frequency of frequent laboratory tests on potential organ donors, the last 72 hours of a patient's stay (defined as ending by ICU discharge or death) ) into 9 blocks of 8 hours each, and only the last value in each time block. The final list of experimental variables used in the model is reported in Table S1 (Supplementary Material).
missing value
The pattern of missing data is likely to be partially missing at random (because it is relevant to the desired outcome) and more likely to be partially missing without random (because of the tendency to avoid clinical laboratory orders, which are likely to betwenty one. Lack of medical specialization was treated as a separate category. To mimic a clinician thinking about missing laboratory values and increase the usefulness of the model in real-world settings, time data were imputed in his two-step method. First, the last value carry-forward (LVCF) method was applied so that the value of the laboratory analysis performed in his previous 8-hour block was carried over to all subsequent blocks until updated by new results. rice field. For values still missing after LVCF, we imputed randomly sampled values from an arbitrarily chosen Gaussian distribution. The mean and variance were chosen such that the distribution encompassed 95% of the physiological normal range for each laboratory analysis. Normal ranges for each laboratory analysis were provided by the laboratory. This decision likely reflected the hypothesis that a clinician’s decision not to order a laboratory test would be expected to have values similar to previous values or within the normal range, and the imputed This was done to reflect the fact that we are adding variability to the calculated values. values.
model development
model structure
We developed two models using different analytical approaches: the neural network temporal model (NN) and the logistic model (LM).
The NN model used the temporal aspect of inspection data combined with static values. Its architecture is shown schematically in Figure S1 and Figure S2 (Supplementary Material). We used a convolutional autoencoder (AE) to extract low-dimensional features from time-series laboratory data.22,23. AE is a subtype of NN architecture used in unsupervised learning. AEs are trained using the same information (images, text, experimental data, etc.) presented at the entrance and exit of the network. The data is compressed and transformed into a reduced size and dimensional representation called the latent representation, and then decoded back to its original form. AE learns how to encode data with minimal loss of information. This type of architecture has been found useful in a number of ways, including embeddings and dimensionality reduction tools to reduce noise.twenty fourfor transfer learning and pre-trainingtwenty five As an anomaly detection approach. We designed a convolutional AE based on ResNet26the well-known convolutional neural network (CNN)27 Using Python (version 3.7.4) and Keras (version 2.2.4) with TensorFlow (version 1.14)28, 29, 30. We adapted it to slide only along the temporal dimension of the inspection data and detect patterns of temporal changes in inspection values. To avoid overfitting, we used a dropout layer and L2 regularization in our model. The AE was used to (1) extract the maximum value from the temporal component of the data, (2) embed the temporal data in a one-dimensional format, and (3) be trained on non-objects only, thus acting as an anomaly detector. . (4) serves as pre-training for classifiers of smaller size. Our AE was trained using only time data from a cohort of non-donor patients only. For the classifier model, we used a deep NN consisting of four fully connected layers ending with a sigmoidal activation layer. The number of layers was chosen to balance capacity and complexity. The temporally embedded data were concatenated with static clinical data. Static data were encoded as binary data or as target mean encoding with smoothing for multiclass data. The final architecture is shown in Figures S1 and S2 (Supplementary Material).
The logistic model (LM) used is a lighter baseline comparison model using only last examination values before ICU discharge or death concatenated with static values using sigmoidal activation strata. This model included only the last time points of the time series, which greatly reduced the complexity of the forecasting algorithm, but also the number of data.
data structure
Potential organ donors represent a rare subpopulation, comprising approximately 2% of ICU patients based on preliminary data surveys. We combined purposeful subsampling and oversampling to tackle the problem of class imbalance.31. 85% of non-donor patients were randomly selected and used as the embedding training set. This allowed for subsampling while using this data for autoencoders that didn’t need the results to develop a latent representation. The remaining patients (15% of non-donors and all potential donors) were randomly divided into training/validation/test sets (60%/20%/20%). Minorities were given proportionally more weight during training. In other words, if the number of non-donors outnumbers the number of donors by 100 to 2, the training weight for donors is 100/2 = 50 and the weight for non-donors is 1. The training and validation sets were used for model development, while the test set was excluded from model development. model development process. After the final model was developed, it was trained on both the training and validation datasets and the results were estimated on the test set.
statistical analysis
The final model was trained by combining the training and validation sets and the results were estimated using the test dataset. We reported the performance, discrimination, and calibration properties of our models (NN and LM). We compared the overall performance of the NN model to the LM model using scaled Bryes scores, discriminant features using AUROC, and calibration using a calibration curve.20. Confidence intervals were obtained using nonparametric bootstrap percentiles (using 2000 resamplings of the test set). AUROC and Brier scores of different models were compared using Z-statistics with bootstrap-based standard errors (2000 resamplings) developed for paired models.32.
In our approach to maximizing detection of potential donors, the goal was to select a sensitive threshold. Due to potential future applications in this study, the optimal cutoff had to be derived from training data rather than test data. Cutoffs were derived using a triple cross-validation approach performed on the training dataset choosing an average threshold giving 90% sensitivity.
Two subgroup analyzes were performed. First, we compared her AUROC curves for the four subgroups by calculating the discrimination of each subgroup compared to the non-donor population. We then simulated a prospective approach to compare accuracy 48, 24, and 8 hours before ICU discharge. This was done using only progressively available data points from hospital discharge. Finally, we performed two sensitivity analyses. First, to ensure the model’s resilience to the removal of rare test values, we iteratively retrained the model after incrementally removing predictors. It started with her presence of the predictor in less than 10% of all her ICU patients, then 20%, then 30%. In doing so, the model was trained with fewer laboratory types at each step, and only the most frequently used laboratory types were used incrementally. We analyzed the performance of the model at each step of the bootstrap approach. We then manually reviewed the files of patients who were not reported as potential organ donors in the death audit, but were predicted as organ donors with a high degree of confidence (>75% predictive probability) by one of the models. bottom. We qualitatively describe this data to help us determine the actual outcome of missing patients in manual audits and to better understand potential systematic biases and error patterns in our models. bottom. Statistical analysis was performed using R (version 4.1.3) and Python (version 3.7.4).29,33.
