Theoretical analysis of MOFs for pharmaceutical applications by predicting load capacity and cell survival using machine learning models.

In this work, we employ a stacked regression approach to calculate cell viability (%) and drug loading capacity (G/g). Several techniques were incorporated to improve model performance and robustness (see Figure 5).

First, we used PCA to reduce the dimensions of the functional space. This technique reduces dimension curses and increases the efficiency of the model.

Second, we used WCA to optimize the hyperparameters in the base model. WCA is a powerful optimization algorithm that effectively explores hyperparameter spaces and leads to improved model performance and generalization.

The base model includes MLP, QR, and RF. MLP, a kind of neural network, can learn complex patterns of data. By modeling the conditional quantiles of the target variables, quantitative regression provides a more complete knowledge of the distribution. An ensemble learning, RF fuses numerous decision trees to improve prediction accuracy and reduce the likelihood of overfitting. Finally, the metamodel is trained to calculate the base model to make the best predictions. By combining these methods, we aimed to create reliable and accurate predictive models of cell viability (%) and drug loading capacity (G/g).

Multilayer Perceptron (MLP)

The MLP architecture can predict target variables using interconnected inputs, hidden, and output layers¹³. MLP is a group of ML-based models in which the output of hidden neurons is calculated via aggregation functions;^14,15:

where w and b See the weight and bias of the mode to be determined by the fitting.

The final output of the last layer of the MLP structure is^15,16:

In this regard, \(\:{got it}\) It refers to the output of the neuron of k-th output layer, while \(\:{w} _{jk} \) Represents weight¹⁵.

Random Forest (RF)

RF is a tree-based ensemble learning algorithm widely used for data classification and non-parametric regression, and can handle the datasets utilized in this study. A collection of decision trees is generated in an RF regression model^17,18. RF model formula k Trees, t(x))^19,20:

This method involves randomly resampling the original dataset leading to the subset without removing the data points on selection. \(\:\left[{\varTheta\:}_{i}\right)]\) It is generated with the same distribution²⁰. Therefore, some data points may be used multiple times, while others may only be displayed once. This approach shows greater resilience to mild variation in the input data, thus improving prediction accuracy and overall stability. Additionally, the RF tree finds the best feature or split points from a larger input dataset by randomly selecting a subset of features^{17, 21, 22}.

Quantile Regression (QR) Model

The QR method evaluates the association between response variables and independent variables at different quantiles of response variables and response distributions. In contrast to the usual least squares (OLS) regression that emphasizes the modelling of conditional mean, QR examines conditional quantiles that include median (50th percentile) and other points such as the 25th or 75th percentile. This allows you to understand in greater detail how predictors affect response variables at different levels^{twenty three}.

In traditional, ordinary least squares (OLS) regression, the model optimizes the residuals to calculate the mean of the response variables. In contrast, QR optimizes the weighted sum of absolute residuals. This makes quantile regression particularly useful when the error terms are not normally distributed or when the data contains data.^{twenty four}. For example, income analysis shows how quantile regression affects lower, middle, high, and high-income levels, factors such as education and experience, revealing insights missed in the mean-based model.

Stacking Ensemble Method

Unlike methods such as bagging and boosting, which usually rely on uniform models, Stacking emphasizes a variety of base models. The metamodel is used to generate final predictions, thereby exploiting the strength of the individual base model.

Here, stacking ensemble methods were used to determine cell viability and drug loading capacity. The framework included two levels of modeling.

1. Base Model: The base model consisted of MLP, RF, and QR. Each model captured various aspects of the data.

• MLP was excellent at learning complex correlations in datasets.

• RF provided robust predictions by reducing overfitting of the dataset.

•Quintile regression provided insight into different parts of the target distribution and was particularly effective in data sets with skewed or hetero-exit flow error structures.

2. Meta-Model: The meta-model was trained on the predictions generated by the base model. This step is important as you learn to assign appropriate weights to the output of the base model.

The stacking process included the following steps:

1. Training-Based Models: Each base model is trained independently, utilizing preprocessed features through principal component analysis (PCA) for dimensional reduction.

2. Generating base predictions: The trained base model generated predictions on holdout validation sets, confirming that an invisible data metamodel was trained to prevent data leakage.

3. Metamodel Training: Trained as input by the base model, and actual target values are trained as output.

4. Final Prediction: For test data, predictions were taken from the base model and the final output was generated using Meta-Model.

Implementing the stacking ensemble method in this study significantly improved the predictive accuracy of both cell viability and drug loading capacity. This highlights the possibility of stacking to handle complex datasets where there is not enough single models to effectively capture all the underlying patterns.

Water circulation algorithm (WCA)

The method of optimization of WCA is inspired by the natural manipulation of the water cycle, which simulates various steps of the water cycle, such as evaporation, river formation, and precipitation, to optimize the ML model.^25,26. During the evaporation stage of the WCA, possible solutions are evaluated according to the fitness rating. Fitness values are essential to check the evaporation rate and adjust the amount of water evaporating from each solution^26,27,28.

The steam vapor condenses into clouds during the precipitation stage and is randomly scattered across the solution set. All clouds could potentially improve the solution. All cloud fitness levels are evaluated. The highest quality one is selected²⁹. The selected cloud creates a river by dictating the current solution. This river indicates a change in solution parameters. The selected cloud platform and current solution branch out and needs to be changed²⁶. Attaching predefined fitness values, exceeding iteration limits, and performing assigned calculation times are all one of the stop criteria for the WCA method^26,30.

In this study, the WCA was set at population size of 60, maximum repetition of 120, and initial river distribution ratio consisted of 0.35, balancing exploration and exploitation, effectively balancing output estimates. The movement of the stream towards the river and the sea followed standard WCA flow dynamics, causing a reactivation of the evaporation and rain processes when stagnation was detected. For fitness assessment, we utilized a 3x cross-validation mean measurement coefficient (R²SCORE) to ensure robust and generalizable performance across different data splits. This fitness feature guides optimization by maximizing the predicted quality of the model.

The WCA computational cost for hyperparameter optimization was approximately 67 minutes to find the optimal hyperparameter for all models on a standard computing setup. Despite this cost, WCA was selected for its effectiveness in improving model performance by thoroughly investigating the hyperparameter space. Dimensional reductions through principal component analysis and parallel processing were used where feasible to mitigate computational demand.

Performance Metrics

Prediction accuracy of the stacking regression model was assessed using three important metrics: R-squared (R²), route mean error (RMSE), and maximum error. It is defined as r² \(\:{r}^{2} = 1- \frac {\sum \:{\left({y}_{i} – \widehat {{y}_{i}}\ri) ght)}^{2}} {\sum \:{\left({y}_{i} – \stackrel { – }{y}\right)}^{2}} \)measure the percentage of variance of the target variables (drug loading ability or cell viability) explained by the model, and values close to 1 indicate a better fit. rmse, as \(\:\text{rmse} = \sqrt{\frac{1}{n}\sum \:{\left({y}_{i} – \widehat{{y}_{i}}\right)}^{2}}}\)quantifies the average magnitude of prediction error. Here, the lower accuracy reflects the higher accuracy. Maximum error, given by \(\:\text {max \:error} = \text {max} \left | {y}_{i} – \widehat {{y}_{i}} \right | \)represents the largest deviation between the predicted and actual values, highlighting the worst performance of the model. These metrics are important to assess the ability of the model to accurately and reliably predict MOF properties and to ensure applicability in optimizing drug delivery systems.