In this study, a comprehensive analysis was conducted to predict the amount of bleeding that would occur during rhinoplasty procedures. In particular, we used several regression models including Linear, random forest (RF), support vector regression (SVR), Extreme Gradient Boosting (XGBoost), Ridge, and least absolute shrinkage and selection operator (LASSO) as primary analytical tools to evaluate the relationship between intraoperative bleeding volume, surgical factors, and patient characteristics. The goal of this strategy is to determine which drug will minimize blood loss during surgery.
The experiments were conducted to implement the models in the Google Colab environment, utilizing a system equipped with an Intel Core i7 processor with 16 GB of RAM running on Windows 11.
Data set description
This retrospective, single-center study analyzed data from patients who were candidates for elective open rhinoplasty and consented to participate in a clinical trial between 2019 and 202237. Initially, a mean comparison algorithm and specific inclusion criteria were employed to screen 135 people. A total of 120 individuals were chosen for the trial after 15 patients were eliminated for non-cooperation. These participants were randomly assigned to one of three groups: the clonidine group (40 patients), the TXA group (40 patients), and the control group (40 patients).
To qualify for participation, individuals had to be between the ages of 18 and 60, provide informed consent, and meet American Society of Anesthesiologists (ASA) classifications I or II, indicating that they did not have any serious systemic issues. However, patients with moderate systemic diseases that were well-controlled were included. Individuals with a history of substance misuse, liver or kidney failure, diabetes, respiratory or cardiovascular issues, coagulation disorders, or allergies to TXA and clonidine were excluded from the study. The outcome of interest was blood loss during the procedure which was measured by weighing pre-weighed blood-soaked gauze and calculating the total volume of blood collected in suction containers. 80% of the trial participants were female, and the average patient weight was 68 kg. Figure 1 illustrates the process used to select patients for experiments.
Patient selection flowchart for this study.
The next section will go into further detail about the variables associated with the data. All predictor variables are shown in Table 1.
Study roadmap
This study included three phases: 1- dataset preprocessing, 2- model implementation, and 3-evaluating the algorithms’ performance. The roadmap of this study is depicted in Fig. 2.
Dataset preprocessing
The raw data obtained from the hospital serves as the foundation for our analysis, but it may contain inconsistencies, inaccuracies, and irrelevant information. Effective preprocessing of these data is essential to guaranteeing reliable and trustworthy conclusions in any analytical investigation. Preprocessing is a critical phase in data analysis that enhances the quality and usability of the dataset, transforming raw data into a structured and usable format38,39. This phase improves the prediction models’ performance in addition to the data’s quality40. Managing missing values, dealing with outliers, balancing and normalizing data, and encoding categorical variables are examples of common preprocessing methods41. Upon reviewing the dataset, we found it to be well-balanced, so no additional balancing techniques were necessary. Additionally, the dataset was complete because no missing values were found. To improve the usability of the data, it was observed that normalization, encoding, and outlier detection were necessary. Each preprocessing step taken on the dataset is described in depth in the sections that follow.
Normalization: To ensure that each feature contributes equally to the analysis, we needed to scale the data. We applied min–max normalization using the following formula, which converts the features to a range between 0 and 142:
$${x}_{normal}=\frac{x- {X}_{min}}{{X}_{max}- {X}_{min}}$$
(1)
In this context, \({X}_{min}\) and \({X}_{max}\) represent the minimum and maximum values of each column, respectively.
Encoding: Most machine learning algorithms require numerical input, so categorical variables need to be converted into a numerical format. Two common techniques for this are label encoding and one-hot encoding. One-hot encoding creates binary columns for each category, allowing algorithms to process categorical data more effectively43.
Outlier detection: Outliers can greatly affect the performance of regression models. Various methods, such as the Z-score method and the IQR method, can be employed to identify and manage outliers44,45. The Z-score is calculated as:
$$Z=\frac{X- \mu }{\sigma }$$
(2)
where, \(\mu\) is the mean and \(\sigma\) is the standard deviation of the dataset. Outliers can be defined as values for which \(\left|Z\right|>3\).
Model implementation
We employed several regression techniques, which are effective for predicting continuous outcomes based on input features, to achieve our objective. Our goal was to thoroughly investigate how different medications, particularly TXA and clonidine, influenced the reduction of blood loss during surgery. By identifying the most effective drug, we aimed to improve surgical efficiency and enhance patient safety.
We chose the recommended regression algorithms due to their proven effectiveness in managing potential overfitting and complex correlations present in the data. These algorithms each offer unique advantages, allowing us to thoroughly evaluate the predictors of blood loss. By analyzing the outcomes from these models, we can identify which drug most effectively reduces bleeding. Using these algorithms systematically helps us better predict blood loss and understand how different preoperative procedures impact surgical outcomes. This approach not only enhances our understanding of the factors influencing blood loss but also provides surgeons with valuable insights to refine their techniques. In the following sections, we will provide a detailed overview of each algorithm used in this study, which includes regression models including Linear, RF, SV, XGBoost, Gradient Boosting, Ridge, and LASSO. Each of these algorithms was chosen for its strengths in predictive modeling, enabling a comprehensive analysis of the data.
Linear regression: A key statistical technique for predicting a continuous dependent variable using one or more independent variables46. Figure 3 illustrates how the linear regression model predicts results based on input features and successfully reflects the relationship between the independent and dependent variables. This method assumes a linear relationship between inputs and outputs, represented as follows:
$$y={\beta }_{0}+{\beta }_{1}{x}_{1}+{\beta }_{2}{x}_{2}+\dots +{\beta }_{n}{x}_{n}+\varepsilon$$
(3)
where, the independent variables are \({x}_{1}, {x}_{2}\), …, \({x}_{n}\), while the dependent variable we are predicting is \(y\). The intercept of the regression line is represented by \({\beta }_{0}\). The coefficients for each independent variable are also denoted by \({\beta }_{1}, {\beta }_{2}\), …, \({\beta }_{n}\). The error term, \(\varepsilon\), reflects the difference between the observed values and the predicted values47.
The Ordinary Least Squares (OLS) method minimizes the sum of the squared differences between observed values and the model’s predicted values to estimate coefficients.
$$Minimize\sum_{i=1}^{n}{\left({y}_{i}-{\widehat{y}}_{i}\right)}^{2}$$
(4)
While linear regression is simple, it is very useful, especially when a true linear relationship exists between the variables.
RF: An ensemble learning technique creates multiple decision trees during the training process and calculates the average prediction from each tree for regression tasks. By averaging the results, this method helps to minimize the risk of overfitting, which is common with individual decision trees48. A prediction made by a random forest can be expressed as follows:
$$\widehat{y}= \frac{1}{M}\sum_{m=1}^{M}{f}_{m}(x)$$
(5)
In the forest, the number of trees is denoted by \(M\). The prediction for the m-th tree is represented as \({f}_{m}(x)\).
A method called bootstrap aggregating, or bagging, is used to build the trees in a random forest. In this approach, portions of the training data are randomly selected with replacements. Additionally, when splitting nodes during the creation of the trees, a random subset of features is considered. This process helps reduce the correlation between the trees. As a result, the model becomes more robust and is capable of effectively handling high-dimensional data and non-linear interactions49.
SVR: Support Vector Machines (SVM) can be adapted for regression tasks, resulting in a method known as SVR50. SVR aims to find a function that is as flat as possible while ensuring that it deviates from the actual target values by no more than a predetermined margin51. As shown in Fig. 4, the SVR model identifies key support vectors to define the regression function. The primary objective is to minimize the following loss function, which allows for errors up to a specific threshold (epsilon):
Support vector regression model.
$$Minimize \frac{1}{2}{\Vert \omega \Vert }^{2}+C\sum_{i=1}^{n}{\vartheta }_{i}$$
(6)
where, \(\Vert \omega \Vert\) is the weight vector’s norm. The balance between flatness and error tolerance is managed by the penalty parameter \(C\). The difference between each anticipated value and the actual value is measured by the slack variables \({\vartheta }_{i}\).
The predicted function is given by:
$$f\left(x\right)= \langle \omega ,x\rangle +b$$
(7)
where, the dot product of the weight vector \(\omega\) and the input vector \(x\) is denoted as \(\langle \omega ,x\rangle\) and \(b\) is the bias term.
To efficiently handle non-linear interactions, SVR can make use of a variety of kernel functions, including linear, polynomial, and radial basis functions52.
XGBoost regressor: XGBoost, is a powerful and flexible machine learning algorithm that effectively implements the gradient boosting framework. It constructs models incrementally, with each new model designed to correct the errors made by the previous ones53. The prediction made by XGBoost can be described as follows:
$$\widehat{y}=\sum_{k=1}^{K}{f}_{k}(x)$$
(8)
where, the variable K represents the total number of trees in the model and \({f}_{k}\left(x\right)\) is the prediction from the k-th tree.
Gradient boosting regressor: Another ensemble method that builds models sequentially is the Gradient Boosting Regressor. It aims to reduce the residual errors of earlier models by fitting new trees to the error terms54. The general formulation of the prediction is:
$$\widehat{y}= {\widehat{y}}_{prev}+ \eta f(x)$$
(9)
where, the prediction from the previous iteration is represented by \({\widehat{y}}_{prev}\). Each tree’s contribution to the final prediction is determined by its learning rate, represented by η. The newly added model to the ensemble is \(f(x)\).
By utilizing gradient descent and making iterative adjustments to the model based on the errors identified in the previous phase, this method minimizes a specific loss function, such as mean squared error. This approach can effectively capture complex relationships within the data and provides flexibility in how the model is specified55.
Ridge regression: To prevent overfitting, particularly when multicollinearity exists among the variables, Ridge Regression—a type of linear regression—employs L2 regularization56. The cost function for Ridge Regression is defined as follows:
$$Cost Function= \sum_{i=1}^{n}{\left({y}_{i}-{\widehat{y}}_{i}\right)}^{2}+ \lambda \sum_{j=1}^{p}{\beta }_{j}^{2}$$
(10)
where λ is a regularization parameter that discourages large coefficients. \({\beta }_{j}\) represents the coefficients of the predictor variables.
The solution is stabilized by a regularization term that prevents the coefficients from becoming excessively large. Ridge regression is particularly useful when there are more predictors than data points or when the predictors exhibit high correlation57. Figure 5 illustrates the Ridge regression model.
Lasso regression: Lasso Regression, or Least Absolute Shrinkage and Selection Operator, is similar to Ridge regression but utilizes L1 regularization, which promotes sparsity in the model58. The cost function for Lasso Regression is expressed as follows:
$$Cost Function= \sum_{i=1}^{n}{\left({y}_{i}-{\widehat{y}}_{i}\right)}^{2}+ \lambda \sum_{j=1}^{p}\left|{\beta }_{j}\right|$$
(11)
Let \(\left|{\beta }_{j}\right|\) represent the absolute values of the coefficients. The L1 penalty in Lasso Regression can shrink some coefficients to zero, which effectively selects a subset of variables. This process simplifies the model and improves its interpretability. Lasso Regression is especially useful for high-dimensional datasets, where it is anticipated that only a few features will be significant59.
Model evaluation criteria
In this study, we predicted the volume of blood loss during rhinoplasty using regression-based machine learning algorithms, including Linear Regression, Random Forest Regression, Support Vector Regression (SVR), XGBoost Regressor, Gradient Boosting Regressor, Ridge Regression, and Lasso Regression. We evaluated the performance of these algorithms with three key metrics: Mean Absolute Error (MAE), Mean Squared Error (MSE), and R-squared (R2)60,61. These metrics provide a comprehensive understanding of the models’ reliability and predictive accuracy in this context.
MAE: This metric evaluates the average magnitude of errors in a set of predictions, ignoring their direction. It is calculated as follows:
$$MAE=\frac{1}{n}\sum_{i=1}^{n}\left|{y}_{i}-{\widehat{y}}_{i}\right|$$
(12)
Here, n refers to the total number of observations, \({y}_{i}\) stands for the actual values, and \({\widehat{y}}_{i}\) denotes the predicted values. Better model performance is indicated by a lower MAE62.
MSE: This metric measures the average of the squared errors, giving more weight to larger errors compared to MAE. It is calculated using the following formula:
$$MSE=\frac{1}{n}\sum_{i=1}^{n}{\left({y}_{i}-{\widehat{y}}_{i}\right)}^{2}$$
(13)
Like MAE, a lower MSE number suggests that the model fits the data more closely63.
R-Square (R2): It shows how effectively the independent variables account for the dependent variable’s variability. It is computed as follows:
$${R}^{2}=1- \frac{\sum_{i=1}^{n}{\left({y}_{i}-{\widehat{y}}_{i}\right)}^{2}}{\sum_{i=1}^{n}{\left({y}_{i}-\overline{y }\right)}^{2}}$$
(14)
where \(\overline{y}\) represents the actual values’ mean. The model explains a sizable amount of the variance when the \({R}^{2}\) value is nearer 1, which denotes a greater link between the independent and dependent variables64,65.
To ensure the robustness of model evaluation, K-fold cross-validation was employed to divide the dataset into training and testing subsets66,67. In this procedure, the data is split into k subsets or folds. Each fold is used as a testing set, while the remaining K—1 folds serve as the training set. By iterating this process, we reduce the risk of overfitting and ensure that the selected algorithms perform well when applied to new data. This approach provides a more accurate assessment of model performance68,69. In conclusion, the combination of these evaluation metrics and the K-fold cross-validation method enhances the reliability of the performance assessment of regression algorithms in predicting blood loss volume during rhinoplasty procedures.
Ethical considerations
This article is extracted from a research project supported by Abadan University of Medical Sciences and all experimental protocols were approved by the ethical committee of Abadan University of Medical Sciences (approved number: IR.ABADANUMS.REC.1403.137). All experiments were performed in accordance with relevant guidelines and regulations of the ethical committee of Abadan University of Medical Sciences. Participation was voluntary and informed consent was obtained from all subjects and/or their legal guardians. Participants had the right to withdraw from the study at any time. Clinical trial number: not applicable.
