Fine tuned CatBoost machine learning approach for early detection of cardiovascular disease through predictive modeling

In this study, the researchers used a Lenovo system with an Intel® Core™ i5-10210U CPU @ 1.60GHz (up to 2.10GHz) to implement machine learning algorithms and models. The experiments were conducted in a web-based application environment using Jupyter Notebook, which provides access to computing resources such as a GPU for deep learning training. The machine was equipped with 16 GB of RAM.

Data collection

The dataset on heart conditions analyzed in this research combines datasets from the Kaggle Repository³⁴. This dataset includes twelve attributes, among which heart failure can be predicted as a serious risk factor for cardiovascular death.

The dependent binary classifier variable of the target is the presence or absence of heart failure. The summary of the target variable is presented in Table 1. Table 2 lists the features and their descriptions.

This data set comprises five originally separate heart datasets and is presently the most extensive heart disease data set available for research. The present heart disease dataset has a total observation of 918 and the total number of features is 12³⁴. Table 3 summarizes the key statistics for the numerical features. The mean age is 53, with a maximum of 77.

Table 4 provides the statistics for categorical attributes. The ChestPainType attribute comprises four distinct categories: ASY, NAP, ATA and TA. “ASY” is the most prevalent category, representing 77.2% of the observations.

Table 5 summarizes the distribution of the Sex variable. The dataset includes male (90.2%) and female (9.8%) patients.

Exploratory data analysis

The dataset represents a balanced proportion of patients with and without heart disease. Out of the 918 patients; 508 out of the total 918 cases were found to have heart failure while 410 were healthy. Those with heart disease had the median age of 57 months, compared to a median age of 51 months among those who do not have heart disease. Figure 2 shows the distribution of patients with heart diseases in different groups according to their age. As shown in the Fig. 3 medians are split between the heart diseased and heart healthy patients. Figure 4 illustrates the distribution between patients with and without heart disease regarding age, systolic blood pressure, cholesterol, heart rate, and ST segment depression. Overall, those patients with a heart disease had higher values of these variables, while younger patients sometimes had outliers. For men and women, heart disease trends are compared in Fig. 5.

Figure 6 comprises two grouped bar charts illustrating the relationship between selected categorical clinical features and the diagnosed presence or absence of heart disease. The left panel, “ExerciseAngina vs Heart Disease,” depicts the distribution of heart disease based on the presence (Y) or absence (N) of exercise-induced angina. It clearly shows a substantially higher count of heart disease cases (n = 316) among individuals with exercise angina compared to those without (n = 192), while individuals without angina predominantly did not have heart disease (n = 355). The right panel, “ST_Slope vs Heart Disease,” visualizes heart disease incidence across different ST_Slope categories (Up, Flat, Down). This chart reveals a strong association of “Flat” (n = 381) and “Down” (n = 49) ST slopes with heart disease, whereas an “Up”sloping ST segment is predominantly observed in individuals without heart disease (n = 317). Collectively, these charts highlight a notable increased incidence of heart disease in patients presenting with exercise-induced angina or with Flat/Down ST segment slopes. Continuing with the analysis of categorical features, Fig. 6 further illustrates the distribution of heart disease cases based on patient sex and chest pain type. The left panel (“Sex vs Heart Disease”) reveals a markedly higher absolute count of heart disease diagnoses among males (M, n = 458) compared to females (F, n = 50), although males also represent a larger portion of the non-diseased group. The right panel (“ChestPainType vs Heart Disease”) indicates that Asymptomatic (ASY) chest pain is associated with the highest number of heart disease cases (n = 392), whereas Atypical Angina (ATA) is more prevalent in individuals without heart disease (n = 149), suggesting differing diagnostic implications for various chest pain presentations. Figure 6 also examines the association of Fasting Blood Sugar (FastingBS) and Resting Electrocardiogram (RestingECG) results with heart disease. The left panel (“FastingBS vs Heart Disease”) shows that elevated FastingBS (category 1) is linked to a considerably higher incidence of heart disease (n = 170) compared to individuals without heart disease in the same category (n = 44). The right panel (“RestingECG vs Heart Disease”) indicates that while ‘Normal’ ECG results are seen in both groups, ST-T wave abnormalities (‘ST’, n = 117) and Left Ventricular Hypertrophy (‘LVH’, n = 106) are more frequently observed in patients diagnosed with heart disease.

Figure 7, the “Correlation Matrix,” is a heatmap that visually displays the linear relationships between several numerical variables from a heart disease dataset, such as Age, Cholesterol, MaxHR, Oldpeak, and the target variable, Heart Disease. Each cell in the matrix shows a correlation coefficient, with the color intensity and hue indicating the strength and direction: dark red for strong positive correlations (values approaching + 1), dark blue for strong negative correlations (values approaching − 1), and lighter shades for weaker links. The matrix reveals that “Oldpeak” has the strongest positive correlation with ‘Heart Disease” (0.40), while “MaxHR” has the strongest negative correlation (− 0.40). Other features like Age (0.28) and FastingBS (0.27) show weaker positive associations with heart disease. Additionally, a notable inter-feature correlation is between “Age” and “MaxHR” (− 0.38), suggesting that maximum heart rate tends to decrease as age increases. This visualization is crucial for identifying which numerical features have more significant linear associations with each other and, importantly, with the presence of heart disease.

Figure 8 presents a collection of six pie charts, each visually summarizing the percentage distribution of different categorical variables within a dataset, likely related to cardiovascular health. The “Sex” chart indicates a significant male majority (90.2%). “Chest Pain Type” is predominantly Asymptomatic (ASY) at 77.2%, while “Fasting Blood Sugar” reveals that 33.5% of subjects have levels greater than 120 mg/dl. Furthermore, “Resting ECG” results are ‘Normal’ for 56.1% of the population. “Exercise Induced Angina” is present in 62.2% of individuals. Lastly, the “ST Slope” chart shows that a ‘Flat’ slope is the most common, occurring in 75.0% of cases. These charts collectively provide a baseline understanding of the prevalence of these characteristics in the sample.

The dataset shows slightly more individuals with heart disease than without, indicating a somewhat balanced distribution. The analysis identified moderate correlations for Oldpeak (0.40) and MaxHR (− 0.40) with heart disease, while others were weaker. Patients with asymptomatic chest pain are notably more likely (approx. 5.7 times) to have heart disease compared to those with atypical angina; respiratory disorders were not assessed. Men in this sample are over twice as likely (approx. 2.4 times) to have heart disease compared to women. Third, impaired ST-T waves (ST or LVH categories) in resting ECG are associated with a higher proportion of heart disease cases. Angina developed during exercise is a strong indicator, with a high proportion (approx. 85%) of such individuals having heart disease. The extent of ST segment depression (Oldpeak) is positively correlated with heart disease and shows a higher mean in affected patients.

Performance evaluation

As with any measure dealing with imbalanced datasets, using accuracy-based calibration of the classifier can be misleading. To deal with this problem, other performance measures are usually applied in organizations. The confusion matrix makes it easier to estimate a classifier’s performance. It shows the correct and incorrect classifications as true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN). Standard performance metrics include accuracy, precision, recall (sensitivity), F1-score and the ROC curve. These metrics can be calculated using the formulas based on the TP, TN, FP and FN values in the test dataset.

Machine learning models

The gathered dataset (approximately 918 rows) was divided into training: 80% and testing 20%. The listed algorithms of machine learning were applied in the study: Extra Tree, Random Forest, Ada Boost, Gradient Boosting and CatBoost. Given these tendencies, a proper evaluation approach must be selected to achieve a more reliable and high-performing classifier. K-fold cross-validation is an example of partitioning a matrix into k equal portions that are equally divided and exclusive. On each iteration, one of the subsets is selected for testing while the rest is used for training. This is done for k times and goes through all the subsets. At each iteration, some metrics are computed from them. In particular, K-fold cross validation provides comparatively good measure of the model’s generalization ability and its ability to operate efficiently on small data amounts. It assists in avoiding overfitting by testing the ability of the model to work on unseen data sets as a s result of cross-validation. Positive or negative values of hyperparameters are crucial for optimizing the machine learning model. The other process of selecting the right algorithm requires identifying the correct parameters for each algorithm. One of the techniques used in hyperparameter optimization is grid search optimization. Table 6 presents the results of hyperparameter optimization for five machine learning models and one proposed model: Extra Trees, Random Forest, AdaBoost, Gradient Boosting, CatBoost, and fine-tuned CatBoost. These findings point to the critical importance of optimization of hyperparameters for development of better performance of the machine learning model.

Model performance on validation test

Figure 9 presents a Receiver Operating Characteristic (ROC) curve comparison for five distinct machine learning models: Random Forest, Gradient Boosting, Extra Trees, AdaBoost, and CatBoost, assessing their performance in classifying heart disease. The curves illustrate the trade-off between the True Positive Rate (TPR) and the False Positive Rate (FPR) at various classification thresholds. Notably, Gradient Boosting (orange line) and CatBoost (purple line) exhibit the highest discriminatory power, both achieving an Area Under the Curve (AUC) of 0.95. Random Forest (blue line) and Extra Trees (green line) also demonstrate strong performance with an AUC of 0.94 each. AdaBoost (red line) shows a slightly lower, yet still good, performance with an AUC of 0.90. These results suggest that Gradient Boosting and CatBoost provide the best balance between sensitivity and specificity among the evaluated models.

Table 7 shows the results for recall, precision and accuracy for the proposed CatBoost model. The presented classifier reveals a high accuracy in detecting heart disease. One should also note that CatBoost is also especially vulnerable to issues related to class imbalance. F1 score is an evaluation of precision and recall, meaning that it provides a balanced perspective on the performance of a model in categorization. The proposed model outstandingly improved heart disease classification with 0.9902 accuracy, 0.9904 precision, 0.992 recall, and 0.992 F1 score.