A hybrid explainable model based on advanced machine learning and deep learning models for classifying brain tumors using MRI images

Proposed framework

Figure 1 illustrates an innovative framework devised for the classification of three distinct types of brain tumors (BTs) from MRI images, alongside the classification of images depicting the absence of tumors. During training, the RRELM model uses fivefold cross-validation to evaluate and save the best-performing model, which is then used for final testing. When tested, the optimized model provides robust four-class classification, effectively distinguishing between different types of brain tumors and non-tumor cases. The entire framework showcases the power of combining a parallel CNN for feature extraction with ridge regression regularization in the classifier, achieving an efficient and accurate medical image classification solution.

An explanation generated using the Shapley Additive Explanation (SHAP) method was employed to gain insights into the black-box nature of the PDSCNN-RRELM model. This innovative visualization technique provided a deeper understanding of the model’s decision-making process and highlighted the most influential features contributing to the classification outcomes. In the subsequent sections, each step of the proposed framework is presented in greater detail, offering a comprehensive explanation of the methodology employed and the remarkable contributions made toward the accurate identification and classification of BT in MRI scan images.

Brain tumor dataset

The brain tumor dataset utilized in this study was acquired from the esteemed Kaggle repository. The dataset comprises four distinct classes: glioma, meningioma, no tumor, and pituitary tumor. It encompasses a total of 7,023 MRI images distributed across the following classes: glioma (1,621 images), meningioma (1,645 images), no tumor (2,000 images), and pituitary (1,757 images). A rigorous evaluation strategy was employed to ensure the reliability and generalizability of the model. Considering the critical role of brain tumor data in medical and clinical applications, ensuring patient privacy and adhering to ethical standards are fundamental to the integrity of this study. This dataset³⁶ is sourced from publicly available repositories (the figshare, SARTAJ, and Br35H datasets), all of which are intended for research purposes. These repositories are typically structured to anonymize personal data, removing any patient-identifying information, thus aligning with the HIPAA and GDPR guidelines for data privacy. Additionally, dataset collection and processing followed the ethical standards and guidelines established for publicly available medical imaging data. No identifying features are linked to individual patients in the dataset, ensuring that the privacy of individuals is maintained throughout the study. The dataset was divided such that 80% of the images were allocated for training purposes, whereas the remaining 20% were reserved for testing the novel RRELM model with a fivefold cross-validation methodology. Figure 2 shows a selection of MRI samples to provide a visual representation of the dataset.

While Fig. 2 visually represents tumor shapes, it is imperative to highlight the inherent variability of the dataset, encompassing background characteristics, imaging artifacts, and tumor appearances. The presence of surrounding tissues such as the brain parenchyma, blood vessels, or ventricles introduces considerable background variations across MRI scans. Additionally, factors such as imaging parameters, scanner settings, and patient-specific traits further contribute to image diversity. Moreover, brain tumors manifest diverse shapes, sizes, and locations, leading to a spectrum of appearances on MRI images. The differentiation of gliomas, which are known for having irregular margins and infiltrative patterns, from healthy tissues is challenging. Conversely, meningiomas typically present as well-defined extra-axial masses, often adjacent to the dura mater.

The complexity of classifying these images extends beyond distinguishing tumors from backgrounds. Accurate identification and categorization of tumor types based on morphological characteristics and spatial relationships within the brain are paramount. Hence, while Fig. 2 provides insight into tumor shape diversity, it is essential to underscore the complexity of the classification task. The proposed method aims to address these challenges by leveraging advanced machine learning techniques to extract discriminative features effectively, facilitating accurate classification across tumor types. By recognizing the nuances in the dataset’s composition and the intricacies of brain tumor characterization, our study endeavors to provide a comprehensive understanding of the complexities involved in MRI-based tumor classification.

Data preprocessing

In the domain of image preprocessing, researchers have recognized the paramount importance of achieving optimal results. To this end, various methodologies have emerged, each aiming to enhance medical images for improved outcomes. In this study, the researchers employed the Contrast Limited Adaptive Histogram Equalization (CLAHE) technique to address the challenge of enhancing image contrast and model accuracy. Originally designed to address the inherent limitations of low-contrast medical images, CLAHE operates by constraining the amplification process within a user-defined threshold referred to as the clip limit³⁷. This threshold plays a crucial role in smoothing out the noise in the histogram, consequently dictating the extent of contrast improvement achieved. For this study, the researchers judiciously set the clip limit at 2.0 while also adopting an (8 × 8) tile grid size to optimize the transformative potential of CLAHE in their pursuit of superior image enhancement. Figure 3 shows a selection of original MRI images alongside their corresponding counterparts, which have undergone enhancement using the CLAHE method. The transformative impact of CLAHE can be observed as it effectively enhances the visual quality and details of the MRI images, enabling improved clarity and contrast for accurate analysis and interpretation. The database encompasses a diverse range of images, each presenting varying sizes, necessitating a harmonization process to facilitate seamless integration into the CNN model. In pursuit of this objective, the images underwent a conversion procedure, aligning them to a standardized size of 124 × 124 pixels. By employing the powerful CLAHE technique, the images were then enhanced, ensuring optimal visual quality and facilitating detailed analysis.

Recognizing the significance of managing complexity and promoting uniformity, the researchers implemented a normalization procedure. Given the abundance of intensity values employed to represent an image, normalization serves as a crucial step to mitigate the intricacies arising from the vast number of pixels. For this purpose, the scale of the images was meticulously adjusted from the original range of 0–255 to a normalized range of 0–1. This strategic adjustment involved dividing the pixel values of each image by 255, effectively streamlining the subsequent analysis by reducing complexity and enhancing computational efficiency.

Parallel depthwise separable convolutional neural network

The primary objective was to develop a CNN model capable of effectively identifying essential features while maintaining a compact structure suitable for diverse real-world applications. To balance the trade-off between parameter limitations and layer complexity, a streamlined PDS-CNN model was crafted and specifically designed to extract meaningful features with minimal resource utilization. The architecture depicted in Fig. 4 exemplifies the elegance of this lightweight approach for capturing distinctive features within the MRI images.

A refined strategy was adopted to simplify the proposed CNN model compared with the TL models. The model comprised nine convolutional layers (CLs) and two fully connected layers (FCs), which were carefully calibrated to strike a balance. Through iterative experimentation, it was observed that a solitary CL would fail to capture crucial features, while a sequence of five CLs would introduce unnecessary depth and complexity. Consequently, the first five CLs were thoughtfully executed in parallel, leveraging the insights gained from meticulous trial and error. Each CL was equipped with 256 kernels of varying sizes, ranging from 11 × 11 to 3 × 3. The choice of employing different filter sizes ranging from 11 × 11 to 3 × 3 is strategic. If identical filter sizes were utilized, they would extract redundant features, ultimately leading to a lack of diversity in the feature space. However, by incorporating varying filter sizes, each filter can capture distinct types of features. Upon concatenating the feature maps obtained from these filters, a rich and diverse set of features is derived. This diversity aids in capturing intricate patterns and nuanced details present in the data, thus enhancing the model’s ability to discern and classify objects accurately. The rationale behind selecting a range from 3 × 3 to 11 × 11 is rooted in the characteristics of the filter sizes. A 3 × 3 filter is considered ideal for detecting local features such as edges, corners, and texture details due to its smaller receptive field. Conversely, an 11 × 11 filter possesses a larger receptive field, enabling it to capture more global features or overarching patterns present in the input data. By encompassing filter sizes across this spectrum, the model can effectively leverage both local and global information, striking a balance between detailed feature extraction and holistic pattern recognition.

A constant padding size was employed across the initial five CLs, enabling comprehensive information extraction even from the border elements. The resulting feature maps generated by the parallel CLs were harmoniously merged and subsequently fed into a sequential CL.

To further refine the model’s simplicity without compromising its classification performance, a novel approach involving depthwise separable convolution (DSC) was adopted, supplanting conventional convolution methods. The DSC expertly dissects the convolution process into depthwise convolution and pointwise convolution, dramatically reducing the parameter count from 2.2 million to a mere 0.53 million. This judicious reduction in computational complexity not only streamlined the model but also enhanced its classification performance. Following the last four CLs, a batch normalization (BN) layer and a max pooling layer were integrated, and their sizes were optimized to 128, 64, 32, and 16, respectively, with 3 × 3 kernels and VALID padding. The integration of the BN expedited execution while bolstering the model’s stability. To introduce nonlinearity and facilitate training, a rectified linear unit (ReLU) activation function was employed across all the CLs. Furthermore, to combat overfitting and expedite training, dropout layers were strategically inserted, randomly disregarding 50% of the nodes. Specifically, two dropout layers were placed after the final two CLs, and an additional layer was positioned after the first FC layer. Considering that the learning rate was set at 0.001 using the ADAM optimizer, the model loss was adeptly managed through the application of the sparse categorical cross-entropy loss function, enabling efficient feature extraction. Employing a systematic trial-and-error approach, 256 distinctive features were thoughtfully selected from the final FC layer, which encapsulates the essence of the model’s capabilities. Algorithm 1 demonstrates the kernel concatenation algorithm used in Separable Conv2d.

Algorithm 1

In a concatenated layer, the number of filters returned after merging depends on the specific configuration of the layer. When concatenating two or more layers, each layer may have a different number of filters, and the resulting concatenated layer will have a total number of filters equal to the sum of the filters in each individual layer.

The total parameters in the sequential model after concatenation depend on the type of layers and their configurations. Each layer in the model contributes a certain number of parameters based on its architecture and the number of filters or units it contains. When concatenating layers, the total number of parameters in the sequential model will be the sum of the parameters from all the layers involved in the concatenation, including the concatenated layer itself.

In this study, the true power of the proposed model was determined by extracting the most discriminative features from intricate MRI images. To convey the remarkable results with the utmost clarity and visual appeal, the exceptional ability of the t-Distributed Stochastic Neighbor Embedding (t-SNE) technique was harnessed, as shown in Fig. 5. With unparalleled finesse, t-SNE seamlessly unraveled the inherent structure and intricate relationships within high-dimensional MRI data. This remarkable algorithm skillfully transforms the complexity of the data into a lower-dimensional representation while preserving the subtle nuances and intricate similarities among the data points.

To ensure the optimal performance of the Extreme Learning Machine (ELM) algorithm, the extracted features underwent a crucial step of standardization using the standard scaler, which transformed the feature values to a standardized scale, thereby enhancing the ELM model’s effectiveness in handling the data. By standardizing all the features, the subsequent feeding of these standardized features into the ELM model facilitated improved performance and more accurate analysis^10,38,39,40.

Interaction between PDSCNN and RRELM

The feature extraction process uses a parallel depthwise separable convolutional neural network (PDSCNN) to capture spatial features efficiently. Through its structure, the PDSCNN extracts 256 prominent features from each MRI image, providing a rich representation of tumor-specific patterns essential for accurate classification. After feature extraction by the PDSCNN, the network produces 256 feature maps (as shown in Fig. 6), each capturing distinct spatial features from the MRI images. These feature maps are then processed to form a single 1D feature vector, which serves as the basis for classification. Mathematically, if each feature map has dimensions of h×w, then flattening these maps results in a vector of size 256×h×w. However, for simplicity and computational efficiency, the mean or another statistical representation across each feature map is often taken, compressing the information into a 256-dimensional feature vector f, where:

Here, \({f}_{i}\) represents the summary value (such as the mean) of the i-th feature map, providing a compact, descriptive vector of the most prominent features detected by the PDSCNN.

Next, normalization is applied to this feature vector to scale the values within a consistent range. Let \({f}_{\text{norm}}\) denotes the normalized vector, where each component \({f}_{\text{i}}\) is transformed as:

where \({\mu}_{i}\) and \({\sigma}_{i}\) represent the mean and standard deviation of the i-th feature across the training set, respectively. This normalization ensures that the features are standardized, making them compatible for input into the next stage without being affected by scale differences.

In this pipeline, the PDSCNN and RRELM work as tightly integrated systems. The PDSCNN acts as the feature extraction module, producing \({f}_{\text{norm}}\) as its output, which becomes the direct input to the RRELM classifier.

The normalized feature vector \({f}_{\text{norm}}\) is then fed into the ridge regression extreme learning machine (RRELM), which is chosen for its robustness in handling high-dimensional and potentially correlated features. The RRELM model is an enhanced version of the classic extreme learning machine (ELM) that incorporates ridge regression regularization to control for overfitting, which is especially useful when dealing with intercorrelated features such as those derived from the PDSCNN.

In RRELM, the classification output y is calculated as:

where W is a matrix of weights learned during training, which maps the 256-dimensional feature vector to the four-class output (three tumor types and one non-tumor class), and b is a bias term added to adjust the final output.

The regularization term in RRELM adds a penalty proportional to the square of the weight matrix W:

where λ is a hyperparameter controlling the regularization strength. This regularization ensures that the model does not overfit, particularly when features are highly intercorrelated or redundant, as it encourages smaller, more stable weights.

Ridge regression extreme learning machine

Huang et al.⁴¹ introduced the ELM, a remarkable forward feed network based on supervised learning, to harness the power of neural networks (NNs) for feature classification. By employing a single hidden layer, the ELM revolutionized the training process by eliminating the need for backpropagation, enabling training times that were a thousandfold faster than those of conventional NNs. This groundbreaking approach endowed the model with exceptional generalization capabilities and unparalleled classification performance. Notably, in the realm of large-scale multiclass classification tasks, the ELM exhibited remarkable progress, surpassing traditional methods^10,42,43,44. While the parameters linking the input and hidden layers were initialized randomly, the parameters connecting the hidden and output layers were determined using the pseudoinverse technique. However, in this study, the pseudoinverse was replaced by ridge regression, which injected an additional layer of sophistication. The input layer boasted a grand ensemble of 256 nodes, the hidden layer consisted of 1,500 nodes, and the output layer was composed of four nodes for the classification of four types of BT from MRI images. The following steps outline the implementation details:

Initialization: The input weight matrix (input_weights) of size (input_size × hidden_size) and biases were randomly initialized using a normal distribution.

Activation function: The rectified linear unit (ReLU) function was used as the activation function for the hidden layer. It can be defined as relu(x) = max (0, x), where x represents the input.

Hidden layer computation: The hidden_nodes() function computes the hidden layer output (H_train) using the following equations.

G = X_train * input_weights + biases, where X_train is the input feature matrix.

H_train = relu(G), where relu(x) is the ReLU activation function applied elementwise to G.

Ridge Regression: Ridge regression was utilized to optimize the output weight matrix (output_weights). The ridge regression formulation can be expressed as follows.

output_weights = inv (H_train. T @ H_train + alpha * I) @ H_train. T @ y_train, where alpha is the regularization parameter, I represents the identity matrix of size hidden_size, and y_train denotes the target labels.

Prediction : The predict() function was introduced to make predictions using the trained model. It computes the output of the hidden layer for the given input features (X) using the following equations.

G = X * input_weights + biases,

H = relu(G),

predictions = H @ output_weights, where predictions represent the final predicted output.

By incorporating Ridge Regression within the ELM (RRELM) framework, this methodology aimed to strike a balance between effective feature learning and regularization, enhancing the model’s ability to generalize and produce accurate predictions.

Assessment matrices and implementation

The effectiveness of the lightweight PDSCNN-RRELM model was evaluated using a confusion matrix (CM), which is a valuable tool for assessing classification performance. The following equations were employed to derive key performance metrics, including accuracy, precision, recall, F1 score, and area under the curve (AUC), from the CM^45,46.

where true positives, true negatives, false positives, and false negatives are represented by \({T}_{P}\), \({T}_{N}\), \({F}_{P}\) and \({F}_{N}\), respectively.

The implementation of all the deep learning algorithms and Explainable Artificial Intelligence (XAI) techniques was accomplished using Keras, a powerful framework, in conjunction with TensorFlow as the backend. The entire system was orchestrated within PyCharm Community Edition (2021.2.3), a reliable software environment. Model training and testing were conducted on a high-performance computing platform comprising an 11th generation Intel(R) Core (TM) i9–11,900 CPU operating at 2.50 GHz, an impressive 128 GB RAM configuration, and an NVIDIA GeForce RTX 3090 graphics processing unit with a substantial 24 GB memory capacity. This robust hardware infrastructure, complemented by the 64-bit Windows 10 Pro operating system, facilitated efficient and expedient execution of the training and testing processes, enabling the exploration of complex deep learning architectures and XAI methodologies.