A model for epileptic EEG detection and recognition based on Multi-Attention mechanism and Spatiotemporal

The structure of MASF model

The model proposed in this paper, the Epileptic Electroencephalogram (EEG) detection and recognition model based on multi-head attention mechanisms and spatio-temporal feature fusion, namely the MASF model, aims to significantly enhance the accuracy of epilepsy seizure detection by integrating the hybrid attention mechanism module, Transformer Encoder Layer, dot-product attention mechanism module, and fully connected module. The MASF model, as shown in Fig. 1, processes EEG data through the hybrid attention mechanism module, learning the spatial local features of each EEG channel and their importance, thereby highlighting specific spatial locations or regions crucial for epilepsy detection. Meanwhile, EEG data also passes through the Transformer Encoder Layer, where the self-attention mechanism captures dependencies over long time ranges. Subsequently, the features from these two modules are concatenated and further processed by the dot-product attention mechanism module, dynamically adjusting the weights of the spatio-temporal features to focus on the most diagnostically valuable information. The hybrid attention mechanism can help the network better select important features, while the dot-product attention mechanism can effectively capture long-range dependencies within the sequence. When these two mechanisms are used together, the model can not only focus on important channel information but also take into account the complex relationships within the sequence. The activation gradient plays a crucial role in the backward propagation process, guiding the update direction and magnitude of the attention mechanism weights.Finally, the weighted spatio-temporal features are fed into two fully connected layers, outputting the predicted result of the epilepsy seizure. In this way, the MASF model not only effectively captures the complex spatiotemporal features in EEG signals but also optimizes the weight distribution of these features, thereby improving the accuracy and reliability of epilepsy seizure ictal detection.The model’s hyperparameters are shown in Table 1.

Mixed attention mechanism

The attention mechanism module, as an important technology for data processing, plays a key role in the field of machine learning and is also a core component of human cognition and decision-making processes. Due to the limited information processing capacity of humans, we naturally prioritize the most important parts when faced with complex information and temporarily ignore other less significant details. This strategy helps us make efficient use of resources, particularly evident in visual information processing. By assigning different weights to different pieces of information, the attention mechanism can intelligently select and focus on the key content in the input data. In machine learning, attention mechanisms are widely used to improve the performance of deep neural networks, allowing them to more accurately capture and utilize the key features of data, thus greatly improving the accuracy and efficiency of task execution. However, in fields like EEG signal processing, when applying traditional channel attention mechanisms (such as Squeeze-and-Excitation Attention and Efficient Channel Attention), these methods often only consider the relationships between channels and ignore the details in the spatial dimension. This leads to the possibility that when the attention weight of a certain channel is low, even if the channel contains important information, it may be overlooked, thus affecting the accuracy of the final category prediction. To address these issues, this paper proposes an innovative hybrid attention mechanism that combines the advantages of local spatial information and global channel information. In global channel attention, as shown in Fig. 2, the paper uses the SE attention mechanism, which mainly operates in two steps: first, the ‘squeeze’ operation, typically using global average pooling to obtain global information for each channel; then, the ‘excitation’ operation, which generates a weight coefficient for each channel through two fully connected layers (the first fully connected layer reduces the dimension, and the second restores the original dimension) and corresponding activation functions (such as ReLU and Sigmoid). Finally, these weight coefficients are multiplied by the original feature map to recalibrate the input features. Specifically, as follows.

First, as shown in Eq. (1), during the squeezing stage, the average value of each channel is calculated through global average pooling, compressing the spatial information of each channel in the input feature map U into a scalar value, resulting in a C-dimensional vector z, where C represents the number of channels in the feature map. The calculation method is:

Among them, z_c represents the global average pooling result of the C channel.

Subsequently, as shown in Eq. (2), during the incentive stage, an adaptive weight generation mechanism is used to calculate a corresponding weight value for each channel, thereby selectively enhancing or suppressing the channels. This process involves two fully connected layers (FC) and nonlinear activation functions (such as ReLU and Sigmoid). The first fully connected layer uses the ReLU activation function to compress the dimensions by half, and then another fully connected layer restores it to the original dimension, using the Sigmoid activation function to generate the weights for each channel.

In Eq. (2), δ represents the ReLU activation function, σ represents the Sigmoid activation function, and W₁ and W₂ are the weighted matrices of two fully connected layers.

In the final feature recalibration stage, as shown in Eq. (3), the SE module multiplies each channel in the original feature map by the corresponding weight to achieve adaptive channel weighting. The output feature map is computed as follows:

In the local attention mechanism, there are mainly two steps. Firstly, after testing and verification, a one-dimensional 7 × 1 convolution kernel Conv1D layer is used to extract broader local information. Then, a 1 × 1 convolution is used to adjust the number of channels and generate local attention weights. Similarly, these local attention weights are applied back to the original feature map through point multiplication. Finally, the features processed by global attention and local attention are added together to fuse the information of these two parts. This process integrates global and local information, which helps to enhance the model’s ability to learn key features and thereby improve the performance of the final task.

Transformer encoder

The Transformer network was originally developed by Google Inc. in 2017, and it uses an attention mechanism to extract features from input data in parallel¹⁹. To ensure the temporal correlation of data in each time series, the Transformer uses position encoding to record the positional information of the input data. As a result, the Transformer structure can still maintain the temporal correlation of data; the parallel input significantly shortens the network training time. Figure 3 shows all the core modules of the Transformer, and Fig. 3(c) illustrates the basic structural unit of the Transformer network. The Transformer model consists of two main parts: the encoder and the decoder. The encoder is primarily used to process the input sequence and generate a series of hidden state representations, while the decoder is generally used to generate the output sequence based on these hidden states, which is very useful in tasks like language translation. However, in tasks related to EEG analysis, such as epilepsy detection, sleep stage classification, or emotion recognition, the main goal is to extract features from EEG signals and perform classification or regression prediction, rather than generating new sequence data. Secondly, EEG data is essentially time-series data, and its purpose is to infer certain physiological or psychological states by analyzing brain activity patterns, which does not require tasks like translating from one language to another as in text generation. Lastly, the task that this study aims to accomplish is a classification task, so only an encoder is needed to extract features.

The encoding module includes the Multi-Head Self-Attention Mechanism layer and the Feed-Forward Neural Network layer. As shown in formula (4) and Fig. 3(b), the function mapping of the multi-head self-attention layer is a query and a set of output key-value pairs. The model is able to compute the query vector (Q), value vector (V), and key vector (K) by multiplying the input vector with three different weight matrices.

The multi-head attention mechanism allows a model to have multiple sets of weight matrices, dividing the input into separate parts and then projecting them into queries, keys, and values. Scaled Dot-Product Attention, as shown in Fig. 3(a), processes the projected inputs in parallel for each version and produces outputs with the same dimensions. The multi-head attention mechanism, as shown in Eqs. (5–6), uses different weight matrices to explore patterns from different domains, with Fig. 3b illustrating the multi-head attention structure.

The parameter matrix of the projection is as follows, \(\:{{W}}_{{i}}^{\text{Q}}\in{{R}}^{{{d}}_{{model}}\times {{d}}_{{k}}}\), \(\:{{W}}_{{i}}^{{K}}\in{{R}}^{{{d}}_{{model}}\times {{d}}_{{k}}}\), \(\:{{W}}_{{i}}^{{V}}\in{{R}}^{{{d}}_{{model}}\times {{d}}_{{v}}}\)and \(\:{{W}}_{{i}}^{{O}}\in{{R}}^{{{d}}_{{model}}\times {{h}{d}}_{{v}}}\).

The output of the self-attention mechanism layer is sent to the feed-forward layer in the Transformer model encoder. The feed-forward layer can run on different values in parallel. According to Eq. (7), the feed-forward layer is created using two linear transformations and a ReLU activation function.

Attention mechanism

When handling epilepsy detection tasks, the sudden nature of seizures and the uncertainty of marker locations may introduce redundant information during feature extraction. This means that, throughout the time series, not all feature vectors contribute equally to epilepsy detection, making it difficult to identify which parts are more critical. To address this challenge, a dot-product attention mechanism was introduced in the later stage of the model. Through this mechanism, we can dynamically assign weights to features at different time steps, effectively enhancing information that is crucial for epilepsy detection while suppressing irrelevant and redundant information. Specifically, this method first feeds the features extracted from the input signals into a specially designed module that can assess the importance of each time step. Then, based on these evaluation results, the weights are adaptively adjusted to emphasize features that help improve detection accuracy, while weakening components that might introduce noise or cause misdirection. First, given three matrices: Query Q, Key K, and Value V, these are typically obtained through linear transformations from the input vectors. As shown in formula (8), the similarity between each query vector q_i and all key vectors k_j is calculated through their dot product.

To stabilize the gradient, as shown in Eq. (9), the dot product result is divided by \(\:\sqrt{{\text{d}}_{\text{k}}}\), where d_k is the dimension of the key vector.

As shown in formula (10), the softmax function is applied to the scaled scores to convert them into a probability distribution, where i represents the index of the query and j represents the index of the key.

Finally, as shown in Eq. (11), the probability distribution obtained in the previous step is used as weights to perform a weighted sum on the corresponding vectors in V, resulting in the final output vector, where v_j is the j-th row vector of matrix V.

In this way, the model effectively learns the correlations between different parts of the input sequence, which not only enhances the model’s sensitivity to epileptic seizure events but also strengthens its overall robustness and reliability. Through this approach, our model can more accurately focus on the time periods with the most diagnostic value, thus achieving more effective epilepsy detection.