A novel hybrid model for emotion detection in text through sequential and transformer-based approaches: LSTM enhanced RoBERTa (LER)

This study aims to propose a hybrid model, LSTM-Enhanced RoBERTa (LER), for emotion detection in textual data, which can facilitate us in various fields of study, including mental health monitoring, customer feedback analysis, social media sentiment analysis, personalized marketing, educational technology for student engagement assessment, and human-computer interaction for adaptive systems. The dataset used in this study has been taken from an online repository, which is discussed in the subsequent section. The overall methodology of this study is presented in Fig. 1, followed by preprocessing, model training, and testing for model performance evaluation. Each phase is further discussed in the subsequent section for better understanding.

Dataset description and preprocessing

The dataset used in this study has been taken from the Kaggle repository online, available at https://www.kaggle.com/datasets/faisalsanto007/isear-dataset. The total number of records in this dataset is 7473, and it has seven different classes: joy, fear, anger, sadness, disgust, shame, and guilt. The statistics of each class are shown in Fig. 2.

Although Fig. 2 illustrates the class distribution of the ISEAR dataset, the potential effects of this imbalance on model performance merit explicit discussion. In our case, certain emotions (e.g., joy and fear) appear more frequently than others (disgust and shame), which could bias the model toward majority classes. To mitigate this, we employed stratified splitting when dividing the dataset, ensuring that the 70/30 train–test ratio preserved the proportional representation of each emotion class. From the training set, 10% was further allocated as a validation set, again following a stratified strategy, resulting in an effective train/validation/test split of 63%/7%/30%. In addition, we incorporated class-weight adjustments during training and monitored performance using macro-averaged precision, recall, and F1-score, which provide a balanced view across both frequent and infrequent classes. This strategy ensured that the proposed LER model did not overfit to dominant emotions and maintained robust performance across all categories, as reflected in the reported results.

To quantitatively justify class-weighting over alternatives, we conducted ablations on a validation set using macro-F1 as the key metric. Weights were computed inversely to class frequencies (\(\:{w}_{c}=\frac{N}{k\cdot\:{n}_{c}}\)), yielding + 6.5% minority F1 gains (e.g., disgust: +7.2%) and overall macro-F1 of 86.9%. SMOTE on TF-IDF features improved minorities by + 4.1% but degraded majority precision by 3.5% due to embedding noise (net macro-F1: 85.7%). Focal loss (\(\:FL=-\alpha\:(1-{p}_{t}{)}^{\gamma\:}\text{l}\text{o}\text{g}({p}_{t})\), \(\:\alpha\:=0.25\), \(\:\gamma\:=2\)) achieved 86.5% but required extra tuning and showed early convergence issues. These results, averaged over 10-fold CV, confirm class-weighting’s efficacy for mild imbalance (IR=10.8:1) in our hybrid architecture¹³. See Table 1 for summary.

Various steps are used for data preprocessing, which are:

a. Data cleaning (Handling missing values)

This step is done to ensure data consistency, and to check that any rows with missing values will be removed:

Let \(\:X\epsilon\:{\mathbb{R}}^{m*n}\) represent the dataset, where \(\:m\) is the sample number and \(\:n\) is the number of features. To remove rows with missing values, we define a filter \(\:f:\:{\mathbb{R}}^{n}\to\:\{0,\:1\}\) such that:

The cleaned dataset \(\:{X}_{cleaned}\) is then:

b. Data augmentation with synonym replacement

To enhance the model generalization and as a variation, we used synonym replacement using WordNet. To augment each text sample \(\:{x}_{i}\) with synonyms using WordNet, we define the following transformations:

1. 1.

   Text to Word Vector: We take \(\:{x}_{i}=\{{w}_{i,1},\:\:{w}_{i,2},\:\dots\:,\:{w}_{i,k}\}\) be the tokenized word sequence of text \(\:{x}_{i}\) where \(\:k={x}_{i}\).

2. 2.

   Synonym Selection: For each word \(\:{w}_{i,1}\in\:\:{x}_{i,}\) that has synonyms \(\:S\left({w}_{i,j}\right)=\:\){\(\:{s}_{1},\:{s}_{2}\), …, \(\:{s}_{n}\)} in WordNet, we randomly select a subset of words \(\:{W}^{{\prime\:}}\subseteq\:\left\{{w}_{i,1},\:{w}_{i,2},\:\dots\:,\:{w}_{i,k}\right\}\) and replace each \(\:{w}_{i,1}\:\in\:W{\prime\:}\) with a synonym \(\:{s}_{i,j}\:\in\:S\left({w}_{i,j}\right)\), yielding an augmented text \(\:{x}_{i}^{{\prime\:}}\):

1. 3.

   Augmented Dataset: We apply the synonym replacement function to each sample in \(\:{X}_{cleaned}\) and combine with the original data to obtain \(\:{X}_{augmented}\):

c. Basic text preprocessing

The basic transformations to standardize the text format have been applied.

1. 1.

   Lowercasing: For each text \(\:{x}_{i}\), we define a lowercase operation \(\:lower\):

\(\:{\mathbb{R}}^{k}\:\to\:\:{\mathbb{R}}^{k}\) that converts each word to lowercase:

1. 2.

   Special Character Removal: Using a regular expression function \(\:re.sub\) to remove special characters \(\:C\), we clean each word in \(\:{x}_{i}\):

Where \(\:C\) represents the set of special characters.

d. Target variable encoding

The target emotion labels \(\:y=\{{y}_{i},\:{y}_{2},\dots\:,\:{y}_{m}\}\) are categorical and need to be encoded into the numerical format for model training.

1. 1.

   Label Encoding: Define a label encoder \(\:L:\left\{{l}_{i},\:{l}_{2},\dots\:,\:{l}_{c}\right\}\:\to\:\left\{0,\:1,\:\dots\:,\:c-1\right\}\) that maps each unique emotion label \(\:l\) to a unique integer.

The encoded labels \(\:\stackrel{\sim}{y}=\{{\stackrel{\sim}{y}}_{1},\:{\stackrel{\sim}{y}}_{2},\:\dots\:,\:{\stackrel{\sim}{y}}_{m}\}\) are given by:

e. Tokenization

For input compatibility with the RoBERTa model, each text sample \(\:{x}_{i}\) is tokenized, padded, and truncated as needed.

1. 1.

   Tokenization: We take \(\:T\) as the RoBERTa tokenizer function. For each sample \(\:{x}_{i}\), apply \(\:T\) to produce a token sequence \(\:{t}_{i}=\{{t}_{i,1},{t}_{i,2},\:\dots\:,\:{t}_{i,p}\}\), where \(\:p=\left|{t}_{i}\right|\):

1. 2.

   Padding and Truncation: Define a fixed sequence length \(\:L\). Each sequence \(\:{t}_{i}\) is either truncated to or padded up to length \(\:L\) to ensure uniformity across samples:

Where 0 represents a padding token and \(\:\left|{t}_{i}\right|\) is the original length of the tokenized sequence.

While synonym replacement via WordNet carries the risk of altering emotional nuance, it was used in a controlled and conservative manner. Candidate synonyms were filtered by part-of-speech and polarity alignment to reduce semantic drift, and only a limited proportion of tokens were replaced to preserve the original emotional context. This ensured that augmentation enriched data variability without introducing significant noise.

Model training and performance evaluation

To prepare the ISEAR dataset for training and testing in our emotion detection model, the data is divided into two sets, which ensures robust evaluation by keeping a portion of the data for testing. A 70 − 30 ratio for data splitting was used, where 70% of the data was allocated for training and the remaining 30% for testing. The performance of the proposed model was compared with state-of-the-art models based on standard assessment criteria presented in Table 1. Where TP represents the true-positive classification count, FN represents the false-negative classification count, TN represents the true-negative classification count, and FP represents the false-positive classification count.

In addition to the train/validation/test split, a 10-fold stratified cross-validation was carried out to further assess robustness. Averaged results across folds showed consistent performance trends with low variance, demonstrating that the proposed LER model generalizes well and is not overly sensitive to data partitioning.

Proposed model

This study focuses on emotion detection, which is an interesting research area and applicable in various study domains, including mental health assessment, customer service, and feedback analysis, sentiment analysis in social media, human-computer interaction, automated content moderation, marketing, consumer behavior analysis, and educational technology for personalized learning. This study is part of a customer service support system in psychology.

To this end, we proposed a hybrid model LER that integrates the powerful contextual understanding of the transformer with the sequential processing strengths of LSTM. RoBERTa captures deep contextual nuances through its attention mechanism, which makes it adept at understanding the complexities of language. The addition of LSTM allows the model to handle long-term dependencies effectively, especially useful for tasks that benefit from sequence retention, such as summarization and emotion detection. This hybrid approach enhances the performance of the model in the task of better interpreting the structure and sentiment of the text, which is depicted with promising results and an accuracy rate on an emotion-detection dataset. The hybrid model is a well-balanced approach that harnesses the best of both architectures to improve results in various NLP applications. Overall, the methodology of the proposed LER hybrid model is as follows:

RoBERTa is first applied for contextualized embeddings of an input text \(\:X=[{x}_{1},\:{x}_{2},\:\dots\:,\:{x}_{n}]\), where \(\:{x}_{i}\) is every token in the sequence, and \(\:n\) is the number of tokens in the sequence. The multi-layer structure of RoBERTa enables processing the tokenized input \(\:X\) to acquire more sophisticated contextual relationships between tokens. At the initial stage, every token \(\:{x}_{i}\) is translated into an embedding vector signifying its semantic meaning. The process is to feed each token of the sequence through RoBERTa layers to calculate its hidden state, leading to a rich contextual representation used to feed the emotion classifier. The token embeddings are done through the following function, where d is the hidden size of RoBERTa:

The next phase is of a self-attention mechanism, where RoBERTa computes attention scores for all token embeddings in a layer. For each token \(\:{x}_{i}\), the attention score is calculated by comparing its query vector \(\:{q}_{i}\) with the key vectors \(\:{k}_{i}\) of all other tokens in the sequence. This can be represented as:

Where:

\(\:{W}_{Q}\) and \(\:{W}_{K}\) are learned weight matrices.

In the contextual embeddings, the attention scores \(\:{\alpha\:}_{ij}\)are used to compute a weighted sum of the value vector \(\:{v}_{j}\), producing a contextualized embedding \(\:{h}_{i}\) for each token:

Where, \(\:{v}_{j}=\:{W}_{{V}^{{e}_{j}}}\)

The output embeddings are the contextual representations of the input tokens:

The contextual embeddings \(\:H\) generated by RoBERTa are fed into a bidirectional LSTM network to capture sequential dependencies in the text. In this layer, the LSTM processes each token embedding \(\:{h}^{th}\) at each time step \(\:t\) and computes a hidden state \(\:{h}_{t}^{LSTM}\)​ along with a cell state \(\:{c}_{t}^{LSTM}\)​. The LSTM operates in both forward and backward directions, allowing it to consider both past and future context in the sequence through its update rules for these states. This step enhances the ability of the model to understand the sequential flow of emotions within the text. In the forward LSTM layer, several states and gates help maintain memory for sequential dependencies and manage the information flow.

• The updated Cell state combines memory, adjusted by the forget gate, with new memory, adjusted by the input gate.

This structured gating process enables the LSTM to selectively remember and forget information, capturing sequential patterns effectively.

In the backward LSTM, embeddings are processed in reverse, generating hidden states from the end of the sequence to the start, denoted as ​\(\:{\overleftarrow{h}}_{t}^{LSTM}\). The final hidden state for each token is then a combination of both directions, created by concatenating the forward ​(\(\:{\overrightarrow{h}}_{t}^{LSTM}\)) and backward (​\(\:{\overleftarrow{h}}_{t}^{LSTM}\)) hidden states, capturing context from both past and future tokens, where \(\:2\text{h}\) is the combined hidden size of the bidirectional LSTM:

The output of the bidirectional LSTM for the entire sequence is:

To summarize the LSTM output for the entire sequence, we can apply a pooling operation, here for the final hidden state pooling, as follows: \(\:z\:\in\:\:{\mathbb{R}}^{2h}\) is the final hidden state from the last time step \(\:n\):

The pooled vector \(\:z\) is passed to a fully connected (dense) layer to output the logits for emotion classification. Let \(\:W\) be the weight matrix of the dense layer and \(\:b\) the bias. The logits \(\:o\) are computed following where \(\:o\in\:{\mathbb{R}}^{C}\) is the vector of logits for \(\:C\) emotion classes:

Finally, the logits are passed through a Softmax function to obtain the predicted probabilities for each emotion class, where \(\:{\widehat{y}}_{i}\) is the predicted probability for class \(\:i\).

The model is trained using the cross-entropy loss, which measures the difference between the true label \(\:y\) and the predicted probabilities \(\:\widehat{y}\)​:

Where \(\:{y}_{i}\) is the true label (one-hot encoded) for the \(\:i-th\) emotion class.

The overall architecture flow is further presented in Algorithm 1, illustrated in Fig. 3.

Table 2 presents the computational complexity of the proposed LSTM Enhanced RoBERTa (LER) model. The RoBERTa encoder, responsible for capturing deep contextual representations, introduces a complexity of O(n×m2×d), primarily due to the self-attention mechanism’s quadratic dependence on sequence length. The LSTM layer adds sequential learning capability with a complexity of O(n×m×h2), where the hidden size h is typically smaller than RoBERTa’s hidden dimension d. Preprocessing operations such as tokenization and label encoding are linear concerning the dataset size. Overall, the total computational cost of the LER model is O(n×m2×d + n×m×h2), which is efficiently managed through sequence truncation, moderate hidden dimensions, mini-batch training, and GPU acceleration.

Beyond theoretical complexity, we also measured actual runtimes. On a single NVIDIA GPU, LER required approximately 45 s per epoch during training and 12 ms per instance for inference. In contrast, transformer-only RoBERTa required ~ 38 s per epoch, while sequential models such as Bi-GRU averaged ~ 29 s per epoch. Although LER is moderately more computationally demanding, its superior accuracy and robustness justify the trade-off, and the runtime remains practical for real-world deployment. These timings were obtained on a single NVIDIA RTX 4090 GPU (24 GB VRAM) with an Intel Core i9-13900 K CPU and 64 GB system RAM, using PyTorch 2.1.0 and CUDA 12.1—a standard mid-range setup for NLP research. For larger datasets (e.g., 10x ISEAR scale), LER scales efficiently via mini-batching and truncation (O(n m² d) dominant cost), achieving < 2 min/epoch with gradient accumulation; further enhancements like distributed training could handle million-sample corpora with minimal accuracy loss (< 1%), supporting deployment in high-volume applications such as social media monitoring.

In our proposed LSTM Enhanced RoBERTa (LER) model, we performed systematic experimentation to determine optimal hyperparameter values that balance performance and computational efficiency. The hyperparameters were selected based on grid search and empirical tuning using a validation set (15% split from the training data). The final configuration is as shown in Table 3.

RoBERTa + BiLSTM integration mechanism

This study’s combination of RoBERTa and BiLSTM makes use of both the sequential learning potential of recurrent neural networks and the contextual power of transformer-based embeddings. The deep contextual feature extractor is called RoBERTa (Robustly Optimized BERT Pretraining Approach). It generates high-dimensional token embeddings that capture bidirectional contextual information at the word and sentence levels after processing each input sentence through several self-attention layers.

A Bidirectional Long Short-Term Memory (BiLSTM) network receives the output from RoBERTa’s last hidden layer, which is usually the contextualized representation of each token. More efficiently than transformer outputs alone, the BiLSTM can model long-range dependencies and the temporal order of words because it receives these embeddings sequentially and processes them both forward and backward.

The BiLSTM improves RoBERTa’s representations by concatenating the forward and backward hidden states, thereby capturing nuanced emotional cues that cut across sentence structures. To predict the corresponding emotion label, the combined representation is then fed into a dense layer and then a Softmax classifier.

When compared to either model alone, this hybrid integration improves performance for emotion recognition tasks by enabling RoBERTa to handle deep semantic understanding and BiLSTM to increase temporal sequence sensitivity.

Model initialization

The RoBERTa model provides a 768-dimensional contextual embedding vector for each token in the input sequence. These high-dimensional vectors capture deep semantic and syntactic information from the text. To further enhance the sequential understanding and capture temporal dependencies relevant to emotion, these embeddings are then passed through a Bidirectional Long Short-Term Memory (BiLSTM) layer, which processes the sequence in both forward and backward directions, enriching the representation for effective emotion classification.

In the proposed LER model, each input sequence of length \(\:m\) is tokenized and passed through the pre-trained RoBERTa encoder, producing contextual embeddings \(\:E\in\:{\mathbb{R}}^{m\times\:d}\), where \(\:d\) is the hidden dimension (768 for Roberta-base). These embeddings are then sequentially fed into a Bidirectional LSTM (BiLSTM) layer with a hidden size \(\:h\), generating forward and backward hidden states \(\:{\overrightarrow{h}}_{t},\:{\overleftarrow{h}}_{t}\in\:{\mathbb{R}}^{h}\) at each time step \(\:t\). The combined hidden representation \(\:{H}_{t}=[{\overrightarrow{h}}_{t},\:{\overleftarrow{h}}_{t}]\) captures both contextual and temporal dependencies, which are then aggregated (e.g., via mean pooling) and passed to the classification head. Formally, the sequence processing can be expressed as:

where \(\:W\) and \(\:b\) are learnable weights of the classification layer, and \(\:Pool\left(\right)\) represents a suitable aggregation function (mean or max). This integration ensures that both deep contextual embeddings and sequential dependencies are leveraged for accurate emotion detection.

• • RoBERTa Component: We initialize the RoBERTa encoder with pre-trained weights from the RoBERTa-base model provided by Hugging Face Transformers. This ensures that the model benefits from rich semantic understanding learned from large corpora (e.g., BookCorpus and English Wikipedia).

• • LSTM Layer: We use Xavier (Glorot) uniform initialization for LSTM weights to maintain a balanced variance in the forward and backward pass. Biases are initialized to zero.

• • Classification Head: We initialize the weights of the linear layer using a normal distribution with mean 0 and standard deviation 0.02, as per standard practice in transformer-based models.

A systematic grid search was performed to identify optimal hyperparameters. The search space included learning rates {1e − 5,2e − 5,3e − 5}, batch sizes {16,32}, dropout values {0.1,0.3,0.5}, and hidden sizes {128,256,512}. The final configuration was chosen based on validation macro-F1 and stability across runs, ensuring a fair comparison with baseline models.

While prior studies have explored hybrid architectures that combine transformer embeddings with recurrent networks, the novelty of the proposed LSTM-Enhanced RoBERTa (LER) model lies in its specific integration strategy, optimization, and robustness analysis, which are tailored for emotion detection. Unlike conventional hybrids that stack RNNs over transformers without systematic optimization, LER introduces a Bidirectional LSTM (BiLSTM) layer with tuned hidden size to refine RoBERTa’s contextual embeddings by explicitly modeling sequential emotional dependencies. Mathematically, the RoBERTa encoder maps an input sequence. \(\:X=\{{x}_{1},\:{x}_{2},\:\dots\:,\:{x}_{m}\}\:\)into contextual embeddings \(\:H=\{{h}_{1},\:{h}_{2},\:\dots\:,\:{h}_{m}\}\), where each \(\:{h}_{1}\in\:{\mathbb{R}}^{d}\). These embeddings are then passed into the BiLSTM, which computes forward and backward hidden states as:

And concatenates them to form the enhanced representation:

where \(\:h\) is the LSTM hidden size (256 in our configuration). This refinement enables the model to retain temporal dependencies that RoBERTa alone may not capture, particularly in phrases that convey emotion. To further balance computational efficiency and accuracy, the model incorporates dropout regularization (0.3), warmup steps (500), and sequence truncation (128 tokens), yielding a combined complexity of:

Which remains tractable under GPU acceleration. Beyond raw performance, LER uniquely integrates uncertainty analysis—both parametric and non-parametric—to assess robustness under noisy or domain-shifted data, an aspect underexplored in prior RoBERTa + LSTM works. Collectively, these design choices differentiate LER from earlier hybrids, as validated by consistent improvements across all metrics (accuracy: 88%, precision: 86.03%, recall: 85.21%, F1: 86.9), establishing its superior suitability for emotion detection.

Benchmark ML and DL models

For fair comparison, all baseline models were re-implemented using the same preprocessing pipeline and stratified data splits. Traditional ML models (SVM, LR, RF, NB) were trained on TF-IDF features with hyperparameters tuned via grid search (e.g., SVM with linear kernel, \(\:\text{C}\in\:\{\text{0.1,1},10\}\); Logistic Regression with \(\:\text{C}\in\:\left\{\text{0.1,1}\right\}\); RF with 200 trees). Sequential DL baselines (LSTM, BiLSTM, GRU, BiGRU) were implemented with embedding size 300, hidden size 256, dropout 0.3, and Adam optimizer with learning rate 2e − 3. Transformer baselines (BERT, RoBERTa) were fine-tuned with maximum sequence length 128, batch size 32, dropout 0.1, and learning rate 2e − 5. Each baseline was trained for 10 epochs with early stopping based on validation loss to ensure robustness and prevent overfitting. Table 4 presents the benchmark ML and DL models used for the comparison with the LER model.