A deception detection model by using integrated LLM with emotion features

To reduce the time cost of manually assigning emotional features and simultaneously improve the efficiency and accuracy of deception detection, we design an automatic lie detection model called LieXBerta. As shown in Fig. 1, LieXBerta is mainly divided into two modules: an emotion classification module and a deception detection module. First, the emotion classification module uses a pretrained large language model, RoBERTa, to perform emotion classification on the text. Then, the emotional features are combined with the dataset’s original action and facial features. Finally, the deception detection module, based on the XGBoost, conducts deception detection by incorporating the new emotional features into the existing feature set.

Emotion classification module

In the emotion classification module, we decide to use the pre-trained large language model RoBERTa. RoBERTa, as a pre-trained language model, excels in emotion classification by capturing subtle emotional nuances through large-scale pre-training and fine-tuning, achieving high accuracy and robustness^25,37,38. Its effectiveness is further demonstrated in various domains: integrating CNN and sentiment lexicons enhances sentiment analysis in bullet screens³⁹, outperforming prompt engineering in hospitality review classification⁴⁰, achieving 92.69% accuracy in social media emotion detection⁴¹, and improving multilabel emotion classification with advanced attention mechanisms⁴². These studies affirm RoBERTa’s adaptability and superior performance across diverse text-based emotion classification tasks.

In this work, we treat each emotional category as an independent classification target. For each emotional label, we construct a separate classification sub-task to determine whether the emotion is present in a given interrogation sentence. This design allows the model to support multilabel outputs, ensuring that multiple emotions can be simultaneously recognized within the same text. To accomplish this, we build an emotion classification model consisting of three major components: a text preprocessing layer, a RoBERTa encoding layer, and a classification output layer.

Text preprocessing layer

After extracting emotional features, each dataset contains the text and the corresponding emotion labels. First, the original text is cleaned by replacing non-printable characters, escape characters, newline characters, and carriage return characters with a space character, and converting all text to lowercase. After cleaning the interrogation texts, we use RoBERTa’s tokenizer to split the cleaned text into tokens. RoBERTa employs a tokenization method based on Byte-Pair Encoding, which effectively handles out-of-vocabulary words and subword units. Next, as shown below, we add [CLS] at the beginning of the token sequence and [SEP] at the end to obtain the original input X,

$$\begin{aligned} X=[\text {CLS}]x_1x_2\cdots x_n[\text {SEP}], \end{aligned}$$

(1)

where n represents the sentence length; [CLS] denotes the special token indicating the start of the text sequence and [SEP] denotes the separator token between text sequences. This helps the model understand the start and end of the sequence.

Then, we convert the token sequence into a corresponding token ID sequence. RoBERTa’s tokenizer assigns a unique ID to each token based on its vocabulary. To ensure all input sequences have consistent lengths, we apply post-padding by appending zeros to the end of sequences that are shorter than the predefined maximum length. Through these steps, we transform the original interrogation texts into input representations $v$ acceptable by the RoBERTa model. The transformation is defined as,

$$\begin{aligned} v = \text {MapAndPad}(X), \end{aligned}$$

(2)

where $v \in \mathbb {R}^{N \times d}$, $N$ denotes the number of tokens in the sequence, and $d$ is the hidden layer dimension of RoBERTa.

RoBERTa encoding layer

After the preprocessing described in section “Text preprocessing layer”, the cleaned and tokenized inputs are fed into the RoBERTa model for training. The training process involves multiple epochs, where each epoch iterates over the entire dataset. To prevent overfitting during training, dropout regularization is applied within the RoBERTa layers. Specifically, certain neurons in the model are randomly deactivated during each forward pass, which encourages the model to learn more robust and generalizable features. In the RoBERTa encoding layer, the input representation passes through multiple layers of Transformers. By leveraging the self-attention mechanism, the model fully learns the semantic associations between each word in the sentence and ultimately obtains the contextual semantic representation of the sentence,

$$\begin{aligned} h = \text {RoBERTa}(v), \end{aligned}$$

(3)

where $h \in \mathbb {R}^{N \times d}$ represents the sequence of hidden states output by RoBERTa. The model uses the hidden layer representation corresponding to the [CLS] position, denoted as $h_0$, whose value is constituted by the first component of h, since [CLS] is the first element of the input sequence.

Classification output layer

After obtaining the hidden layer representation $h_0$ at the [CLS] position, a fully connected layer is used to predict the classification label corresponding to the input text. Since our task is a binary classification for each emotion label, the output layer applies a softmax activation to produce a probability distribution $P \in \mathbb {R}^2$, indicating whether the given emotion is present or absent in the text. The prediction is computed as,

$$\begin{aligned} P=\text {Softmax}{\left( h_0W^o+b^o\right) }, \end{aligned}$$

(4)

where $W^o\in R^{d\times 2}$ represents the weights of the fully connected layer; $b^o \in \mathbb {R}^2$ denotes the bias of the fully connected layer.

After obtaining the classification probability distribution P, we compute the cross-entropy loss with the true classification label y and update the model parameters through backpropagation,

$$\begin{aligned} \mathcal {L}_{\text {CE}} = -\sum _{i=0}^{1} y_i \log P_i, \end{aligned}$$

(5)

where $y = [y_0, y_1]$ is the one-hot encoded true label vector. Specifically, $y = [0, 1]$ indicates that the emotion is present in the input sample, while $y = [1, 0]$ indicates that the emotion is absent. Similarly, $P = [P_0, P_1]$ denotes the predicted probability distribution over the two classes, where $P_0$ is the predicted probability that the emotion is absent, and $P_1$ is the predicted probability that the emotion is present.

The model parameters $\theta = \{W^o, b^o, \theta _{\text {RoBERTa}}\}$ are optimized using gradient descent,

$$\begin{aligned} \theta \leftarrow \theta – \alpha \cdot \nabla _\theta \mathcal {L}_{\text {CE}}, \end{aligned}$$

(6)

where $\alpha$ is the learning rate. After each parameter update in Eq. (6), the updated parameters $\theta$ are propagated back to the RoBERTa encoder in Eq. (3), enabling continued refinement of the contextual representation h. This training process is repeated iteratively over multiple epochs. After each epoch, the model evaluates the average training loss,

$$\begin{aligned} \mathcal {L}_{\text {train}} = \frac{1}{M} \sum _{j=1}^{M} \mathcal {L}_{\text {CE}}^{(j)}, \end{aligned}$$

(7)

where M denotes the total number of training samples, and $\mathcal {L}_{\text {CE}}^{(j)}$ represents the cross-entropy loss of the j-th training sample. If this loss drops below a minimal threshold $\delta$, the training is terminated early to prevent overfitting.

To ensure the model retains the best-performing parameters, we monitor the validation loss at each epoch. If the validation loss decreases compared to the previous best, the current model state is preserved. Once the optimal parameters are determined through the training procedure described above, the model utilizes these parameters to predict emotion labels for test set. Specifically, the input texts are reprocessed using the same tokenization and encoding pipeline, and passed through the trained RoBERTa encoder and classification layer. The predicted emotion label $\hat{y}$ for each input sequence is computed as,

$$\begin{aligned} \hat{y} = \arg \max _i P_i, \end{aligned}$$

(8)

where $\hat{y}$ is obtained by selecting the index i with the highest predicted probability $P_i$ from P . Specifically, $\hat{y} = 1$ indicates that the emotion is predicted to be present, while $\hat{y} = 0$ indicates that the emotion is predicted to be absent.

This process is performed independently for each emotion category, resulting in individual binary predictions. Finally, we obtain the predicted results for each emotional category in each interrogation text, which are used in the subsequent deception detection module. The process of the Emotion Classification Module is presented in Algorithm 1.

Deception detection module

XGBoost has proven to be highly effective in fraud detection, excelling in tasks like credit card fraud⁴³ and customer transaction fraud⁴⁴ due to its superior accuracy and F1 score. It employs regression trees based on the CART (Classification and Regression Tree) structure, optimized through gradient boosting to minimize loss. Its robustness is further validated in scam profile detection on Instagram, where it outperforms other models with 90% accuracy⁴⁵, and in a large-scale fraud detection framework, achieving an impressive 99% accuracy⁴⁶.

We adopt a feature-level fusion strategy by applying horizontal concatenation of the emotional, facial, and action feature vectors. All features are horizontally concatenated according to the sample index, ensuring that each feature vector represents a consistent multimodal description of the same data instance. Because XGBoost builds its trees incrementally and selects the most discriminative features at each split automatically, it can adaptively learn which features are most relevant for deception detection^47,48. Therefore, we did not assign weights or priorities to features.

After concatenating the features, we form a unified input feature vector $\textbf{x} \in \mathbb {R}^d$, where d denotes the dimension of the input feature vector. For the input feature vector $x_i$ of the i-th sample, after training the t-th decision tree, its predicted value is updated as,

$$\begin{aligned} \hat{y}_i^{\left( t\right) } = \hat{y}_i^{\left( t-1\right) } + \eta \,f_t\left( x_i\right) , \end{aligned}$$

(9)

where $\eta$ is the learning rate, and $f_t\left( x_i\right)$ is the prediction of the t-th tree for the sample $x_i$. In the first round of training, since the model has not yet learned any information, the initial predicted value for all samples is set to 0.5,

$$\begin{aligned} \hat{y}_i^{\left( 0\right) } = 0.5, \end{aligned}$$

(10)

where $\hat{y}_i^{\left( 0\right) }$ denotes the predicted value for the i-th sample before the first iteration. The objective of the $t$-th boosting iteration is to minimize the following function,

$$\begin{aligned} L_t = \sum _{i=1}^{n} \ell \left( y_i, \hat{y}_i^{\left( t-1\right) } + f_t(x_i)\right) + \Omega (F_{t-1}) + \Omega (f_t), \end{aligned}$$

(11)

where n is the total number of samples and $\ell$ denotes the loss function. $y_i \in \{0, 1\}$ represents the ground truth deception label for the $i$-th sample, where 1 denotes a truthful instance and 0 denotes a deceptive one. $\Omega (F_{t-1})$ represents the regularization term of all trees up to iteration $t-1$, and $\Omega (f_t)$ is the complexity regularization term of the newly added tree at iteration $t$. The definition of $\Omega (f)$ is given as follows,

$$\begin{aligned} \Omega \left( f\right) =\gamma T+\frac{1}{2}\lambda \sum _{j=1}^{T}w_j^2, \end{aligned}$$

(12)

where T is the number of leaf nodes in the current tree, $\gamma$ is the regularization cost associated with adding a leaf node and $\lambda$ is the L2 regularization coefficient. In a regression tree, the prediction for a sample $x_i$ is determined by the score of the leaf node it falls into. If sample $x_i$ falls into leaf $j$, its prediction is determined as,

$$\begin{aligned} f_t(x_i) = w_j, \quad \text {with } j = q(x_i). \end{aligned}$$

(13)

where $q(x_i)$ is the structure function that maps sample $x_i$ to the index of the leaf it belongs to, and $w_j$ is the score assigned to leaf $j.$

To optimize efficiently, XGBoost uses a second-order Taylor expansion to approximate the loss function $\ell$. By ignoring constant terms, we obtain,

$$\begin{aligned} L_t[f_t] \approx \sum _{i=1}^{n} \left[ g_i f_t(x_i) + \frac{1}{2} z_i f_t(x_i)^2 \right] + \Omega (f_t), \end{aligned}$$

(14)

where $g_i=\partial _{{\hat{y}}_i}l\left( y_i,{\hat{y}}_i\right)$ is the first-order derivative and $z_i=\partial _{{\hat{y}}_i}^2l\left( y_i,{\hat{y}}_i\right)$ is the second-order derivative.

By integrating the regularization term in Eq. (12) and the leaf-wise prediction formulation in Eq. (13) into the second-order approximation in Eq. (14), the objective function can be reformulated as,

$$\begin{aligned} L_t[f_t] = \sum _{j=1}^{T} \left( G_j w_j + \frac{1}{2} (Z_j + \lambda ) w_j^2 \right) + \gamma T, \end{aligned}$$

(15)

where $G_j = \sum _{i \in I_j} g_i$ is the sum of first-order gradients of the samples in leaf node $j$. $Z_j = \sum _{i \in I_j} z_i$ is the sum of second-order gradients of the samples in leaf node $j.$

By taking the derivative of the objective function with respect to $w_j$ and setting it to zero, the optimal value for each leaf node is obtained as,

$$\begin{aligned} w_j = -\frac{G_j}{Z_j + \lambda }, \end{aligned}$$

(16)

where $w_j$ denotes the optimal prediction score for leaf node $j$. By substituting Eq. (16) into Eq. (15), we obtain,

$$\begin{aligned} \min _{f_t} L_t[f_t] = -\frac{1}{2} \sum _{j=1}^{T} \frac{G_j^2}{Z_j + \lambda } + \gamma T, \end{aligned}$$

(17)

where $\min _{f_t} L_t[f_t]$ denotes the minimum value of the objective function achievable by the optimal decision tree $f_t$ in the t-th boosting iteration. Then,

$$\begin{aligned} L_{\text {before}} = -\frac{1}{2} \cdot \frac{G_j^2}{Z_j + \lambda } + \gamma T, \end{aligned}$$

(18)

where $L_{\text {before}}$ denotes the minimized value of the objective function before splitting a node. After splitting the node into left child L and right child R, the minimized objective becomes,

$$\begin{aligned} L_{\text {after}} = -\frac{1}{2} \left( \frac{G_L^2}{Z_L + \lambda } + \frac{G_R^2}{Z_R + \lambda } \right) + \gamma (T + 1), \end{aligned}$$

(19)

where $G_L$ is the sum of first-order gradients of all samples in the left child node L, and $G_R$ is the sum of first-order gradients in the right child node R. Similarly, $Z_L$ and $Z_R$ denote the sums of second-order gradients in the left and right child nodes, respectively. Since the parent node’s gradient and Hessian sums are equal to the total sums of its left and right children, we have,

$$\begin{aligned} G_j= & G_L + G_R, \end{aligned}$$

(20)

$$\begin{aligned} Z_j= & Z_L + Z_R, \end{aligned}$$

(21)

where $G_j$ and $Z_j$ denote the first-order gradient sum and second-order gradient sum of the parent node, respectively. Then,

$$\begin{aligned} \text {Gain} = L_{\text {before}} – L_{\text {after}}, \end{aligned}$$

(22)

where Gain represents the improvement in the objective function by splitting the node. By substituting Eqs. (18), (19), (20), and (21) into Eq. (22), we can derive the expression for Gain,

$$\begin{aligned} \text {Gain} = \frac{1}{2} \left( \frac{G_L^2}{Z_L + \lambda } + \frac{G_R^2}{Z_R + \lambda } – \frac{(G_L + G_R)^2}{Z_L + Z_R + \lambda } \right) – \gamma , \end{aligned}$$

(23)

where Gain can be used to compute feature importance. During the node splitting process, we adopt a top-down greedy strategy as the split finding method. Specifically, for each node, the model processes each feature sequentially by sorting all values of that feature among the samples in the node and enumerating all possible split points. For each candidate split, the samples in the node are divided into left and right subsets. The corresponding Gain value is then calculated. Finally, the feature and split point that yield the highest Gain are selected to perform the node split. If the highest Gain is positive, the number of samples in the node exceeds the minimum child weight threshold, and the maximum depth constraint is not violated, then the node is split.

After completing the above steps, we can obtain the updated prediction \(\hat{y}_i^{
(24)

Finally, the predicted label is determined by thresholding the probability at 0.5, where values greater than or equal to 0.5 are classified as 1, meaning truthful, and those below 0.5 as 0, meaning deceptive. The process of the Deception Detection Module is presented in Algorithm 2.