Ten healthy male individuals (mean age 30.8 ± 3.68) participated in the experiments. Each patient had normal or corrected-to-normal eyesight. Of the 10 subjects, nine were right-handed. None of the subjects had any history of mental or neurological illness. The card game was known to all subjects. Informed consent was obtained from all subjects, and the experiments were performed in accordance with the latest Declaration of Helsinki. The framework proposed in this study is illustrated in Figure 1. The framework is divided into two blocks: a training block (blue dotted lines) and a testing block (green dotted lines). The training black was used to train the neural network models on the given data, whereas the testing block was used to classify the data into truth and lie classes based on the model trained in the training block. Information on the individual blocks is provided in the respective chapters of the article.

The subjects were seated comfortably in front of a second person (referred to as the victim). The subject and victims underwent three practice sessions, and a brief explanation of the experiment before the experiment was provided to ensure that they fully understood the guidelines.

A well-selected experimental paradigm was used in this study. The experimental paradigm was a card game known as bluff or cheat. The bluff game was chosen because the rules of the game are ideal for testing our hypotheses. Our objective was to distinguish between deceptive actions when the subject is speaking the truth and when they are intentionally deceiving the victim with a falsehood.

The game rules are straightforward. The subject received 20 randomly selected cards, with the consideration that a minimum of six of these cards had no corresponding matches. Therefore, the subject had to lie at least four or five times in order to get rid of those cards. The subject had to play out all of their cards without revealing any signs of bluffing. The subjects had 1 min to carefully organize all their cards prior to starting the experiment. The duration of each experiment was approximately 10 min, with each experiment having a maximum of 20 sessions, each lasting approximately 30 s. In each session, the first 15 s were allotted for card arrangement. The subject had to lie to the victim in the next 5 s (called “claim time”) by laying his cards face down on the table and declaring what kind of cards they are (for instance, “three sevens”). Depending on his claim, the subject could select any number of cards between two and four. However, this assertion may or may not be correct. The victim then had 10 s to react to the subject’s assertion (response time). If the victim believed that the subject is telling the truth, they could choose to pass, removing the pile from the table. However, if the victim suspected that the subject had lied in their claim, they had the option to flip the cards face up. If the subject lied, the pile was returned to the subject. However, if the subject was truthful, the pile was removed from the table, and the next session commenced. The game continued for 20 sessions. A prize of 10,000 Korean Won was to be awarded to the subject if they managed to play all their cards within 20 sessions; however, if they were to do so, they would not receive the prize money. There were 12 total subjects in this trial. Two respondents’ data were excluded from the analysis as they consistently spoke the truth at the beginning of trials and only lied towards the end, rendering their responses predictable. Eight out of ten subjects were able to play all of their cards. One administrator continuously monitored the experiment and documented the trials in which the subject deceived the victim.

A lab-built multichannel continuous-wave imaging system captured the brain signals (Bhutta and Hong, 2013). The optical probe of the fNIRS system was positioned on the forehead of the subject such that the FP1 and FP2 locations were in the middle of the probe, as shown in Figure 2. To connect the flexible probe and ensure excellent contact between the its emitters and detectors and the subject’s scalp, hair was brushed backward. Self-adhesive bandages were used to secure the probe to the subject’s head. The emitters and detectors were systematically positioned within a 4.3 × 13 cm² area according to a source-detector distance of approximately 3 cm. A sampling rate of approximately 3.8 Hz was used to collect the data. A Velcro band was used to hold the probe at the appropriate location throughout the experiment.

MATLAB (MathWorks, United States) was used to import and further analyze the signals from the fNIRS equipment offline. The data were stored in a host computer text file as digitized raw intensity values from the fNIRS system. The hemoglobin conversion block of the framework was used to convert the intensity values to concentration changes of HbO and HbR using the Modified Beer–Lambert law (Bhutta et al., 2015). The change in optical density (ΔOD) was calculated using these raw intensity values at each discrete time k as:

Where I_out is the intensity of detected light; I_in, intensity of incident light; d, differential path length factor; l, distance between the emitter and detector; and Δμ_a, absorption change of the tissue. The changes of HbO (Δc_HbO) and HbR (Δc_HbR) were measured using the modified Beer–Lambert Law as (Bhutta et al., 2015):

with λ₁ = 640 nm, λ₂ = 910 nm, d^λ₁ = 6.63, and d^λ₂ = 2.765, according to the values for the wavelength-dependent absorption coefficients α_HbO, α_HbR. fNIRS, while detecting the hemodynamic responses, picks up the physiological noise of respiration, pulse, and low-frequency drift fluctuations. A second-order low-pass filter with a cutoff frequency of 0.15 Hz was used to eliminate such noises (Hu et al., 2012; Bhutta et al., 2015). The HbO was considered for further analysis in this study because it is a more sensitive and reliable activity indicator than HbR (Hoshi, 2003, 2007).

Once the data were preprocessed, classification was performed on the Δc_HbO(k) signals. We conducted this classification to distinguish between lie and truth responses based on the features extracted from fNIRS signals. The features selected in this study were the signal mean (SM) and signal slope (SS) of the HbO signal during the 5-s claim period of the stimulus. We used this claim period because it is the actual time at which the subject attempted to deceive the victim by either telling the truth or lying. The average HbO signal for each subject was obtained by averaging all 12 channels of the corresponding subject. SM and SS values over a 5-s window can yield better results in binary classification (Khan et al., 2014; Bhutta et al., 2015).

In this study, we performed the classification using various classifiers categorized into linear and nonlinear categories. LDA and SVM are the main linear classifiers, whereas the ANN is a nonlinear classifier. Both the LDA and SVM algorithms classify different classes of data based on hyperplanes. In LDA, a separating hyperplane is generated to minimize the interclass variance and maximize the distance between the class means (Lotte et al., 2007). For the SVM classifier, a separating hyperplane is designed such that the distance between the hyperplane and the nearest training point(s) is maximized (Naseer et al., 2014).

Mainstream machine learning techniques can be categorized as linear or nonlinear classifiers. Linear classifiers classify a sample based on the value of the linear combination of its features. For example, assume that we have an input feature vector x. A linear classifier then constructs a function that directly assigns the input vector x to a specific class:

A linear SVM is a linear classifier that makes decisions according to a linear hyperplane capable of effectively segregating data. SVM finds an optimal hyperplane by maximizing the margin, which is the minimum distance between the hyperplane and any of the data samples. Such classifiers perform well when the problem is linearly separable. However, if the data are not linearly separable, they will have poor generalization ability. In this case, we could map the input vector onto a higher-dimensional space using the kernel function K and find the separating hyperplane in that particular dimension. Quadratic SVM and Cubic SVM are examples of kernelized versions of SVM that utilize second and third-degree polynomial kernels.

In the machine learning literature, several other algorithms handle nonlinear cases using a completely different computational approach; one of the simplest algorithms is the K-Nearest Neighbor (KNN). The main idea of this algorithm is that, for a new instance to be classified, the algorithm searches for the K-nearest points in the feature space and assigns it to the class that prevails among its neighbors. Similarly, the decision tree constructs a classification model with a tree-like structure. It partitions a feature space into smaller regions containing homogenous instances and simultaneously incrementally constructs an associated decision tree. The partitions of the feature space are usually based on criteria such as the Gini impurity, information gain, or distance measure.

In recent years, artificial neural networks have flourished in the machine learning and pattern recognition domains. They consist of many interconnected processing units, called neurons. The outputs of the hidden layer neurons are transmitted to the inputs of the next hidden layer within the network (Ullah et al., 2020). Thus, they communicate with each other by emitting signals over numerous weighted connections. During training, each neuron can update its weight, allowing the network to learn hidden patterns from the data. In this study, we designed three ANN architectures (M1, M2, and M3) to conduct experiments on our dataset. These structures were designed based on ideas from state-of-the-art convolutional neural network models, including AlexNet, ResNet, and GoogleNet. The numbers of input and output nodes and hidden layers of these neural networks are the same; however, the number of nodes in each hidden layer varies. The first two layers of M1 contain 10 neurons; the subsequent two hidden layers consist of eight and five neurons, respectively; and finally, the prediction layer contains two SoftMax classifiers. The M2 topology is similar to that of M1; however, we introduced two pairs of hidden layers with the same number of neurons in this structure. The first two layers had eight neurons, and the next pair had layers containing four neurons. We designed a third neural network architecture that differed from the aforementioned architecture. In this structure, we first increased the number of neuron dimensions from two to six and six to eight and then decreased it from eight to six and six to two neurons for the final class prediction. The architectures of the three ANNs are shown in Figure 3. Neural networks have weights that are initially randomly initialized, and later in the training process, these weights are optimized. The initial weights of our neural networks were determined using Kaiming uniform initialization (also known as HE initialization). This method is tailored for layers activated by the ReLU function and provides an advantage over random initialization. Specifically, HE initialization mitigates issues such as vanishing and exploding gradient problems, thereby enabling faster convergence during training. Aligning with the characteristics of ReLU, it also minimizes the occurrence of inactive neurons at the start of training. The empirical robustness of this method makes it a superior choice for deep network initialization compared to other simplistic methods. We selected four intermediate layers to achieve an optimal balance for our dataset. With only two features present in the input, it is essential to project them into a higher-dimensional space for feature extraction and subsequently condense the dimensions as we approach the classification layer. If we were to increase the number of hidden layers, the model would risk succumbing to the vanishing gradient problem. This is especially pertinent when processing only two features across excessive layers, as this is not advisable.

In a binary classification problem, the output class is usually labeled as positive or negative. The classification results can be represented in a structured form called a confusion matrix. However, the confusion matrix only provides true- and false-positive results. Therefore, to check the performance of the classification model at different thresholds, we calculated the ROC curves for all classifiers. This ROC curve plots the True Positive Rate (TPR) and False Positive Rate (FPR) at various thresholds, where TPR is a synonym for recall. These can be defined as follows:

Moreover, for further evaluation, it is crucial to compute the ROC points, which is a resource-intensive method. An efficient sorting-based algorithm called the AUC, provides this information for evaluation. It measures the entire area under the ROC curve from (0,0) to (1,1). AUC offers an aggregate measure of performance at all possible thresholds. Thus, we calculated these values and obtained promising results for both the ROC curves and AUC values for all classifiers. The obtained AUC values and ROC curves for all classifiers are shown in Figure 5. The SVM classifiers achieved better AUC values and ROC curves, obtaining 86%, 84%, and 83% AUC for linear, quadratic, and cubic SVM, respectively. In contrast, the KNN, simple decision tree, and complex decision tree achieved AUCs of 64%, 78%, and 73%, respectively. Linear SVM has better accuracy than other statistical machine-learning techniques. However, it is still ineffective for sensitive problems, such as lie detection. To achieve better performance, we proposed three different neural network structures that increased the accuracy of lie detection from 8% to 10%.

In the proposed method, we conducted experiments on our data using the three neural network models discussed in detail in the proposed methodology section. The models were trained for 50 epochs, and the data were divided into training, validation, and test sets of 60%, 20%, and 20%, respectively. The confusion matrices, ROC curves, and AUC for the three models are shown in Figure 6, and the overall accuracies are listed in Table 2. The proposed neural network models outperformed statistical machine learning approaches by a large margin, reaching 90% overall accuracy for the M3 neural network model, which is a 10% increase in accuracy. We trained our models five times and obtained the highest accuracies of 88%, 88%, and 87% for the fourth folds of M1, M2, and M3, respectively. The confusion matrices of the three models were almost identical, demonstrating the effectiveness of the models for lie detection.

The proposed neural network models were also evaluated for subject-wise performance, which is illustrated in Figure 7. In the entire dataset, we had a total of 10 subjects. For this experiment, we trained our models on nine subjects and tested the models on the remaining one subject. This experiment showed the accuracy of our models when applied to unseen data. Subjects 1 and 9 achieved the highest accuracy of 90% and 95% on each model, respectively; only subjects 2 and 7 were had accuracies less than 70%. The remaining subjects had accuracies greater than 80% for all three models. The average accuracies achieved for M1, M2, and M3 were 81%, 80%, and 82%, respectively, demonstrating that the models are very effective and robust for unseen data.

Figure 7 displays the results for the test set of each subject’s data. We randomly selected three samples from each subject to check the robustness of our models for different subject’s data. The third column represents the actual label of the test sample, and the other columns represent the results of its corresponding machine-learning algorithm. The proposed three neural network models achieved better performance of 80%, 80%, and 90% subject-wise accuracy for neural network1, neural network2, and neural network3, respectively. On the other hand, the machine learning-based methods, namely KNN, SDT, CDT, L-SVM, Q-SVM, and C-SVM achieved 60%, 72%, 60%, 70%, 77%, and 73% accuracies, respectively. The proposed models have low accuracy for only three samples’ data, including the first sample of subject 2 and the third sample of subjects 4 and 6. However, for this data, other machine learning algorithms also faced challenges in detection. Subsequent examination of data revealed that these particular samples significantly differed from the rest of the dataset and exhibited substantial noise; therefore, the outcomes for these three samples were unsatisfactory.