Demonstrating brain-level interactions between working memory load and frustration while driving using functional near-infrared spectroscopy
-
1
University of Oldenburg, Department of Psychology, Germany
-
2
Deutsches Zentrum für Luft- und Raumfahrt (DLR), Institute of Transportation Systems, Germany
Introduction and Goal
Mental workload is a popular concept in ergonomics as it provides an intuitive explanation why exceedingly cognitive task demands result in a decrease in task performance and increase the risk of fatal incidents while driving. At the same time, affective states such as frustration, also play a role in traffic safety as they increase the tendency for speedy and aggressive driving and may even degrade cognitive processing capacities. To reduce accidents due to dangerous effects of degraded cognitive processing capacities and affective biases causing human errors, it is necessary to continuously assess multiple user states simultaneously to better understand potential interactions. In two previous studies, we measured brain activity with functional near-infrared spectroscopy (fNIRS) for separate brain based prediction of working memory load (WML) (Unni et al., 2017) and frustration levels (Ihme et al. submitted) while driving. Here, we report results from a study designed to predict simultaneously manipulated WML and frustration using data driven machine learning approaches from whole-head fNIRS brain activation measurements.
Methods and Data Analyses
We implemented a realistic driving task on city roads with concurrent traffic in the Virtual Reality-lab at DLR, Braunschweig (Fischer et al., 2014), in which we simultaneously varied memory workload demands and induced frustration through a combination of time pressure and blocking goal-directed behavior. A cover story instructed the participants to deliver a parcel within a certain time with the incentive of gaining 2€ per successful delivery in addition to a basic incentive. In the frustrating drives (FRUST), the participants were blocked by heavy traffic, while in the non-frustrating drives (NOFRUST), the participants could drive mostly unblocked. In parallel, the participants performed a digit-span n-back speed regulation task at one of two levels: 0-back or 2-back. For this task, speed limit signs were introduced every 15 seconds. Depending on the current n-back task, the participant was supposed to remember the previous ‘n’ speed sequences and adjust his/her speed accordingly (for details, see Unni et al., 2017). The participants repeatedly performed six drives of approximately 4 minutes duration for each of the four combinations of FRUST/NOFRUST with the two WML levels. We recorded whole-head fNIRS oxyhemoglobin (HbO) and deoxyhemoglobin (Hb) data using a 64-channel NIRScout Extended system while the participants performed the driving task. We then used all fNIRS channels in a multivariate logistic ridge regression with a 5-fold cross-validation for temporally resolved prediction of frustration and WML levels. The theoretical guessing level in the two-class classifications was 50%. Further, we used an encoding modeling approach to gain insights into the localization of neural correlates for frustration and WML. The magnitudes of encoding model weights reflect the contribution of the respective condition to the hemodynamic response measured by a particular channel. In this approach, we used univariate linear ridge regression to predict the fNIRS sensor time-series data of the hemodynamic response separately in single channels using binary coded conditions (FRUST and WML) as predictor variables and averaged the weights over participants.
Results
For the predictions of WML level dependent on frustration (i.e. separate classifiers trained for each frustration level), we achieved a mean accuracy of 74% (precision: 0.75, recall: 0.74) over 19 participants. The mean accuracy dropped to 67.7% (precision: 0.68, recall: 0.68) for WML predictions independent of the frustration level (i.e. all frustration levels combined for WML classification). However, a two-tailed paired t-test did not reveal a significant difference between the prediction accuracies of the dependent and independent models (p = 0.13).
For predictions of induced frustration levels dependent on WML levels (i.e. separate classifiers trained for each WML level), we achieved a mean accuracy of 71.9% (precision: 0.73, recall: 0.72). In addition, the frustration prediction differed only slightly across WML levels (0-back = 74%, 2-back = 69.8%, p = 0.40). Conversely, a two-tailed paired t-test showed that the prediction of frustration independent of WML levels (i.e. all WML levels combined for classification) was significantly lower with a mean classification accuracy close to chance level (accuracy: 50.9%, precision: 0.51, recall: 0.51 than the dependent model (p = 0.0003).
Figure 1 depicts the group-level brain maps specific to frustration and WML levels derived from the combined linear encoding model. A large overlap in the predictive maxima of the model weights (marked by white shapes) can be observed for both, frustration (marked by white shapes) and WML in the bilateral frontal, left superior parietal and temporo-occipital areas. However, the weights appear to be higher for WML compared to frustration, indicating a stronger influence of our WML level manipulation on the brain activations.
Figure 1. Group-averaged linear encoding model weights for HbO and Hb showing brain areas predictive to FRUST and WML. White shapes mark the prediction maxima for FRUST in all maps.
Discussion and Conclusion
In this study, we used a data driven machine learning approach to investigate brain level indicators for WML and frustration as well as potential interactions. We were able to predict WML dependent on and independent of frustration levels. Conversely, the prediction accuracy for frustration was almost as high as for WML in the dependent model but dropped to chance-level in the independent model. Combined, these results suggest an interaction between WML and frustration at the brain level with a stronger effect of WML level on frustration related networks than reverse.
Previously, we demonstrated an overlap in brain activation patterns in the bilateral frontal and temporo-occipital areas when we predicted WML and frustration separately in two different studies. We reproduce this pattern of overlap in the current combined study. The overlap of effects in fNIRS sensors suggests that both frustration and WML variations modulate similar brain areas or brain areas in close spatial vicinity. Bilateral frontal areas in the inferior frontal gyrus (BA44/45) are known to be involved in WML processing (D'Esposito et al. 2000). Furthermore, Abler et al., 2005 reported that the anterior insula, located somewhat deeper below these superficial cortical areas, is involved in processing frustration. Future studies are required to test this hypothesis.
Acknowledgements
This work was funded by a grant of the Volkswagen Foundations to the Center for Critical Systems Engineering, a DFG-grant RI1511/2-1 to JWR, and KO 1990/5-1 to FK.
References
Abler, B., Walter, H., & Erk, S. (2005). Neural correlates of frustration. Neuroreport, 16(7), 669–72. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/15858403
D’Esposito, M., Postle, B. R., & Rypma, B. (2000). Prefrontal cortical contributions to working memory: evidence from event-related fMRI studies. In Executive control and the frontal lobe: Current issues (pp. 3-11). Springer, Berlin, Heidelberg.
Fischer, M., Richter, A., Schindler, J., Plättner, J., Temme, G., Kelsch, J., & et al. (2014). Modular and Scalable Driving Simulator Hardware and Software for the Development of Future Driver Assistence and Automation Systems. In New Developments in Driving Simulation Design and Experiments: Driving Simulation Conference 2014. Paris, France. Retrieved from http://elib.dlr.de/90638/
Unni, A., Ihme, K., Jipp, M., & Rieger, J. W. (2017). Assessing the driver’s current level of working memory load with high density functional near-infrared spectroscopy: A realistic driving simulator study. Frontiers in Human Neuroscience, 11. https://doi.org/10.3389/fnhum.2017.00167
Keywords:
Working memory load,
Frustration,
functional near-infrared spectroscopy (fNIRS),
driving,
brain-level interactions,
Multivariate predictions
Conference:
2nd International Neuroergonomics Conference, Philadelphia, PA, United States, 27 Jun - 29 Jun, 2018.
Presentation Type:
Oral Presentation
Topic:
Neuroergonomics
Citation:
Unni
A,
Kretzmeyer
B,
Ihme
K,
Koester
F,
Jipp
M and
Rieger
JW
(2019). Demonstrating brain-level interactions between working memory load and frustration while driving using functional near-infrared spectroscopy.
Conference Abstract:
2nd International Neuroergonomics Conference.
doi: 10.3389/conf.fnhum.2018.227.00091
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
27 Mar 2018;
Published Online:
27 Sep 2019.
*
Correspondence:
Mr. Anirudh Unni, University of Oldenburg, Department of Psychology, Oldenburg, Germany, anirudh.unni@uni-oldenburg.de