fNIRS, EEG, ECG, and GSR reveals an effect of complex, dynamically changing environments on cognitive load, affective state, and performance, but not physiological stress

Henrikke Dybvik^1,2,3*Christian Kuster Erichsen²Chris Snider¹Martin Steinert²

• ¹University of Bristol, Bristol, United Kingdom
• ²TrollLABS, Norwegian University of Science and Technology, Trondheim, Sør-Trøndelag, Norway
• ³SCANCOR, Graduate School of Education, Stanford University, Palo Alto, United States

This work used functional near-infrared spectroscopy (fNIRS), electroencephalography (EEG), electrocardiography (ECG), electrodermal activity (EDA), performance, and subjective self-reports to investigate cognitive load and stress in a complex, dynamically changing environment. N=30 participants were tasked with three Tetris gameplays: Easy and Hard had constant difficulties, and Ramp started at a low difficulty before successively ramping up to a very high difficulty level. Participants performed significantly best in Easy, followed by Ramp and Hard. In general, increased workload resulted in increased cognitive load and stress, but only until a certain threshold, after which fNIRS activation reduced, possibly due to mental fatigue or disengagement. We furthermore found a temporal effect of workload in the constant workload conditions, evidenced by increased fNIRS activation (HbO increase and HbR decrease), and mental fatigue measured by EEG (Delta power increase). Despite large differences in cognitive load, we found little between-condition differences in physiological stress response as measured by ECG and EDA. At the same time, Easy yielded significantly higher participant-ratings of valence, enjoyment, workload acceptability, and subjective performance, compared to Hard, indicating differing affective states. The combination of indistinguishable physiological stress and different affective states suggests participants experienced more a state of eustress in Easy and distress in Hard.