Prefrontal Cortex Activation Upon a Demanding Virtual Hand-Controlled Task: A New Frontier for Neuroergonomics

Introduction

The term neuroergonomics was first introduced in 1997 for depicting an interdisciplinary area of research which involves the intersection of two disciplines: neuroscience and ergonomics (Parasuraman and Rizzo, 2007). The studies in this field, previously carried out using either mobile or immobile neuroimaging techniques, have been nicely reviewed by Mehta and Parasuraman (2013). Virtual reality (VR), a computer-based technology that allows the creation of multisensory simulated environments in which users can interact and receive real-time feedbacks on their performance, was claimed by Kearney et al. (2007) to be highly relevant for neuroergonomics. This because VR can replicate, with a greater control than that applicable in the real world, a wide range of conditions that are impractical or impossible to observe in the real situations; then allowing behavioral and neurophysiological observations of the mind and brain at work. Given its peculiarity, VR is also effectively used by human operators to accomplish their work in dangerous environments, thus avoiding any physical risk. For instance, applying gesture-based control interfaces, VR is usually employed for tele-operated applications such as driving robots, rovers and other devices remotely, with the operators at a certain distance from them (Chen et al., 2015; Liu and Zhang, 2015; Wei et al., 2015). The tele-operated systems are very expensive, unique neither replicable nor quickly replaceable, and from their proper use depends the success or failure of long-planned, critical, costly and challenging operations. Taking into account the high degree of responsibility inherent to operators’ duties, their considerable physical/cognitive work should be evaluated objectively by neuroimaging techniques in the framework of neuroergonomics (for review, see Gramann et al., 2011, 2014; Mehta and Parasuraman, 2013).

Although the most widely used immobile functional neuroimaging modality has been undoubtedly represented by functional magnetic resonance imaging (fMRI), the development of portable and wearable neuroimaging devices, comprising electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS; for review, see Mehta and Parasuraman, 2013; Gramann et al., 2014; Scholkmann et al., 2014) has considerably facilitated the approach of neuroergonomics. fNIRS, with its high degree of ecological validity and its very limited requirement of physical constraints to subjects, represents a valuable tool for monitoring cortical responses in the research fields of neuroergonomics (for reviews, see Ayaz et al., 2013; Derosière et al., 2013). Furthermore, compared to fMRI, fNIRS is silent, allowing to avoid any bias in the results due to difficulties in focusing on the task because of the high level of noise. Briefly, fNIRS is a non-invasive vascular-based functional neuroimaging technology which assesses, simultaneously from multiple measurement sites, concentration changes in oxygenated-deoxygenated hemoglobin (O₂Hb/HHb, respectively) at the level of the cortical microcirculation blood vessels (Scholkmann et al., 2014). O₂Hb/HHb, indeed, interact differently with near infrared light, so that both physiological indexes can be recovered from the measured signal. This is a further advantage of fNIRS over fMRI, the latter being able to recover only a single physiological index, namely the blood-oxygen level dependent (BOLD) signal. When a specific brain region is activated, cerebral blood flow increases in a temporally and spatially coordinated manner through a complex sequence of coordinated events, tightly linked to changes in neural activity (i.e., neurovascular coupling). The coupling between the neuronal activity and the cerebral blood flow is fundamental to brain function. fNIRS relies exactly on this coupling to reveal the activated cortical region by measuring the associated cortical blood oxygenation changes (i.e., the increase in O₂Hb and the decrease in HHb).

Since 1993, fNIRS has been employed for evaluating the spatiotemporal characteristics of the cortical activation during different motor tasks related to upper and/or lower limb exercise (for a review, see Leff et al., 2011). In most previous fNIRS studies, the activation of the sensory-motor cortex and the PFC has been widely investigated in different tasks of the lower limb like walking (Koenraadt et al., 2014; Mirelman et al., 2014), stepping (Huppert et al., 2013), precision stepping (Koenraadt et al., 2014), etc. Several fNIRS studies have also investigated the PFC activation during hand tasks such as: finger movement (Wriessnegger et al., 2012), passive finger movement (Chang et al., 2014), isometric grasping/grasping (Mandrick et al., 2013), handgrip exercise (Derosière et al., 2014), learning a hand motor skill (Hatakenaka et al., 2007), etc.

Since 2000, fNIRS has been also employed in real-world activities realized in VR environment for evaluating the PFC activation during the simulation of different hand-related demanding/fatiguing activities, like airplane piloting (Ayaz et al., 2012a; Durantin et al., 2014; Gateau et al., 2015), car driving (Tomioka et al., 2009), grasping (Holper et al., 2010, 2012), natural orifice transluminal endoscopic surgery (James et al., 2011), etc. Interestingly, the inferior frontal gyrus (Ayaz et al., 2012a) and the dorsolateral PFC (DLPFC; Durantin et al., 2014; Gateau et al., 2015) were found activated during airplane piloting tasks. The bilateral ventrolateral PFC (VLPFC) was found activated during natural orifice transluminal endoscopic surgery when the simulation required a more difficult navigation path through an orifice (James et al., 2011).

It is well-known that the PFC, and in particular the VLPFC and DLPFC, are involved in the control of the motor actions. On one hand, it has been demonstrated that the VLPFC is involved in visuo-motor learning tasks (Yamagata et al., 2012; Hoshi, 2013). Moreover the reflexive orienting seems to be controlled by the right VLPFC (Corbetta et al., 2008), whereas the goal relevant information for the action control seems to be maintained and retrieved by the left VLPFC (Badre and Wagner, 2007; Souza et al., 2009). On the other hand, the DLPFC apparently plays a specific role in learning by trial and error (Halsband and Lange, 2006). Furthermore DLPFC, for its involvement in mediating and monitoring of actions, is considered to be the major anatomical correlate of the central executive (Baddeley, 2003; Gateau et al., 2015). Therefore, the VLPFC and the DLPFC are involved in associating visual information with motor responses (Halsband and Lange, 2006; Tanji and Hoshi, 2008).

The PFC plays not only a crucial role in single cognitive or motor tasks, but also in combined sensorimotor-cognitive task (i.e., dual-task; Gentili et al., 2013; Mandrick et al., 2013; Mirelman et al., 2014). Several fNIRS studies have reported that, in comparison to a single task, the attention-demanding dual-tasks (e.g., walking while talking, calculating while stepping, balancing a ball while walking, etc.) induced an increase of the PFC activation due to a greater cognitive load (Holtzer et al., 2011). For instance, Mandrick et al. (2013) investigated how an additional mental load (i.e., arithmetic task) during isometric grasping affects the PFC activation. The performance of the mental task was impaired when the motor task difficulty increased, suggesting that performing a dual-task requires more attentional resources than performing a single task.

In the last few years, the use of VR interfaces, driven by natural hand movements for remote control, is growing-up thanks to the development of innovative optical 3-dimensional (3D) systems for gesture recognition (Erden and Çetin, 2014). The key advantage of gesture recognition technology is that no physical contact is required between the human body and the gesture recognition device, so that the subjects can move freely. One of the most recent optical 3D sensors, based on stereo-vision, is the LEAP Motion Controller^® (LEAP). The LEAP is a high-resolution 3D hand-sensing device, which allows the freehand natural interaction crucial for the implementation of real-time, realistic VR systems. This low cost non-bulky device has an extremely accurate reactivity (Bachmann et al., 2014). In addition, the 3D rendering technologies, including state of the art displays and visors specifically designed for VR (Dodgson, 2013; Desai et al., 2014; Nan et al., 2014), have permitted a large development of VR techniques. In particular, their high visual/rendering fidelity and an immersive wide field of view enables the sensation of presence and the feeling being actually inside the virtual scene with the possibility to have multi-sensorial feedbacks.

The aim of the present study was to investigate non-invasively by fNIRS the PFC responses in healthy subjects while performing a complex hand-controlled task (HCT) in a VR environment. This task emulated the interaction with a real, remotely-driven, system operating into a critical environment. The hand movements were captured by the LEAP, a high spatial and temporal resolution 3D hand-sensing device. The subjects were asked to move their right hand/forearm with the purpose of guiding a virtual ball (VB) over a virtual route (VROU). The VROU can be easily and purposely designed to replicate a real track that an operator should travel to carry on a given challenging operation. The HCT-related PFC response was monitored non-invasively by a 20-channel fNIRS system. The HCT involved the control of the hand/forearm movement, and the active interaction with the virtual environment through the hand/forearm motor actions. The attentional resources allocation and the integration of different executive functions (e.g., coordination, planning, decision making, etc.) are needed in performing the HCT. Taking into account the well-known role played by the VLPFC and the DLPFC in motor action preparation and in the allocation of the attentional resources to generate goals from current situations, it was hypothesized that either VLPFC or DLPFC or both of them would be activated bilaterally in subjects while performing the HCT.

Materials and Methods

Participants

Fifteen University students (all males, age: 26.6 ± 2.9 years; level of education: 14.4 ± 2.1 years), without neurological or psychiatric illness and normal or corrected-to-normal vision were recruited in the study. In order to prevent any gender differences in emotional responses (Matud, 2004) and in visuo-motor abilities (Wang et al., 2015), only men were enrolled. To exclude left-handed subjects, all participants completed the Edinburgh Handedness Inventory assessing hand dominance. Following a full explanation of the protocol and its non-invasiveness, and prior to the starting of the experimental procedure, a written informed consent was obtained from each participant. All procedures were conducted in accordance with the Declaration of Helsinki and approved by the University Ethics Committee.

Experimental Setup

Hand-Controlled Task (HCT)

A VR HCT was implemented by integrating a LEAP Motion Controller^® with a real-time 3D engine. The LEAP provides both a 3D hand model and real-time hand tracking information for enabling subjects to transpose their hand movements within the virtual 3D HCT (Figure 1). The LEAP is a small (1.3 cm × 3.2 cm × 8 cm) 3D sensor which uses two internal infra-red (IR) cameras and three IR light emitting diodes to detect objects within a dome of approximately 0.22 m³ above it. Its spatial and temporal resolution is 1 mm and 15 ms, respectively. The LEAP, connected to a computer via a USB cable, is designed specifically to detect, in real-time, hand and finger motions and gestures, such as pinching fingers, closing hand, tapping, etc. This device, positioned under the palm center of the right hand at a distance of about 25 cm (Figure 1), was utilized to: (1) capture the movements of the hand; (2) associate hand movements to a virtual hand model; and (3) translate the movements of the virtual hand model to a set of commands in order to drive a VB within a virtual environment.

FIGURE 1

Figure 1. Experimental setting for the execution of a virtual reality (VR) hand-controlled task (HCT). The upper large image shows the subject with the functional near-infrared spectroscopy (fNIRS) probe holder while sitting in front of a PC, and the positioning of the LEAP motion controller below the operator hand. The four columns of small images show (from left to right): the real position of the subject hand/forearm (first column); the corresponding hand/forearm virtual model (second column); the influence of the commands on the virtual ball (VB) during the HCT (third column); and the corresponding effects on a real, remote, spider-like rover (fourth column, in dashed line to indicate that the presence of the rover is just supposed). The yellow arrow indicates the direction of the guided VB and the red arrow indicates the corresponding effect on the rover. The length of the arrows indicates the force amplitude impressed by the operator. Note that, during the HCT, the hand/forearm virtual model (second column) was not shown to the operator and the visualization of the arrows (third column) was disabled.

In the present study, the adopted virtual environment was aimed at simulating the driving action of a spider-like robot similar to the one developed by the National Aeronautics and Space Administration (NASA). Due to its high stability, equilibrium and ability to change quickly direction, this robot has been proven to be adapted for moving in very rough environments and in environments designed for humans (to go up and down stairs). The movements of the ball, in fact, simulated fairly those ones performed by the considered robot (lateral, ahead, behind, and stop). The aim of this adopted HCT was to move the VB over a VROU of a fixed length (42 m; Figure 2), trying to travel as long as possible the distance in a fixed time without falling (2 min). Either in the case of VB falling or in the case of accomplishment of a VROU in advance, subjects were requested to restart the VROU from the beginning. To calculate the whole distance traveled by each subject over the HCT, the completed VROU and the distance traveled until the task end were considered. The VROUs with failure were not considered. The VROU adopted in this study (Figure 2) was purposely designed to reproduce a narrow road including some critical points (i.e., stairs, turns, a slippery part, and climbs). The Torque 3D Engine¹, a cross-platform high performance real-time 3D engine, was used both for the editing and the rendering of the whole virtual 3D HCT. A controllable VB and a 3D VROU were created by using a customized version of Marble Motion, a well-known game². In particular, the whole native source code was rewritten and enriched in order to fulfill the requirements of the task: (1) a time driven version of the task in which both the start and the end of the HCT were established by a fixed time interval; (2) the possibility of storing information related to the operator-task interaction process (including the number of times in which the VB was fallen out of the VROU, the position of the VB falls over the VROU, the distance traveled by the VB in the given time). The software allowed also the calculation of some dynamic parameters (followed trajectory, VB speed and acceleration). These last parameters have been considered to calculate the VB speed along the VROU, to get information about resting periods occurred during the HCT, and to evaluate the subject skills while executing the HCT. Moreover, to increase the subject’s concentration and motivation during the HCT, the whole technical items, including elements of the VROU (textures and materials), were redefined utilizing some predefined items of the 3D engine editor (Torque 3D Editor)³.

FIGURE 2

Figure 2. A perspective view of the designed virtual route (VROU) which reproduces a narrow road including some critical points. 1: stairs; 2: turns; 3: slippery part; 4: climbs; and 5: sequential turns.

The subject was asked to place his right forearm on a fixed and firm support in order to allow the hand capture (Figure 1). This support ensured the maintenance of the correct position of the forearm, and consequently of the hand, during the execution of the HCT. The task started with a stationary VB placed at the beginning of the VROU. The subject had to maintain his right hand opened over the LEAP device by keeping his forearm on the support with the center of the palm perpendicular to the center of the device and with all the five fingers extended (Figure 1). At the beginning of the HCT, the subject had to guide the VB over the VROU by using four commands (Figure 1). The first command (hand flexion) made the VB to proceed forward; the second command (hand extension) made the VB to decrease the speed (up to stop the VB) and to proceed backward; the third and the fourth commands (counter clockwise and clockwise rotations of the wrist) made the VB to move toward left or right, respectively (a combined use of the hand flexion/extension movements or the rotation of the wrist made the VB to stop). These hand movements had a real-time proportional impact on the VB. More specifically, the command chosen by the subject transmitted the direction and the “force” to the VB (e.g., a low hand downward flexion corresponded to a low “force” application to the VB in the forward direction), especially when the VB speed was depending on the inclination degree of the downward flexion and the time during which the hand was maintained at the same position. Thus, the HCT was purposely designed to combine the four main commands and when the subject assumed a pose of his hand halfway between two commands, the system merged both directions and “force” amplitude. In this way, the subject had the feeling to guide the VB without restrictions or constraints.

The effective range of the LEAP tracking system is limited to roughly 60 cm due to the low near infrared light intensity. In this study, the distance between the fNIRS head probe and the LEAP was always greater than 60 cm. Therefore, the LEAP should have not interfered with the fNIRS measurements. Interestingly, several fNIRS studies in combination with other near infrared based tracking systems such as Kinect or eye trackers have been published (Kita et al., 2010; Sukal-Moulton et al., 2014; Urakawa et al., 2015). Although those systems have utilized more powerful light emitters (than the LEAP), and the light emitters have been directly pointed toward the fNIRS head probe, the potential interference with the fNIRS data was not mentioned.

fNIRS Instrumentation and Data Processing

A two-wavelength continuous wave 20-channel fNIRS system (Oxymon Mk III, Artinis Medical Systems, Netherlands) was utilized to map non-invasively the changes in O₂Hb and HHb over the bilateral PFC. The details of this instrumentation have been previously reported (Basso Moro et al., 2013). The O₂Hb/HHb data from the 20 channels were acquired at 10 Hz. The O₂Hb/HHb concentration changes (expressed in ΔμM), obtained by using the modified Beer-Lambert law and the age-dependent differential pathlenght factor (4.99 + 0.067 × Age^0.814) were displayed in real-time on a PC monitor. Eight optical fiber bundles (length: 3.15 m; diameter: 4.5 mm) were utilized to transport the light to the left and the right PFC (four for each hemisphere), whereas ten optical fiber bundles of the same size (five for each hemisphere) were utilized to collect the light emerging from the PFC. The illuminating and collecting bundles were assembled into a flexible probe holder, consisting of two mirror-like units (9.7 cm × 8.9 cm each) held together by three flexible junctions. In 16 out of the 20 channels the illuminator-detector distance was set at 3.5 cm, while in the remaining four channels the illuminator-detector distance was set at 1 cm (short-separation channels or SS channels). In the 16 channels, the measurement points were defined as the midpoint of the corresponding illuminator-detector pairs. The probe holder was placed over the subject head by a Velcro brand fastener in order to get a stable optical contact with the scalp (Figure 1). In particular, the two frontopolar fibers bundles, collecting the light at the bottom of the holder, were centered (according to the International 10–20 system for the EEG electrode placement) on the Fp1 and Fp2 locations for the left and right hemisphere, respectively. The pressure created by the fastener was sufficient to induce a partial transient blockage of the skin circulation during the fNIRS study. The adopted procedure would suggest that a consistent reduction of forehead skin blood flow was occurring as a result of this approach. The Montreal Neurological Institute coordinates of the optodes and the relative 16 measurement points were calculated using a probe placement method. For the details of this procedure see Basso Moro et al. (2013). The measurement points 1, 2, 3, 9, 10, 11 corresponded to the DLPFC, which includes part of the Brodmann’s Area (BA) 46; the measurement points 5, 6, 13, 14 corresponded to the frontopolar cortex, which includes part of the BA 10; and measurement points 4, 7, 8, 12, 15, 16 corresponded to the VLPFC, which includes part of the BA 45.

During the data collection procedure, the fNIRS signal quality as well as the absence of movement artifacts were verified on the PC monitor. The subject’s heart rate (HR) was monitored by a pulse oximeter (N-600, Nellcor, Puritan Bennett, St. Louis, MO, USA) with the sensor clipped to the index finger of the left hand. The Homer2 NIRS processing package⁴ was employed to analyze the data. Raw intensity data in each channel were converted into optical density changes (OD). Channels showing low intensity values were excluded from further analyses. The Wavelet motion correction method was employed to correct motion artifacts. Based on the method developed by Molavi and Dumont (2012), it sets to zero all wavelet detail coefficients exceeding a predefined threshold (iqr = 0.1). The modified Beer-Lambert law was then applied to convert the corrected OD data into concentration changes. A General Linear Model (GLM) approach (hmrDeconvHRF_DriftSS) was utilized to recover the mean hemodynamic response function (HRF) for each subject and channel. The approach, consisting in adding the SS channel signal with the highest correlation with the analyzed standard channel signal in the design matrix, was able to reduce the contribution of the mean arterial blood pressure changes in task-evoked fNIRS signal. The less restrictive set of Gaussian functions with standard deviation (SD) of 3 s and with their means separated by 2 s was chosen as temporal basis functions (ranging between −20 before and 210 s after the starting of the HCT; Gagnon et al., 2011).

Experimental Design

A familiarization/training phase was carried out 3 days before the study. The subjects were informed about the procedures and familiarized with both the experimental setting and the HCT. During this phase, the fNIRS probe holder was placed over the head of the subjects who were trained to stay as firm as possible to avoid movement artifacts in fNIRS measurement during the HCT execution. After evaluating the joint mobility of their hand and wrist, the subjects were requested to pay attention to the presented four commands to be used for guiding properly the VB (hand flexion, hand extension, counter clockwise and clockwise rotations of the wrist). Later, the subjects were asked to place their right hand opened over the LEAP in order to verify the correctness of the estimated hand virtual model. Once this phase was completed, the subjects were asked to guide a VB over a VROU. In order to avoid a potential learning effect, a different VROU was used in this training phase. For each subject, the training phase was considered completed when he demonstrated his ability to guide properly the VB. After 3 days, at the same time of the training phase, the subjects participated in the study. A monetary reward was not given.

This study was carried out in a quiet and dimly lit room. The subject was asked to sit on a comfortable high-backed chair in front of a 17″ PC monitor, and to keep his forearm on the firm support with the right hand opened over the LEAP (see “The Hand-Controlled Task” Section; Figure 1). The HCT protocol lasted 6 min. Specifically, the protocol started with a 3 min baseline, during which subjects were asked to relax (observing a white fixation cross presented on a black screen) in order to get stable fNIRS signals. Then, a stationary VB came into view on the PC monitor, and a visual instruction informed the subjects that the 2 min HCT was starting. During the HCT, the subject had to guide the VB over the VROU through the four hand movements (see “The Hand-Controlled Task” Section). When the subject failed in guiding the VB over the VROU or the subject completed the VROU in less than 2 min, the VB was repositioned at the VROU starting point, and the route restarted. At the end of the HCT execution, there was a recovery period (1 min), in which the subject was requested to relax while observing a white fixation cross presented on a black screen. In order to evaluate the potential “state anxiety” provoked by HCT, all the subjects completed the 20-items of the State Trait Anxiety Inventory Form Y-1 (STAI) before and after the protocol.

Data Analysis and Statistics

The integral values of the O₂Hb/HHb (_INTO₂Hb/HHb) changes of the HRFs were calculated from the beginning (at 0 s) until the end of the HCT (at 120 s), for each measurement point and subject, and were used as metric for the following statistical analysis. The mean values of the HR changes (analyzed as percentage of control) were calculated from the beginning (at 0 s) until the end of the HCT (at 120 s). Both the _INTO₂Hb/HHb changes and the HR mean values were corrected for the baseline periods, calculated over the last 20 s before the starting of the HCT. The median of the values of the distance traveled by the guided VB was calculated to subdivide the subjects in two groups: best performers (above the median) and worst performers (below the median). The subject of the median value was not included in any group. Student’s t-test was conducted in order to evaluate the presence of any difference, in terms of the distance traveled by the guided VB between the worst and best performers.

All data were examined for normality and sphericity using Shapiro–Wilk and Mauchly’s Sphericity tests, respectively. Each level of the independent variables followed a normal or approximately normal distribution in all the dependent variables (O₂Hb, HHb and HR), permitting the use of parametric statistical analyses. When the sphericity was not assumed, the Greenhouse-Geisser correction was utilized.

In order to investigate the PFC activation in response to HCT, the two-way analysis of variance (ANOVA) was applied to _INTO₂Hb/HHb changes. The ANOVA included two factors: measurement point (16 levels) and cortical hemodynamic response (CHR; i.e., corrected task period vs. zero; 2 levels). To control for multiple significance tests, the Fisher’s least significant difference adjustment was applied. A series of one-way ANOVAs was performed for the HCT in order to evaluate the influence of the CHR (2 levels) on the _INTO₂Hb/HHb changes. In particular, the one-way series of ANOVAs were performed only for the _INTO₂Hb/HHb changes related to the measurement points 7, 8, 15 and 16, chosen as descriptive measurement points of the hemodynamic response. For the HCT, the Pearson’s correlation coefficient was calculated in order to evaluate the relation between the distance traveled by the guided VB and the _INTO₂Hb/HHb changes in the 7, 8, 15 and 16 measurement points. A one-way ANOVA was performed for the HCT in order to evaluate the influence of the CHR (2 levels) on the HR changes. The Pearson’s correlation coefficient was also calculated for evaluating the relation between the distance traveled by the guided VB and the HR. Student’s t-tests were conducted in order to evaluate the presence of any difference in: (1) the anxiety state before and after the protocol; (2) _INTO₂Hb/HHb changes in the 7, 8, 15 and 16 measurement points between the worst performers and best performers; and (3) the distance traveled by the guided VB of worst and best performers.

All statistical analyses were conducted with SPSS 20.0 (SPSS Inc., Chicago, IL, USA). Data were expressed as mean ± SD. The criterion for significance was p < 0.05.

Results

The behavioral data analysis revealed the following main results. There was no significant difference (t = 0.64, p = 0.53) in the anxiety state before (28.8 ± 6.4) and after the protocol (28.9 ± 5.8). The distance traveled by the guided VB in the 15 subjects was: 21, 26, 36, 43, 51, 53, 56, 58, 61, 83, 88, 96, 101, 132, and 148 m, respectively. The median value was 58 m. The mean distance was 70.2 ± 37.2 m. The less skilled subjects failed several times in guiding the VB over the VROU. Therefore, the distance traveled by their guided VB was shorter. Indeed, the distance traveled by the guided VB was significantly different (t = −4.89, p < 0.001) between the worst performers (40.9 ± 13.7 m) and best performers (101.3 ± 29.7 m). The fNIRS data evidenced a heterogeneous O₂Hb/HHb response over the mapped cortical area in the subjects while performing the HCT (Figure 3). In particular, since the beginning of the HCT, a progressive O₂Hb increase and a concomitant progressive HHb decrease were observed in the measurement points 7, 8, 15 and 16, corresponding to the VLPFC, which includes part of the BA 45. About 15 s after the end of the HCT, a gradual return of O₂Hb/HHb to the corresponding baseline values, was observed. This delay is reasonable considering that the cerebral blood flow increase lasts over the period of the HCT. The statistical analysis revealed the following main results. The two-way ANOVA analysis, carried out on the _INTO₂Hb changes, revealed a significant main effect of: (1) the measurement point (F_(2.67,37.47) = 9.52, p < 0.001), and (2) the measurement point * CHR interaction (F_(2.67,37.47) = 9.52, p < 0.001). The two-way ANOVA analysis, carried out on the _INTHHb changes, revealed a significant main effect of the: (1) measurement point (F_(2.83,39.67) = 15.70, p < 0.001); (2) CHR (F_(1.00,14.00) = 13.83, p = 0.002); and (3) measurement point * CHR interaction (F_(2.83,39.67) = 15.70, p < 0.001). The two-way ANOVAs, carried out on the _INTO₂Hb/HHb changes, revealed the main significant differences between the measurement points 7, 8, 15, 16 and all the others (ps < 0.05). The series of one-way ANOVAs, carried out on the _INTO₂Hb changes of the measurement points 7, 8, 15, and 16, revealed a significant activation in all the measurement points (F_(1,14) = 5.32, p = 0.037; F_(1,14) = 23.79, p < 0.001; F_(1,4) = 7.33, p = 0.017; F_(1,14) = 16.89, p = 0.001).The series of one-way ANOVAs carried out on the _INTHHb changes of the measurement points 7, 8, 15, and 16 revealed a significant cortical activation in all the measurement points (F_(1,14) = 41.61, p < 0.001; F_(1,14) = 58.57, p < 0.001; F_(1,14) = 19.12, p = 0.001; F_(1,14) = 26.23, p < 0.001). In the HCT, no correlation was found between the distance traveled by the guided VB and the corresponding _INTO₂Hb/HHb changes (ps > 0.05). During the HCT, no differences (ps > 0.05) were found in the _INTO₂Hb/HHb changes in the 7, 8, 15 and 16 measurement points between the worst performers and best performers. The one-way ANOVA analysis for the HR mean values revealed a significant main effect of the CHR (F_(2.88,40.40) = 8.06, p < 0.001). However, the mean values of the HR changes during the execution of the HCT increased only of about 15% with respect to the mean value of the baseline. No correlation was found between the HR and the distance traveled by the guided VB.

FIGURE 3

Figure 3. Grand average of O₂Hb (red line) and HHb (blue line) changes observed over the bilateral PFC [16 measurement points of the right (from 1–8) and left (from 9–16) hemisphere] in response to the VR HCT. The corresponding numbers in the brackets refer to the associated Brodmann’s Area (BA). The vertical solid lines limit the duration of the task execution. The major cortical activation was observed in the bilateral VLPFC (measurement points 7, 8, 15, and 16). N = 15; means ± SD.

Discussion

In this feasibility study, the bilateral PFC was investigated by a multi-channel fNIRS system while subjects performed a demanding VR HCT, a remotely-driven operation simulated by a high-resolution and low-cost 3D hand-sensing device. The observed involvement of the bilateral VLPFC supports the formulated hypothesis.

The results of the present study have indicated a consistent bilateral VLPFC activation (measured as O₂Hb increase and a concomitant HHb decrease) in response to the execution of the VR HCT (Figure 3). It has been reported that VLPFC is involved in associating visual information with motor responses (Tanji and Hoshi, 2008). In fact, the execution of the adopted HCT requires the combination of the contextual visual information in order to coordinate the hand/forearm movements for guiding the VB over the VROU. Although the tested subjects had the same age and level of education (University students), their skills in performing the HCT were different. Indeed, the distance traveled by the VB was heterogeneous: the distance traveled by the VB guided by the best performer was about seven times longer than the distance traveled by the VB guided by the worst performer. These results clearly confirm that the designed VROU (Figure 2) was really demanding. The diverse skills of the subjects could not be attributable to emotional factors, because no difference between anxiety state before and after the HCT, and no correlation between the distance traveled by the guided VB and the HR of the subjects were found. However, a bilateral VLPFC activation was observed indiscriminately in all the tested subjects, including the ones who never completed at least one VROU. Therefore, the present fNIRS data did not provide the possibility to discriminate the subjects according to their performance. This could be partly explained by the fact that other cortical areas were not investigated in this study and that subcortical areas and/or cortical-subcortical network are supposed to be responsible of the differences between the best and the worst performers. As well known, PFC is involved in the executive functions (e.g., attention, coordination, planning, decision making, etc.), the same required to perform the HCT regardless of the performer’s skills. The combined use of a fNIRS-EEG system has evidenced a greater involvement of the deeper structures (e.g., hippocampus) in the “good performers group” compared with the “bad performers group” while executing a spatial navigation task (Kober et al., 2013). Moreover, it has been reported that a complex sensorimotor-cognitive task, such as the adopted HCT, would require the involvement of different cortical-subcortical networks including: PFC, spinal cord, brainstem, cerebellum, basal ganglia, and motor cortex (Takakusaki, 2008). This suggests that the degree of the cognitive demand (measured by fNIRS), required for executing the HCT, is not associated with the subjects performance (measured as the distance traveled by the guided VB). This in part confirms the results of other VR fNIRS studies in which a dissociation between the mental work demanded to execute a complex task and the performance output was observed (Ayaz et al., 2012a; Boyer et al., 2015). In the present study, this dissociation could be also explained by the fact that tested subjects had only a short familiarization/training phase. A higher activation of the PFC was usually observed in non-expert subjects while executing a novel VR task compared to expert (Ayaz et al., 2012b). The non-expertise requires the employment of more attentional resources to perform a novel task. On the contrary, the expertise implies an increase of automaticity and does not require the same high level of attention and control (Ayaz et al., 2012b). The VLPFC activation, observed in the present study in non-expert subjects while performing the novel VR HCT, suggests that some of the well-known executive functions (e.g., attention, coordination, planning, decision making, etc.) are required in the learning phase. Then, it could be supposed that the amplitude of the observed VLPFC activation would become lower or even disappear in subjects very familiar with the VR HCT.

Several studies evidenced the advantages of using fNIRS technology for investigating non-invasively cortical responses in subjects while performing different VR tasks; the most representative studies are listed in Table 1. The common relevant finding is represented by the activation of different regions of the frontal cortex and the PFC. For example, the involvement of the medial PFC (mPFC) and the frontopolar cortex (Ayaz et al., 2012b) and the inferior frontal gyrus (Harrison et al., 2014) was observed in subjects while executing air traffic control tasks; an activation of the overall frontal cortex was observed in subjects while performing a train piloting task (Kojima et al., 2005). Very recently, the usefulness of fNIRS as a tool to conduct driving research has been nicely reviewed (Liu et al., 2015). For example, an activation of the right PFC (Tomioka et al., 2009) and an activation of the overall PFC (Tsunashima and Yanagisawa, 2009) were found in subjects while executing different simulated car driving tasks. However, in all of the above reported studies, no activation of the VLPFC, and in particular of the BA 45, was found.

TABLE 1

Table 1. Selected fNIRS studies about the effects of VR tasks on different cortical areas.

The combined use of fNIRS-EEG would be an ideal tool for carrying out studies in the field of neuroergonomics. The pros of lightweight, high-density EEG and fNIRS recording to study natural human cognition have been previously reviewed (Gramann et al., 2011, 2014). Simultaneous fNIRS-EEG measurements offer complementary functional information about neuronal activity and hemodynamic changes in order to provide a wider perspective on different aspects of the cortical processes. To the best of our knowledge, Kober et al. (2013) have first compared neuronal responses of good and bad navigators during a VR spatial navigation task by a combined fNIRS-EEG system. The commercial fNIRS systems, utilized either by Kober et al. (2013) or in the present study, are equipped with fiber optic bundles. The disadvantage of using fiber optic bundles is that the fibers are often heavy and with a limited flexibility. Therefore, this kind of fNIRS instrumentation is not the best choice for studies in the neuroergonomics field. Since 2009, different battery operated multi-channel wearable/wireless fNIRS systems have been commercialized (Scholkmann et al., 2014). A 64-measurement point wireless fNIRS system was developed and integrated with simultaneous EEG and electrocardiography (ECG) monitoring in order to record data up to several days (Zhang et al., 2014). These most advanced versions of integrated fNIRS-EEG systems represent a suitable tool for evaluating brain activation in response to cognitive tasks executed in normal daily activities. To make fNIRS technology more suitable for neuroergonomics studies, in terms of robustness, mobility, user-friendliness and customization, a very recent and successful effort has been made to realize a dedicated fNIRS device (von Lühmann et al., 2015). Further future hardware developments will increase the production of miniaturized wireless integrated fNIRS-EEG devices and their use in neuroergonomics. It is noteworthy to mention that a 16-measurement point wireless fNIRS system has been recently coupled with transcranial direct current stimulation (tDCS) in order to investigate the effects of tDCS on spatial working memory (McKendrick et al., 2015). These authors have suggested the utility of using the combination of simultaneous tDCS and fNIRS techniques for future applications in the field of neuroergonomics: from enhanced/accelerated learning and training of complex human-machine systems to optimization of task load for improved safety and productivity. The neuroergonomics approaches can also provide sensitive and reliable assessment of mental workload in complex tasks and naturalistic work settings (Parasuraman, 2011). The mental workload has been defined as the degree of the effort to be made by the brain to meet the task demands (Young et al., 2015). Recently, Peck et al. (2014) have reviewed the use of fNIRS to measure mental workload in the real world tasks, and the different approaches for automatic detection of the workload.

The strengths and the limitations of the fNIRS technique have been previously discussed in detail (for review, see Scholkmann et al., 2014). The fNIRS equipment is transportable, completely safe and non-invasive. These advantages allow for the investigation of brain activity in natural conditions (e.g., while sitting on a chair) and during daily life activities (e.g., standing and/or walking). Therefore, with respect to other functional neuroimaging methods such as fMRI, fNIRS represents an useful tool for neuroergonomics research (for review, see Ayaz et al., 2013; Derosière et al., 2013), for studies in other fields of neuroscience such as brain-computer interface (for review, see Naseer and Hong, 2015), human-robot interaction (for review, see Canning and Scheutz, 2013), and cognitive states measurements (for review, see Strait and Scheutz, 2014). In addition, very recently the integration of fNIRS with a wearable technology, such as Google Glass, has been demonstrated (Afergan et al., 2015). For an adequate understanding of the current findings, some limitations should be pointed out: (1) this study has been conducted in a small sample of healthy young male adults subjects and the subjective cognitive load was not tested by NASA Task Load Index; (2) the duration of the adopted task and the length of the route were relatively short, hence, the effect of a longer duration of the VR HCT on the PFC hemodynamic response remains unknown; (3) this study did not imply a control session for example including/not including motor task with/without VR; (4) this study did not contemplate repeated trials on separate days in order to verify the reproducibility and the potential learning effect of the HCT; (5) the limited number of measurement points (16) made possible by the utilized fNIRS system did not allow the investigation of the supposed connectivity between the PFC and other cortical areas (e.g., premotor and motor cortices) likely involved in performing HCT; and (6) this study did not take into account the impact of the variability of the skull thickness amongst the 16 measurement points within subject and amongst subjects. This anatomical variability could be examined by acquiring structural T1-weighted MRI scans from each subject.

Conclusion

The results of the present study confirm the promising application of fNIRS technology to objectively evaluate cortical hemodynamic changes occurring in VR environments. The ongoing development of fNIRS technology, finalized to deliver more dedicated, sophisticated and wireless devices, together with the most advanced VR solutions, could provide the best combined approach for monitoring operators training and assessing mental work. Future studies could give a contribution to a better understanding of the cognitive mechanisms underlying human performance either in expert or non-expert operators.

Author Contributions

Study design and protocols were conceived by MC, AP, SL, SBM, MS, MF, GP and VQ. Data collection was performed by MC, AP, SL, SBM, MS, MF, GP and VQ. Data analysis was performed by MC, AP, SL, SBM, SB, MS, MF, GP and VQ. The manuscript was written by MC, AP, SL, SBM, SB, MS, MF, GP and VQ.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The study has been performed in the framework of the “Interdepartmental Research Centre for Molecular Diagnostics and Advanced Therapies”. This work was supported in part by: (1) the 2014 grant from the “Fondazione Cassa di Risparmio della Provincia dell’Aquila”, and (2) the “Abruzzo Earthquake Relief Fund” (Toronto, ON; purchase of the Artinis system). The authors wish to thank Dr. Simone Cutini, “Department of Developmental Psychology and Socialization”, University of Padua, Italy for helping in the identification of the relative Brodmann’s Areas, and Dr. Danilo Avola, “Department of Mathematics and Informatics”, University of Udine, Italy for HCT implementation.

Footnotes

1. ^ www.garagegames.com/products/torque-3d
2. ^ http://mit.garagegames.com/MarbleMotion-1-0b.zip
3. ^ www.garagegames.com/products/torque-3d/overview/editor
4. ^ www.nmr.mgh.harvard.edu/PMI/resources/homer2/home.htm

References

Afergan, D., Hincks, S. W., Shibata, T., and Jacob, R. J. K. (2015). Phylter: a system for modulating notifications in wearables using physiological sensing. Lect. Notes Comput. Sc. 9183, 167–177. doi: 10.1007/978-3-319-20816-9_17

CrossRef Full Text | Google Scholar

Ayaz, H., Cakir, M. P., Izzetoglu, K., Curtin, A., Shewokis, P. A., Bunce, S., et al. (2012a). “Monitoring expertise development during simulated UAV piloting tasks using optical brain imaging,” in Proceedings of the IEEE Aerospace Conference (Big Sky, MN), 1–11.

Ayaz, H., Shewokis, P. A., Bunce, S., Izzetoglu, K., Willems, B., and Onaral, B. (2012b). Optical brain monitoring for operator training and mental workload assessment. Neuroimage 59, 36–47. doi: 10.1016/j.neuroimage.2011.06.023

PubMed Abstract | CrossRef Full Text | Google Scholar

Ayaz, H., Onaral, B., Izzetoglu, K., Shewokis, P. A., McKendrick, R., and Parasuraman, R. (2013). Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development. Front. Hum. Neurosci. 7:871. doi: 10.3389/fnhum.2013.00871

PubMed Abstract | CrossRef Full Text | Google Scholar

Ayaz, H., Shewokis, P. A., Curtin, A., Izzetoglu, M., Izzetoglu, K., and Onaral, B. (2011). Using MazeSuite and functional near infrared spectroscopy to study learning in spatial navigation. J. Vis. Exp. 56:e3443. doi: 10.3791/3443

PubMed Abstract | CrossRef Full Text | Google Scholar

Bachmann, D., Weichert, F., and Rinkenauer, G. (2014). Evaluation of the leap motion controller as a new contact-free pointing device. Sensors (Basel) 15, 214–233. doi: 10.3390/s150100214

PubMed Abstract | CrossRef Full Text | Google Scholar

Baddeley, A. (2003). Working memory: looking back and looking forward. Nat. Rev. Neurosci. 4, 829–839. doi: 10.1038/nrn1201

PubMed Abstract | CrossRef Full Text | Google Scholar

Badre, D., and Wagner, A. D. (2007). Left ventrolateral prefrontal cortex and the cognitive control of memory. Neuropsychologia 45, 2883–2901. doi: 10.1016/j.neuropsychologia.2007.06.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Basso Moro, S., Bisconti, S., Muthalib, M., Spezialetti, M., Cutini, S., Ferrari, M., et al. (2014). A semi-immersive virtual reality incremental swing balance task activates prefrontal cortex: a functional near-infrared spectroscopy study. Neuroimage 85, 451–460. doi: 10.1016/j.neuroimage.2013.05.031

PubMed Abstract | CrossRef Full Text | Google Scholar

Basso Moro, S., Cutini, S., Ursini, M. L., Ferrari, M., and Quaresima, V. (2013). Prefrontal cortex activation during story encoding/retrieval: a multi-channel functional near-infrared spectroscopy study. Front. Hum. Neurosci. 7:925. doi: 10.3389/fnhum.2013.00925

PubMed Abstract | CrossRef Full Text | Google Scholar

Boyer, M., Cummings, M. L., Spence, L. B., and Solovey, E. T. (2015). Investigating mental workload changes in a long duration supervisory control task. Interact. Comput. 27, 512–520. doi: 10.1093/iwc/iwv012

CrossRef Full Text | Google Scholar

Brand, J., Geisseler, O., Holper, L., Hepp-Reimond, M. C., Morari, M., Kiper, D., et al. (2011). “The effects of manipulation of visual feedback in virtual reality on cortical activity: a pilot study,” in Proceedings of the IEEE International Conference on Virtual Rehab., Zurich, 1–2.

Google Scholar

Canning, C., and Scheutz, M. (2013). Functional near-infrared spectroscopy in human-robot interaction. J. Hum. Robot Interact. 2, 62–84. doi: 10.5898/jhri.2.3.canning

CrossRef Full Text | Google Scholar

Chang, P. H., Lee, S. H., Gu, G. M., Lee, S. H., Jin, S. H., Yeo, S. S., et al. (2014). The cortical activation pattern by a rehabilitation robotic hand: a functional NIRS study. Front. Hum. Neurosci. 8:49. doi: 10.3389/fnhum.2014.00049

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, S., Ma, H., Yang, C., and Fu, M. (2015). “Hand gesture based robot control system using leap motion,” in Intelligent Robotics and Applications, eds H. Liu, N. Kubota, X. Zhu, R. Dillmann, and D. Zhou (Cham: Springer International Publishing), 581–591.

Google Scholar

Corbetta, M., Patel, G., and Shulman, G. L. (2008). The reorienting system of the human brain: from environment to theory of mind. Neuron 58, 306–324. doi: 10.1016/j.neuron.2008.04.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Derosière, G., Alexandre, F., Bourdillon, N., Mandrick, K., Ward, T. E., and Perrey, S. (2014). Similar scaling of contralateral and ipsilateral cortical responses during graded unimanual force generation. Neuroimage 85, 471–477. doi: 10.1016/j.neuroimage.2013.02.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Derosière, G., Mandrick, K., Dray, G., Ward, T. E., and Perrey, S. (2013). NIRS-measured prefrontal cortex activity in neuroergonomics: strengths and weaknesses. Front. Hum. Neurosci. 7:583. doi: 10.3389/fnhum.2013.00583

PubMed Abstract | CrossRef Full Text | Google Scholar

Desai, R. P., Desai, N. P., Ajmera, D. K., and Mehta, K. (2014). A review paper on oculus rift-a virtual reality headset. Int. J. Eng. Trends Tech. 13, 175–179. doi: 10.14445/22315381/IJETT-V13P237

CrossRef Full Text | Google Scholar

Dodgson, N. (2013). Optical devices: 3D without the glasses. Nature 495, 316–317. doi: 10.1038/495316a

PubMed Abstract | CrossRef Full Text | Google Scholar

Durantin, G., Gagnon, J. F., Tremblay, S., and Dehais, F. (2014). Using near infrared spectroscopy and heart rate variability to detect mental overload. Behav. Brain Res. 259, 16–23. doi: 10.1016/j.bbr.2013.10.042

PubMed Abstract | CrossRef Full Text | Google Scholar

Erden, F., and Çetin, A. (2014). Hand gesture based remote control system using infrared sensors and a camera. IEEE Trans. Consum. Electron. 60, 675–680. doi: 10.1109/tce.2014.7027342

CrossRef Full Text | Google Scholar

Ferrari, M., Bisconti, S., Spezialetti, M., Basso Moro, S., Di Palo, C., Placidi, G., et al. (2014). Prefrontal cortex activated bilaterally by a tilt board balance task: a functional near-infrared spectroscopy study in a semi-immersive virtual reality environment. Brain Topogr. 27, 353–365. doi: 10.1007/s10548-013-0320-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Gagnon, L., Perdue, K., Greve, D. N., Goldenholz, D., Kaskhedikar, G., and Boas, D. A. (2011). Improved recovery of the hemodynamic response in diffuse optical imaging using short optode separations and state-space modeling. Neuroimage 56, 1362–1371. doi: 10.1016/j.neuroimage.2011.03.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Gateau, T., Durantin, G., Lancelot, F., Scannella, S., and Dehais, F. (2015). Real-time state estimation in a flight simulator using fNIRS. PLoS One 10:e0121279. doi: 10.1371/journal.pone.0121279

PubMed Abstract | CrossRef Full Text | Google Scholar

Gentili, R. J., Shewokis, P. A., Ayaz, H., and Contreras-Vidal, J. L. (2013). Functional near-infrared spectroscopy-based correlates of prefrontal cortical dynamics during a cognitive-motor executive adaptation task. Front. Hum. Neurosci. 7:277. doi: 10.3389/fnhum.2013.00277

PubMed Abstract | CrossRef Full Text | Google Scholar

Gramann, K., Ferris, D. P., Gwin, J., and Makeig, S. (2014). Imaging natural cognition in action. Int. J. Psychophysiol. 91, 22–29. doi: 10.1016/j.ijpsycho.2013.09.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Gramann, K., Gwin, J. T., Ferris, D. P., Oie, K., Jung, T. P., Lin, C. T., et al. (2011). Cognition in action: imaging brain/body dynamics in mobile humans. Rev. Neurosci. 22, 593–608. doi: 10.1515/RNS.2011.047

PubMed Abstract | CrossRef Full Text | Google Scholar

Halsband, U., and Lange, R. K. (2006). Motor learning in man: a review of functional and clinical studies. J. Physiol. Paris 99, 414–424. doi: 10.1016/j.jphysparis.2006.03.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Harrison, J., Izzetoglu, K., Ayaz, H., Willems, B., Hah, S., Ahlstrom, U., et al. (2014). Cognitive workload and learning assessment during the implementation of a next-generation air traffic control technology using functional near-infrared spectroscopy. Proc. IEEE Trans. Hum. Mach. Sys. 44, 429–440. doi: 10.1109/thms.2014.2319822

CrossRef Full Text | Google Scholar

Hatakenaka, M., Miyai, I., Mihara, M., Sakoda, S., and Kubota, K. (2007). Frontal regions involved in learning of motor skill- A functional NIRS study. Neuroimage 34, 109–116. doi: 10.1016/j.neuroimage.2006.08.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Holper, L., Kobashi, N., Kiper, D., Scholkmann, F., Wolf, M., and Eng, K. (2012). Trial-to-trial variability differentiates motor imagery during observation between low versus high responders: a functional near-infrared spectroscopy study. Behav. Brain Res. 229, 29–40. doi: 10.1016/j.bbr.2011.12.038

PubMed Abstract | CrossRef Full Text | Google Scholar

Holper, L., Muehlemann, T., Scholkmann, F., Eng, K., Kiper, D., and Wolf, M. (2010). Testing the potential of a virtual reality neurorehabilitation system during performance of observation, imagery and imitation of motor actions recorded by wireless functional near-infrared spectroscopy (fNIRS). J. Neuroeng. Rehabil. 7:57. doi: 10.1186/1743-0003-7-57

PubMed Abstract | CrossRef Full Text | Google Scholar

Holtzer, R., Mahoney, J. R., Izzetoglu, M., Izzetoglu, K., Onaral, B., and Verghese, J. (2011). fNIRS study of walking and walking while talking in young and old individuals. J. Gerontol. A Biol. Sci. Med. Sci. 66, 879–887. doi: 10.1093/gerona/glr068

PubMed Abstract | CrossRef Full Text | Google Scholar

Hoshi, E. (2013). Cortico-basal ganglia networks subserving goal-directed behavior mediated by conditional visuo-goal association. Front. Neural Circuits 7:158. doi: 10.3389/fncir.2013.00158

PubMed Abstract | CrossRef Full Text | Google Scholar

Hou, L., and Watanuki, K. (2012). Measurement of brain activity under virtual reality skills training using near-infrared spectroscopy. J. Adv. Mech. Des. Syst. 6, 168–178. doi: 10.1299/jamdsm.6.168

CrossRef Full Text | Google Scholar

Huppert, T., Schmidt, B., Beluk, N., Furman, J., and Sparto, P. (2013). Measurement of brain activation during an upright stepping reaction task using functional near-infrared spectroscopy. Hum. Brain Mapp. 34, 2817–2828. doi: 10.1002/hbm.22106

PubMed Abstract | CrossRef Full Text | Google Scholar

James, D. R. C., Orihuela-Espina, F., Leff, D. R., Sodergren, M. H., Athanasiou, T., Darzi, A. W., et al. (2011). The ergonomics of natural orifice transluminal endoscopic surgery (NOTES) navigation in terms of performance, stress and cognitive behavior. Surgery 149, 525–533. doi: 10.1016/j.surg.2010.11.019

PubMed Abstract | CrossRef Full Text | Google Scholar

Karim, H., Schmidt, B., Dart, D., Beluk, N., and Huppert, T. (2012). Functional near-infrared spectroscopy (fNIRS) of brain function during active balancing using a video game system. Gait Posture 35, 367–372. doi: 10.1016/j.gaitpost.2011.10.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Kearney, J. K., Rizzo, M., and Severson, J. (2007). “Virtual reality and neuroergonomics,” in Neuroergonomics: the Brain at Work, eds R. Parasuraman and M. Rizzo (New York, NY: Oxford University Press), 253–274.

Google Scholar

Kim, S., Biocca, F., Lee, K. M., Lee, S. W., and Chung, J. Y. (2015). “Neural correlates of embodied experiences through avatar self-identification,” in Proceeding of the 21st Annual Meeting of the Organization for Human Brain Mapping, Hawaii, HI.

Kita, Y., Gunji, A., Sakihara, K., Inagaki, M., Kaga, M., Nakagawa, E., et al. (2010). Scanning strategies do not modulate face identification: eye-tracking and near-infrared spectroscopy study. PLoS One 5:e11050. doi: 10.1371/journal.pone.0011050

PubMed Abstract | CrossRef Full Text | Google Scholar

Kober, S. E., Wood, G., and Neuper, C. (2013). “Measuring brain activation during spatial navigation in virtual reality: a combined EEG-NIRS study,” in Virtual Environments: Developments, Applications and Challenges, eds S. Trautman and F. Julien (New York, NY: NOVA Publisher), 1–24.

Google Scholar

Koenraadt, K. L., Roelofsen, E. G., Duysens, J., Keijsers, N. L., Kojima, T., and Tsunashima, H. (2014). Cortical control of normal gait and precision stepping: an fNIRS study. Neuroimage 85, 415–422. doi: 10.1016/j.neuroimage.2013.04.070

PubMed Abstract | CrossRef Full Text | Google Scholar

Kojima, T., Tsunashima, H., Shizawa, T., Takada, H., and Sakai, T. (2005). Measurement of train driver’s brain activity by functional near-infrared spectroscopy (fNIRS). Opt. Quant. Electron 37, 1319–1338. doi: 10.1007/s11082-005-4202-9

CrossRef Full Text

Leff, D. R., James, D. R. C., Orihuela-Espina, F., Kwok, K., Sun, L. W., Mylonas, G., et al. (2015). The impact of expert visual guidance on trainee visual search strategy, visual attention and motor skills. Front. Hum. Neurosci. 9:526. doi: 10.3389/fnhum.2015.00526

PubMed Abstract | CrossRef Full Text | Google Scholar

Leff, D. R., Orihuela-Espina, F., Elwell, C. E., Athanasiou, T., Delpy, D. T., Darzi, A. W., et al. (2011). Assessment of the cerebral cortex during motor task behaviours in adults: a systematic review of functional near infrared spectroscopy (fNIRS) studies. Neuroimage 54, 2922–2936. doi: 10.1016/j.neuroimage.2010.10.058

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, T., Pelowski, M., Pang, C., Zhou, Y., and Cai, J. (2015). Near-infrared spectroscopy as a tool for driving research. Ergonomics doi: 10.1080/00140139.2015.1076057 [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, Y., and Zhang, Y. (2015). “Toward welding robot with human knowledge: a remotely-controlled approach,” in Vol. 12, Proceedings of the IEEE Transaction on Automation Science and Engineering (Gothenburg: IEEE), 769–774.

Google Scholar

Mandrick, K., Derosière, G., Dray, G., Coulon, D., Micallef, J. P., and Perrey, S. (2013). Prefrontal cortex activity during motor tasks with additional mental load requiring attentional demand: a near-infrared spectroscopy study. Neurosci. Res. 76, 156–162. doi: 10.1016/j.neures.2013.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Matud, M. P. (2004). Gender differences in stress and coping styles. Pers. Individ. Dif. 37, 1401–1415. doi: 10.1016/j.paid.2004.01.010

CrossRef Full Text | Google Scholar

McKendrick, R., Parasuraman, R., and Ayaz, H. (2015). Wearable functional near infrared spectroscopy (fNIRS) and transcranial direct current stimulation (tDCS): expanding vistas for neurocognitive augmentation. Front. Syst. Neurosci. 9:27. doi: 10.3389/fnsys.2015.00027

PubMed Abstract | CrossRef Full Text | Google Scholar

Mehta, R. K., and Parasuraman, R. (2013). Neuroergonomics: a review of applications to physical and cognitive work. Front. Hum. Neurosci. 7:889. doi: 10.3389/fnhum.2013.00889

PubMed Abstract | CrossRef Full Text | Google Scholar

Mirelman, A., Maidan, I., Bernad-Elazari, H., Nieuwhof, F., Reelick, M., Giladi, N., et al. (2014). Increased frontal brain activation during walking while dual tasking: an fNIRS study in healthy young adults. J. Neuroeng. Rehabil. 11:85. doi: 10.1186/1743-0003-11-85

PubMed Abstract | CrossRef Full Text | Google Scholar

Miura, S., Kobayashi, Y., Kawamura, K., Nakashima, Y., and Fujie, M. G. (2015). Brain activation in parietal area during manipulation with a surgical robot simulator. Int. J. Comput. Assist. Radiol. Surg. 10, 783–790. doi: 10.1007/s11548-015-1178-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Molavi, B., and Dumont, G. A. (2012). Wavelet-based motion artifact removal for functional near-infrared spectroscopy. Physiol. Meas. 33, 259–270. doi: 10.1088/0967-3334/33/2/259

PubMed Abstract | CrossRef Full Text | Google Scholar

Nan, X., Zhang, Z., Zhang, N., Guo, F., He, Y., and Guan, L. (2014). vDesign: a CAVE-based virtual design environment using hand interactions. J. Multimodal User Interfaces 8, 367–379. doi: 10.1007/s12193-014-0168-x

CrossRef Full Text | Google Scholar

Naseer, N., and Hong, K. S. (2015). fNIRS-based brain-computer interfaces: a review. Front. Hum. Neurosci. 9:3. doi: 10.3389/fnhum.2015.00003

PubMed Abstract | CrossRef Full Text | Google Scholar

Noah, J. A., Ono, Y., Nomoto, Y., Shimada, S., Tachibana, A., Zhang, X., et al. (2015). fMRI validation of fNIRS measurements during a naturalistic task. J. Vis. Exp. 100:e52116. doi: 10.3791/52116

PubMed Abstract | CrossRef Full Text | Google Scholar

Okahashi, S., Mizumoto, H., Komae, A., Ueno, K., Yokoyama, M., Nagano, A., et al. (2014). An fNIRS-based study on prefrontal cortex activity during a virtual shopping test with different task difficulties in brain-damaged patients. J. Behav. Brain Sci. 4, 247–255. doi: 10.4236/jbbs.2014.46026

CrossRef Full Text | Google Scholar

Ono, Y., Noah, J. A., Zhang, X., Nomoto, Y., Suzuki, T., Shimada, S., et al. (2015). Motor learning and modulation of prefrontal cortex: an fNIRS assessment. J. Neural Eng. 12:066004. doi: 10.1088/1741-2560/12/6/066004 [Epub ahead of print].

CrossRef Full Text | Google Scholar

Parasuraman, R. (2011). Neuroergonomics: brain, cognition and performance at work. Curr. Dir. Psychol. Sci. 20, 181–186. doi: 10.1177/0963721411409176

CrossRef Full Text | Google Scholar

Parasuraman, R., and Rizzo, M. (2007). Neuroergonomics: The Brain at Work. New York, NY: Oxford University Press.

Google Scholar

Peck, E. M., Afergan, D., Yuksel, B. F., Lalooses, F., and Jacob, R. J. K. (2014). “Using fNIRS to measure mental workload in the real world,” in Advances in Physiological Computing, eds S. H. Fairclough and K. Gilleade (London: Springer-Verlag), 117–139.

Google Scholar

Sangani, S., Lamontagne, A., and Fung, J. (2015). Cortical mechanisms underlying sensorimotor enhancement promoted by walking with haptic inputs in a virtual environment. Progr. Brain Res. 218, 313–330. doi: 10.1016/bs.pbr.2014.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Scholkmann, F., Kleiser, S., Metz, A., Zimmermann, R., Mata Pavia, J., Wolf, U., et al. (2014). A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology. Neuroimage 85, 6–27. doi: 10.1016/j.neuroimage.2013.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Seraglia, B., Gamberini, L., Priftis, K., Scatturin, P., Martinelli, M., and Cutini, S. (2011). An exploratory fNIRS study with immersive virtual reality: a new method for technical implementation. Front. Hum. Neurosci. 5:176. doi: 10.3389/fnhum.2011.00176

PubMed Abstract | CrossRef Full Text | Google Scholar

Souza, M. J., Donohue, S. E., and Bunge, S. A. (2009). Controlled retrieval and selection of action-relevant knowledge mediated by partially overlapping regions in left ventrolateral prefrontal cortex. Neuroimage 46, 299–307. doi: 10.1016/j.neuroimage.2009.01.046

PubMed Abstract | CrossRef Full Text | Google Scholar

Strait, M., and Scheutz, M. (2014). What we can and cannot (yet) do with functional near infrared spectroscopy. Front. Neurosci. 8:117. doi: 10.3389/fnins.2014.00117

PubMed Abstract | CrossRef Full Text | Google Scholar

Sukal-Moulton, T., de Campos, A. C., Stanley, C. J., and Damiano, D. L. (2014). Functional near infrared spectroscopy of the sensory and motor brain regions with simultaneous kinematic and EMG monitoring during motor tasks. J. Vis. Exp. 94:e52391. doi: 10.3791/52391

PubMed Abstract | CrossRef Full Text | Google Scholar

Takakusaki, K. (2008). Forebrain control of locomotor behaviors. Brain Res. Rev. 57, 192–198. doi: 10.1016/j.brainresrev.2007.06.024

PubMed Abstract | CrossRef Full Text | Google Scholar

Takeuchi, Y. (2000). Change in blood volume in the brain during a simulated aircraft landing task. J. Occup. Health 42, 60–65. doi: 10.1539/joh.42.60

CrossRef Full Text | Google Scholar

Tanji, J., and Hoshi, E. (2008). Role of the lateral prefrontal cortex in executive behavioral control. Physiol. Rev. 88, 37–57. doi: 10.1152/physrev.00014.2007

PubMed Abstract | CrossRef Full Text | Google Scholar

Tomioka, H., Yamagata, B., Takahashi, T., Yano, M., Isomura, A. J., Kobayashi, H., et al. (2009). Detection of hypofrontality in drivers with Alzheimer’s disease by near-infrared spectroscopy. Neurosci. Lett. 451, 252–256. doi: 10.1016/j.neulet.2008.12.059

PubMed Abstract | CrossRef Full Text | Google Scholar

Tsunashima, H., and Yanagisawa, K. (2009). Measurement of brain function of car driver using functional near-infrared spectroscopy (fNIRS). Comput. Intell. Neurosci. 2009:164958. doi: 10.1155/2009/164958

PubMed Abstract | CrossRef Full Text | Google Scholar

Urakawa, S., Takamoto, K., Ishikawa, A., Ono, T., and Nishijo, H. (2015). Selective medial prefrontal cortex responses during live mutual gaze interactions in human infants: an fNIRS study. Brain Topogr. 28, 691–701. doi: 10.1007/s10548-014-0414-2

PubMed Abstract | CrossRef Full Text | Google Scholar

von Lühmann, A., Herff, C., Heger, D., and Schultz, T. (2015). Towards a wireless open source instrument: functional near-infrared spectroscopy in mobile neuroergonomics and BCI applications. Front. Hum. Neurosci. 9:617. doi: 10.3389/fnhum.2015.00617

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, L., Krasich, K., Bel-Bahar, T., Hughes, L., Mitroff, S. R., and Appelbaum, L. G. (2015). Mapping the structure of perceptual and visual-motor abilities in healthy young adults. Acta Psychol. (Amst) 157, 74–84. doi: 10.1016/j.actpsy.2015.02.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Wei, J. L., Sen, W. L., and Sani, Z. (2015). Leap motion underwater robotic arm control. J. Teknologi 74, 153–159. doi: 10.11113/jt.v74.4819

CrossRef Full Text | Google Scholar

Wriessnegger, S. C., Bauernfeind, G., Schweitzer, K., Kober, S., Neuper, C., and Müller-Putz, G. R. (2012). The interplay of prefrontal and sensorimotor cortices during inhibitory control of learned motor behavior. Front. Neuroeng. 5:17. doi: 10.3389/fneng.2012.00017

PubMed Abstract | CrossRef Full Text | Google Scholar

Yamagata, T., Nakayama, Y., Tanji, J., and Hoshi, E. (2012). Distinct information representation and processing for goal-directed behavior in the dorsolateral and ventrolateral prefrontal cortex and the dorsal premotor cortex. J. Neurosci. 32, 12934–12949. doi: 10.1523/JNEUROSCI.2398-12.2012

PubMed Abstract | CrossRef Full Text | Google Scholar

Young, M. S., Brookhuis, K. A., Wickens, C. D., and Hancock, P. A. (2015). State of science: mental workload in ergonomics. Ergonomics 58, 1–17. doi: 10.1080/00140139.2014.956151

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, Q., Ivkovic, V., Hu, G., and Strangman, G. E. (2014). Twenty-four-hour ambulatory recording of cerebral hemodynamics, systemic hemodynamics, electrocardiography and actigraphy during people’s daily activities. J. Biomed. Opt. 19:047003. doi: 10.1117/1.JBO.19.4.047003

PubMed Abstract | CrossRef Full Text | Google Scholar