Challenges and Opportunities in Consumer Neuroergonomics
Anne-Marie Brouwer- TNO The Netherlands Organisation for Applied Scientific Research, Soesterberg, Netherlands
Introduction
As described in the Field Grand Challenge article of this journal (Dehais et al., 2020a), Neuroergonomics is all about understanding the brain at work and in everyday life. Understanding the brain in everyday life is essential for clinical, psychological, and social neuroscience as fundamental research fields, as well as for harnessing neuroscience knowledge for applications, which has been at the core interest of this field from the beginning (Parasuraman, 2003).
In Consumer Neuroergonomics, we focus on the application—products and services that benefit the everyday consumer, in her or his professional capacity or in free time.
The field of Consumer Neuroergonomics can be understood in two ways. One concerns developing and validating neuroscientific consumer goods. For purchase and use by the everyday consumer, these goods need to be not only useful or fun, but also cheap, easy and comfortable to use. The other concerns the application of neuroscientific methodologies to study the user in her/his capacity of a consumer, examining user experience, product usability and product marketing.
With respect to Consumer Neuroergonomics goods, a broad range of neuroergonomic studies work toward their development. Examples are studies on predicting missing auditory alerts (Dehais et al., 2014), predicting memorized visual information (Brouwer et al., 2017b), detecting workload, fatigue and mind wandering (Borghini et al., 2014), usually with the aim of adapting semi-automated systems to better fit the current state of the user (Putze et al., 2018; Dehais et al., 2020b; Roy et al., 2020). Another example is monitoring group attention and engagement using wearable technology for possible use in educational settings (Dikker et al., 2017; Stuldreher et al., 2020; Van Beers et al., 2020). In parallel with scientific work toward developing neuroergonomic applications, an industry emerged providing products and services that (claim to) relate physiological measures to mental state, and that give advice or feedback based on these. Examples of products in this industry are wrist and headbands to monitor and reduce own stress levels, or gadgets to detect mood for entertaining purposes. A trending subfield in scientific studies and industry are tools to modulate brain activity directly through neurostimulation (Tyler et al., 2017; Vosskuhl et al., 2018).
Consumer Neuroergonomics as a discipline that uses neuroscience to study individuals in their capacity as consumers encompasses neuromarketing (Lee et al., 2007; Ariely and Berns, 2010; Stasi et al., 2018), neuroeconomics (Sanfey et al., 2006; Clithero et al., 2008), and consumer neuroscience (Yoon et al., 2012; Plassmann et al., 2015). This application area aims to better understand consumers and their interaction with products and services beyond the traditional self-report surveys and articulated responses from focus groups. Examples are studies on neuroscientific indicators of willingness to pay (Ramsøy et al., 2018) and purchasing behavior (Çakir et al., 2018); using neuroscience to investigate the role of emotions in decision-making (Rampl et al., 2016) and to evaluate advertising or marketing campaigns (Cartocci et al., 2017; Krampe et al., 2018). Similar to Consumer Neuroergonomics as a discipline that develops neuroscientific consumer goods, for Consumer Neuroergonomics as a discipline that studies the consumer, there is already a neuromarketing industry that commercializes this discipline.
Industry and commercial developments represent an essential element of Consumer Neuroergonomics, in that they directly target actual use, taking into account user comfort and user interests. On the other hand, Consumer Neuroergonomics as a science plays a role in the independent assessment of the validity of these products, and drive their improvement.
Below I present drivers that leverage Consumer Neuroergonomics and challenges that, within the broader field of Neuroergonomics, are quite specific for Consumer Neuroergonomic studies. These challenges relate to bridging the gap between science and application, to methodological issues, and to ethics.
Drivers
There is a great interest in applying information from (neuro)physiological signals. Compared to other research and technology domains, applied neuroscience gets a large share of attention in the media, from students and start-ups as well as larger companies. Three important drivers of this interest can be identified.
Firstly, there is a need in the outside world for accessible methods and technology that can probe mental state through (neuro)physiological sensing in everyday consumers. We ever more often interact with increasingly complex technology throughout the day, both in our personal and professional lives. While we can process a certain amount of information provided by technology, the bandwidth of information from us toward the technology is quite limited, prohibiting optimal adjustment from the technology to the user. Besides an increase in interaction with technology, spurring the need for information about the human to the system, globalization and centralization increases human-human interaction mediated through technology. Especially now during the COVID epidemic, real life meetings related to work or leisure are being replaced by virtual meetings. While this has large advantages e.g. from an environmental point of view, it also drastically reduces the continuous source of information about partners' mental states, such as interest or confusion, that is normally available. Neuroergonomics tools may help to reduce this problem. Another, different need for accessible methods and technology that can probe mental state, directly relates to Consumer Neuroergonomics as a discipline that studies the consumer. It is formulated by developers of products that have reached a level of maturity or quality such that usual tests and questionnaires can no longer predict market success. The hope is that alternative methods, probing mental state in a different way, still can (Köster and Mojet, 2015; Venkatraman et al., 2015; Kaneko et al., 2018). A final example of the need for technology that can monitor mental state is that of monitoring and predicting stress-related problems. Traditional interventions have been found to have positive, but only small effect sizes (Panagioti et al., 2017; Dreison et al., 2018)—neuroergonomic tools may support and expand these traditional approaches in designing timely and personalized interventions.
Besides a drive coming from the need, there is also the drive from technology. There is a huge development of relatively cheap, wearable sensors to measure brain signals, heart rate, and electrodermal activity, often together with different other types of information such as temperature and movement (Michard, 2017; Radüntz and Meffert, 2019; Stojanova et al., 2019; Witt et al., 2019; Masè et al., 2020; Sawangjai et al., 2020). Also, it is possible to extract physiological information from camera images such as heart rate, breathing as well as behavioral information like eye gaze, posture and facial expression (van der Kooij and Naber, 2019; Kong et al., 2021). This enables collecting information about an individual sitting at a laptop without any interference and without requiring any special equipment. These developments open up an unprecedented range of possibilities.
In alignment with technological developments, and fed by (sometimes oversimplified) stories in the popular media and by companies, the third driver is a strong belief in part of the general public that mental state or thoughts can currently be inferred with great accuracy and precision from neurophysiolocal signals. Wrist bands that indicate stress level, and emotion as indicated by lit up brain areas are seen as more “true” than what individuals report to feel. Neurophysiological measures and the addition of irrelevant neuroscience information can unjustifiably lend credence to conclusions and explanations (Canli and Amin, 2002; Farah, 2002; Weisberg et al., 2008; Howard-Jones, 2009). Here, the science of Consumer Neuroergonomics has an important role to play as also discussed below.
Challenges
Important current challenges in Neuroergonomics as a whole include improving wearable sensors, dealing with artifacts, dealing with limited spatiotemporal resolution neuroimaging, improving generalization across users and context and finding (other) ways to minimize the requirement of training or calibration data (Brouwer et al., 2015; Lotte et al., 2018; Dehais et al., 2020a). Clearly, meeting these challenges are essential for Consumer Neuroergonomics as a subfield as well.
The core of Neuroergonomics, but of Consumer Neuroergonomics in particular, is the integration between fundamental science and its application; between scientists, developers and users. The focus on the benefit of current, and near-future applications calls for close collaboration between (potential) users, engineers, designers and scientists; identification and exploration of suitable application areas; and solid validation where we could focus more on validation in terms of the beneficial effects that we are ultimately interested in. The immediate connection with potential large scale use by laymen, with recording from users rather than only laboratory participants, and dealing with technology that aims to detect mental state without having to ask but therewith also blurring the boundary of consent, also brings ethical challenges. These application-related and ethical challenges are outlined below.
Neuroergonomics research is multidisciplinary by nature, involving research fields ranging from machine learning to psychology. Bringing high-level expertise together on all these areas can be challenging and requires a good team. However, especially important for Consumer Neuroergonomics is the collaboration between scientists, developers, companies and consumers, as well as interdisciplinarity in expertise of different possible sources of information, besides the brain. Since Consumer Neuroergonomics is focused on applications in the relative short term, it is important to know the needs and wishes of the users, and the practical boundaries in the context of (envisioned) use. See for an example of how to implement this Derks et al. (2019), who describe the development of a biofeedback app using a User Centered Design framework and a cyclic developmental process involving the different user groups. With respect to practical boundaries in the context of use, multi-sensor brain measurements or collecting extensive amounts of calibration data may be prohibited due to the burden put on the user. Utilizing other physiological signals, as well as various kinds of information that can be obtained unobtrusively from e.g., a (web)camera, speech and movement can improve interpretation of other (brain) measures, as well as serve as the desired, multifaceted information itself (e.g., Brouwer et al., 2017a; Sargent et al., 2020). There is still a scarcity of studies that computationally combine different types of signals in another way than treating them as features in a model.
While this and other papers identified general areas of Consumer Neuroergonomics applications, e.g., enhancing man-machine teaming and evaluation of products without having to rely only on explicit measures, more specific cases could and should be explored where it is clear that the needs in the application area inherently match the strengths of Consumer Neuroergonomics. Promising application areas are those that clearly gain from continuous, implicit measures since easy alternatives (e.g., self-report or overt performance measures) are not available or usable, and that build on relatively easy interpretable signals. Also, such applications should be envisioned to be useful within the inherent limitations of limited accuracy, obtrusiveness of sensors, calibration data and noise in the context of use. Some examples in this respect include identification of attended speakers through around-the-ear EEG to improve hearing aids (Mirkovic et al., 2016), predicting aggressive outbursts of patients in mental health institutions based on physiology recorded through a wristband (De Looff et al., 2019), and predicting upcoming head rotation through EEG to improve video streaming in Head Mounted Displays (Brouwer et al., 2018). Another promising and timely area may be in awareness training on implicit biased judgement of others, or, as discussed in the beginning, recovering social cues that are lost when moving from real to virtual meetings. The latter may range from indicating where a meeting's attendee is gazing at, to monitoring shared attention through interindividual synchrony in heart rate as recorded through the webcam (cf. Stuldreher et al., 2020). In the context of evaluating consumers' mental state when interacting with systems or products, Consumer Neuroergonomics is especially relevant when mental state is studied over the course of interaction and repeated questioning would influence the mental state of interest itself (Ayaz et al., 2012; Shewokis et al., 2017; Brouwer et al., 2019), or when systematic response biases are expected, e.g., because consumers under study systematically differ in cultural background (Torrico et al., 2018, 2019; Kaneko et al., in press).
With developing applications comes testing and validation. For applications that monitor mental state, e.g., stress, this is usually done by comparing the mental state as estimated by the application to the state as reported by the individual, or by comparing the estimates between different conditions that are assumed to differentially influence mental state of interest. However, for many (envisioned) applications we are not interested in estimating a certain mental state in itself, but in a means to change individuals' behavior, improve individuals' performance or general well-being. We want to do this beyond what is possible with current methods, or in an easier way than currently used methods. Thus, in order to know whether an application works, it is important to connect results as directly as possible to what we are ultimately interested in. From the research perspective, this can be considered as a blessing and a curse. On the one hand, this approach stays clear of the issue of defining mental states, of which it is often unsure how or whether they can be mapped on certain physiological signals (Cacioppo and Tassinary, 1990), and that are muddled and confounded in real life situations. Additionally, it is difficult to retrieve independent, “ground truth” measures of these mental states to train models and validate results, especially when the motivation of the neuroergonomic research or application is to circumvent the use of self-report measures (Brouwer et al., 2020). Finally, for many types of mental states, it is unknown what the “appropriate level” is, i.e., the level corresponding to high performance and high well-being. On the other hand, focusing on performance measures of (ultimate) interest rather than mental state, is difficult since these can be hard to obtain (e.g., success in the marketplace, or pilot's noticing a very rare combination of events that can cause a fatal air traffic accident). Obtaining self-reports on the mental state that is believed to be related to the ultimate goal is easy to do.
The vicinity of Consumer Neuroergonomics to large scale use outside the scientific or clinical sphere generates imminent ethical challenges. One class of ethical challenges lies in a range of issues that comes with monitoring of individuals out of the context of ethically approved experiments, where the aim of the monitoring is to extract information from individuals without them being (continuously) aware of this. While the purpose of this is to not burden the user, or to derive information that is beneficial but otherwise is difficult to obtain, this brings the requirement to treat these private data with care and with consent of the individuals monitored. Ienca et al. (2018) express their concern in this respect and propose safeguarding legal regulations. They also argue for legislation for non-invasive neurostimulation devices, that pose special health risks to the user. Mecacci and Haselager (2019) propose a framework, or a list of criteria, along which ethicists, policy makers and other stakeholders can systematically evaluate (upcoming) “brain reading” technology from an ethical perspective. Criteria include weighing the relevance of the information for the purpose at hand (do we really need this information), and whether the method could be used unknown to the individual. When developing neuroergonomic products, criteria such as these should be taken into account, and possible problems alleviated, from the start.
Another class of ethical challenges lies in the communication with the public and maintenance of high scientific standards. As Neuroergonomics scientists, we should take care that shortened explanations and interpretations of our results do not mislead the general public. This is an especially important role in Consumer Neuroergonomics, where the validity of certain neuroergonomic tools and services cannot always readily be known or judged, and neurotechnology companies may oversell their products (Wexler and Thibault, 2018). Wexler and colleagues argue that this class of ethical issues is the most prominent: “.. the problem with consumer EEG devices is not that the data they gather are rich, accurate, and revealing; the problem is that consumer EEG companies are misleading consumers into thinking it is revealing” (Wexler, 2019).
It is clear that Consumer Neuroergonomics scientists have an important role to play in assisting commercial parties, professionals and consumers in interpreting findings and products related to Consumer Neuroergonomics, both for outlining its possibilities and opportunities to enhance performance and well-being, as well as its (current) limitations, e.g., with respect to generalization across situations. Helping interpret findings and products is also essential to contribute and to shape the public discussion on the possible dangers associated with ethical concerns concerning neuroergonomic applications, ranging from privacy issues to possible consequences of overreliance on neuroergonomic applications. We should note that high scientific standards do not imply that research has to follow common practice as originated from traditional laboratory research. It is, especially in this field, important to go beyond the strictly controlled laboratory experiments, and to explore the sensitivity of the various innovative sensors and techniques under multiple types of daily life conditions. However, we should be transparent on the downsides this may bring, discuss, and examine possible influence of sensor noise and confounds, and be careful in interpreting the meaning of the findings, especially in terms of what this means for applications. Rayatdoost et al. (2020a,b) follow an approach that is sound from an applied, scientific and ethical view. They investigate classification of emotion based on signals recorded by EEG sensors, show and discuss that the major part of classification success can be attributed to muscle artifacts, and subsequently use these ‘artifacts' as information to improve their classification. They thus show that EEG sensors can be useful to classify emotions (especially in cases that the face is hidden under a Head Mounted Display), but do not claim that this information is obtained from the brain.
Final Remarks
In sum, Consumer Neuroergonomics can flourish through benefits that Neuroergonomics can bring, through a push from technology and interest of the public. Studies and products that would propel Consumer Neuroergonomics involve potential users and expertise from different fields; they harness, if needed, multiple modalities; they are targeted to solve a well-circumscribed problem that is hard to solve in other (cheaper or simpler) ways; they show that the neuroergonomic product or approach actually benefits the user; they respect the user's safety and privacy; and they are transparent on the functioning and limitations of the product or approach.
With this paper, I hope to have acknowledged existing challenges and at the same time illustrated the opportunities in this exciting new field. Opportunities, open questions and enthusiasm are also manifested in the Frontiers in Consumer Neuroergonomics Research Topics that are already in place. There is no doubt that the establishment of Frontiers Neuroergonomics will help to bring the field forward.
Author Contributions
The author confirms being the sole contributor of this work and has approved it for publication.
Funding
A-MB was supported by the TNO Early Research Program Body Brain Interactions.
Conflict of Interest
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Ariely, D., and Berns, G. S. (2010). Neuromarketing: the hope and hype of neuroimaging in business. Nat. Rev. Neurosci. 11, 284–292. doi: 10.1038/nrn2795
Ayaz, H., Shewokis, P. A., Bunce, S., Izzetoglu, K., Willems, B., and Onaral, B. (2012). Optical brain monitoring for operator training and mental workload assessment. Neuroimage 59, 36–47. doi: 10.1016/j.neuroimage.2011.06.023
Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., and Babiloni, F. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci. Biobehav. Rev. 44, 58–75. doi: 10.1016/j.neubiorev.2012.10.003
Brouwer, A.-M., Hogervorst, M. A., Grootjen, M., van Erp, J. B. F., and Zandstra, E. H. (2017a). Neurophysiological responses during cooking food associated with different emotions. Food Q. Prefer. 62, 307–316. doi: 10.1016/j.foodqual.2017.03.005
Brouwer, A.-M., Hogervorst, M. A., Oudejans, B., Ries, A. J., and Touryan, J. (2017b). EEG and eye tracking signatures of target encoding during structured visual search. Front. Hum. Neurosci. 11:264. doi: 10.3389/fnhum.2017.00264
Brouwer, A.-M., Hogervorst, M. A., van Erp, J. B. F., Grootjen, M., van Dam, E., and Zandstra, E. H. (2019). Measuring cooking experience implicitly and explicitly: Physiology, facial expression and subjective ratings. Food Qual. Prefer. 78:103726. doi: 10.1016/j.foodqual.2019.103726
Brouwer, A.-M., van den Broek, T., Hogervorst, M. A., Kaneko, D., Toet, A., Kallen, V., et al. (2020). Estimating affective taste experience using combined implicit behavioral and neurophysiological measures. IEEE Trans. Affect. Comp. 99:1. doi: 10.1109/TAFFC.2020.3032236
Brouwer, A.-M., van der Waa, J., and Stokking, H. (2018). BCI to potentially enhance streaming images to a VR headset by predicting head rotation. Front. Hum. Neurosci. 12:420. doi: 10.3389/fnhum.2018.00420
Brouwer, A.-M., Zander, T. O., van Erp, J. B. F., Korteling, J. E., and Bronkhorst, A. W. (2015). Using neurophysiological signals that reflect cognitive or affective state: six recommendations to avoid common pitfalls. Front. Neurosci. 9:136. doi: 10.3389/fnins.2015.00136
Cacioppo, J. T., and Tassinary, L. G. (1990). Inferring psychological significance from physiological signals. Am. Psychol. 45,16–28. doi: 10.1037/0003-066X.45.1.16
Çakir, M. P., Çakar, T., Girisken, Y., and Yurdakul, D. (2018). An investigation of the neural correlates of purchase behavior through fNIRS. Eur. J. Market. 52, 224–243. doi: 10.1108/EJM-12-2016-0864
Canli, T., and Amin, Z. (2002). Neuroimaging of emotion and personality: scientific evidence and ethical considerations. Brain Cogn. 50, 414–431. doi: 10.1016/S0278-2626(02)00517-1
Cartocci, G., Caratù, M., Modica, E., Maglione, A. G., Rossi, D., Cherubino, P., et al. (2017). Electroencephalographic, heart rate, and galvanic skin response assessment for an advertising perception study: application to antismoking public service announcements. J. Vis. Exp. 28:55872. doi: 10.3791/55872
Clithero, J. A., Tankersley, D., and Huettel, S. A. (2008). Foundations of neuroeconomics: from philosophy to practice. PLoS Biol. 6:e298. doi: 10.1371/journal.pbio.0060298
De Looff, P., Noordzij, M. L., Moerbeek, M., Nijman, H., Didden, R., and Embregts, P. (2019). Changes in heart rate and skin conductance in the 30 min preceding aggressive behavior. Psychophysiology 56:e13420. doi: 10.1111/psyp.13420
Dehais, F., Causse, M., Vachon, F., Régis, N., Menant, E., and Tremblay, S. (2014). Failure to detect critical auditory alerts in the cockpit: evidence for inattentional deafness. Hum. Fact. 56, 631–644. doi: 10.1177/0018720813510735
Dehais, F., Karwowski, W., and Ayaz, H. (2020a). Brain at work and in everyday life as the next frontier: grand field challenges for neuroergonomics. Front. Neuroergonomics 1:58373., doi: 10.3389/fnrgo.2020.583733
Dehais, F., Lafont, A., Roy, R., and Fairclough, S. (2020b). A neuroergonomics approach to mental workload, engagement and human performance. Front. Neurosci. 14:268. doi: 10.3389/fnins.2020.00268
Derks, Y. P., Klaassen, R., Westerhof, G. J., Bohlmeijer, E. T., and Noordzij, M. L. (2019). Development of an ambulatory biofeedback app to enhance emotional awareness in patients with borderline personality disorder: multicycle usability testing study. JMIR Mhealth Uhealth 7:e13479 doi: 10.2196/13479
Dikker, S., Wan, L., Davidesco, I., Kaggen, L., Oostrik, M., McClintock, J., et al. (2017). Brain-to-brain synchrony tracks real-world dynamic group interactions in the classroom. Curr. Biol. 27, 1375–1380. doi: 10.1016/j.cub.2017.04.002
Dreison, K. C., Luther, L., Bonfils, K. A., Sliter, M. T., McGrew, J. H., and Salyers, M. P. (2018). Job burnout in mental health providers: a meta-analysis of 35 years of intervention research. J. Occup. Health Psychol. 23, 18–30. doi: 10.1037/ocp0000047
Farah, M. J. (2002). Emerging ethical issues in neuroscience. Nat. Neurosci. 5, 1123–1129. doi: 10.1038/nn1102-1123
Howard-Jones, P. A. (2009). Scepticism is not enough. Cortex 45, 550–551. doi: 10.1016/j.cortex.2008.06.002
Ienca, M., Haselager, P., and Emanuel, E. J. (2018). Brain leaks and consumer neurotechnology. Nat. Biotechnol. 36, 805–810. doi: 10.1038/nbt.4240
Kaneko, D., Stuldreher, I. V., Reuten, A., Toet, A., van Erp, J. B. F., and Brouwer, A.-M. (in press). Comparing explicit implicit measures for assessing cross-cultural food experience. Front. Neuroergon.
Kaneko, D., Toet, A., Brouwer, A.-M., Kallen, V., and van Erp, J. B. F. (2018). Methods for evaluating emotions evoked by food experiences: a literature review. Front. Psychol. 9:911. doi: 10.3389/fpsyg.2018.00911
Kong, Y., Posada-Quintero, H. F., Daley, M. S., Chon, K. H., and Bolkhovsky, J. (2021). Facial features and head movements obtained with a webcam correlate with performance deterioration during prolonged wakefulness. Atten. Percept. Psychophys. 83, 525–540. doi: 10.3758/s13414-020-02199-5
Köster, E. P., and Mojet, J. (2015). From mood to food and from food to mood: a psychological perspective on the measurement of food-related emotions in consumer research. Food Res. Int. 76, 180–191. doi: 10.1016/j.foodres.2015.04.006
Krampe, C., Gier, N. R., and Kenning, P. (2018). The application of mobile fnirs in marketing research—detecting the “first-choice-brand” effect. Front. Hum. Neurosci. 12:433. doi: 10.3389/fnhum.2018.00433
Lee, N., Broderick, A. J., and Chamberlain, L. (2007). What is ‘neuromarketing'? A discussion and agenda for future research. Int. J. Psychophys. 63, 199–204. doi: 10.1016/j.ijpsycho.2006.03.007
Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., et al. (2018). A review of classification algorithms for EEGbased brain–computer interfaces: a 10 year update. J. Neural. Eng. 15:031005. doi: 10.1088/1741-2552/aab2f2
Masè, M., Micarelli, A., and Strapazzon, G. (2020). Hearables: new perspectives and pitfalls of in-ear devices for physiological monitoring. A scoping review. Front. Physiol. 11:568886. doi: 10.3389/fphys.2020.568886
Mecacci, G., and Haselager, P. (2019). Identifying criteria for the evaluation of the implications of brain reading for mental privacy. Sci. Eng. Ethics 25, 443–461. doi: 10.1007/s11948-017-0003-3
Michard, F. (2017). A sneak peek into digital innovations and wearable sensors for cardiac monitoring. J. Clin. Monit. Comput. 31, 253–259. doi: 10.1007/s10877-016-9925-6
Mirkovic, B., Bleichner, M. G., De Vos, M., and Debener, S. (2016). Target speaker detection with concealed EEG and around the ear. Front. Neurosci. 10:349. doi: 10.3389/fnins.2016.00349
Panagioti, M., Panagopoulou, E., Bower, P., Lewith, G., Kontopantelis, E., Chew-Graham, C., et al. (2017). Controlled interventions to reduce burnout in physicians: a systematic review and meta-analysis. JAMA Intern. Med. 177:195–205. doi: 10.1001/jamainternmed.2016.7674
Parasuraman, R. (2003). Neuroergonomics: research and practice. Theoret. Issues Ergon. Sci. 4, 5–20. doi: 10.1080/14639220210199753
Plassmann, H., Venkatraman, V., Huettel, S., and Yoon, C. (2015). Consumer neuroscience: applications, challenges, and possible solutions. J. Market. Res. 52, 427–435. doi: 10.1509/jmr.14.0048
Putze, F., Mühl, C., Lotte, F., Fairclough, S., and Herff, C. (2018). Editorial: detection and estimation of working memory states and cognitive functions based on neurophysiological measures. Front. Hum. Neurosci. 12:440. doi: 10.3389/fnhum.2018.00440
Radüntz, T., and Meffert, B. (2019). User Experience of 7 mobile electroencephalography devices: comparative study. JMIR Mhealth Uhealth 7:e14474. doi: 10.2196/14474
Rampl, L. V., Opitz, C., Welpe, I. M., and Kenning, P. (2016). The role of emotions in decision-making on employer brands: insights from functional magnetic resonance imaging (fMRI). Mark. Lett. 27, 361–374. doi: 10.1007/s11002-014-9335-9
Ramsøy, T. Z., Skov, M., Christensen, M. K., and Stahlhut, C. (2018). Frontal brain asymmetry and willingness to pay. Front. Neurosci. 12:138. doi: 10.3389/fnins.2018.00138
Rayatdoost, S., Radrauf, D., and Soleymani, M. (2020b). “Multimodal gated information fusion for emotion recognition from EEG signals and facial behaviors,” in Proceedings of the 22nd ACM International Conference on Multimodal Interaction, ICMI'20 (New York, NY: ACM) doi: 10.1145/3382507.3418867
Rayatdoost, S., Rudrauf, D., and Soleymani, M. (2020a). “Expression-guided EEG representation learning for emotion recognition,” in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (Barcelona), 3222–3226. doi: 10.1109/ICASSP40776.2020.9053004
Roy, R. N., Drougard, N., Gateau, T., Dehais, F., and Chanel, C. P. C. (2020). How can physiological computing benefit human-robot interaction? Robotics 9:100. doi: 10.3390/robotics9040100
Sanfey, A. G., Loewenstein, G., McClure, S. M., and Cohen, J. D. (2006). Neuroeconomics: cross-currents in research on decision-making. Trends Cogn. Sci. 10: 108–116. doi: 10.1016/j.tics.2006.01.009
Sargent, A., Watson, J., Ye, H., Suri, R., and Ayaz, H. (2020). Neuroergonomic assessment of hot beverage preparation and consumption: an EEG, and EDA study. Front. Hum. Neurosci. 14:175. doi: 10.3389/fnhum.2020.00175
Sawangjai, P., Hompoonsup, S., Leelaarporn, P., Kongwudhikunakorn, S., and Wilaiprasitporn, T. (2020). Consumer grade EEG measuring sensors as research tools: a review. IEEE Sens. J. 20, 3996–4024. doi: 10.1109/JSEN.2019.2962874
Shewokis, P. A., Shariff, F. U., Liu, Y., Ayaz, H., Castellanos, A., and Lind, D. S. (2017). Acquisition, retention and transfer of simulated laparoscopic tasks using fNIR and a contextual interference paradigm. Am. J. Surg. 213, 336–345. doi: 10.1016/j.amjsurg.2016.11.043
Stasi, A., Songa, G., Mauri, M., Ciceri, A., Diotallevi, F., Nardone, G., et al. (2018). Neuromarketing empirical approaches and food choice: a systematic review. Food Res. Int. 108, 650–664. doi: 10.1016/j.foodres.2017.11.049
Stojanova, A., Koceski, S., and Koceska, N. (2019). Continuous blood pressure monitoring as a basis for ambient assisted living (AAL) – review of methodologies and devices. J. Med. Syst. 43:24. doi: 10.1007/s10916-018-1138-8
Stuldreher, I., Thammasan, N., van Erp, J. B. F., and Brouwer, A. M. (2020). Physiological synchrony in EEG, electrodermal activity and heart rate reflects shared selective auditory attention. J. Neural Eng. 17:046028. doi: 10.1088/1741-2552/aba87d
Torrico, D. D., Fuentes, S., Viejo, C. G., Ashman, H., and Dunshea, F. R. (2019). Cross-cultural effects of food product familiarity on sensory acceptability and non-invasive physiological responses of consumers. Food Res. Int. 115, 439–450. doi: 10.1016/j.foodres.2018.10.054
Torrico, D. D., Fuentes, S., Viejo, C. G., Ashman, H., Gunaratne, N. M., Gunaratne, T. M., et al. (2018). Images and chocolate stimuli affect physiological and affective responses of consumers: a cross-cultural study. Food Qual. Prefer 65, 60–71. doi: 10.1016/j.foodqual.2017.11.010
Tyler, W. J., Sanguinetti, J. L., Fini, M., and Hool, N. (2017). “Non-invasive neural stimulation,” in Proc. SPIE 10194, Micro- and Nanotechnology Sensors, Systems, and Applications IX, 101941L.
Van Beers, J. J., Stuldreher, I. V., Thammasan, N., and Brouwer, A.-M. (2020). “A comparison between laboratory and wearable sensors in the context of physiological synchrony,” ICMI ‘20: Proceedings of the 2020 International Conference on Multimodal Interaction, 604–608. doi: 10.1145/3382507.3418837
van der Kooij, K. M., and Naber, M. (2019). An open-source remote heart rate imaging method with practical apparatus and algorithms. Behav. Res. 51, 2106–2119. doi: 10.3758/s13428-019-01256-8
Venkatraman, V., Dimoka, A., Pavlou, P. A., Vo, K., Hampton, W., Bollinger, B., et al. (2015). Predicting advertising success beyond traditional measures: new insights from neurophysiological methods and market response modeling. J. Market. Res. 52, 436–452. doi: 10.1509/jmr.13.0593
Vosskuhl, J., Strüber, D., and Herrmann, C. S. (2018). Non-invasive brain stimulation: a paradigm shift in understanding brain oscillations. Front. Hum. Neurosci. 12:211. doi: 10.3389/fnhum.2018.00211
Weisberg, D. S., Keil, F. C., Goodstein, J., Rawson, E., and Gray, J. R. (2008). The seductive allure of neuroscience explanations. J. Cogn. Neurosci. 20, 470–477. doi: 10.1162/jocn.2008.20040
Wexler, A. (2019). Separating neuroethics from neurohype. Nat. Biotechnol. 37, 988–990. doi: 10.1038/s41587-019-0230-z
Wexler, A., and Thibault, R. J. (2018). Mind-reading or misleading? assessing direct-to-consumer electroencephalography (EEG) devices marketed for wellness and their ethical and regulatory implications. Cogn. Enhanc. 3, 131–137. doi: 10.1007/s41465-018-0091-2
Witt, D., Kellogg, R., Snyder, M., and Dunn, J. (2019). Windows into human health through wearables data analytics. Curr. Opin. Biomed. Eng. 9, 28–46. doi: 10.1016/j.cobme.2019.01.001
Keywords: neuroergonomics, psychophysiology, affective computing, brain-computer interfaces, human-machine systems, mental state monitoring, wearables
Citation: Brouwer A-M (2021) Challenges and Opportunities in Consumer Neuroergonomics. Front. Neuroergon. 2:606646. doi: 10.3389/fnrgo.2021.606646
Received: 15 September 2020; Accepted: 08 February 2021;
Published: 05 March 2021.
Edited and reviewed by: Hasan Ayaz, Drexel University, United States
Copyright © 2021 Brouwer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Anne-Marie Brouwer, anne-marie.brouwer@tno.nl