Event Abstract

Back to Event

Cognitive and Affective Probing for Neuroergonomics

Laurens R. Krol1* and Thorsten O. Zander1, 2
  • 1 Technische Universität Berlin, Biological Psychology and Neuroergonomics, Germany
  • 2 Zander Laboratories B.V., Netherlands

INTRODUCTION With increasingly portable and user-friendly electroencephalography (EEG) systems, this brain monitoring technique continues its path towards commercial applications (Mullen et al., 2015; Zander et al., 2017). Research into passive brain-computer interfacing (pBCI; Zander & Kothe, 2011) has focused on EEG to enable human-computer interaction (HCI) to make use of implicit information—information that is not explicitly communicated by the operator, but is instead obtained from brain activity (Zander, Brönstrup, Lorenz, & Krol, 2014). For example, a cognitive workload index can be established based on reference recordings of an operator's brain activity (Gevins & Smith, 2003). This can then be used in the form of a pBCI to automatically adapt automation levels (Parasuraman, Mouloua, & Hilburn, 1999). pBCI-based adaptive automation is an example of a closed-loop system (Krol, Andreessen, & Zander, 2018). Access to an operator's brain state, however, allows a computer system more flexibility than that. Through cognitive probing, a system can autonomously learn specific pieces of information, independent of pre-defined loops. This paper is part of an effort to discuss the wider implications of cognitive probing in a number of different disciplines. It overlaps with and continues a paper presented earlier (Krol & Zander, in press). COGNITIVE PROBING We proposed earlier (Krol & Zander, in press) to define cognitive probing as utilising cognitive probes. A cognitive probe is a single autogenous system adaptation that is initiated or co-opted by that system in order to learn from the user's contextual, cognitive brain response to it. 'System adaptation' refers to anything the computer does, be it stimulus presentation, feedback, response to input, etc. Cognitive probing involves measuring the operator's responses to computer actions and contextual factors in such a way that a causal effect can be determined. Importantly, cognitive probes are computer actions that are autonomously controlled by the computer for the purpose of learning such effects. When it is learned what actions cause what cognitive responses, the computer can act to support the operator. The proposed definition encompasses a number of different methods already in use. For example, a secondary oddball task to infer workload (Kohlmorgen et al., 2007) is a form of cognitive probing: the computer purposefully plays sounds in order to learn from the operator's response to those sounds. The framework highlights a number of aspects of cognitive probing. One aspect concerns the probe's intrusiveness. Whereas an oddball paradigm must necessarily interfere with the operator's tasks, probes can also be embedded unobtrusively. For example, a computer may learn that whenever certain 'naturally' (Krol & Zander, 2017) occurring notifications pop up, workload increases. It could then suppress such notifications when workload levels are already high. Of course, this particular rule has been suggested before (Kirchner et al., 2016). Cognitive probing however would grant a computer the autonomy to learn such a rule by itself—or, indeed, any other rule. Cognitive probing enables the system to learn each operator's individual cognitive properties based on their responses to any number of probes. AFFECTIVE PROBING The proposed definition focuses on cognition, but can easily be extended to include affect. For example, Zander, Krol, Birbaumer, and Gramann (2016) gave a speculative example of a neuroadaptive book. In this book, the events that unfold as part of the written story are co-opted as probes: the system monitors the readers' brain activity, and registers their implicit responses to specific passages. It may thus learn, for example, which character is the reader's favourite. It can then adapt itself by re-writing subsequent passages to manipulate the reader's experience of the book. Affect influences performance in the workplace (Brief & Weiss, 2002). For example, many potential sources of anger have been identified in the workplace (Schieman, 2010), which, along with many other negative emotions, often lead to counterproductive work behaviour (Spector & Fox, 2005). Thus, HCI may benefit when the computer autonomously can probe, interpret, and adapt to the operator's affective state. Systems capable of both cognitive and affective probing would be able to construct a more complete model of the operator's preferences. DISCUSSION Probes can be 'hidden', and the operators may not be capable of suppressing their responses to them. This makes cognitive probing a highly privacy-sensitive issue. We have argued earlier that strict ethical guidelines must be adhered to, that data ownership must remain with the operator, and that full disclosure and informed consent are necessary (Krol & Zander, in press). Ethical issues may be even more sensitive in the workplace. Managers prescribe what hard- and software must be used. Should they be allowed to prescribe systems that utilise cognitive probes, and build models of their users' needs, preferences, and intentions? We argue that, at least, employees must remain in control of all access to the information derived from probes. This includes momentary responses to single probes, as well as general models derived form larger numbers of probes. We believe regulation must be in place to prevent anyone from obtaining unauthorised access to these cognitive profiles before any cognitive probes are used in professional settings. Affective probing means that computers must be able to influence their operator's emotions. In private settings, such manipulation may be desirable—horror movies, for example, provide an example of adults deliberately choosing to experience negative affect. Similarly, neuroadaptive emotional manipulation may be desirable in some contexts. However, aside from the earlier- mentioned ethical considerations, we at least suggest the need for additional mechanisms to avoid positive feedback loops leading to extreme emotions. Looking specifically at the workplace, furthermore, we believe an open debate must be held concerning whether or not neuroadaptive emotional manipulation is desirable at all, even when the above-mentioned requirements are met. Cognitive and affective probing may lead to uniquely personalised, cooperative systems, and thus increased productivity and satisfaction. However, there are a number of ethical issues that must be considered now—before the first such systems are developed. Where people may largely decide for themselves in their private lives, a neuroadaptive workplace requires additional considerations ahead of time.

Acknowledgements

Part of this work was supported by the Deutsche Forschungsgemeinschaft (ZA 821/3-1).

References

Brief, A. P., & Weiss, H. M. (2002). Organizational behavior: Affect in the workplace. Annual Review of Psychology, 53 (1), 279–307. doi: 10.1146/annurev.psych.53.100901.135156
Gevins, A., & Smith, M. E. (2003). Neurophysiological measures of cognitive workload during human-computer interaction. Theoretical Issues in Ergonomics Science, 4 (1–2), 113–131. doi: 10.1080/14639220210159717
Kirchner, E. A., Kim, S. K., Tabie, M., Wöhrle, H., Maurus, M., & Kirchner, F. (2016). An intelligent man-machine interface—multi-robot control adapted for task engagement based on single-trial detectability of P300. Frontiers in Human Neuroscience, 10, 291. doi: 10.3389/fnhum.2016.00291
Kohlmorgen, J., Dornhege, G., Braun, M., Blankertz, B., Curio, G., Hagemann, K., . . . Kincses, W. (2007). Improving human performance in a real operating environment through real-time mental workload detection. In G. Dornhege (Ed.), Toward Brain-Computer Interfacing (pp. 409–422). Cambridge, MA, USA: MIT Press.
Krol, L. R., Andreessen, L. M., & Zander, T. O. (2018). Passive Brain-Computer Interfaces: A Perspective on Increased Interactivity. In C. S. Nam, A. Nijholt, & F. Lotte (Eds.), Brain-Computer Interfaces Handbook: Technological and Theoretical Advances (pp. 69–86). Boca Raton, FL, USA: CRC Press.
Krol, L. R., & Zander, T. O. (2017). Passive BCI-based neuroadaptive systems. In Proceedings of the 7th Graz brain-computer interface conference 2017 (pp. 248–253). doi: 10.3217/978-3-85125-533-1-46
Krol, L. R., & Zander, T. O. (in press). Towards a conceptual framework for cognitive probing. In Symbiotic Interaction, Lecture Notes in Computer Science. Cham, Switzerland: Springer.
Mullen, T. R., Kothe, C. A. E., Chi, Y. M., Ojeda, A., Kerth, T., Makeig, S., . . . Cauwenberghs, G. (2015). Real-time neuroimaging and cognitive monitoring using wearable dry EEG. IEEE Transactions on Biomedical Engineering, 62 (11), 2553–2567. doi: 10.1109/TBME.2015.2481482
Parasuraman, R., Mouloua, M., & Hilburn, B. (1999). Adaptive aiding and adaptive task allocation enhance human-machine interaction. Automation technology and human performance: Current research and trends, 119–123.
Schieman, S. (2010). The sociological study of anger: Basic social patterns and contexts. In M. Potegal, G. Stemmler, & C. Spielberger (Eds.), International handbook of anger: Constituent and concomitant biological, psychological, and social processes (pp. 329–347). New York, NY, USA: Springer. doi: 10.1007/978-0-387-89676-2 19
Spector, P. E., & Fox, S. (2005). The stressor-emotion model of counterproductive work behavior. In S. Fox & P. E. Spector (Eds.), Counterproductive work behavior: Investigations of actors and targets (pp. 151–174). Washington, DC, USA: American Psychological Association. doi: 10.1037/10893-007
Zander, T. O., Andreessen, L. M., Berg, A., Bleuel, M., Pawlitzki, J., Zawallich, L., Krol, L. R. & Gramann, K. (2017). Evaluation of a dry EEG system for application of passive brain-computer interfaces in autonomous driving. Frontiers in Human Neuroscience, 11, 78. doi: 10.3389/fnhum.2017.00078
Zander, T. O., Brönstrup, J., Lorenz, R., & Krol, L. R. (2014). Towards BCI-based Implicit Control in Human-Computer Interaction. In S. H. Fairclough & K. Gilleade (Eds.), Advances in Physiological Computing (pp. 67–90). Berlin, Germany: Springer. doi: 10.1007/978-1-4471-6392-3_4
Zander, T. O., & Kothe, C. A. (2011). Towards passive brain-computer interfaces: applying brain-computer interface technology to human-machine systems in general. Journal of Neural Engineering, 8 (2), 025005. doi: 10.1088/1741-2560/8/2/025005
Zander, T. O., Krol, L. R., Birbaumer, N. P., & Gramann, K. (2016). Neuroadaptive technology enables implicit cursor control based on medial prefrontal cortex activity. Proceedings of the National Academy of Sciences, 113 (52), 14898–14903. doi: 10.1073/pnas.1605155114

Keywords: Passive Brain-Computer Interface, neuroadaptive technology, neuroergonomics, cognitive probing, affective probing, Neuroethics

Conference: 2nd International Neuroergonomics Conference, Philadelphia, PA, United States, 27 Jun - 29 Jun, 2018.

Presentation Type: Oral Presentation

Topic: Neuroergonomics

Citation: Krol LR and Zander TO (2019). Cognitive and Affective Probing for Neuroergonomics. Conference Abstract: 2nd International Neuroergonomics Conference. doi: 10.3389/conf.fnhum.2018.227.00087

Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.

The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.

Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.

For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.

Received: 03 Apr 2018; Published Online: 27 Sep 2019.

* Correspondence: Mr. Laurens R Krol, Technische Universität Berlin, Biological Psychology and Neuroergonomics, Berlin, Germany, lrkrol@gmail.com