1. Introduction
The day seems not too far away, in which robots will be an active part of our daily life, just like electric appliances already are. Hence, there is an increasing need for paradigms, tools, and techniques to design proper human-robot interaction in a human-centered fashion (Beckerle et al., 2017). To this end, appropriate Human-Machine Interfaces (HMIs) are required, and there is a growing body of research showing how the Peripheral Nervous System (PNS) might be the ideal channel through which this interaction could proficiently happen.
During daily motor tasks such as grasping, walking, or speaking, the central nervous system (CNS) recruits a number of α-motoneurons in the ventral horn of the spinal cord and modulates the rate at which they discharge action potentials. The α-motoneurons are further modulated by supraspinal, afferent volleys and intrinsic motoneuron properties (Heckman et al., 2005; Enoka, 2008). The motoneuronal axonal action potentials are transformed into forces by a group of muscle fibers (the muscle unit) innervated by one axon. The muscle unit and the motoneuron form the final ensemble of all motor actions, the so-called motor unit. Translating neural commands into muscular forces, (spinal) motor units represent a promising interface to the CNS. However, there are some physiological constraints of motor control that must be taken into account by robotic applications.
In this opinion paper, we claim that better user experience would lead to more intuitive control and tighter human-robot interaction or even human-machine integration and vice-versa (see Figure 1).
Figure 1
Using PNS data for intent detection as well as for online assessment of user experience renders such interfaces technically promising and a tool to understand human behaviors and reactions (Beckerle et al., 2019). To improve on this, we discuss developments in intent detection and user feedback and user feedback emphasizing on anthropomorphic systems, which are directly controlled by humans, e.g., prostheses and teleoperation, and aiming to create novel sensorimotor paradigms.
2. Peripheral-Nervous-System–Machine Interfaces (PNS-MIs)
Interfaces for controlling anthropomorphic robotic systems, e.g., HMIs for self-powered prostheses, cannot function like a joystick or a touch-screen for instance, since the user cannot physically operate such devices. These HMIs must rather resort to interpreting the user's intent based on signals the user is able to produce - usually, relevant biological signals related to the intended muscle activation (Beckerle et al., 2019). Surface electromyography (Merletti et al., 2011) is a primary example, although different kinds of signals are currently being explored, e.g., tactile information (Beckerle et al., 2018, 2019) and also promising for other applications such as anthropomorphic teleoperation (Nostadt et al., 2020) or teaching collaborative robots (Cansev et al., 2021).
In principle, all signals generated by the users can and should be used to interpret their intent, but clearly, the optimal choice of signals and sensors depends on a tradeoff involving several factors. This includes, e.g., how well the sensors can be worn (ergonomy), how expensive the processing is (both economically and computationally), and how invasive their setup is. Furthermore, intent detection does not necessarily coincide with classification of signal patterns; rather, it's the ability to provide the user a seamless control experience based upon such signals, e.g., using regression instead of classification.
2.1. The Pros and Cons of CNS/PNS-MIs
Broadly speaking, signals related to movement and muscle activation can be classified according to whether they are recorded from the CNS or the PNS (Castellini et al., 2014). HMIs relying on CNS signals include brain-machine interfaces using surface electroencephalography as well as electrocorticography with direct implants on the motor cortex and spinal implants (Micera et al., 2010), or by decoding spinal motoneurons from high-density EMG signals (Farina et al., 2017; Del Vecchio and Farina, 2019). PNS-based HMIs, on the contrary, are those using signals recorded from the limbs, either invasively or non-invasively, e.g., implanted electromyography and direct connections to peripheral nerves vs. surface electromyography, force- and magneto-myography (Fang et al., 2015). Although EMG interfaces are placed in the periphery (muscles) the signal carried by the electrical activity generated by muscle fibers is in a one-to-one relation with spinal motoneurons. Moreover, minimally invasive approaches like local tomography of the limbs, which entail no surgery but indeed the injection of energy into the body, exist (Sierra González and Castellini, 2013; Gibas et al., 2019).
Given the extreme density of neural cells found in the CNS, most signals useful for robotic control to be potentially found in it are physically unavailable for direct inspection, unless one resorts to very invasive methods, e.g., the Braingate (Hochberg et al., 2012). Non-invasive methods, on the other hand, strongly suffer from cross-talk: the main problem is to tell signals pertaining to the intent under examination from “all the rest.” Surface electroencephalography, for instance, poses extremely complex problems to interpret and discern the neural firing patterns of interest, since each sensor can only record potentials from a large pool of neurons. Accordingly, damping and distorting effects due to skull bone tissue complicate pattern recognition (Lazarou et al., 2018). As opposed to that, an excellent signal-to-noise ratio can be obtained at the price of getting in contact with the cerebral cortex or the spinal cells (Hochberg et al., 2012).
PNS-based systems, on the other hand, can use better separated and physiologically relevant signals, naturally enforced by the anatomical branching of nerves and neurons as they depart from the brain, brainstem, and spinal cord. If one is interested in detecting the intent to move, exert forces and torques, and/or activate one's own muscles, then detecting such activity from the PNS appears to be a better choice especially if non-invasiveness is desired (Castellini et al., 2014). On top of this, if minimal invasiveness is permitted or desired, PNS approaches are probably even the best choice nowadays. Ultrasound scanning and electromyographic sensors implanted during osseointegration (Ortiz-Catalan et al., 2014) or injected into the muscles (Becerra-Fajardo and Ivorra, 2019) offer high signal-to-noise ratios while entailing rather low risk.
2.2. Improving on PNS-MIs
It has been known to physiologists for the last three decades that the neural activation that is transmitted by the motoneuron is delayed by the muscle tissue over a large range of values, from roughly 50 to more than 200 ms (Partridge, 1965; Baldissera et al., 1998). During fast motor tasks the nervous system compensates this delay by increasing the motoneuron firing frequency and the delay between the recruitment of successive motor units. Therefore, the CNS tunes this delay dynamically. Previous experiments in animal preparations demonstrated that changes in stimulation frequency alters the delay between the myoelectrical signal and the force produced by the muscle tissues in a very large range (Partridge, 1965; Baldissera et al., 1998). Recently, by decoding the activity of a large population of motoneurons during contraction at different speeds, we also found that the human nervous system modulates such delays in a very broad range (50–250 ms for hand and leg muscles Del Vecchio et al., 2018).
In virtually all prosthetic applications, however, this delay is fixed (Farina et al., 2014) yielding devices that do not follow the physiological modulation during natural processes like muscle fatigue (Zhou et al., 1998), adaptation of contraction speed (Del Vecchio et al., 2018), and muscular force output (Del Vecchio et al., 2018). Still, neuroergonomics should indeed translate these basic physiological findings into novel interface designs and, potentially, prosthetic applications for improving human robot-interactions. One potential solution to overcome this limitation is to decode surface EMG in real-time. We have previously shown that it is possible to retrieve the motoneuron discharge timings with delays smaller than 2 ms (Glaser et al., 2013; Barsakcioglu and Farina, 2018; Chen et al., 2020; Ting et al., 2021). Moreover, the potential to identify individual motor unit discharge times allows to label each motor unit to its unique motor space, e.g., encoding flexion/extension or which digit. Therefore, classification of EMG activity can be performed in a highly accurate way by associating each motor unit to its specific spatiotemporal space, as shown in a spinal cord injury case (Ting et al., 2021).
2.3. Considering User Experience Through PNS-MIs
Recent research outlines that PNS-MIs also have potential in directly assessing user experience going beyond established psychometric and physiological methods. An interesting example is the embodiment of robotic systems such as prostheses or teleoperation systems (Beckerle et al., 2019; Nostadt et al., 2020): the embodiment of artificial limbs can be assessed through surveying subjective experience with questionnaires (Longo et al., 2008), objective behavioral measures (e.g., proprioceptive drift), or (neuro)physiology (Christ and Reiner, 2014). This effect was also shown for artificial limbs with myoelectric control (Romano et al., 2015; Sato et al., 2018), but we might ask ourselves whether myoelectric measurements could also be used to analyze neuroergonomics of interaction with such devices. Recent work by Preatoni et al. for instance (Preatoni et al., 2021) indicates that proper sensory feedback makes a leg prosthesis feel lighter.
For patients suffering from stroke, the experience of device embodiment seems to have similar influence on electromyographic activity as for other physiological measures, i.e., electrodermal activity and skin temperature (Llorens et al., 2017). While, Tsuji et al. (2013) even report subjective survey results to be better represented by electromyography than by electrodermal activity, (Llorens et al., 2017) state that interactions between their subjective and neurophysiological results were inconclusive. Besides embodiment, the perception of pleasantness of affective touch can be related to electromyographic as well as to electrodermal measurements (Ree et al., 2019). This is very interesting since providing affective information through touch was shown to increase the embodiment of artificial limbs (Crucianelli et al., 2013, 2018; van Stralen et al., 2014) and, hence, appears worth considering in human-robot interaction (Beckerle et al., 2018).
Although myographic activity was measured at different sites, i.e., hand and face (Tsuji et al., 2013; Llorens et al., 2017; Ree et al., 2019), considering it in the assessment of user experience seems promising. We have ourselves recently put forward the potential connection between control based upon muscle activation and action schemes in the sense developed by Piaget (Piaget, 1950). Here, a proper PNS-MI could foster the creation of novel circular reactions, leading to embodiment as a natural consequence (Bettoni and Castellini, 2021). The factors influencing the effect remain unexplored so far. Understanding and shaping these interactions might be supported by multimodal data from an interface integrating myography with other physiological data, e.g., electrodermal activity or heart rate.
3. Discussion
With this position paper, we advocate peripheral neuroergonomics as an elegant way to improve HRI. Non-invasively interfacing the peripheral nervous system seems to provide very good interpretability and is currently advantageous over CNS-based interfaces, which outline higher invasiveness as well. Moreover, peripheral interfaces can augment or complement other modalities such as eye-tracking and electroencephalography to improve the recognition of user intentd and cognitive status. Generally, we expect considering neuromechanical insights in novel interfaces designs to foster improved HRI characteristics of robotic systems and devices. An accurate closed-loop control of the neuromechanical delays matching the physiological pathways would likely improve sensorimotor interactions. In addition, peripheral neural information can complement psychometric and physiological methods to assess user experience, which indicates that integrating myographic assessment in multimodal PNS-MIs would bring the neuroergonomics of human-robot interaction to a new level of quality.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Statements
Author contributions
PB coordinated its development as well as the integration of individual contributions. All authors conceptualized the structure, contributed content, perspectives, and references as well as discussed and revised the manuscript.
Funding
This work received support from the German Research Foundation (DFG) through the projects Active transfer learning with neural networks through human-robot interactions (TRAIN) (no. BE 5729/16) and Deep-Hand: deep learning + deep sensing for myocontrol of the upper limb (no. CA1389/1-2).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1
BaldisseraF.CavallariP.CerriG. (1998). Motoneuronal pre-compensation for the low-pass filter characteristics of muscle. A quantitative appraisal in cat muscle units. J. Physiol. 511, 611–627. 10.1111/j.1469-7793.1998.611bh.x
2
BarsakciogluD. Y.FarinaD. (2018). A real-time surface EMG decomposition system for non-invasive human-machine interfaces, in 2018 IEEE Biomedical Circuits and Systems Conference (BioCAS) (Cleveland, OH: IEEE), 1–4. 10.1109/BIOCAS.2018.8584659
3
Becerra-FajardoL.IvorraA. (2019). First steps towards an implantable electromyography (EMG) sensor powered and controlled by galvanic coupling, in IFMBE Proceedings, World Congress on Medical Physics and Biomedical Engineering (Prague). 10.1007/978-981-10-9023-3_4
4
BeckerleP.CastelliniC.LenggenhagerB. (2019). Robotic interfaces for cognitive psychology and embodiment research: a research roadmap. Wiley Interdiscipl. Rev. Cogn. Sci. 10:e1486. 10.1002/wcs.1486
5
BeckerleP.KõivaR.KirchnerE. A.Bekrater-BodmannR.DosenS.ChristO.et al. (2018). Feel-good robotics: requirements on touch for embodiment in assistive robotics. Front. Neurorobot. 12:84. 10.3389/fnbot.2018.00084
6
BeckerleP.SalviettiG.UnalR.PrattichizzoD.RossiS.CastelliniC.et al. (2017). A human-robot interaction perspective on assistive and rehabilitation robotics. Front. Neurorobot. 11:24. 10.3389/fnbot.2017.00024
7
BettoniM. C.CastelliniC. (2021). Interaction in assistive robotics: a radical constructivist design framework. Front. Neurorobot. 15:67. 10.3389/fnbot.2021.675657
8
CansevM. E.XueH.RottmannN.BliekA.MillerL. E.RückertE.et al. (2021). Interactive human-robot skill transfer: a review of learning methods and user experience (accepted). Adv. Intell. Syst. 3:2000247. 10.1002/aisy.202000247
9
CastelliniC.ArtemiadisP.WiningerM.AjoudaniA.AlimusajM.BicchiA.et al. (2014). Proceedings of the first workshop on Peripheral Machine Interfaces: going beyond traditional surface electromyography. Front. Neurorobot. 8:22. 10.3389/fnbot.2014.00022
10
ChenC.MaS.ShengX.FarinaD.ZhuX. (2020). Adaptive real-time identification of motor unit discharges from non-stationary high-density surface electromyographic signals. IEEE Trans. Biomed. Eng. 67, 3501–3509. 10.1109/TBME.2020.2989311
11
ChristO.ReinerM. (2014). Perspectives and possible applications of the rubber hand and virtual hand illusion in non-invasive rehabilitation: technological improvements and their consequences. Neurosci. Biobehav. Rev. 44, 33–44. 10.1016/j.neubiorev.2014.02.013
12
CrucianelliL.KrahéC.JenkinsonP. M.FotopoulouA. K. (2018). Interoceptive ingredients of body ownership: affective touch and cardiac awareness in the rubber hand illusion. Cortex104, 180–192. 10.1016/j.cortex.2017.04.018
13
CrucianelliL.MetcalfN. K.FotopoulouA. K.JenkinsonP. M. (2013). Bodily pleasure matters: velocity of touch modulates body ownership during the rubber hand illusion. Front. Psychology4:703. 10.3389/fpsyg.2013.00703
14
Del VecchioA.FarinaD. (2019). Interfacing the neural output of the spinal cord: robust and reliable longitudinal identification of motor neurons in humans. J. Neural Eng. 17:016003. 10.1088/1741-2552/ab4d05
15
Del VecchioA.UbedaA.SartoriM.AzorinJ. M.FeliciF.FarinaD. (2018). Central nervous system modulates the neuromechanical delay in a broad range for the control of muscle force. J. Appl. Physiol. 125, 1404–1410. 10.1152/japplphysiol.00135.2018
16
EnokaR. M. (2008). Neuromechanics of Human Movement. Campaign, IL: Human Kinetics.
17
FangY.HettiarachchiN.ZhouD.LiuH. (2015). Multi-modal sensing techniques for interfacing hand prostheses: a review. IEEE Sens. J. 15, 6065–6076. 10.1109/JSEN.2015.2450211
18
FarinaD.JiangN.RehbaumH.HolobarA.GraimannB.DietlH.et al. (2014). The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges. IEEE Trans. Neural Syst. Rehabil. Eng. 22, 797–809. 10.1109/TNSRE.2014.2305111
19
FarinaD.VujaklijaI.SartoriM.KapelnerT.NegroF.JiangN.et al. (2017). Man/machine interface based on the discharge timings of spinal motor neurons after targeted muscle reinnervation. Nat. Biomed. Eng. 1, 1–12. 10.1038/s41551-016-0025
20
GibasC.GrunewaldA.WunderlichH. W.MarxP.BruckR. (2019). A wearable EIT system for detection of muscular activity in the extremities, in 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Berlin), 2496–2499. 10.1109/EMBC.2019.8856792
21
GlaserV.HolobarA.ZazulaD. (2013). Real-time motor unit identification from high-density surface EMG. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 949–958. 10.1109/TNSRE.2013.2247631
22
HeckmanC.GorassiniM. A.BennettD. J. (2005). Persistent inward currents in motoneuron dendrites: implications for motor output. Muscle Nerve31, 135–156. 10.1002/mus.20261
23
HochbergL. R.BacherD.JarosiewiczB.MasseN. Y.SimeralJ. D.VogelJ.et al. (2012). Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature7398, 372–375. 10.1038/nature11076
24
LazarouI.NikolopoulosS.PetrantonakisP. C.KompatsiarisI.TsolakiM. (2018). EEG-based brain-computer interfaces for communication and rehabilitation of people with motor impairment: a novel approach of the 21st century. Front. Hum. Neurosci. 12:14. 10.3389/fnhum.2018.00014
25
LlorensR.BorregoA.PalomoP.CebollaA.NoéE.i BadiaS. B.et al. (2017). Body schema plasticity after stroke: subjective and neurophysiological correlates of the rubber hand illusion. Neuropsychologia96, 61–69. 10.1016/j.neuropsychologia.2017.01.007
26
LongoM. R.SchüürF.KammersM. P.TsakirisM.HaggardP. (2008). What is embodiment? A psychometric approach. Cognition107, 978–998. 10.1016/j.cognition.2007.12.004
27
MerlettiR.BotterA.CesconC.MinettoM.VieiraT. (2011). Advances in surface EMG: Recent progress in clinical research applications. Crit. Rev. Biomed. Eng. 38, 347–379. 10.1615/CritRevBiomedEng.v38.i4.20
28
MiceraS.CarpanetoJ.RaspopovicS. (2010). Control of hand prostheses using peripheral information. IEEE Rev. Biomed. Eng. 3, 48–68. 10.1109/RBME.2010.2085429
29
NostadtN.AbbinkD. A.ChristO.BeckerleP. (2020). Embodiment, presence, and their intersections: teleoperation and beyond. ACM Trans. Hum. Robot Interact. 9, 1–19. 10.1145/3389210
30
Ortiz-CatalanM.HåkanssonB.BrånemarkR. (2014). An osseointegrated human-machine gateway for long-term sensory feedback and motor control of artificial limbs. Sci. Transl. Med. 6:257re6. 10.1126/scitranslmed.3008933
31
PartridgeL. D. (1965). Modifications of neural output signals by muscles: a frequency response study. J. Appl. Physiol. 20, 150–156. 10.1152/jappl.1965.20.1.150
32
PiagetJ. (1950). The Psychology of Intelligence. London: Routledge and Kegan Paul.
33
PreatoniG.ValleG.PetriniF. M.RaspopovicS. (2021). Lightening the perceived prosthesis weight with neural embodiment promoted by sensory feedback. Curr. Biol. 31, 1065–1071. 10.1016/j.cub.2020.11.069
34
ReeA.MayoL. M.LeknesS.SailerU. (2019). Touch targeting C-tactile afferent fibers has a unique physiological pattern: a combined electrodermal and facial electromyography study. Biol. Psychol. 140, 55–63. 10.1016/j.biopsycho.2018.11.006
35
RomanoD.CaffaE.Hernandez-ArietaA.BruggerP.MaravitaA. (2015). The robot hand illusion: Inducing proprioceptive drift through visuo-motor congruency. Neuropsychologia70, 414–420. 10.1016/j.neuropsychologia.2014.10.033
36
SatoY.KawaseT.TakanoK.SpenceC.KansakuK. (2018). Body ownership and agency altered by an electromyographically controlled robotic arm. R. Soc. Open Sci. 5:172170. 10.1098/rsos.172170
37
Sierra GonzálezD.CastelliniC. (2013). A realistic implementation of ultrasound imaging as a human-machine interface for upper-limb amputees. Front. Neurorobot. 7:17. 10.3389/fnbot.2013.00017
38
TingJ. E.Del VecchioA.SarmaD.ColachisS. C.AnnettaN. V.CollingerJ. L.et al. (2021). Sensing and decoding the neural drive to paralyzed muscles during attempted movements of a person with tetraplegia using a sleeve array. medRxiv. 10.1101/2021.02.24.21250962
39
TsujiT.YamakawaH.YamashitaA.TakakusakiK.MaedaT.KatoM.et al. (2013). Analysis of electromyography and skin conductance response during rubber hand illusion, in 2013 IEEE Workshop on Advanced Robotics and Its Social Impacts (Tokyo: IEEE), 88–93. 10.1109/ARSO.2013.6705511
40
van StralenH. E.van ZandvoortM. J.HoppenbrouwersS. S.VissersL. M.KappelleL. J.DijkermanH. C. (2014). Affective touch modulates the rubber hand illusion. Cognition131, 147–158. 10.1016/j.cognition.2013.11.020
41
ZhouS.CareyM. F.SnowR. J.LawsonD. L.MorrisonW. E. (1998). Effects of muscle fatigue and temperature on electromechanical delay. Electromyogr. Clin. Neurophysiol. 38, 67–73.
Summary
Keywords
peripheral interfaces, myography, user experience, neuroergonomics, human-robot interaction
Citation
Del Vecchio A, Castellini C and Beckerle P (2021) Peripheral Neuroergonomics – An Elegant Way to Improve Human-Robot Interaction?. Front. Neurorobot. 15:691508. doi: 10.3389/fnbot.2021.691508
Received
06 April 2021
Accepted
28 July 2021
Published
20 August 2021
Volume
15 - 2021
Edited by
Ehsan T. Esfahani, University at Buffalo, United States
Reviewed by
Hemanth Manjunatha, Georgia Institute of Technology, United States; Ranjana K. Mehta, Texas A&M University, United States
Updates
Copyright
© 2021 Del Vecchio, Castellini and Beckerle.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Philipp Beckerle philipp.beckerle@fau.de
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.