Feedback Method of Force Controlled by Electrical Muscle Stimulation Based on Infrared Optical Sensing

1 Introduction

1.1 Background

Our experience in daily life is being augmented with the improvement in the technology of virtual reality or augmented reality. In the technology of connecting people and avatars in the metaverse, it is common to transfer people’s intentions to the avatar (Bourdin et al., 2019). It is important that the transmission of body movement information be interactive. If we can follow the avatar’s motion passively, we can experience movement that we have never experienced through the metaverse.

Human body sharing is a technology enriching our lives by connecting us with other people in the real world or avatars in the metaverse. Since human motion is controlled based on electrical signals, the human motion information can be expressed as electrical signal data. In particular, electric muscle stimulation (EMS) enables us to share electrical signals actuating muscles between humans. Since the EMS approach ensures the user’s hands-free and small application compared to mechanical actuators, the EMS technology has a potential for general use as a daily application.

Human body physique is different among individuals; thus, reaction to EMS is different. If the EMS signal and human movement can be converted, we can experience other people’s motion history. Physical movements during sports and playing musical instruments are transmitted from a person to a person. If it is possible to transmit movement directly, we can be instructed directly to know the proper motion, such as cooking, massage, or pet handling expert. It is important to compensate for the individual difference in the relationship between EMS parameters and body movement.

1.2 Related Works

One of the main challenges of EMS is haptics providing a human with realism. The EMS technologies ensure hands-free haptic feedback. EMS to arm muscles applied interactive force feedback, such as resistance force from a game object to a human when playing video games (Kruijff et al., 2006; Lopes and Baudisch, 2013). A miniaturized force feedback device is applicable to mobile gaming. In addition, EMS to the forearm provided the haptics feeling of grabbing an object (Pfeiffer et al., 2014). EMS to the wrist muscles was used to provide the sensation of touching an object or counterforce from an object (Lopes et al., 2018). In order to create the sensation of picking up a heavy cube, the EMS system actuates the respective opposite muscles, actuating the triceps for applying a load to the user’s biceps and actuating the user’s shoulder muscle for applying a load to the user’s pectoralis (Lopes et al., 2017). EMS to fingertip provided perceptions of softness-hardness and stickiness (Yem et al., 2018). EMS provided the sensation when tapping a rigid object in mid-air (Miyamoto et al., 2015). These techniques can involve interaction with virtual objects, which enables us to feel like that these objects exist in front of us.

Additionally, EMS has been researched for sharing body movement information between humans. EMS actuation systems mainly focus on methods of controlling fingers (Nith et al., 2021; Takahashi et al., 2021; Tamaki et al., 2011, 2010). EMS can actuate each finger independently (Takahashi et al., 2021). Controlling other people’s fingers was used as a remote conversation tool by Hanagata and Kakehi (2018). Muscle-plotter steered a wrist to plot charts as an interaction system (Lopes et al., 2016). EMS synchronized muscle tension interactively between two users (Nishida and Suzuki, 2017). Conveying a body motion as communication with EMS was useful to augment video conferencing (Hanagata and Kakehi, 2018). In another study, EMS conveyed emotions sensed by electroencephalography across distances by body gesture (Hassib et al., 2017). For better dexterity control of a finger, exoskeleton and EMS electrodes are combined (Nith et al., 2021).

Since EMS is an actuator system moving a body of a human, the EMS technology can be used as training or skill transfer tools. The use of EMS has the potential to communicate and teach physical movements to learners from an instructor. Several studies have been proposed in which physical movements during sports or playing a musical instrument are transmitted and taught from the instructor to the learner using EMS (Tamaki et al., 2011; Nishida and Suzuki, 2017). EMS enables users to learn how to manipulate tools that the users have not known how to use (Lopes et al., 2015). EMS training enabled a faster motor response as an after-effect (Kasahara et al., 2021). In addition, the walking direction was modified by EMS to the sartorius muscle for pedestrian navigation, and users need not pay attention to the navigation task (Pfeiffer et al., 2015). EMS has also been used for rehabilitation, which facilitates nerve regeneration and recovery (Fu et al., 2020). Applying low-intensity EMS was helpful for increasing the muscle strength and cardiorespiratory fitness (Miyamoto et al., 2016). In addition, EMS actuates the calf muscles of the swing foot during running to control the foot angle for coaching a better way of striking the ground (Hassan et al., 2017). However, since the actual physical movement is not detected by the EMS system, it is difficult to confirm automatically whether the muscle movement is what the instructor intends.

1.3 Objective

In this study, we focused on the fingertip motion because many EMS research studies have focused on finger motions (Nith et al., 2021; Takahashi et al., 2021; Tamaki et al., 2011, 2010). The goal of the study was to develop a system that can adjust the EMS parameters for individuals when sharing experiences with stimulation by sensing the degree of muscle contraction during electrical stimulation. Since how much a body is moved by EMS is user-dependent, the EMS parameters need to be adjusted for each user. If we do not know the appropriate amount of current for stimulation for an individual, the muscles would not contract as we aimed, and we will not be able to share the experience as we expected.

To adjust parameters, feedback control is necessary (Crago et al., 1980; Kurosawa et al., 2005). For conducting closed-loop feedback control, output values must be obtained. The measurement of output values of the fingertip requires a dedicated sensor, such as a force sensor. However, it is not practical because these sensors need to be directly attached to the fingertip and disturb the fingertip motion. Estimation of the fingertip motion is important for transmitting the feedback of EMS to the system.

There is a method to estimate the active muscle movements of the user by electromyography (EMG). However, it is difficult to use EMG to estimate the movements of learners who are passively moved by EMS. Since the EMG sensors are placed on the skin surface, the EMS current flows through the EMG sensors while the EMS is being applied. Although there was research using EMG and EMS (Kasuya et al., 2013), the muscle state activated by EMS was not able to be obtained.

The objective of this study was to ensure the method obtaining a feedback EMS output for adjusting the EMS parameter for the individual. We assume that infrared optical sensing can be used for estimating the fingertip force without disturbance from EMS current. The attachment of EMS pads limits the attachment position of the infrared optical sensor. We assume that circumference sensor arrays attached to the forearm between EMS pads can extract the feature of the fingertip motion during EMS based on a machine learning algorithm, even though the sensor attachment position changes. In this study, we presented a system estimating the fingertip force as the output of EMS by monitoring the muscle state based on infrared optical sensing.

2 System

The proposed system obtains the state of the muscles during EMS for adjusting the parameters of EMS for each individual. To monitor the finger motion, the infrared information observing the forearm muscle correlated with the finger motion was applied. A FirstVR machine (H2L Inc., Tokyo, Japan), which has 14 channels of infrared optical sensors, retrieves the muscle and skin deformation information, as shown in Figure 1. The reason why we used this sensor in this study is that 14 channels of the infrared optical sensor can obtain information about the forearm circumference. Infrared optical sensing consists of infrared light-emitting and infrared light-receiving sources. The reading value for each channel changes in correspondence with the muscle or skin deformation. The FirstVR machine reads the intensity of infrared rays of photoreflectors which emit near-infrared lights and receive reflected lights and send data to a computer through Bluetooth Low Energy. FirstVR was banded around the same part of the body as the placement of EMS pads. Infrared optical sensing observes the state of the muscle which is deformed corresponding to the motion of the endpoint of a limb (Miyake et al., 2021, 2019). The optical sensing method has resistance against sweating and can eliminate noise. In addition, the method is less expensive to implement than other methods for obtaining biological information. The reading value for each channel does not indicate the muscle state directly. Therefore, the reading values were converted to the force value (or movement amount) based on a machine learning–based algorithm.

FIGURE 1

FIGURE 1. Infrared optical sensor FirstVR.

Support vector regression (SVR) derives the pushing force value of the fingertip which is controlled by EMS. Input data on SVR are optical data of the FirstVR machine. Soft margin SVR is applied because it accepts small errors so that the SVR avoids over-fitting. SVR has high regression performance using an acceptable error margin and kernel function transformation of input data, which leads to its adaptability to a complicated data structure with a small number of inputs (14 from FirstVR) (Cortes and Vapnik, 1995). The regression equation of SVR f(x) is as follows:

$f\left(\mathbf{x}\right)={\mathbf{w}}^{\mathbf{T}}\varphi \left(\mathbf{x}\right)+\mathbf{b},(1)$

where x indicates a vector of input data of infrared readings (14 signals from FirstVR), ϕ indicates a kernel function, w indicates a vector of the weight parameter, and b indicates a bias parameter. To handle non-linear data structure, the radial basis function is implemented as a kernel function as

$\varphi \left(\mathbf{x}\right)=\mathit{exp}\left(-\gamma \Vert \mathbf{x}-{\mathbf{x}}^{\prime }{\Vert }^2\right),(2)$

where x′ indicates the mean value of input training data, and γ indicates a Gaussian function. To train the parameters of w and b, the target function consisting of C is minimized, and ϵ represents parameters which control the acceptance rate of errors and the penalty, respectively. The target function L(w) is formulated as follows:

$L\left(\mathbf{w}\right)=\frac{1}{2}\Vert \mathbf{w}{\Vert }^2+C\sum _{i=1}^N\mathit{max}\left(0,|y_i-f\left(x_i\right)|-ϵ\right),(3)$

where N indicates the total number of the vector, i indicates the sample number, and y_i indicates a target value. The target function L(w) is minimized based on the Lagrange multiplier using slack variables (Smola and Schölkopf, 2004).

Figure 2 shows the system diagram. The estimated force based on infrared optical sensing is feedbacked for closed-loop feedback control of EMS. It estimates the force applied to the moving finger when the EMS is applied to a user with several parameters. It is possible to find the better parameter (amount of current) whose output becomes closer to the target value.

FIGURE 2

FIGURE 2. Block diagram of the EMS amplitude control system. The pushing force estimated by infrared optical sensing values was used as a feedback value.

3 Experiment

3.1 Experimental Condition

The objective of this experiment was to investigate whether the degree of muscle contraction can be estimated from the infrared optical sensor values when the EMS parameters are changed and the finger is moved. Therefore, we gathered data that were composed of EMS parameters, the deformation of the muscles when EMS was applied, and the force applied to the fingertips by EMS.

There were 10 subjects (three females and seven males) with a mean age of 25.7 years (22–33 years old). During data acquisition, electrical stimulation with a pulse width of 200 µs and a frequency of 40 Hz was applied, while the current level varied from 20 to 40 mA. The range of current was determined by considering whether EMS can move a finger or not, referring to a previous study sharing a body motion (Hassib et al., 2017). Other parameters were set heuristically for considering conditions where the finger is moved and discomfort is less before performing an experiment. The parameters of the EMS used in the experiment are shown in Table 1.

TABLE 1

TABLE 1. Stimulation parameters per finger (mA), frequency (hz), pulse width (µs) and duration (s).

3.2 Procedure

In the experiment, we explained the experiment contents to all participants and obtained their consent before conducting the experiment. We also explained how to interrupt the experiment so that subjects could stop the experiment at any time. No explanation was given regarding the purpose of the experiment. The subjects were asked not to exert the fingertip force so that subjects could not intentionally manipulate their fingers. This experiment was approved by the Japanese Society for Wellbeing Science and Assistive Technology (No. 240).

The pushing force was measured based on the value of the voltage applied to the force sensor (FSR402, Interlink Electronics Inc., California, United States) when the 5 V power supply was divided by the FSR and a resistor. The microcontroller (Arduino Uno, Italy) read the voltage value that was changed by pushing the force sensor with a resolution of 8 bits.

EMS was applied with iStim TENS EV-820 (iStim, California, United States). EMS was applied to actuate the index and middle fingers. The electrode pads were attached to the positions where the PIP and DIP joints of the target finger moved.

The procedure of the experiment is divided into the following three steps.

STEP 1: the electrodes are placed at the position where only the finger to be tested moves.

STEP 2: a FirstVR machine is attached to the subject’s forearm.

STEP 3: a force sensor (FSR402) is placed under the subject’s finger to be tested for data acquisition.

In STEP 1, the location on the forearm of the electrodes where only the finger moves was identified, and the electrodes were placed. At this time, electrodes were not placed at the position where the FirstVR was placed. The number of electrodes was two, which is the minimum number required for EMS. In STEP 2, the FirstVR was attached to the subject. The electrodes and FirstVR on the forearm are shown in Figure 3A. In Figure 3A, the experimental instrument was placed in the electrode-FirstVR-electrode order, but this arrangement varies depending on the subject and the fingers to be tested. In STEP 3, data acquisition was conducted. The experiment during data acquisition is shown in Figure 3B. During data acquisition, a force sensor was placed under the finger to be tested, as shown in Figure 3B, in order to measure the force generated when the finger was bent. Data acquisition was performed by varying the parameters of the EMS (current, pulse width, and frequency) while acquiring the force sensor values and the muscle displacement by FirstVR. The wrist was on the table in a relaxed state, and the posture of the subjects was consistent during the experiment. The experiment was conducted under the same conditions for all subjects.

FIGURE 3

FIGURE 3. Experiment images. (A) Arrangement of electrodes and FirstVR. (B) Scene of the experiment.

In addition, we checked whether the proposed model can provide feedback on the estimated force in real-time or not. Because adjusting the EMS strength was out of limitation in this study, we performed just a simple test. We set the target force value as 480 g. There were two cases: the minimum EMS current can output the target force or higher, and the minimum EMS current was not enough to output the target force. We manually increased the EMS current by reading the estimated value if the estimated value was lower than the target value.

3.3 Data Analysis

We investigated whether it is possible to estimate the degree of force application from the infrared optical sensor values. SVR regression accuracy was evaluated after SVR was trained from all subjects' data. After combining the data for all the subjects, the data were divided into training and test data. The training data were used to train the model, and then, the performance of the model was validated using testing data. Since the test data were unknown to the model, this can be an indicator of generalization performance. Of the acquired data, 75% were extracted as training data, and 25% were used as test data. The validation method was a nested cross validation method (five-fold), which divides the data differently multiple times and then adds up the validation results with test data. We did not acquire data on the EMS parameter for which the subject felt pain.

The combination of the values of the infrared optical sensor and the EMS parameter of each frame was used as the input. The sample number of test data was more than 200 for each finger. The regression equation that the value of the force on the fingertip when the finger was moved by the EMS was made as output, and correlation, R2 (coefficient of determination), and root mean square error (RMSE) were derived for accuracy evaluation. We performed the significance test of the correlation coefficient to investigate whether the relationship between estimation and measure values was a correlation or not. RMSE was derived using the total sample number N as

$RMSE\left(\mathit{x}\right)=\sqrt{\frac{1}{N}\sum _{i=1}^N{\left(f\left(x^{\left(i\right)}\right)-y^{\left(i\right)}\right)}^2}.(4)$

4 Result

Figures 4, 5 show the relationship between the force values measured by the force sensor and force values estimated by infrared optical sensing with SVR. The finger was controlled passively (the human did not intend to push the force sensor) by EMS. The results show a significant correlation. The values of C and ϵ were 100,000 and 0.0001, respectively. Figure 6 shows an example of the original data on the signals read by each IR sensor. This figure also shows the pushing force with a dotted line. In this figure, EMS was applied at approximately 40 frames (50 Hz), and a large amount of IR data changed afterward. Table 2 shows the results of aggregating the data on all the subjects. Negative values are rarely calculated when estimating the force, but since these values can be regarded as outliers systematically, these data are eliminated. The correlation coefficients of Table 2 show that there is a strong correlation between the estimated and measured values of the force by EMS. In addition, the values of RMSE also show that the index finger has a larger variation in data than the middle finger. Figures 4, 5 also show that there is variation in the data for the index finger, and the accuracy of the estimated force of the index finger is lower than that of the middle finger.

FIGURE 4

FIGURE 4. Results of the relationship between measured and estimated values of the force of the index finger. The horizontal axis is the measured value acquired by the force sensor, and the vertical axis is the estimated value by FirstVR. p-value indicates the significant result of the correlation coefficient. The result shows a significant correlation.

FIGURE 5

FIGURE 5. Results of relationship between measured and estimated values of the force of the middle finger. The horizontal axis is the measured value acquired by the force sensor, and the vertical axis is the estimated value by FirstVR. p-value indicates the significant result of the correlation coefficient. The result shows a significant correlation.

FIGURE 6

FIGURE 6. Example of the original data on the signals read by each IR sensor and the pushing force for the middle finger. EMS was applied at approximately 40 frames (50 Hz), and a large amount of IR data changed afterward.

TABLE 2

TABLE 2. Evaluation results of the experiment. Correlation, R², and RMSE of the index and middle fingers, respectively, calculated by summing up the results of all subjects are shown.

In addition, the relationship between the EMS parameter and pushing force was obtained to discuss the effect of the difference in EMS conditions and individual differences. Figure 7 shows an example result of the measured pushing force of index and middle fingers in each EMS condition for one subject. Conditions 1, 2, and 3 have the same values of pulse width and frequency, 200 µs and 40 Hz, but different current values, 20 mA, 30 mA, and 40 mA, respectively. The pushing force of the middle finger became stronger as the EMS signal became stronger, while the pushing force of the index finger did not always become stronger. Figure 8 shows the pushing force of the middle finger for each individual in EMS condition 1, pulse width 200 µs, frequency 40 Hz, and current 20 mA. There was an individual difference in the pushing force values among individuals.

FIGURE 7

FIGURE 7. Example of the pushing force measured by the force sensor in each EMS condition for one subject. The pushing motion was controlled by only EMS. (A) shows the pushing force of the index finger, and (B) shows the pushing force of the middle finger. EMS was performed by adding a current of 20 mA for condition 1, 30 mA for condition 2, and 40 mA for condition 3.

FIGURE 8

FIGURE 8. Individual difference of the pushing force of the middle finger when EMS parameters were as follows: pulse width: 200 µs, frequency: 40 Hz, and current value: 20 mA.

Based on the trained SVR, the feedback value of the estimated force for the middle finger was obtained in real time. We manually increased the EMS current value (20–30 mA) by reading the estimated value if the estimated value was lower than the target value. Figure 9 shows pushing force in cases where the EMS current value changed (case 1) and where the EMS current value did not change (case 2). In case 1, the value approaches the target value by increasing the EMS current based on the estimated value of the pushing force.

FIGURE 9

FIGURE 9. Pushing force for the middle finger in cases where the EMS current value changed (case 1) and where the EMS current value did not change (case 2). We manually increased the EMS current by reading the estimated value if the estimated value was lower than 480 g.

5 Discussion

We estimated the fingertip force based on infrared optical sensing values, which was activated by EMS. As shown in Table 2 and Figures 4, 5, the relationship between the estimated and measured forces showed a fairly strong positive relationship, and the coefficient of determination was also high. As shown in Figure 6, a large amount of IR data changed after EMS. Although there was a difference between fingers, fingertip force estimation with infrared optical sensing during EMS was possible. It indicates that the system can monitor a feedback value of electrical muscle stimulation fingertip control based on the muscle state observation of the forearm with infrared optical sensing. Although the appropriate gain might vary from an individual to an individual, the EMS parameter is adjustable using the estimated force. If the EMS current causes a higher force than the target value, EMS becomes lower, and if it is lower, EMS becomes higher.

The results show that the accuracy differs depending on the finger. The middle finger is located in the middle of the hand; thus, it is easier for hand posture to be stable during the push-in movement. When the fingers are moved by EMS, the wrist is subjected to the reaction force. When the index finger is moved by EMS, we assume that a moment is generated in the direction of abduction of the wrist. Since the transmission of the pushing force of the index finger to the force sensor becomes poor, the estimation accuracy of the pushing force becomes worse. As shown in Figure 7, the pushing force did not increase proportionally as the current of EMS became stronger. In fact, we observed a case in which the pushing position of the index finger deviated from the sensor, and the pushing force was not transmitted to the sensor appropriately. Therefore, the ground truth of the estimated value here may be wrong in some frames. In the future, it might be important to use a simple exoskeleton to smooth the transmission of power.

As shown in Figure 8, we reconfirmed that the pushing force was different among individuals even though the EMS strength was the same. It means that the degree of the pushing movement of the fingertip controlled by EMS was different among individuals. Nonetheless, the estimation result based on the infrared optical sensing value was not affected by the individual difference. The feature of the muscle state obtained by infrared optical sensing was consistent among individuals.

The system showed the feasibility of infrared optical sensing for the closed-loop feedback control system of the EMS parameters for the individual. In Figure 7, the difference in the pushing force of the middle finger due to ampere difference among conditions was more than 60 g for this subject. The RMSE of force estimation of the middle finger was less than the error among conditions in this experiment, that is, 10 mA adjustment of EMS was possible. For the index finger, RMSE was less than the difference in the pushing force between conditions 1 and 2. The highest ampere of EMS could not increase the force strength, which might mean that the EMS system cannot apply higher pushing force any further. As mentioned earlier, the force might not increase due to hand rotation. The simple exoskeleton would be better for more range of force adjustment for the index finger. It might happen to other fingers except for the middle fingers, considering the finger placement. In addition, for more detailed adjustment, it would be better to control the current value more finely, further investigation increasing the experimental conditions.

Our system enables people to share the fingertip force. The force range can be 0.6 N to more than 50 N for playing the piano (Parlitz et al., 1998; Kinoshita et al., 2007). Moderately loud pushing (mezzo forte) requires approximately 10 N (approximately 1,000 g). In the case of pushing force of 10 N for such cases, the error of our system accounts for less than 10. The difference among stronger forces was approximately 10 N, while the difference among weaker forces was approximately 2 N. These differences in the pushing force can be discriminated by our system. Therefore, our current system has a limitation to convey the smaller force information, but it can be used to convey larger force information with higher accuracy, such as mezzo forte. In addition, as shown in Figure 9, the force value could approach the target value through real-time feedback. Because we did not develop the EMS stimulator, it is difficult to make a fine adjustment of force. The development of the method for fine-tuning fingertip force with EMS would be needed in future work.

For the current system, a difference in the attachment placement was not considered. In this study, since the training data include sensing information of all subjects, the SVM training process might be able to select the channels of FirstVR that are important for regression. For general use in the future, it would be helpful for better usability and higher estimation accuracy to develop a methodology to identify which channel corresponds to which finger’s muscles for each individual. Furthermore, increasing the number of human samplings would be better for making the SVR model more robust and investigating the feasibility of more tasks in future work. We did not develop an EMS simulator system, and it is difficult to make a fine adjustment of force. The development of the method for fine-tuning the fingertip force with EMS would be needed in the future work. We aimed to know the natural motion controlled by EMS without any constraint because hands-free use is the most natural condition considering the practical use of EMS. We consider how the hand or wrist constrained should be investigated in future work.

Data Availability Statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Japanese Society for Wellbeing Science and Assistive Technology. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

SH and TM conceptualized the research. SH implemented the system. SH and SM conducted the experiment. ET aided in the conceptualization of this study. All authors wrote parts of the manuscript.

Conflict of Interest

SH, TM, and SM were fully or partially employed by the company H2L Inc. ET was the employer of the company H2L Inc.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Bourdin, P., Martini, M., and Sanchez-Vives, M. V. (2019). Altered Visual Feedback from an Embodied Avatar Unconsciously Influences Movement Amplitude and Muscle Activity. Sci. Rep. 9, 19747–19749. doi:10.1038/s41598-019-56034-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Cortes, C., and Vapnik, V. (1995). Support-vector Networks. Mach. Learn 20, 273–297. doi:10.1007/bf00994018

CrossRef Full Text | Google Scholar

Crago, P. E., Mortimer, J. T., and Peckham, P. H. (1980). Closed-loop Control of Force during Electrical Stimulation of Muscle. IEEE Trans. Biomed. Eng. BME-27, 306–312. doi:10.1109/tbme.1980.326738

PubMed Abstract | CrossRef Full Text | Google Scholar

Fu, T., Jiang, L., Peng, Y., Li, Z., Liu, S., Lu, J., et al. (2020). Electrical Muscle Stimulation Accelerates Functional Recovery after Nerve Injury. Neuroscience 426, 179–188. doi:10.1016/j.neuroscience.2019.10.052

PubMed Abstract | CrossRef Full Text | Google Scholar

Hanagata, S., and Kakehi, Y. (2018). “Paralogue: A Remote Conversation System Using a Hand Avatar Which Postures Are Controlled with Electrical Muscle Stimulation,” in Proceedings of the 9th Augmented Human International Conference, 1–3.

Google Scholar

Hassan, M., Daiber, F., Wiehr, F., Kosmalla, F., and Krüger, A. (2017). FootStriker. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 1–18. doi:10.1145/3053332

CrossRef Full Text | Google Scholar

Hassib, M., Pfeiffer, M., Schneegass, S., Rohs, M., and Alt, F. (2017). “Emotion Actuator: Embodied Emotional Feedback through Electroencephalography and Electrical Muscle Stimulation,” in Proceedings of the 2017 chi conference on human factors in computing systems, 6133–6146.

Google Scholar

Kasahara, S., Takada, K., Nishida, J., Shibata, K., Shimojo, S., and Lopes, P. (2021). “Preserving Agency during Electrical Muscle Stimulation Training Speeds up Reaction Time Directly after Removing Ems,” in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–9. doi:10.1145/3411764.3445147

CrossRef Full Text | Google Scholar

Kasuya, M., Seki, M., Kawamura, K., Kobayashi, Y., Fujie, M. G., and Yokoi, H. (2013). “Robust Grip Force Estimation under Electric Feedback Using Muscle Stiffness and Electromyography for Powered Prosthetic Hand,” in 2013 IEEE International Conference on Robotics and Automation (Karlsruhe, Germany: IEEE), 93–98. doi:10.1109/icra.2013.6630561

CrossRef Full Text | Google Scholar

Kinoshita, H., Furuya, S., Aoki, T., and Altenmüller, E. (2007). Loudness Control in Pianists as Exemplified in Keystroke Force Measurements on Different Touches. J. Acoust. Soc. Am. 121, 2959–2969. doi:10.1121/1.2717493

PubMed Abstract | CrossRef Full Text | Google Scholar

Kruijff, E., Schmalstieg, D., and Beckhaus, S. (2006). “Using Neuromuscular Electrical Stimulation for Pseudo-haptic Feedback,” in Proceedings of the ACM symposium on Virtual reality software and technology, 316–319. doi:10.1145/1180495.1180558

CrossRef Full Text | Google Scholar

Kurosawa, K., Futami, R., Watanabe, T., and Hoshimiya, N. (2005). Joint Angle Control by Fes Using a Feedback Error Learning Controller. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 359–371. doi:10.1109/tnsre.2005.847355

PubMed Abstract | CrossRef Full Text | Google Scholar

Lopes, P., and Baudisch, P. (2013). “Muscle-propelled Force Feedback: Bringing Force Feedback to Mobile Devices,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2577–2580.

Google Scholar

Lopes, P., Jonell, P., and Baudisch, P. (2015). “Affordance++ Allowing Objects to Communicate Dynamic Use,” in Proceedings of the 33rd annual acm conference on human factors in computing systems, 2515–2524.

Lopes, P., You, S., Cheng, L.-P., Marwecki, S., and Baudisch, P. (2017). “Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation,” in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 1471–1482.

Google Scholar

Lopes, P., You, S., Ion, A., and Baudisch, P. (2018). “Adding Force Feedback to Mixed Reality Experiences and Games Using Electrical Muscle Stimulation,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1–13. doi:10.1145/3173574.3174020

CrossRef Full Text | Google Scholar

Lopes, P., Yüksel, D., Guimbretière, F., and Baudisch, P. (2016). “Muscle-plotter: An Interactive System Based on Electrical Muscle Stimulation that Produces Spatial Output,” in Proceedings of the 29th Annual Symposium on User Interface Software and Technology, 207–217.

Google Scholar

Miyake, T., Cheng, Z., Hosono, S., Yamamoto, S., Funabashi, S., Zhang, C., et al. (2019). “Heel-contact Gait Phase Detection Based on Specific Poses with Muscle Deformation,” in 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO) (Dali, China: IEEE), 977–982. doi:10.1109/robio49542.2019.8961661

CrossRef Full Text | Google Scholar

Miyake, T., Yamamoto, S., Hosono, S., Funabashi, S., Cheng, Z., Zhang, C., et al. (2021). Gait Phase Detection Based on Muscle Deformation with Static Standing-Based Calibration. Sensors 21, 1081. doi:10.3390/s21041081

PubMed Abstract | CrossRef Full Text | Google Scholar

Miyamoto, N., Aoyama, K., Furukawa, M., Maeda, T., and Ando, H. (2015). “Air Tap: The Sensation of Tapping a Rigid Object in Mid-air,” in Haptic Interaction (Berlin, Germany: Springer), 285–291. doi:10.1007/978-4-431-55690-9_52

CrossRef Full Text | Google Scholar

Miyamoto, T., Kamada, H., Tamaki, A., and Moritani, T. (2016). Low-intensity Electrical Muscle Stimulation Induces Significant Increases in Muscle Strength and Cardiorespiratory Fitness. Eur. J. sport Sci. 16, 1104–1110. doi:10.1080/17461391.2016.1151944

PubMed Abstract | CrossRef Full Text | Google Scholar

Nishida, J., and Suzuki, K. (2017). “Biosync: A Paired Wearable Device for Blending Kinesthetic Experience,” in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 3316–3327.

Google Scholar

Nith, R., Teng, S.-Y., Li, P., Tao, Y., and Lopes, P. (2021). “Dextrems: Increasing Dexterity in Electrical Muscle Stimulation by Combining it with Brakes,” in The 34th Annual ACM Symposium on User Interface Software and Technology, 414–430. doi:10.1145/3472749.3474759

CrossRef Full Text | Google Scholar

Parlitz, D., Peschel, T., and Altenmüller, E. (1998). “Assessment of Dynamic Finger Forces in Pianists: Effects of Training and Expertise,”, 1063–1067. doi:10.1016/s0021-9290(98)00113-4J. biomechanics31

CrossRef Full Text | Google Scholar

Pfeiffer, M., Dünte, T., Schneegass, S., Alt, F., and Rohs, M. (2015). “Cruise Control for Pedestrians: Controlling Walking Direction Using Electrical Muscle Stimulation,” in Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2505–2514.

Google Scholar

Pfeiffer, M., Schneegass, S., Alt, F., and Rohs, M. (2014). “Let Me Grab This: a Comparison of Ems and Vibration for Haptic Feedback in Free-Hand Interaction,” in Proceedings of the 5th augmented human international conference, 1–8.

Google Scholar

Smola, A. J., and Schölkopf, B. (2004). A Tutorial on Support Vector Regression. Statistics Comput. 14, 199–222. doi:10.1023/b:stco.0000035301.49549.88

CrossRef Full Text | Google Scholar

Takahashi, A., Brooks, J., Kajimoto, H., and Lopes, P. (2021). “Increasing Electrical Muscle Stimulation’s Dexterity by Means of Back of the Hand Actuation,” in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–12. doi:10.1145/3411764.3445761

CrossRef Full Text | Google Scholar

Tamaki, E., Miyaki, T., and Rekimoto, J. (2010). “Possessedhand: a Hand Gesture Manipulation System Using Electrical Stimuli,” in Proceedings of the 1st augmented human international conference, 1–5.

Google Scholar

Tamaki, E., Miyaki, T., and Rekimoto, J. (2011). “Possessedhand: Techniques for Controlling Human Hands Using Electrical Muscles Stimuli,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 543–552.

Google Scholar

Yem, V., Vu, K., Kon, Y., and Kajimoto, H. (2018). “Effect of Electrical Stimulation Haptic Feedback on Perceptions of Softness-Hardness and Stickiness while Touching a Virtual Object,” in 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (Tuebingen, Germany: IEEE), 89–96. doi:10.1109/vr.2018.8446403

CrossRef Full Text | Google Scholar