Neural-enhanced motion-to-EMG: refining simulated muscle activity from musculoskeletal models using a Seq2Seq approach

Electromyography (EMG) is essential for accurate assessment of motor function in rehabilitation, sports science, and robotics. However, its various time-consuming human operations (e.g., electromagnetic noise countermeasures) limit its widespread use. Meanwhile, motion capture technology has become more accessible, leading to increasing interest in musculoskeletal simulation models such as OpenSim. Although advances have been made in individualizing the model parameters, accurately estimating muscle activity remains a significant challenge. Previous efforts to optimize the parameters in musculoskeletal model simulators have yielded limited improvements in estimation accuracy. A key source of error that is identified in this study is the spatio-temporal distortion between the estimated and actual muscle activity, which has been inadequately addressed in previous research. To address this problem, this study proposes the Neural-Enhanced Motion-to-EMG (NEM2E) framework, which mitigates spatio-temporal distortions in simulated muscle activity using the Spatio-Temporal Distortion Refinement Network (STDR-Net). The STDR-Net is implemented via a Sequence-to-Sequence model with attention mechanisms to refine the estimates. Validation on two public datasets (walking and running motions) confirms significant accuracy improvements: enhanced estimations for all five muscles in the running dataset and for two of five muscles in the walking dataset. These findings demonstrate the potential of the NEM2E framework to refine OpenSim-generated muscle activity estimates and advance personalized applications in muscle activity analysis.

1 Introduction

Electromyography (EMG) is essential for accurate assessment of motor function in rehabilitation, sports science, and robotics. However, it is not suitable for routine measurement because it requires various time-consuming human operations, such as the application of electrodes by specialists and electromagnetic noise countermeasures, for correct measurement. In addition, it is affected by sweat and changes in skin conditions, making it unsuitable for long-term measurement. In contrast, markerless motion capture systems such as THEIA three-dimensional (3D), Azure Kinect, and mediapipe allow for easy motion measurement. Mobile force plates that can measure ground reaction forces simply by putting on shoes have also been investigated (Liu et al., 2010; Adachi et al., 2011) and commercialized by Tec Gihan Co., Ltd. (M3D Force Plate). In addition, VMOCAP (Ohashi et al., 2020), which estimates motion and muscle activity using only an RGB camera, has been proposed. Owing to these technological advances, muscle activity estimation using a musculoskeletal model simulator with measurements of the motions and ground reaction forces has become a useful system for measuring muscle activity in the elderly and in athletes who wish to reduce their measurement burden.

In this context, musculoskeletal model simulators (anybody, 2025; Dhaibaworks, 2025; ARMO, 2025; SIMM, 2025) such as OpenSim (Opensim, 2025; Delp et al., 2007) have attracted increasing attention for muscle activity estimation. A key challenge in musculoskeletal simulation is the accurate modeling of individual human dynamics. Optimizing model parameters such as the bone length, muscle force characteristics, and joint stiffness is crucial for achieving more realistic analyses (Dembia et al., 2021).

Among these simulators, OpenSim is one of the most widely used open-source platforms for musculoskeletal simulation (Seth et al., 2018; Mokhtarzadeh et al., 2023; Bedo et al., 2021). OpenSim facilitates the dissemination of research within the SimTK community and is used extensively in academic and clinical research (Reinbolt et al., 2011; Mansouri and Reinbolt, 2012; Lee and Umberger, 2016; Blache et al., 2017; Nasiri et al., 2022; Willson et al., 2023; Gao et al., 2024; Zhang et al., 2024). It supports integration with skeletal models (Saul et al., 2005; Christophy et al., 2012; Arnold et al., 2010) and optimization algorithms (Seth et al., 2018; Dembia et al., 2021) as add-on components. Recent advancements, such as Myosuite (Caggiano et al., 2022) combined with Multi-Joint dynamics with Contact (MuJoCo) (Todorov et al., 2012) have further expanded the capabilities of musculoskeletal simulations.

OpenSim operates through two key layers: torque estimation and muscle activity estimation (see Supplementary Appendix SA for details on the muscle activation estimation procedure). The torque estimation layer accurately predicts joint torques by customizing musculoskeletal models based on user-specific physical characteristics and ground reaction forces (Koller et al., 2018; Saul et al., 2005; Christophy et al., 2012; Arnold et al., 2010). Conversely, the muscle activity estimation layer solves an inverse problem to derive muscle activations from joint torques. However, this inverse problem lacks a unique solution owing to the redundancy of muscle actuators in the OpenSim musculoskeletal model. OpenSim addresses this challenge by minimizing the sum of muscle activity to achieve local optimal solutions (Thelen et al., 2003); however, this often leads to significant discrepancies between the estimated and measured muscle activities.

Several sources of error in the estimation of muscle activity estimation using OpenSim have been identified. Modeling errors, particularly in muscle dynamics, are one major cause. For example, prior research (Hamner and Delp, 2013) observed a consistent time delay of approximately 75 ms between the estimated and measured muscle activities in running datasets. In addition, other studies (Nasiri et al., 2022; Gastaldi et al., 2021; Liu et al., 2008) reported shifts in the peak of muscle activity. Another critical source of error arises from the non-unique solutions to the inverse problem of determining muscle activity from joint torque owing to the redundancy of muscle actuators, which are employed to reproduce the human musculoskeletal system. OpenSim uses Computed Muscle Control (CMC) (Thelen et al., 2003) to solve this problem; however, its objective function, which minimizes activation while achieving the target torques, does not guarantee alignment with actual muscle activity patterns. To the best of our knowledge, no previous study has generically addressed the motion-to-EMG problem using a neural network, as musculoskeletal simulators do.

Based on these findings, this study hypothesizes that errors in muscle activity estimation stem from two primary factors: temporal mismatches and spatial redundancy in musculoskeletal models. As the results of previous studies have shown, the resolution of these errors is limited only by the conventional optimization of the model parameters. Therefore, as opposed to optimizing the musculoskeletal parameters, this study explores an alternative approach by introducing a compensation model to refine the OpenSim outputs.

The Neural-Enhanced Motion-to-EMG (NEM2E) framework (Figure 1) is introduced to address the above problems. It estimates realistic muscle activity from motion and ground reaction force data by incorporating the Spatio-Temporal Distortion Refinement Network (STDR-Net). Specifically, the STDR-Net refines the muscle activity outputs of OpenSim by compensating for spatio-temporal distortion. The network leverages a Sequence-to-Sequence (Seq2Seq) model (Sutskever et al., 2014), which is a recent development in natural language and image processing, enhanced with attention mechanisms (Luong et al., 2015). The framework implements Seq2Seq models with spatial and temporal attention mechanisms to investigate the spatial and temporal error contributions separately.

Figure 1

Figure 1. NEM2E framework. NEM2E refines the muscle activity estimated from motion and ground reaction force data using OpenSim, enhancing it with the STDR-Net.

The analysis was performed using the walk and running motion of elderly people and athletes, as well as extractable discrete motions such as baseball batting and golf swinging, which are commonly applied in sports and rehabilitation science. This study used two publicly available datasets (Liu et al., 2008; Hamner and Delp, 2013) that pair the muscle activity estimations of OpenSim with actual EMG data for lower-limb walking and running motions. In addition, the accuracy was tested on unknown subjects through cross-validation between subjects based on open data to enable cross-sectional measurement.

The contributions of this study are summarized as follows.

• The NEM2E framework is developed to enhance the realism of musculoskeletal model simulators.

• The study hypothesizes that discrepancies between simulated muscle activity and measured EMG data are owing to spatio-temporal distortions that are inherent in conventional musculoskeletal models.

• The STDR-Net is proposed using Seq2Seq and attention mechanisms to refine the muscle activity estimations.

• Validation is performed using public datasets, demonstrating significant accuracy improvements for all five muscles in running data and two out of five muscles in walking data.

By addressing a significant error source in musculoskeletal modeling, this study opens pathways for further innovations in refining biomechanical simulations and integrating neural networks into computational modeling.

The remainder of this paper is organized as follows: Section 2 reviews related studies, Section 3 describes the NEM2E framework, Section 4 explains the model learning and statistical analysis, Section 5 presents the validation results, Section 6 discusses the findings, and Section 7 concludes the paper.

2 Related works

One of the key roles of OpenSim is to estimate muscle activity. Thelen et al. (2003) demonstrated that integrating joint angular acceleration feedback into the muscle activity optimization routine (static optimization) of OpenSim yields muscle activity estimates with timing similar to EMG-based measurements. This method is implemented in OpenSim as CMC.

However, estimating muscle activity from joint torque remains an ill-posed problem owing to the redundancy of actuators in musculoskeletal models. Several studies have reported discrepancies between the simulated and experimental muscle activity, which have often been attributed to changes in muscle loading and the effects of assisted movement (Nasiri et al., 2022; Gastaldi et al., 2021).

Efforts to improve the accuracy of OpenSim musculoskeletal models have followed two main approaches:

2.1 Development of sophisticated models

The first approach involves creating more detailed models. For instance, studies have developed precise models of the lumbar spine, addressing areas that are not adequately represented by the base model of OpenSim (Christophy et al., 2012; Raabe and Chaudhari, 2016). Tools such as NMSBuilder developed by G. Valente et al. have also been introduced to aid researchers in developing customized OpenSim models (Valente et al., 2017). However, there are inherent limitations to the accuracy that is achievable in simulations owing to challenges in replicating the complexity of the human body. In addition, measurements such as CT scans are required to capture individual differences accurately, which may not be feasible for all users.

2.2 Optimization of model parameters

The second approach focuses on optimizing the model parameters. OpenSim moco (Dembia et al., 2021) introduces functions to calibrate parameters such as the muscle length and maximum tension using individual user data. Although this improves the personalization, discrepancies between the estimated muscle activity and measured EMG data persist. Valente et al. proposed robust optimization methods to address uncertainties in muscle models (Valente et al., 2014). However, parameter optimization alone cannot resolve fundamental modeling errors in musculoskeletal models.

2.3 Our concept

In contrast to previous studies that focused only on optimizing the parameters or models within the OpenSim framework, this study introduces the NEM2E framework. By addressing temporal and spatial modeling errors separately, the STDR-Net compensates for these errors using a Seq2Seq model with an attention mechanism. This approach has the potential to mitigate the limitations of conventional methods and improve the accuracy of muscle activity estimation.

3 Materials and methods

This section presents an overview of the methods and experimental setup.

3.1 Methods

NEM2E framework and STDR-Net.

3.1.1 NEM2E framework

The NEM2E framework refines the muscle activity estimations generated by OpenSim (Figure 1). It starts with the standard workflow of OpenSim, which includes 1) scaling (skeletal model parameter adjustment), 2) inverse kinematics (joint angle calculation), 3) residual reduction algorithm (joint torque estimation), and 4) CMC (muscle activity estimation) (see Supplementary Appendix SA for a detailed description). The resulting muscle activity data are then refined by STDR-Net to compensate for spatio-temporal distortions that are introduced by the OpenSim musculoskeletal model.

3.1.2 STDR-Net

STDR-Net addresses spatio-temporal distortions in OpenSim musculoskeletal models. Several time-series modeling techniques (e.g., linear regression, recurrent neural networks, long short-term memory (LSTM), and the Transformer) can be applied; however, this study implements STDR-Net using a Seq2Seq model with an attention mechanism (Luong et al., 2015) because this approach leverages attention layers to analyze the spatio-temporal relationships during data refinement. A schematic is shown in Figure 2. Two distinct models, (a) and (b), are constructed to analyze the temporal and spatial distortion, respectively.

Figure 2

Figure 2. STDR-Net. STDR-Net combines the Seq2Seq model and an attention mechanism that transforms time-series data ${\mathbf{y}}_s$ using encoders and decoders with LSTM. There are two types of attention mechanism: (a) an attention mechanism for temporal analysis and (b) an attention mechanism for spatial analysis.

These models use the muscle activity estimated by OpenSim as the input ${\mathbf{x}}_s$ and the measured real muscle activity as the output ${\mathbf{y}}_s$, and train a neural network (encoder ${\mathbf{h}}_s$, decoder ${\mathbf{h}}_t$, and attention ${\mathbf{a}}_t$) that compensates for the spatio-temporal distortions between the input and output. By analyzing the attention mechanism that is embedded in the network, the information that is used to correct the distortion can be revealed, and the contribution of spatio-temporal information to refining the estimated muscle activity can be verified. See Supplementary Appendix SB for details on the Seq2Seq with attention model.

3.2 Experiment

The proposed NEM2E framework was validated using two publicly available datasets (Liu et al., 2008): published in https://simtk.org/projects/mspeedwalksims and the datasets used in the literature (Hamner and Delp, 2013) published in https://simtk.org/projects/nmbl_running. The contents of the dataset are shown in Table 1 These datasets contain the estimated muscle activation results for which individual body parameter tuning and other standard tuning has already been performed in the respected studies.

Table 1

Table 1. Datasets.

3.2.1 Model learning

The parameters of the STDR-Net were trained using a dataset, where the simulated muscle activation from OpenSim served as the input and the muscle activation obtained from the measured EMG served as the output. The squared error was used as the loss function and the optimization process was carried out using the Adam algorithm (Kingma and Ba, 2015).

For all conditions, the STDR-Net was configured with 20 units corresponding to the number of muscle actuators in OpenSim for the encoder (input layer), 700 units for each intermediate layer, and 20 units for the decoder (output layer). Training was performed with a batch size of 1 and 700 epochs. The data were standardized by thinning the input and output data to 20 samples each. For the temporal attention models, the input dimension corresponded to the single dimension of the target muscle.

3.2.2 Statistical analysis

The performance of the proposed model was compared with the baseline muscle activity estimations of OpenSim. All EMG measurements in each dataset (running data: 16 muscles, walking data: 5 muscles) were verified. The names and abbreviations of each muscle are as follows: the soleus (soleus), biceps femoris long head (bi), semimembranosus (sem), tibialis anterior (tib), rectus femoris (rect), gluteus maximus (glmax1, glmax2, and glmax3), gluteus medius (glmed1, glmed2, and glmed3), gastrocnemius medial head (gasmed), gastrocnemius lateral head (gaslat), vastus lateralis (vaslat), and vastus medialis (vasmed) muscles. These names follow the column labels used in publicly available OpenSim muscle activity datasets and may differ from standard anatomical abbreviations. All data were segmented into individual gait cycles. A leave-one-out cross-validation approach was used, with data blocked by subject for both the walking and running datasets.

The accuracy of the model was assessed using the root mean squared error (RMSE) between the model output $\widehat{y}$ and measured EMG data $y$, which is calculated using the following (Equation 1):

$E=\sqrt{\frac{1}{N}{\displaystyle \sum _{i=0}^N}{\left(y_i-{\widehat{y}}_i\right)}^2},(1)$

where $N=20$ is the number of samples. Assuming that the error in the refined muscle activity obtained by the proposed method is $E_{\text{pro}}$ and the error in the estimated muscle activity obtained from OpenSim is $E_{\text{OpenSim}}$, the error improvement rate $Er$ is calculated as

$Er=\frac{E_{\text{pro}}}{E_{\text{OpenSim}}}-1.(2)$

Because each subject has data for four different speeds, the average of the four RMSE values is treated as the representative value for each subject. The null hypothesis $H_0:Er=0$ was tested using a T-test. The effect size is calculated using Cohen’s d in the following (Equation 3).

$d=\frac{E{r}_{\text{ave}}-{\mu }_0}{E{r}_{\text{std}}},(3)$

where, $E{r}_{\text{ave}}$ and $E{r}_{\text{std}}$ denote average and standard deviation of $Er$, respectively. ${\mu }_0$ is hypothesized population mean.

The 95% confidence interval (CI) is calculated using the following (Equation 4).

$CI=E{r}_{\text{ave}}\pm t_{\frac{\alpha }{2},n-1}\frac{E{r}_{\text{std}}}{\sqrt{n}},(4)$

where, $t_{\frac{\alpha }{2},n-1}$ represents the two-tailed t-value of $\frac{\alpha }{2}$ in the t-distribution with n-1 degrees of freedom. n is sample number. $\alpha$ is 0.05. Three input data patterns were defined for validation.

• One muscle, quarter cycle (One-Quarter): Sequence data for a quarter cycle of one muscle of the target muscle.

• One muscle, entire-cycle (One-Entire): Sequence data for an entire cycle of one muscle of the target muscle.

• All muscles, entire cycle (All-Entire): Sequence data for an entire cycle of all muscle sets in the musculoskeletal model.

One-Entire tested the temporal distortion compensation. Because the proposed framework emphasizes the importance of utilizing the entire sequence data to address temporal distortion, the effect of using only a portion of the sequence as input was examined in the One-Quarter model. All-Entire assessed the spatial distortion compensation by incorporating all muscle activity data. Temporal attention (Figure 2a) was used for One-Entire and One-Quarter, whereas spatial attention (Figure 2b) was applied to All-Entire. Tukey’s multiple comparison tests were performed for each muscle to analyze significant differences among the three input patterns.

4 Results

4.1 Refinement results of NEM2E

Figure 3 shows representative results from the NEM2E framework. The results for the One-Entire condition in the walking dataset are shown in (a) and the results for the All-Entire condition in the running dataset are shown in (b). The blue lines indicate simulated muscle activity from OpenSim, the red lines show the refined output from the STDR-Net, and the black lines represent measured EMG data. The horizontal axis denotes the gait cycle (right heel strike to left heel strike), whereas the vertical axis represents muscle activation. In all cases, the refined muscle activation (red) closely matched the measured EMG (black), demonstrating the ability of the framework to improve upon the baseline OpenSim estimates (blue).

Figure 3

Figure 3. Refined muscle activities with our proposed method. (a) shows the walking dataset results for the One-Entire condition. (b) shows the running dataset results for the All-Entire condition. The blue, red, and black lines represent simulated, refined, and measured muscle activity. The horizontal axis represents the gait cycle (right heel strike to left heel strike), whereas the vertical axis represents muscle activity.

4.2 Statistical analysis

Tables 2 and 3 summarize the statistical results for the improvement rates in the estimation accuracy for the walking and running datasets, respectively. $\ast$ indicates that the null hypothesis $H_0:Er=0$ is rejected in the estimation error improvement rate for each muscle calculated by Equation (2). In the walking data, significant improvements in accuracy were observed for the soleus, tib, and rect under the All-Entire condition. No significant improvements were observed for bi and sem under any conditions. In the running dataset, significant improvements were evident for all muscles in the All-Entire condition. Under the One-Quarter condition, only 4 out of the 16 muscles showed significant improvement, and under the One-Entire condition, 10 out of the 16 muscles demonstrated significant improvement.

Table 2

Table 2. Statistical analysis for walking dataset.

Table 3

Table 3. Statistical analysis for running dataset.

Figure 4 show the improvement rates in the estimation error for the walking and running datasets. A value of 0 represents the baseline accuracy, whereas negative values indicate improvement. In the walking dataset, there were no significant differences between conditions. Conversely, in the running data, significant differences were observed between One-Quarter and All-Entire in 13 out of 16 muscles as well as between One-Entire and All-Entire in bi and glmax2.

Figure 4

Figure 4. Comparison of estimation accuracy improvement. (a–c) show the results for the running dataset, and (d) shows the results for the walking dataset. The green, blue, and gray boxes represent the One-Entire, One-Quarter, and All-Entire conditions, respectively. The horizontal axis indicates target muscles. The vertical axis represents the error rate $Er$. Statistical significance is indicated by *p $<0.05$, **p $<0.01$, and ***p $<0.001$ (Tukey’s multiple comparisons test).

In the running dataset, significant improvements were observed in all muscles under the All-Entire condition. However, under the One-Entire condition, no significant improvements were observed in 6 out of the 16 muscles. Furthermore, in the walking dataset, significant improvements in the accuracy of the gastrocnemius muscle were observed under all conditions. The rectus femoris muscle showed improvements only under the All-Entire condition, suggesting that the spatial attention mechanism in this condition plays a crucial role in enhancing muscle activity estimation. Furthermore, under the One-Quarter condition, significant improvements were observed in only 5 out of 21 muscles across both running and walking data, indicating the need to consider the entire sequence rather than applying window processing to the sequence data. Moreover, as the models in this study were trained only on running and walking data, shortcut learning may have occurred, in which average muscle activity patterns were learned that are dependent on the movement duration. Then, for the One-Entire model trained on the soleus walk data, which showed strong improvement results in the validation, the improvement rate of the refined results was tested with a T-test by entering 32 randomly frequency- and phase-changed data, which was the same number of test trials in the walk dataset of the original paper. The average improvement rate (- indicates improvement and + indicates degradation) was +0.48, with a p-value of 0.001, showing significant degradation. Thus, the results demonstrate that shortcut learning, which is dependent on the operating time, did not occur.

Furthermore, Figure 5 shows the loss trends for the training and validation data across each learning trial. In the running data, as shown in Figure 5a, the validation loss converged, whereas, in the walking data, as shown in Figure 5b, the loss temporarily increased without decreasing and then converged. For the running data, the validation loss demonstrates a stable relationship with the training loss, consistently decreasing alongside it. This trend suggests that overfitting is unlikely for this dataset, even with the relatively high number of epochs (700). The observed generalization across individuals further indicates that the model is successfully learning the underlying patterns for this specific sequence task without memorizing the training data.

Figure 5

Figure 5. Transition of loss over trials (a) is the value of LOSS when learning the soleus data of the run. The blue line is the loss for the training data, and the orange line is the LOSS for the validation data. (b) is the result for the soleus of walking.

Conversely, for the walk data, the validation loss exhibits an increasing trend from early epochs. However, the validation loss remains stagnant and elevated from the outset, suggesting that the intrinsic complexity or inherent noise within this dataset may hinder the model’s ability to achieve substantial learning improvements from the beginning. This observation implies that for the walk data, increasing the dataset size is considered more critical for improving model performance and generalization than simply extending the training duration or adjusting architectural parameters.

4.3 Attention

Figure 6 depicts two representative examples of temporal attention weights in the One-Entire condition. In (I), the attention mechanism primarily focuses on the central part of the input sequence to refine the initial portion of the output waveform of the model. Conversely (II) highlights attention on the latter part of the input sequence, corresponding to the refinement of the second half of the output waveform.

Figure 6

Figure 6. Temporal attention results. (I) and (II) show two typical examples of temporal attention. (a) Attention weight matrix, with the color bar indicating the weight magnitude (red for higher weights and blue for lower weights). The horizontal axis represents the input sample index and the vertical axis shows the output sample index. (b) Muscle activity, where the horizontal axis is the time and the vertical axis is the muscle activity. The blue lines represent the estimated muscle activity of OpenSim, the red lines indicate the refined muscle activity, and the black lines show the measured EMG.

The temporal attention results (Figure 6) reveal that the model allocated higher weights to data near the center of the input sequence when refining the muscle activity. This suggests that the temporal distortions in the estimations of OpenSim can be effectively learned and corrected by the attention mechanism. However, the lack of improvement in the One-Quarter condition for certain muscles, such as the biceps femoris long head, underscores the need for longer temporal sequences to capture the full distortion. The fact that the attention was focused on the peak shift in the attention weight results suggests that the model learned as intended, correcting for temporal bias rather than movement duration.

Figure 7 depicts the spatial attention results for the tibialis anterior in the All-Entire condition. Each skeleton visualizes the gait cycle motions at 0%, 50%, and 100%, with the top 10 muscles having the highest attention weights. At 0% and 100%, corresponding to the heel strike phase, similar muscle groups with high attention weights were identified. At 50%, additional trunk and hip muscles were selected, reflecting a shift in the coordination required during the swing phase.

Figure 7

Figure 7. Spatial attention results. An example of spatial attention is shown. The vertical axis of the graph indicates the muscle activity channels and the horizontal axis represents the gait cycle. The color bar denotes the weight magnitude (yellow for higher weights and blue for lower weights). The skeleton models highlight the top 10 muscles with the highest motion and attention weights at 0%, 50%, and 100% of the gait cycle.

The spatial attention revealed that the top 10 muscles with the highest attention weights for the tibialis anterior, which exhibited improved accuracy only in the All-Entire condition, varied at 0%, 50%, and 100% of the gait cycle. This variation may be explained by differences in the active muscle groups during the stance and swing phases of gait.

In addition, the top 10 muscles with the highest attention weights included muscles from the trunk, hip, and lower back on the opposite leg, in addition to the peripheral muscles of the target muscle. The peripheral muscles may have complemented the information obtained through muscle synergy during gait. Regarding the involvement of the hip, pelvis, and trunk muscles of the opposite leg, previous studies have reported that the trailing limb angle is related to ankle moments during gait (Hsiao et al., 2016). This suggests that ankle torque is not solely generated by muscle activity in the target leg, but also involves coordinated actions with the trunk and opposite leg. Owing to the significant number of muscles associated with the hip and trunk, these regions likely act as a counterweight for ankle movement. The results suggest that incorporating information from these muscles improved the accuracy of the muscle activity estimations by accounting for the broader coordination required during gait.

5 Discussion

We have proposed the NEM2E framework, which corrects the spatio-temporal distortion between OpenSim estimated and real muscle activity. Within this framework, a refined model for estimated muscle activity was trained using Seq2Seq with attention based on open data, and the improvement in estimation accuracy for new users was verified.

The temporal attention results shown in Figure 5a indicate that attention is focused on the 50% gait cycle muscle activity from OpenSim to estimate the 0%–40% gait cycle muscle activity. This result suggests that the proposed method refines the approximately 75 ms temporal shift observed in some muscles, as reported by a previous study (Hamner and Delp, 2013) using the running dataset. Because the temporal attention results are generalized across individuals, the results support the previous study’s suggestion that some muscles exhibit a common time delay.

In this study, we specifically address the persistent issue of spatio-temporal distortion in OpenSim muscle activity estimations. Previous studies (Koller et al., 2018; Saul et al., 2005; Christophy et al., 2012; Arnold et al., 2010) have focused on model tuning within the OpenSim environment to improve the accuracy of musculoskeletal simulators. These include tuning body parameters, aligning joint torque with floor reaction force data, and accounting for uncertainty in tuning parameters, such as muscle length. However, our proposed NEM2E framework introduces a novel external refinement approach. By treating OpenSim outputs as initial estimates and subsequently correcting their spatio-temporal properties with a dedicated refinement network, we provide a complementary solution that enhances the accuracy of existing musculoskeletal models without modifying OpenSim’s core parameters. In some cases, the improvement in accuracy could not be confirmed in certain walk data. One possible explanation is the increased variation in spatio-temporal distortions due to individual differences in balance and control strategies during the double support phase of gait. This variation may have reduced the generalization performance of the model owing to the limited dataset. Because the loss in the verification data in Figure 5b does not decrease, it is necessary to increase the size of the learning data and verify its effectiveness. Increasing the dataset size could address this issue and improve the effectiveness of the model.

A limitation of this work is the use of open data for walking and running as the validation dataset. Therefore, this method can primarily be applied to discrete motions, and the accuracy of the model for entirely unknown motions cannot be guaranteed. However, as generalization to unknown users is allowed, the method can be widely used in the analysis of movements related to walking and running, such as walking analysis of the elderly and athletics. The framework can also be applied to other discrete motions, such as golf swinging and baseball batting and pitching. However, verification regarding motions other than walking and running is crucial. In the future, it is necessary to build a benchmark dataset with a wide range of motions and large volume of data to validate the proposed framework and design appropriate refinement models.

The observed improvements in estimation accuracy for new users, particularly in generalization performance, highlights the importance of explicitly addressing spatio-temporal distortions. These findings directly support the hypothesis that spatio-temporal distortions contribute significantly to errors in OpenSim’s estimations. Furthermore, the effectiveness of the Seq2Seq with attention architecture in isolating and correcting these errors indicates that data-driven models offer a promising means of enhancing the fidelity of physiologically informed models. By externally addressing these distortions through a refinement network, the proposed framework provides a promising approach for enhancing the accuracy and applicability of musculoskeletal models. As a refined model in this study, the Seq2Seq with attention model was used as an example implementation to analyze spatio-temporal distortions; however, more suitable models may exist for this design. If a sufficient dataset can be constructed to train and compare models, the proposed framework could incorporate methods that more accuratey refine the spatio-temporal distortions discussed in this study. As a future direction, if a sufficient dataset can be constructed, research on direct estimation of muscle activity using neural network methods may become feasible.

The code to train the public dataset and perform CV between subjects under the All-Entier condition of this paper is available at SimTK (https://simtk.org/projects/nem2e).

Data availability statement

Publicly available datasets were analyzed in this study. This data can be found here: https://simtk.org/projects/mspeedwalksims https://simtk.org/projects/nmbl_running.

Author contributions

TT: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Resources, Software, Validation, Visualization, Writing – original draft, Writing – review and editing. TM: Conceptualization, Methodology, Supervision, Writing – review and editing. TN: Conceptualization, Funding acquisition, Project administration, Writing – review and editing. JM: Conceptualization, Funding acquisition, Project administration, Writing – review and editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This research was supported by JST Moonshot R&D (Grant Number JPMJMS2034), NEDO (Grant Number JPNP20006), and JSPS KAKENHI (Grant Number 24K21325, 23K24925).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fbioe.2025.1611414/full#supplementary-material

References

Adachi, W., Tsujiuchi, N., Koizumi, T., Shiojima, K., Tsuchiya, Y., and Inoue, Y. (2011). Development of walking analysis system using by motion sensor with Mobile force plate. J. Syst. Des. Dyn. 6, 655–664. doi:10.1299/jsdd.6.655

CrossRef Full Text | Google Scholar

AnyBody (2025). AnyBody Modeling System. Aalborg, Denmark: AnyBody Technology. Available online at: https://www.anybodytech.com/.

Google Scholar

ARMO (2025). Healthcare Tech Solves Corporate and Social Issues and Improves Business Productivity. Tokyo, Japan: G-Sport Inc. Available online at: https://www.gsport.co.jp/products-2/armo/.

Google Scholar

Arnold, E. M., Ward, S. R., Lieber, R. L., and Delp, S. L. (2010). A model of the lower limb for analysis of human movement. Ann. Biomed. Eng. 38, 269–279. doi:10.1007/s10439-009-9852-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Bedo, B. L. S., Mantoan, A., Catelli, D. S., Cruaud, W., Reggiani, M., and Lamontagne, M. (2021). Bops: a matlab toolbox to batch musculoskeletal data processing for opensim. Comput. Methods Biomechanics Biomed. Eng. 24, 1104–1114. doi:10.1080/10255842.2020.1867978

PubMed Abstract | CrossRef Full Text | Google Scholar

Blache, Y., Begon, M., Michaud, B., Desmoulins, L., Allard, P., and Dal Maso, F. (2017). Muscle function in glenohumeral joint stability during lifting task. PLos One 12, e0189406. doi:10.1371/journal.pone.0189406

PubMed Abstract | CrossRef Full Text | Google Scholar

Caggiano, V., Wang, H., Durandau, G., Sartori, M., and Kumar, V. (2022). Myosuite: a contact-rich simulation suite for musculoskeletal motor control. In: R. Firoozi, N. Mehr, E. Yel, R. Antonova, J. Bohg, and M. Schwager, editors. Proceedings of the 4th annual learning for dynamics and control conference. New York, NY: Proceedings of Machine Learning Research, Vol. 168. p. 492–507.

Google Scholar

Christophy, M., Faruk Senan, N. A., Lotz, J. C., and O’Reilly, O. M. (2012). A musculoskeletal model for the lumbar spine. Biomechanics Model. Mechanobiol. 11, 19–34. doi:10.1007/s10237-011-0290-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Delp, S. L., Anderson, F. C., Arnold, A. S., Loan, P., Habib, A., John, C. T., et al. (2007). Opensim: open-Source software to create and analyze dynamic simulations of movement. IEEE Trans. Biomed. Eng. 54, 1940–1950. doi:10.1109/tbme.2007.901024

PubMed Abstract | CrossRef Full Text | Google Scholar

Dembia, C. L., Bianco, N. A., Falisse, A., Hicks, J. L., and Delp, S. L. (2021). Opensim moco: musculoskeletal optimal control. PLOS Comput. Biol. 16, e1008493–21. doi:10.1371/journal.pcbi.1008493

PubMed Abstract | CrossRef Full Text | Google Scholar

Dhaibaworks (2025). Preface. Tokyo, Japan: DhaibaWorks. Available online at: https://www.dhaibaworks.com/.

Google Scholar

Gao, X., Xu, D., Baker, J. S., Ee-Chon, T., Liang, M., and Gu, Y. (2024). Exploring biomechanical variations in ankle joint injuries among Latin dancers with different stance patterns: utilizing opensim musculoskeletal models. Front. Bioeng. Biotechnol. 12, 1359337. doi:10.3389/fbioe.2024.1359337

PubMed Abstract | CrossRef Full Text | Google Scholar

Gastaldi, L., Panero, E., Rosso, V., Pastorelli, S., Vieira, T., and Botter, A. (2021). Upper limbs musculoskeletal opensim model: customization and assessment. In: V. Niola, and A. Gasparetto, editors. Advances in Italian mechanism science. Cham: Springer International Publishing. p. 162–170.

CrossRef Full Text | Google Scholar

Hamner, S. R., and Delp, S. L. (2013). Muscle contributions to fore-aft and vertical body mass center accelerations over a range of running speeds. J. Biomechanics 46, 780–787. doi:10.1016/j.jbiomech.2012.11.024

PubMed Abstract | CrossRef Full Text | Google Scholar

Hamner, S. R., Seth, A., and Delp, S. L. (2010). Muscle contributions to propulsion and support during running. J. Biomech. 43, 2709–2716. doi:10.1016/j.jbiomech.2010.06.025

PubMed Abstract | CrossRef Full Text | Google Scholar

Hsiao, H., Knarr, B. A., Pohlig, R. T., Higginson, J. S., and Binder-Macleod, S. A. (2016). Mechanisms used to increase peak propulsive force following 12-weeks of gait training in individuals poststroke. J. Biomechanics 49, 388–395. doi:10.1016/j.jbiomech.2015.12.040

PubMed Abstract | CrossRef Full Text | Google Scholar

Kingma, D., and Ba, J. (2015). Adam: a method for stochastic optimization. In: International conference on learning representations (ICLR); 2015 May 7–9; San Diega, CA, USA. Ithaca, NY: ArXiv. p. 1–15.

Google Scholar

Koller, J. R., Remy, C. D., and Ferris, D. P. (2018). Biomechanics and energetics of walking in powered ankle exoskeletons using myoelectric control versus mechanically intrinsic control. J. Neuroeng Rehabil. 15, 42. doi:10.1186/s12984-018-0379-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, L.-F., and Umberger, B. R. (2016). Generating optimal control simulations of musculoskeletal movement using opensim and matlab. PeerJ 4, e1638. doi:10.7717/peerj.1638

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, M. Q., Anderson, F. C., Schwartz, M. H., and Delp, S. L. (2008). Muscle contributions to support and progression over a range of walking speeds. J. Biomechanics 41, 3243–3252. doi:10.1016/j.jbiomech.2008.07.031

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, T., Inoue, Y., and Shibata, K. (2010). A wearable force plate system for the continuous measurement of triaxial ground reaction force in biomechanical applications. Meas. Sci. Technol. 21, 085804. doi:10.1088/0957-0233/21/8/085804

CrossRef Full Text | Google Scholar

Luong, T., Pham, H., and Manning, C. D. (2015). Effective approaches to attention-based neural machine translation. In: L. Màrquez, C. Callison-Burch, and J. Su, editors. Proceedings of the 2015 conference on empirical methods in natural language processing. Lisbon, Portugal: Association for Computational Linguistics. p. 1412–1421.

Google Scholar

Mansouri, M., and Reinbolt, J. A. (2012). A platform for dynamic simulation and control of movement based on opensim and matlab. J. biomechanics 45, 1517–1521. doi:10.1016/j.jbiomech.2012.03.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Mokhtarzadeh, H., Jiang, F., Zhao, S., and Malekipour, F. (2023). Opencolab project: Opensim in google colaboratory to explore biomechanics on the web. Comput. Methods Biomechanics Biomed. Eng. 26, 1055–1063. doi:10.1080/10255842.2022.2104607

PubMed Abstract | CrossRef Full Text | Google Scholar

Nasiri, R., Aftabi, H., and Ahmadabadi, M. N. (2022). Human-in-the-loop weight compensation in upper limb wearable robots towards total muscles’ effort minimization. IEEE Robotics Automation Lett. 7, 3273–3278. doi:10.1109/lra.2022.3144519

CrossRef Full Text | Google Scholar

Ohashi, T., Ikegami, Y., and Nakamura, Y. (2020). Synergetic reconstruction from 2D pose and 3D motion for wide-space multi-person video motion capture in the wild. Image Vis. Comput. 104, 104028. doi:10.1016/j.imavis.2020.104028

CrossRef Full Text | Google Scholar

Opensim (2025). OpenSim Software Ecosystem. Houston, Texas: Opensim. Available online at: https://opensim.stanford.edu/.

Google Scholar

Raabe, M. E., and Chaudhari, A. M. (2016). An investigation of jogging biomechanics using the full-body lumbar spine model: model development and validation. J. Biomechanics 49, 1238–1243. doi:10.1016/j.jbiomech.2016.02.046

PubMed Abstract | CrossRef Full Text | Google Scholar

Reinbolt, J. A., Seth, A., and Delp, S. L. (2011). Simulation of human movement: applications using opensim. Procedia Iutam 2, 186–198. doi:10.1016/j.piutam.2011.04.019

CrossRef Full Text | Google Scholar

Saul, K., Murray, W., and Delp, S. (2005). A model of the upper extremity for simulating musculoskeletal surgery and analyzing neuromuscular control. Ann. Biomed. Eng. 33, 829–840. doi:10.1007/s10439-005-3320-7

CrossRef Full Text | Google Scholar

Seth, A., Hicks, J. L., Uchida, T. K., Habib, A., Dembia, C. L., Dunne, J. J., et al. (2018). Opensim: simulating musculoskeletal dynamics and neuromuscular control to study human and animal movement. PLOS Comput. Biol. 14, e1006223–20. doi:10.1371/journal.pcbi.1006223

PubMed Abstract | CrossRef Full Text | Google Scholar

SIMM (2025). Singapore: SIMM. Available online at: https://www.motionanalysis.com/videos/simm/.

Google Scholar

Sutskever, I., Vinyals, O., and Le, Q. V. (2014). Sequence to sequence learning with neural networks. Adv. neural Inf. Process. Syst. 27. doi:10.48550/arXiv.1409.3215

CrossRef Full Text | Google Scholar

Thelen, D. G., Anderson, F. C., and Delp, S. L. (2003). Generating dynamic simulations of movement using computed muscle control. J. Biomechanics 36, 321–328. doi:10.1016/s0021-9290(02)00432-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Todorov, E., Erez, T., and Tassa, Y. (2012). Mujoco: a physics engine for model-based control. In: 2012 IEEE/RSJ international conference on intelligent robots and systems; 2012 October 07–12; Vilamoura-Algarve, Portugal: IEEE. p. 5026–5033. doi:10.1109/IROS.2012.6386109

CrossRef Full Text | Google Scholar

Valente, G., Crimi, G., Vanella, N., Schileo, E., and Taddei, F. (2017). Nmsbuilder: freeware to create subject-specific musculoskeletal models for opensim. Comput. Methods. Prog. Biomed. 152, 85–92. doi:10.1016/j.cmpb.2017.09.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Valente, G., Pitto, L., Testi, D., Seth, A., Delp, S. L., Stagni, R., et al. (2014). Are subject-specific musculoskeletal models robust to the uncertainties in parameter identification? PLos One 9, e112625–10. doi:10.1371/journal.pone.0112625

PubMed Abstract | CrossRef Full Text | Google Scholar

Willson, A. M., Anderson, A. J., Richburg, C. A., Muir, B. C., Czerniecki, J., Steele, K. M., et al. (2023). Full body musculoskeletal model for simulations of gait in persons with transtibial amputation. Comput. Methods Biomechanics Biomed. Eng. 26, 412–423. doi:10.1080/10255842.2022.2065630

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, Z., Zou, J., Lu, P., Hu, J., Cai, Y., Xiao, C., et al. (2024). Analysis of lumbar spine loading during walking in patients with chronic low back pain and healthy controls: an opensim-based study. Front. Bioeng. Biotechnol. 12, 1377767. doi:10.3389/fbioe.2024.1377767

PubMed Abstract | CrossRef Full Text | Google Scholar