Abstract
Compliant and soft sensors that detect machinal deformations become prevalent in emerging soft robots for closed-loop feedback control. In contrast to conventional sensing applications, the stretchy body of the soft robot enables programmable actuating behaviors and automated manipulations across a wide strain range, which poses high requirements for the integrated sensors of customized sensor characteristics, high-throughput data processing, and timely decision-making. As various soft robotic sensors (strain, pressure, shear, etc.) meet similar challenges, in this perspective, we choose strain sensor as a representative example and summarize the latest advancement of strain sensor-integrated soft robotic design driven by machine learning techniques, including sensor materials optimization, sensor signal analyses, and in-sensor computing. These machine learning implementations greatly accelerate robot automation, reduce resource consumption, and expand the working scenarios of soft robots. We also discuss the prospects of fusing machine learning and soft sensing technology for creating next-generation intelligent soft robots.
Introduction
Emerging soft robots have attracted more and more attention in robotic research due to their easy body/shape transformation (Shepherd et al., 2011; Rus and Tolley, 2015), high degree of freedom (DOF) deformation (Rus and Tolley, 2018; Walsh, 2018), energy-efficient actuation (Hu et al., 2018; Kim et al., 2018), and safe human-machine interfaces (Laschi et al., 2016; Kim et al., 2019). Efficient and accurate feedback control is an essential function for soft robots to safely interact in dynamic environments (Thuruthel et al., 2019; Wang et al., 2020; Zhou et al., 2020). Conventional robotic control strategy also involves simulation and coaching, which depends on advanced algorithms or mathematical models. But for soft robots, due to the very high degree of freedom of their soft body, it is challenging to model the robotic dynamics mathematically, making the simulation and coaching not suitable. As a result, feedback control, which relays on the integrated soft sensors to back actual electronic signals in response to external stimulus, becomes prevalent in soft robotic research. Among various soft sensing technologies, the compliant strain sensor, as a representative sensing technology with the facile device fabrication and testing, is widely attached to the robotic body, enabling the robot itself to capture mechanical deformations and achieve closed-loop control (Mirvakili and Hunter, 2018; Zhang et al., 2019). Until now, diverse strain sensing components are integrated into soft robots to enable bio-mimicking dexterous manipulations, such as fish-like robot to explore deep ocean (Li et al., 2021), ultra-gentle gripper to capture fragile lives (Sinatra et al., 2019; Sundaram et al., 2019), cheetah-inspired soft robot with fast-moving (Tang et al., 2020), and soft human exoskeletons for rehabilitation (Zhao et al., 2016; Cianchetti et al., 2018), etc.
With the prosperity of soft robots, the soft robotic body with high DOF manipulations proposes high requirements of the integrated strain sensors. Firstly, soft robots often exhibit multimodal locomotion, whereas different robotic units show very different actuating behaviors with strain deformations ranging from <5% (Araromi et al., 2020) to >200% (Wang et al., 2018; Yang et al., 2020). To track the coordinated locomotion of a soft robot, multiple strain sensors with on-demand sensor characteristics are necessary for tracking separate robotic units with distinct kinematics (Mengüç et al., 2014; Yang et al., 2020), yet it is hindered by the huge resource consumption to validate a large number of design principles and trial-and-error experiments. Secondly, with the increasing number of integrated strain sensors, the mass of sensing data increases rapidly, leading to a heavy load of data storage, transmission, and processing, as well as the concern of high energy consumption (Faisal et al., 2019; Wang et al., 2020; Yang et al., 2020). Besides, to achieve the ultimate goal of an intelligent soft robot, timely decision-making is significant, which relays on local sensor data processing (i.e., in-sensor computing) rather than remote external computing devices (Zhou and Chai, 2020; Moin et al., 2021), yet is rarely explored in the soft robotic system. To push the boundary of strain sensor-integrated soft robots, it is significant to develop new principles to address the above issues of customized sensor characteristics, high-throughput data processing, and timely decision-making.
To address these challenges, data-driven exploration, prediction, and recommendation by machine learning (ML) provide an alternative approach (Schmidt et al., 2019; Xia et al., 2020). ML is a powerful tool that could create a model to uncover the unknown relations or principles in data-rich systems (Butler et al., 2018). Recently, ML experiences a boom in the investigation of various scientific areas including material science (Kitchin, 2018; Zahrt et al., 2019; Toyao et al., 2020), statistics (Yu and Kumbier, 2018), medicine (Ekins et al., 2019; Vamathevan et al., 2019), neuroscience (Savage, 2019), and machine automation (Zhou et al., 2022), where a large number of data points are provided to train an accurate ML model. Considering the inherently strong capabilities and recent advancements in handling data, including 1) uncovering the material/device characteristics with absent first principles, 2) parallel computing for high-throughput data, and 3) edge chip to achieve localized data processing, it is of high value to fuse ML technology for addressing robotic sensing challenges (Shih et al., 2020). Herein, we choose strain sensor as a representative example of robotic sensing technologies and briefly summarize the latest advancement of using ML techniques to design strain sensor-integrated soft robots, including sensor materials optimization, sensor signal analyses, and in-sensor computing. These ML implementations address the above challenges of customized sensor characteristics, high-throughput data processing, and timely decision-making, respectively. In the end, several prospects are also discussed. We expect this mini perspective could provide a brief but timely review of the current status and future directions.
ML for sensor material optimization
There are generally two types of strain sensors: piezoresistive sensor and capacitive sensor. Under mechanical deformations, the piezoresistive sensor shows resistance changes while the capacitive sensor demonstrates changes in capacitance value. Compared with the capacitive sensor that requires a sandwich structure, the piezoresistive sensor with a simple single-layer structure shows the advantages of facile fabrication and tight integration with the robotic body. As a result, at this very initial research stage, most ML techniques in strain sensor-integrated soft robots are implemented on the piezoresistive sensor. Therefore, piezoresistive sensor is selected as the main topic to review recent ML advancements.
A variety of intrinsically conductive materials ranging from metallic materials (e.g., gold foil (Yang et al., 2016), liquid metal (Lin et al., 2022)) to nanomaterials (e.g., fullerenes (Shi et al., 2018a), carbon nanotubes (Yamada et al., 2011; Sun et al., 2020), graphene (Wang et al., 2014; Yang et al., 2018), MXene (Yang et al., 2020; Saeidi-Javash et al., 2021; Cao et al., 2022)) have been adopted for the fabrication of piezoresistive strain sensors. The sensing mechanism is based on varying conductive pathways of the sensor layer under strains (e.g., in-plane crack propagation), leading to increased resistance as the electrical indicator. Two characteristics of a strain sensor are generally evaluated, including strain sensitivity and linear working window. The sensitivity of a strain sensor is characterized by gauge factor (GF), as defined in the equation below,where ε denotes the applied uniaxial strain, R0 and R represent the resistances before and after the applied strain, respectively. On the other hand, the linear working window of a strain sensor is determined by the strain range where the resistance of the sensor increased linearly with the applied strain.
The research experiences from past decades have revealed that the GF and linear working window of a strain sensor are highly correlated and balanced (Qiu et al., 2019; Afsarimanesh et al., 2020; Souri et al., 2020; Pei et al., 2021). A high GF generally exists within a narrow linear range, while a broad strain range is usually accompanied by a low GF (Amjadi et al., 2016; Shi et al., 2018b; Jayathilaka et al., 2019). This trade-off is caused by the different fracture behaviors of piezoresistive sensors: high GF comes from quick and drastic crack propagation under a narrow strain range, yet a broad linear working window requires slow and mild crack growth across a large strain. Based on these principles, many influential fabrication parameters including sensor material composition (Shi et al., 2018b; Cai et al., 2018), sensor shape/dimension (Jiang et al., 2018; Xin et al., 2020), and sensor topography (Yang et al., 2016; Pei et al., 2021) were proposed to engineer the sensor characteristics. For example, Shi et al. used fullerene as a lubricant agent to lower the friction between 2D graphene layers (Figure 1A). Under uniaxial strains, the graphene layer with lubricant experienced a mild fracture behavior instead of inducing long cracks to cut off the conductive pathways quickly, enabling the resulted strain sensor with a large linear working window (Shi et al., 2018a). Besides, X. Chen’s group designed an auxetic mechanical structure to improve the sensitivity (Jiang et al., 2018). As shown in Figure 1B, compared with conventional flat film, the auxetic structure showed a negative Poisson’s ratio and underwent 2D expansion in both parallel and perpendicular directions. Such auxetic metamaterials were incorporated into stretchable strain sensors to significantly enhance the crack density under strains and improved the sensitivity by up to 24-fold. In addition, P.-Y. Chen’s group demonstrated a hierarchical topography of MXene sensor with micro-crumples/wrinkles (Figure 1C) to regulate the in-plane crack propagation, which could control the crack growth in a programmable manner and realized highly tunable strain sensor characteristics (Yang et al., 2020).
FIGURE 1
Through various design principles, as discussed by multiple important review articles (Amjadi et al., 2016; Jayathilaka et al., 2019; Qiu et al., 2019; Afsarimanesh et al., 2020), the state-of-the-art research studies focus on developing high-performance strain sensor that shows high sensitivity (GF > 100) and large linear working window (>100%), simultaneously. With the great progress in recent years, a lot of strain sensors with outstanding sensor characteristics were created (Tao et al., 2017; Shi et al., 2018a; Shi et al., 2018b; Cai et al., 2018; Jiang et al., 2018; Lee et al., 2019; Li and Du, 2019; Yang et al., 2020; Zhang et al., 2021), which gradually fulfilled the above challenging region. With this regard, instead of continuing to pursue high sensor performance, it is time to move to the next stage of developing strain sensors with customized sensor characteristics for desired applications such as soft robots.
Nevertheless, due to the complex relationship between sensing materials and sensor characteristics, the exploration efficiency is greatly limited through conventional trial-and-error experiments. Under this condition, ML tools show a unique opportunity to underlay the unclear relations, which greatly reduces time, manpower, and resource consumption (Butler et al., 2018; Wei et al., 2019). Recently, H. Yang et al. proposed an ML framework that enabled automatic strain sensor design across a wide strain range from <0.5% to 350% (Yang et al., 2022). As shown in Figure 1D, there was a three-stage machine learning process, including boundary definition, active learning, and data augmentation. Firstly, multiple influential variables of varying material components, varying sensor thicknesses, and varying sensor morphologies were included. Adequate delamination tests were conducted to examine the fabrication feasibility which was used to train a support vector machine (SVM) classifier that only allowed the fabrication recipes with high successful chances to be suggested by the model. Secondly, with the guidance of active learning, an increasing number of strain sensors were stagewise fabricated to serve as the representative dataset to enrich the multi-dimensional dataset of the ML model. Finally, data augmentation methods were implemented to create more than 10,000 virtual data points to cultivate the prediction model. After the three-stage process, the ultimate prediction model was capable of conducting two-way tasks of automatic strain sensor design for diverse soft robots, including 1) high-accuracy prediction of sensor characteristics based on proposed fabrication recipes and 2) suggestion of feasible fabrication recipes for interested strain ranges. In addition, by implementing statistical analyses based on the collected data points, the underlying sensing mechanisms were revealed that dictated data-driven design principles.
ML for sensor signal analyses
Besides the on-demand sensor characteristics, soft robots also face difficulties in processing large amounts of data generated from multiple sensors on separate robotic units (Shih et al., 2020). For example, when applying wearable robots to track full-body human motions, more than 70 sensors are needed to match various body joints from shoulders to toe. Also, to achieve high sensing resolution, S. Sundaram et al. showed a knitted soft glove with 548 sensors (Sundaram et al., 2019). Such a large number of integrated sensors lead to a heavy load in multi-channeled sensor data collection, transmission, and processing (Zhou and Chai, 2020). Emerging approaches adopt wireless technology such as Bluetooth or near-field communication modules to facilitate data collection and wireless transmission (Lin et al., 2020; Yang et al., 2020; Jeong et al., 2021; Lin et al., 2022), yet they failed in the aspect of signal analyses due to the absence of computing unit.
With the rapid development of mobile computing platforms like Raspberry Pi, an emerging strategy is to connect sensors, wireless modules, and ML-embedded computing hardware together to establish a comprehensive sensor platform (Ha et al., 2020; Wang et al., 2021). By introducing ML tools, without manual data calibration, one can adopt the well-known algorithms from the open-source library (e.g., pandas, scikit-learn, PyTorch, TensorFlow) to pre-process the sensor signals easily such as de-noising, filtering, and grouping (Manie et al., 2020; Zhou et al., 2020). On the other hand, powerful machine learning neural networks enable the ML model to solve more complex tasks like signal pattern recognition. For example, X. Chen group adopted deep convolutional neural networks to analyze the strain sensor signals from five fingers of a human hand to distinguish different hand gestures (Wang et al., 2020). In addition, by combining tactile sensing information and a multilayer perceptron model, G, Li et al. developed a smart robotic hand that could precisely recognize different shapes, sizes, and materials in a diverse set of garbage (Figure 1E) (Li et al., 2020). On the other hand, recurrent neural network is often used as a powerful architecture to process variables along a temporal sequence like speech recognition (Jung et al., 2020).
Besides executing a single task, multiple ML models and computing tools with different functions could be assembled and work together to satisfy the diverse needs of soft robots. For instance, J. Chen group recently reported a wearable sign-to-speech translation system based on a soft glove with stretchable strain sensor arrays (Zhou et al., 2020), which involved a three-step computing process including signal pre-processing (the generated sensor signals are merged into a matrix), feature extraction (by principal component analysis), and speech translation (by multi-class SVM classifier). In addition, as shown in Figure 1F, Y. Luo et al. reported an ML-powered sensing textile suit which had three models with different network architectures to cover self-supervised calibration, interaction identification, and full-body motion prediction, respectively (Luo et al., 2021).
ML for in-sensor computing
The integration of sensors, wireless modules, and ML-embedded computing devices ensured facile sensor data transmission and processing, but with the increasing number of sensors, continuous data steaming raises high concerns of bandwidth requirement, graphics processing units expense, and power consumption (Zhou and Chai, 2020; Iwendi et al., 2021). On the other hand, wireless communication usually faces interruption and disconnection challenges, especially for soft robotic navigation in distant fields or underwater scenarios (Shih et al., 2019; Pillai and Lohani, 2020). Additionally, the radio frequency communication between wireless hardware and external computing devices faces the inevitable risks of data security (Lin et al., 2020).
An innovative approach to address the above difficulties is to process the real-time sensor data locally to achieve in-sensor computing without steaming sensor data out (Zhou and Chai, 2020; Moin et al., 2021). The goal of in-sensor computing is to perform both data processing and decision-making tasks locally to achieve higher levels of robotic autonomy with onboard machine intelligence, without further analyses or external computing modules. For instance, J. M. Rabaey group recently proposed an in-sensor adaptive machine learning framework for hand gesture recognition (Figure 1G) (Moin et al., 2021). In particular, they implemented a neuro-inspired hyperdimensional computing algorithm that could analyze sensor data locally for real-time hand gesture classification, as well as automatic model training and updating under variable conditions such as different arm positions and sensor replacement. That report well filled the gap of self-supervised model training and updating during continuous usage, enabling optimal model performance under long-term operations. On the other hand, S. Ma et al. developed a neuromorphic computing chip of TianjicX, which enabled multiple neural network models to be executed locally (Ma et al., 2022). As shown in Figure 1H, by equipping with TianjicX and multimodal sensors, without steaming sensor data out and external computing devices, a robotic car is able to simultaneously process multisensory information locally for diverse tasks, including object recognition, obstacle avoidance, and decision-making. In comparison with the commercial NVIDIA Jetson TX2 (ML-embedded computing hardware developed by NVIDIA), the developed TianjicX chip showed less latency and 50% lower power consumption.
Prospects
Fusing ML and soft sensing technology is an emerging multidisciplinary field and shows great potential for soft robots (see a summary in Table 1). But there is still a long way to achieve the ultimate goal of the intelligent soft robot, and several viewpoints are discussed here. Firstly, the sensing capability of soft robot is not limited to body strain deformation to estimate the kinematics of the robot itself but also should include more functions such as contact modeling (by pressure sensor) and mapping of the surroundings (by chemical and biological sensors). Therefore, important progress is expected in integrating ML technology with multiple types of sensors (e.g., strain, pressure, chemical, biomolecule) to create a multifunctional soft robot with comprehensive perception capability. We believe these ML techniques, including sensor material optimization, sensor signal analyses, and in-sensor computing, are also workable for other types of soft sensors, as they meet similar challenges to realize the goal of intelligent soft robot. On the other hand, for ML-assisted sensor material optimization, although some promising examples have been identified, we envisioned that critical challenges still exist at this early stage, such as the lack of sufficient high-quality datasets and absent evaluation standards to fairly compare the developed models. Besides, considering different kind of sensor types, their unique sensing mechanisms and structures may lead to different ML implementations yet is rarely studied, which would be an important future research direction.
TABLE 1
| ML technique | References(s) | Advantages | Disadvantages |
|---|---|---|---|
| Sensor material optimization | Yang et al. (2022)Nat. Mach. Intell. | ▪ Accurate prediction of sensor characteristics | ▪ Manual device fabrication and test |
| ▪ Automatic sensor design for diverse soft robots | ▪ Manual signal processing | ||
| Sensor signal analyses | Li et al. (2020)Sci. Robot. | ▪ Wireless transmission and good mobility | ▪ High bandwidth requirement |
| ▪ Multi-channeled data processing | ▪ Signal interruption and disconnection | ||
| Luo et al. (2021)Nat. Electron. | ▪ Multiple tasks implementation | ▪ Energy consumption | |
| ▪ Data security | |||
| In-sensor computing | Moin et al. (2021)Nat. Electron. | ▪ Less signal latency | ▪ Separated sensor and computing units |
| ▪ Real-time response | ▪ Mismatch of mechanical modulus between rigid chips and soft robots | ||
| Ma et al. (2022)Sci. Robot. | ▪ Power efficiency | ▪ Circuit miniaturization | |
| ▪ High data security |
Summary of ML implementations for strain sensor-integrated soft robots.
Secondly, many recent advances are conceptual studies to examine the feasibility of ML tools for soft robots yet lack the assessments of system robustness in practical conditions. For example, both compliant sensors and soft robotic bodies consist of soft matters, which will experience stress fatigue under long-term repeated mechanical deformations. This kind of phenomenon is commonly seen in robotics and affects the sensor signal and body kinematics a lot. It remains to explore suitable ML frameworks to ensure accurate feedback and control under such mechanical coupling conditions. Also, many other powerful ML algorithms like active learning, meta-learning, and reinforcement learning are waiting to be fully utilized for guiding soft robots to complete more challenging missions like automatic navigation or swarm under dynamic, unstructured, and even unknown environments.
At last, module integration and miniaturization are always pursued. For soft robotic applications, the separated sensors and computing modules inevitably increase the overall weight, cause signal deviation, and affect robotic locomotion. Therefore, the tight integration of multiple sensors, data collection/transmission modules, and computing units as well as the subsequent miniaturized circuit design are desired to achieve the goals of creating light, compliant, power-efficient, and eventually intelligent robots with multifunctionality. Moreover, it is expected to see potential innovations in constructing the data processing and computing modules with total soft materials, which would reduce the design restrictions of soft robots and expand their application scenarios.
Statements
Author contributions
All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.
Acknowledgments
We are grateful for the financial support of the National Natural Science Foundation of China (52103228, 22075199).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1
AfsarimaneshN.NagA.SarkarS.SabetG. S.HanT.MukhopadhyayS. C. (2020). A review on fabrication, characterization and implementation of wearable strain sensors. Sensors Actuators A Phys.315, 112355. 10.1016/j.sna.2020.112355
2
AmjadiM.KyungK. U.ParkI.SittiM. (2016). Stretchable, skin‐mountable, and wearable strain sensors and their potential applications: A review. Adv. Funct. Mat.26, 1678–1698. 10.1002/adfm.201504755
3
AraromiO. A.GrauleM. A.DorseyK. L.CastellanosS.FosterJ. R.HsuW.-H.et al (2020). Ultra-sensitive and resilient compliant strain gauges for soft machines. Nature587, 219–224. 10.1038/s41586-020-2892-6
4
ButlerK. T.DaviesD. W.CartwrightH.IsayevO.WalshA. (2018). Machine learning for molecular and materials science. Nature559, 547–555. 10.1038/s41586-018-0337-2
5
CaiY.ShenJ.GeG.ZhangY.JinW.HuangW.et al (2018). Stretchable Ti3C2TX MXene/carbon nanotube composite based strain sensor with ultrahigh sensitivity and tunable sensing range. ACS Nano12, 56–62. 10.1021/acsnano.7b06251
6
CaoY.GuoY.ChenZ.YangW.LiK.HeX.et al (2022). Highly sensitive self-powered pressure and strain sensor based on crumpled MXene film for wireless human motion detection. Nano Energy92, 106689. 10.1016/j.nanoen.2021.106689
7
CianchettiM.LaschiC.MenciassiA.DarioP. (2018). Biomedical applications of soft robotics. Nat. Rev. Mat.3, 143–153. 10.1038/s41578-018-0022-y
8
EkinsS.PuhlA. C.ZornK. M.LaneT. R.RussoD. P.KleinJ. J.et al (2019). Exploiting machine learning for end-to-end drug discovery and development. Nat. Mat.18, 435–441. 10.1038/s41563-019-0338-z
9
FaisalA. I.MajumderS.MondalT.CowanD.NasehS.DeenM. J. (2019). Monitoring methods of human body joints: State-of-the-Art and research challenges. Sensors19, 2629. 10.3390/s19112629
10
HaN.XuK.RenG.MitchellA.OuJ. Z. (2020). Machine learning‐enabled smart sensor systems. Adv. Intell. Syst.2, 2000063. 10.1002/aisy.202000063
11
HuW.LumG. Z.MastrangeliM.SittiM. (2018). Small-scale soft-bodied robot with multimodal locomotion. Nature554, 81–85. 10.1038/nature25443
12
IwendiC.MaddikuntaP. K. R.GadekalluT. R.LakshmannaK.BashirA. K.PiranM. J. (2021). A metaheuristic optimization approach for energy efficiency in the IoT networks. Softw. Pract. Exper.51, 2558–2571. 10.1002/spe.2797
13
JayathilakaW. a. D. M.QiK.QinY.ChinnappanA.Serrano‐GarcíaW.BaskarC.et al (2019). Significance of nanomaterials in wearables: A review on wearable actuators and sensors. Adv. Mat.31, 1805921. 10.1002/adma.201805921
14
JeongH.KwakS. S.SohnS.LeeJ. Y.LeeY. J.O’brienM. K.et al (2021). Miniaturized wireless, skin-integrated sensor networks for quantifying full-body movement behaviors and vital signs in infants. Proc. Natl. Acad. Sci. U. S. A.118, e2104925118. 10.1073/pnas.2104925118
15
JiangY.LiuZ.MatsuhisaN.QiD.LeowW. R.YangH.et al (2018). Auxetic mechanical metamaterials to enhance sensitivity of stretchable strain sensors. Adv. Mat.30, 1706589. 10.1002/adma.201706589
16
JungY. H.HongS. K.WangH. S.HanJ. H.PhamT. X.ParkH.et al (2020). Flexible piezoelectric acoustic sensors and machine learning for speech processing. Adv. Mat.32, 1904020. 10.1002/adma.201904020
17
KimY.ParadaG. A.LiuS.ZhaoX. (2019). Ferromagnetic soft continuum robots. Sci. Robot.4, eaax7329. 10.1126/scirobotics.aax7329
18
KimY.YukH.ZhaoR.ChesterS. A.ZhaoX. (2018). Printing ferromagnetic domains for untethered fast-transforming soft materials. Nature558, 274–279. 10.1038/s41586-018-0185-0
19
KitchinJ. R. (2018). Machine learning in catalysis. Nat. Catal.1, 230–232. 10.1038/s41929-018-0056-y
20
LaschiC.MazzolaiB.CianchettiM. (2016). Soft robotics: technologies and systems pushing the boundaries of robot abilities. Sci. Robot.1, eaah3690. 10.1126/scirobotics.aah3690
21
LeeJ.PyoS.KwonD. S.JoE.KimW.KimJ. (2019). Ultrasensitive strain sensor based on separation of overlapped carbon nanotubes. Small15, 1805120. 10.1002/smll.201805120
22
LiG.ChenX.ZhouF.LiangY.XiaoY.CaoX.et al (2021). Self-powered soft robot in the mariana trench. Nature591, 66–71. 10.1038/s41586-020-03153-z
23
LiG.LiuS.WangL.ZhuR. (2020). Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition. Sci. Robot.5, eabc8134. 10.1126/scirobotics.abc8134
24
LiH.DuZ. (2019). Preparation of a highly sensitive and stretchable strain sensor of MXene/silver nanocomposite-based yarn and wearable applications. ACS Appl. Mat. Interfaces11, 45930–45938. 10.1021/acsami.9b19242
25
LinR.KimH.-J.AchavananthadithS.KurtS. A.TanS. C.YaoH.et al (2020). Wireless battery-free body sensor networks using near-field-enabled clothing. Nat. Commun.11, 444–510. 10.1038/s41467-020-14311-2
26
LinR.KimH.-J.AchavananthadithS.XiongZ.LeeJ. K.KongY. L.et al (2022). Digitally-embroidered liquid metal electronic textiles for wearable wireless systems. Nat. Commun.13, 2190–2210. 10.1038/s41467-022-29859-4
27
LuoY.LiY.SharmaP.ShouW.WuK.FosheyM.et al (2021). Learning human–environment interactions using conformal tactile textiles. Nat. Electron.4, 193–201. 10.1038/s41928-021-00558-0
28
MaS.PeiJ.ZhangW.WangG.FengD.YuF.et al (2022). Neuromorphic computing chip with spatiotemporal elasticity for multi-intelligent-tasking robots. Sci. Robot.7, eabk2948. 10.1126/scirobotics.abk2948
29
ManieY. C.LiJ.-W.PengP.-C.ShiuR.-K.ChenY.-Y.HsuY.-T. (2020). Using a machine learning algorithm integrated with data de-noising techniques to optimize the multipoint sensor network. Sensors20, 1070. 10.3390/s20041070
30
MengüçY.ParkY.-L.PeiH.VogtD.AubinP. M.WinchellE.et al (2014). Wearable soft sensing suit for human gait measurement. Int. J. Rob. Res.33, 1748–1764. 10.1177/0278364914543793
31
MirvakiliS. M.HunterI. W. (2018). Artificial muscles: Mechanisms, applications, and challenges. Adv. Mat.30, 1704407. 10.1002/adma.201704407
32
MoinA.ZhouA.RahimiA.MenonA.BenattiS.AlexandrovG.et al (2021). A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition. Nat. Electron.4, 54–63. 10.1038/s41928-020-00510-8
33
PeiY.ZhangX.HuiZ.ZhouJ.HuangX.SunG.et al (2021). Ti3C2TX MXene for sensing applications: Recent progress, design principles, and future perspectives. ACS Nano15, 3996–4017. 10.1021/acsnano.1c00248
34
PillaiR. R.LohaniR. B. (2020). “Emergency data detection using hidden markov model during temporary disconnection of wireless body area networks,” in 2020 International Conference for Emerging Technology (INCET), Belgaum, India, Jun 5-7, 1–5.
35
QiuA.LiP.YangZ.YaoY.LeeI.MaJ. (2019). A path beyond metal and silicon: Polymer/nanomaterial composites for stretchable strain sensors. Adv. Funct. Mat.29, 1806306. 10.1002/adfm.201806306
36
RusD.TolleyM. T. (2018). Design, fabrication and control of origami robots. Nat. Rev. Mat.3, 101–112. 10.1038/s41578-018-0009-8
37
RusD.TolleyM. T. (2015). Design, fabrication and control of soft robots. Nature521, 467–475. 10.1038/nature14543
38
Saeidi-JavashM.DuY.ZengM.WyattB. C.ZhangB.KempfN.et al (2021). All-printed MXene–graphene nanosheet-based bimodal sensors for simultaneous strain and temperature sensing. ACS Appl. Electron. Mat.3, 2341–2348. 10.1021/acsaelm.1c00218
39
SavageN. (2019). How AI and neuroscience drive each other forwards. Nature571, S15–S17. 10.1038/d41586-019-02212-4
40
SchmidtJ.MarquesM. R. G.BottiS.MarquesM. a. L. (2019). Recent advances and applications of machine learning in solid-state materials science. npj Comput. Mat.5, 83. 10.1038/s41524-019-0221-0
41
ShepherdR. F.IlievskiF.ChoiW.MorinS. A.StokesA. A.MazzeoA. D.et al (2011). Multigait soft robot. Proc. Natl. Acad. Sci. U. S. A.108, 20400–20403. 10.1073/pnas.1116564108
42
ShiX.LiuS.SunY.LiangJ.ChenY. (2018a). Lowering internal friction of 0D–1D–2D ternary nanocomposite‐based strain sensor by fullerene to boost the sensing performance. Adv. Funct. Mat.28, 1800850. 10.1002/adfm.201800850
43
ShiX.WangH.XieX.XueQ.ZhangJ.KangS.et al (2018b). Bioinspired ultrasensitive and stretchable MXene-based strain sensor via nacre-mimetic microscale “brick-and-mortar” architecture. ACS Nano13, 649–659. 10.1021/acsnano.8b07805
44
ShihB.ShahD.LiJ.ThuruthelT. G.ParkY.-L.IidaF.et al (2020). Electronic skins and machine learning for intelligent soft robots. Sci. Robot.5, eaaz9239. 10.1126/scirobotics.aaz9239
45
ShihY.HsiuP.PangA. (2019). A data parasitizing scheme for effective health monitoring in wireless body area networks. IEEE Trans. Mob. Comput.18, 13–27. 10.1109/TMC.2018.2830779
46
SinatraN. R.TeepleC. B.VogtD. M.ParkerK. K.GruberD. F.WoodR. J. (2019). Ultragentle manipulation of delicate structures using a soft robotic gripper. Sci. Robot.4, eaax5425. 10.1126/scirobotics.aax5425
47
SouriH.BanerjeeH.JusufiA.RadacsiN.StokesA. A.ParkI.et al (2020). Wearable and stretchable strain sensors: Materials, sensing mechanisms, and applications. Adv. Intell. Syst.2, 2000039. 10.1002/aisy.202000039
48
SunX.QinZ.YeL.ZhangH.YuQ.WuX.et al (2020). Carbon nanotubes reinforced hydrogel as flexible strain sensor with high stretchability and mechanically toughness. Chem. Eng. J.382, 122832. 10.1016/j.cej.2019.122832
49
SundaramS.KellnhoferP.LiY.ZhuJ.-Y.TorralbaA.MatusikW. (2019). Learning the signatures of the human grasp using a scalable tactile glove. Nature569, 698–702. 10.1038/s41586-019-1234-z
50
TangY.ChiY.SunJ.HuangT.-H.MaghsoudiO. H.SpenceA.et al (2020). Leveraging elastic instabilities for amplified performance: Spine-inspired high-speed and high-force soft robots. Sci. Adv.6, eaaz6912. 10.1126/sciadv.aaz6912
51
TaoL.-Q.WangD.-Y.TianH.JuZ.-Y.LiuY.PangY.et al (2017). Self-adapted and tunable graphene strain sensors for detecting both subtle and large human motions. Nanoscale9, 8266–8273. 10.1039/C7NR01862B
52
ThuruthelT. G.ShihB.LaschiC.TolleyM. T. (2019). Soft robot perception using embedded soft sensors and recurrent neural networks. Sci. Robot.4, eaav1488. 10.1126/scirobotics.aav1488
53
ToyaoT.MaenoZ.TakakusagiS.KamachiT.TakigawaI.ShimizuK.-I. (2020). Machine learning for catalysis informatics: Recent applications and prospects. ACS Catal.10, 2260–2297. 10.1021/acscatal.9b04186
54
VamathevanJ.ClarkD.CzodrowskiP.DunhamI.FerranE.LeeG.et al (2019). Applications of machine learning in drug discovery and development. Nat. Rev. Drug Discov.18, 463–477. 10.1038/s41573-019-0024-5
55
WalshC. (2018). Human-in-the-Loop development of soft wearable robots. Nat. Rev. Mat.3, 78–80. 10.1038/s41578-018-0011-1
56
WangH.TotaroM.BeccaiL. (2018). Toward perceptive soft robots: Progress and challenges. Adv. Sci.5, 1800541. 10.1002/advs.201800541
57
WangM.WangT.LuoY.HeK.PanL.LiZ.et al (2021). Fusing stretchable sensing technology with machine learning for human–machine interfaces. Adv. Funct. Mat.31, 2008807. 10.1002/adfm.202008807
58
WangM.YanZ.WangT.CaiP.GaoS.ZengY.et al (2020). Gesture recognition using a bioinspired learning architecture that integrates visual data with somatosensory data from stretchable sensors. Nat. Electron.3, 563–570. 10.1038/s41928-020-0422-z
59
WangY.WangL.YangT.LiX.ZangX.ZhuM.et al (2014). Wearable and highly sensitive graphene strain sensors for human motion monitoring. Adv. Funct. Mat.24, 4666–4670. 10.1002/adfm.201400379
60
WeiJ.ChuX.SunX. Y.XuK.DengH. X.ChenJ.et al (2019). Machine learning in materials science. InfoMat1, 338–358. 10.1002/inf2.12028
61
XiaB.MiriyevA.TrujilloC.ChenN.CartolanoM.VartakS.et al (2020). Improving the actuation speed and multi-cyclic actuation characteristics of silicone/ethanol soft actuators. Actuators9, 62. 10.3390/act9030062
62
XinM.LiJ.MaZ.PanL.ShiY. (2020). MXenes and their applications in wearable sensors. Front. Chem.8, 297. 10.3389/fchem.2020.00297
63
YamadaT.HayamizuY.YamamotoY.YomogidaY.Izadi-NajafabadiA.FutabaD. N.et al (2011). A stretchable carbon nanotube strain sensor for human-motion detection. Nat. Nanotechnol.6, 296–301. 10.1038/nnano.2011.36
64
YangH.LiJ.LimK. Z.PanC.Van TruongT.WangQ.et al (2022). Automatic strain sensor design via active learning and data augmentation for soft machines. Nat. Mach. Intell.4, 84–94. 10.1038/s42256-021-00434-8
65
YangH.XiaoX.LiZ.LiK.ChengN.LiS.et al (2020). Wireless Ti3C2TX MXene strain sensor with ultrahigh sensitivity and designated working windows for soft exoskeletons. ACS Nano14, 11860–11875. 10.1021/acsnano.0c04730
66
YangT.LiX.JiangX.LinS.LaoJ.ShiJ.et al (2016). Structural engineering of gold thin films with channel cracks for ultrasensitive strain sensing. Mat. Horiz.3, 248–255. 10.1039/C6MH00027D
67
YangZ.PangY.HanX.-L.YangY.LingJ.JianM.et al (2018). Graphene textile strain sensor with negative resistance variation for human motion detection. ACS Nano12, 9134–9141. 10.1021/acsnano.8b03391
68
YuB.KumbierK. (2018). Artificial intelligence and statistics. Front. Inf. Technol. Electron. Eng.19, 6–9. 10.1631/FITEE.1700813
69
ZahrtA. F.HenleJ. J.RoseB. T.WangY.DarrowW. T.DenmarkS. E. (2019). Prediction of higher-selectivity catalysts by computer-driven workflow and machine learning. Science363, eaau5631. 10.1126/science.aau5631
70
ZhangH.LiuD.LeeJ.-H.ChenH.KimE.ShenX.et al (2021). Anisotropic, wrinkled, and crack-bridging structure for ultrasensitive, highly selective multidirectional strain sensors. Nano-Micro Lett.13, 122–215. 10.1007/s40820-021-00615-5
71
ZhangJ.ShengJ.O’neillC. T.WalshC. J.WoodR. J.RyuJ.et al (2019). Robotic artificial muscles: Current progress and future perspectives. IEEE Trans. Robot.35, 761–781. 10.1109/TRO.2019.2894371
72
ZhaoH.O’brienK.LiS.ShepherdR. F. (2016). Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides. Sci. Robot.1, eaai7529. 10.1126/scirobotics.aai7529
73
ZhouF.ChaiY. (2020). Near-sensor and in-sensor computing. Nat. Electron.3, 664–671. 10.1038/s41928-020-00501-9
74
ZhouX.WenX.WangZ.GaoY.LiH.WangQ.et al (2022). Swarm of micro flying robots in the wild. Sci. Robot.7, eabm5954. 10.1126/scirobotics.abm5954
75
ZhouZ.ChenK.LiX.ZhangS.WuY.ZhouY.et al (2020). Sign-to-Speech translation using machine-learning-assisted stretchable sensor arrays. Nat. Electron.3, 571–578. 10.1038/s41928-020-0428-6
Summary
Keywords
artificial intelligence, strain sensors, soft robots, machine intelligence, closed-loop control
Citation
Yang H and Wu W (2022) A review: Machine learning for strain sensor-integrated soft robots. Front. Electron. Mater. 2:1000781. doi: 10.3389/femat.2022.1000781
Received
22 July 2022
Accepted
31 October 2022
Published
14 November 2022
Volume
2 - 2022
Edited by
Ctirad Uher, Department of Physics, University of Michigan, United States
Reviewed by
Kean C. Aw, The University of Auckland, New Zealand
Updates
Copyright
© 2022 Yang and Wu.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Wenbo Wu, wuwb@tju.edu.cn
This article was submitted to Optoelectronic Materials, a section of the journal Frontiers in Electronic Materials
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.