These authors have contributed equally to this work and share first authorship
Ivo Kuhlemann, Ibeo Automotive Systems GmbH, Hamburg, Germany
This article was submitted to Robotic Control Systems, a section of the journal Frontiers in Robotics and AI
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Reliable force-driven robot-interaction requires precise contact wrench measurements. In most robot systems these measurements are severely incorrect and in most manipulation tasks expensive additional force sensors are installed. We follow a learning approach to train the dependencies between joint torques and end-effector contact wrenches. We used a redundant serial light-weight manipulator (KUKA iiwa 7 R800) with integrated force estimation based on the joint torques measured in each of the robot’s seven axes. Firstly, a simulated dataset is created to let a feed-forward net learn the relationship between end-effector contact wrenches and joint torques for a static case. Secondly, an extensive real training dataset was acquired with 330,000 randomized robot positions and end-effector contact wrenches and used for retraining the simulated trained feed-forward net. We can show that the wrench prediction error could be reduced by around 57% for the forces compared to the manufacturer’s proprietary force estimation model. In addition, we show that the number of high outliers can be reduced substantially. Furthermore we prove that the approach could be also transferred to another robot (KUKA iiwa 14 R820) with reasonable prediction accuracy and without the need of acquiring new robot specific data.
Compliant robotic arms have increasingly gained importance during the last years, where anthropomorphic kinematically redundant serial manipulators with seven degrees of freedom (DoF) are frequently used for various new applications. Integrated joint torque sensors provide crucial functionalities for safe human-robot-interactions. Based on the joint torques
Furthermore, robots are usually not used in their full performance range, but operate in a very small range of forces due to the rather restricted type of specialized applications. Therefore, an optimized calibration within this specific range of forces could lead to very precise results, minimized errors and drastically increased overall system accuracy.
One approach to increase the accuracy of the contact force calculation is machine learning. Artificial neural networks show impressive capabilities in solving direct
An approach using deep learning for reducing errors in identification of dynamic parameters of a 6-DoF robot is presented in
The use of neural networks for calibrating the robotic system provides several advantages. The system is capable of learning unique mechanical characteristics of the manipulator and the robot can therefore be calibrated in a highly specialized way. Furthermore, critical arm positions and singularities can be directly learned from the network as training points. Using a sufficient amount of diverse input data, these points can be uniquely identified and integrated into the model, resulting in a more robust calibration.
The aim of this work is to improve the accuracy of static end-effector contact wrench estimation using the robot’s integrated sensory technology. A scenario with comparatively small contact forces (up to 20 N) well below the maximal capacity of the robot and small distances to the force application point in the tool (up to 0.15 m) is chosen. This scenario represents our use case holding an ultrasound probe, which is attached to the end-effector, in safe contact with the body. Even though this task can be considered quasi-static since the robot may move slightly, the dynamic effects due to the movement are very small compared with the gravitational forces as well as the external force exerted by the human body being in contact with. Thus, the movement can be seen as a point-to-point motion, with a wrench estimation being taken at each point while the robot is not moving and therefore is in a static state at the time of measurement.
Standard approaches for acquiring ground truth data for contact wrenches by using force/torque sensors or additional collaborating manipulators require expensive hardware and introduce various sources of errors due to mechanical issues. We present an alternative method to generate contact wrenches by mounting calibration loads to the end-effector. By using the gravity force in combination with different robot base orientations, we obtain a homogeneous representation of contact forces in all directions. An extensive database consisting of 330,000 randomized data points was created.
Our database allows for a detailed analysis of the robot’s integrated sensors and the accuracy of the proprietary force estimation model (PFEM).
We present a data-driven method based on linear regression to approximate the static gravity torques without any knowledge of the link masses nor the centers of gravity of the links. The approach is easy to implement, data-saving and performs slightly better than the robots integrated estimation of gravity torques in a static case.
We follow a learning approach to train deep feed-forward artificial neural networks (ANNs) with simulated created as well as real data to estimate the contact wrenches applied to the end-effector based on the measured joint torques and information about the robot’s current pose. Estimation error and robustness close to singular joint configurations will be improved. Moreover we show that the approach could be transferred to another, similar robot (
We used a
A detailed description of the proprietary robot control architecture is given in
The aim of this work is to determine contact wrenches at the end-effector. It is investigated whether the contact wrench can be predicted from corresponding joint positions as well as joint torque data. Different datasets, both simulated and real have been acquired to train and evaluate neural network models.
A large simulated dataset is generated, varying the applied forces, the distances of the end-effector to the force application point as well as the applied external torques for randomized joint configurations. Thus, randomized end-effector contact wrenches are generated and the corresponding joint torques for an ideally static case can be calculated. In our scenario forces between −20 and 20 N are applied at a point with a distance between 0 and 0.15 m to the end-effector. The range results from our application to measure the force acting on an ultrasonic probe attached to the end-effector of a robot. Moreover an additional external torque between −2 and 2 Nm is applied at the end-effector. The steps of generating the simulated training data are explained as follows: Let
The aim of this work is to precisely determine corresponding contact wrenches at the end-effector from given joint positions and torque data. Contact wrenches at the end-effector can result from various impacts: by external forces such as pushing or pulling by hand, or while the robot actively pushes against an object with its tool attached to the end-effector. From a mechanical point of view, the resulting wrenches are the same for both cases and can be measured via the joint torques. In this study, the wrenches are simulated by mounting different masses on the end-effector. Gravity produces an equivalent force, which pulls the mass towards the ground. Obviously, this static force will always point in the same direction in the world coordinate system, but for varying robot positions it will create different contact forces in end-effector coordinates. With only one direction in world coordinates not all contact forces can be represented. Hence, six different base rotations were used to solve this problem. The robot was therefore mounted in different orientations, shown in
Calibration weights were stacked on a metallic rod attached to the end-effector of the robot to generate 10 equidistantly distributed end-effector forces in our target range from 0–20 N. Examples of appropriately stacked calibration weights to generate an end-effector force of 8 N
The exact contact forces for the ground truth can be determined as follows:
For a serial manipulator we can compute the position and orientation of the end-effector by coordinate transformations from the base along the joints by
Let
To calculate the six base transformations
i | Base rotation | Ψ [°] | Θ [°] | Φ [°] |
---|---|---|---|---|
1 | g → z − | 0 | 0 | 0 |
2 | g → z + | 0 | 0 | 180 |
3 | g → x − | 180 | −90 | 0 |
4 | g → x + | 0 | 90 | 0 |
5 | g → y − | 90 | 0 | 90 |
6 | g → y + | −90 | 0 | −90 |
The forces are not acting directly in the end-effector coordinate system, but in the centre of mass of the metallic rod with the appropriately stacked calibration weights. Thus, a moment m
As described in
Base orientations of directional generalization
Data points | Ψ [°] | Θ [°] | Φ [°] | Load [kg] |
---|---|---|---|---|
1,500 | 0.0 | 0.2 | 0.1 | 1.0 |
500 | 0.0 | 0.0 | 31.1 | 1.0 |
1,500 | 0.5 | 29.1 | 0.2 | 1.0 |
1,500 | 0.0 | 27.8 | −14.6 | 1.0 |
1,500 | 0.0 | −43.3 | 0.7 | 1.0 |
1,500 | 0.0 | −31.7 | 39.4 | 1.0 |
The LBR iiwa was mounted on a hexapod to acquire the test data (
Generating a large robot specific training dataset as described in
For training of the neural network the isolated torques
For a 7-DOF manipulator the relationship between a joint torque
Taking into account all of the joints of the manipulator, the following equation system represents the relationship from
To determine
External forces applied to a certain point of an attached tool lead to corresponding contact wrenches at the end-effector. Our aim was the estimation of these wrenches w
Firstly, a deep feed forward neural network was trained with the idealized simulated
Hyperparameter optimization - static and dynamic parameters.
Parameter | Type | Value/Value range |
---|---|---|
Layers | Dynamic | 1–20 |
Neurons | Dynamic | 1–1,000 |
L2 regularization | Dynamic | 1 |
Loss | Dynamic | Mean squared error, mean absolute error |
Epochs | Static | 500 |
Batch size | Static | 6,000 |
Early stopping patience | Static | 1,000 |
The optimized neural network model
To take into account the described inaccuracies, the model based on simulated data
The goal is to precisely estimate gravity joint torques as well as contact wrenches at the end-effector from given joint positions and torque data. A data-driven method based on linear regression is used to approximate the static gravity torques without any knowledge of the link masses nor the centers of gravity of the links. To estimate the end-effector contact wrench an extensive database, consisting of both simulated and real data, was acquired to develop and evaluate artificial neural network models. For generating real ground truth contact wrenches, ten specially manufactured weights in the range of 0–2 kg (0–20 N) were mounted to the end-effector. By using the constant gravity force and different robot base orientations, a homogeneous representation of contact forces in all directions was realized. Due to various combinations of base orientations, calibration weights and robot poses, the database consists of 330,000 randomized data points. See
Firstly the performance of our proposed gravity compensation model is evaluated in
To determine the imaginary masses
The determined imaginary masses
Absolute error of the proposed gravity compensation model compared to the model of the robot manufacturer, evaluated on the
The obtained results show the applicability of the presented approach to determine static gravity torques of a robot without knowing the inertial parameters of the links. It represents a simple to implement and time-saving option, since only joint torque data of a few randomly generated poses need to be acquired.
For evaluation purposes, the contact forces and moments acting at the end-effector are considered separately. The prediction accuracies of the
To find the optimal network architecture for the estimation of contact wrenches, an autonomous hyperparameter optimization was done using the
It must also be noted, that our ground truth for the contact moments is based on the center of mass of the metallic rod with the mounted calibration weights. These were determined from an appropriate created CAD model, so small uncertainties can possibly result. Nevertheless it could be shown, that the relationship between end-effector forces, the distance to the force application point and corresponding end-effector moments can be learned and precisely predicted by the neural network. Due to the problematic issues pointed out regarding the contact moment, in the following chapters the accuracy of contact forces acting at the end-effector is investigated in more detail. These are also of higher relevance for our application of precisely estimating the force acting on an ultrasound probe attached to the robot end-effector.
As shown in
In order to analyze the problems of incalculable contact forces at singular positions and numerical instabilities close to these positions in more detail, we looked at the contact force errors as a function of the arm position. To acquire this, the manipulators’ reach was used as a relevant parameter. The reach indicates how far the end-effector is displaced from the shoulder. At maximum reach, the arm is therefore fully extended and in a singular configuration.
To examine the transferability of our approach to another robot, the accuracy of contact force estimation of the additional testing dataset, which has been acquired with the
Absolute end-effector force error
However, it is shown, that the estimation of the contact forces could be improved by our neural network models - even if another robot type is used. Especially the
The aim of this work was to precisely determine corresponding contact wrenches at the end-effector from given joint position and torque data of a redundant serial lightweight manipulator (
After the promising results of this work, minor limitations remain. The calibration was performed in a limited force range of 0–20 N, which is significantly below the maximum loads of the manipulator. However, compliant robots are often used in specialized practical applications within limited load ranges. Moreover, our results show good generalization performance, even for estimation of the neural network, which was soley trained with simulated data. Thus using a huge simulation dataset with an increased force range and a small real calibration dataset with larger distances between the calibration weights could be used. In addition we plan to further investigate the transferability of our method model to robots from other manufacturers, which have a more different geometry. In this context, a transfer to a robot with a different degree of freedom and thus an input of a different size must also be considered. A robot specific simulated dataset could be generated easily for any robot whose geometric data are known. Afterwards a small number of robot specific new recorded data points could be acquired and used for retraining and thus the prediction performance could be improved.
The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.
All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.
This study was partially funded by Deutsche Forschungsgemeinschaft (grants ER 817/1–1, ER 817/1-2 and ER 817/4-1) and by the German Federal Ministry of Education and Research (grant 13GW0228 and 01IS19069).
IK was employed by Ibeo Automotive Systems GmbH.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
We acknowledge financial support by Land Schleswig-Holstein within the funding programme Open Access Publikationfonds.