Deep Recurrent Neural Networks Based Obstacle Avoidance Control for Redundant Manipulators

Obstacle avoidance is an important subject in the control of robot manipulators, but is remains challenging for robots with redundant degrees of freedom, especially when there exist complex physical constraints. In this paper, we propose a novel controller based on deep recurrent neural networks. By abstracting robots and obstacles into critical point sets respectively, the distance between the robot and obstacles can be described in a simpler way, then the obstacle avoidance strategy is established in form of inequality constraints by general class-K functions. Using minimal-velocity-norm (MVN) scheme, the control problem is formulated as a quadratic-programming case under multiple constraints. Then a deep recurrent neural network considering system models is established to solve the QP problem online. Theoretical conduction and numerical simulations show that the controller is capable of avoiding static or dynamic obstacles, while tracking the predefined trajectories under physical constraints.


INTRODUCTION
As industrial automation develops, robot manipulators have been used in a wide range of applications such as painting, welding, assembly, etc., (Cheng et al., 2009;Yang et al., 2018a). With the evolution of intelligent manufacturing, the way robot works is also changing. In order to fulfill more difficult tasks in complex environment, the robot is required to have better execution capabilities (Pan et al., 2018). Therefore, robots with redundant DOFs have attracted much attention in the field of robotic control since its wonderful flexibility (Chan and Dubey, 1995;Zhang, 2015).
Obstacle avoidance is a core problem in the control of redundant manipulators, in order to realize human-machine collaboration and integration, robots no longer work in a separate environment that is completely isolated (Ge and Cui, 2000;Sugie et al., 2003;Lee and Buss, 2007). Instead, collaboration is required between human or other robots, as a result, the obstacle avoidance control is becoming a matter of urgency: robots need to achieve real-time avoidance of static or dynamic obstacles while completing given motion tasks.
Many obstacle avoidance methods for robot manipulators haven been reported, which are designed online or off-line. Based on stochastic sampling algorithm, a series of obstacle avoidance methods are proposed, these methods could obtain effective solutions even in ultra-redundant systems. In Wei and Ren (2018), Wei et al. propose a modified RRT based method, namely Smoothly RRT, in which a maximum curvature constraint is built to obtain a smooth curve when avoiding obstacles, simulation results also show that the method achieves faster convergence than traditional RRT based ones. In Hsu et al. (2006), Hsu discusses the probabilistic foundations of PRM based methods, a conclusion is drew that the visibility properties rather than dimensionality of C has a greater impact on the probability, and the convergence would be faster if extract partial knowledge could be introduced. However, due to the large computational costs, those methods can be hardly used online.
Different from stochastic results obtained by above mentioned methods, artificial potential field methods plan the same path each time in the same environment, which is important in industrial applications (Khatib, 1986). The basic idea of artificial potential field methods is that the target bears as an attractive pole while the obstacle creates repulsion on the robot, then the robot will be controlled to converge to the target without colliding with obstacles. At the same time, artificial potential field methods have shown great ability in tracking dynamic targets as well as avoiding dynamic obstacles. In Csiszar et al. (2011), a modified method is proposed, which describes the obstacles by different geometrical forms, both theoretical conduction and experimental tests validate the proposed method. Considering the local minimum problem that may be caused by multi-link structures, in Badawy (2016), a two minima is introduced to construct potential field, such that a dual attraction between links enables faster maneuvers comparing with traditional methods. Other improvements to artificial potential field method can be found in Tsai et al. (2001); Tsuji et al. (2002); Wen et al. (2017). Taking advantage of redundant DOFs, obstacles can be avoided by the self-motion in the null space, by calculating pseudoinverse of Jacobian matrix, the solution can be formulated as the sum of a minimum-norm particular solution and homogeneous solutions (Cao et al., 1999;Moosavian and Papadopoulos, 2007;Krzysztof and Joanna, 2016).
The application of artificial intelligence algorithms based on neural networks provide a new idea for robotic control, these methods are considered to be very promising since its excellent learning ability (Jung and Kim, 2007). For instance, in , the authors propose a command-filtered back-stepping method, in which a neural network based learning scheme is introduced to deal with functional uncertainties. In , a biomimetic hybrid controller is established, in which the control strategy consist of a feed-forward predictive machine based on a RBF Neural Network and a feedback servo machine based on a proportional-derivative controller. In Fu et al. (2018), a fuzzy logic controller is proposed for long-term navigation of quad-rotor UAV systems with input uncertainties. Experiment results show that the controller can achieve better control performance when compared to their singleton counterparts. In Fu et al. (2019), an online learning mechanism is built for visual tracking systems. The controller uses both positive and negative sample importances as input, and it is shown that the proposed weighted multiple instance learning scheme achieves wonderful tracking performance in challenging environments. Typically, the structure of a neural network may be complex in order to achieve better performance. Although the model of robot manipulator is highly nonlinear, by introducing the priori information of the system model, the neural network can be optimized, i.e., the number of nodes in neural networks can be reduced effectively while maintaining the learning efficiency (Fontaine and Germain, 2001). Inspired by this, a series dynamic neural networks are proposed to realize robotic control in realtime Li et al., 2017;Yang et al., 2018b). Based on the idea of constraint-optimization, quadraticprogramming approaches haven been introduced for kinematic control of redundant manipulators. The designed outer-loop controller is described as equality constraints, and objective functions are established to describe certain performance of the system. Using the learning and parallel calculation ability, dynamic neural networks are established to solve the quadraticprogramming problem online. The kinematic control is thus achieved by ensuring the equality constraints, and the flexibility is used by optimizing the objective functions. On the other hand, these methods is capable of handling inequality constraints and model uncertainties (Zhang et al., 2018;Li et al., 2019;Xu et al., 2019b). In Cheng et al. (1993), the obstacle avoidance strategy is described as equality constraints, but the parameters of escape velocity is difficult to obtain. In Zhang and Wang (2004), Zhang et al. propose an inequality based method, in which the distance between the robot and obstacles are formulated as a group of distances from critical points and robot links. On this basis, an improved method is proposed by Guo et al. in Guo and Zhang (2012), which is capable of suppressing undesirable discontinuity in the original solutions.
Motivated by the above observations, in this paper, we proposed a novel obstacle avoidance strategy based on deep recurrent neural networks. By abstracting robot and obstacles as a set of critical points, the distances between the robot and obstacles are approximately described by a group of pointto-point distances. And the obstacle avoidance is realized by inequality constraint described by class-K functions. Then the obstacle avoidance problem is reformulated as a QP problem in the speed level, and a deep recurrent neural network is designed to solve the QP online. Numerical results show that the robot is capable of avoiding the obstacles while tracking the predefined trajectories. Before ending this section, the main contributions of this paper are summarized as below • The proposed deep RNN based controller is able to achieve both path tracking and obstacle avoidance, at the same time, physical constraints such as angular joints and velocities are satisfied. • In this paper, we propose a class-K function based obstacle avoidance strategy, which has a more general form of description than traditional linear escape velocity methods. • By abstracting robots and obstacles into critical point sets respectively, the distance between the robot and the obstacle can be described in a simpler way. Besides, numerical results show that the control algorithm can realize the avoidance of static and dynamic obstacles.

Basic Description
When a redundant robot is controlled to track a particular trajectory in the cartesian space, the positional description of the end-effector can be formulated as: where x ∈ R m and θ ∈ R n are the end-effector ′ s positional vector and joint angles, respectively. In the velocity level, the kinematic mapping betweenẋ andθ can be described as: where J(θ ) ∈ R m×n is the Jacobian matrix from the end-effector to joint space. In engineering applications, obstacles are inevitable in the workspace of a robot manipulator. For example, robot manipulators usually work in a limited workspace restricted by fences, which are used to isolated robots from humans or other robots. This problem could be even more acute in tasks which requires collaboration of multiple robots. Let C 1 be the set of all the points on a robot body, and C 2 be the set of all the points on the obstacles, then the purpose of obstacle avoidance of a robot manipulator is to ensure C 1 ∪ C 2 = ∅ at all times. By introducing d as a safety distance between the robot and obstacles, the obstacle avoidance is reformulated as Equation (3) gives a basic description of obstacle avoidance problem in form of inequalities. However, there are too many elements in sets C 1 and C 2 , the vast majority of which are actually unnecessary. Therefore, by uniformly selecting points of representative significance from C 1 and C 2 , and increasing d properly, Equation (3) can be approximately described as below: with A i , i = 1, . . . , a and O j , j = 1, . . . , b being the representative points of the robot and obstacles, respectively. The schematic diagram of Equation (4) in shown in Figure 1.
Remark. 1 In real implementations, there are many ways to measure |O j A i |. For instance, since physical structure of FIGURE 1 | The basic idea of obstacle avoidance in this paper. the a manipulator is known, the key points A i are available in advance, both positions and velocities of those points can be calculated directly using the feedback of robot joints. The real-time measurement of obstacles can be achieved through industrial cameras. Therefore, the information of A i and B j are all available. As to measurement noise, by introducing a bigger safety distance d, e.g., d = 1.5(d 1 + d 2 ), the safety can be ensured.

Reformulation of Inequality in Speed Level
In order to guarantee the inequality (4), by defining D = |O j A i |− d, an inequality is rebuilt in speed level as: in which g(•) belongs to class-K. Remarkable that the velocities of critical points A i can be described by joint velocities: where J ai ∈ R m×n is the Jacobian matrix from the critical point A i to joint space. If O j is prior known, the left-side of Equation (5) can be reformulated as: where Therefore, the collision between critical point A i and object O j can be obtained by ensuring the following inequality: where Based on the inequality description (8), the collision between robot and obstacle can be avoided by ensuring: where Remark 2: According to 5 the definition of class-K functions, the original escape velocity based obstacle avoidance methods in Zhang and Wang (2004); Guo and Zhang (2012) can be regarded as a special case of 5, in which G(|D|) is selected as G(|D|) = k|D|. Therefore, in this paper, the proposed obstacle avoidance strategy is more general than traditional methods.

QP Type Problem Description
As to redundant manipulators, in order to take full advantage of the redundant DOFs, the robot can be designed to fulfill a secondary task when tracking a desired trajectory. In this paper, the secondary task is set to minimize joint velocity while avoiding obstacles. In real implementations, both joint angles and velocities are limited because of physical limitations such as mechanical constraints and actuator saturation. Because of the fact that rank (J) < n, there might be infinity solutions to achieve kinematic control. In this paper, we aim to design a kinematic controller which is capable of avoiding obstacles while tracking a pre-defined trajectory in the cartesian space. For safety ′ s sake, the robot is wished to move at a low speed, on the other hand, lower energy consumption is guaranteed. By defining an objective function scaling joint velocity asθ Tθ /2, the tracking control of a redundant manipulator while avoiding obstacles can be described as: It is remarkable that the constraints 10b-10e and the objective function 10a to be optimized are built in different levels, which is very difficult to solve directly. Therefore, we will transform the original QP problem (10) in the velocity level. In order to realize precise tracking control to the desired trajectory x d , by introducing a negative feedback in the outer-loop, the equality constraint can be ensured by letting the end-effector moves at a velocity ofẋ =ẋ d + k(x d − x). In terms with (10c), according to escape velocity method, it can be obtained by limiting joint speed as α(θ − − θ ) ≤θ ≤ α(θ + − θ ), where α is a positive constant. Combing the kinematic property (2), the reformulated QP problem is: It is noteworthy that both the formula (11a) and (11d) are nonlinear. The QP problem cannot be solved directly by traditional methods. Using the parallel computing and learning ability, a deep RNN will be established later.

DEEP RNN BASED SOLVER DESIGN
In this section, a deep RNN is proposed to solve the obstacle avoidance problem (11) online. To ensure the constraints (11b), (11c), and (11d), a group of state variables are introduced in the deep RNN. The stability is also proved by Lyapunov theory.

Deep RNN Design
Firstly, by defining a group of state variables λ 1 ∈ R m , λ 2 ∈ R ab , a Lagrange function is selected as: λ 1 and λ 2 are the dual variables corresponding to the constraints (11b) and (11d). According to Karush-Kuhn-Tucker conditions, the optimal solution of the optimization problem (12) can be equivalently formulated as: where P (x) = argmin y∈ ||y − x|| is a projection operator to a restricted interval , which is defined as = {θ ∈ R n |max(α(θ − − θ ),θ − ) ≤θ ≤ min(θ + , α(θ + − θ ))}. Notable that Equation (13c) can be simply described as: where (•) + is a projection operation to the non-negative space, in the sense that y + = max(y, 0). Although the solution of (13) is exact the optimal solution of the constrained-optimization problem (11), it is still a challenging issue to solve (13) online since its inherent nonlinearity. In this paper, in order to solve (13), a deep recurrent neural network is designed as: with ǫ is a positive constant scaling the convergence of (15). Remark. 3 As to the established deep RNN (15), the first dynamic equation is also the output dynamics, which combines the dynamics of state variables λ 1 and λ 2 , as well as the physical limitations including joint angles and velocities. State variable λ 1 is used to ensure the equality constraint (11b), as shown in (15b), λ 1 is updated according to the difference between reference speeḋ x d + k(x d − x) and actually speed J(θ )θ. Similarly, λ 2 is used to ensure the inequality constraint 11d, which will be further discussed later. It is remarkable that ǫ plays an important role in the convergence of the deep RNN. The smaller ǫ, the faster the deep RNN converges.
Remark. 4 By introducing the model information such as J, J o , etc., the established deep RNN consists of three class of nodes, namelyθ, λ 1 and λ 2 , and the total number of nodes is n+m+ab. Comparing to traditional neural networks in Jung and Kim (2007), the complexity of neural networks is greatly reduced.

Stability Analysis
In this subsection, we offer stability analysis of the obstacle avoidance method based on deep RNN based. First of all, some basic Lemmas are given as below.
Definition 1: A continuously differentiable function F(•) is said to be monotone, if ∇F +∇F T is positive semi-definite, where ∇F is the gradient of F(•).
Lemma 1: A dynamic neural network is said to converge to the equilibrium point if it satisfies: where κ > 0 and ̺ > 0 are constant parameters, and P S = argmin y∈S ||y − x|| is a projection operator to closed set S. Lemma 2: (Slotine and Li, 2004)

then
x = 0 is a uniformly asymptotically stable equilibrium point if there exist some class-K function g defined on [0, d), to make the following inequality hold: About the stability of the closed-loop system, we offer the following theorem. Theorem 1: Given the obstacle avoidance problem for a redundant manipulator in kinematic control tasks, the proposed deep recurrent neural network is stable and will globally converge to the optimal solution of (10).
Proof: The stability analysis consists of two parts: firstly, we will show that the equilibrium of the deep RNN is exactly the optimal solution of the control objective described in (11), which prove that the output of deep RNN will realize obstacle avoidance while tracking a given trajectory, and then we will prove that the deep recurrent neural network is stable in sense of Lyapunov.
Part I. As to the deep recurrent neural network (15), let (θ * , λ * 1 , λ * 2 ) be the equilibrium of the deep RNN, then (θ * , λ * 1 , λ * 2 ) satisfies: withθ * be the output of deep RNN. By comparing Equation (18) and (13), we can readily obtain that they are identical, i.e., the equilibrium point satisfies the KKT condition of (10). Then we will show that the equilibrium point(output of the proposed deep RNN) is capable of dealing with kinematic tracking as well as obstacle avoidance problem. Define a Lyapunov function V about the tracking error e = x d − x as V = e T e/2, by differentiating V with respect to time and combining (11b), we have:V in which the dynamic Equation 18b is also used. It can readily obtained that the tracking error would eventually converge to 0. It is notable that the dynamic (Equation 18c) satisfies: According to the property of projection operator (•) + , y−(y) + ≤ 0 holds for any y, then we have J oθ * − B ≤ 0, together with (5), the inequality (5) is satisfied. Notable that (5) can be rewritten as: As to (21), we first consider the situation when equality holds.
Since g(|D|) is a function belonging to class K, it can be easily obtained that D = 0 is the only equilibrium ofḊ = −sgn(D)g(|D|). Define a Lyapunov function as V 2 (t, D) = D 2 /2, and select two functions as α 1 (|D|) = α 2 (|D|) = D 2 /2. It is obvious that α 1 (|D|) = α 2 (|D|) belong to class-K, and the following inequality will always hold: Taking the time derivative of V 2 (t, D), we have: According to Lemma 2, the equilibrium x = 0 is uniformly asymptotically stable. Then we arrive at the conclusion that if the equality d(|O j A i |)/dt = −sgn(D)g(|D|) holds, |D| = 0 will be guaranteed, i.e., |O j A i | − d for all i = 1 · · · a, = 1 · · · b. Based on comparison principle, we can readily obtain that |O j A i | ≥ d when d(|O j A i |)/dt ≥ −sgn(D)g(|D|). Part II. Then we will show the stability of the deep RNN (15). Let ξ = [θ T , λ T 1 , λ T 2 ] T be the a concatenated vector of state variables of the proposed deep RNN, then (15) can be rewritten as: where P S (•) is a projection operator to a set S, and F(ξ ) = [F 1 (ξ ), F 2 (ξ ), F 3 (ξ )] T ∈ R n+m+ab , in which: Let ∇F = ∂F/∂ξ , we have: According to the definition of monotone function, we can readily obtain that F(ξ ) is monotone. From the description of (24), the projection operator P S can be formulated as P S = [P ; P R ; P ], in which P is defined in (13), P R can be regarded as a projection operator of λ 1 to R, with the upper and lower bounds being ±∞, and P = (•) + is a special projection operator to closed set R ab + . Therefore, P S is a projection operator to closed set [ ; R m ; R ab + ]. Based on Lemma 1, the proposed neural network (15) is stable and will globally converge to the optimal solution of (10). The proof is completed.

NUMERICAL RESULTS
In this section, the proposed deep RNN based controller is applied on a planar 4-DOF robot. Firstly, a basic case where the obstacle is described as a single point is discussed, and then the controller is expanded to multiple obstacles and dynamic ones. Comparisons with existing methods are also listed to indicate the superiority of the deep RNN based scheme.

Simulation Setup
The physical structure of the 4-link planar robot to be simulated in shown in Figure 2, in which the critical points of the robot are also marked. As shown in Figure 2A, critical points A 2 , A 4 , A 6 are selected at the joint centers, and A 1 , A 3 , A 5 , A 7 are selected at the center of robot links. The D-H parameters are given in Figure 2B. It is notable that A i and the Jacobian matrix J o i are essential in the proposed control scheme. Based on the above description of A i , the D-H parameters of A 1 is a 1 = 0.15, a 2 = a 3 = 0, α 1 = α 2 = α 3 = 0, d 1 = d 2 = d 3 = 0, then both the position and Jacobian matrix J a1 of A 1 can be calculated readily. Based on the definition in Equation 8, J o1 can be obtained. A i and J oi can be calculated similarly. The control parameters are set as ǫ = 0.001, α = 8, k = 8. As to the physical constraints, the limits of joint angles and velocities are selected as θ − i = −3rad, θ + i = 3rad, θ − i = −1rad/s,θ + i = 1rad/s for i = 1 . . . 4. The safety distance d is set to be 0.1m.

Single Obstacle Avoidance
In this simulation, the obstacle is assumed to be centered at [−0.1, 0.2] T m, the desired path is set as x d = [0.4 + 0.1cos(0.5t), 0.4 + 0.1sin(0.5t)] T m, and the initial joint angles are set to be θ 0 = [π/2, −π/3, −π/4, 0] T rad. The class-K function is selected as G(|D|) = K 1 |D| with K 1 = 200. In order to show the effectiveness of the proposed obstacle avoidance method, contrast simulations with and without inequality constraint (10e) are conducted. Simulation results are shown in Figure 3. When ignoring the obstacle, the end-effector trajectories and the corresponding incremental configurations are shown in Figure 3A, although the robot achieves task space tracking to x d , obviously the first link of the robot would collide with the obstacle. After introducing obstacle avoidance scheme, the robot moves other joints rather than the first joint, and then avoids the obstacle effectively ( Figure 3B). Simultaneously, the tracking errors when tracking the given circle are shown in Figure 3C. From the initial state, the end-effector moves toward the circle quickly and smoothly, after that, the tracking error in stable state keeps < 1 × 10 −4 m, showing that the robot could achieve kinematic control as well as obstacle avoidance tasks. To show more details of the proposed deep RNN based method, some important process data is given. As the obstacle is close to the first joint, critical points A 1 and A 2 are more likely to collide with obstacle, therefore, as a result, the distances between the obstacle O 1 and A 1 , A 2 are shown in Figure 3D, from t = 2s to t = 6.5s, ||A 1 O 1 || remains at the minimum value d = 0.1, that is to say, using the proposed obstacle avoidance method, the robot maintains minimum distance from the obstacle. The profile of joint velocities are shown in Figure 3E, at the beginning of simulation, the robot moves at maximum speed, which leads to the fast convergence of tracking errors. The curve of joint angles change over time is shown in Figure 3F.

Discussion on Class-K Functions
In this part, we will discuss the influence of different class-K functions in the avoidance scheme (5). Four functions are selected as G 1 (|D|) = K|D| 2 , G 2 (|D|) = K|D|, G 3 (|D|) = Ktanh(5|D|), G 4 (|D|) = Ktanh(10|D|), Figure 4A shows the comparative curves the these functions. Other simulation settings are the same as the previous one. Simulation results are shown in Figure 4B. When selecting the same positive gain K, the minimum distance is about 0.08m, which shows the robot can avoid colliding with the obstacle using the avoidance scheme (5). The close-up graph of the tracking error is also shown, it is remarkable that the minimum distance deceases, as the gradient of the class-K function increases near 0.  Therefore, one conclusion can be drawn that the function can be more similar with Sign function, to achieve better obstacle avoidance.

Multiple Obstacles Avoidance
In this part, we consider the case where there are two obstacles in the workspace. The obstacles are set at [0.1, 0.25] T m and [0, 0.4] T m, respectively. Simulation results are shown in Figure 5. The desired path is defined as x d = [0.45 + 0.1cos(0.5t), 0.4 + 0.1sin(0.5t)] T . The initial joint angle of the robot is selected as θ 0 = [1.5, −1 − 1, 0] T . To further show the effectiveness of the proposed obstacle avoidance strategy 5, g|D| is selected as g|D| = K 1 /(1 + e −|D| ) − K 1 /2 with K 1 = 200. When λ 2 is set to 0, as shown in Figure 5A, the inequality constraint (11d) will not work, in other words, only kinematic tracking problem in considered rather than obstacle avoidance, in this case, the robot would collide with the obstacles. After introducing online training of λ 2 , the simulation results are given in Figures 5B-H. The tracking errors are shown in Figure 5C, with the transient time being about 4s, and steady state error < 1 × 10 −3 m. Correspondingly, the robot moves fast in the transient stage, ensuring the quick convergence of the tracking errors. It is remarkable that the distances between the critical points and obstacle points are kept larger than 0.1m at all times, showing the effectiveness of the proposed method. At t = 14s, from Figures 5D,G, when the distance between A 3 and O 1 is close to 0.1m, the corresponding dual variable λ 2 becomes positive, making the inequality constraint (11d) hold, and the boundedness between the robot and obstacle is thus guaranteed. After t = 18s, ||A 3 O 1 || becomes greater, then λ 2 converges to 0. Notable that although λ 1 and λ 2 do not converge to certain values, the dynamic change of λ 1 and λ 2 ensures the regulation of the proposed deep RNN.

Enveloping Shape Obstacles
In this part, we consider obstacles of general significance. Suppose that there is a rectangular obstacle in the workspace, with the vertices being [0, 0  Figure 6B, in which a local amplification diagram is also given. showing that the critical points A 3 is capable of avoiding O 2 and O 3 . It is worth noting that by selecting proper point group and safety distance, the obstacle can be described by the envelope shape effectively. While in Figure 6A, when obstacle avoidance is ignored, the collision emerges. Figures 6C-H also give important process data of the system under the proposed controller, including tracking errors, joint angles, angular velocities, and state variables of deep RNNs. We can observe that the physical constraints as well as kinematic control task are realized using the controller.

Dynamic Obstacles
In this part, we consider dynamic obstacles moving in the workspace. In real applications, pedestrian or other obstacles always tend to be mobile. Obstacle avoidance for dynamic obstacles is of more general significance. In real time, static obstacles can be considered a special case. In this simulation, the simulation duration is selected as 20s, and the trajectory of a dynamic obstacle is defined as x d = [−0.1 + 0.01t, 0.3] T . The snapshots in the control process are shown in Figure 8. While ensuring effective tracking of the defined path, the robot is able to use its self-motion to avoid the dynamic obstacle effectively, and maintain a safe distance. The distances between critical points and the dynamic O is shown in Figure 7B. At the beginning of simulation, the tracking error is big, in order to ensure the convergence of tracking error, the joints move a big range except J1. It is worth noting that since the critical point A 2 is next to O, joint 1 rotates in the direction which conforms to the movement of O. In the stable state, tracking error is < 5 × 10 −4 m (Figure 7A). At about t = 14s, A 2 O decreases to 0.1m, accordingly, the joint velocities change obviously (as shown the significant change of joint velocities in Figure 7C, the tracking error changes to 10 −3 m, and then converges quickly. At t = 18s, although A 2 and A 3 are near O, the robot is still capable of avoiding the dynamic obstacle. During the control process, joint angles are ensured not to exceed the limits, as shown in Figure 7D.

Obstacle Performance on 7-DOF Manipulator in 3-Dimensional Space
To further verify the effectiveness of the control scheme, another simulation on a 7DOF manipulator iiwa is carried out. The desired path to be tracked is also a planar circular, which is centered at [0, −0.6, 0.1] T m with radius being 0.15m. The physical parameters can be found in Xu et al. (2019a). Suppose that there exist a cylinder obstacle in the workspace, the obstacle is centered as [−0.13, −0.3, 0] T m, with the radius and height being 0.15m and 2m, respectively. Simulation results are shown in Figure 9. It can be readily found that the proposed schemes can obtain satisfying performance in 3-dimensional spaces.

Comparisons
To illustrate the priority of the proposed scheme, a group of comparisons are carried out. As shown in Table 1, all the controllers in Zhang and Wang (2004); Csiszar et al. (2011); Guo and Zhang (2012); Krzysztof and Joanna (2016) achieve the avoidance of obstacles. Comparing to APF method in Csiszar et al. (2011); Krzysztof and Joanna (2016) of JP based method in Csiszar et al. (2011); Krzysztof and Joanna (2016), the proposed controller can realize a secondary task, at the same time, we present a more general formulation of the obstacle avoidance strategy, which is helpful to gain a deeper understanding of the mechanism for avoidance of obstacles. Moreover, in this paper, both dynamic trajectories and obstacles are considered. The comparisons above also highlight the main contributions of this paper.

CONCLUSIONS
In this paper, a novel obstacle avoidance strategy is proposed based on a deep recurrent neural network. The robots are obstacles are presented by sets of critical points, then the distance between the robot and obstacle can be approximately describes as point-to-points distances. By understanding the nature escape velocity methods, a more general description  Zhang and Wang (2004); Guo and Zhang (2012); Krzysztof and Joanna (2016), dynamic obstacles are not considered. **Regular escape velocity method is used, which is only a special case of 5.
of obstacle avoidance strategy is proposed. Using minimumvelocity-norm (MVN) scheme, the obstacle avoidance together with path tracking problem is formulated as a QP problem, in which physical limits are also considered. By introducing model information, a deep RNN with simple structure is established to solve the QP problem online. Simulation results show that the proposed method can realize the avoidance of static and dynamic obstacles.

DATA AVAILABILITY
All datasets analyzed for this study are included in the manuscript and the supplementary files.

AUTHOR CONTRIBUTIONS
Theoretical derivation was done by ZX. Simulation research was performed by ZX, with important advice from XZ. The paper was revised by XZ and SL. All authors approved the manuscript.