Energy-Efficiency-Oriented Vision Feedback Control of QCSP Systems: Linear ADRC Approach

1 Introduction

With the development of unmanned aerial vehicle (UAV) technology (Wu et al., 2018), drone transport has become an important branch of UAV applications. The quadrotor with the cable-suspended payload (QCSP) (Lv et al., 2020; Lv et al., 2021) equipped with a camera and an embedded platform, is of great relevance to the realization of rescue and transport tasks. The QCSP actively adjusts the UAV’s own attitude to quickly reduce the oscillation of the suspended load and then runs the vision algorithm through the embedded platform to process the images from the camera to obtain an accurate target position for the drone. Based on the target position information, the QCSP needs to reach the target position quickly and stably. However, in the transportation process, in addition to the energy required for drone flight, the object detection algorithm and the QCSP flight control strategy also consume great energy. Therefore, by considering the limited battery capacity of the drone, it is important to improve the energy efficiency of the QCSP systems.

Recent decades witness great progress in object detection with the development of convolutional neural networks (CNNs). To obtain a powerful network, numerous efforts are made to build large and complex architecture with high computation and energy consumption, which restricts its application on embedding devices such as a drone. To get the light network, Z. Liu et al. (Liu et al., 2017) utilize scaling factors to value the significance of connections and remove these under a threshold. This efficient method works well for classification networks. However, it lacks effectiveness for detection networks. To make the network compression method suitable to detection network, Z. Xie et al. (Xie et al., 2020) introduce location-aware loss for network compression, which helps in preserving the comprehensive ability of the detection network. These methods are not optimal for saving energy because they do not adopt an energy-aware function during compressing.

To realize the path following for the QCSP, Qian, et al. (Qian and Liu, 2019) propose a controller based on uncertainty and an interference estimator. Hao, et al. (Hao et al., 2021) propose a nonlinear, robust, fault-tolerant, position-tracking, control law for a tilt tri-rotor UAV, thus avoiding rear servo’s stuck fault together with parametric uncertainties and unknown external disturbances. To enable a multirotor UAV to achieve static hovering, Mochida, et al. (Mochida et al., 2021) propose a geometric method that reveals the relationship between the position of the center of mass (CoM) and the rotor placement of a multirotor UAV with upward-oriented rotors. These methods can effectively help UAVs accomplish their tasks, but they do not take into account the energy limitation of UAVs; the algorithm is complex and not applicable to the QCSP.

Although many researchers have done meaningful work and achieved results, there are still some challenges in vision processing and position control in the energy-efficiency-oriented QCSP, mainly as follows: 1) Object detection technique–based CNNs can obtain accurate target information for a UAV, but the improved detection performance of deep neural networks also brings huge energy consumption, which is not friendly to the QCSP. 2) The QCSP needs to control the load stably and reach the target position quickly, whereas the traditional PID controller usually needs a long adjustment time for the UAV to reach the target position, which is also not conducive to the QCSP system to save energy for long-distance transportation.

To improve the energy efficiency of the QCSP systems, we propose a lightweight object detection algorithm and an LADRC payload position control strategy for the QCSP. Specifically, the object detection model is compressed by network scaling factors and an energy-aware penalty, which enables the YOLOV3 network to run on the Jetson NX embedded platform of the QCSP with low energy consumption. In addition, an efficient control strategy in the form of a string stage is used to overcome the under-actuated characteristics of the QCSP, which includes attitude, swing angle, load velocity, and load position subcontrollers. The contributions of this paper to the energy saving of QCSP mainly include 1) a new QCSP experimental platform with embedded vision detection is constructed, and a lightweight object detection network is used to obtain position information; 2) an LADRC algorithm is used to control the payload position quickly and efficiently.

The remainder of this paper is structured as follows: Section 2 introduces the dynamic model and object detection algorithm of the QCSP in detail; controller design, including the LADRC position control algorithm for the QCSP is introduced in Section 3; in Section 4, the effectiveness of the proposed QCSP system is verified through experiments. Conclusions are drawn in Section 5.

2 Dynamic Model and Object Detection Algorithm of the QCSP

There are three reference frames to describe the QCSP (Lv et al., 2020) system (see left of Figure 1): the inertial frame $\mathcal{I}\{X_i,Y_i,Z_i\}$, the quadrotor body frame $\mathcal{B}\{X_b,Y_b,Z_b\}$, and the payload body frame ${\mathcal{B}}_p\{X_p,Y_p,Z_p\}$. What needs to be mentioned is that the inertial frame follows the North-East-Down (NED) notation. For the quadrotor body frame, Z_b points down, the X_b toward the front direction, and the Y_b toward the right direction. Based on the reference frames, some variables are defined. The generalized coordinates $\mathit{q}={[{\mathit{\xi }}^{\top }{\mathit{\eta }}^{\top }{\mathit{\sigma }}^{\top }]}^{\top }\in {\mathbb{R}}^8$, where $\mathit{\xi }={[xyz]}^{\top }\in {\mathbb{R}}^3$ denotes the coordinate of the quadrotor’s CoG under the inertial frame $\mathcal{I}$; $\mathit{\eta }={[\varphi \theta \psi ]}^{\top }\in {\mathbb{R}}^3$ denotes the attitude angle of the quadrotor in the Euler coordinate system, and ϕ means the roll angle, θ means pitch angle, and ψ means yaw angle; $\mathit{\sigma }={[\alpha \beta ]}^{\top }\in {\mathbb{R}}^2$ denotes the swing angle of the payload, where α and β are the roll and pitch angles of the cable, respectively. The boundaries of the quadrotor attitude and the swing angle are limited as

$\varphi ,\theta ,\alpha ,\beta \in \left(-\pi /2,\pi /2\right).(1)$

FIGURE 1

FIGURE 1. The schematic diagram and the control block diagram of the QCSP.

The coordinate $\mathit{\delta }={[x_p{y}_p{z}_p]}^{\top }$ of the payload’s CoG in inertial frame $\mathcal{I}$ can be given by ξ and σ:

$x_p=x+l\mathit{c}\alpha \mathit{s}\beta ,y_p=y-l\mathit{s}\alpha ,z_p=z+l\mathit{c}\alpha \mathit{c}\beta ,(2)$

and the velocity of the payload is expressed by $\dot{\mathit{\delta }}={[{\dot{x}}_p{\dot{y}}_p{\dot{z}}_p]}^{\top }$. In addition, I_n and 0_m×n represent the n-dimensional identity matrix and m × n dimensional null matrix, respectively. c⋅ and s⋅ are used to represent cos ⋅ and sin ⋅, respectively.

Following previous work (Lv et al., 2020), the dynamic model of the QCSP system is described by the following equations:

$\ddot{\mathit{\delta }}=\left({\mathit{R}}_G{F}_l-m_q\ddot{\mathit{\xi }}+m_q\mathit{g}+{\mathit{D}}_{\mathit{\xi }}+{\mathit{D}}_{\mathit{\delta }}\right)/m_p+\mathit{g},(\mathrm{3a})$

$\ddot{\mathit{\eta }}={\mathit{J}}_q^{-1}\left({\mathit{\tau }}_{\mathit{\eta }}+\mathit{A}-{\dot{\mathit{J}}}_{\mathit{q}}\dot{\mathit{\eta }}+{\mathit{D}}_{\mathit{\eta }}\right),(\mathrm{3b})$

$\ddot{\mathit{\sigma }}=-{\mathit{M}}_1^{-1}\left({\mathit{M}}_2\ddot{\mathit{\xi }}+{\mathit{V}}_{\mathit{d}\mathit{\sigma }}-\mathit{C}-{\mathit{D}}_{\mathit{\sigma }}\right),(\mathrm{3c})$

where ${\mathit{F}}_t={\mathit{R}}_G{F}_l-m_q\ddot{\mathit{\xi }}+m_q\mathit{g}+{\mathit{D}}_{\mathit{\xi }}$ is the tensile force of the cable on the payload.

For dynamic model described by Eq. 3a, m_q is the mass of the quadrotor, m_p is the mass of the payload, g is the gravity acceleration, D_ξ and D_δ are the air drag forces that act on the quadrotor and the payload, respectively. R_G is the projection vector in the inertial frame $\mathcal{I}$ of the unit vector on the axis Z_b.

For the dynamic submodel described by Eq. (3b), D_η denotes the aerodynamic drag torque on the quadrotor. What needs to be mentioned is that $\mathit{A}={[A_{\varphi }{A}_{\theta }0]}^{\top }$ is given in Eq. 12a of (Lv et al., 2020), and the inertial matrix J_q is given in Eq. 4 of (Lv et al., 2020).

For the dynamic model described by Eq. 3c, the drag torque D_σ on the payload is given by

${\mathit{D}}_{\mathit{\sigma }}={\mathit{I}}_{2\times 3}\left(\mathit{l}\times {\mathit{R}}_i^{bp}{\mathit{D}}_{\mathit{\sigma }}\right),(4)$

where ${\mathit{I}}_{\mathit{2}\times \mathit{3}}=\left[\begin{array}{ccc}\hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \end{array}\right]$, l = [0 0 l]^⊤, l is the length of cable. M₁ = m_pl²  diag (1, c²α), ${\mathit{M}}_2=-m_{p}l\left[\begin{array}{ccc}\hfill \mathit{s}\alpha \mathit{s}\beta \hfill & \hfill \mathit{c}\alpha \hfill & \hfill \mathit{s}\alpha \mathit{c}\beta \hfill \\ \hfill -\mathit{c}\alpha \mathit{c}\beta \hfill & \hfill 0\hfill & \hfill \mathit{c}\alpha \mathit{s}\beta \hfill \end{array}\right]$, ${\mathit{V}}_{d\mathit{\sigma }}={[m_{p}gl\mathit{s}\alpha \mathit{c}\beta m_{p}gl\mathit{c}\alpha \mathit{s}\beta ]}^{\top }$, $\mathit{C}={\left[C_{\alpha }{C}_{\beta }\right]}^{\top }$, with $C_{\alpha }=-m_p{l}^2\mathit{s}\alpha \mathit{c}\alpha {\dot{\beta }}^2,C_{\beta }=2m_p{l}^2\mathit{s}\alpha \mathit{c}\alpha \dot{\alpha }\dot{\beta }$. To facilitate the controller design, the dynamic model (see Eq. 3)is rewritten as

$\ddot{\mathit{\delta }}=\left({\mathit{F}}_{\mathit{t}}+{\mathit{D}}_{\mathit{\sigma }}\right)/m_p+\mathit{g},(\mathrm{5a})$

$\ddot{\mathit{\eta }}={\mathit{J}}_q^{-1}\left({\mathit{\tau }}_{\mathit{\eta }}+{\mathit{\tau }}_{\mathit{\eta }\mathit{e}}\right),(\mathrm{5b})$

$\ddot{\mathit{\sigma }}={\mathit{M}}_{\mathit{\sigma }}\ddot{\mathit{\xi }}+{\mathit{F}}_{\mathit{\sigma }\mathit{e}}+{\mathit{M}}_1^{-1}{\mathit{D}}_{\mathit{\sigma }},(\mathrm{5c})$

where a = 1/m_p, ${\mathit{\tau }}_{\mathit{\eta }\mathit{e}}=\mathit{A}-{\dot{\mathit{J}}}_{\mathit{q}}\dot{\mathit{\eta }}+{\mathit{D}}_{\mathit{\eta }}$,

${\mathit{F}}_{\mathit{\sigma }\mathit{e}}={\mathit{M}}^{-1}\left(\mathit{C}-{\mathit{V}}_{\mathit{d}\mathit{\sigma }}\right)=\left[\begin{array}{c}\hfill -\mathit{s}\alpha \mathit{c}\alpha {\dot{\beta }}^2-\mathit{s}\alpha \mathit{s}\beta g/l\hfill \\ \hfill 2\mathit{s}\alpha /\mathit{c}\alpha \dot{\alpha }\dot{\beta }-\mathit{s}\alpha \mathit{s}\beta g/l\hfill \end{array}\right],$

${\mathit{M}}_{\mathit{\sigma }}=-{\mathit{M}}_{\mathbf{1}}^{-1}{\mathit{M}}_{\mathbf{2}}=\left[\begin{array}{ccc}\hfill \mathit{s}\alpha \mathit{s}\beta \hfill & \hfill \mathit{c}\alpha \hfill & \hfill \mathit{s}\alpha \mathit{c}\beta \hfill \\ \hfill -\mathit{c}\beta /\mathit{c}\alpha \hfill & \hfill 0\hfill & \hfill \mathit{s}\beta /\mathit{c}\alpha \hfill \end{array}\right]/l.$

It can be found that M_σ, F_δe, and F_σe do not contain a.

Apart from dynamic model controlling the basic attitude of the quadrotor, the motion of the quadrotor depends on the guide of the object detection network (ODN). Currently, because of accuracy and effectiveness, YOLOV3 (Redmon and Farhadi, 2018) is adopted in a growing number of real-world situations. However, this ODN method is computationally expensive; hence, it creates huge energy consumption, which is not friendly to the QCSP system. Therefore, compressing the ODN to obtain a lightweight ODN is essential for deploying YOLOV3 on the quadrotor. As mentioned, preserving computational performance and saving energy cost simultaneously are challenging issues. For preserving performance of the network, we utilize the sparsity-induced penalty to retrain a sparsity network indicated by scaling factors. Then, these low-significance connections distinguished by scaling factors are removed to achieve network compression. Considering the energy consumption, we add an energy-aware penalty to supervise the compression process. Specifically, the retrain objective is given by

$L=\sum _{\mathbf{X},\mathbf{Y}}l\left(f\left(\mathbf{X},\mathbf{W}\right),\mathbf{Y}\right)+\lambda \sum _{\gamma \in \mathrm{\Gamma }}\Vert \gamma {\Vert }_1+\alpha \sum _{i\in N}\left(E_{comp}^{\left(i\right)}+E_{data}^{\left(i\right)}\right),(6)$

where l (⋅) denotes the supervised training loss; for YOLOV3, l (⋅) is the detection loss proposed in (Redmon and Farhadi, 2018). (X, Y) denote the retrain input and label, W denotes the learnable weights, ‖ ⋅‖₁ denotes the L1-norm function, γ is a scaling factor and Γ is the scaling factors set, ‖γ‖₁ is used as a sparsity-induced penalty. In practice, the learnable γ in batch normalization (Ioffe and Szegedy, 2015) is widely adopted as a scaling factor. $E_{comp}^{(i)}$ denotes the energy consumption for computation of the ith layer, whereas $E_{data}^{(i)}$ denotes the energy consumption for data access of the ith layer, N means the whole network consisting of N layers. $E_{comp}^{(i)}+E_{data}^{(i)}$ is utilized as the energy-aware penalty. λ and α balance three items. Following the principles of (Yang et al., 2018), the $E_{comp}^{(i)}$ of the normal convolutional layer is given by

$\left\{\begin{array}{c}{E}_{comp}^{\left(i\right)}=e_{MAC}\cdot h^{\prime }\cdot w^{\prime }\cdot \Vert {\mathbf{W}}^{\left(i\right)}{\Vert }_0\hfill \\ h^{\prime }=\lfloor \left(h+2p-r\right)/s\rfloor +1\hfill \\ w^{\prime }=\lfloor \left(w+2p-r\right)/s\rfloor +1\hfill \end{array}\right.,(7)$

where e_MAC denotes the energy consumption of one systolic array MAC (Kung, 1982) (a kind of hardware widely used in GPU or TPU) operation, whereas h and w denote the height and weight of the convolutional layer input, ‖ ⋅‖₀ denotes the L0-norm function, p, r, s denote the convolutional arguments, i.e., padding, kernel size, and stride. Data access energy $E_{data}^{(i)}$ depends on the hardware architecture, i.e., systolic array, which is complex and not helpful for understanding our method. We just describe it as a function of X⁽ⁱ⁾, h, w, p, r, s: $E_{data}^{(i)}=E_{data}^{(i)}({\mathbf{X}}^{(i)},h,w,p,r,s),$ where X⁽ⁱ⁾ denotes the input of the ith convolutional layer. The other omitted items depend on the specific architecture of the hardware, e.g., bus bandwidth.

Obviously, Eq. 6 gives a meaningful objective. However, it is hard to optimize because of $E_{data}^{(i)}$ and $E_{comp}^{(i)}$ are not functions of scaling factors γ. To make the energy-aware penalty influence γ, we redescribe Eq. 6 as

$L^{\prime }=\sum _{\mathbf{X},\mathbf{Y}}l\left(f\left(\mathbf{X},\mathbf{W}\right),\mathbf{Y}\right)+\lambda \sum _{i\in N}\left(E_{comp}^{\left(i\right)}+E_{data}^{\left(i\right)}\right)\cdot \Vert {\mathit{\gamma }}^{\left(i\right)}{\Vert }_1,$

where γ⁽ⁱ⁾ is a scaling factor vector of the ith layer. Then, we optimize the above equation to obtain a sparsity distribution of scaling factors and remove these low-significance connections. After that, the compact network is fine-tuned for several iterations to resume.

Finally, a network deployed on a computation and energy-limited platform could be accessed. We utilize this compact network to provide location and category information of the target object to the quadrotor as a basis for flight adjustment.

3 Controller Design

Because of the underactuated character of the QCSP, the proposed controller mainly consists of two parts: the cascade controller for attitude self-stabilization and the active disturbance rejection controller for position control (see the right subfigure of Figure 1). Referring to (Lv et al., 2020), the design process of the cascade controller mainly consists of three parts: inner-loop attitude, middle-loop swing angle, and outer-loop velocity subcontrollers.

3.1 Tracking Errors

Errors associated with the dynamics of the QCSP are given as follows:

${\mathit{e}}_{\mathit{\eta },{\mathit{p}}_{\mathit{\eta }}}={\left[{{\mathit{e}}_{\mathit{\eta }}}^{\top }{{\mathit{e}}_{{\mathit{p}}_{\mathit{\eta }}}}^{\top }\right]}^{\top },(8)$

${\mathit{e}}_{\mathit{\eta }}={\mathit{\eta }}_d-\mathit{\eta },{\mathit{e}}_{{\mathit{p}}_{\mathit{\eta }}}={\dot{\mathit{\eta }}}_d-\dot{\mathit{\eta }}+{\mathit{K}}_{\mathit{\eta }}{\mathit{e}}_{\mathit{\eta }},(9)$

${\mathit{e}}_{\mathit{\sigma },{\mathit{p}}_{\mathit{\sigma }}}={\left[{{\mathit{e}}_{\mathit{\sigma }}}^{\top }{{\mathit{e}}_{{\mathit{p}}_{\mathit{\sigma }}}}^{\top }\right]}^{\top },(10)$

${\mathit{e}}_{\mathit{\sigma }}={\mathit{\sigma }}_d-\mathit{\sigma },{\mathit{e}}_{{\mathit{p}}_{\mathit{\sigma }}}={\dot{\mathit{\sigma }}}_d-\dot{\mathit{\sigma }}+{\mathit{K}}_{\mathit{\sigma }}{\mathit{e}}_{\mathit{\sigma }},(11)$

${\mathit{e}}_{\dot{\mathit{\delta }}}={\dot{\mathit{\delta }}}_d-\dot{\mathit{\delta }},{\dot{e}}_{\dot{\mathit{\delta }}}={\ddot{\mathit{\delta }}}_d-\ddot{\mathit{\delta }},(12)$

where ${\dot{\mathit{\delta }}}_d={[{\dot{x}}_{pd}{\dot{y}}_{pd}{\dot{z}}_{pd}]}^{\top }$, ${\mathit{\sigma }}_d={[{\alpha }_d{\beta }_d]}^{\top }$, and ${\mathit{\eta }}_d={[{\varphi }_d{\theta }_d{\psi }_d]}^{\top }$ with the desired attitude η_d, and the desired position δ_d, which can be determined by the object detection algorithm proposed in the above section. The positive definite diagonal matrixes K_σ = diag (k_α, k_β) and K_η = diag (k_ϕ, k_θ, k_ψ). The attitude η in (9) and the velocity $\dot{\mathit{\xi }}$ in (12) of the quadrotor are measured by the IMU integrated in the flight control system. The payload’s velocity $\dot{\mathit{\delta }}$ in (12) can be calculated by η and $\dot{\mathit{\xi }}$. From Eq. 9, the attitude error dynamic of the quadrotor is obtained as

${\dot{e}}_{\mathit{\eta }}={\mathit{e}}_{{\mathit{p}}_{\mathit{\eta }}}-{\mathit{K}}_{\mathit{\eta }}{\mathit{e}}_{\mathit{\eta }},{\dot{e}}_{{\mathit{p}}_{\mathit{\eta }}}={\ddot{\mathit{\eta }}}_d-\ddot{\mathit{\eta }}+{\mathit{K}}_{\mathit{\eta }}\left({\mathit{e}}_{{\mathit{p}}_{\mathit{\eta }}}-{\mathit{K}}_{\mathit{\eta }}{\mathit{e}}_{\mathit{\eta }}\right).(13)$

The swing angle error dynamics of the payload are deduced from Eq. 11:

${\dot{e}}_{\mathit{\sigma }}={\mathit{e}}_{{\mathit{p}}_{\mathit{\sigma }}}-{\mathit{K}}_{\mathit{\sigma }}{\mathit{e}}_{\mathit{\sigma }},{\dot{e}}_{{\mathit{p}}_{\mathit{\sigma }}}={\ddot{\mathit{\sigma }}}_d-\ddot{\mathit{\sigma }}+{\mathit{K}}_{\mathit{\sigma }}\left({\mathit{e}}_{{\mathit{p}}_{\mathit{\sigma }}}-{\mathit{K}}_{\mathit{\sigma }}{\mathit{e}}_{\mathit{\sigma }}\right).(14)$

3.2 Load Velocity Controller

3.2.1 Inner-Loop Attitude Controller

Considering the subsystem (see Eq. 5b), the inner-loop subcontroller (see the right of Figure 1) is used to control the attitude η of the quadrotor, which is measured by the inertial measurement unit (IMU) integrated in the flight control system of the quadrotor. The control torque ${\mathit{\tau }}_{\mathit{\eta }}\in {\mathbb{R}}^3$ of the inner-loop controller is given by

${\mathit{\tau }}_{\mathit{\eta }}={\mathit{J}}_q\left[\left({\mathit{I}}_3-{{\mathit{K}}_{\mathit{\eta }}}^2\right){\mathit{e}}_{\mathit{\eta }}+\left({\mathit{K}}_{\mathit{\eta }}+{\mathit{K}}_{{\mathit{p}}_{\mathit{\eta }}}\right){\mathit{e}}_{{\mathit{p}}_{\mathit{\eta }}}\right]-{\mathit{\tau }}_{\mathit{\eta }\mathit{e}},(15)$

where ${\mathit{K}}_{{\mathit{p}}_{\mathit{\eta }}}=\text{diag}(k_{p\varphi },k_{p\theta },k_{p\psi })$ denotes a constant positive definite matrix.

3.2.2 Middle-Loop Swing Angle Controller

Referring to Eq. 16 in (Lv et al., 2020), the adaptive swing angle controller is applied to make σ follow the desired σ_d. Noting Eq. 5c, ${\mathit{M}}_{\mathit{\sigma }}\ddot{\mathit{\xi }}$ is taken as the visual control input. The desired visual control input is designed as

${\mathit{M}}_{\mathit{\sigma }}{\ddot{\mathit{\xi }}}_{\mathit{d}}=\left({\mathit{I}}_2-{{\mathit{K}}_{\mathit{\sigma }}}^2\right){\mathit{e}}_{\mathit{\sigma }}+\left({\mathit{K}}_{\mathit{\sigma }}+{\mathit{K}}_{\mathit{p}\mathit{\sigma }}\right){\mathit{e}}_{\mathit{p}\mathit{\sigma }}-{\mathit{F}}_{\mathit{\sigma }\mathit{e}}-{\mathit{M}}_{\mathbf{1}}^{-1}{\mathit{D}}_{\mathit{\sigma }},(16)$

where K_pσ = diag (k_pα, k_pβ) is constant positive definite.

3.3.3 Decoupler

For the desired acceleration ${\mathit{M}}_{\mathit{\sigma }}{\ddot{\mathit{\xi }}}_d$ in Eq. 16 generated by the aforementioned adaptive swing angle controller, the quadrotor’s lift force F_l and the desired attitude ϕ_d can be decoupled by ${\mathit{M}}_{\mathit{\sigma }}{\ddot{\mathit{\xi }}}_d$ and F_td. Considering the constraint given in (1) and the mechanisms of the quadrotor maneuvers, we have F_lzd > 0, θ_d, ϕ_d ∈ ( − π/2, π/2). The decoupling result is given by

${\theta }_d=arctan\left(\left(F_{lxd}\mathit{c}\psi +F_{lyd}\mathit{s}\psi \right)/F_{lzd}\right),(\mathrm{17a})$

${\varphi }_d=-arctan\left(\left(-F_{lxd}\mathit{s}\psi +F_{lyd}\mathit{c}\psi \right)\mathit{c}{\theta }_d/F_{lzd}\right),(\mathrm{17b})$

$F_l=F_{lzd}/\left(\mathit{c}{\varphi }_d\mathit{c}{\theta }_d\right).(\mathrm{17c})$

3.3.4 Outer-Loop Velocity Controller

As illustrated in the outer-loop part of the right subfigure in Figure 1, the outer-loop velocity controller is utilized to track the desired velocity ${\dot{\mathit{\delta }}}_d$ for the translational dynamic (see Eq. 5a) of the payload. The desired tensile force ${\mathit{F}}_{\mathit{t}\mathit{d}}={[F_{txd},F_{tyd}{F}_{tzd}]}^{\top }$ is designed as

${\mathit{F}}_{\mathit{t}\mathit{d}}={\mathit{K}}_{\dot{\mathit{\delta }}}{\mathit{e}}_{\dot{\mathit{\delta }}}-{\mathit{F}}_{\mathit{\delta }\mathit{e}}-m_p\mathit{g},(18)$

with a constant positive definite diagonal matrix ${\mathit{K}}_{\dot{\mathit{\delta }}}=\text{diag}(k_{{\dot{x}}_p},k_{{\dot{y}}_p},k_{{\dot{z}}_p})$. Considering the constraints in (1) and that the cable is always taut and there is tensile force on the cable, F_tzd > 0, α_d, β_d ∈ ( − π/2, π/2). Referring to Eq. 26 in (Lv et al., 2020), the desired magnitude F_td of tensile force and the desired swing angles α_d and β_d are given by

$F_{td}=F_{tzd}/\left(\mathit{c}{\alpha }_d\mathit{c}{\beta }_d\right),(\mathrm{19a})$

${\beta }_d=arctan\left(F_{txd}/F_{tzd}\right),(\mathrm{19b})$

${\alpha }_d=-arctan\left(F_{tyd}\mathit{c}{\beta }_d/F_{tzd}\right).(\mathrm{19c})$

3.3 LADRC Based Position Controller

LADRC is utilized for the position control of quadrotors. There are two parts in the LADRC, including the linear extended state observer (LESO) and PD controller. The LESO estimates the internal and external disturbances of the system through an extended state, which is called total disturbance, and compensates the control variables. Therefore, the integrator used in traditional PID to eliminate static error under constant disturbance is no longer necessary. The system can be stabilized by PD controller.

Referring to the controller built by Gao in (Han, 2009) and taking δ_d as the expected input of the controller and δ as feedback, the designed LADRC block diagram is as follows:

3.3.1 LESO

Compared with ESO (Han, 2009), LESO introduces the frequency domain method. It connects the parameters with the observer bandwidth, making the parameter tuning more convenient.

The LESO for the position control of the QSCP is designed as

$\dot{z}=\left[A-LC\right]\mathit{z}-\left[B,L\right]{\mathit{u}}_{\mathit{c}},(\mathrm{20a})$

${\mathit{y}}_{\mathit{c}}=\mathit{z},(\mathrm{20b})$

where u_c = [u δ]^⊤ is the input of LESO, and y_c is the output. Besides this, $A=\left[\begin{array}{ccc}\hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \end{array}\right]$, $B=\left[\begin{array}{c}\hfill 0\hfill \\ \hfill b_0\hfill \\ \hfill 0\hfill \end{array}\right]$, $C=\left[\begin{array}{ccc}\hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill \end{array}\right]$, $L={\left[\begin{array}{ccc}\hfill 3{\omega }_0\hfill & \hfill 3{\omega }_0^2\hfill & \hfill {\omega }_0^3\hfill \end{array}\right]}^{\top }$. Here, b₀ can be adjusted according to the step response of the system. ω₀ is the observer bandwidth.

3.3.2 PD Controller

Under the action of LESO, the linear PD controller can stabilize the system. Besides this, the proportional coefficient and differential time constant are related to the controller bandwidth, which simplifies the tuning of the controller.

The PD controller is designed as

${\mathit{u}}_{\mathbf{0}}=k_p\left({\mathit{\delta }}_{\mathit{d}}-{\mathit{z}}_{\mathbf{1}}\right)-k_d{\mathit{z}}_{\mathbf{2}}(21)$

where δ_d is the expected input of the controller. z₁ and z₂ are observer states from LESO. k_p and k_d are the parameters of the controller gain matrix $K={[k_d{k}_p]}^{\top }$ to be designed. We choose $k_p={\omega }_c^2,k_d=2{\omega }_c$ with the controller bandwidth ω_c.

Finally, the control quantity u₀ with the total disturbance z₃ has to be compensated, and the control quantity u is $\mathit{u}=\frac{{\mathit{u}}_0-{\mathit{z}}_3}{\mathit{b}}$, where b is the gain factor.

For the position control, it is necessary to obtain the position coordinates of the target point, but the camera feeds back the pixel coordinates, which should be compensated by attitude angle and height information. Besides this, due to the relative displacement between the quadcopter and the load, the coordinates of the target point collected by the camera relative to the quadcopter should be transformed into the coordinates relative to the load.

4 Experiment

To verify the effectiveness of the proposed algorithm, a QCSP experimental platform was created. The payload is connected to the bottom of the F450 quadcopter by a Cadan joint, and the Jetson NX board is fixed to the bottom plate with the camera as shown in Figure 3A.

Before the flight test, we simulated the designed LADRC control strategy and compared it with the conventional PID algorithm as shown in Figure 2. The parameters of LADRC and PID are obtained by many experiments according to overshoot and response time. Among them, “desired” is the target position curve after transformation, “f_pid” is the position curve of the payload under PID control. “f_ladrc” is the position curve of the payload under LADRC control. It can be seen that, at 20 s, given a target position of 80 cm, the payload achieved steady state in 7 s without overshoot under the LADRC control, while the state of the system oscillated and took more than three times as long to stabilize under the PID controller. When a sin disturbance signal is added in 80 s, the LADRC can obviously suppress the disturbance. As a result, it can be deduced that the LADRC controller is effective in saving QCSP energy.

FIGURE 2

FIGURE 2. The comparison results of PID and the LADRC position controller.

The QCSP vision deployment hardware is Jetson NX, which runs aarch64 Ubuntu 18.04 as the operating system. PyTorch (Paszke et al., 2019) is used as the retraining, fine-tuning, and inference deep learning software. For efficient inference, popular object detection model YOLOV3(Redmon and Farhadi, 2018) is adopted, whose backbone is replaced by MobileNet (Howard et al., 2017) from DarkNet. The image size requires 416 × 416. The original network is retrained on a PASCAL VOC data set (Everingham et al., 2010) for 50,000 iterations and removes the connections whose scaling factors are lower than the threshold 0.01. Then, the compact network is fine-tuned for 12,000 iterations. The retraining and fine-tuning, which cost a large amount of computing power, are carried out on NVIDIA RTX 2080Ti, and only inference is done on the embedding Jetson NX platform. The target pattern is the helicopter landing area-“H”, and the recognition effect is shown in part (b) of Figure 3, where four objects are detected with confidence 0.89, 0.95, 0.98, and 0.97.

FIGURE 3

FIGURE 3. Experimental platform of QCSP object detection and position control flight test.

Compression results are reported in Table 1. Through our method, we saved about 52% energy of the whole network with only 0.7 mean average precision (mAP) dropping. In addition, we also demonstrate the necessary of using sparsity-induced and energy-aware penalties simultaneously. In the case of only the sparsity-induced penalty used, the energy saved is not satisfactory (43% saved), but performance drops a lot (1.4) when only the energy-aware penalty is used.

TABLE 1

TABLE 1. Object detection performance on PASCAL VOC. “Penalty” denotes the penalty item that we added for training loss. “Energy” denotes the energy cost for detecting one image. “Energy ↓” denotes the energy saved comparing to original network whose backbone is MobileNet; “mAP” is a common indicator evaluating the performance of detection network.

During the actual flight experiment, the quadcopter flew along the positive direction of the X axis at the speed of 20 cm/s. When the target point is identified by Jetson NX, it turn into position control mode as shown in Figure 3C. From the experiment, it can be seen that the QCSP system can control the payload stably, and when the target position is detected, it can reach the destination quickly and remain stable.

5 Conclusion

Energy-efficiency plays a crucial role in the development of UAVs. In this paper, a lightweight YOLOV3 object detection network with a LADRC-based position controller is proposed to reduce the energy consumption of the QCSP system. The experimental results show that the compressed network can save more than 50% of energy compared with the original network with little accuracy loss, and the LADRC controller has three times faster stabilization time and no overshoot compared with the classic PID controller and has a suppression effect on disturbing signals. Therefore, the work done in this paper can effectively save the energy of the QCSP and improve its range, anti-interference performance, and robustness.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author Contributions

SL contributed to the architecture, property, and training algorithm, and design of feedback controller of the QCSP systems. SL, and LF drafted the work and contributed to the experiments and conclusions. All authors agree to be accountable for the content of the work.

Funding

National Natural Science Foundation of China No. 61972064; LiaoNingRevitalizationTalentsProgram No. XLYC1806006.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A. (2010). The Pascal Visual Object Classes (Voc) challenge. Int. J. Comput. Vis. 88 (2), 303–338. doi:10.1007/s11263-009-0275-4

CrossRef Full Text | Google Scholar

Han, J. (2009). From Pid to Active Disturbance Rejection Control. IEEE Trans. Ind. Electron. 56 (3), 900–906. doi:10.1109/tie.2008.2011621

CrossRef Full Text | Google Scholar

Hao, W., Xian, B., and Xie, T. (2021). Fault Tolerant Position Tracking Control Design for an Tilt Tri-rotor Unmanned Aerial Vehicle. IEEE Trans. Ind. Electron. 99, 1. doi:10.1109/TIE.2021.3050384

CrossRef Full Text | Google Scholar

Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., et al. (2017). Mobilenets: Efficient Convolutional Neural Networks for mobile Vision Applications. California: Google Inc. arXiv preprint arXiv:1704.04861. Available at: https://arxiv.53yu.com/pdf/1704.04861.pdf%EF%BC%89.

Google Scholar

Ioffe, S., and Szegedy, C. (2015). “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,” in International conference on machine learning (PMLR), Lille, France, July 7–9, 2015, 448–456.

Google Scholar

Kung, H.-T. (1982). Why Systolic Architectures? Computer 15 (01), 37–46. doi:10.1109/mc.1982.1653825

CrossRef Full Text | Google Scholar

Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., and Zhang, C. (2017). “Learning Efficient Convolutional Networks through Network Slimming,” in Proceedings of the IEEE international conference on computer vision, Venice, Italy, October 22–29, 2017, 2736–2744. doi:10.1109/iccv.2017.298

CrossRef Full Text | Google Scholar

Lv, Z.-Y., Li, S., Wu, Y., and Wang, Q.-G. (2021). Adaptive Control for a Quadrotor Transporting a cable-suspended Payload with Unknown Mass in the Presence of Rotor Downwash. IEEE Trans. Veh. Technol. 70 (9), 8505–8518. doi:10.1109/tvt.2021.3096234

CrossRef Full Text | Google Scholar

Lv, Z.-Y., Wu, Y., and Rui, W. (2020). Nonlinear Motion Control for a Quadrotor Transporting a cable-suspended Payload. IEEE Trans. Veh. Technol. 69 (8), 8192–8206. doi:10.1109/tvt.2020.2997733

CrossRef Full Text | Google Scholar

Mochida, S., Matsuda, R., Ibuki, T., and Sampei, M. (2021). A Geometric Method of Hoverability Analysis for Multirotor Uavs with Upward-Oriented Rotors. IEEE Trans. Robotics 99, 1–15. doi:10.1109/tro.2021.3064101

CrossRef Full Text | Google Scholar

Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., et al. (2019). “PyTorch: An Imperative Style, High-Performance Deep Learning Library,” in Advances in Neural Information Processing Systems. Editors Wallach, H., Larochelle, H., Beygelzimer, A., d\textquotesingle Alch\’{e}-Buc, F., Fox, E., and Garnett, R. (Curran Associates, Inc.) 32. Available at: https://proceedings.neurips.cc/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf.

Google Scholar

Qian, L., and Liu, H. (2019). Path Following Control of a Quadrotor Uav with a cable Suspended Payload under Wind Disturbances. IEEE Trans. Ind. Electron. 67, 1. doi:10.1109/TIE.2019.2905811

CrossRef Full Text | Google Scholar

Redmon, J., and Farhadi, A. (2018). Yolov3: An Incremental Improvement. Washington D.C: University of Washington. Available at: https://arxiv.53yu.com/pdf/1804.02767.pdf.

Google Scholar

Wu, Y., Hu, K., and Sun, X.-M. (2018). Modeling and Control Design for Quadrotors: A Controlled Hamiltonian Systems Approach. IEEE Trans. Vehicular Tech. 67 (12), 11 365. doi:10.1109/tvt.2018.2877440

CrossRef Full Text | Google Scholar

Xie, Z., Zhu, L., Zhao, L., Tao, B., Liu, L., and Tao, W. (2020). Localization-aware Channel Pruning for Object Detection. Neurocomputing 403, 400–408. doi:10.1016/j.neucom.2020.03.056

CrossRef Full Text | Google Scholar

Yang, H., Zhu, Y., and Liu, J. (2018). Energy-constrained Compression for Deep Neural Networks via Weighted Sparse Projection and Layer Input Masking. New York, NY: Cornell University. arXiv preprint arXiv:1806.04321.

Google Scholar