Energy-Efficiency-Oriented Vision Feedback Control of QCSP Systems: Linear ADRC Approach

How to save the energy of unmanned aerial vehicles (UAVs) and then enable long-distance transport is a very real and difficult task. However, for UAVs, the classic object detection algorithm, such as the deep convolutional neural network–based object detection algorithm and the classic flight control algorithm, such as the PID-based position control algorithm, require significant energy, which limits the application scenarios of the UAV system. In view of this problem, this paper proposes a lightweight object detection network and a linear active disturbance rejection controller (LADRC) for the quadrotor with the cable-suspended payload (QCSP) system to improve energy efficiency. The system uses a YOLOV3 network and embeds it into the Jesson NX mobile platform to accurately detect the target position. Furthermore, a nonlinear velocity controller with a cable-suspended structure to control the velocity of the payload, a LADRC algorithm is adopted to achieve fast and accurate control of the payload position. Simulations and real flight experiments show that the proposed object detection algorithm and the LADRC control strategy can save the energy of drone effectively.


INTRODUCTION
With the development of unmanned aerial vehicle (UAV) technology (Wu et al., 2018), drone transport has become an important branch of UAV applications. The quadrotor with the cablesuspended payload (QCSP) (Lv et al., 2020;Lv et al., 2021) equipped with a camera and an embedded platform, is of great relevance to the realization of rescue and transport tasks. The QCSP actively adjusts the UAV's own attitude to quickly reduce the oscillation of the suspended load and then runs the vision algorithm through the embedded platform to process the images from the camera to obtain an accurate target position for the drone. Based on the target position information, the QCSP needs to reach the target position quickly and stably. However, in the transportation process, in addition to the energy required for drone flight, the object detection algorithm and the QCSP flight control strategy also consume great energy. Therefore, by considering the limited battery capacity of the drone, it is important to improve the energy efficiency of the QCSP systems.
Recent decades witness great progress in object detection with the development of convolutional neural networks (CNNs). To obtain a powerful network, numerous efforts are made to build large and complex architecture with high computation and energy consumption, which restricts its application on embedding devices such as a drone. To get the light network, Z. Liu et al. (Liu et al., 2017) utilize scaling factors to value the significance of connections and remove these under a threshold. This efficient method works well for classification networks. However, it lacks effectiveness for detection networks. To make the network compression method suitable to detection network, Z. Xie et al. (Xie et al., 2020) introduce location-aware loss for network compression, which helps in preserving the comprehensive ability of the detection network. These methods are not optimal for saving energy because they do not adopt an energy-aware function during compressing.
To realize the path following for the QCSP, Qian, et al. (Qian and Liu, 2019) propose a controller based on uncertainty and an interference estimator. Hao, et al. (Hao et al., 2021) propose a nonlinear, robust, fault-tolerant, position-tracking, control law for a tilt tri-rotor UAV, thus avoiding rear servo's stuck fault together with parametric uncertainties and unknown external disturbances. To enable a multirotor UAV to achieve static hovering, Mochida, et al. (Mochida et al., 2021) propose a geometric method that reveals the relationship between the position of the center of mass (CoM) and the rotor placement of a multirotor UAV with upward-oriented rotors. These methods can effectively help UAVs accomplish their tasks, but they do not take into account the energy limitation of UAVs; the algorithm is complex and not applicable to the QCSP.
Although many researchers have done meaningful work and achieved results, there are still some challenges in vision processing and position control in the energy-efficiencyoriented QCSP, mainly as follows: 1) Object detection technique-based CNNs can obtain accurate target information for a UAV, but the improved detection performance of deep neural networks also brings huge energy consumption, which is not friendly to the QCSP. 2) The QCSP needs to control the load stably and reach the target position quickly, whereas the traditional PID controller usually needs a long adjustment time for the UAV to reach the target position, which is also not conducive to the QCSP system to save energy for longdistance transportation.
To improve the energy efficiency of the QCSP systems, we propose a lightweight object detection algorithm and an LADRC payload position control strategy for the QCSP. Specifically, the object detection model is compressed by network scaling factors and an energy-aware penalty, which enables the YOLOV3 network to run on the Jetson NX embedded platform of the QCSP with low energy consumption. In addition, an efficient control strategy in the form of a string stage is used to overcome the under-actuated characteristics of the QCSP, which includes attitude, swing angle, load velocity, and load position subcontrollers. The contributions of this paper to the energy saving of QCSP mainly include 1) a new QCSP experimental platform with embedded vision detection is constructed, and a lightweight object detection network is used to obtain position information; 2) an LADRC algorithm is used to control the payload position quickly and efficiently.
The remainder of this paper is structured as follows: Section 2 introduces the dynamic model and object detection algorithm of the QCSP in detail; controller design, including the LADRC position control algorithm for the QCSP is introduced in Section 3; in Section 4, the effectiveness of the proposed QCSP system is verified through experiments. Conclusions are drawn in Section 5.

DYNAMIC MODEL AND OBJECT DETECTION ALGORITHM OF THE QCSP
There are three reference frames to describe the QCSP (Lv et al., 2020) system (see left of Figure 1): the inertial frame I {X i , Y i , Z i }, the quadrotor body frame B{X b , Y b , Z b }, and the payload body frame B p {X p , Y p , Z p }. What needs to be mentioned is that the inertial frame follows the North-East-Down (NED) notation. For the quadrotor body frame, Z b points down, the X b toward the front direction, and the Y b toward the right direction. Based on the reference frames, some variables are defined. The generalized coordinates q [ξ ⊤ η ⊤ σ ⊤ ] ⊤ ∈ R 8 , where ξ [x y z] ⊤ ∈ R 3 denotes the coordinate of the quadrotor's CoG under the inertial frame I ; η [ϕ θ ψ] ⊤ ∈ R 3 denotes the attitude angle of the quadrotor in the Euler coordinate system, and ϕ means the roll angle, θ means pitch angle, and ψ means yaw angle; σ [α β] ⊤ ∈ R 2 denotes the swing angle of the payload, where α and β are the roll and pitch angles of the cable, respectively. The boundaries of the quadrotor attitude and the swing angle are limited as ϕ, θ, α, β ∈ −π 2, π 2 .
( 1 ) The coordinate δ [x p y p z p ] ⊤ of the payload's CoG in inertial frame I can be given by ξ and σ: and the velocity of the payload is expressed by In addition, I n and 0 m×n represent the n-dimensional identity matrix and m × n dimensional null matrix, respectively. c· and s· are used to represent cos · and sin ·, respectively.
Following previous work (Lv et al., 2020), the dynamic model of the QCSP system is described by the following equations: where F t R G F l − m q € ξ + m q g + D ξ is the tensile force of the cable on the payload.
For dynamic model described by Eq. 3a, m q is the mass of the quadrotor, m p is the mass of the payload, g is the gravity acceleration, D ξ and D δ are the air drag forces that act on the quadrotor and the payload, respectively. R G is the projection vector in the inertial frame I of the unit vector on the axis Z b .
For the dynamic submodel described by Eq. (3b), D η denotes the aerodynamic drag torque on the quadrotor. What needs to be mentioned is that A [A ϕ A θ 0] ⊤ is given in Eq. 12a of (Lv et al., 2020), and the inertial matrix J q is given in Eq. 4 of (Lv et al., 2020). For the dynamic model described by Eq. 3c, the drag torque D σ on the payload is given by To facilitate the controller design, the dynamic model (see Eq. 3)is rewritten as It can be found that M σ , F δe , and F σe do not contain a. Apart from dynamic model controlling the basic attitude of the quadrotor, the motion of the quadrotor depends on the guide of the object detection network (ODN). Currently, because of accuracy and effectiveness, YOLOV3 (Redmon and Farhadi, 2018) is adopted in a growing number of real-world situations. However, this ODN method is computationally expensive; hence, it creates huge energy consumption, which is not friendly to the QCSP system. Therefore, compressing the ODN to obtain a lightweight ODN is essential for deploying YOLOV3 on the quadrotor. As mentioned, preserving computational performance and saving energy cost simultaneously are challenging issues. For preserving performance of the network, we utilize the sparsity-induced penalty to retrain a sparsity network indicated by scaling factors. Then, these lowsignificance connections distinguished by scaling factors are removed to achieve network compression. Considering the energy consumption, we add an energy-aware penalty to supervise the compression process. Specifically, the retrain objective is given by where l (·) denotes the supervised training loss; for YOLOV3, l (·) is the detection loss proposed in (Redmon and Farhadi, 2018). (X, Y) denote the retrain input and label, W denotes the learnable weights, · 1 denotes the L1-norm function, γ is a scaling factor and Γ is the scaling factors set, γ 1 is used as a sparsity-induced penalty. In practice, the learnable γ in batch normalization (Ioffe and Szegedy, 2015) is widely adopted as a scaling factor. E (i) comp denotes the energy consumption for computation of the ith layer, whereas E (i) data denotes the energy consumption for data access of the ith layer, N means the whole network consisting of N layers.
data is utilized as the energy-aware penalty. λ and α balance three items. Following the principles of (Yang et al., 2018), the E (i) comp of the normal convolutional layer is given by where e MAC denotes the energy consumption of one systolic array MAC (Kung, 1982) (a kind of hardware widely used in GPU or TPU) operation, whereas h and w denote the height and weight of the convolutional layer input, · 0 denotes the L0-norm function, p, r, s denote the convolutional arguments, i.e., padding, kernel size, and stride. Data access energy E (i) data depends on the hardware architecture, i.e., systolic array, which is complex and not helpful for understanding our method. We just describe it as a function of X (i) , h, w, p, r, s: data (X (i) , h, w, p, r, s), where X (i) denotes the input of the ith convolutional layer. The other omitted items depend on the specific architecture of the hardware, e.g., bus bandwidth.
Obviously, Eq. 6 gives a meaningful objective. However, it is hard to optimize because of E (i) data and E (i) comp are not functions of scaling factors γ. To make the energy-aware penalty influence γ, we redescribe Eq. 6 as where γ (i) is a scaling factor vector of the ith layer. Then, we optimize the above equation to obtain a sparsity distribution of scaling factors and remove these low-significance connections. After that, the compact network is fine-tuned for several iterations to resume. Finally, a network deployed on a computation and energylimited platform could be accessed. We utilize this compact network to provide location and category information of the target object to the quadrotor as a basis for flight adjustment.

CONTROLLER DESIGN
Because of the underactuated character of the QCSP, the proposed controller mainly consists of two parts: the cascade controller for attitude self-stabilization and the active disturbance rejection controller for position control (see the right subfigure of Figure 1). Referring to (Lv et al., 2020), the design process of the cascade controller mainly consists of three parts: inner-loop attitude, middle-loop swing angle, and outer-loop velocity subcontrollers.

Tracking Errors
Errors associated with the dynamics of the QCSP are given as follows: where with the desired attitude η d , and the desired position δ d , which can be determined by the object detection algorithm proposed in the above section. The positive definite diagonal matrixes K σ = diag (k α , k β ) and K η = diag (k ϕ , k θ , k ψ ). The attitude η in (9) and the velocity _ ξ in (12) of the quadrotor are measured by the IMU integrated in the flight control system. The payload's velocity _ δ in (12) can be calculated by η and _ ξ. From Eq. 9, the attitude error dynamic of the quadrotor is obtained as The swing angle error dynamics of the payload are deduced from Eq. 11:

Inner-Loop Attitude Controller
Considering the subsystem (see Eq. 5b), the inner-loop subcontroller (see the right of Figure 1) is used to control the attitude η of the quadrotor, which is measured by the inertial measurement unit (IMU) integrated in the flight control system of the quadrotor. The control torque τ η ∈ R 3 of the inner-loop controller is given by where K p η diag(k pϕ , k pθ , k pψ ) denotes a constant positive definite matrix.

Middle-Loop Swing Angle Controller
Referring to Eq. 16 in (Lv et al., 2020), the adaptive swing angle controller is applied to make σ follow the desired σ d . Noting Eq. 5c, M σ € ξ is taken as the visual control input. The desired visual control input is designed as where K pσ = diag (k pα , k pβ ) is constant positive definite.

Decoupler
For the desired acceleration M σ € ξ d in Eq. 16 generated by the aforementioned adaptive swing angle controller, the quadrotor's lift force F l and the desired attitude ϕ d can be decoupled by M σ € ξ d and F td . Considering the constraint given in (1) and the mechanisms of the quadrotor maneuvers, we have F lzd > 0, θ d , ϕ d ∈ ( − π/2, π/2). The decoupling result is given by

Outer-Loop Velocity Controller
As illustrated in the outer-loop part of the right subfigure in Figure 1, the outer-loop velocity controller is utilized to track the desired velocity _ δ d for the translational dynamic (see Eq. 5a) of the payload. The desired tensile force F td [F txd , F tyd F tzd ] ⊤ is designed as with a constant positive definite diagonal matrix K _ δ diag(k _ x p , k _ y p , k _ z p ). Considering the constraints in (1) and that the cable is always taut and there is tensile force on the cable, F tzd > 0, α d , β d ∈ ( − π/2, π/2). Referring to Eq. 26 in (Lv et al., 2020), the desired magnitude F td of tensile force and the desired swing angles α d and β d are given by

LADRC Based Position Controller
LADRC is utilized for the position control of quadrotors. There are two parts in the LADRC, including the linear extended state observer (LESO) and PD controller. The LESO estimates the internal and external disturbances of the system through an extended state, which is called total disturbance, and compensates the control variables. Therefore, the integrator used in traditional PID to eliminate static error under constant disturbance is no longer necessary. The system can be stabilized by PD controller.
Referring to the controller built by Gao in (Han, 2009) and taking δ d as the expected input of the controller and δ as feedback, the designed LADRC block diagram is as follows:

LESO
Compared with ESO (Han, 2009), LESO introduces the frequency domain method. It connects the parameters with the observer bandwidth, making the parameter tuning more convenient.
The LESO for the position control of the QSCP is designed as where u c = [u δ] ⊤ is the input of LESO, and y c is the output.
Besides this, A Here, b 0 can be adjusted according to the step response of the system. ω 0 is the observer bandwidth.

PD Controller
Under the action of LESO, the linear PD controller can stabilize the system. Besides this, the proportional coefficient and differential time constant are related to the controller bandwidth, which simplifies the tuning of the controller.
The PD controller is designed as where δ d is the expected input of the controller. z 1 and z 2 are observer states from LESO. k p and k d are the parameters of the controller gain matrix K [k d k p ] ⊤ to be designed. We choose k p ω 2 c , k d 2ω c with the controller bandwidth ω c . Finally, the control quantity u 0 with the total disturbance z 3 has to be compensated, and the control quantity u is u u0−z3 b , where b is the gain factor.
For the position control, it is necessary to obtain the position coordinates of the target point, but the camera feeds back the pixel coordinates, which should be compensated by attitude angle and height information. Besides this, due to the relative displacement between the quadcopter and the load, the coordinates of the target point collected by the camera relative to the quadcopter should be transformed into the coordinates relative to the load.

EXPERIMENT
To verify the effectiveness of the proposed algorithm, a QCSP experimental platform was created. The payload is connected to the bottom of the F450 quadcopter by a Cadan joint, and the Jetson NX board is fixed to the bottom plate with the camera as shown in Figure 3A.
Before the flight test, we simulated the designed LADRC control strategy and compared it with the conventional PID algorithm as shown in Figure 2. The parameters of LADRC and PID are obtained by many experiments according to overshoot and response time. Among them, "desired" is the target position curve after transformation, "f pid " is the position curve of the payload under PID control. "f ladrc " is the position curve of the payload under LADRC control. It can be seen that, at 20 s, given a target position of 80 cm, the payload achieved steady state in 7 s without overshoot under the LADRC control, while the state of the system oscillated and took more than three times as long to stabilize under the PID controller. When a sin disturbance signal is added in 80 s, the LADRC can obviously suppress the disturbance. As a result, it can be deduced that the LADRC controller is effective in saving QCSP energy.
The QCSP vision deployment hardware is Jetson NX, which runs aarch64 Ubuntu 18.04 as the operating system. PyTorch (Paszke et al., 2019) is used as the retraining, fine-tuning, and inference deep learning software. For efficient inference, popular object detection model YOLOV3 (Redmon and Farhadi, 2018) is adopted, whose backbone is replaced by MobileNet (Howard et al., 2017) from DarkNet. The image size requires 416 × 416. The original network is retrained on a PASCAL VOC data set (Everingham et al., 2010) for 50,000 iterations and removes the connections whose scaling factors are lower than the threshold 0.01. Then, the compact network is fine-tuned for 12,000 iterations. The retraining and finetuning, which cost a large amount of computing power, are carried out on NVIDIA RTX 2080Ti, and only inference is done on the embedding Jetson NX platform. The target pattern is the helicopter landing area-"H", and the recognition effect is shown in part (b) of Figure 3, where four objects are detected with confidence 0.89, 0.95, 0.98, and 0.97.
Compression results are reported in Table 1. Through our method, we saved about 52% energy of the whole network with only 0.7 mean average precision (mAP) dropping. In addition, we also demonstrate the necessary of using sparsity-induced and energy-aware penalties simultaneously. In the case of only the sparsity-induced penalty used, the energy saved is not satisfactory (43% saved), but performance drops a lot (1.4) when only the energy-aware penalty is used. During the actual flight experiment, the quadcopter flew along the positive direction of the X axis at the speed of 20 cm/s. When the target point is identified by Jetson NX, it turn into position control mode as shown in Figure 3C. From the experiment, it can be seen that the QCSP system can control the payload stably, and when the target position is detected, it can reach the destination quickly and remain stable.

CONCLUSION
Energy-efficiency plays a crucial role in the development of UAVs. In this paper, a lightweight YOLOV3 object detection network with a LADRC-based position controller is proposed to reduce the energy consumption of the QCSP system. The experimental results show that the compressed network can save more than 50% of energy compared with the original network with little accuracy loss, and the LADRC controller has three times faster stabilization time and no overshoot compared with the classic PID controller and has a suppression effect on disturbing signals. Therefore, the work done in this paper can effectively save the energy of the QCSP and improve its range, anti-interference performance, and robustness. TABLE 1 | Object detection performance on PASCAL VOC. "Penalty" denotes the penalty item that we added for training loss. "Energy" denotes the energy cost for detecting one image. "Energy ↓" denotes the energy saved comparing to original network whose backbone is MobileNet; "mAP" is a common indicator evaluating the performance of detection network.

Backbone
Penalty Energy (J) Energy ↓ mAP