# Deep learning method teaches aircraft to fly around the clock in 12 minutes

“One of the more interesting applications of Neural-Fly is flying cars or urban air mobility. As air traffic density increases, humans will need more precise, smarter, safer, and more adaptive decision-making. Now we are putting Neural-Fly is applied to UAV systems other than quadrotors, especially electric UAVs that can take off and land vertically. Other application scenarios include UAV cargo delivery, flying ambulances, etc.” Coming in 2023 in the entry card Shi Guanya, an assistant professor at the Institute of Robotics at Kimmelon University, said.

Recently, he developed a deep learning-based robot control method called Neural-Fly. The methodological and theoretical part of Neural-Fly is not limited to flying robots, but applies to all robotic systems that can be described by Euler-Lagrange equations. Currently, the team is considering extending the method to footed robots, and has made some progress.

The related paper was published under the title “Neural-Fly can quickly learn agile flying in strong winds”. Guanya Shi is the co-first author (in alphabetical order), and Soon-Jo Chung, a professor in the Department of Engineering and Applied Science at Caltech, is the corresponding author.

Project Motivation and R&D Process

The research, which began in late 2019, was initially initiated because although the team’s previous work “N eural-Lander” solved the problem of learning aerodynamics in a single environment, it did not adapt online in a dynamic environment. .

Another motivation for the project was the establishment of the Robotics and Automation Center at Caltech in 2017. What makes Shi Guanya very excited is that the center has introduced a wind wall that can control wind conditions in real time. Unlike traditional wind tunnels, the researchers can fly drones in front of this wind wall, which usually requires the aircraft to be fixed.

Therefore, he naturally had the following idea: why not collect data from multiple wind conditions, use deep learning tools to “learn the rules” from these data, and then do online adaptive control?

Soon, they put ideas into action. The first step of the project is to collect data, and finally 6 sets of flight data under different wind conditions and a total of 12 minutes are collected.

After collecting the data, the research group developed a meta-learning algorithm, DAIML, to learn a set of basis functions that can represent all wind conditions. This set of basis functions is represented by a deep neural network. Essentially, DAIML approximately solves a two-layer optimization problem with an adversarial discriminator.

After obtaining this set of basis functions, they developed a matching nonlinear adaptive control algorithm for online weight update. Specifically, the developed adaptive control algorithm is based on composite adaptive control, and spectral normalization and L2 regularization are introduced to ensure exponential stability and robustness.

Once the algorithm is developed, the next step is testing. The research group first tested the control accuracy of Neural-Fly in the wind wall, and made a lateral comparison with some other adaptive control algorithms. It is worth noting that the maximum wind speed they tested was 12 m/s, and the data collection process only included wind speeds of 0 to 6 m/s. That is to say, the basis functions learned by the DAIML algorithm developed by it have strong generalization ability.

At the same time, they also tested the performance of Neural-Fly flying outdoors. It was found that Neural-Fly performed very well and robustly outdoors even without an indoor positioning system.

After completing the algorithm development and experiments, the next step is some interesting “muscle show” scenes. For example, demonstrations such as drones accurately traversing obstacles in the wind.

Of course, the research process was by no means smooth. From the initiation of the project to the publication of the paper, the research team has damaged two UAVs and 27 rotors successively. This once again shows that if you want to apply deep learning-based methods in systems with extremely high safety requirements, such as drones, you must be cautious.

Another interesting thing happened when the team was interviewed by a foreign media. On the day of the interview, they bought an umbrella with the Caltech logo printed on it, and found that the wind speed tested could easily break the umbrella, while the drone could maintain its position and attitude stably with an error of only about 1 cm.

Technical core: stable and precise flight in changing strong winds

As far as the relevant technical core is concerned, the team is interested in how the robot can quickly and reliably learn online adaptively in an unknown and dynamic environment. For example, it hopes that a footed robot can adapt to the complex surface environment, so that it can walk smoothly on different ground; and hope that a drone can adapt to complex aerodynamics online, so as to maintain stable and precise flight in changing strong winds .

The challenge of this problem comes from three aspects:

(1) The complex dynamic environment determines the dynamic equation of the robot, which is highly nonlinear and time-varying, which requires the robot’s control algorithm to be able to quickly adapt online.

(2) For robotic systems such as drones, reliability is very important, and the team needs to ensure that the control algorithm is theoretically robust, stable, and safe.

(3) The computing resources that the robot can mobilize online are limited. For example, only the Raspberry Pi is used in this project. Therefore, the research team needs to design an efficient and feasible control algorithm.

Given the powerful learning ability of deep neural networks for complex models, control algorithms based on deep learning have made progress in many problems. However, purely deep learning based methods do not meet the above requirements.

On the one hand, even methods such as transfer learning, meta-learning, and representation learning can fine-tune and adapt the learned model to a certain extent. But generally speaking, it is impossible for researchers to fine-tune a multi-layer neural network model at high frequency using only airborne resources.

For example, in this project, the research group needs a model update frequency faster than 50 Hz to cope with the rapidly changing aerodynamics, which far exceeds the upper limit of the online update speed of the neural network. On the other hand, due to the black-box nature of the model, it is often difficult for deep learning-based control algorithms to guarantee the safety, robustness, and stability of closed-loop systems.

In control theory, adaptive control focuses on how to design a controller to adapt to an unknown dynamic model. Among them, the most representative framework is called Model Reference Adaptive Control.

Different from deep learning-based methods, model-referenced adaptive control first models the dynamic system, and some parameters in the model are unknown, such as the payload weight of the UAV. Then, based on these unknown parameters, model reference adaptive control designs a control law and an adaptive law to update the unknown parameters, thereby achieving the convergence and Lyapunov stability of the closed-loop system.

However, traditional adaptive control can only solve the problem of parametric uncertainty, that is, it must know the “structure” or “form” of the uncertainty of the dynamic model. In addition, it is difficult for traditional adaptive control to learn rules from offline data to optimize the online adaptive process.

Neural-Fly, on the other hand, combines the philosophy of model reference adaptive control with the methodology of modern deep learning. Specifically, the team proposes a new adaptive control framework: meta-adaptive control.

Its core idea is as follows. First, the team collected flight data under several different wind conditions. Afterwards, they developed a meta-learning algorithm (DAIML) to optimize a set of basis functions representing all wind conditions, represented by a deep neural network.

Later, it developed a matching nonlinear adaptive control algorithm to update the online weights. In other words, Neural-Fly in the offline stage learns a set of basis functions that can represent aerodynamics under different wind conditions, and then recombines the set of basis functions through the framework of model reference adaptive control in the online stage.

Two things are worth noting: First, the team did not learn the full dynamics model, but instead focused on residual aerodynamics after removing relatively simple rigid body dynamics. The main advantage of this model-based residual learning is that it greatly improves the efficiency of data utilization and can provide the basic structure of an adaptive controller. Second, benefiting from the framework of model-referenced adaptive control, the team’s method has strict Lyapunov stability and robustness guarantees.

Such research has also been well received by reviewers, who agreed that the team’s work presents a novel, deep learning-based framework for adaptive control.

Its main advantages are: on the one hand, data efficiency and computational efficiency are very high, only 12 minutes of data are required, and it can be run on an airborne microcomputer; on the other hand, there is a safe and reliable theoretical guarantee; finally, the control effect of this research is far better than a number of baseline methods.