An AI system designed by researchers at the University of Zurich has beaten world champions in a series of drone races.
The Swift AI won multiple races against three world-class champions in first-person view (FPV) drone racing, where pilots fly quadcopters at speeds exceeding 100km/h (62mph), controlling them remotely while wearing a headset linked to an onboard camera.
The Swift AI pilot reacts in real time to data collected by a camera onboard the drone, like the one used by human racers. An integrated inertial measurement unit measures acceleration and speed while an artificial neural network processes the data from the camera to localize the drone in space and detect the gates along the racetrack.
The information is fed to a control unit, also based on a deep neural network that chooses the best action to finish the circuit as fast as possible.
Davide Scaramuzza, head of the Robotics and Perception Group at the University of Zurich said, “Physical sports are more challenging for AI because they are less predictable than board or video games. We don’t have a perfect knowledge of the drone and environment models, so the AI needs to learn them by interacting with the physical world.
Most unmanned drone racers up to this point use an external position-tracking system to precisely control their trajectories.
Swift was trained in a simulated environment where it taught itself to fly by trial and error, using a type of machine learning called reinforcement learning. The use of simulation helped avoid destroying multiple drones in the early stages of learning when the system often crashes.
In this phase, the drone used a position-tracking system, while also recording data from its camera. This way it learned to autocorrect errors it made interpreting data from the onboard sensors.
“To make sure that the consequences of actions in the simulator were as close as possible to the ones in the real world, we designed a method to optimize the simulator with real data,” says Elia Kaufmann, a researcher from the University of Zurich on the project.
Human pilots still adapt better
After a month of simulated flight time, which corresponds to less than an hour on a desktop PC, Swift was ready to challenge its human competitors: the 2019 Drone Racing League champion Alex Vanover, the 2019 MultiGP Drone Racing champion Thomas Bitmatta, and three-times Swiss champion Marvin Schaepper.
The races took place between 5 and 13 June 2022, on a purpose-built track in a hangar of the Dübendorf Airport, near Zurich. The track covered an area of 25 by 25 meters, with seven square gates that had to be passed in the right order to complete a lap, including challenging maneuvers including a Split-S, an acrobatic feature that involves half-rolling the drone and executing a descending half-loop at full speed.
Overall, Swift achieved the fastest lap, with a half-second lead over the best lap by a human pilot. On the other hand, human pilots proved more adaptable than the autonomous drone, which failed when the conditions were different from what it was trained for, e.g., if there was too much light in the room.
Forest fires and rescue missions?
The researchers believe real-world applications for the Swift AI could include environmental monitoring or disaster response.
In applications such as forest monitoring or space exploration, for example, flying fast is important to cover large spaces in a limited time. In the film industry, fast autonomous drones could be used for shooting action scenes. And the ability to fly at high speeds could make a huge difference for rescue drones sent inside a building on fire.
Scaramuzza said, “Drones have a limited battery capacity; they need most of their energy just to stay airborne. Thus, by flying faster we increase their utility.”
