MonoRace: Winning Champion-Level Drone Racing with Robust Monocular AI

arXiv:2601.15222 · cs.RO · Submitted 2026-01-21 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "MonoRace: Winning Champion-Level Drone Racing with Robust Monocular AI".

Dev: The gist The proposed approach won the 2025 Abu Dhabi Autonomous Drone Racing Competition (A2RL),

Rosa: First, who's behind it and why it matters.

Paper summary: Rosa: So we're looking at MonoRace today. This paper claims they developed an onboard approach for winning champion-level drone racing using just a monocular camera and an IMU, without any external motion tracking system. It says this is a major step because current state of the art systems use stereo cameras and IMUs to beat human champions in controlled indoor settings, but MonoRace generalizes that method to any competition environment.

Dev: That’s the core thesis, Rosa. The paper argues that you can get robust state estimation by combining neural network-based gate segmentation with a drone model. It also mentions an offline optimization procedure that uses known geometry of gates to refine those estimation parameters based purely on onboard flight data.

Taro: I'm interested in how they tackle the perception side, because at high speeds, you get motion blur and inter-frame displacements which really mess up computer vision. How does this monocular approach handle that without external tracking?

Rosa: They capture images of eight hundred twenty by six hundred sixteen pixels at ninety Hertz, and they adaptively crop and resize those images to three hundred eighty-four by three hundred eighty-four based on predicted gate locations. Then, these crops go through a GateNet architecture for segmentation, which produces five output maps for multi-scale predictions.

Dev: And after that segmentation, they use something called QuAdGate to extract precise gate corners without relying only on the mask extremities. QuAdGate uses line detectors to identify line segments and computes their intersections to determine corner candidates.

Taro: So they aren't just using a simple bounding box or boundary detection for the gates; they are actively computing intersection points from detected lines. That sounds like it handles occlusion better, right?

Rosa: Exactly. Corner matching then involves matching those candidates with prior gate corners projected from an Extended Kalman Filter estimate using handcrafted descriptors. The drone's pose is estimated via a Perspective-n-Point solution combined with attitude estimates from an EKF.

Dev: And that state is fused with high-rate IMU data in another EKF to make sure the final state estimation is both robust and accurate. That’s critical for the control loop, Dev, because you need a reliable input signal at every step.

Taro: But what about when things go wrong? The paper mentions they have a model-based acceleration prediction mechanism if there's a discrepancy between measured and predicted accelerations that exceeds a certain threshold. What does that look like in practice when the drone hits something unexpected?

Paper summary: Rosa: It’s essentially an internal check. If the IMU data tells you one thing about acceleration, but your drone model predicts another, and that difference is too big, the system uses a prediction to smooth things out before the control gets messed up. It's a safety mechanism against sensor saturation.

Dev: And this whole pipeline feeds into their guidance and control system, which is a Guidance-and-Control Network or G andCNet. This network directly maps the estimated state to motor commands, running on the flight controller at five hundred Hertz. It skips inner loop controllers entirely by just sending those raw motor commands.

Taro: That direct mapping sounds fast, but it means if the state estimation has even a tiny bit of drift, that error gets passed straight to the motors without any intermediate stabilization from a traditional PID loop or anything. How do they manage that latency?

Dev: They rely on the high-rate IMU data feeding into their final EKF for the state estimate, and they tune everything for this five hundred Hertz rate to keep the control loop tight enough. The paper shows that this approach won the two thousand twenty-five Abu Dhabi Autonomous Drone Racing Competition A2RL <ref:2601.15222#pg3>.

Rosa: That win is significant because they outperformed all competing AI teams and three human world champion pilots in a direct knockout tournament. They also set a new milestone in research, reaching speeds up to one hundred kilometers per hour on the competition track.

Taro: So they managed to cope with things like camera interference and IMU saturation while achieving those high speeds? What were the main hurdles they overcame that weren't just standard sensor noise?

Dev: They addressed those issues through both the gating segmentation refinement and that offline optimization procedure based purely on onboard flight data. This offline optimization refines vital external camera calibration parameters by minimizing prediction error using a metric called Intersection over Union, or IoU.

Rosa: That means they are using past flight data to fine-tune how the camera sees things in real-time, which is important for those calibration parameters they mentioned. It’s a self-tuning aspect that doesn't need external infrastructure for the main flight.

Paper summary: Taro: So, what does this mean for someone who just watches drone racing? It means you can have an AI system that works reliably on a single camera and internal sensors, even when things get tough in a real competition setting.

Dev: It moves away from needing external motion tracking infrastructure, which substantially reduces the economic costs of these systems and opens up many more types of environments for autonomous drone racing applications.

Rosa: The MonoRace approach is all about proving that you can get champion-level performance using just a monocular camera and IMU. It shows how robust state estimation can be built from perception, geometry, and optimization without relying on external aids.

Taro: If the system runs outside the lab for a long time, Rosa, what’s the biggest thing that usually misbehaves? Is it lighting changes or something else?

Rosa: Outside trials show that handling unexpected environmental variations is tough. The paper notes they have to handle problems like camera interference and IMU saturation gracefully when the world isn't perfectly controlled. It shows they can cope with those kinds of issues.

Dev: And from a control engineering standpoint, the fact that they use a direct guidance network running at five hundred Hertz without inner loop controllers is a design choice that prioritizes speed and low latency for the competition environment. That’s how you get those high speeds on track.

Taro: It’s interesting because it shows that sometimes cutting out extra layers of control just to keep up with the required response time is a necessary trade-off when you're fighting against physical constraints.

Rosa: So, to wrap up this look at MonoRace, it’s about achieving champion-level drone racing performance using a monocular camera and IMU by combining neural network perception with an offline optimization procedure for calibration. It pushes the limits of what you can get from onboard AI under tight resource constraints.

Dev: And the implication is that we can build more independent, cost-effective autonomous aerial systems that don't require expensive external tracking infrastructure to operate in competitive settings.

Taro: The challenge going forward is making sure that this robust state estimation holds up when the drone encounters completely novel situations it hasn't seen during its offline optimization phase.

Rosa: That’s the next big question for autonomy research, figuring out how to make those systems truly general and resilient in unpredictable, real-world scenarios.

Conclusion: Rosa: So, to wrap up MonoRace, we're looking at how they managed to take that title for the Abu Dhabi Autonomous Drone Racing Competition.

Dev: It’s a paper about using just a monocular camera and an IMU to win against human champions without any external motion tracking system.

Taro: The authors, Rosa and Dev, are focused on making this onboard approach robust enough for real competition conditions.

Rosa: They set out to beat the current state of the art which often uses stereo cameras and IMUs in controlled indoor settings.

Dev: But their goal was to make it work in a competition environment where things get messy, like camera interference or IMU saturation.

Taro: And they achieved that by combining neural network gate segmentation with an offline optimization procedure for the camera calibration.

Rosa: So, this approach is about generalizing state estimation without needing that extra hardware for motion tracking.

Dev: It’s a major step because it shows you can get champion-level performance using just what's on board the drone.

Taro: It pushes the limits of what you can get from onboard AI under tight resource constraints for racing applications.

Rosa: The implication is that we can build more independent, cost-effective autonomous aerial systems that don't need expensive external tracking infrastructure to operate in competition settings.

Dev: But there’s still the question of how resilient it is when the drone sees something completely new during a race.

Taro: Exactly. If the system hasn't seen a situation during its offline optimization phase, will it handle that novel situation gracefully?

Rosa: That’s what we need to figure out next: making these systems truly general and resilient in unpredictable, real-world scenarios.

Delft University of Technology

cs.RO

Submitted: 2026-01-21

Updated: 2026-01-21

DOI: 10.1126/scirobotics.aee9250

Code: https://github.com/ethz-asl/kalibr

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: The gist The proposed approach won the 2025 Abu Dhabi Autonomous Drone Racing Competition (A2RL), outperforming all competing AI teams and three human world champion pilots in a direct knockout

Key concepts

MonoRace: A novel onboard drone racing approach that uses a single camera (monocular) and an Inertial Measurement Unit (IMU) for state estimation. It generalizes well to competition environments without needing external motion tracking systems, allowing the drone to navigate and control itself autonomously during high-speed racing.
GateNet: A U-Net style neural network architecture used for gate segmentation. This network processes images to identify specific areas of interest—in this case, gates—producing five different output maps at various scales. It helps precisely locate the boundaries of the gates in the drone's view.
QuAdGate: A component that extracts precise gate corners from segmentation masks. Instead of just using the edges of a mask, QuAdGate uses line detectors to find line segments and calculates their intersections to pinpoint exact corner candidates for more accurate gate identification.
Offline Optimization: A procedure performed purely on onboard flight data to fine-tune external camera calibration parameters. It minimizes prediction error by using the Intersection over Union (IoU) metric between expected and observed gate pixels, refining the system's accuracy within competition timing constraints.

Terminology

Artificial Intelligence — 1× in this paper
Artificial Intelligence (AI) — 1× in this paper · explained in 2 episodes
Calibration — 1× in this paper · explained in 9 episodes
Fine-Tuning — 1× in this paper · explained in 5 episodes

Summary

The gist The proposed approach won the 2025 Abu Dhabi Autonomous Drone Racing Competition (A2RL), outperforming all competing AI teams and three human world champion pilots in a direct knockout tournament

How it works

The MonoRace approach utilizes a monocular, rolling-shutter camera and an Inertial Measurement Unit (IMU) to achieve robust state estimation without external motion tracking systems

This pipeline features several key components for perception and control

The state estimation begins by capturing images of 820 × 616 at 90 Hz, which are adaptively cropped and resized to 384×384 based on predicted gate locations

The resulting crops are processed by GateNet for gate segmentation

GateNet is a U-Net–style architecture that produces five output maps for multi-scale predictions

To extract precise gate corners from the segmentation masks without relying solely on mask extremities is QuAdGate

QuAdGate uses line detectors to identify line segments and computes their intersections to determine corner candidates

Corner matching involves matching these candidates with prior gate corners projected from the Extended Kalman Filter (EKF) estimate using handcrafted descriptors

The drone's pose is estimated using a Perspective-n-Point (PnP) solution combined with attitude estimates from an EKF

The state is then fused with high-rate IMU data in an EKF to ensure robust and accurate state estimation

Guidance and control are performed by a Guidance-and-Control Network (G&CNet) that directly maps the estimated state to motor commands

The G&CNet runs on the flight controller at 500 Hz and foregoes inner loop controllers by directly sending motor commands

Robustness and Optimization

The system incorporates several mechanisms to handle challenging real-world conditions

To address IMU saturation during high-g maneuvers, a model-based acceleration prediction mechanism is implemented when the discrepancy between measured and predicted accelerations exceeds a threshold

The system also uses an offline optimization procedure based purely on onboard flight data to refine vital external camera calibration parameters

This refinement is achieved by minimizing prediction error through a metric called Intersection over Union (IoU) between expected and observed gate pixels

The optimization process uses Bayesian optimization to fine-tune these parameters within the strict timing constraints of the competition

Performance and Results

The MonoRace approach achieved a maximum speed of 28.23 m/s on a 76 × 18 × 5.4 m track

The controller achieved the fastest completion time of 16.

Improvements for AI systems

Improve state estimation robustness by implementing an onboard algorithm that continuously compares predicted accelerations from a dynamic drone model with measured accelerations during all high-speed competition flights to correct for IMU saturation. This allows the system to maintain accurate state estimation even when the accelerometer saturates, which previously led to multiple crashes.
Enhance gate localization accuracy by utilizing an offline optimization procedure that leverages known geometry, specifically using prediction error minimization via comparing the expected racing gate pixels and the actually observed racing gate pixels in the gate segmentation images to refine parameters like external camera calibration parameters.
Increase perception robustness against sensor corruption by integrating a fail-safe strategy where the system uses a RANSAC-based outlier rejection and a Kalman filter that filters out measurements that deviate too much from the predicted state, based on the uncertainty level of the Kalman Filter, allowing survival despite 50% image corruption.
Improve control latency and responsiveness by using a Guidance-and-Control Network (G&CNet) trained via Proximal Policy Optimization (PPO) which directly outputs motor commands, enabling the controller to operate at 500 Hz with full throttle authority and changing commands in 2 ms.
Enable generalization across different hardware by applying extensive domain randomization, as the RL policy is trained using this method to bridge the gap between simulation and reality, ensuring that real-world completion times closely match the average simulated times.

Abstract

Autonomous drone racing represents a major frontier in robotics research. It requires an Artificial Intelligence (AI) that can run on board light-weight flying robots under tight resource and time constraints, while pushing the physical system to its limits. The state of the art in this area consists of a system with a stereo camera and an inertial measurement unit (IMU) that beat human drone racing champions in a controlled indoor environment. Here, we present MonoRace: an onboard drone racing approach that uses a monocular, rolling-shutter camera and IMU that generalizes to a competition environment without any external motion tracking system. The approach features robust state estimation that combines neural-network-based gate segmentation with a drone model. Moreover, it includes an offline optimization procedure that leverages the known geometry of gates to refine any state estimation parameter. This offline optimization is based purely on onboard flight data and is important for fine-tuning the vital external camera calibration parameters. Furthermore, the guidance and control are performed by a neural network that foregoes inner loop controllers by directly sending motor commands. This small network runs on the flight controller at 500Hz. The proposed approach won the 2025 Abu Dhabi Autonomous Drone Racing Competition (A2RL), outperforming all competing AI teams and three human world champion pilots in a direct knockout tournament. It set a new milestone in autonomous drone racing research, reaching speeds up to 100 km/h on the competition track and successfully coping with problems such as camera interference and IMU saturation.