Onboard UAV State Estimation for Fast Flights Without External Localization Systems: A2RL Drone Racing Finalist Approach

arXiv:2602.01860 · cs.RO · Submitted 2026-02-02 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Hi, I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "Onboard UAV State Estimation for Fast Flights Without External Localization Systems".

Dev: Here's the gist of the paper. The gist: This work presents an approach for onboard state estimation of a high-speed UAV using a monocular RGB camera and an IMU, which fuses data from Visual–Inertial Odometry (VIO), an onboard landmarkbased camera measurement system, and an IMU to produce an accurate state estimate.

Rosa: How it works The proposed method decomposes the problem into planning, state estimation, and an MPC control framework where Lyapunov stability theory provides formal guarantees <ref:2602.01860#pg2> The approach runs at 100 Hz leveraging VIO

10: , but compared to the state-of-the-art methods

6: ,

7: we correct all UAV states, including position, orientation, linear velocity and angular velocity <ref:2602.01860#pg4> It also uses our onboard landmark-based camera measurement system which uses known landmarks in an environment that are detectable in camera images to provide an estimate of the UAV’s position and orientation <ref:2602.01860#pg5> The state estimation pipeline involves several key components: 1.

Dev: VIO algorithm

10: provides drifting and inaccurate UAV state estimates which are fed into the VIO drift estimator <ref:2602.01860#pg6> 2.

Rosa: The Landmark detector provides estimated UAV states in the W frame using predefined static landmarks, such as racing gates, computing the UAV’s position p w ld and orientation Θw ld in the W frame at 30 Hz <ref:2602.01860#pg8> 3.

Paper summary: Rosa: So, this paper is about needing fast, reliable state estimation for high-speed UAVs when you don't have GPS available. It tackles the problem of keeping track of where a drone is going when it’s flying fast and in cluttered areas.

Dev: The core idea here is fusing data from three different things: Visual-Inertial Odometry, which uses the camera and the IMU, an onboard landmark measurement system, and that IMU data itself.

Rosa: And what they claim is that by using this combination, they can produce a really accurate state estimate.

Dev: They do this by taking the information from their onboard measurements to fix or compensate for any drift that happens in just the Visual-Inertial Odometry system alone through a new mathematical model for drift.

Rosa: It sounds like they're trying to solve the problem where standard methods often rely on more complicated hardware, like stereo cameras or rangefinders, and they end up using uncorrected velocities and orientations from the VIO, which causes errors during those fast maneuvers.

Dev: But this approach they present corrects all the UAV states including position, orientation, linear velocity and angular velocity.

Rosa: That means even when things are moving really fast and changing quickly, their estimation stays accurate because they are correcting everything that goes wrong in the VIO system.

Dev: They also incorporate this onboard landmark-based camera measurement system which uses known landmarks in an environment that are detectable in camera images to give an estimate of the UAV’s position and orientation.

Taro: So, if I'm driving, would I expect a drone to just keep getting lost when it turns really sharply or flies fast?

Rosa: Yeah, you would expect it to lose track quickly because the sensors can only see what they are currently seeing and interpreting the movement.

Dev: This paper is addressing that by making sure the estimation pipeline doesn't just rely on one source of truth, like VIO alone.

Taro: What do you mean by correcting all those states? Does that mean it fixes position, orientation, linear velocity, and angular velocity?

Dev: Exactly. They are correcting position first by taking the difference between the Visual-Inertial Odometry position transformed into a world frame and an estimated position drift.

Rosa: That sounds like a way to fight the drift problem directly rather than just hoping it stays small enough.

Conclusion: Rosa: So, looking at this paper, "Onboard UAV State Estimation for Fast Flights Without External Localization Systems: A2RL Drone Racing Finalist Approach," it really boils down to how you can keep a drone tracking itself accurately when you're doing aggressive maneuvers without any outside help like GPS.

Dev: The authors are using a combination of VIO and landmark detection to build this robust state estimation pipeline.

Rosa: It means they’ve designed a system that is good enough for the demanding environment of drone racing, which is very cluttered and requires quick reactions.

Dev: The implication here for anyone working on autonomous systems is that you don't necessarily need those expensive, complex external localization systems to achieve high performance if your onboard estimation can handle the dynamics.

Rosa: It shows that by correctly modeling the drift in the VIO process and adding a measurement system from known features, you can get state estimates that are accurate even during rapid motion.

Dev: The authors did this by introducing a novel mathematical drift model that includes translation, linear velocity, yaw angle, and yaw rate where those velocities are influenced by artificial friction to help with accuracy when the camera data is temporarily missing.

Rosa: That bit about adding artificial friction to increase accuracy when the camera stops providing usable data is something I find interesting because it directly addresses a failure mode of the system.

Dev: And they validated this whole setup through one thousand six hundred simulations and numerous real-world experiments, showing significant reductions in error compared to previous methods.

Rosa: So, what does this mean for people who only listen to the show? It means that if you're building a fast drone system, you can aim for accuracy during aggressive flight without needing those external localization systems if your estimation framework is strong enough.

Taro: So when you talk about those results in simulations, what’s the actual difference they found compared to the state-of-the-art methods they were comparing against?

Dev: They showed that their approach reduces position RMSE by a factor of twenty-seven using their method and reduces the error for orientation by seventy percent compared to other papers.

Rosa: That's a big reduction in error, especially when you consider how much harder it is to maintain those states during high-speed flight.

Taro: What about the limitations they mentioned? Did they say anything about where this method stops working or what it doesn't do?

Dev: They did flag that their proposed method relies on a set of predefined static landmarks for the measurement system to work effectively, and any other method providing noisy partial UAV state estimates within the operating area could also be used.

Rosa: So the limitation is tied to those landmarks; if you're in an environment completely devoid of recognizable features, this specific landmark measurement part won't help as much.

Taro: That makes sense. If I were testing it outside, would I expect it to handle a sudden loss of visual features gracefully?

Rosa: Well, the paper suggests that even with delays and noise in the landmark detector in Gazebo simulations, their system still operates effectively. It shows resilience when the environment isn't perfect.

Dev: So if you're thinking about implementing this, what is your immediate concern regarding putting this into a real-time loop?

Rosa: My concern is keeping that one hundred Hz update rate stable under real-world conditions where things get noisy <ref:2602.01860#pg1>.

Filip Novak®, Matej Petrl šk, Matej Novosad, Parakh M. Gupta, Robert Peni Ŭka, Martin Saska š

Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague

cs.RO

Submitted: 2026-02-02

Updated: 2026-10-21

Comments: Visit our webpage for more details: https://mrs.fel.cvut.cz/papers/onboard-uav-state-estimation-a2rl

Project page: https://mrs.fel.cvut.cz/papers/onboard-uav-state-estimation-a2rl

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 77/100

The gist: The gist: This work presents an approach for onboard state estimation of a high-speed UAV using a monocular RGB camera and an IMU, which fuses data from Visual–Inertial Odometry (VIO), an onboard

Key concepts

Visual-Inertial Odometry (VIO): VIO is a technique that combines data from an RGB camera and an Inertial Measurement Unit (IMU) to estimate the UAV's motion. While VIO is fast, it suffers from drift over time, meaning its position and orientation estimates become inaccurate during long or rapid movements. This method provides the initial state estimation but requires correction for sustained accuracy.
Landmark-based Camera Measurement System: This system uses known static objects in the environment, such as racing gates, visible in camera images. By detecting these landmarks, the system can provide an independent estimate of the UAV's position and orientation relative to those known points. This measurement helps correct the accumulated errors from VIO drift.
VIO Drift Model: A novel mathematical model is introduced to explicitly estimate and compensate for the inaccuracies inherent in VIO. This model tracks translational drift, linear velocity, yaw angle, and yaw rate. By modeling how these drifts occur—even with artificial friction—the system can predict and subtract the error from the raw VIO data to produce a highly accurate state estimate.
State Estimation Pipeline: This is the overall process where various sensor inputs are combined to determine the UAV's full 6-degree-of-freedom state (position, orientation, linear velocity, and angular rates). The pipeline involves feeding VIO outputs into a drift estimator, using landmark measurements to correct position/orientation errors in a world frame (W), and fusing IMU data for fast response during dynamic flight.

Terminology

Visual-Inertial Odometry — 1× in this paper

Summary

The gist: This work presents an approach for onboard state estimation of a high-speed UAV using a monocular RGB camera and an IMU, which fuses data from Visual–Inertial Odometry (VIO), an onboard landmarkbased camera measurement system, and an IMU to produce an accurate state estimate.

How it works

The proposed method decomposes the problem into planning, state estimation, and an MPC control framework where Lyapunov stability theory provides formal guarantees <ref:2602.01860#pg2> The approach runs at 100 Hz leveraging VIO [10], but compared to the state-of-the-art methods [6], [7] we correct all UAV states, including position, orientation, linear velocity and angular velocity <ref:2602.01860#pg4> It also uses our onboard landmark-based camera measurement system which uses known landmarks in an environment that are detectable in camera images to provide an estimate of the UAV’s position and orientation <ref:2602.01860#pg5>

The state estimation pipeline involves several key components:

VIO algorithm [10] provides drifting and inaccurate UAV state estimates which are fed into the VIO drift estimator <ref:2602.01860#pg6>
The Landmark detector provides estimated UAV states in the W frame using predefined static landmarks, such as racing gates, computing the UAV’s position p w ld and orientation Θw ld in the W frame at 30 Hz <ref:2602.01860#pg8>
A novel model of VIO drift is introduced which includes translation, linear velocity, yaw angle, and yaw rate where the velocities are influenced by artificial friction to increase accuracy even when the onboard camera system temporarily stops providing usable data
IMU data are directly fused into the attitude estimation in our pipeline enabling higher accuracy and faster response to rapid maneuvers compared to state-of-the-art methods [6], [7]

State Estimation Details

The full 6 DOF UAV state is estimated as xuav = (x, y, z, ϕ, θ, ψ, x˙ y˙ z˙ p q r ˙) The VIO algorithm outputs the position and orientation in a local reference frame V defined at the initialization point of the VIO method These states are then transformed into the W frame using a static transformation T W V constructed from the known initial UAV position and orientation r w init in the W frame The ground-truth position and orientation of the UAV in the W frame are denoted as r w gt

The corrected states are computed by fusing data from the VIO algorithm, the estimated VIO drift, and the IMU measurements in the State estimator block Specifically, the corrected UAV position puav is computed as the difference between the VIO position transformed into the W frame p w vio and the estimated position drift pd as puav(t) = p w vio(t) − pd(t) The roll ϕvio and pitch θvio obtained from the VIO algorithm are inaccurate and delayed due to z-axis alignment issues and the lack of incorporation into the graph optimization

Drift Compensation Mechanism

To estimate VIO drift, a discrete linear system is modeled where the drift state xd consists of the translational drift pd = (xd, yd, zd) with corresponding drift velocities vd = (˙xd, y˙d, z˙d) as well as the yaw drift ψd with its corresponding drift rate rd The measurement equation defines how the system states are observed through available measurements z at time t is given by zd(t) = Hdxd(t) + vnoise(t)

The VIO drift is then computed as the difference between the VIO position and yaw (xvio, yvio, zvio, ψvio) expressed in W frame (2) and the camera-based estimate x w ld z = (xvio, yvio, zvio, ψvio) − (xld, yld, zld, ψld) This measurement is used to update the drift state using a Linear Kalman Filter (LKF) which consists of two steps: 1) the prediction step and 2. the correction step The LKF uses the confidence cld of the Landmark detector as follows, where R is defined as Rrxd, Ryd, Rzd, and Rψd = arv / (1 + exp(−brv(cld − crv)) + drv)

Performance and Validation

The proposed method was thoroughly validated through 1600 simulations and numerous real-world experiments In Gazebo simulations, the results show that our system operates effectively even in the presence of delays and noise in the Landmark detector A significant increase in RMSE occurs for all states when σ2 pld and σ2 ψld reach 1.0 In real-world experiments, the position RMSE of the VIO is reduced by a factor of 27 using our approach Furthermore, our approach reduces RMSE of Θ by 70 % and RMSE of ω by a factor of 8 compared to [7] and [6]

Conclusion

This estimation pipeline enables fast flights through gates in drone racing providing accurate UAV states even during aggressive maneuvers Our method uses VIO odometry which does not require predefined landmarks and is subsequently corrected using the estimated drift states The direct fusion of IMU data enables fast and accurate estimation of roll, pitch, and their corresponding rates during aggressive maneuvers This estimation pipeline enables fast flights through gates in drone racing providing accurate UAV states even during aggressive maneuvers The approach was extensively evaluated in simulations and real-world experiments outperforming current state-of-the-art methods Finally, we applied the proposed approach during the A2RL Drone Racing Challenge 2025 where our team reached the final four out of 210 teams

References

[1] G. Silano, T. Baca, R. Penicka, D. Liuzza, and M. Saska, “Power line inspection tasks with multi-aerial robot systems via signal temporal logic specifications,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 4169–4176, 2021 <ref:2602.01860#pg4>

[5] E. Kaufmann, M. Gehrig, P. Foehn, R. Ranftl, A. Dosovitskiy, V. Koltun, and D. Scaramuzza, “Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing,” in 2019 International Conference on Robotics and Automation (ICRA), May 2019, pp. 690–696

[7] E. Kaufmann, L. Bauersfeld, A. Loquercio, M. Muller, V. Koltun, and ¨ D. Scaramuzza, “Champion-level drone racing using deep reinforcement learning,” Nature, vol. 620, no. 7976, pp. 982–987

[10] T. Qin, P. Li, and S. Shen, “VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator,” IEEE Transactions on Robotics, vol. 34, no. 4, pp. 1004–1020

[16] P. Geneva, K. Eckenhoff, W. Lee, Y. Yang, and G. Huang, “OpenVINS: A Research Platform for Visual-Inertial Estimation,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), May 2020, pp. 4666–4672 <ref:

Improvements for AI systems

Correct all VIO states (position, orientation, linear and angular velocity) instead of using uncorrected drifting VIO velocities or attitude/angular rates to ensure accuracy during rapid and dynamic motion. This results in accurate state estimation even during rapid and dynamic motion.
Introduce a novel model of VIO drift that includes translation, linear velocity, yaw angle, and yaw rate where the velocities are influenced by artificial friction to increase accuracy even when the onboard camera system temporarily stops providing usable data.
Directly fuse IMU data into the attitude estimation in our pipeline, enabling higher accuracy and faster response to rapid maneuvers compared to state-of-the-art methods. This is crucial for maintaining high precision during aggressive maneuvers.
Correct translational, rotational, and both linear and angular velocity drifts in the VIO state estimates by fusing estimated states obtained from detections of known landmarks with VIO and IMU data. This fusion corrects all drift components systematically.
Implement a Linear Kalman Filter (LKF) for VIO drift estimation using the Landmark detector's noisy estimates, where the measurement confidence is defined by an exponential function of the reprojection error. This LKF provides a robust mechanism to track and correct accumulated errors in VIO states.
Use IMU data to estimate roll and pitch angles, filtering them with a low-pass filter and a dynamic notch filter to ensure fast and accurate response during aggressive maneuvers in agile flights. This ensures rapid attitude changes are accurately reflected in the control system.
Correct the yaw rate by subtracting the estimated yaw drift from the VIO yaw angle, resulting in psi uav(t) = ψvio(t) - ψd(t). This directly mitigates yaw drift for more accurate heading estimation.

Abstract

Fast flights with aggressive maneuvers in cluttered GNSS-denied environments require fast, reliable, and accurate UAV state estimation. In this paper, we present an approach for onboard state estimation of a high-speed UAV using a monocular RGB camera and an IMU. Our approach fuses data from Visual-Inertial Odometry (VIO), an onboard landmark-based camera measurement system, and an IMU to produce an accurate state estimate. Using onboard measurement data, we estimate and compensate for VIO drift through a novel mathematical drift model. State-of-the-art approaches often rely on more complex hardware (e.g., stereo cameras or rangefinders) and use uncorrected drifting VIO velocities, orientation, and angular rates, leading to errors during fast maneuvers. In contrast, our method corrects all VIO states (position, orientation, linear and angular velocity), resulting in accurate state estimation even during rapid and dynamic motion. Our approach was thoroughly validated through 1600 simulations and numerous real-world experiments. Furthermore, we applied the proposed method in the A2RL Drone Racing Challenge 2025, where our team advanced to the final four out of 210 teams and earned a medal.