Efficient On-Board Processing of Oblique UAV Video for Rapid Flood Extent Mapping

arXiv:2601.11290 · cs.CV · Submitted 2026-01-16 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Efficient On-Board Processing of Oblique UAV Video for Rapid Flood Extent Mapping".

Jane: Here's the gist of the paper. The gist The Temporal Token Reuse (TTR) framework accelerates video semantic segmentation by exploiting spatiotemporal redundancy in oblique UAV video to achieve significant computational efficiency with minimal loss in accuracy.

Tom: Problem and Motivation Effective disaster response relies on rapid disaster response, where oblique aerial video is the primary modality for initial scouting due to its ability to maximize spatial coverage and situational awareness in limited flight time The computational density required to process these wide-field-of-view streams precludes low-latency inference on standard edge hardware This inefficiency is not a minor technical issue; it results directly into operational limitations: shorter flight times due to the high power draw of the on-board processing Temporal Token Reuse (TTR) Framework The primary contribution of this paper is the proposed Temporal Token Reuse (TTR) framework, which directly targets the temporal redundancy TTR exploits the intrinsic spatiotemporal redundancy of aerial video by formulating image patches as tokens It utilizes a lightweight similarity metric to dynamically identify static regions and propagate their precomputed deep features, thereby bypassing redundant backbone computations The framework modifies the core mechanism: instead of merely reducing the processing resolution, TTR skips computation entirely for patches that have not changed significantly from the previous frame The TTR process for each frame is formalized into two key stages, detailed in Algorithms 1 and 2 First, a lightweight decision mechanism determines which image regions require re-computation For each incoming frame It, we compare its patches against those from the preceding frame It−1 using the Cosine Similarity Index Once a SparsityMask is generated, it governs the data flow within the model’s forward pass A critical component enabling this sparse processing is our BlockSkip module, a modification of the original BlockCopy mechanism tailored for temporal reuse By simulating a dense feature map locally, BlockSkip allows standard convolutional layers to operate correctly on sparse inputs Experimental Validation Experimental results on edge-grade hardware demonstrate that TTR achieves a 30% reduction in inference latency with negligible degradation in segmentation accuracy («0.5% mIoU) The impact of the threshold hyperparameter τ is analyzed in Figure 3 This highly favorable trade-off, where a significant boost in FPS is achieved for a small drop in mIoU, makes TTR a compelling solution for resource-constrained devices The strong visual alignment and high Pearson correlation coefficients (r > 0.8) demonstrate that TTR effectively adapts its processing, allocating more computational resources during periods of high motion and significantly reducing computation when the scene is static 0.

Paper summary: Tom: So, looking at "Efficient On-Board Processing of Oblique UAV Video for Rapid Flood Extent Mapping," the authors proposed Temporal Token Reuse to solve a major problem where processing high-resolution oblique video on small drones is too slow.

Jane: They showed that by formulating image patches as tokens and using a similarity metric to skip redundant computations, they can cut inference latency by thirty percent while keeping the segmentation accuracy above zero point five percent mIoU.

Lu: The implication here is that we can deploy AI models for real-time situational awareness in disaster zones much faster and more affordably than before because the system adapts its effort based on what’s actually happening in the scene.

Meng: It means that instead of waiting for massive cloud processing, we can get immediate, high-quality results right where the action is happening on a drone. That changes how quickly responders can get actionable intelligence during an emergency.

Lalam: The paper confirms that TTR effectively shifts the operational Pareto frontier by making high-fidelity understanding of oblique video achievable in real time for these missions.

Jane: It's about making complex remote sensing tasks accessible to smaller, faster platforms without needing massive computational resources on board.

Conclusion: Tom: So, this paper is about taking those high-resolution oblique videos from drones—the kind you use for scouting disaster zones—and making them actually work on the drone itself without it overheating or running out of battery too fast.

Jane: Right, so instead of sending all that raw video back to a huge computer somewhere else to process it, the authors developed this Temporal Token Reuse framework. It’s like they figured out how to only look at the parts of the video that actually changed between frames.

Lu: Exactly. They treat image patches as tokens and use a simple similarity measure, like cosine similarity, to see if a patch is redundant from the last frame or not. If it hasn't changed much, they just skip the heavy math on it.

Meng: What I find really interesting is that this isn't just about reducing the data size; it’s about skipping entire chunks of computation. That means less power draw, which directly translates to longer flight times for those critical scouting missions you mentioned earlier.

Tom: And they showed a thirty percent reduction in inference latency on edge hardware, while keeping segmentation accuracy very close—less than zero point five percent mean Intersection over Union. That number behind the performance claim is what makes it compelling.

Jane: So, what does this actually mean for someone who just has to watch the show? It means that instead of waiting for a slow cloud process that takes forever to give you an update on a flood, you get a high-quality visual understanding right away from the drone feed.

Lu: The implication is shifting the operational frontier. You can move from slow, delayed intelligence gathering to real-time understanding using what’s available on resource-constrained devices.

Meng: For us in production, this means we can deploy these kinds of models onto smaller systems that wouldn't otherwise handle the load, which is a huge win for scalability.

Tom: It boils down to making complex video analysis practical for immediate deployment in time-sensitive scenarios like disaster response.

Jane: And now that we’ve seen how it works on benchmarks, we need to look at how this kind of adaptive processing holds up when the video quality gets really messy or the scene is extremely fast-moving.

Vishisht Sharma, Sam Leroux, Lisa Landuyt, Nick Witvrouwen, Pieter Simoens

IDLab, Ghent University - imec · VITO (Flemish Institute for Technological Research)

cs.CV

Submitted: 2026-01-16

Updated: 2026-01-16

DOI: 10.1109/JSTARS.2026.3729310

Code: https://github.com/decide-ugent/floodwater-dataset

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 89/100

The gist: The gist The Temporal Token Reuse (TTR) framework accelerates video semantic segmentation by exploiting spatiotemporal redundancy in oblique UAV video to achieve significant computational efficiency

Key concepts

Oblique Aerial Video: This refers to video captured by an unmanned aerial vehicle (UAV) from an angle that is not directly overhead. This perspective is crucial for disaster response because it allows the drone to maximize spatial coverage quickly while maintaining situational awareness over large areas, which is essential for initial scouting in flood scenarios.
Temporal Token Reuse (TTR): TTR is a novel inference framework that speeds up video processing by recognizing that consecutive frames are often very similar. It breaks the video into image patches called tokens and uses a similarity check to determine if a patch has changed significantly since the last frame. If it hasn't, the model reuses its existing calculations, saving significant computational power.
Spatiotemporal Redundancy: This concept describes how visual information in video is redundant both across space (different parts of the image) and time (the same area appearing in consecutive frames). The TTR framework leverages this by assuming that if a region looks the same between two frames, its deep features can be reused, allowing the system to skip expensive computations for those stable areas.
Edge-Grade Hardware: This refers to standard computing devices with limited power and processing capabilities, such as those found on small drones or embedded systems. The TTR framework is specifically designed to run efficiently on this hardware. By reducing the computational load through token reuse, it allows for low-latency inference necessary for real-time decision-making during urgent missions.

Terminology

Accuracy — 1× in this paper · explained in 1 episode
Benchmark — 1× in this paper · explained in 2 episodes
Fidelity — 1× in this paper · explained in 2 episodes
Redundancy — 1× in this paper · explained in 2 episodes
Temporal Token Reuse — 1× in this paper

Summary

The gist The Temporal Token Reuse (TTR) framework accelerates video semantic segmentation by exploiting spatiotemporal redundancy in oblique UAV video to achieve significant computational efficiency with minimal loss in accuracy.

Problem and Motivation

Effective disaster response relies on rapid disaster response, where oblique aerial video is the primary modality for initial scouting due to its ability to maximize spatial coverage and situational awareness in limited flight time The computational density required to process these wide-field-of-view streams precludes low-latency inference on standard edge hardware This inefficiency is not a minor technical issue; it results directly into operational limitations: shorter flight times due to the high power draw of the on-board processing

Temporal Token Reuse (TTR) Framework

The primary contribution of this paper is the proposed Temporal Token Reuse (TTR) framework, which directly targets the temporal redundancy TTR exploits the intrinsic spatiotemporal redundancy of aerial video by formulating image patches as tokens It utilizes a lightweight similarity metric to dynamically identify static regions and propagate their precomputed deep features, thereby bypassing redundant backbone computations The framework modifies the core mechanism: instead of merely reducing the processing resolution, TTR skips computation entirely for patches that have not changed significantly from the previous frame

Methodology Details

The TTR process for each frame is formalized into two key stages, detailed in Algorithms 1 and 2 First, a lightweight decision mechanism determines which image regions require re-computation For each incoming frame It, we compare its patches against those from the preceding frame It−1 using the Cosine Similarity Index Once a SparsityMask is generated, it governs the data flow within the model’s forward pass A critical component enabling this sparse processing is our BlockSkip module, a modification of the original BlockCopy mechanism tailored for temporal reuse By simulating a dense feature map locally, BlockSkip allows standard convolutional layers to operate correctly on sparse inputs

Experimental Validation

Experimental results on edge-grade hardware demonstrate that TTR achieves a 30% reduction in inference latency with negligible degradation in segmentation accuracy («0.5% mIoU) The impact of the threshold hyperparameter τ is analyzed in Figure 3 This highly favorable trade-off, where a significant boost in FPS is achieved for a small drop in mIoU, makes TTR a compelling solution for resource-constrained devices The strong visual alignment and high Pearson correlation coefficients (r > 0.8) demonstrate that TTR effectively adapts its processing, allocating more computational resources during periods of high motion and significantly reducing computation when the scene is static 0.

Improvements for AI systems

Improved on-board processing for UAV video by implementing Temporal Token Reuse (TTR), which exploits the intrinsic spatiotemporal redundancy of aerial video by formulating image patches as tokens to dynamically identify static regions and propagate their precomputed deep features, thereby bypassing redundant backbone computations. This allows the system to achieve a 30% reduction in inference latency with negligible degradation in segmentation accuracy («0.5% mIoU).
Adaptive computation allocation based on scene dynamics by using the TTR framework to modulate computational load in direct proportion to scene complexity, as demonstrated by the finding that TTR's computational load shows a "remarkably tight coupling between the normalized FLOPs executed by our model and the normalized magnitude of inter-frame optical flow (r > 0.8). This enables TTR immediately scales up its computation in response to these events" like sudden movements, ensuring high-fidelity processing where accuracy is important.
Enhanced real-time flood extent mapping capability by integrating TTR with a CNN backbone (e.g., EfficientNet-B4) to achieve a deployable 25 FPS on edge devices like the Jetson Orin Nano for floodwater monitoring, while maintaining an accuracy of 62.1% mIoU, confirming that the framework can handle high-fidelity semantic classification under power constraints.
Robustness against varying scene dynamism by utilizing a lightweight similarity metric based on cosine similarity to determine patch activity, which is inherently robust to uniform brightness changes, ensuring that static elements are correctly cached and reused even under illumination shifts prevalent in outdoor environments.