A 2D Image Feedback-Based Autonomous Endovascular Navigation Method: A Perception and Control Framework

arXiv:2610.21481 · cs.RO · Submitted 2026-10-15 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "A 2D Image Feedback-Based Autonomous Endovascular Navigation Method".

Dev: The gist The method formulates a perception and control framework for two-dimensional image feedback to enable autonomous navigation in endovascular procedures by converting vessel masks into a labeled centerline graph…

Rosa: First, who's behind it and why it matters.

Paper summary: Rosa: So we're looking at this paper, "A 2D Image Feedback-Based Autonomous Endovascular Navigation Method: A Perception and Control Framework <ref:2610.21481#pg1,A 2D Image Feedback-Based Autonomous Endovascular Navigation Method: A Perception and>." Basically, the authors are tackling the problem that doctors use planar DSA and X-ray fluoroscopy to see things but translating those images into something a robot can actually use for navigation is really hard.

Dev: Exactly. The core idea here is to create a perception and control method specifically designed for two-dimensional image feedback in these endovascular procedures. They convert vessel masks into what they call a labeled centerline graph, and then project the device tip estimates onto that path to get progress, deviation, and direction information.

Taro: So it’s not just looking at an image; it’s building this graph from the vessel masks and using those estimates to calculate how far you are ahead of plan, how off course you are, and which way you need to go Taro. But what does that actually mean for a physical robot trying to navigate?

Rosa: It means they use this quantified progress, deviation, and direction information to drive a rule-based controller. This controller handles things like delivering the device, choosing which branch of the vessel to probe next, performing rollback if things go wrong, and even recovery actions Rosa. It’s a complete set of maneuvers driven by what the image feedback tells it.

Dev: And they test this framework on a physical dual-device robotic platform using sixty-five closed-loop runs with simulator-rendered planar images across three different paths <ref:2610.21481#pg1,on a physical dual-device robotic platform>. The main result they report is that all runs reached at least a maximum observed guidewire progress of zero point nine five, and two runs, which was three point one percent of the total, met their prespecified failure rule of three or more autonomous stop events Dev <ref:2610.21481#pg1,prespecified failure rule of three or more autonomous stop events>.

Taro: That's the number on the page, but what does that progress mean in a real procedure? Does it mean we're actually moving through the vessel effectively, or is that just some metric for success?

Rosa: It establishes route-level performance on this tested platform. The paper states that clinical image sequences and broader anatomy are still things they need to evaluate further Rosa. It shows what works under these controlled conditions using a fixed planar projection.

Paper summary: Dev: The central contribution, according to the paper, is this topology-constrained, path-relative state abstraction that converts planar device and vessel observations into interpretable progress and branch-decision variables for physical-robot navigation Dev. They are separating the static vessel topology from the dynamic device motion and expressing the latter in a route-relative coordinate system.

Taro: So it’s like they’re creating a simplified, traceable way to navigate macroscopic movement using only what you see on the screen, even though you don't need three-dimensional reconstruction for this specific route-level formulation Taro. Is that abstraction robust enough?

Rosa: The evidence supports this path-relative image state as a practical abstraction for traceable macroscopic navigation under a fixed planar projection Rosa. It allows the system to support target-directed delivery, branch probing, rollback, recovery, and catheter following across those sixty-five closed-loop runs Rosa <ref:2610.21481#pg1>.

Dev: From my side looking at the controls engineer stuff, they are using a deterministic hierarchical finite-state machine driven by these measured states. The modes respond to image observations of device vessel interaction like Risk Point Recovery or Off-Path Recovery Dev. They have explicit thresholds for when to trigger actions like retraction, rotation scans, insertion tests, or rollback Dev.

Taro: So when the world misbehaves—when the robot deviates from the plan—it doesn't just stop blindly; it enters these defined recovery modes based on how far off track it is Taro. How gracefully does that transition happen in practice?

Rosa: The paper shows that they tied each control action, like rollback or catheter following, to explicit thresholds or mode transitions Rosa. For instance, deviation correction enters when the deviation d w t is greater than twenty-two pixels and releases when it's less than sixteen pixels Rosa.

Dev: And at a route junction, the controller scans discrete rotation targets and evaluates a score J to select the best branch using an empirical weighted score Dev. This means the choice of which path to take is based on a calculated value derived from those image observations.

Taro: That scoring mechanism for branches sounds interesting because it’s not just picking the visually best looking option; it’s a scored decision based on progress and deviation metrics Taro. So, if you only listen to this, what does this mean for someone who doesn't work with robots?

Paper summary: Rosa: It means that even with limited visual feedback from a fixed projection, you can still achieve route-level navigation decisions—like choosing a branch or deciding when to retract—based on progress and deviation metrics derived from the image Rosa. The system is designed to be interpretable in those terms.

Dev: The authors are concluding that this formulation separates the static vessel topology from the dynamic device motion and expresses the latter in a route-relative coordinate system Dev. They show that this state supports target-directed delivery, branch probing, rollback, recovery, and catheter following across sixty-five closed-loop runs Dev <ref:2610.21481#pg1>.

Taro: But what’s missing? The paper says clinical image sequences and broader anatomy are still things they need to evaluate for generalization Taro. That tells us the current testbed is a very specific environment.

Rosa: Right, so while this framework works on the Mentice platform with simulator-rendered images across three paths, it’s not yet proven that it will work with all possible clinical image sequences or complex anatomy Rosa. It’s a strong demonstration of the concept under constrained conditions.

Dev: And they did mention that they used one fixed projection and one parameter set for the study Dev. That's a key limitation when thinking about real-world applicability across different patient setups.

Taro: So the question for future work is taking it beyond this controlled environment, right? The authors are clearly setting up the next steps to test this more broadly.

Rosa: Right. It sets the stage for testing this on changing projections, calibrated localization methods, and incorporating more complex catheter endpoint trials to see if these route-level decisions hold up in a wider clinical context Rosa. That’s where we need to look next.

Dev: So, to wrap up the paper "A 2D Image Feedback-Based Autonomous Endovascular Navigation Method: A Perception and Control Framework," they introduce a perception and control framework that turns planar image feedback into progress, deviation, and direction variables for autonomous navigation Dev <ref:2610.21481#pg1,A 2D Image Feedback-Based Autonomous Endovascular Navigation Method: A Perception and>. They show this works on a physical robot in sixty-five runs, hitting at least zero point nine five maximum observed guidewire progress Dev <ref:2610.21481#pg1,at least 0.95 maximum observed guidewire progress>.

Taro: And the main point for us is that they create an interpretable state abstraction that converts observations into branch decisions and recovery triggers for macroscopic navigation Taro. It’s a way to make sense of the visual data even when you only have a fixed planar projection Taro.

Paper summary: Rosa: So, in simple terms, it means they took complex 2D images and turned them into measurable metrics—progress, deviation—that drive the robot's decision-making process for things like moving forward or choosing a path Rosa <ref:2610.21481#pg1>. That’s the essence of this paper.

Dev: It's a solid piece of work demonstrating how to create a traceable state for route-level decisions under incomplete visual observations Dev. The engineering challenge they solved was creating that bridge between the 2D image and the physical action loop Dev <ref:2610.21481#pg1>.

Taro: And it gives us a concrete mechanism for how autonomy can handle things like probing, rollback, and recovery based on what’s happening visually in real-time Taro. It moves beyond just simple obstacle avoidance to actual procedural decision making Taro.

Rosa: That's the interesting part. The paper shows that by defining these quantities precisely—like target-directed one-step progress—the system can make decisions about delivery and probing with a high degree of consistency Rosa.

Dev: But we have to keep in mind the caveat they mentioned: clinical image sequences and broader anatomy are still things they need to evaluate for generalization Dev. It’s not just that it works on their platform; it needs to work everywhere.

Taro: So the next big question is, how does this system handle situations where the vessel geometry changes in ways that weren't perfectly captured in those initial masks? That’s a real-world failure mode Taro.

Rosa: Exactly. They are looking ahead to incorporating force-related sensing and multimodal feedback to extend the control state beyond just the visual measurements they are using now Rosa. They aren't stopping at this visual abstraction.

Dev: So, in conclusion for this paper, it lays out a clear path: convert 2D masks into a labeled graph, project tips onto that graph for progress and deviation, and use that state to drive a rule-based controller for navigation Dev <ref:2610.21481#pg1>. It’s about creating an interpretable state abstraction for route-level decisions Dev.

Taro: And the results show it achieves at least zero point nine five maximum observed guidewire progress across those tests, even with two runs hitting their failure criterion Taro <ref:2610.21481#pg1,at least 0.95 maximum observed guidewire progress>. That's a solid performance metric for a physical robot in this setup Taro.

Rosa: It’s a useful step because it proves that we can translate visual observation into actionable navigation variables for complex procedures like endovascular intervention Rosa. It gives us a framework to think about how AI can interpret medical imaging in these ways.

Conclusion: Rosa: So we're wrapping up this look at "A 2D Image Feedback-Based Autonomous Endovascular Navigation Method: A Perception and Control Framework <ref:2610.21481#pg1,A 2D Image Feedback-Based Autonomous Endovascular Navigation Method: A Perception and>."

Dev: Basically, the paper lays out a way to take those two dimensional images from an endovascular procedure and turn them into actual progress and deviation numbers that a robot can use.

Taro: It’s about creating this mathematical language for navigation based on what you see on the screen, rather than just following a pre-programmed path blindly.

Rosa: That's right. They took vessel masks, turned them into a graph, and projected the device tip onto that graph to get these metrics like progress and direction.

Dev: And they use those metrics to drive a rule-based controller that handles things like moving forward or picking the right branch when there’s a split in the vessel.

Taro: The big point is that this gives you an interpretable state for route-level decisions, even when you don't have perfect three dimensional data.

Rosa: It means you get traceable navigation under a fixed planar projection, which is pretty powerful for what they tested on their physical robot platform.

Dev: They showed it worked in sixty five closed loop runs hitting at least ninety five percent maximum observed guidewire progress across three different test paths.

Taro: But the caveat is that this works on a specific setup—a fixed projection and simulator images—so we need to keep asking how it handles real clinical image sequences later on.

Rosa: Exactly. The authors are separating the static vessel shape from the moving device motion into a route-relative coordinate system for these macroscopic navigation decisions.

Dev: So, while they proved this framework can support recovery and probing across those runs, we still need to see how it behaves when the underlying anatomy is more complex than what's in that initial model.

Taro: That’s the next thing to test. We need to check if this abstraction holds up when the world misbehaves in a way that wasn't fully captured by their training data.

Rosa: Right, so they’ve built a solid foundation for using visual feedback for autonomous navigation, but proving its broad clinical relevance is what comes next.

Dev: And that leads us into how we test these kinds of control loops when the real-world latency and uncertainty start creeping in.

Sheng Cao, Shuxiang Guo, Yonggan Yan, Jian Guo, Chunying Li, Bin Wang, Mingchao Ding, Yongxin Zhang, Yongwei Zhang, Pengfei Yang,, Jianmin Liu

Beijing Institute of Technology · Southern University of Science and Technology, Shenzhen Institute of Advanced Biomedical Robot Company Limited · Aerospace Center Hospital

cs.RO

Submitted: 2026-10-15

Updated: 2026-10-15

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 69/100

The gist: The gist The method formulates a perception and control framework for two-dimensional image feedback to enable autonomous navigation in endovascular procedures by converting vessel masks into a

Key concepts

Labeled Centerline Graph: This is a representation of the vessel anatomy derived from image masks. Instead of just an outline, it's a graph where specific points along the vessel path are labeled with information. This structure allows the system to understand the vessel's topology and plan movement relative to known anatomical features.
Device-Tip Estimates: These are calculated positions of the endovascular device tip derived from processing 2D images. The system uses segmentation models to find the device in each frame and then calculates its location. These estimates are crucial for determining how far the device has moved, where it is relative to the planned path, and which way it needs to go.
Path-Guided Control (HFSM): This is a deterministic control system that uses a finite-state machine structure. It takes the measured state from the images (like progress and deviation) as input. Based on predefined thresholds, the controller decides which action to take, such as advancing, rotating, or recovering from an error.
Route-Relative Coordinate System: This system transforms device motion into coordinates relative to a pre-selected planned route. By using this system, the control framework can make decisions based on progress along the intended path rather than just absolute image coordinates. This abstraction helps in making traceable, macroscopic navigation choices.

Summary

The gist The method formulates a perception and control framework for two-dimensional image feedback to enable autonomous navigation in endovascular procedures by converting vessel masks into a labeled centerline graph and projecting device-tip estimates onto this graph to obtain progress, deviation, and direction

System Overview and Sensing Design

The platform integrates the Mentice VIST G7 vascular simulator, a physical dual-device robot, an autonomous-navigation workstation, and a physician-side controller in a closed loop The sequence involves Mentice providing the planar projection, the robot changing device configuration, and the workstation estimating the next visual state before issuing a command The system isolates dynamic image measurements from the static route map because the vessel representation is initialized once and locked, while physical robot changes later frames update masks Translation and rotation do not determine final device motion by themselves; instead, contact, friction, and lumen geometry redirect the flexible guidewire The controller estimates tip displacement, path progress, path distance, direction consistency, and wire–catheter separation using successive images

Path-Based Visual Measurement Model

The sensing problem involves estimating device tips, local directions, and residuals relative to the planned path from an image The projected start and target coordinates are defined by the operator-selected route, which is parameterized by arc length The resulting measurements include raw increments, target-directed one-step progress, interval progress, and wire–catheter separation The action set includes guidewire insertion, retraction, left or right rotation, and catheter insertion and retraction The next image determines the realized outcome of each command

Image-Based Perception and State Estimation

Two independently trained segmentation models use a custom U-Net with squeeze-and-excitation (SE) blocks to generate masks The device model targets the two dynamic structures that determine the visual state: guidewire and catheter The vessel model targets the static route reference, outputting ten anatomical branch channels and one vesselunion channel Online perception uses right 1080×1080 crop and uses FP32 ONNX checkpoints converted to TensorRT engines Device masks are updated with each processed frame because their temporal stability directly determines tip and direction measurements Tip measurement is achieved by skeletonizing the largest connected component of the device mask and performing a two-sweep geodesic search Local image-plane direction is estimated using principal-component fitting, with a directional exponential filter reducing frame-to-frame fluctuations

Path-Guided Control Implementation

The path-guided control implementation is a deterministic hierarchical finite-state machine (HFSM) driven by the measured state in (5) The modes respond to image-observed outcomes of device–vessel interaction, including Risk-Point Recovery, Off-Path Recovery, Junction Probing, and Auto-Advance In Auto-Advance, advancement is eligible no more frequently than every 0.35 s and motion continues when d w t ≤ 35 pixels and ρ w t ≥ −0.30 Deviation correction enters when d w t ≥ 22 pixels and releases when d w t ≤ 16 pixels, selecting rotation based on the sign of xˆw t − x w t At a route junction, the controller scans discrete rotation targets and evaluates a score J to select the best branch using an empirical weighted score

Perception and Closed-Loop Navigation Results

All 65 runs reached at least 0.95 maximum observed guidewire progress, and two Path 2 runs met the prespecified three-stop failure criterion Table IV reports sensing measures, showing median wire-mask IoU of 0.890, 0.882, and 0.897 for Paths 1–3 The lower and more variable wire direction cosine reflects the reorientation and retreat states encountered during navigation The state exposed the reason for each control action, tying retraction, rotation scans, insertion tests, rollback, risk retreat, and catheter following to explicit thresholds or mode transitions The evidence supports a path-relative image state as a practical abstraction for traceable macroscopic navigation under a fixed planar projection

The study concludes that the presented formulation separates static vessel topology from dynamic device motion and expresses the latter in a route-relative coordinate system, forming an interpretable state for routelevel decisions under incomplete visual observations The state supported target-directed delivery, branch probing, rollback, recovery, and catheter-follow triggering across 65 closed-loop runs All runs reached at least 0.95 maximum observed guidewire progress, and two Path 2 runs met the prespecified threestop failure criterion The evidence supports a path-relative image state as a practical abstraction for traceable macroscopic navigation under a fixed planar projection Clinical image sequences, changing projections, calibrated localization, forcerelated state variables, broader anatomy, and complete catheter endpoint trials remain necessary to establish generalization and clinical relevance The study used three routes, 65 physical-robot runs, one fixed projection, and one parameter set

References

[1] L. P. Palaniappan, et al., “2026 heart disease and stroke statistics: A report of US and global data from the American Heart Association,” Circulation, vol. 153, no. 9, pp. e275–e906, Mar. 2026, doi: 10.1161/CIR.0000000000084

[2] B. C. Lengyel, P. Chinnadurai, S. J. Corr, et al., “Robot-assisted vascular surgery: Literature review, clinical applications, and future perspectives,” J. Robot. Surg., vol. 18, no. 1, p. 328, 2024

[3] S. R. Mandhumula and M. A. Tavallaei, “Robotic systems in cardiovascular interventions: Evolving platforms and the path forward,” Methodist DeBakey Cardiovasc. J., vol. 21, no. 5, pp. 16–34, 2025

[4] S. Miyachi, Y. Nagano, R. Kawaguchi, and H. Tadauchi, “Robotics for neuroendovascular therapy,” J. Neuroendovascular Ther., vol. 19, no. 1, pp. ra.2024-0112, 2025

[5] J. Sun, R. Tang, Q. Zhang, et al., “Advancements and challenges in autonomous endovascular interventional robotics: A comprehensive review,” iScience, vol. 28, no. 12, Art. no. 114024, 2025

[6] T. Yao, et al., “Advancing embodied intelligence in roboticassisted endovascular procedures: A systematic review of AI solutions,” IEEE Rev. Biomed. Eng., vol. 19, pp. 248–266, 2026

[7] P. Ambrosini, D. Ruijters, W. J. Niessen, A. Moelker, and T. van Walsum, “Fully automatic and real-time catheter segmentation in X-ray fluoroscopy,” in Proc. MICCAI, 2017, pp.

Improvements for AI systems

Bold Header: Topology-Constrained Path Abstraction for State Representation

This improves AI systems by converting image observations into a state that can be inspected at every control transition via a topology-constrained, pathrelative state abstraction that converts planar device and vessel observations into interpretable progress and branch-decision variables. This allows the controller to focus on route-level behavior rather than pixel-level variations.

Bold Header: Deterministic Hierarchical Finite-State Machine (HFSM) Control

The system utilizes a deterministic hierarchical finite-state machine (HFSM) driven by the measured state in (5) to manage navigation modes like Risk-Point Recovery, Off-Path Recovery, Junction Probing, and Auto-Advance. This eliminates reliance on learned policies for core navigation logic.

Bold Header: Explicit Failure Accounting and Run-Level Performance Metrics

The control system explicitly tracks failure conditions by classifying runs based on specific criteria: A run was classified as a failure when it contained three or more such autonomous stop events. This provides traceable, quantifiable performance metrics beyond simple success/failure flags.

Bold Header: Multi-Modal Feedback Integration for State Extension

The state vector is comprehensive, including progress, transverse deviation, directional consistency, and inter-device lag, derived from projections of tips onto the route. This ensures the control state captures critical physical interactions necessary for complex maneuvers like branch selection.

Bold Header: Mixed-Precision Inference Optimization

The system employs mixed-precision deployment (FP16/INT8) on TensorRT engines, specifically retaining FP16 for device masks because it reduces the propagation of quantization error into closed-loop decisions. This balances computational speed with necessary accuracy in safety-critical navigation.

Bold Header: Operator-Marked Interaction Risk Trigger

The system incorporates a heuristic response where Operator-marked interaction-risk points form a separate early-trigger channel, which invokes priority retreat before the delayed off-path condition, without modifying the graph or replanning the route. This provides an immediate safety mechanism for known undesired regions.

Sources

Autonomous Guidewire Navigation for Robot-assisted Endovascular Interventions: A Knowledge-Driven Visual Guidance Approach