Vision-Guided Closed-Loop Autonomous Navigation of Light-Driven Janus Microbots

The gist: This work establishes an integrated vision-guided framework, termed VIMAN (Vision-guided Microbot Autonomous Navigation), for light-driven hematite/titania Janus microbots that combines programmable trajectory control, environment-aware path planning, real-time adaptation, and task execution within a single system.

Microbot Platform and Propulsion

The researchers developed cube-shaped hematite/titania Janus microbots with an edge length of approximately 3 μm as the active platform for autonomous microrobotic navigation The microbots consist of a hematite (α-Fe2O3) cube asymmetrically coated with a titania (TiO2) layer on one exposed face, forming a Janus architecture<ref:2610.21222#pg6> This Janus architecture provides the microbots with distinct photo-responsive and magnetic properties, which can be exploited for propulsion and directional control, respectively<ref:2610.21222#pg6> Under blue-light illumination (465 nm), the hematite/titania Janus microbots exhibited spontaneous propulsion in a low-concentration aqueous hydrogen peroxide medium (0.5% w/v) <ref:2610.21222#pg3> As schematically illustrated in Figure 2b, blue-light irradiation generates photogenerated charge carriers in the hematite/titania system, which participate in photocatalytic reactions involving H2O2<ref:2610.21222#pg3> Interfacial chargetransfer processes and the asymmetric Janus structure produce spatially nonuniform reaction rates around the microbot, resulting in local concentration gradients and phoretic slip that drive diffusiophoretic propulsion<ref:2610.21222#pg3> To regulate this otherwise unconstrained or random motion, they exploited the magnetic responsiveness of the hematite component using a two-dimensional Helmholtz-coil system (Figure 2c)<ref:2610.21222#pg7> This separation of propulsion and directional control, in which blue light drives the microbot while magnetic actuation determines its direction of motion, provides the physical basis for the closed-loop navigation framework developed in the following sections<ref:2610.21222#pg7>

Vision-Based Closed-Loop Control

The VIMAN framework integrates real-time visual localization using a custom-trained YOLOv11 object-detection model with discretized waypoint-based magnetic steering to enable precise navigation along complex prescribed trajectories<ref:2610.21222#pg2> The detected positions are continuously processed by the computer-based control program to determine the required steering commands, which are subsequently transmitted to the microcontroller for magnetic actuation<ref:2610.21222#pg8> To convert a desired trajectory into an experimentally executable path, they developed a path-discretization strategy in which the prescribed continuous trajectory is divided into a sequence of equidistant waypoints<ref:2610.21222#pg9> For navigation between successive waypoints, they implemented a base-to-perpendicular steering strategy where the instantaneous microbot position is treated as the base point and the next waypoint defines the target direction<ref:2610.21222#pg9> The complete implementation of the VIMAN framework is illustrated in Figure 2f<ref:2610.21222#pg9> Images acquired using the microscope camera are processed by the YOLOv11-based vision model to determine the instantaneous microbot position, while the control program determines the appropriate steering command based on its instantaneous position and the next target waypoint<ref:2610.21222#pg9> This closed-loop architecture established continuous feedback between visual detection and magnetic steering, enabling real-time correction of the microbot position and trajectory during navigation<ref:2610.21222#pg9>

Trajectory Following and Programming

The system allows for programmable trajectory control through two main methods: prescribed trajectories and user-defined freehand paths<ref:2610.21222#pg2> Desired trajectories were generated using a custom Python-based program that accepts either user-defined points or mathematical equations describing different geometries<ref:2610.21222#pg9> The microbot successfully navigated along a diverse range of prescribed trajectories, including polygonal, circular, lemniscate (∞), spiral, heart-shaped, and Lissajous paths<ref:2610.21222#pg10> The cumulative RMSE converged to approximately 2-3 μm for the linear reference trajectory and approximately 5–6 μm for the more complex trajectories shown in Figure 3a–f<ref:2610.21222#pg10> Importantly, more than 90% of the measured trajectory deviations remained below 5 μm across the tested path geometries<ref:2610.21222#pg10> The VIMAN framework can directly translate user-drawn free-hand paths into executable microbot trajectories, providing an intuitive approach for programmable and task-specific microbot navigation without requiring the user to define the underlying mathematical trajectory or steering commands<ref:2610.21222#pg13>

Environment-Aware Navigation

The framework extends to autonomous navigation through physically confined and dynamically changing environments<ref:2610.21222#pg4> In a physically confined maze, the custom-developed navigation algorithm generated an executable collision-free path connecting the microbot to the destination while avoiding the maze walls<ref:2610.21222#pg14> For dynamic obstacle avoidance, approximately 5 μm spherical Cu2O/SiO2 Janus micromotors were used as independently swimming active obstacles<ref:2610.21222#pg16> The overall navigation strategy is schematically illustrated in Figure 6a<ref:2610.21222#pg16> When the updated obstacle positions rendered the current path unsafe, a new collision-free A∗ path was generated in real time while accounting for the updated obstacle safety zones<ref:2610.21222#pg17> The rapid reconfiguration of the navigation path demonstrates the ability of the closed-loop system to respond to changes in the surrounding environment in real time<ref:2610.21222#pg17>

Task Execution and Cargo Manipulation

The framework is further extended to task-oriented cargo manipulation, enabling selective and sequential capture of multiple 2 μm PLA cargo particles, and their autonomous transport and delivery to a predefined location while continuously avoiding dynamic obstacles<ref:2610.21222#pg20> The custom-trained YOLOv11 object-detection model simultaneously identified the microbot, 5 μm SiO2 particles used as dynamic obstacles, and 2 μm PLA particles used as cargo<ref:2610.21222#pg8> Selected cargo particles were approached and captured by the microbot through local phoretic interactions and subsequently transported under closed-loop magnetic guidance<ref:2610.21222#pg8> Cargo release at the predefined delivery location was achieved by rapid programmed switching of the external magnetic field, inducing repeated reorientation of the microbot and destabilizing the phoretic interaction responsible for cargo retention<ref:2610.21222#pg20> This experiment demonstrates the integration of real-time object detection, target selection, dynamic path planning, sequential cargo capture, obstacle avoidance, transport, and controlled release within a single autonomous navigation framework<ref:2610.21222#pg20>

Conclusion

Overall this work establishes an integrated vision-guided framework that combines programmable trajectory control, environment-aware path planning, real-time adaptive navigation, and task execution within a single light-driven and magnetically steerable Janus microbot platform<ref:2610.21222#pg25> The developed approach provides a versatile basis for future autonomous microscale operations in lab-on-a-chip and organ-on-a-chip systems, microfluidic environments, and in vitro transport and delivery applications<ref:2610.21222#pg25>

How it works

1. Vision Localization: A custom YOLOv11 object-detection model is integrated to determine the positions of the microbot, cargo particles, and obstacles in real time<ref:2610.21222#pg8> This model enables reliable extraction of object positions for real-time feedback control<ref:2610.21222#pg8>

2. Path Planning and Control Loop: The detected positions are processed by a computer-based control program to determine required steering commands<ref:2610.21222#pg9> For prescribed paths, the trajectory is discretized into equidistant waypoints, and the microbot is guided sequentially toward individual waypoints using closed-loop magnetic steering<ref:2610.21222#pg9>

3. Dynamic Adaptation: For dynamic environments, an A∗ path planning algorithm generates a collision-free path in real time while accounting for detected obstacle positions and safety zones<ref:2610.21222#pg17> When obstacles move, the A∗ algorithm is reinitiated to generate a new collision-free shortest path based on the updated obstacle configuration<ref:2610.21222#pg17>

4. Task Execution: For cargo transport, local chemical gradient induced diffusiophoretic interactions enable capture when the microbot comes into close proximity to the target<ref:2610.21222#pg8> Cargo release is achieved by rapid programmed switching of the external magnetic field to destabilize the phoretic interaction<ref:2610.21222#pg20>

Key Components

% Light-Driven Propulsion: The hematite/titania Janus microbots exhibit spontaneous propulsion under blue-light illumination in aqueous hydrogen peroxide<ref:2610.21222#pg3>

% Magnetic Steering: A two-dimensional Helmholtz-coil system generates magnetic fields along the x- and y-directions to control the microbot's orientation<ref:2610.21222#pg7>

% Feedback Integration: The system closes the perception–control–actuation loop by continuously feeding positional feedback from vision detection back to the magnetic steering commands<ref:2610.21222#pg9>

% Path Generation Flexibility: The framework supports both predefined geometric trajectories and arbitrary user-defined freehand paths, allowing direct translation into executable microbot trajectories<ref:2610.21222#pg13>

Experimental Setup

The experimental setup involves an inverted optical microscope equipped with a Thorlabs scientific CMOS camera interfaced with a Linux-based computer<ref:2610.21222#pg4> The control system is driven by an Arduino Mega microcontroller interfacing with the magnetic field generation system<ref:2610.21222#pg4> The workspace is discretized into a grid based on detected positions, and the A∗ path-planning algorithm generates collision-free paths between waypoints<ref:2610.21222#pg9>

References

[1] X. Chen, C. Zhou, W. Wang, Chem. – Asian J. 2019, 14, 2388.<ref:2610.21222#pg26>

[2] S. Chen, D. E.

1. Bold header: Vision-Guided Closed-Loop Navigation Framework Implementation

This system can perform precise navigation along complex prescribed trajectories by integrating a custom-trained YOLOv11 object-detection model with discretized waypoint-based magnetic steering.

2. Bold header: Arbitrary Trajectory Execution

The improved AI system can translate arbitrary user-defined freehand paths into executable microbot trajectories, allowing for intuitive navigation by accepting inputs like a damped sine-wave pattern or handwritten text.

3. Bold header: Dynamic Obstacle Avoidance via Real-Time Replanning

The AI can enable navigation through dynamically changing environments by using A∗ path planning enables real-time path replanning while avoiding multiple independently swimming 5 μm spherical Janus micromotors.

4. Bold header: Autonomous Maze Solving

The system can navigate physically confined spaces by using image-based boundary detection to generate an executable collision-free path, demonstrating the ability to incorporate the detected environmental geometry into navigation.

5. Bold header: Task-Oriented Sequential Cargo Manipulation

The framework can execute complex tasks by enabling selective and sequential cargo capture of multiple 2 μm PLA cargo particles, and their autonomous transport and delivery to a predefined location while continuously avoiding dynamic obstacles.

Artificial microbots capable of precise and autonomous navigation could provide versatile platforms for microscale transport, manipulation, and task execution in complex environments. However, reliable navigation becomes increasingly challenging at the micrometre scale, particularly when the surrounding environment changes dynamically. Here, we develop a vision-guided closed-loop navigation framework for light-driven cube-shaped hematite/TiO2 Janus microbots with a characteristic size of approximately 3 μm. Real-time visual localization using a custom-trained YOLOv11 object-detection model is integrated with discretized waypoint-based magnetic steering to enable precise navigation along complex prescribed trajectories, with more than 90% of the measured trajectory deviations remaining below 6 μm across the tested path geometries. The framework further enables arbitrary user-defined free-hand trajectories and autonomous navigation through a physically confined maze. In dynamically changing environments, A* path planning enables real-time path replanning while avoiding multiple independently swimming 5 μm spherical Janus micromotors. Finally, the microbot selectively captures multiple cargo particles and autonomously transports and delivers them to a predefined location while simultaneously avoiding dynamic obstacles. These results establish an integrated platform combining programmable trajectory control, autonomous path planning, real-time environmental adaptation, and task execution within a single light-driven Janus microbot system.