Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories

arXiv:2603.25903 · cs.RO · Submitted 2026-03-26 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Hi, I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "Emergent Neural Automaton Policies".

Dev: Here's the gist of the paper. The gist The Emergent Neural Automaton Policy (ENAP) framework allows for unsupervised discovery of task-specific logic and autonomous failure recovery by learning a bi-level neuro-symbolic policy from visuomotor demonstrations, which addresses the challenge of scaling robot learning to long-horizon tasks How it works ENAP introduces a framework that allows a bi-level neuro-symbolic policy to emerge adaptively from visuomotor demonstrations.

Rosa: Specifically, the framework first employs adaptive clustering and an extension of the L∗ algorithm to infer a Mealy state machine from visuomotor data, which serves as an interpretable high-level planner capturing latent task modes.

Dev: Then, this discrete structure guides a low-level reactive residual network to learn precise continuous control via behavior cloning (BC).

Rosa: This bi-level architecture unifies discrete reasoning (where humans excel) and continuous control (where robots excel), aiming for both high interpretability and strong compositionality.

Title and authors: Rosa: So we're looking at this paper called "Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories," and the authors are Pan, Luo, Hu, Yu, Liu from Carnegie Mellon. They're basically talking about how to get robot learning to handle long-horizon tasks without needing all that pre-programmed logic.

Dev: Right. It sounds like they’re moving away from those rigid symbolic rules that you have to manually build in and instead letting the system figure out the structure itself just from watching robots do things.

Taro: I'm curious if this means we can get a system that learns how to plan over many steps without us telling it every single rule beforehand.

Rosa: Exactly. The paper introduces this framework called ENAP, which is meant to let a bi-level policy emerge adaptively from just looking at visuomotor demonstrations. It’s about unlocking task-specific logic and figuring out how the robot can recover autonomously if things go wrong during those long sequences of actions.

Dev: So, the core idea here is using this bi-level neuro-symbolic policy structure. That means you get high-level symbolic reasoning, which is great for planning, and a low-level network that handles the actual continuous movement and control.

Taro: And how does it actually figure out that structure from just video data? Is it some kind of black box learning that just spits out a policy?

Rosa: Not quite. The framework starts with inferring a Mealy state machine directly from the visuomotor data using adaptive clustering and an extension of the L star algorithm. This gives them this interpretable high-level planner, which essentially captures different task modes.

Dev: That Mealy machine part is interesting because it’s a discrete structure, which is what humans are good at—the planning part—but it’s derived from the data without any labels we have to provide ourselves.

Taro: If they can discover those task modes unsupervised, that means the system isn't stuck following a single script; it’s discovering the different ways to achieve a goal on its own.

Rosa: That's right. Then, this discrete structure then guides a low-level reactive residual network to learn the precise continuous control through behavior cloning. This lets you get both high interpretability and strong compositionality in one go.

Dev: I like that idea of unifying the discrete reasoning and the continuous control parts into a single architecture, rather than having them be two completely separate systems that have to communicate perfectly.

Title and authors: Taro: But what about when things go wrong? If the inferred structure isn't perfect for a specific situation, how does it handle failure instead of just freezing up?

Rosa: That’s where one of the key findings comes in. They show that this extracted PMM, or Probabilistic Mealy Machine, acts as a high-level planner giving a coarse action prior at every time step. This is then combined with a learned residual term to synthesize the final precise action, which helps focus the neural network on just the local details.

Dev: So you’ve got this PMM capturing the global dynamics and intent, and then a separate neural network handling that small correction term, a t, which is what makes it reactive.

Taro: I saw something about autonomous failure recovery too. The paper suggests they found structural loops in this automaton that let the agent exploit those loops on certain nodes to retry actions when it hits an error signature.

Rosa: That’s right, and it shows that the system doesn't just stop when it fails; it uses the discovered structure to transition back to a previous state, like going from node q2 back to node q1. It learns how to retry insertion upon error because of that learned topology.

Dev: That sounds like a robust way to handle uncertainty during execution because the recovery mechanism is built into the structure they inferred, not just tacked on at the end.

Taro: And what about the initial learning phase? They use an iterative Expectation–Maximization training paradigm where they jointly train both the encoder that maps observations to symbols and that residual policy.

Rosa: That E-Step involves discovering the structure—updating those cluster assignments and refining the PMM—and then in the M-Step, they optimize the residual policy through behavior cloning while fine-tuning that observation encoder.

Dev: So, it’s a tight loop where structure discovery and control optimization happen at the same time, which is how they ensure that whatever structure they find is also actually useful for making precise movements.

Taro: I wonder about the semantic part too. They mention generating human-understandable concepts using GPT based on those cluster representatives to bridge the gap between robot control and symbolic meaning, like generating a concept called 'Hold'.

Rosa: That’s true, they use GPT based on the cluster representatives to generate these semantic meanings. It's an attempt to give us a way to understand what the robot is actually doing beyond just its raw motor commands.

Title and authors: Dev: So it provides that explicit, interpretable map of latent decision logic that we talked about earlier, not just a black box policy outputting vectors.

Taro: If this works on long-horizon tasks and handles failure recovery through those loops, what does that mean for real-world deployment outside of the lab?

Rosa: Well, the experiments show it performs well even in low-data regimes compared to state-of-the-art end-toend vision language agent policies by up to twenty-seven percent <ref:2603.25903#pg3>. It validates that this extracted PMM captures the true underlying task topology, acting as a strong inductive bias that generalizes beyond just what was seen during training.

Dev: That comparison to state-of-the-art VLA policies in low data regimes is a solid number because it suggests the structural prior they learned isn't just overfitting to the training set.

Taro: And I think one of the biggest implications is that we’re moving toward hierarchical cognitive architectures where a robot can actually plan those long sequences of manipulation steps by figuring out its own logical phases.

Rosa: Yeah, it gives us a way to tackle multi-stage manipulation and long horizon tasks because the system models those global task phases explicitly through that Probabilistic Mealy Machine.

Dev: It’s about moving from end-to-end reasoning to something more structured, where the high-level symbolic planning is guided by data rather than being manually defined by a programmer.

Taro: And for me, the autonomous failure recovery via those structural loops means we can build in robustness against uncertainties during execution, which is pretty important when you’re dealing with unpredictable real-world physics.

Rosa: Exactly. And that’s what makes it more than just a better policy; it gives us an explicit map of the agent's latent decision logic, which is really valuable for debugging and understanding complex behaviors.

Dev: So, to wrap up on "Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories," this framework unifies active automata inference with residual control to learn emergent neural automaton policies from visuomotor trajectories.

Taro: I just think the ability to discover those task structures and recover autonomously means we can build agents that are much more reliable when they run outside of a perfectly controlled simulation.

Rosa: It’s about getting the robot learning capability scaled for long-horizon tasks by letting it learn its own hierarchical reasoning structure from demonstrations.

Dev: It gives us a way to get interpretable intelligence that is tied directly to the physical actions the robot performs, not just abstract mathematical predictions.

The paper's summary: Rosa: So, to wrap up what we just talked about, this paper is showing how an AI can figure out complex task logic and even how to recover from mistakes just by watching a robot do things in the real world.

Dev: Right. It’s about taking those messy videos of a robot and turning them into a structured plan that’s both interpretable for us and robust enough to actually work outside the lab.

Taro: I mean, the summary talks about this bi-level policy emerging adaptively from just visuomotor demonstrations, which is pretty neat because it’s not about programming every single step.

Rosa: Exactly. They use something called adaptive clustering and an extension of the L star algorithm to infer a Mealy state machine from that raw data. That gives them a high-level planner that captures the task modes without any pre-labeled rules from us.

Dev: That discrete structure is key because it’s what we understand as symbolic reasoning, and then it guides a low-level residual network to handle the precise continuous control part. It’s like having a smart map guiding a precise steering wheel.

Taro: The paper points out that this PMM, or Probabilistic Mealy Machine, isn't just for planning; it actually discovers decision points autonomously, like figuring out when the robot needs to switch from one way of grasping something to another.

Rosa: They found this emergent structure where the agent exploits self-loops on certain nodes when things go wrong. So, if the robot messes up a step, instead of just crashing, it uses that learned topology to try again.

Dev: That sounds like a solid safety case for real-time systems because the recovery isn't arbitrary; it’s derived from the structure they inferred from the data itself. But what about training? How do you get this structure to be useful without having all those perfect demonstrations?

Taro: They use an iterative expectation maximization training paradigm where they jointly train the encoder that maps observations to symbols and that residual network for control. So, they discover the structure and tune the control at the same time in a loop.

Rosa: And they claim this method performs well even in low-data regimes, outperforming some state-of-the-art end-toend vision language agent policies by up to twenty seven percent. That suggests the structural prior is really strong.

Dev: That comparison is interesting because it shows that having a learned structure acts like a powerful inductive bias that helps generalize beyond just the specific data they used for training.

Taro: It also mentions generating human-understandable concepts using GPT based on those cluster representatives, which bridges the gap between robot movement and actual symbolic meaning, like giving the concept of 'Hold'.

Rosa: That’s a big deal because it means we can actually interpret what the robot is doing beyond just its raw motor commands. So this isn't just about better control; it’s about building intelligence that we can actually understand.

Dev: It offers an explicit, interpretable map of the agent's latent decision logic, which is something you really need when you’re designing systems that need a high degree of trust and reliability.

Taro: It gives us a way to tackle multi-stage manipulation and long horizon tasks because the system models those global task phases through that Probabilistic Mealy Machine. That's how we move past just seeing the next immediate step.

Rosa: So this framework unifies active automata inference with residual control to learn these emergent neural automaton policies from visuomotor trajectories, giving us a path toward generalizable and interpretable intelligence. Where do you think this stuff is going next?

The paper's improvements: Rosa: So, to wrap up what we just talked about, this paper is showing how an AI can figure out complex task logic and even how to recover from mistakes just by watching a robot do things in the real world.

Dev: Right. It’s about taking those messy videos of a robot and turning them into a structured plan that’s both interpretable for us and robust enough to actually work outside the lab.

Taro: I mean, the summary talks about this bi-level policy emerging adaptively from just visuomotor demonstrations, which is pretty neat because it’s not about programming every single step.

Conclusion: Rosa: So we’ve seen how this framework, "Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories," lets an AI figure out complex task logic just from watching robots perform actions.

Dev: Yeah, it’s about using that data to build a structure that lets the robot plan long sequences and recover when things go wrong.

Taro: I mean, what this means for real-world trials is that we can build systems that are much more robust because they learn how to handle those misbehaves structurally.

Rosa: That’s right. The system shows it can handle long horizon tasks better and perform well even in low-data regimes compared to some other end-toend vision language agent policies.

Dev: For us on the controls side, the idea of having that inferred PMM act as a strong inductive bias for the global dynamics is something we need to look at closely for our latency requirements.

Taro: And I think being able to see that structural loop recovery is crucial because it means when the world throws an unexpected variable at it, it doesn't just freeze; it uses its internal logic to find a way back.

Rosa: It gives us an explicit map of the agent’s latent decision logic, which is valuable for debugging and understanding complex behaviors we see in the field.

Dev: So we're moving toward systems where the high-level planning isn't just a black box prediction but something grounded in observable, inferred structure.

Taro: It’s about getting more reliable intelligence tied directly to the physical actions because it’s not just some abstract math guessing what to do next.

Rosa: That’s the core of it. We're seeing a path toward generalizable and interpretable intelligence by letting the robot learn its own hierarchical reasoning structure from demonstrations through this framework.

Dev: It's a lot of work getting that structure to be consistent across different task modes, but if it works in the trials, that’s a huge win for building safer real-time loops.

Taro: I think the next thing we should watch is how they handle those semantic meanings generated by GPT; bridging that gap between control and true understanding is where the real long-term potential lies.

Yiyuan Pan, Xusheng Luo, Hanjiang Hu, Peiqi Yu, Changliu Liu

Robotics Institute, Carnegie Mellon University

cs.RO

Submitted: 2026-03-26

Updated: 2026-04-16

Journal ref: Proceedings of Robotics: Science and Systems XXII (RSS 2026), Paper 144

DOI: 10.15607/RSS.2026.XXII.144

Project page: https://intelligent-control-lab.github.io/ENAP-project-webpage

License: http://creativecommons.org/licenses/by/4.0/

The gist: The gist The Emergent Neural Automaton Policy (ENAP) framework allows for unsupervised discovery of task-specific logic and autonomous failure recovery by learning a bi-level neuro-symbolic policy

Key concepts

Bi-level Neuro-symbolic Policy: This is a two-part system: a high-level planner (neuro) and a precise controller (symbolic). The high level uses learned structure to decide the overall task phase, while the low level executes fine motor actions. This combination leverages the strengths of both deep learning for continuous control and symbolic reasoning for structured planning.
Probabilistic Mealy Machine (PMM): This is a mathematical model inferred from movement data that represents a state machine. It captures both discrete decisions (states) and continuous dynamics (transitions). The PMM acts as the robot's high-level planner, providing coarse guidance on which task mode to be in at any given moment.
Symbol Abstraction: This process converts continuous visual observations into a discrete set of symbols. It involves mapping complex movement features into an alphabet where each symbol represents a distinct task phase or state. This allows the system to treat continuous input as a sequence of meaningful, interpretable steps.
Iterative Expectation–Maximization (EM) Training: This is the training method used to jointly optimize two components: the structure discovery part (the encoder) and the control part (the residual network). The system iteratively refines both by first discovering a better task structure and then updating the policy based on that discovered structure, ensuring consistency between planning and control.

Terminology

Clustering — 1× in this paper · explained in 2 episodes
Emergent Neural Automaton Policy — 1× in this paper
Interpretability — 1× in this paper · explained in 15 episodes
Planner — 1× in this paper · explained in 3 episodes
Sample Efficiency — 1× in this paper · explained in 7 episodes

Summary

The gist The Emergent Neural Automaton Policy (ENAP) framework allows for unsupervised discovery of task-specific logic and autonomous failure recovery by learning a bi-level neuro-symbolic policy from visuomotor demonstrations, which addresses the challenge of scaling robot learning to long-horizon tasks

How it works

ENAP introduces a framework that allows a bi-level neuro-symbolic policy to emerge adaptively from visuomotor demonstrations. Specifically, the framework first employs adaptive clustering and an extension of the L∗ algorithm to infer a Mealy state machine from visuomotor data, which serves as an interpretable high-level planner capturing latent task modes. Then, this discrete structure guides a low-level reactive residual network to learn precise continuous control via behavior cloning (BC). This bi-level architecture unifies discrete reasoning (where humans excel) and continuous control (where robots excel), aiming for both high interpretability and strong compositionality.

Symbol Abstraction

The first step in the ENAP framework is symbol abstraction, where continuous observation space is discretized into an alphabet Σ, and sequences of symbols are compressed into task phases using a Recurrent Neural Network (RNN). This is achieved by mapping observations ot:= (It, pt) to a semantic feature vector zt = ϕθ(ot) ∈ R z via an encoder. Following this, HDBSCAN is employed to unsupervisedly discover the alphabet Σ within the feature space. Each cluster leads to one symbol. For policy discovery scenarios, an RNN is trained to map the history (a0:t, c0:t) to a fixed-size embedding ht ∈ H. This RNN induces clustering of continuous hidden states around saturated centers, in which each component is driven toward ±1 under tanh activations.

Structure Extraction via Extended L∗ Algorithm

The paper introduces an extension of the L∗ algorithm for continuous trajectory data to infer a Probabilistic Mealy Machine (PMM). This extended algorithm maintains an Observation Table (U, E, T) to infer the automaton. The process involves two redefined query mechanisms: a Self-Supervised Membership Query (MQ) and a Non-deterministic Equivalence Query (EQ). The MQD(u) function retrieves the immediate action at and next-step embeddings hnext that are similar to the representative history u. The EQ validates the hypothesis by tracking all possible transition sequences in the PMM induced by a test trajectory.

Bi-level Control Pipeline

The learned PMM serves as a high-level planner, providing a coarse action prior at each time step. This is combined with a learned residual term ∆at ∼ πψ(·qt, ot) to synthesize the final precise action aˆt = a tbase + ∆at. The PMM captures the global dynamics (δ) and coarse intent (λ), leaving the neural network to focus solely on the local residual term πψ. The next state is determined as qt+1 = arg max q′∈δ(·qt,ct) h Ict+1∈NIS(q′) + ϵ · P(q′qt, ct).

Iterative Residual Learning and Refinement

The framework employs an iterative Expectation–Maximization (EM) training paradigm to jointly train the encoder ϕ(θ) and the residual network π(ψ). The E-Step involves structure discovery, yielding updated cluster assignments Σ(k+1) and a refined PMM M(k+1). The M-Step optimizes the residual policy πψ and fine-tunes the encoder ϕθ via behavior cloning. This EM-style training procedure ensures that the encoder learns a task-relevant manifold where “states” are both structurally consistent (for PMM) and control-effective (for residuals).

Emergent Structure Analysis

The extracted PMM autonomously discovers a decision point at node q2 (the “Grasp” phase), where multiple valid task branches exist due to the various task configurations. The residual policy guides the transition into the correct logical mode given the current observation. Furthermore, ENAP exhibits autonomous failure recovery through structural loops, where the agent exploits self-loops on nodes like “Align” to retry insertion upon error.

Conclusion

ENAP presents a framework for learning emergent neural automaton policies from visuomotor trajectories by unifying active automata inference with residual control. This representation not only enables sample-efficient learning in long-horizon tasks but also provides an explicit, interpretable map of the agent’s latent decision logic. This offers a viable path toward generalizable and interpretable intelligence >

How it works

Improvements for AI systems

Structural Priors for Long-Horizon Reasoning: The ENAP framework learns a “structured task abstraction and low-level reactive network in an emergent manner,” which allows agents to transition from end-toend reasoning to hierarchical cognitive architectures. This enables systems to tackle multi-stage manipulation and long horizon tasks by explicitly modeling the global task phases (System 2) through a Probabilistic Mealy Machine.
Label-Free Structure Discovery: The system can autonomously discover interpretable state machines directly from raw visuomotor data without requiring task-specific labels. This capability is achieved by employing an adaptive clustering and an extended L∗ algorithm to infer a Mealy state machine from visuomotor data.
Enhanced Control Efficacy in Low-Data Regimes: The improved system maintains high performance even in challenging scenarios, as experiments show ENAP outperforms state-of-the-art (SoTA) end-toend VLA policies by up to 27% in low-data regimes. This validates that the extracted PMM captures the true underlying task topology, acting as a strong inductive bias that generalizes beyond the training distribution.
Autonomous Failure Recovery: The policy gains robustness through structural loops, where the controller frequently exploits the self-loop on the “Align” node to retry actions when visual feedback does not match expected signatures. This capability is derived from the PMM structure, ensuring robustness against uncertainties during execution.
Semantic Meaning Generation: The framework generates human-understandable concepts by using GPT based on cluster representatives, allowing for a bridge between robotic control and symbolic meaning, such as generating semantic meanings like ‘Hold’.

Abstract

Scaling robot learning to long-horizon tasks remains a formidable challenge. While end-to-end policies often lack the structural priors needed for effective long-term reasoning, traditional neuro-symbolic methods rely heavily on hand-crafted symbolic priors. To address the issue, we introduce ENAP (Emergent Neural Automaton Policy), a framework that allows a bi-level neuro-symbolic policy adaptively emerge from visuomotor demonstrations. Specifically, we first employ adaptive clustering and an extension of the L* algorithm to infer a Mealy state machine from visuomotor data, which serves as an interpretable high-level planner capturing latent task modes. Then, this discrete structure guides a low-level reactive residual network to learn precise continuous control via behavior cloning (BC). By explicitly modeling the task structure with discrete transitions and continuous residuals, ENAP achieves high sample efficiency and interpretability without requiring task-specific labels. Extensive experiments on complex manipulation and long-horizon tasks demonstrate that ENAP outperforms state-of-the-art (SoTA) end-to-end VLA policies by up to 27% in low-data regimes, while offering a structured representation of robotic intent (Fig. 1).