Novelty Adaptation Through Hybrid Large Language Model (LLM)-Symbolic Planning and LLM-guided Reinforcement Learning
cs.RO, cs.AI
Submitted: 2026-03-11
Updated: 2026-10-02
Comments: Accepted at IEEE/RSJ International Conference on Intelligent Robots & Systems (IROS) 2026
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: In dynamic open-world environments, autonomous agents often encounter novelties that hinder their ability to find plans to achieve their goals.
Terminology
Abstract
In dynamic open-world environments, autonomous agents often encounter novelties that hinder their ability to find plans to achieve their goals. Specifically, traditional symbolic planners fail to generate plans when the robot's planning domain lacks the operators that enable it to interact appropriately with novel objects in the environment. We propose a neuro-symbolic architecture that integrates symbolic planning, reinforcement learning, and a large language model (LLM) to learn how to handle novel objects. In particular, we leverage the common sense reasoning capability of the LLM to identify missing operators, generate plans with the symbolic AI planner, and write reward functions to guide the reinforcement learning agent in learning control policies for newly identified operators. Our method outperforms the state-of-the-art methods in operator discovery as well as operator learning in continuous robotic domains.Our webpage and code can be access here: https://helenlu66.github.io/hybridLLMguided/
Sources
- SPOTTER: Extending Symbolic Planning Operators through Targeted Reinforcement Learning
- Large Language Models as Commonsense Knowledge for Large-Scale Task Planning
- MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
- Neuro-Symbolic World Models for Adapting to Open World Novelty
- Learning Neuro-Symbolic Skills for Bilevel Planning
- ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
- Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents
- ReAct: Synergizing Reasoning and Acting in Language Models
- Inner Monologue: Embodied Reasoning through Planning with Language Models
- Dynamic Planning with a LLM
- Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents
- LLM+P: Empowering Large Language Models with Optimal Planning Proficiency
- Exploiting Contextual Structure to Generate Useful Auxiliary Tasks
- Reward Design with Language Models
- Self-Refined Large Language Model as Automated Reward Function Designer for Deep Reinforcement Learning in Robotics
- Text2Reward: Reward Shaping with Language Models for Reinforcement Learning
- Eureka: Human-Level Reward Design via Coding Large Language Models
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies