Learning State-Action Control Barrier Functions for Model-Free Control under Constraints
eess.SY, cs.SY
Submitted: 2025-05-21
Updated: 2026-09-16
License: http://creativecommons.org/licenses/by/4.0/
The gist: Ensuring constraint satisfaction for learning-based control is a critical challenge, especially in the model-free case.
Terminology
Abstract
Ensuring constraint satisfaction for learning-based control is a critical challenge, especially in the model-free case. While safety filters address this challenge in the model-based setting, they typically rely on predictive models derived from physics or data. This reliance limits their applicability for advanced model-free learning control methods. To address this gap, we propose a new optimization-based control framework that determines safe control inputs directly from data. The benefit of the framework is that both the objective function and the safety constraint can be updated using model-free learning algorithms, enabling performance optimization while maintaining safety. As a key component, the concept of direct data-driven safety filters (3DSF) is first proposed. The framework employs a novel safety certificate, called the state-action control barrier function (SACBF). We present two approaches for synthesizing SACBFs, including learning from an expert controller and reinforcement learning. We develop a robustness framework that guarantees safety and recursive feasibility for the expert-guided approach under learning errors. The proposed control framework bridges the gap between model-free learning-based control and constrained control, by decoupling performance optimization from safety enforcement. Simulations on vehicle control illustrate the superior performance regarding constraint satisfaction and task achievement compared to model-based methods and reward shaping.
Sources
- Approximate predictive control barrier function for discrete-time systems
- Control Barrier Function-Based Safety Filters: Characterization of Undesired Equilibria, Unbounded Trajectories, and Limit Cycles
- Learning Performance-Oriented Control Barrier Functions Under Complex Safety Constraints and Limited Actuation
- Value Functions are Control Barrier Functions: Verification of Safe Policies using Control Theory
- Safety Filters for Black-Box Dynamical Systems by Learning Discriminating Hyperplanes
- A Forward Reachability Perspective on Control Barrier Functions and Discount Factors in Reachability Analysis
- Discrete GCBF Proximal Policy Optimization for Multi-agent Safe Optimal Control
- Predictive Control with Learning-Based Terminal Costs Using Approximate Value Iteration
- Safe Exploration in Continuous Action Spaces
- Safety with Agency: Human-Centered Safety Filter with Application to AI-Assisted Motorsports
Related papers
- One Request, Multiple Experts: LLM Orchestrates Domain Specific Models via Adaptive Task Routing
- A Geometric Decision Procedure for STL Feasibility and Repair
- From Asymptotic to Designer-Assigned-Time Control: A Review of Stability Notions, Design Mechanisms, and Controller Architectures
- Battery Second Life: A Review of Experimental Studies
- Submodular Multi-Agent Policy Learning for Online Distributed Task Allocation in Open Multi-Agent Systems
- Policy-Level Recursive Self-Improvement for Embodied AI with a Criticality World Model