Discriminative Barrier Functions for Safe Adversarial Imitation Learning from Observation

arXiv:2607.13938 · cs.RO · Submitted 2026-07-15 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "Discriminative Barrier Functions for Safe Adversarial Imitation Learning from Observation".

Dev: The gist:

Rosa: First, who's behind it and why it matters.

Title and authors: Rosa: So we're looking at this paper titled Discriminative Barrier Functions for Safe Adversarial Imitation Learning from Observation, and the authors are Anubhav Vishwakarma, Bhaumik Mehta, Caleb Hsu, Byron Boots, Karen Leung, Tyler Han. What do these names tell us about what they're doing?

Dev: They’re tackling the problem of using Inverse Reinforcement Learning to learn from expert demonstrations but making it safe for the real world where we don't have explicit safety labels.

Rosa: That makes sense. The title points directly at the core idea, which is using these barrier functions to constrain how the reward function can be learned during that imitation process.

Dev: It’s about taking something like Inverse Reinforcement Learning, which is good for learning what an expert wants but it usually leaves a lot of room for unsafe exploration, and we want to bring in Control Barrier Functions to put some hard safety limits on that exploration.

Taro: So you're saying the goal is to learn a policy from demonstrations while making sure that whatever you learn respects some fundamental safety rules, even if the expert didn't explicitly show you every possible unsafe situation.

Rosa: Exactly. The paper suggests this approach yields a formulation that has safe online control with continuous improvement, which means as the system learns, it gets safer without needing human intervention or hand-designed constraints for every single scenario.

Dev: It’s about constraining the reward function candidacy to the space of Control Barrier Functions, which is a way to ensure that if you are in a state where an unsafe observation occurs, the resulting learned reward structure will steer you away from it.

The paper's summary: Rosa: To summarize what this paper is about, they introduce something called Discriminative Barrier Functions, or DBFs, which are ways to restrict the class of cost functions we allow the Inverse Reinforcement Learning process to choose from.

Dev: The main point is that by constraining the reward function search to these CBFs, you get a formulation that exhibits safe online control with continuous experiential improvement.

Rosa: This framework lets you achieve something really important: the data-driven recovery of barrier functions directly from unlabeled expert observations, which means they don't need those explicit unsafe labels we usually have to engineer.

Taro: So if I understand correctly, this means the learned barrier function is robust to any unsafe states that were completely absent from the expert demonstrations, which is a big deal for real-world deployment.

Dev: That’s right. They demonstrate that this recovered barrier function generalizes zero-shot zero collisions to a novel obstacle configuration unseen during training without needing retraining.

Rosa: And they show this works in both policy-based and planning-based Inverse Reinforcement Learning frameworks, which means it’s not tied to one specific way of planning or learning, it’s quite general.

Dev: It is. They benchmarked this against standard IRL baselines in a simulated navigation environment and showed improved safety performance compared to those baselines.

The paper's improvements: Rosa: A key improvement they highlight is that by using the DBF objective, you can significantly reduce collisions in online navigation settings, which really mitigates the risks of environment interaction during Inverse Reinforcement Learning from Observation.

Dev: That’s because incorporating this DBF regularization into the reward function search actually makes it possible to enable safe online imitation learning without needing action supervision or hand-designed constraints.

Taro: So instead of you having to manually design a safety layer for every potential problem, the AI learns the necessary inhibitory control directly from just observing expert trajectories.

Rosa: Right. And they show that this method consistently reduces cost rate as learning progresses, unlike other methods like GAIL which collapses entirely because they fail to discriminate against unseen obstacles.

Dev: They also found that DBF-MPAIL achieves the lowest cost rate at convergence, while DBF-GAIL has a higher cost rate but better reward; it shows you have to trade off some performance for safety, and this is what DBFs are designed to manage.

Taro: The real world validation shows that DBF variants achieve zero collisions across all trials, and they successfully trace paths during inference that aren't even in the expert demonstrations, which means they can navigate around obstacles unseen in the expert data.

Conclusion: Rosa: So to wrap up, this paper introduces Discriminative Barrier Functions for Safe Adversarial Imitation Learning from Observation, a framework that incorporates Control Barrier Function theory directly into the AI framework.

Dev: By constraining the learned discriminator through these barrier-function objectives, DBFs enable AI methods to recover notions of safety and inhibitory control directly from unlabeled demonstrations without requiring explicit unsafe labels or hand-engineered safety costs.

Taro: This means they recover coherent safety boundaries that are robust to previously unseen obstacles and environmental changes, which is important for real world deployment.

Rosa: Across both policy-based and planning-based AI algorithms, they demonstrated that DBFs improve the generalization of learned safe sets beyond the expert data distribution without sacrificing imitation performance.

Dev: The implication here is that incorporating control-theoretic structure into adversarial imitation learning provides a promising direction toward safer and more robust online learning from observation.

Taro: It’s about getting better at recovering safety boundaries from just watching demonstrations, which is crucial for making these systems trustworthy when they go out in the real world.

Rosa: That’s what this work does: it shows how we can recover safety boundaries that generalize and induce safer exploration during online learning in both simulation and the real world.

University of Washington

cs.RO

Submitted: 2026-07-15

Updated: 2026-10-14

Comments: 20 pages, 5 figures

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 91/100

The gist: The gist: This work introduces Discriminative Barrier Functions (DBFs) to constrain reward function candidacy during Inverse Reinforcement Learning from Observation (IRLfO) to yield a formulation

Key concepts

Discriminative Barrier Functions (DBFs)
DBFs are a constrained class of reward functions derived from Control Barrier Function theory. They are designed to capture expert intent while inherently discriminating against unsafe states, such as obstacles completely absent from the expert demonstrations. This constraint allows the system to learn safe boundaries directly from unlabeled data.
Inverse Reinforcement Learning from Observation (IRLfO)
IRLfO is a framework where an agent learns the underlying reward function of an expert solely by observing their actions and states without explicit labels. The paper modifies this by constraining the search space for the reward function to be within the set of DBFs, ensuring that any learned policy adheres to safety guarantees.
Control Barrier Functions (CBFs)
CBFs are mathematical functions used in control theory to guarantee system safety by defining a safe region. In this context, they are used as the structure for the reward function. By using CBFs, the framework gains inherent control-theoretic properties that ensure safe online navigation and collision avoidance.
Generalization to Unseen Obstacles
This refers to the ability of DBF-MPAIL to maintain safety when encountering obstacles not present in the expert training data. The learned barrier function generalizes zero-shot safety, correctly classifying novel, unseen obstacles as unsafe without needing retraining. This demonstrates that the learned safety constraints are robust and not just memorized from the expert examples.

Terminology

Summary

The gist: This work introduces Discriminative Barrier Functions (DBFs) to constrain reward function candidacy during Inverse Reinforcement Learning from Observation (IRLfO) to yield a formulation exhibiting safe online control with continuous experiential improvement.

Learning Interpretable Barrier Functions

The research demonstrates that restricting the cost function class within the IRLfO framework to Discrete-time CBFs yields highly interpretable and generalizable barrier functions, which is called DBF This constrained objective to synthesize cost function through IRLfO is called DBF. These functions successfully capture expert intent while inherently discriminating against unsafe observations, such as obstacles completely absent from the expert demonstrations.

Safe Online Learning from Observation

Incorporating DBF regularization into the reward function search significantly reduces collisions in online navigation settings, which mitigates the risks of environment interaction during IRLfO and paves the way toward safe online deployment in the real world. The framework enables safe online imitation learning without action supervision and handdesigned constraints.

Empirical Validation of the DBF Formulation

The method is tested across both policybased and planning-based IRLfO frameworks, demonstrating its efficacy in obstacle avoidance tasks through both simulation and real-world robotic experiments unseen during training time, thereby showcasing the generalization capability of DBF. The results show that DBF objective strongly influences the performance of planning-based method, since the reward function (i.e., the CBF constraint) is directly used as a stage cost of MPPI planner, providing strong guidance toward safe actions.

Generalization to Unseen Obstacles

The DBF-MPAIL method successfully navigates both scenarios, demonstrating generalization to unseen obstacle configurations. The learned barrier function generalizes zero-shot (zero collisions) to a novel obstacle configuration unseen during training, correctly classifying the new obstacle as unsafe without retraining. DBF-MPAIL consistently trace path during inference not directly present in the expert demonstrations, as shown in Figure 1, to avoid obstacles while imitating the behavior needed to reach the goal.

Safety Performance Comparison

DBF-based methods consistently reduce cost rate as learning progresses, while GAIL collapses entirely, failing to imitate the expert and accumulating high constraint violations due to failure to discriminate against unseen obstacle configurations absent from the expert demonstrations. DBF-GAIL achieves the lowest cost rate at convergence, while DBF-MPAIL trades marginally higher cost rate for significantly better reward than DBF-GAIL, highlighting the safety-performance trade-off that DBFs are designed to address. The results suggest that incorporating control-theoretic structure into adversarial imitation learning provides a promising direction toward safer and more robust online learning from observation.

Real World Validation

DBF-MPAIL consistently trace path during inference not directly present in the expert demonstrations, as shown in Figure 1, to avoid obstacles while imitating the behavior needed to reach the goal. DBF variants achieve zero collisions across all trials. In addition, Figure 1 and Figure 7 illustrate a key advantage of planning-based DBFAIL: success in imitating goal-reaching behavior while avoiding obstacles unseen in the expert data and at unseen positions in the training environment. DBF-based methods enhance the safety of imitation learning algorithms in real-world safety-critical deployments.

Conclusion

In this work, we introduce Discriminative Barrier Functions (DBFs), a framework for incorporating Control Barrier Function (CBF) theory directly into the AIL framework. By constraining the learned discriminator through barrier-function objectives, DBFs enable AIL methods to recover notions of safety and inhibitory control directly from unlabeled demonstrations without requiring explicit unsafe labels or hand-engineered safety costs. Across both policy-based and planning-based AIL algorithms, we demonstrate that DBFs improve the generalization of learned safe sets beyond the expert data distribution without sacrificing imitation performance. DBFs recover coherent safety boundaries, exhibit robustness to previously unseen obstacles and environmental changes, and induce safer exploration during online learning in both simulation and the real world.

Limitations and Future Work

Future work will consider approximate Hamilton-Jacobi reachability methods to modify the generalized advantage estimator, synthesizing a safety-aware value function for improved performance. Additionally, DBFs may become overly conservative when expert data covers only a narrow subset of valid strategies, incorrectly suppressing safe but underrepresented behaviors. Future work may investigate uncertainty-aware DBFs, active data collection, or mechanisms for reasoning about demonstration coverage and ambiguity. The K-class function is currently specified by hand; prior work suggests these functions may themselves be learnable, potentially enabling more adaptive barrier representations across tasks with complex or heterogeneous safety constraints.

Acknowledgments

All acknowledgments go at the end of the paper, including thanks to reviewers who gave useful comments, to colleagues who contributed to the ideas, and to funding agencies and corporate sponsors that provided financial support.

References

[1] A. Y. Ng and S. J. Russell. Algorithms for inverse reinforcement learning

[2] P. Abbeel and A. Y. Ng, Apprenticeship learning via inverse reinforcement learning

[5] F. Torabi, G. Warnell, and P. Stone, Generative Adversarial Imitation from Observation

[12] A. D. Ames et al., Control Barrier Functions: Theory and Applications

[23] A. Robey et al., Learning Control Barrier Functions from Expert Demonstrations

[45] V. Sharma et al., Learning differentiable and safe multi-robot control for generalization to novel environments using control barrier functions

[44] I. Mitchell, A. Bayen, and C. Tomlin, A time-dependent hamilton-jacobi formulation of reachable sets for continuous dynamic games

[25] Y. Yang et al., Enhancing Safety in Learning from Demonstration Algorithms via Control Barrier Function Shielding

[30] S. Gu et al., A Review of Safe Reinforcement Learning: Methods, Theories, and Applications

[41] A. Ray, J. Achiam, and D. Amodei, Benchmarking safe exploration in deep reinforcement learning

[42] J. Sun et al., Adversarial Inverse Reinforcement Learning With Self-Attention Dynamics Model

[40] NVIDIA,:, M. Mittal, P. Roth, J. Tigue, A. Richard, O. Zhang, P. Du, A. Serrano-Munoz, X. Yao et al., Isaac lab: A gpu-accelerated simulation framework for multi-modal robot learning

[43] J. F. Fisac et al., Bridging hamiltonjacobi safety analysis and reinforcement learning

[45] V. Sharma, N. Mehr, and N. Hovakimyan, Learning differentiable and safe multi-robot control for generalization to novel environments using control barrier functions

[42] J. Sun et al., Adversarial Inverse Reinforcement Learning With Self-Attention Dynamics Model

[45] V. Sharma, N. Mehr, and N. Hovakimyan, Learning differentiable and safe multi-robot control for generalization to novel environments using control barrier functions

[42] J. Sun et al., Adversarial Inverse Reinforcement Learning With Self-Attention Dynamics Model

[45] V.

Improvements for AI systems

  1. Improved AI systems can recover a learned barrier function directly from unlabeled expert observations, enabling safe online imitation learning without action supervision and handdesigned constraints. This allows robots to learn inhibitory control near unsafe states simply by observing expert trajectories, which is crucial for real-world deployment where action labels are unavailable.

  2. The framework enables the data-driven recovery of barrier functions directly from unlabeled expert observations, ensuring that the recovered barrier function is robust to unsafe states entirely absent from the expert data. This means the learned safety boundaries generalize effectively to novel obstacle configurations unseen during training, as demonstrated by DBF-MPAIL achieving zero collisions in unseen scenarios.

  3. DBF-based algorithms can be improved for online learning by exhibiting safety performance that consistently reduces cost rate while maintaining competitive reward, unlike GAIL which collapses entirely. This ensures that as the robot learns, it actively learns to discriminate against unsafe states, mitigating risks during the exploration phase of Imitation Learning from Observation.

  4. The system can be enhanced by utilizing a planning-based approach (DBF-MPAIL) to achieve superior performance in real-world tasks, as it consistently trace[s] path during inference not directly present in the expert demonstrations. This capability, achieved by using the discriminator as a stage cost for an MPPI planner, provides strong guidance toward safe actions that are robust to distribution shifts between simulation and reality.

Sources

Related papers