Agentic Self-Awareness: Aligning Knowing, Acting, and Holding
cs.AI
Submitted: 2026-10-13
Updated: 2026-10-13
License: http://creativecommons.org/licenses/by/4.0/
The gist: Frontier AI agents increasingly act in environments where seemingly solvable tasks may lack prerequisites, contain conflicting requirements, or reveal blockers only during execution.
Abstract
Frontier AI agents increasingly act in environments where seemingly solvable tasks may lack prerequisites, contain conflicting requirements, or reveal blockers only during execution. We formulate agentic self-awareness (ASA) as the alignment of task-grounded feasibility judgments (know), warranted behavior (act), and responses to reconsideration (hold), distinguishing what a task permits from what an agent can solve. We build paired feasible and blocked tasks in Terminal-Bench 2.1 and retail τ squared-bench and introduce an authorship-aware M0-M3 rubric that separates recognizing a blocker from taking the certified deferral action for the correct reason. Separate probes assess stated feasibility beliefs and responses to a single challenge. In a Terminal-Bench protocol baseline, blocker recognition reaches 84.4%, but strict self-awareness (SA) reaches only 40.6%. Harness search without model-weight updates increases strict SA from 45.3% to 93.8% on reused Terminal-Bench search tasks; selection for abstention alone, however, can reduce success on feasible tasks. Independently initialized joint-objective searches improve both blocked-task SA and feasible-task solve on their own search sets. Retail held-out gains remain limited, and Terminal-Bench blocked-task transfer is not measured. This evaluation pipeline makes the recognition-action gap measurable and sets a two-sided target for improving agent decisions under changing task evidence.
Sources
- $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
- HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry
- Do Agents Know What They Can't Do? Evaluating Feasibility Awareness in Tool-Using Agents
- UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making
- Overconfidence is Key: Verbalized Uncertainty Evaluation in Large Language and Vision-Language Models
- Automated Design of Agentic Systems
- Language Models (Mostly) Know What They Know
- Agentic Uncertainty Reveals Agentic Overconfidence
- Why Language Models Hallucinate
- AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
- Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment
- From Knowing to Acting: Benchmarking Self-Awareness Capability of LLM Agents
- AgentAbstain: Do LLM Agents Know When Not to Act?
- Agentic Abstention: Do Agents Know When to Stop Instead of Act?
- Towards Understanding Sycophancy in Language Models
- HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?
- MIRROR: A Hierarchical Benchmark for Metacognitive Calibration in Large Language Models
- On Calibration of Large Language Models: From Response To Capability
- Agentic Uncertainty Quantification
- Agentic Confidence Calibration
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems