Judges Follow the Reference: The Derivation Gap in Verifying Tool-Call Arguments
cs.AI, cs.CL
Submitted: 2026-10-13
Updated: 2026-10-13
Comments: 28 pages, 6 figures, 18 tables
Code: https://github.com/SarielMa/DeRef_release
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- ToolRM: Outcome Reward Models for Tool-Calling Large Language Models
- FinTrace: Holistic Trajectory-Level Evaluation of LLM Tool Calling for Long-Horizon Financial Tasks
- LLM Judges Can Be Too Generous When There Is No Reference Answer
- TwinCheck: Evidence-Grounded Negative-Twin Verification for Stateful Tool Agents
- The Granularity Mismatch in Agent Security: Argument-Level Provenance Solves Enforcement and Isolates the LLM Reasoning Bottleneck
- The Llama 3 Herd of Models
- Mistral 7B
- LongFuncEval: Measuring the effectiveness of long context models for function calling
- No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding
- Judging Against the Reference: Uncovering Knowledge-Driven Failures in LLM-Judges on QA Evaluation
- JEV-as-a-Judge: Accept When Confident, Escalate When Unsure
- Backbone-Adaptive Evidence Routing for Robust Pairwise LLM Judging
- Agent-Sentry: Bounding LLM Agents via Execution Provenance
- AgentJudgeBench: A Multi-Difficulty Benchmark for Evaluating LLM Judges on Agentic Tool-Calling
- Qwen3 Technical Report
- ComplexFuncBench: Exploring Multi-Step and Constrained Function Calling under Long-Context Scenario
- More Convincing, Not More Correct: Self-Play Reward Hacking of Reference-Free LLM Judges
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems