Same Attack, Different Judge: A First Controlled Adversarial Comparison of Jev, LLM Judges, and Purpose-Built Safety Classifiers

arXiv:2610.17820 · cs.CR, cs.AI · Submitted 2026-10-13 · Read on arXiv

cs.CR, cs.AI

Submitted: 2026-10-13

Updated: 2026-10-13

Comments: 21 pages, 10 figures

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• LLM judge — 1× in this paper · explained in 2 episodes

Sources

• Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failures
• Assessing Automated Prompt Injection Attacks in Agentic Environments
• JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models
• Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
• The Llama 3 Herd of Models
• Prompt Injection Attack to Tool Selection in LLM Agents
• ToolTweak: An Attack on Tool Selection in LLM-based Agents
• Type-Safe Is Not Error-Free: Typed Decision Models Follow the Option Name, Not the Definition Bound to It
• Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills
• Token Composition: A Graph Based on EVM Logs