JevOut: Natural Context Can Flip Decision Models

arXiv:2609.30243 · cs.CL · Submitted 2026-09-24 · Read on arXiv

cs.CL

Submitted: 2026-09-24

Updated: 2026-10-08

Comments: 46 pages, 8 figures, 30 tables. V2: substantially strengthened empirical evaluation with blinded human validation, a budget-matched independent-generation baseline, and repeatability analyses; expanded related work and updated author list. Homepage: https://xzx34.github.io/jevout/ ; Code: https://github.com/xzx34/JevOut

Code: https://github.com/xzx34/JevOut

Project page: https://xzx34.github.io/jevout

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Adaptation — 6× in this paper · explained in 1 episode
• Benchmark — 5× in this paper · explained in 2 episodes
• LLM-as-a-Judge — 5× in this paper · explained in 3 episodes
• Robustness — 5× in this paper · explained in 5 episodes
• Targeted Flip Rate — 5× in this paper
• Accuracy — 4× in this paper · explained in 1 episode
• Cross-Model Transfer — 4× in this paper · explained in 2 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Calibration — 2× in this paper · explained in 7 episodes
• Accountability — 1× in this paper · explained in 2 episodes
• Adversarial Attacks — 1× in this paper · explained in 1 episode
• Agentic Evaluation — 1× in this paper
• Aggregation — 1× in this paper
• Benchmarking — 1× in this paper · explained in 4 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Provenance — 1× in this paper · explained in 5 episodes
• Steering — 1× in this paper · explained in 2 episodes
• Trace — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• Gemma 4 Technical Report
• LoRA: Low-Rank Adaptation of Large Language Models
• SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines
• Qwen3 Technical Report
• Type-Safe Is Not Error-Free: Typed Decision Models Follow the Option Name, Not the Definition Bound to It
• MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
• SATA-BENCH: Select All That Apply Benchmark for Multiple Choice Questions
• Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering