RuleTrek: Benchmarking Agentic Rule Reasoning in Complex Scenarios
cs.AI
Submitted: 2026-10-14
Updated: 2026-10-14
Code: https://github.com/yuan247/RuleTrek
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Beyond Case Law: Evaluating Structure-Aware Retrieval and Safety in Statute-Centric Legal QA
- Transformers as Soft Reasoners over Language
- GLM-5: from Vibe Coding to Agentic Engineering
- LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
- AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning
- STAGE-Claw: Automated State-based Agent Benchmarking for Realistic Scenarios
- EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings
- Beyond Instruction Following: Evaluating Inferential Rule Following of Large Language Models
- Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies
- Policy-Invisible Violations in LLM-Based Agents
- TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks
- RuleWeaver: Benchmarking Rule-Centered Scenario Reasoning for Large Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems