Discovery or Memorization? A Faithful Benchmark for Real-World Business-Logic Vulnerability Discovery

arXiv:2610.17542 · cs.CR · Submitted 2026-10-13 · Read on arXiv

cs.CR

Submitted: 2026-10-13

Updated: 2026-10-13

Comments: 22 pages, 11 figures, 4 tables

Project page: https://www.ndss-symposium.org/ndss-paper/auto-draft-207

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 1× in this paper · explained in 2 episodes

Sources

• Theory-Scale Auto-Formalization of Logics for Computer Science
• Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilities
• Detecting Privilege Escalation in Polyglot Microservices via Agentic Program Analysis
• CVE-Factory: Scaling Expert-Level Agentic Tasks for Code Security Vulnerability
• HoF-Bench: Rediscovering Real AI-Discovered CVEs Without Frontier Models
• ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?
• CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wild
• Same Request, Different Boundary: Evaluating Cybersecurity Assistance across Conversational Contexts
• SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
• Locus: Agentic Predicate Synthesis for Directed Fuzzing

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Estimating the Social Cost of Corporate Data Breaches
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills