HarnessSecurity-Bench: Do Security Mechanisms Really Protect Coding Agent Harnesses?
cs.CR, cs.SE
Submitted: 2026-10-06
Updated: 2026-10-10
Comments: 23 pages, 6 figures, 11 tables. Project website: https://tsingpig.github.io/HarnessSecurity-Benchmark/
Code: https://github.com/TsingPig/HarnessSecurity-Benchmark
Project page: https://tsingpig.github.io/HarnessSecurity-Benchmark/ABSTRACT
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents
- Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents
- An Empirical Study of Harness Design for Coding Agents
- The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey
- Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses
- Auditing Agent Harness Safety
- "Your AI, My Shell": Demystifying Prompt Injection Attacks on Agentic AI Coding Editors
- SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction
- SHE: Trajectory-driven Safety Harness Evolution for LLM Agents
- Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems
- The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities
- Beyond Handcrafted Security: Towards Self-Evolving Defense for LLM Agents
- Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
- IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests
- The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluation
- ClawSafety: "Safe" LLMs, Unsafe Agents
- QueryIPI: Query-agnostic Indirect Prompt Injection on Coding Agents
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs