BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views
cs.CR, cs.LG
Submitted: 2026-10-12
Updated: 2026-10-12
License: http://creativecommons.org/licenses/by/4.0/
The gist: Browser agents need to make decisions and execute actions on behalf of users while consuming untrusted webpages, creating opportunities for malicious instructions embedded in webpages to redirect
Terminology
Abstract
Browser agents need to make decisions and execute actions on behalf of users while consuming untrusted webpages, creating opportunities for malicious instructions embedded in webpages to redirect legitimate tasks. We introduce BrowseSafe++, which extends BrowseSafe from 14,719 pages to 38,675 pairs of pages. Most of its prompt injections were observed in the wild on production websites. We keep each injection unchanged and rewrite the sensitive parts of the page around it to protect privacy. Each attacked page has a clean copy of the same page as its control. Both are provided as HTML, Markdown, an accessibility tree, and a screenshot. Every pair includes a user task and its expected answer. The benchmark includes twelve types of websites, from news articles to shared documents and forums, with 24 attack types appearing in 18 different locations on a page. We use these paired pages to train BrowseSafe-Guard, two 0.8B-parameter detectors for text and screenshots. Both achieve higher F1 scores than the detector based on frontier models, while keeping false-positive rates on benign pages and inference cost low. BrowseSafe-Adapt revises benchmark injections from browser-agent feedback, and attacks that succeed on only a few percent of tasks when replayed directly reach up to 95.41% success once adapted.
Sources
- LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge
- Qwen3-VL Technical Report
- CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models
- VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents
- The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injections
- Defending Against Prompt Injection With a Few DefensiveTokens
- SecAlign: Defending Against Prompt Injection with Preference Optimization
- Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents
- DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents
- Securing AI Agents with Information-Flow Control
- Defeating Prompt Injections by Design
- DeepSeek-V3 Technical Report
- WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks
- The Llama 3 Herd of Models
- Defending Against Indirect Prompt Injection Attacks With Spotlighting
- Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents
- OverThink: Slowdown Attacks on Reasoning LLMs
- ACE: A Security Architecture for LLM-Integrated App Systems
- EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
- RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Estimating the Social Cost of Corporate Data Breaches
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills