From Observations to Authorized Actions: Measuring Source Information in Android Agents

Android agents read content from applications and web pages and act on it with the user's authority, yet they cannot reliably tell where that content came from -- its source, the application or document that supplied it. The WebView accessibility fields we inspected expose the host application but omit the document origins available inside the browser, so an agent reading through this interface cannot check which document supplied an object; a limited reproduction on an unmodified WebView yields the same finding. We build and separately evaluate two prototypes to examine how source information supports authorization. Our application-source monitor withholds side-effecting calls whose signed source record contains an untrusted application. With the source set derived from the OS accessibility log, it blocked every selected attack write in the tested runs, but it also withheld benign sends issued after reading untrusted content, flagged benign tasks that use external content, and carried source records into a later clean task. Our WebView source-reference gate uses records from a modified browser to check object origins and reference lifetime through direct calls; in a separate 24-trial evaluation, the live agent selected no disallowed object. Supporting benchmark and planner measurements characterize textual argument matches and attacker-target selection, without establishing causal dataflow or a controlled model ranking. The prototypes are not an integrated defense: the WebView gate still lacks authenticated browser delivery and atomic capture of selected bytes and origin. Together, the results show which source information the tested policies need and why it must stay linked to the object used in execution.