What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds
cs.CL, cs.AI, cs.SI
Submitted: 2026-10-14
Updated: 2026-10-14
License: http://creativecommons.org/licenses/by/4.0/
The gist: Effective social interaction requires more than fluent responses: agents must reach plans that fit the participants' needs and circumstances.
Abstract
Effective social interaction requires more than fluent responses: agents must reach plans that fit the participants' needs and circumstances. Existing evaluations of social agents mainly rely on LLM judges that rate the conversation, without checking the resulting plan. We introduce HICO-Bench, a benchmark of 180 everyday coordination scenarios, such as resource allocation, scheduling, and authorization, in which two agents with private information converse and then independently record the plan they agreed on. A program checks Feasibility (the two records match and satisfy both participants' private conditions), and an LLM judge checks Integrity (the conversation supports the recorded plan); passing both is a Verified Success. Each scenario has two versions that share the same public context but differ in one private condition, so the correct plan differs between them. Across thirteen LLMs, 36% of conversations rated highly for goal completion fail Verified Success. To learn from outcomes that arrive only at the end of a conversation, we propose Paired-Credit Policy Optimization (PaCPO). After each turn, PaCPO asks the agent for a tentative plan, checks it in both versions of the scenario, and rewards the turns after which the tentative plans fit their respective versions. PaCPO raises Qwen3-8B's Verified Success from 20.4% to 28.3%.
Sources
- STaR-GATE: Teaching Language Models to Ask Clarifying Questions
- RUDDER: Return Decomposition for Delayed Rewards
- $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
- Towards Interactive Intelligence for Digital Humans
- Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learning
- SAVOIR: Learning Social Savoir-Faire via Shapley-based Reward Attribution
- GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experience
- Vinci2: Providing Proactive Assistance in Continuous Egocentric Videos
- Social Gym and SPaRTan: Benchmarking and Improving LLM Social Reasoning via Multi-Agent Game Tournaments
- LoRA: Low-Rank Adaptation of Large Language Models
- Can MLLMs Read the Room? A Multimodal Benchmark for Assessing Deception in Multi-Party Social Interactions
- Can MLLMs Read the Room? A Multimodal Benchmark for Verifying Truthfulness in Multi-Party Social Interactions
- Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality?
- Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems
- Understanding R1-Zero-Like Training: A Critical Perspective
- GPT-4o System Card
- SocialDirector: Training-Free Social Interaction Control for Multi-Person Video Generation
- High-Dimensional Continuous Control Using Generalized Advantage Estimation
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering