Empathy Is Not the Scapegoat for Declines in Factuality and Reliability
cs.CL
Submitted: 2026-10-13
Updated: 2026-10-13
Comments: Accepted by AACL 2026
Code: https://github.com/unitaryai/detoxify
License: http://creativecommons.org/licenses/by/4.0/
The gist: Language models are increasingly expected to support users in stressful or uncertain situations, where warmth and empathy are desirable.
Abstract
Language models are increasingly expected to support users in stressful or uncertain situations, where warmth and empathy are desirable. While recent work associates warmer models with reduced reliability, we show that this conclusion is incomplete. Prior studies often train primarily on empathetic dialogue, making it difficult to separate tone effects from SFT-induced distribution shift. We revisit this issue at both the model and evaluation levels. We study prompting, SFT, and RL for inducing warm behavior, and comprehensively evaluate capability preservation using an 11-dataset benchmark suite and EMOFACT under emotional contexts. Across multiple model families, we find that warmth effects are task- and model-dependent, while task-aligned adaptive tone can improve empathy and targeted reliability, although some capability trade-offs remain. We further introduce human-validated empathy evaluators specifically developed to assess empathetic model responses. Our results suggest that models can be more supportive in social interactions while remaining reliable.
Sources
- Evaluating Large Language Models Trained on Code
- Training Verifiers to Solve Math Word Problems
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- The Llama 3 Herd of Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing
- Training language models to be warm and empathetic makes them less reliable and more sycophantic
- GPT-4o System Card
- Beacon: Single-Turn Diagnosis and Mitigation of Latent Sycophancy in Large Language Models
- Qwen2.5 Technical Report
- The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems
- Steering Llama 2 via Contrastive Activation Addition
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Towards Understanding Sycophancy in Language Models
- From Measurement to Expertise: Empathetic Expert Adapters for Context-Based Empathy in Conversational AI Agents
- SENSE-7: Taxonomy and Dataset for Measuring User Perceptions of Empathy in Sustained Human-AI Conversations
- PERM: Psychology-grounded Empathetic Reward Modeling for Large Language Models
- Simple synthetic data reduces sycophancy in large language models
- Qwen3 Technical Report
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering