Accuracy and Uncertainty in Machine Deception Detection (Research White Paper)
cs.LG, cs.AI, cs.CR, stat.ML
Submitted: 2026-10-11
Updated: 2026-10-11
Comments: 18 pages, 2 figures
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: A deception detector is only as useful as the trust its alarms deserve.
Terminology
Abstract
A deception detector is only as useful as the trust its alarms deserve. Like a polygraph, detectors that read a language model's internal activations can catch deceptive responses that look honest. Yet current detectors depend on which layers they read, weaken on broader tests, and do not state how certain their alarms are. This white paper proposes the Machine Deception Detector, which targets both detection accuracy and the uncertainty of detector scores. The detector reads every layer of the language model, learns which tokens carry evidence of deception using only response labels, and calibrates its alarm threshold so that its false--positive rate stays below a prescribed level. An ensemble of such detectors measures the uncertainty of each score and raises an alarm only when its members agree, with a confidence level that sets how certain each alarm must be. The proposed evaluation trains on instructed honest and deceptive statements, tests on five unseen deceptive tasks of Liars' Bench read by six Qwen3 models from 0.6B to 32B parameters, and is designed to also show which layers carry the deception feature and how this changes with model size.
Sources
- Understanding intermediate layers using linear classifier probes
- OpenAI o1 System Card
- Diverse and Adaptable Arm Coordination for Octopus-Crawling via Diffusion-Based Uncertainty-Aware Optimization
- Liars' Bench: Evaluating Lie Detectors for Language Models
- One Probe Won't Catch Them All: Towards Targeted Deception Detection
- Linear Probe Accuracy Scales with Model Size and Benefits from Multi-Layer Ensembling
- The Linear Representation Hypothesis and the Geometry of Large Language Models
- The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes
- Qwen3 Technical Report
- Representation Engineering: A Top-Down Approach to AI Transparency
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks