Training-Free versus Training-Based Intent Classification in LLMs: Accuracy, Robustness, and Failure Modes

arXiv:2608.02415 · cs.CL · Submitted 2026-08-03 · Read on arXiv

Nan Chen, Zhouhao Yang, Soufiane Hayou

cs.CL

Submitted: 2026-08-03

Comments: Accepted at the Conference on Language Modeling (COLM 2026)

Code: https://github.com/Zhouhao-Yang/Training-Free-versus-Training-Based-Intent-Classification-in-LLMs

License: http://creativecommons.org/licenses/by/4.0/

The gist: Intent classification in Large Language Models (LLMs) involves categorizing user prompts into predefined classes.

Terminology

Abstract

Intent classification in Large Language Models (LLMs) involves categorizing user prompts into predefined classes. For instance, given a user prompt, the system must determine whether it primarily concerns mathematics, coding, or general text processing. Such classification enables routing prompts to specialized models optimized for specific domains, improving both accuracy and computational efficiency. In this work, we conduct a systematic study comparing training-free vs training-based approaches for intent classification. For this purpose, we consider two lightweight, training-free methods based on statistics of internal representations and compare them against MLP classifiers and linear probes. Our comprehensive empirical evaluation reveals that 1) Both training-free and training-based methods saturate easy benchmarks (mathematics vs. coding vs. natural language), 2) Training-based classifiers have an advantage on harder classification tasks (e.g. Java vs Python), and 3) Training-free methods are generally more robust to mixed-intent and adversarial prompts.

Sources

Related papers