Exemplars in Disguise: Pure Exemplar Models Mimic Abstraction-First Learning

arXiv:2608.00821 · cs.CL · Submitted 2026-08-01 · Read on arXiv

cs.CL

Submitted: 2026-08-01

Updated: 2026-10-03

Code: https://github.com/znhoughton/exemplar-abstraction-sims

License: http://creativecommons.org/licenses/by/4.0/

The gist: Whether idiosyncratic, item-specific knowledge is learned before abstract class-level generalizations, or vice versa, is a central question in language learning, with exemplar and abstraction-based

Terminology

Abstract

Whether idiosyncratic, item-specific knowledge is learned before abstract class-level generalizations, or vice versa, is a central question in language learning, with exemplar and abstraction-based theories making opposite predictions. Recent methods have claimed to show that, at least for large language models, abstract knowledge is learned first. We show that these methods fall short: pure memorizer models with no abstract representations can appear, by the same criteria, to learn either item-specific or class-level knowledge first, depending on their sensitivity to individual observations, with the transition point governed by the distributional properties of the input. We further argue that the distinction between item-specific and abstract knowledge may be ill-defined for distributed representations, as a word's class-level properties may not be separable from its item-specific properties.

Sources

Related papers