What Makes a Latent Thought Valid? Defining and Auditing Thought Representations in LLMs

The synthesis below integrates their findings into a comprehensive, detailed description of the paper's core contribution, methodology, and conclusions.

Comprehensive Research Synthesis: Auditing Latent Thought Representations via Axiomatic and Geometric Frameworks

This body of research introduces rigorous frameworks for defining, evaluating, and auditing the quality of latent thought representations generated by Large Language Models (LLMs). The central thesis is that existing downstream benchmark scores are insufficient; they conflate representation quality with model capacity or training procedure. The authors argue that true representational failures—the inability of a thought representation to capture the necessary information about a problem—are structural properties inherent to the representation itself, independent of the specific LLM or its downstream performance.

The work presents two complementary, yet distinct, methodologies: an Axiomatic Evaluation Framework and a Geometric Similarity Scale Analysis.

I. The Axiomatic Evaluation Framework (What Makes a Latent Thought Valid?)

The primary contribution here is the formalization of thought representations through four functional axioms designed to measure intrinsic representational properties rather than relying on external performance metrics. This framework allows researchers to diagnose why a representation might fail, attributing the failure to a specific representational property rather than an aggregate benchmark score.

Axioms and Quantitative Measures:

1. Causality: Measured by KL substitution error. A lower value indicates that the thought representation (T) effectively encapsulates the effect of the input prompt (y pre) on subsequent generation, suggesting a causal link to the desired outcome.

2. Minimality: Measured by the IB residual gap (CE), defined as CE(X T) - CE(Y T). A higher gap signifies a representation that is highly efficient: it compresses the input information (X) while retaining only the output-relevant information (Y), adding minimal extraneous noise.

3. Separability: Measured by Discriminator accuracy. This requires distinct inputs to yield resolvable representations under a bounded hypothesis class H with a margin delta. This tests whether different problems are encoded into sufficiently distinct latent spaces.

4. Stability: Measured by the Distributional Consistency Score (DCS), which quantifies the entropy of P(Sx), reflecting robustness against lexical variations and stochastic sampling noise.

Empirical Audit and Findings:

The framework was applied to audit candidate thought representations extracted from various LLMs (including Soft Thinking with/without Gumbel noise, Latent Thinking variants, and last-input-token hidden states) across 23 diverse reasoning tasks (e.g., Spatial Reasoning, Factual QA). The key empirical findings are:

• Failure of Discrimination: No candidate representation satisfies all four axioms simultaneously. Crucially, the representations reliably distinguish between different task types but fail to distinguish between two questions within the same task. This confirms that a model can achieve high downstream accuracy while its internal thought process collapses at the per-question identity level.

• Information Compression: The audit reveals that candidate representations encode very little information beyond what is already present in the input embedding, suggesting a severe representational collapse.

• Structural Gap: The consistent failure across dense, reasoning-distilled, and RL-trained model families indicates that the gap is structural to how LLMs generate thought representations, rather than being an artifact of model size or specific training procedures.

• Diagnostic Power: The framework serves as a principled diagnostic tool: researchers can score candidates on each axiom independently to pinpoint the exact representational property causing performance degradation, allowing them to identify the binding constraint before undertaking costly retraining efforts.

II. Geometric Similarity Scale Analysis (Anisotropy-Adjusted Similarity)

This complementary analysis focuses on geometric properties of the latent space itself, using an anisotropy-adjusted similarity scale to diagnose within-task separability failures.

Key Diagnostic:

The authors introduce = within-task - cross-task, where cross-task serves as the anisotropy baseline inherent to self-attention mechanisms. This subtraction isolates the within-task scale beyond what standard transformer geometry alone produces, focusing on the scale of similarity rather than just cluster structure.

Findings:

• Geometric Constraint: The analysis demonstrates that while latent thought representations possess cross-task discrimination capability, their within-task geometry is inherently collapsed. This collapse manifests as either a narrow subspace (low PR) or noise drift (low purity).

• Failure Confirmation: The failure to discriminate within a task is confirmed geometrically: every candidate, except the direct Output Embedding, exhibits low PR or low purity in the relevant panels.

• Robustness Across Models: This geometric verdict holds across all tested LLMs and does not depend on a trained discriminator. Furthermore, probe capacity ablation experiments confirm that neither Last Input Token nor Soft Thinking can distinguish between multiple problems when conditioned on T, reinforcing the idea that this is a property of the latent computation itself.

• Minimality Confirmation: The minimality formulation (CE) is also examined, showing that direct thought representations do not produce a more compressed, output-relevant summary than simply embedding the input prompt.

Conclusion and Overall Contribution

The combined findings from both frameworks converge on a powerful conclusion: The failure to discriminate between questions within the same task is an intrinsic property of the geometry produced by latent computation, not an artifact of any single LLM or its downstream task difficulty.

The overarching contribution of this research is twofold:

1. Axiomatic Formalization: Providing a rigorous, logically consistent definition of thought representations based on four explicit functional axioms (Causality, Minimality, Separability, Stability). This moves the study of thought from subjective evaluation to objective property measurement.

2. Principled Measurement Protocol: Establishing a measurement protocol that exposes representational gaps at the representation level without requiring downstream benchmarks or retraining of the source model. This allows researchers to identify which specific axiom is the binding constraint on an existing representation, offering a direct, decomposable path for future thought representation development and auditing.

1. Axiomatic Thought Representation Evaluation: Implement a new evaluation protocol that audits latent thought representations using four functional axioms (Causality, Minimality, Separability, and Stability) directly on the source LLM without retraining. This allows researchers to attribute downstream accuracy changes to specific representational properties rather than model size or training procedure.

2. Constraint-Aware Thought Extraction: Develop a method for extracting candidate thought representations that is explicitly optimized to satisfy the four axioms simultaneously. This system can identify which axiom is the binding constraint on an existing representation before any retraining is undertaken and compare competing representations directly on a four-metric profile rather than a single accuracy number.

3. Representation Quality Diagnostics: Create diagnostic readouts for thought representations based on the measured axioms, such as KL substitution error for Causality, the Minimality Gap measure for Minimality, same- and cross-task discriminator accuracy for Separability, and the Distributional Consistency Score (DCS) for Stability. This provides explicit optimization targets and a diagnostic resolution to attribute downstream accuracy changes to specific representational properties.

4. Structural Collapse Identification: The system can identify fine-grained representational collapse on per-question identity that is masked by aggregate benchmark scores. By measuring the PR (Within-task participation ratio) and k-NN task purity of thought vectors, it can reveal whether representations distinguish task type reliably but cannot distinguish between two questions within the same task.

Chain-of-Thought (CoT) reasoning improves the performance of Large Language Models (LLMs), yet it writes steps as text and follows one reasoning path at a time, which limits the capabilities of such models. Latent reasoning methods replace these steps with continuous thought representations and claim that a thought representation substitutes for the written reasoning, compresses trajectories, distinguishes semantic content, and achieves superposition of multiple exploration paths at once. These claims concern the content of the thought representation, whereas the evaluations of these methods rely on downstream task accuracy, token counts, and analyses specific to each method. We formalize these claims as a valid thought representation space and prove that a space is valid if and only if its thought representations attain four properties, namely causality, minimality, separability, and stability, each of which follows from one claim. Translating these properties into four measures and auditing three training-free and three trained methods shows gaps in these claims that accuracy may not necessarily reveal. Training-free methods fail all four properties and do not encode the reasoning they are meant to replace, whereas trained methods generally improve substitution. The four properties and measures therefore provide a theoretically grounded evaluation beyond downstream task accuracy and expose failure modes that may guide future development of latent reasoning methods.