DeepThink: A Conceptual Model for Knowledge-Addressable Recurrent Language Modeling
cs.CL
Submitted: 2026-10-12
Updated: 2026-10-12
Comments: 31 pages,7 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: This manuscript presents DeepThink, a knowledge-addressable framework for recurrent language modeling.
Abstract
This manuscript presents DeepThink, a knowledge-addressable framework for recurrent language modeling. A Knowledge Bank stores typed Knowledge Units, a Knowledge Activator selects units from the current execution state, Knowledge Assembly produces an executable configuration, and a Shared Thinking Operator updates that state. An output stage then organizes generation. This organization exposes knowledge capacity, activation width, and recurrent depth as separate design variables, supporting both complete parameter packages and fine-grained expert or low-rank modules. We formalize the interfaces and their execution semantics, including state-dependent selection, parameter-space assembly, and thinking followed by generation. For compatible templates and complete runtime-state configurations, we establish an exact Fixed-Schedule Realization of an existing fixed-depth model. Exact shared-base reparameterization and pretrained output-tail reuse provide complementary initialization constructions. Together, these results define a route from pretrained models to dynamically selected and assembled recurrent computation. A synthesis of conditional computation, recurrent models, modular parameters, and knowledge interfaces positions DeepThink within the research landscape and identifies the design questions arising from their integration.
Sources
- Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA
- Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- MoEUT: Mixture-of-Experts Universal Transformers
- DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
- Universal Transformers
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- Adaptive Computation Time for Recurrent Neural Networks
- HyperNetworks
- Training Large Language Models to Reason in a Continuous Latent Space
- Mixture of A Million Experts
- Distilling the Knowledge in a Neural Network
- Large Memory Layers with Product Keys
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
- Fixed Universal Transformers
- Soft Merging of Experts with Adaptive Routing
- MoRE: Mixture of Reused Experts
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- Attention Is All You Need
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering