Gluing Local Contexts into Global Meaning: A Sheaf-Theoretic Decomposition of Transformer Representations
cs.CL, cs.LG
Submitted: 2026-10-12
Updated: 2026-10-12
Comments: NeurIPS 2026 Poster, ICLR Unifying Concept Representation Learning
Code: https://github.com/CWRU-AISM/gluing-lc
Project page: https://cwru-aism.github.io/gluing-lc-page
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Sheaf Neural Networks with Connection Laplacians
- Sheaves, Cosheaves and Applications
- Learning Sheaf Laplacian Optimizing Restriction Maps
- Sheaf Hypergraph Networks
- Toy Models of Superposition
- Gemma 2: Improving Open Language Models at a Practical Size
- The Llama 3 Herd of Models
- Linearity of Relation Decoding in Transformer Language Models
- A Unified Understanding and Evaluation of Steering Methods
- A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations
- Mistral 7B
- Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Models
- Textbooks Are All You Need II: phi-1.5 technical report
- Directional Sheaf Hypergraph Networks: Unifying Learning on Directed and Undirected Hypergraphs
- 2 OLMo 2 Furious
- Sheaf Cohomology of Linear Predictive Coding Networks
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Steering Language Models With Activation Engineering
- Representation Learning with Contrastive Predictive Coding
- Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering