A Sovereign, Indic-First, Reasoning Model
cs.CL, cs.AI
Submitted: 2026-10-11
Updated: 2026-10-11
Comments: 69 pages, 24 figures, 49 tables. Technical Report
Code: https://github.com/NVIDIA/Mode
License: http://creativecommons.org/licenses/by/4.0/
The gist: Gnani-Evon-v3.3-30B-A3B is a long-context hybrid Mamba-Transformer mixture-of-experts model for reasoning, instruction following, and tool use in English and eight Indian languages.
Abstract
Gnani-Evon-v3.3-30B-A3B is a long-context hybrid Mamba-Transformer mixture-of-experts model for reasoning, instruction following, and tool use in English and eight Indian languages. Starting from NVIDIA's Nemotron-3-Nano-30B-A3B checkpoint, it retains the 52-layer backbone and changes the vocabulary, training data, and post-training objectives. An in-place expansion from 131,072 to 163,072 tokens reduces mean Indian-language fertility by 38.6%. The new embedding rows are initialized from their upstream decompositions and warmed before the backbone is updated. Continual pretraining then adds broad English-Indic data, high-quality annealing, and direct training at a 131,072-token sequence length. The central result appears after post-training. Reinforcement learning across four verifiable task environments improves mathematics, graduate-level science, and instruction following, but degrades Indic extractive question answering because the rewards do not require an answer in the requested language. The share of Indic extraction items answered in English rises from 8.7% to 91.7%. Accuracy conditional on answering in the target script roughly doubles, which is consistent with preserved reading ability but is measured on a self-selected subset. Two unreleased studies test repairs. A scoped Indic reward restores the language convention. A 26B-A4B generative judge outperforms a 235B-parameter model on RM-Bench, and distillation restores 81% of the capabilities displaced by optimizing against it. Relative to the architecture-identical starting checkpoint, the released model gains 5.2 points on an Indic aggregate while giving up 1.7 points of English retention. The unreleased Indic-aware branch trades further, gaining 14.6 Indic points at a cost of 5.4 in English. A planned FP8 release reduces the artifact to 32GB with near-zero mean change across 32 paired benchmarks.
Sources
- A Sovereign, Open-Source Foundation Model for German and English
- In-Place Tokenizer Expansion for Pre-trained LLMs
- The Art of Breaking Words: Rethinking Multilingual Tokenizer Design
- DataComp-LM: In search of the next generation of training sets for language models
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- IndicIFEval: A Benchmark for Verifiable Instruction-Following Evaluation in 14 Indic Languages
- Language Models are Multilingual Chain-of-Thought Reasoners
- Measuring Massive Multitask Language Understanding
- Measuring Mathematical Problem Solving With the MATH Dataset
- Training Verifiers to Solve Math Word Problems
- GPQA: A Graduate-Level Google-Proof Q&A Benchmark
- Evaluating Large Language Models Trained on Code
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- Instruction-Following Evaluation for Large Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering