MJev-Spatial: System One Spatial Intelligence via Native Attention Matching

The gist MJev-Spatial introduces a System 1 multimodal decision model that unifies spatial understanding tasks—option selection, numerical estimation, and image pointing—into a single classification framework using pretrained attention projections without adding parameters.

How it works

MJev-Spatial treats each candidate as a class and scores it using the backbone’s pretrained query and key projections, without adding parameters The model predicts by selecting candidates rather than generating text. This unified classification framework casts prediction as classification over finite candidate sets where each class represents a supplied answer option, a numerical bin, or an image region depending on the task.

The inference process involves multiple stages depending on the task type

1. For choice tasks, it selects one class which gives the chosen option.

2. For numerical prediction, it uses up to two stages; single-point prediction uses three stages, re-encoding the selected image crop before each later decision.

3. Point prediction involves three nine-way region decisions with crop reencoding between stages.

Model Architecture

The model is built on Qwen3.5-4B and leverages the final attention layer’s pretrained query and key projections to compute class scores, which are then passed through a softmax to get class probabilities without additional prediction heads or trainable parameters. The scoring mechanism involves computing logits by taking the mean scaled inner product of the decision state query and each candidate key. This allows the model to reuse existing parameters for candidate scoring without adding a prediction head or new trainable parameters.

Training Recipe

To train efficiently on heterogeneous spatial data, MJev-Spatial supervises all coarse-to-fine stages jointly with teacher forcing in one forward pass. This is combined with length-aware global batching and cross-GPU load balancing, which raises training throughput by 30.8% over the original batching strategy. The training data consists of 12.3M question–answer pairs converted from existing spatial datasets and includes multiple task types such as choice, point localization, length estimation, angle estimation, and counting.

Performance and Efficiency

MJev-Spatial achieves the highest overall score (65.64%) among eight tested models and ranks first on seven of twelve benchmarks. It outperforms its unmodified autoregressive base model on all twelve benchmarks, while running 2.5× faster than the fastest autoregressive baseline and 5.6× faster than the base model. Across all twelve benchmarks, it achieves 2.5× and 5.6× speedups over the fastest autoregressive baseline and the base model, respectively. Its mean batch-one query latency is 160.6 ms across the twelve benchmarks, giving 2.5× and 5.6× speedups over the fastest autoregressive baseline and a 5.6× speedup over its base model under the reported timing protocol.

Key Findings in Ablation Studies

Ablation studies show that native Q/K scoring achieves the best RefSpatial accuracy and lowest length-estimation MAE, while the position classifier performs better on Where2Place and RoboSpatial pointing. For numerical prediction, using a 4/8 fine-bin design reduces VST-300 MAE from 0.3277 to 0.3202 m, showing that finer representational precision does not necessarily translate into better learned predictions. Visual refinement experiments show that crop re-encoding achieves the highest RefSpatial and Point-Bench scores among the four visual strategies. Furthermore, multi-point decoding can be extended to evaluate multiple targets without additional training by expanding multiple paths and assigning each final cell the product of its conditional probabilities along the path.

The final Qwen3.5-4B-based model achieves the highest overall score among eight evaluated models, leads on seven benchmarks, and exceeds its local autoregressive base model on all twelve.

1. Bold header: System 1 Decision Paradigm Implementation

MJev-Spatial unifies these three output types by casting prediction as classification over finite candidate sets, enabling fast-thinking decisions with high accuracy and low inference latency. This allows the system to perform spatial relation selection, numerical estimation (e.g., distance), and point prediction in a single forward pass without generating answer tokens or requiring text parsing overhead.

2. Bold header: Unified Training Recipe for Heterogeneous Data

The system employs joint supervision across spatial tasks and refinement stages by training all coarse-to-fine stages jointly with teacher forcing in one forward pass, which significantly raises training throughput by 30.8% over the original batching strategy. This enables efficient learning from large, heterogeneous datasets like the 12.3M QA corpus containing choice, point, and length estimation tasks simultaneously.

3. Bold header: Optimized Inference Latency via Native Query/Key Projections

The system leverages native Q/K scoring by reusing the final attention layer’s pretrained query and key projections, which allows each classification decision to take a single forward pass and generate no answer tokens. This results in an inference latency of 160.6 ms, achieving a 2.5× speedup over the fastest autoregressive baseline.

4. Bold header: Task-Specific Refinement via Coarse-to-Fine Classification

For numerical and point prediction, the system uses a multi-stage process where numerical prediction uses up to two stages; single-point prediction uses three stages, with new image crops introduced between stages for refinement. This allows the model to achieve high precision while maintaining a unified classification framework across all output types.

5. Bold header: Robust Multi-Point Prediction via Probability Tree Expansion

The system extends point prediction using a probability tree approach, where it evaluates multiple spatial paths and assign[s] each final cell the product of its conditional probabilities along the path. This allows for multiple points prediction without additional training, enabling the model to recover several locations from a single trained model.

Spatial intelligence lets an agent perceive and act in the physical world, and many of its judgments must be made at interaction rates: which spatial relation holds, how far an object is, or where to place it. Vision-language models (VLMs) produce these answers by free-form text generation, paying for autoregressive decoding and answer parsing even though the output space is compact: one of a few options, a scalar, or a 2D point. In light of this, we introduce MJev-Spatial, a System 1 multimodal decision model that tackles all three types of spatial understanding tasks within a unified classification framework. Specifically, MJev-Spatial treats each candidate as a class and scores it using the backbone's pretrained query and key projections, without adding parameters. The candidates are the given options for choice tasks, or value bins and image regions refined through coarse-to-fine classification for numerical and point prediction, respectively. At inference, each classification decision takes a single forward pass and generates no answer tokens. To train on heterogeneous spatial data efficiently, we supervise all coarse-to-fine stages jointly with teacher forcing in one forward pass, and combine this with length-aware global batching and cross-GPU load balancing, raising training throughput by 30.8% over the original batching strategy. Built on Qwen3.5-4B and trained on 12.3M question-answer pairs converted from existing spatial datasets, MJev-Spatial achieves the highest overall score (65.64%) among eight tested models, ranks first on seven of twelve benchmarks, and outperforms its unmodified autoregressive base model on all twelve, while running 2.5 times faster than the fastest autoregressive baseline and 5.6 times faster than the base model.