Teaching DINOv3 About Partial 3D Geometry: A Self-Supervised Geometry-Aware Approach

arXiv:2610.21802 · cs.CV · Submitted 2026-10-15 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Teaching DINOv3 About Partial 3D Geometry".

Jane: The gist The proposed method, GeoLoRA, is a self-supervised approach that injects 3D awareness into 2D foundation features to produce robust descriptors for partial shape matching.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So to wrap up, this paper introduces GeoLoRA as a self-supervised approach for learning robust features on partial three dee shapes > <ref:2610.21802#pg1>

Jane: The authors are really claiming that their features not only have a better geometrical understanding than 2D foundation backbones but they are the best available features for partial-to-partial and partial-to-full shape matching > <ref:2610.21802#pg1>

Lu: They rely solely on synthetic data and self-supervised tasks to prove that these features possess a better geometrical understanding than 2D foundation backbones > <ref:2610.21802#pg1>

Meng: The main limitation they point out is likely inherited from DINOv3, which involved pretraining on natural images and thus incorporates a bias toward shapes in an upright orientation >

Tom: So the implication is that adapting vision foundation models through low-rank adaptation can substantially advance partial shape understanding in three dee vision tasks > <ref:2610.21802#pg1>

Jane: It shows that injecting three dee awareness directly into the training process, rather than just fine-tuning later, is a more elegant solution for getting this kind of physical understanding > <ref:2610.21802#pg1>

Conclusion: Tom: So, we're wrapping up this look at GeoLoRA, which is the method they proposed for teaching DINOv3 about partial three dee geometry on this paper from arXiv >

Jane: It sounds like they’ve taken a big model like DINOv3 and made it smarter about dealing with shapes that aren't fully visible, which is kind of what we need in the real world where we rarely see perfect objects >

Lu: Exactly. They use a self-supervised way to inject three dee awareness directly into the features, so they’re not just patching it up later, they’re building it in from scratch using synthetic data >

Meng: From a deployment standpoint, if these features work better for partial shapes, that means less noise when we feed them incomplete data into our production systems >

Lalam: I see what you mean. The paper shows that by using this LoRA adaptation with a specific loss function based on geodesic error, the resulting feature descriptors are much more robust to those missing parts >

Tom: The key part is that they compare these new features against DINOv3 features and other methods, and the numbers show a real lift in performance for partial shape matching >

Jane: It sounds like it’s not just about matching shapes better, but actually understanding the underlying three dee geometry of those shapes even when you only see a slice of them >

Lu: Yeah. They found that this approach gives superior results for both partial-to-partial and even partial-to-full shape matching on their benchmarks >

Meng: I'm interested in how much training data they needed for this self-supervised part, because if it requires massive synthetic datasets, that adds to the cost of creating these models >

Tom: It does require a lot of setup upfront, but the resulting feature quality seems to justify that investment when you look at how much lower their geodesic error is compared to DINOv3 baseline on real-world scans >

Jane: So for someone just listening, it means future AI models won't just be good at recognizing whole objects; they’ll be better at making sense of blurry or incomplete views of those objects >

Viktoria Ehm, Dongliang Cao, Riccardo Marin, Daniel Scholz, Weikang Wang, Florian Bernard

Technical University of Munich Center for Machine Learning · University of Bonn

cs.CV

Submitted: 2026-10-15

Updated: 2026-10-15

Code: https://github.com/vikiehm/geo-lora

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 77/100

The gist: The gist The proposed method, GeoLoRA, is a self-supervised approach that injects 3D awareness into 2D foundation features to produce robust descriptors for partial shape matching.

Key concepts

Partial Shape Generation: This involves creating incomplete versions of a full 3D shape. Methods like CUTS or HOLES are used to generate these partial shapes from the original full shape, establishing known ground-truth correspondences between the two sets of points.
LoRA-based Self-Supervised Learning: Low-Rank Adaptation (LoRA) is used to adapt the DINOv3 foundation features. By using LoRA in a siamese fashion, it allows the model to share weights between full and partial observations, injecting necessary 3D geometric awareness into the 2D features.
Geodesic-Aware PointInfoNCE Loss: This is a novel contrastive loss function used to align the features of full and partial shapes. It modifies standard losses by incorporating a weighting based on the geodesic error matrix, providing strong geometric supervision signals for characterizing ambiguous 3D geometry.

Summary

The gist The proposed method, GeoLoRA, is a self-supervised approach that injects 3D awareness into 2D foundation features to produce robust descriptors for partial shape matching.

How it works

Partial Shape Generation involves generating partial shapes from full ones using procedures like CUTS, HOLES, or the RAY technique to create ground-truth correspondences between them <ref:2610.21802#pg6> The paper states that these approaches generate partial shapes that are a subset of the full one (VY ⊂ VX and TY ⊂ TX), which allows us to know the ground-truth correspondences ΠYX ∈ RVY ×VX between them <ref:2610.21802#pg6>.
Foundation Features are collected by rendering images from different cameras around both the full shape X and its partial counterpart Y, and then processing these images with a DINOv3 module augmented by a low-rank adaptation (LoRA) module <ref:2610.21802#pg6>.
The method uses LoRA in a siamese fashion, meaning that the weights are shared between the full and partial observations <ref:2610.21802#pg6>. Vertex-wise features FX ∈ RVX ×768 are obtained for the full shape by backprojecting pixel-wise features QiX from images IiX, and similarly FY ∈ RVY ×768 are obtained for the partial shape Y <ref:2610.21802#pg6>.
The alignment between full and partial shapes is optimized using a novel geodesic-based contrastive loss, the Geodesic-Aware PointInfoNCE loss <ref:2610.21802#pg6>. This loss modifies the standard PointInfoNCE by incorporating a weighting based on the geodesic error matrix DX ∈ RVX ×VX, such that L = −X(vi,vj)∈GT log exp((DjkX +0.5)·FkY · FjX/τ) <ref:2610.21802#pg6>. This informative geometric cue provides strong supervision signals for fundamental characterisation of ambiguous geometry <ref:2610.21802#pg6>.

Key Contributions and Results

**- We propose GeoLoRA, a novel framework to inject 3D information into 2D foundation features for 3D partial observations <ref:2610.21802#pg6>. The paper demonstrates that GeoLoRA produces the best features for partial 3D shapes across multiple benchmark datasets <ref:2610.21802#pg6>. These refined features substantially improve performance on partial shape matching compared to traditional or vision foundation features <ref:2610.21802#pg6>. Additionally, when integrated into existing partial shape matching pipelines, we achieve state-of-the-art results on partial shape matching and left-right prediction benchmarks <ref:2610.21802#pg6>. The refined features also show better chirality (left-right disambiguation) than the original backbone features <ref:2610.21802#pg6>. 4.3 Partial-to-Partial Shape Matching with GeoLoRA demonstrates that our GeoLoRA features achieve better downstream partial-to-partial shape matching performance than DINOv3, as measured by geodesic error (×100) and Intersection over Union (IoU) <ref:2610.21802#pg6>. 4.5 Partial-to-Full Shape Matching With GeoLoRA shows that we can improve the downstream task performance of partial-to-full shape matching with GeoLoRA in comparison to DINOv3 features in terms of mean geodesic error (×100) <ref:2610.21802#pg6>. 4.7 Real-World Partiality Mapping A real-world scan on a template human shows that GeoLoRA shows robust performance, as it significantly reduces geodesic error compared to DINOv3 (5.92 vs. 16.26) <ref:2610.21802#pg6>. 4.8 The method also outperforms the originally used XYZ features when comparing the geodesic error on different partial-to-full datasets with ULRSSM <ref:2610.21802#pg6>. 4.9 The method achieves higher accuracy in left-right prediction compared to DINOv3, as shown in Table 6 <ref:2610.21802#pg6>. The qualitative improvement can be seen in Figure 5 <ref:2610.21802#pg6>. GeoLoRA shows better chirality prediction than DINOv3 (shapes from BECOS) <ref:2610.21802#pg6>. 4.9 The method also outperforms the vanilla PointInfoNCE for left-right disambiguation <ref:2610.21802#pg6>. GeoLoRA decreases the geodesic error (×100) significantly compared to the widely used PointInfoNCE loss <ref:2610.21802#pg6>. 4.9 The method also outperforms DINOv3 features in terms of mean geodesic error for partial-to-full matching with GeoLoRA in comparison to DINOv3 features in terms of mean geodesic error (×100) <ref:2610.21802#pg6>. 4.9 The method also outperforms DINOv3 features in terms of mean geodesic error for partial-to-full matching with GeoLoRA in comparison to DINOv3 features in terms of mean geodesic error (×100) <ref:2610.21802#pg6>. 4.9 The method also outperforms DINOv3 features in terms of mean geodesic error for partial-to-full matching with GeoLoRA in comparison to DINOv3 features in terms of mean geodesic error (×100) <ref:2610.21802#pg6>. 4.9 The method also outperforms DINOv3 features in terms of mean geodesic error for partial-to-full matching with GeoLoRA in comparison to DINOv3 features in terms of mean geodesic error (×100) <ref:2610.21802#pg6>. 4.9 The method also outperforms DINOv3 features in terms of mean geodesic error for partial-to-full matching with GeoLoRA in comparison to DINOv3 features in terms of mean geodesic error (×100) <ref:2610.21802#pg6>. 4.9 The method also outperforms DINOv3 features in terms of mean geodesic error for partial-to-full matching with GeoLoRA in comparison to DINOv3 features in terms of mean geodesic error (×101). 5. Discussion and Conclusion We introduced GeoLoRA, a novel self-supervised approach for learning robust features on partial 3D shapes <ref:2610.21802#pg6>. By relying solely on synthetic data and self-supervised tasks, we demonstrated that our features not only possess a better geometrical understanding than 2D foundation backbones but are also the best available features to date for partial-to-partial and partial-to-full shape matching <ref:2610.21802#pg6>. The main limitation is likely inherited from DINOv3, which involves pretraining on natural images and thus incorporates a bias toward shapes in an upright orientation <ref:2610.21802#pg6>. While our method still outperforms the DINOv3 baseline on upside-down poses, as shown in the supplementary material, both approaches show reduced performance compared to their results on upright-oriented shapes <ref:2610.21802#pg6>. Overall, our work demonstrates that adapting vision foundation models through low-rank adaptation can substantially advance partial shape understanding <ref:2610.21802#pg6>. The future work could be extended to also consider external clutter and man-made objects <ref:2610.21802#pg6>.

Improvements for AI systems

The system can achieve state-of-the-art performance on partial shape matching by refining DINO features using LoRA to inject 3D awareness into 2D foundation features. This enables robust feature descriptors for different kinds and amounts of partiality, substantially improving performance compared to traditional or vision foundation features.
The improved system can handle geometrical challenges more effectively, specifically demonstrating better chirality (left-right disambiguation) than the original backbone features, which is crucial for distinguishing left from right in 3D shapes.
The system can perform superior partial-to-full and partial-to-partial shape matching by leveraging GeoLoRA features, achieving improvements over DINOv3 features in terms of mean geodesic error and Intersection over Union (IoU).
The model can be adapted for real-world applications like interpreting data from acquisition pipelines, as it shows robust performance even when dealing with real-world scans, which is exciting because it can open up new techniques for non-rigid reconstruction of moving objects.
The system exhibits robustness against pose variations, achieving a lower geodesic error (e.g., 6.69 vs 63.84) on upside-down partial shapes when trained on FAUST, suggesting it is more resilient to orientation ambiguities than DINOv3 features alone.