PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling

arXiv:2606.25430 · cs.CV · Submitted 2026-06-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling".

Jane: Here's the gist of the paper. The gist: PRISM proposes a feed-forward framework that decomposes single-image 3D reconstruction into a parameter-free geometric prior and a learned residual correction, eliminating diffusion sampling at inference.

Tom: Reconstructing 3D scenes from a single image is a fundamental challenge in computer vision, with broad applications in virtual reality, robotics, and content creation <ref:2606.25430#pg2>.

Jane: Recent methods achieve outstanding performance by leveraging camera-controlled video diffusion models, but rely on iterative diffusion sampling, which greatly limits their practical deployment <ref:2606.25430#pg2>.

Tom: The authors observe that geometric forward warping alone can cover up to 90% of a target view directly from the input image, with only a compact residual left for the encoder to correct <ref:2606.25430#pg2> (Page 2).

Paper summary: Tom: So we're talking about PRISM today. The paper is titled "PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling," and it’s about figuring out how to build a complete three dee scene from just one image <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Jane: Right, and the big deal here is that it gets rid of those slow iterative diffusion sampling steps you see in some of the top video diffusion models. The authors claim they can achieve this using a feed-forward framework instead.

Lu: Basically, their thesis is that you don't need all that complex sampling at inference because geometric forward warping alone can cover up to ninety percent of the target view directly from the input image, leaving only a small residual for a learned encoder to fix > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg2> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Meng: So what does that mean for deployment? If you don't need iterative sampling, that’s a huge win because it means faster inference times, right? I’m really interested in how they handle the training side because getting it to work on real stuff is always the tricky part > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg3> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Lalam: From my perspective, the key mechanism they propose is decomposing the multi-view latent prediction into two parts: a parameter-free geometric prior and a learned residual correction, which is what makes it feed-forward > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg3> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Tom: Exactly. Tom here, focusing on the mechanics, they use this decomposition to replace the video diffusion model at inference with a lightweight residual encoder instead > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg2> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Jane: And how do they actually get that geometric prior? I'm trying to picture it without reading the math too much.

Lu: They first estimate monocular depth using MoGe, and then they use this estimated depth to perform z-buffered forward splatting directly in the Cosmos latent space to form the geometric prior > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg3> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Paper summary: Meng: That sounds like a lot of steps for inference. What’s the actual residual encoder doing? Is it just patching things up?

Tom: Not just patching, Meng. They have a lightweight transformer encoder, f theta, that takes those warped latents, validity masks, the reference latent, and Plucker ray embeddings as input to predict those residual corrections > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg4> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Jane: And how do they train this residual encoder if they don't have perfect multi-view supervision readily available? That’s a big hurdle for real world use, isn't it?

Lu: They address that with a two-stage training strategy, combining latents supervised distillation and perceptual fine-tuning > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg3> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Meng: Can you elaborate on those stages? What's the difference between supervised distillation and perceptual fine-tuning in this context? I need to know what kind of data they are using for each part > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg4> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Tom: For the distillation part, they supervise the encoder through the intermediate features of a frozen three deeGS decoder instead of just raw latent values, specifically targeting crossview attention layers to encode multiview geometric consistency > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg4> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Jane: So they're teaching the encoder to understand the geometry from how a frozen decoder processes features across different views, which makes sense for enforcing consistency > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg4> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Lu: Then, in the second stage, they do perceptual fine-tuning using losses like LPIPS and MSE on rendered images through the frozen three deeGS decoder to optimize appearance quality > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg4> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Meng: So it’s a two-step process, first learning the structure from features, then polishing the look from final renders. That sounds like they are trying to get the best of both worlds > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg4> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Paper summary: Lalam: And in terms of results, they show that this approach achieves reconstruction quality competitive with diffusion-based methods, and inference time is reduced to only thirty-six seconds per scene, which is a two hundred seventy-seven times speedup compared to LYRA > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg1> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Tom: That speedup figure is significant. And they mention that on the harder T andT benchmark, PRISM outperforms LYRA across all three metrics, which is a strong indicator of its performance > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg7> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Jane: That leads us to what this means for someone who only listens to the show. It means that for things like virtual reality or robotics where you need fast reconstruction, you could potentially use a method that’s much faster than the current diffusion-based ones, even if it doesn't quite match their absolute peak fidelity yet > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg1> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Lu: And one thing they point out is that the warp-residual decomposition generalizes effectively beyond the synthetic training domain, which suggests it might actually work better on real scenes than expected > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg9> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Meng: But what about limitations? If you’re deploying this on a production system, what’s the catch? What doesn't this method do well?

Tom: The paper notes that removing the residual encoder drops the PSNR by one point seven seven dB and LPIPS goes up from zero point two four two to zero point four five seven, which shows that just relying on the geometric warp prior alone isn't enough to recover fine-grained appearance detail > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg8>.

Jane: So the residual encoder is absolutely necessary for that high fidelity, meaning it’s not just a neat trick but a required part of the architecture if you want good visuals > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg8> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Lu: It also mentions that the warp coverage and PSNR have a positive correlation, with r equals zero point six five, and scene image gradient magnitude correlates negatively with mean PSNR, with r equals negative zero point eight two > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg8>.

Paper summary: Meng: That gradient correlation is interesting because it tells you that scenes with sharper edges or more contrast are better reconstructed than blurry ones, which is something we have to consider when building our pipeline > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg8> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Lalam: Overall, the PRISM paper shows that diffusion sampling isn't a prerequisite for high quality novel view synthesis, and it separates what geometry can already explain from what remains uncertain, allowing that lightweight feed-forward encoder to replace iterative diffusion sampling entirely > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg5> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Tom: It’s a clean way to separate the geometric structure from the texture detail, and that separation is what makes this approach so efficient > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg5> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Jane: So if you’re looking at future work, where do you think this goes next? Are there any other areas where we can apply the idea of geometric prior and learned residual correction?

Lu: I think the direction for future work is exploring how to make that two-stage training strategy even more robust so it can generalize even further to completely unseen real-world environments > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg9> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Meng: From a deployment standpoint, I’m interested in making sure the residual encoder is as small and fast as possible so it doesn't bottleneck our actual serving infrastructure > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg8> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Tom: That’s the engineering reality, Meng. The paper shows that this feed-forward approach matches the speed of geometry-based reconstruction while keeping it competitive with diffusion methods in fidelity > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg1> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Jane: So to wrap up, PRISM is a feed-forward framework that uses geometric warp and a learned residual correction to reconstruct three dee scenes from a single image, doing it without diffusion sampling at inference > PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg1> <ref:2606.25430#pg1,PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling>.

Conclusion: Tom: So we’re wrapping up our look at PRISM, which is titled "PRISM: Feed-Forward Single-Image three dee Reconstruction via Geometric Warp-Residual Modeling."

Jane: Right, and the authors are working on making single image three dee reconstruction much faster by ditching iterative diffusion sampling.

Lu: The core idea is splitting the problem into a parameter-free geometric prior and a learned residual correction.

Meng: So, what does that mean for deployment? Does this actually make it fast enough to run on our actual infrastructure?

Tom: It’s about replacing the slow video diffusion model at inference with just a lightweight residual encoder.

Jane: And how do they handle training if they don't have perfect multi-view supervision from the start?

Lu: They use a two-stage training strategy, first latents supervised distillation, and then perceptual fine-tuning for appearance quality.

Meng: Can you explain what that means in practice? What’s the difference between those two stages for a production model?

Tom: For the distillation stage, they supervise the encoder through intermediate features of a frozen three deeGS decoder to encode geometric consistency.

Jane: And then the perceptual fine-tuning stage uses losses like LPIPS and MSE on rendered images to polish up how it looks.

Lu: That two-stage approach lets them train a model without any real-world supervision and still transfer well to real scenes.

Meng: So, what’s the final verdict on the paper? Is this method ready for anything beyond the synthetic training data they used?

Tom: Extensive experiments show PRISM achieves reconstruction quality competitive with diffusion methods while cutting inference time dramatically to only thirty-six seconds per scene.

Jane: That speedup is huge—a two hundred seventy-seven times improvement over a baseline method.

Lu: It also showed that the warp and residual decomposition generalizes effectively beyond the synthetic training domain they used.

Meng: What does this mean for us when we look at deploying models to real world data, like actual photos?

Tom: The authors found that removing the residual encoder drops PSNR by one point seven seven decibels and LPIPS goes up significantly, showing the learned correction is essential for fine detail.

Jane: So, it’s not just about getting a rough shape; you need that correction to get the texture right.

Lu: The conclusion is that PRISM separates what geometry can explain from what remains uncertain, allowing a feed-forward encoder to replace iterative sampling entirely.

Meng: This really boils down to separating structure from detail, which is exactly what we need for efficient deployment.

Tom: PRISM proves that diffusion sampling isn't a prerequisite for high quality novel view synthesis if you use this decomposition approach.

Jane: It’s a clean way to get the speed you want without sacrificing reconstruction fidelity.

Lu: We’ll be looking at how this geometric prior and residual correction idea can be applied to other areas of scene understanding next.

Zhijie Zheng, Xinhao Xiang, Jiawei Zhang

University of California, Davis

cs.CV

Submitted: 2026-06-24

Updated: 2026-10-17

Code: https://github.com/microjie372/PRISM

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: The gist: PRISM proposes a feed-forward framework that decomposes single-image 3D reconstruction into a parameter-free geometric prior and a learned residual correction, eliminating diffusion

Key concepts

Geometric Decomposition
This step splits the 3D prediction into two parts: a fixed geometric prior (z v blend,t) and a learned correction ($\Delta$v $\theta$,t). The geometric part handles the large-scale structure based on camera geometry, while the residual part learns to fix small errors that the geometry alone cannot capture.
Geometry-guided Forward Warping
This involves using estimated depth and reference latents to perform a forward splatting operation directly in latent space. This process creates a geometrically consistent prior ($z_{ref}$) that covers most of the target view, significantly reducing the work needed for the subsequent correction step.
Residual Encoder
A lightweight transformer encoder takes warped latents and other geometric information to predict small residual corrections ($\Delta$v $\theta$,t). This network learns how to refine the geometrically predicted latent, ensuring fine details and appearance quality are corrected without needing iterative sampling.

Terminology

Summary

The gist: PRISM proposes a feed-forward framework that decomposes single-image 3D reconstruction into a parameter-free geometric prior and a learned residual correction, eliminating diffusion sampling at inference.

Motivation and Problem

Reconstructing 3D scenes from a single image is a fundamental challenge in computer vision, with broad applications in virtual reality, robotics, and content creation <ref:2606.25430#pg2>. Recent methods achieve outstanding performance by leveraging camera-controlled video diffusion models, but rely on iterative diffusion sampling, which greatly limits their practical deployment <ref:2606.25430#pg2>. The authors observe that geometric forward warping alone can cover up to 90% of a target view directly from the input image, with only a compact residual left for the encoder to correct <ref:2606.25430#pg2> (Page 2). This observation motivates PRISM, which replaces the video diffusion model at inference with a lightweight residual encoder <ref:2606.25430#pg2> (Page 2).

PRISM Architecture

PRISM decomposes multi-view latent prediction into a parameter-free geometric prior and a learned residual correction <ref:2606.25430#pg2> (Page 2). The predicted latents are decoded by a frozen 3DGS decoder into a complete 3D scene, with no diffusion sampling at inference <ref:2606.25430#pg2> (Page 2).

The core of the method involves several key components:

  1. Geometric decomposition: The prediction is decomposed into a parameter-free geometric prior z v blend,t and a learned residual correction ∆v θ,t such that zˆ v t = z v blend,t + ∆v θ,t <ref:2606.25430#pg2> (Page 3).

  2. Geometry-guided forward warping: The reference latent is encoded with the Cosmos VAE to obtain zref = E(I) ∈ R C×h×w, and monocular depth D is estimated with MoGe [49] <ref:2606.25430#pg2> (Page 3). This process performs z-buffered forward splatting directly in the Cosmos latent space to form the geometric prior <ref:2606.25430#pg2> (Page 3).

  3. Residual encoder: A lightweight transformer encoder fθ takes the warped latents, validity masks, reference latent, and Plucker ray embeddings as input to predict residual corrections <ref:2606.25430#pg2> (Page 4). This joint processing is essential for geometric consistency <ref:2606.25430#pg2> (Page 4).

Training Strategy

To enable generalization from purely synthetic training data to real-world scenes, PRISM employs a two-stage training strategy <ref:2606.25430#pg2> (Page 3).

  1. Latents Supervised Distillation: The encoder is supervised through the intermediate features of the frozen 3DGS decoder D rather than raw latent values, targeting crossview attention layers to encode multiview geometric consistency <ref:2606.25430#pg2> (Page 4). This is achieved by maximizing the cosine similarity between predicted and ground truth features <ref:2606.25430#pg2> (Page 4).

  2. Perceptual Fine-tuning: The encoder is fine-tuned from the Stage 1 checkpoint with a perceptual loss computed on rendered images through the frozen 3DGS decoder, using losses like LPIPS and MSE <ref:2606.25430#pg2> (Page 4).

Experimental Results

Extensive experiments on three benchmarks demonstrate that PRISM achieves competitive reconstruction quality compared with diffusion-based methods, while reducing inference time dramatically to only 36 seconds per scene, a 277× speedup compared to LYRA [1] <ref:2606.25430#pg2> (Page 1). On the more challenging T&T benchmark, PRISM outperforms LYRA across all three metrics, the strongest result among all evaluated methods <ref:2606.25430#pg2> (Page 7). Ablation studies show that removing the residual encoder drops PSNR by 1.77 dB and LPIPS degrades from 0.242 to 0.457, revealing that the geometric warp prior alone cannot recover fine-grained appearance detail <ref:2606.25430#pg2> (Page 8).

Efficiency and Generalization

The average inference time of PRISM is 36.2 seconds per scene, with the 3DGS Decoder dominating at 35.04 seconds, accounting for 97% of total inference time <ref:2606.25430#pg2> (Page 8). The warp-residual decomposition shows that per-view warp coverage correlates positively with PSNR (r = 0.65), and scene image gradient magnitude correlates negatively with mean PSNR (r = −0.82) <ref:2606.25430#pg2> (Page 8). Qualitative visualizations show that PRISM produces renders that closely match LYRA across all three viewpoints, with consistent geometry, lighting, and texture appearance <ref:2606.25430#pg2> (Page 9). The results demonstrate that the warp-residual decomposition generalizes effectively beyond the synthetic training domain <ref:2606.25430#pg2> (Page 9).

Conclusion

PRISM presents a feed-forward framework for single-image 3D reconstruction that shows diffusion sampling is not a prerequisite for high-quality novel view synthesis <ref:2606.25430#pg2> (Page 5). PRISM separates what geometry can already explain from what remains uncertain, which allows a lightweight feed-forward encoder to replace iterative diffusion sampling entirely <ref:2606.25430#pg2> (Page 5). The two-stage training strategy enables a model trained without any real-world supervision to transfer to real-world scenes <ref:2606.25430#pg2> (Page 9). Consequently, our method matches the efficiency of geometry-based reconstruction while remaining competitive with diffusion-based methods in reconstruction fidelity >.

--- Page 1 ---

PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling <ref:2606.25430#pg2> (Page 1)

The gist: PRISM proposes a feed-forward framework that decomposes single-image 3D reconstruction into a parameter-free geometric prior and a learned residual correction, eliminating diffusion sampling at inference.

Motivation and Problem

Reconstructing 3D scenes from a single image is a fundamental challenge in computer vision, with broad applications in virtual reality, robotics, and content creation <ref:2606.25430#pg2> (Page 1). Recent methods achieve outstanding performance by leveraging camera-controlled video diffusion models, but rely on iterative diffusion sampling, which greatly limits their practical deployment <ref:2606.25430#pg2> (Page 1). The authors observe that geometric forward warping alone can cover up to 90% of a target view directly from the input image, with only a compact residual left for the encoder to correct <ref:2606.25430#pg2> (Page 2). This observation motivates PRISM, which replaces the video diffusion model at inference with a lightweight residual encoder <ref:2606.25430#pg2> (Page 2).

The core of the method involves several key components:

  1. Geometric decomposition: The prediction is decomposed into a parameter-free geometric prior z v blend,t and a learned residual correction ∆v θ,t such that zˆ v t = z v blend,t + ∆v θ,t <ref:2606.25430#pg2> (Page 3).

  2. Geometry-guided forward warping: The reference latent is encoded with the Cosmos VAE to obtain zref = E(I) ∈ R C×h×w, and monocular depth D is estimated with MoGe [49] <ref:2606.25430#pg2> (Page 3). This process performs z-buffered forward splatting directly in the Cosmos latent space to form the geometric prior <ref:2606.25430#pg2> (Page 3).

  3. Residual encoder: A lightweight transformer encoder fθ takes the warped latents, validity masks, reference latent, and Plucker ray embeddings as input to predict residual corrections <ref:2606.25430#pg2> (Page 4). This joint processing is essential for geometric consistency <ref:2606.

Improvements for AI systems

  1. Feed-forward Inference for Single-Image 3D Reconstruction The system can synthesize geometrically consistent novel views across diverse scenes without requiring iterative diffusion sampling at inference, achieving a 277× speedup over LYRA [1] and reducing inference time to just 36 seconds per scene.

  2. Geometric Prior Generation from Depth PRISM utilizes monocular depth estimation to create a parameter-free geometric prior via z-buffered forward splatting in the Cosmos latent space, which covers up to ∼90% of a target view directly from the input image.

  3. Residual Correction for Incomplete Views A lightweight residual encoder predicts corrections, decomposing the problem into a parameter-free geometric prior and a learned residual correction, where the encoder focuses on disoccluded areas and depth estimation errors.

  4. Generalization via Two-Stage Training The system can generalize from synthetic training data to real-world scenes by employing a two-stage training strategy combining latents supervised distillation for geometric generalization and perceptual finetuning for appearance quality optimization.

  5. Cross-View Geometric Consistency Supervision Stage 1 supervision targets the intermediate features of the frozen 3DGS decoder D across all views, ensuring that latent predictions encode multiview geometric consistency rather than per-view appearance, which is crucial for reconstructing a globally consistent 3D scene.

  6. Perceptual Quality Alignment Stage 2 fine-tuning uses a loss function like LPIPS(ˆI, I∗) + λdyn MSE LMSE(ˆI, I∗) to optimize rendered images against ground truth, bridging the quality gap by refining fine-grained texture and sharpness.

Sources

Related papers