CF-Splat: Closed-Form Triangle Splatting with Mesh Priors

The gist CF-Splat proposes a novel closed-form triangle-splatting framework for efficient scene appearance estimation over a known mesh by formulating the problem as a global linear least-squares (LLS) problem, eliminating iterative nonlinear optimization and substantially reducing computation cost.

How it works

CF-Splat leverages a known mesh to establish a linear relationship between rendered pixel colors and per-vertex spherical harmonics (SH) coefficients, allowing multi-view scene appearance estimation to be formulated as a global LLS problem without the need of iterative nonlinear optimization. This approach is based on the insight that the rendered pixel colors can be expressed linearly with respect to the per-vertex spherical harmonics (SH) coefficients which enables a closed-form solution.

The core of the method involves several key components:

1. A global LLS formulation is derived by exploiting the linear dependence of rendered pixel colors on per-vertex SH coefficients, thereby eliminating iterative nonlinear optimization and substantially reducing computation cost.

2. To guarantee a well-posed solution under uneven multi-view observation coverage, the framework introduces adaptive SH regularization that constrains insufficiently supported high-order SH coefficients and yields a unique global minimizer.

3. To ensure scalability to large-scale scenes, a triangle-structured matrix-free preconditioned conjugate-gradient solver that avoids explicitly constructing the large sparse normal system is developed.

Closed-Form Formulation and Regularization

The problem statement defines the known mesh M as (V, F) and calibrated RGB observations O as (Ik, Kk, Tk). View-dependent appearance is modeled by encoding each vertex in V by SH signals f(c, d) = X L sum l X l sum m c ml y ml(d). The photometric objective is expressed as Ephoto(X) = 1/2 sum k bIk(X M, O) - I k T 2/F.

The global multi-view observation model is constructed by defining the per-view observation matrix Ak in R(HW x qNv), where [Ak]r,is k,r = phi s k,r T for the row corresponding to pk,r. This leads to the global observation matrix A and B such that Bb = AX.

Triangle-Structured Matrix-Free Solver

For large-scale scenes, explicit construction of the global normal matrix H is avoided by developing a triangle-structured matrix-free PCG solver. This solver evaluates the required matrixvector product directly from rasterization records without explicitly constructing H.

The PCG iteration involves:

1. Constructing the right-hand side G in (33) by accumulating contributions over all valid pixels and views.

2. Evaluating the normal-matrix product H(u) using rasterization records without explicitly constructing H in (36).

3. Using a vertex-block preconditioner where each vertex has its own q x q preconditioner block Pi.

Scalability and Robustness

The scalability analysis shows that the computation time grows approximately linearly with the number of input observations KHW, while peak GPU memory complexity is O(Nvq + HWq). The matrix-free formulation avoids explicitly storing A and H in (29), allowing the working memory to be governed primarily by the number of vertices Nv.

The analysis of adaptive SH regularization demonstrates its effectiveness under uneven observations, showing that adaptive regularization substantially improves the test-view PSNR and the PCG residual decreases more rapidly compared to unregularized systems. Furthermore, CF-Splat demonstrates robustness to input noise, maintaining better reconstruction quality than 3DGS under camera pose perturbations and mesh inaccuracies.

Conclusion

CF-Splat achieves competitive or superior NVS quality while substantially reducing computation time and peak GPU memory consumption on a consumer-grade laptop. The paper concludes that CF-Splat provides a new closed-form perspective on splatting-based scene representation, demonstrating that high-quality appearance estimation can be achieved through a globally structured linear formulation rather than iterative nonlinear scene fitting.

How it works

The final analytical form of the solution is X⋆ = H−1G = A⊤WA + Λ−1A⊤WB. The triangle-structured matrix-free PCG solver is then used to obtain the final appearance estimate X∗. This system admits a unique global minimizer because the adaptive SH regularization makes the resulting normal system strictly positive definite.

How it works

The efficiency gain is primarily due to the proposed closed-form formulation, which exploits the linear dependence of image observations on the per-vertex SH coefficients and formulates appearance estimation as a global LLS formulation in (28). This contrasts with iterative methods that require repeatedly processing highresolution multi-view observations dominates the computational cost of iterative methods on large-scale scenes.

How it works

The geometric accuracy and appearance fidelity are evaluated by measuring the bidirectional mean surface distance between the ground-truth and reconstructed meshes, as well as the mean CIEDE2000 color difference. CF-Splat consistently achieves the lowest bidirectional geometric error across all evaluated scenes while also producing lower surface appearance error.

How it works

The ablation study confirms the benefits of various design choices, showing that increasing L generally improves NVS quality by expanding the directional appearance capacity of the per-vertex SH representation. Similarly, increasing spatial vertex support through gradient-guided triangle densification improves fine-scale appearance reconstruction. The block-Jacobi preconditioner is shown to converge more efficiently while maintaining a comparable memory footprint.

How it works

CF-Splat preserves the geometry of the input mesh and focuses on appearance estimation over this surface representation, deliberately simplifying the splatting problem by avoiding joint optimization of geometry during scene fitting<ref:2610.

1. Bold header: Global LLS Formulation for Appearance Estimation

This formulation allows multi-view scene appearance estimation to be formulated as a global linear least-squares (LLS) problem without the need of iterative nonlinear optimization, fundamentally replacing the computationally expensive, potentially non-convergent process used in methods like 3DGS.

2. Bold header: Triangle-Structured Matrix-Free PCG Solver

The development of this solver avoids explicitly constructing the large sparse normal system while producing the same LLS solution, which reduces GPU memory consumption and enabling the proposed LLS formulation to large-scale scenes.

3. Bold header: Adaptive SH Regularization for Robustness

The introduction of this regularization ensures a well-posed LLS problem with a unique global minimizer by constraining insufficiently supported high-order SH coefficients, which improves cross-view stability of the solution.

4. Bold header: Gradient-Guided Triangle Densification Preprocessing

Applying this step before the LLS system construction allows for adaptive allocation of spatial appearance capacity without altering the mapped surface, improving fine-scale reconstruction by selecting triangles with large multiview image-space variation.

5. Bold header: Scalable Solution for Large Scenes

The resulting framework maintains stability and efficiency on large scenes because the larger problem does not introduce additional nonlinear optimization variables or local minima, as the formulation remains linear, allowing the system to be solved efficiently via the matrix-free PCG solver.