PCM-NeRF: Probabilistic Camera Modeling for Neural Radiance Fields under Pose Uncertainty

Neural surface reconstruction methods typically treat camera poses as fixed values, assuming perfect accuracy from Structure-from-Motion (SfM) systems. This assumption breaks down with imperfect pose estimates, leading to distorted or incomplete reconstructions. We present PCM-NeRF, a probabilistic framework that augments neural surface reconstruction with per-camera learnable uncertainty, built directly on SG-NeRF. Rather than treating all cameras equally throughout optimization, we represent each pose as a distribution with a learnable mean and variance, initialized from SfM correspondence quality. An uncertainty regularization loss couples the learned variance to view confidence — derived from both geometric correspondences and rendering quality — and the resulting uncertainty directly modulates the effective pose learning rate: uncertain cameras receive damped gradient updates, preventing poorly initialized views from corrupting the reconstruction. Experiments on challenging scenes with severe pose outliers demonstrate that PCM-NeRF consistently outperforms state-of-the-art methods in both Chamfer Distance and F-Score, particularly for geometrically complex structures, without requiring foreground masks.

Contributions

• We introduce a probabilistic per-camera pose representation for NSR in which each pose is modeled as a distribution in SE(3) with a learnable mean and per-axis variance, initialized from SfM correspondence quality and evolved jointly with the scene.
• We derive an uncertainty-modulated pose learning rate that inversely scales each camera's effective gradient step by its learned uncertainty, automatically preventing outlier views from destabilizing reconstruction without manual filtering or threshold selection.
• We propose a cross-modal consistency loss that aligns learned variance parameters with view confidence derived from both geometric correspondences and rendering quality, anchoring uncertainty to observable evidence rather than rendering gradients alone.
• We demonstrate state-of-the-art surface reconstruction on challenging outlier-pose scenes, achieving the lowest Chamfer Distance on 6 of 8 scenes and the highest F-Score on 6 of 8 scenes.

PCM-NeRF builds directly on SG-NeRF, extending gradient-based pose refinement with a lightweight probabilistic pose representation. Each camera pose P_i is modeled as a distribution in SE(3) with learnable mean μ_i = (r_i, t_i) and diagonal covariance Σ_i = diag(σ²_r,i, σ²_t,i), parameterized in log-space for numerical stability. All rendering uses the mean pose directly — adding no inference-time overhead — while the variance parameters are optimized through the uncertainty regularization loss.

View reliability is estimated from SfM correspondence density and a rolling buffer of per-camera PSNR scores, blended as γ_i^(t) = (1−α)γ_i⁽⁰⁾ + α·γ̂_i^(t) with α = 0.7. The uncertainty regularization loss encourages σ̄_i ≈ (1−γ_i), and the learned uncertainty modulates the effective pose learning rate: η_i^pose = η₀ / (1 + σ̄_i · κ), where κ = 5.0. Geometric consistency is enforced via the volumetric IoU loss over Mixture-of-Gaussian density distributions along matched rays, inherited from SG-NeRF.