---

• Description: GAVIS paper note — uncertainty quantification for 3DGS via a per-particle anisotropic visibility field; spherical harmonics representation, 200+ FPS real-time UQ, outperforms FisherRF/VIMC/NVF across all image-quality metrics
• My Notion Note ID: K2E-B-G2-7
• Created: 2026-07-01
• Updated: 2026-07-13
• License: Free to share: please credit Yu Zhang and link back to yuzhang.io

---

Table of Contents

• 1. Summary
• 2. Key Contributions
• 3. Method
  • 3.1 Anisotropic Visibility Field Definition
  • 3.2 Spherical Harmonics Representation
  • 3.3 Uncertainty-Aware Rasterizer
  • 3.4 Density Control: Virtual Particles
  • 3.5 Active Mapping Pipeline
• 4. Experiments & Results
• 5. Ablation & Discussion
• 6. Strengths / Limitations / Future Work
• References

---

1. Summary

Title: Uncertainty-driven 3D Gaussian Splatting Active Mapping via Anisotropic Visibility Field Authors: S. Xue*, J. Dill*, D. Ahuja*, F. Dellaert, P. Tsiotras, D. Xu (Georgia Institute of Technology) Paper: arXiv:2605.30342 (CVPR 2026) Github: xsj01/gavis · project page

GAVIS (Gaussian Splatting Anisotropic Visibility Field, Georgia Tech, 2026) — UQ and active mapping framework for 3DGS (3D Gaussian Splatting, an explicit scene representation using Gaussian primitives).

Core problem: Active mapping (robot autonomously selects next viewpoint to maximize map quality) requires uncertainty estimates. Current methods — FisherRF and VIMC (parametric UQ); NVF (neural network learning an isotropic visibility field) — are either inaccurate or too slow (NVF requires minutes of retraining per planning step).

Key insight: Regions not covered by training views are inherently unreliable → use visibility as a proxy for uncertainty. Unlike NVF's isotropic position function, GAVIS models anisotropic (direction-dependent) visibility: observing a Gaussian particle from one direction does not imply observing it from the opposite side (wall example). Visibility must be a function of render direction $\mathbf{d}$.

Main results: Outperforms FisherRF and VIMC across all four datasets (NeRF Synthetic / Space / Gibson / HM3D). Vs. NVF: dominant on image quality (PSNR/SSIM/LPIPS) and efficiency across all datasets; only Gibson mesh metrics (CR/VIS) slightly worse (HM3D: GAVIS edges NVF on CR 0.820 vs. 0.819, VIS 0.876 vs. 0.873). Visibility field built 500× faster than NVF (< 1 s vs. minutes); UQ at 200+ FPS. Works as a post-hoc plug-in: Fisher+GAVIS substantially improves FisherRF.

2. Key Contributions

• Anisotropic visibility field: per-particle direction-dependent visibility $V^{(i)}(\mathbf{d})$ with three factors — FOV indicator × transmittance × directional similarity (von Mises-Fisher-style function); handles self-occlusion and multi-view coverage
• Efficient SH representation: $L=2$ spherical harmonics ($(L+1)^2 = 9$ params/particle) → build complexity $O(|\mathcal{P}|)$, query $O(1)$ (independent of trajectory length); vs. naive SH representation at $O(|\mathcal{P}|^4 L^3)$ build cost
• Bayesian-network rasterizer: integrates visibility probability $v_i$ into a GMM pixel-color PDF; entropy used directly as active-mapping objective
• Virtual particles for density control: zero-opacity particles distinguish unexplored regions from free space → high uncertainty assigned to unexplored areas
• General post-hoc module: plug into FisherRF or VIMC to substantially improve both

3. Method

3.1 Anisotropic Visibility Field Definition

Single-view visibility (from camera pose $\mathbf{p}$ for particle $i$):

$$V^{(i)}_{\mathbf{p}}(\mathbf{d}) = \underbrace{\Phi_{i,\mathbf{p}}}_{\text{FOV}} \cdot \underbrace{T_{\mathbf{p}}(t_i^{\mathbf{p}})}_{\text{transmittance}} \cdot \underbrace{\nu(\mathbf{d};\,\mathbf{d}_p)}_{\text{dir. similarity}}$$

Three factors:

• $\Phi_{i,\mathbf{p}} \in \{0,1\}$: binary FOV indicator — whether particle $i$ falls inside camera $\mathbf{p}$'s field of view
• $T_{\mathbf{p}}(t_i^{\mathbf{p}})$: transmittance — probability of reaching particle $i$ without occlusion along the ray from camera $\mathbf{p}$; read directly from the 3DGS rasterizer
• $\nu(\mathbf{d};\,\mathbf{d}_p) = \zeta\exp(\kappa\,\mathbf{d}\cdot\mathbf{d}_p)$: directional similarity between render direction $\mathbf{d}$ and training view direction $\mathbf{d}_p$; $\zeta = \exp(-\kappa)$ normalizes so $\nu(\mathbf{d}_p;\mathbf{d}_p)=1$; $\kappa$ controls concentration (inverse of spread)

The first two terms $\Phi_{i,\mathbf{p}}\,T_{\mathbf{p}}(t_i^{\mathbf{p}})$ form an isotropic visibility (same as NVF); the third introduces directional anisotropy.

Full-dataset visibility (probability of being observed from at least one training view):

$$V^{(i)}(\mathbf{d}) = 1 - \prod_{\mathbf{p}\in\mathcal{P}}\!\bigl(1 - V^{(i)}_{\mathbf{p}}(\mathbf{d})\bigr)$$

$\mathcal{P}$ — set of all observed camera poses.

3.2 Spherical Harmonics Representation

Two naive approaches are unscalable:

• Direct product evaluation (§3.1 closed-form): must access all training view directions $\mathbf{d}_p$ at query time → runtime and memory scale linearly with trajectory length $O(|\mathcal{P}|)$
• Naive SH representation of $V^{(i)}(\mathbf{d})$ (substituting the SH expansion of $\nu$ into the product formula): build $O(|\mathcal{P}|^4 L^3)$, memory $O(|\mathcal{P}|^2 L^2)$, query $O(|\mathcal{P}|^2)$

Both blow up as the trajectory grows. The fix: expand the directional similarity function $\nu$ in SH analytically.

SH decomposition of $\nu$:

$$\nu(\mathbf{d};\mathbf{d}_p) = \zeta\sum_{\ell=0}^{\infty}\sum_{m=-\ell}^{\ell} a_{\ell m}\,Y_{\ell}^m(\mathbf{d}), \qquad a_{\ell m} = 4\pi\,i_\ell(\kappa)\,Y_{\ell}^{m*}(\mathbf{d}_p)$$

$Y_{\ell}^m$ — spherical harmonic basis; $i_\ell(\kappa)$ — modified spherical Bessel function of the first kind; $Y_{\ell}^{m*}$ — complex conjugate.

Auxiliary function $\tilde{V}^{(i)}(\mathbf{d})$ (sum over all views):

$$\tilde{V}^{(i)}(\mathbf{d}) = \sum_{\mathbf{p}} V^{(i)}_{\mathbf{p}}(\mathbf{d}) = \sum_{\ell=0}^{L}\sum_{m=-\ell}^{\ell} \gamma_{\ell m}^{\mathcal{P}}\,Y_{\ell}^m(\mathbf{d})$$

SH coefficients computed analytically (no training needed):

$$\gamma_{\ell m}^{\mathcal{P}} = 4\pi\,\zeta\,i_\ell(\kappa)\!\sum_{\mathbf{p}\in\mathcal{P}}\Phi_{i,\mathbf{p}}\,T_{\mathbf{p}}(t_i^{\mathbf{p}})\,Y_{\ell}^{m*}(\mathbf{d}_p)$$

Adding a new training frame → accumulate $\Phi_{i,\mathbf{p}}\,T_{\mathbf{p}}(t_i^{\mathbf{p}})\,Y_{\ell}^{m*}(\mathbf{d}_p)$ into coefficients → build $O(|\mathcal{P}|)$. Each particle stores $(L+1)^2 = 9$ coefficients ($L=2$, matching 3DGS color SH order).

AM-GM lower bound for $O(1)$ query:

$$V^{(i)}(\mathbf{d}) \geq 1 - \Bigl(1 - \frac{\tilde{V}^{(i)}(\mathbf{d})}{|\mathcal{P}|}\Bigr)^{|\mathcal{P}|}$$

Compute $\tilde{V}^{(i)}(\mathbf{d})$ from SH coefficients in $O(1)$, substitute into the bound — no access to the training trajectory at query time.

3.3 Uncertainty-Aware Rasterizer

Pixel-color PDF modeled as a GMM with visibility-weighted components:

$$p(\mathbf{z}_0) = \sum_i w_i^*\,v_i\,\mathcal{N}(\boldsymbol{\mu}_{c_i}, \mathbf{Q}_{c_i}) + \mathcal{N}(\boldsymbol{\mu}_0, \mathbf{Q}_0)\sum_i w_i^*(1-v_i)$$

• $w_i^*$: visibility-corrected mixture weights (derived from standard 3DGS alpha-blending)
• $v_i = V^{(i)}(\mathbf{d})$: visibility probability of particle $i$ in the current render direction
• $\mathcal{N}(\boldsymbol{\mu}_{c_i}, \mathbf{Q}_{c_i})$: color distribution of particle $i$ (mean $\boldsymbol{\mu}_{c_i}$, covariance $\mathbf{Q}_{c_i}$)
• $\mathcal{N}(\boldsymbol{\mu}_0, \mathbf{Q}_0)$: prior for unobserved regions (large variance → high uncertainty)

GMM entropy $\mathcal{H}$ drives next-best-view (NBV) selection:

$$\tau^* = \arg\max_{\tau}\,\mathcal{H}(\mathbf{Z}_\tau)$$

$\tau$ — candidate camera pose; $\mathbf{Z}_\tau$ — pixel-color random variable observed from $\tau$.

3.4 Density Control: Virtual Particles

Problem: 3DGS adaptive density control prunes particles in empty regions. A "no particle" region can be either: ① free space (explored, pruned) or ② unexplored (insufficient initialization, zero gradient, never densified). Standard 3DGS cannot distinguish them — particle-based UQ assigns low uncertainty to both, causing unexplored areas to be ignored.

Solution: uniformly sample virtual particles (zero opacity, not rendered) and compute their isotropic visibility:

$$V^{(i)}_\text{virtual}(\mathbf{d}) = 1 - \prod_{\mathbf{p}\in\mathcal{P}}\bigl(1-\Phi_{i,\mathbf{p}}\,T_{\mathbf{p}}(t_i^{\mathbf{p}})\bigr)$$

(isotropic only, since virtual particles have no directional preference)

• Low-visibility virtual particle → unexplored region → keep, set $V^{(i)}(\mathbf{d}) = 0$ (maximum uncertainty)
• High-visibility virtual particle → free space → prune

Virtual particles ≈ 5–10% of total particle count; concatenated with trained 3DGS particles for UQ.

3.5 Active Mapping Pipeline

1. Train 3DGS on current observation frames
2. Build visibility field: analytically compute SH coefficients $\{\gamma_{\ell m}^{\mathcal{P}}\}$ for all particles (§3.2); add virtual particles
3. Candidate-view UQ: sample candidate poses $\tau$ from prior; compute GMM entropy via uncertainty-aware rasterizer (§3.3) for each
4. Select NBV: $\tau^* = \arg\max \mathcal{H}(\mathbf{Z}_\tau)$; execute observation; repeat

Post-hoc plug-in mode: overlay GAVIS visibility $v_i$ onto existing UQ estimates (FisherRF, VIMC) to upweight low-visibility regions.

4. Experiments & Results

Settings: 3 domains, 4 datasets

• NeRF Synthetic (object reconstruction)
• Space dataset (Hubble HST + ISS)
• Gibson and HM3D (indoor home-robot environments, 8 scenes each)

Baselines:

• FisherRF — Fisher information UQ for 3DGS
• VIMC — Monte-Carlo manifold-sampling UQ (sampling-based variational inference; co-trains an uncertainty model)
• NVF — Neural Visibility Field, learns an isotropic visibility field for NeRF-based UQ

Metrics: PSNR/SSIM/LPIPS (image quality); CR (completion ratio, mesh coverage); VIS (visual coverage); $T_{UP}$ (uncertainty preparation time — visibility field build for GAVIS/NVF, Hessian estimation for FisherRF, training overhead for VIMC); UQ FPS (per candidate view)

Dataset	Method	PSNR ↑	SSIM ↑	LPIPS ↓	CR ↑	VIS ↑	UQ FPS ↑	$T_{UP}$ ↓
NeRF Syn.	FisherRF	22.34	0.870	0.119	0.626	0.376	146	0.42s
	VIMC	23.14	0.880	0.107	0.651	0.407	145	9.48s
	NVF	22.59	0.859	0.147	0.549	0.431	11.9	149.1s
	GAVIS	24.26	0.894	0.097	0.711	0.437	252	0.17s
Gibson	FisherRF	18.11	0.720	0.419	0.431	0.469	39.8	0.90s
	VIMC	15.70	0.668	0.465	0.337	0.366	57.0	90.5s
	NVF	23.29	0.798	0.402	0.880	0.915	4.2	219.9s
	GAVIS	24.42	0.812	0.323	0.831	0.890	207	0.42s

NVF beats GAVIS on CR/VIS only in Gibson (CR 0.880 vs. 0.831, VIS 0.915 vs. 0.890); on HM3D GAVIS edges NVF (CR 0.820 vs. 0.819, VIS 0.876 vs. 0.873). NVF trails on all image-quality metrics, UQ ≈30× slower on average (paper's headline figure; up to ~50× on Gibson/HM3D), visibility build ≈hundreds× slower. Space dataset: GAVIS PSNR 26.14 vs. NVF 23.76 (+2.4 dB), $T_{UP}$ 0.17s vs. 140.5s (826× speedup).

5. Ablation & Discussion

Ablation (Table 2, averaged over 4 datasets):

Configuration	PSNR ↑	SSIM ↑	LPIPS ↓	CR ↑	VIS ↑
Full GAVIS	24.70	0.839	0.224	0.748	0.697
Isotropic (no $\nu$ term)	23.97	0.827	0.231	0.741	0.671
No density control (no virtual particles)	24.18	0.830	0.234	0.712	0.668
Isotropic + no density control (≈ NVF ported to 3DGS)	23.38	0.819	0.240	0.691	0.625

Both components necessary: directional anisotropy contributes more to image quality (PSNR/SSIM/LPIPS); density control matters more for mesh coverage (CR/VIS). Isotropic + no density control (naive NVF→3DGS port) is worst.

GAVIS as post-hoc module (Table 3, averaged):

Method	PSNR ↑	SSIM ↑	LPIPS ↓	CR ↑	VIS ↑
FisherRF	20.73	0.779	0.285	0.513	0.469
FisherRF+GAVIS	24.70	0.842	0.220	0.748	0.699
VIMC	20.14	0.758	0.300	0.519	0.475
VIMC+GAVIS	24.21	0.833	0.227	0.719	0.672

FisherRF+GAVIS nearly matches standalone GAVIS (24.70 vs. 24.70) → visibility modeling is the dominant factor in active mapping; parametric UQ adds little on top. VIMC+GAVIS slightly weaker, as VIMC's sampling noise interferes with the visibility signal.

Uncertainty quality (AUSE, 5 scenes):

Method	AUSE-D ↓	AUSE-V ↓
FisherRF	0.463	0.496
VIMC	0.504	0.447
NVF	0.381	0.231
GAVIS	0.224	0.176

AUSE-D (area under sparsification error curve by depth error) and AUSE-V (by visibility error, more aligned with active mapping goal) both best.

6. Strengths / Limitations / Future Work

Strengths

• Analytic visibility field — no neural network, built in < 1 s (NVF: minutes)
• 200+ FPS UQ → real-time robot planning loop
• Outperforms all existing 3DGS UQ methods; composable as post-hoc module with any existing approach
• Virtual particles solve the UQ blind spot from 3DGS density pruning

Limitations

• Evaluated on static scenes only; dynamic environments not validated (not stated by the paper — inferred from experimental scope)
• Isotropic mesh metrics (CR/VIS) cannot fully credit anisotropic modeling (a mesh face counts as covered from any direction) — grounded in the paper's own §5.2 discussion
• Candidate-view sampling and NBV selection efficiency not analyzed for long planning horizons

Future Work

• Extend to dynamic scenes (visibility estimation for moving objects) — editorial extension, not stated by the paper
• Apply anisotropic visibility to other 3DGS tasks (inpainting, relighting, occlusion-aware view synthesis) — grounded in the paper's conclusion + related-work framing
• Integrate with front-end SLAM systems (SplaTAM, MonoGS, etc.) for online active SLAM — editorial extension, not stated by the paper

References

• Xue, S., Dill, J., Ahuja, D., Dellaert, F., Tsiotras, P., & Xu, D. (2026). Uncertainty-driven 3D Gaussian Splatting Active Mapping via Anisotropic Visibility Field. CVPR 2026. arXiv:2605.30342, code.
• Jiang, W., Lei, B., & Daniilidis, K. (2024). FisherRF: Active view selection and mapping with radiance fields using Fisher information. ECCV 2024.
• Lyu, L., Tewari, A., Habermann, M., Saito, S., Zollhöfer, M., Leimkühler, T., & Theobalt, C. (2024). Manifold sampling for differentiable uncertainty in radiance fields (VIMC). SIGGRAPH Asia 2024 Conference Papers. [paper ref 54]
• Xue, S., Dill, J., Mathur, P., Dellaert, F., Tsiotras, P., & Xu, D. (2024). Neural visibility field for uncertainty-driven active mapping (NVF). CVPR 2024. [paper ref 104]
• SplaTAM note: 3DGS SLAM base framework
• 3D Gaussian Splatting note: 3DGS representation basics