Skip to content

Commit c74c4e3

Browse files
committed
Add 3DGS render depth certification plan
1 parent 9244bd7 commit c74c4e3

1 file changed

Lines changed: 207 additions & 0 deletions

File tree

Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
# 3DGS Render-Depth Certification Plan — ndarray
2+
3+
## Goal
4+
5+
Capture the mathematical-bounds cross-pollination for rendering depth, visibility, occlusion, and progressive refinement.
6+
7+
This plan connects:
8+
9+
```text
10+
3DGS EWA projection
11+
Cesium screen-space error
12+
Blender/CAD depth buffers
13+
CPU-SIMD HHTL traversal
14+
statistical error certificates
15+
```
16+
17+
The renderer should not only decide what to draw. It should know how deep, how certain, and how much refinement is mathematically justified.
18+
19+
## Core problem
20+
21+
Depth in 3DGS and CAD/BIM scenes is not a single scalar problem.
22+
23+
A view may contain:
24+
25+
```text
26+
mesh surfaces
27+
transparent splats
28+
overlapping Gaussian support
29+
scan noise
30+
quantized local frames
31+
LOD substitutions
32+
query-relevant hidden objects
33+
```
34+
35+
A naive depth test is insufficient for certified traversal.
36+
37+
## Depth certificate components
38+
39+
```rust
40+
pub struct RenderDepthCertificate {
41+
pub min_depth: f32,
42+
pub max_depth: f32,
43+
pub depth_variance: f32,
44+
pub projected_radius_px: f32,
45+
pub occlusion_confidence: f32,
46+
pub ordering_uncertainty: f32,
47+
pub quantization_depth_error: f32,
48+
pub covariance_depth_error: f32,
49+
pub total_depth_error: f32,
50+
pub passed: bool,
51+
}
52+
```
53+
54+
## Error terms
55+
56+
```text
57+
E_depth_total =
58+
E_camera_transform
59+
+ E_local_frame_quantization
60+
+ E_covariance_projection
61+
+ E_splat_support_overlap
62+
+ E_sort_bucket_width
63+
+ E_lod_substitution
64+
+ E_sampling_discrepancy
65+
```
66+
67+
The exact formula can evolve, but the certificate should keep terms separate for auditability.
68+
69+
## Projection path
70+
71+
For each splat or block:
72+
73+
```text
74+
world/local center
75+
-> view-space position
76+
-> projected screen center
77+
-> EWA image covariance
78+
-> approximate depth support interval
79+
-> depth certificate
80+
```
81+
82+
Depth support interval:
83+
84+
```text
85+
z_center ± k * sigma_z
86+
```
87+
88+
where `k` is selected by the render budget / confidence target.
89+
90+
## HHTL usage
91+
92+
```text
93+
HEEL
94+
reject blocks outside frustum or behind near/far planes
95+
96+
HIP
97+
estimate depth interval and projected radius
98+
99+
TWIG
100+
refine overlap/ordering uncertainty
101+
102+
LEAF
103+
exact projection and optional raster/composite path
104+
```
105+
106+
## Blender/CAD relevance
107+
108+
Blender/CAD scenes add exact mesh depth or proxy depth:
109+
110+
```text
111+
mesh object depth
112+
-> exact or rasterized depth interval
113+
114+
3DGS texture/radiance skin
115+
-> probabilistic splat depth interval
116+
117+
combined object
118+
-> mesh depth anchors splat depth confidence
119+
```
120+
121+
This enables reality-skinned CAD/BIM:
122+
123+
```text
124+
CAD mesh says wall is here.
125+
3DGS scan says visual surface is here.
126+
Certificate reports whether the visual skin aligns within tolerance.
127+
```
128+
129+
## Cesium relevance
130+
131+
Cesium-style traversal uses screen-space error.
132+
133+
3DGS depth certification adds:
134+
135+
```text
136+
screen-space error
137+
+ depth uncertainty
138+
+ ordering uncertainty
139+
+ occlusion confidence
140+
```
141+
142+
This improves decisions for:
143+
144+
```text
145+
skip LOD
146+
refine tile
147+
hydrate exact mesh
148+
render splat preview
149+
reject visually hidden block
150+
```
151+
152+
## CPU-SIMD kernels
153+
154+
Candidate kernels:
155+
156+
```text
157+
batch view-space transform
158+
batch depth interval estimate
159+
batch frustum/depth rejection
160+
batch projected-radius estimate
161+
batch occlusion bucket assignment
162+
batch certificate reduction
163+
```
164+
165+
The exact alpha compositing path may remain GPU/WGPU optional. CPU-SIMD owns planning, prefiltering, and certification.
166+
167+
## Sorting and buckets
168+
169+
Full exact splat sorting can be expensive.
170+
171+
Use tiered depth buckets:
172+
173+
```text
174+
coarse bucket
175+
-> render/skip/refine decision
176+
177+
uncertain bucket overlap
178+
-> TWIG refinement or exact LEAF sort
179+
```
180+
181+
Certificate reports whether bucket uncertainty is acceptable.
182+
183+
## Acceptance criteria
184+
185+
- Depth certificate DTO exists at plan/API level before implementation.
186+
- Scalar reference computes depth interval and projected-radius estimates.
187+
- SIMD path can batch depth rejection.
188+
- Certificate terms remain separate, not only one opaque score.
189+
- Blender/CAD mesh depth can be used as an anchor for 3DGS skin alignment.
190+
- Cesium-style SSE decisions can include depth uncertainty.
191+
192+
## First demo
193+
194+
```text
195+
one camera
196+
one mesh plane
197+
one small splat block near the plane
198+
-> compute mesh depth
199+
-> compute splat depth interval
200+
-> report alignment / uncertainty / pass-fail
201+
```
202+
203+
## Wall sentence
204+
205+
```text
206+
Depth is not just where something is; depth is how confidently the renderer may stop looking behind it.
207+
```

0 commit comments

Comments
 (0)