Skip to content

Commit 751cca1

Browse files
committed
Add 3DGS columnar splat codec plan
1 parent c386f55 commit 751cca1

1 file changed

Lines changed: 168 additions & 0 deletions

File tree

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
# 3DGS Columnar Splat Codec Plan — ndarray
2+
3+
## Goal
4+
5+
Define the ndarray-side columnar representation and codec primitives for 3DGS splat blocks.
6+
7+
The representation must be friendly to:
8+
9+
- SIMD batch projection
10+
- Arrow/Lance sidecar storage in `lance-graph`
11+
- quantization and compression
12+
- deterministic validation
13+
- future glTF/SPZ/3D Tiles import-export bridges
14+
15+
## Non-goals
16+
17+
- Do not implement 3D Tiles package writing here.
18+
- Do not own metadata schemas for full geospatial tilesets.
19+
- Do not implement a server.
20+
21+
## Core representation
22+
23+
Prefer struct-of-arrays for hot paths:
24+
25+
```rust
26+
pub struct SplatBlockColumns<T> {
27+
pub pos_x: Vec<T>,
28+
pub pos_y: Vec<T>,
29+
pub pos_z: Vec<T>,
30+
pub scale_x: Vec<T>,
31+
pub scale_y: Vec<T>,
32+
pub scale_z: Vec<T>,
33+
pub quat_w: Vec<T>,
34+
pub quat_x: Vec<T>,
35+
pub quat_y: Vec<T>,
36+
pub quat_z: Vec<T>,
37+
pub opacity: Vec<T>,
38+
pub color_rgba: Vec<[u8; 4]>,
39+
pub feature_id: Vec<u32>,
40+
}
41+
```
42+
43+
For zero-copy interop, expose borrowed views:
44+
45+
```rust
46+
pub struct SplatBlockView<'a, T> {
47+
pub pos_x: &'a [T],
48+
pub pos_y: &'a [T],
49+
pub pos_z: &'a [T],
50+
pub scale_x: &'a [T],
51+
pub scale_y: &'a [T],
52+
pub scale_z: &'a [T],
53+
pub quat_w: &'a [T],
54+
pub quat_x: &'a [T],
55+
pub quat_y: &'a [T],
56+
pub quat_z: &'a [T],
57+
pub opacity: &'a [T],
58+
pub color_rgba: &'a [[u8; 4]],
59+
pub feature_id: &'a [u32],
60+
}
61+
```
62+
63+
## Quantization tiers
64+
65+
Implement explicit tiers, not hidden magic:
66+
67+
1. `F32Raw`
68+
- reference format
69+
- easiest validation
70+
71+
2. `F16Packed`
72+
- scale/quaternion/opacity where tolerable
73+
- use existing f16 carrier machinery
74+
75+
3. `Bf16Packed`
76+
- wider exponent for geospatial scale variation
77+
- useful for world-space values after local origin rebasing
78+
79+
4. `I16LocalTile`
80+
- local tile coordinate frame
81+
- positions stored as i16/u16 with scale/offset
82+
83+
5. `PaletteBlock`
84+
- optional future tier for repeated covariance/color/SH patterns
85+
86+
## Tile-local coordinate policy
87+
88+
World geospatial coordinates should not be projected directly in f32 at large scales.
89+
90+
Use:
91+
92+
```text
93+
ECEF / projected world coordinate
94+
->
95+
tile local origin
96+
->
97+
local f32 or quantized i16 block coordinate
98+
```
99+
100+
The codec should support a `TileLocalFrame`:
101+
102+
```rust
103+
pub struct TileLocalFrame {
104+
pub origin_world: [f64; 3],
105+
pub axis_x: [f64; 3],
106+
pub axis_y: [f64; 3],
107+
pub axis_z: [f64; 3],
108+
pub scale: f64,
109+
}
110+
```
111+
112+
## Stats emitted by codec
113+
114+
Every encode/decode must produce stats:
115+
116+
```rust
117+
pub struct SplatCodecStats {
118+
pub splat_count: usize,
119+
pub bytes_raw: usize,
120+
pub bytes_encoded: usize,
121+
pub max_position_error: f32,
122+
pub rms_position_error: f32,
123+
pub max_scale_error: f32,
124+
pub max_opacity_error: f32,
125+
pub invalid_quaternion_count: usize,
126+
pub invalid_scale_count: usize,
127+
}
128+
```
129+
130+
These stats feed `3DGS-error-certification-pillars-plan.md`.
131+
132+
## Module layout
133+
134+
```text
135+
src/hpc/splat3d/codec/
136+
mod.rs
137+
columns.rs
138+
local_frame.rs
139+
quant_f16.rs
140+
quant_bf16.rs
141+
quant_i16_tile.rs
142+
palette.rs
143+
stats.rs
144+
```
145+
146+
## Acceptance criteria
147+
148+
- Lossless f32 roundtrip test.
149+
- Quantized roundtrip tests with explicit max/RMS error checks.
150+
- Tile-local coordinate tests for large ECEF-like origins.
151+
- SIMD renderer accepts borrowed views without copying.
152+
- Codec stats can feed certificate generation.
153+
154+
## Cross-repo integration
155+
156+
`lance-graph` should own Arrow/Lance schema and durable storage.
157+
158+
`ndarray` should own the hot-path memory layout and conversion kernels.
159+
160+
Recommended bridge:
161+
162+
```text
163+
lance-graph Arrow RecordBatch
164+
->
165+
borrowed ndarray SplatBlockView
166+
->
167+
projection / certification / report
168+
```

0 commit comments

Comments
 (0)