|
| 1 | +# 3DGS Certified Field Kernel Substrate Plan — ndarray |
| 2 | + |
| 3 | +## Goal |
| 4 | + |
| 5 | +Generalize the ndarray-side 3DGS math into a domain-neutral certified field-kernel substrate. |
| 6 | + |
| 7 | +The 3DGS renderer is the first concrete consumer, but the numerical pattern is wider: |
| 8 | + |
| 9 | +```text |
| 10 | +local field block |
| 11 | + -> kernel summary |
| 12 | + -> SIMD / BLAS / tensor operation |
| 13 | + -> error certificate |
| 14 | + -> skip / refine / hydrate decision support |
| 15 | +``` |
| 16 | + |
| 17 | +## What stays in ndarray |
| 18 | + |
| 19 | +`ndarray` owns reusable numerical kernels: |
| 20 | + |
| 21 | +```text |
| 22 | +EWA sandwich |
| 23 | +SPD covariance operations |
| 24 | +Mat4 / Sym4 / Block4 carriers |
| 25 | +SIMD HHTL scoring kernels |
| 26 | +BLAS/SYRK/GEMM backend dispatch |
| 27 | +popcount / distance / palette lookup kernels |
| 28 | +quantization and codec helpers |
| 29 | +pillar probes and parity tests |
| 30 | +``` |
| 31 | + |
| 32 | +It does not own domain semantics. |
| 33 | + |
| 34 | +## Field kernel abstraction |
| 35 | + |
| 36 | +A field kernel is a compact local approximation of a larger signal. |
| 37 | + |
| 38 | +Examples: |
| 39 | + |
| 40 | +```text |
| 41 | +3DGS: |
| 42 | + anisotropic Gaussian over 3D position, color, opacity, covariance |
| 43 | +
|
| 44 | +Datalake: |
| 45 | + statistics / centroid / covariance / bloom over a fragment |
| 46 | +
|
| 47 | +RAG: |
| 48 | + semantic centroid + provenance + uncertainty over a chunk family |
| 49 | +
|
| 50 | +Ultrasound: |
| 51 | + PSF covariance + amplitude / Doppler / phase over a frame block |
| 52 | +
|
| 53 | +Genetics: |
| 54 | + motif / transition / expression kernel over a sequence window |
| 55 | +
|
| 56 | +Neuronal: |
| 57 | + activation / edge uncertainty kernel over a microcircuit block |
| 58 | +``` |
| 59 | + |
| 60 | +## Suggested module layout |
| 61 | + |
| 62 | +```text |
| 63 | +src/hpc/field_kernel/ |
| 64 | + mod.rs |
| 65 | + block.rs |
| 66 | + summary.rs |
| 67 | + certificate.rs |
| 68 | + decision_support.rs |
| 69 | + simd_score.rs |
| 70 | + blas_backend.rs |
| 71 | + block4.rs |
| 72 | +``` |
| 73 | + |
| 74 | +This can start as a plan only. Do not create a generic abstraction until two concrete consumers need it. |
| 75 | + |
| 76 | +## Core DTO sketches |
| 77 | + |
| 78 | +```rust |
| 79 | +pub struct FieldKernelSummary { |
| 80 | + pub block_len: usize, |
| 81 | + pub density: f32, |
| 82 | + pub energy: f32, |
| 83 | + pub variance: f32, |
| 84 | + pub max_error: f32, |
| 85 | + pub confidence: f32, |
| 86 | +} |
| 87 | + |
| 88 | +pub struct FieldKernelCertificate { |
| 89 | + pub total_error: f32, |
| 90 | + pub confidence: f32, |
| 91 | + pub passed: bool, |
| 92 | + pub reason_codes: Vec<FieldKernelReason>, |
| 93 | +} |
| 94 | + |
| 95 | +pub enum FieldKernelReason { |
| 96 | + BelowErrorBudget, |
| 97 | + AboveErrorBudget, |
| 98 | + NonPsdCovariance, |
| 99 | + QuantizationTooHigh, |
| 100 | + SamplingTooSparse, |
| 101 | + DependenceInflationHigh, |
| 102 | + BackendParityFailed, |
| 103 | +} |
| 104 | +``` |
| 105 | + |
| 106 | +## Relation to 3DGS |
| 107 | + |
| 108 | +3DGS remains the first hard implementation path: |
| 109 | + |
| 110 | +```text |
| 111 | +GaussianBatch / SplatBlockView |
| 112 | + -> EWA projection |
| 113 | + -> HHTL cascade |
| 114 | + -> field-kernel summary |
| 115 | + -> certificate |
| 116 | +``` |
| 117 | + |
| 118 | +Do not weaken `splat3d` into an over-generic module too early. Extract common pieces only after they repeat. |
| 119 | + |
| 120 | +## Relation to 4x4 carrier |
| 121 | + |
| 122 | +The 4x4 cognitive-shader carrier is the preferred experimental block grammar: |
| 123 | + |
| 124 | +```text |
| 125 | +lane0: coordinate / source |
| 126 | +lane1: state / covariance / transition |
| 127 | +lane2: signal / activation / confidence |
| 128 | +lane3: time / provenance / role |
| 129 | +``` |
| 130 | + |
| 131 | +The certified field-kernel substrate should be able to score and certify 4x4 block summaries, but it should not require every domain to become 4x4 on day one. |
| 132 | + |
| 133 | +## Backend types |
| 134 | + |
| 135 | +```text |
| 136 | +Native scalar reference |
| 137 | +Native SIMD |
| 138 | +BLAS / SYRK / GEMM |
| 139 | +AMX / BF16 experimental |
| 140 | +popcount / Hamming |
| 141 | +palette table lookup |
| 142 | +``` |
| 143 | + |
| 144 | +Every backend must have a reference path and parity tests. |
| 145 | + |
| 146 | +## Acceptance criteria |
| 147 | + |
| 148 | +- 3DGS code remains concrete and fast. |
| 149 | +- Reusable kernel/certificate pieces can be extracted without domain semantics. |
| 150 | +- Backend parity is testable. |
| 151 | +- Field-kernel certificates can be consumed by `lance-graph` decisions. |
| 152 | +- No domain-specific adapter logic is added to ndarray. |
| 153 | + |
| 154 | +## Implementation trigger |
| 155 | + |
| 156 | +Do not implement this as a big abstraction immediately. |
| 157 | + |
| 158 | +Trigger extraction when at least two of these are active: |
| 159 | + |
| 160 | +```text |
| 161 | +3DGS splat blocks |
| 162 | +Datalake HHTL summaries |
| 163 | +RAG retrieval blocks |
| 164 | +Ultrasound PSF blocks |
| 165 | +4x4 cognitive-shader blocks |
| 166 | +``` |
| 167 | + |
| 168 | +Until then, keep this as the north-star substrate plan. |
0 commit comments