|
| 1 | +# Blackboard — ndarray |
| 2 | + |
| 3 | +> Single-binary architecture: already Rust. Integrates as `crate::simd` + `crate::linalg`. |
| 4 | +
|
| 5 | +## What Exists |
| 6 | + |
| 7 | +Production-grade N-dimensional array library with three-tier SIMD dispatch (AVX-512 → AVX2 → Scalar), pluggable BLAS backends (Native/MKL/OpenBLAS), and 55 HPC extension modules. |
| 8 | + |
| 9 | +## Core Data Structure |
| 10 | + |
| 11 | +```rust |
| 12 | +pub struct ArrayBase<S, D, A = <S as RawData>::Elem> { |
| 13 | + data: S, // Ownership: Owned, View, ArcArray, CowArray |
| 14 | + parts: ArrayPartsSized<A, D>, // ptr + dim + strides |
| 15 | +} |
| 16 | + |
| 17 | +// Type aliases |
| 18 | +type Array<A, D> = ArrayBase<OwnedRepr<A>, D>; // Owned |
| 19 | +type ArrayView<'a, A, D> = ArrayBase<ViewRepr<&'a A>, D>; // Read-only view |
| 20 | +``` |
| 21 | + |
| 22 | +## SIMD Dispatch |
| 23 | + |
| 24 | +``` |
| 25 | +LazyLock<Tier> detected once at first call: |
| 26 | + AVX-512F → Tier::Avx512 (F32x16, F64x8) |
| 27 | + AVX2+FMA → Tier::Avx2 (F32x8, F64x4) |
| 28 | + Fallback → Tier::Scalar |
| 29 | +``` |
| 30 | + |
| 31 | +dispatch! macro generates one-line stubs per function. |
| 32 | + |
| 33 | +## BLAS Operations |
| 34 | + |
| 35 | +### Level 1 (Vector-Vector) |
| 36 | +`dot_f32/f64`, `axpy_f32/f64`, `scal_f32/f64`, `nrm2_f32/f64`, `asum_f32/f64` |
| 37 | + |
| 38 | +### Level 2 (Matrix-Vector) |
| 39 | +`gemv_f32/f64`, `ger_f32/f64` |
| 40 | + |
| 41 | +### Level 3 (Matrix-Matrix) |
| 42 | +`gemm_f32/f64` via `matrixmultiply` crate (Goto BLAS kernel) |
| 43 | + |
| 44 | +## HPC Extensions (`src/hpc/`, 55 modules) |
| 45 | + |
| 46 | +| Module | Purpose | |
| 47 | +|---|---| |
| 48 | +| `blas_level1/2/3.rs` | BLAS trait extensions | |
| 49 | +| `statistics.rs` | median, variance, percentiles | |
| 50 | +| `activations.rs` | sigmoid, softmax, relu | |
| 51 | +| `fft.rs` | Cooley-Tukey FFT | |
| 52 | +| `fingerprint.rs` | 32/256/512-bit containers | |
| 53 | +| `cascade.rs` | Hamming distance bands | |
| 54 | +| `nars.rs` | NARS reasoning | |
| 55 | +| `arrow_bridge.rs` | Apache Arrow integration | |
| 56 | +| `clam.rs` | Hierarchical clustering | |
| 57 | + |
| 58 | +## Integration Points for Binary |
| 59 | + |
| 60 | +- lance-graph's BlasGraph calls ndarray for SIMD Hamming distance |
| 61 | +- Query result DataFrames use Arrow bridge |
| 62 | +- Fingerprint/cascade search for semantic retrieval |
| 63 | + |
| 64 | +## Key Files |
| 65 | + |
| 66 | +| File | Size | Purpose | |
| 67 | +|---|---|---| |
| 68 | +| `src/lib.rs` | 66KB | ArrayBase definition, exports | |
| 69 | +| `src/backend/mod.rs` | 5KB | BlasFloat trait, backend selection | |
| 70 | +| `src/backend/native.rs` | 23KB | SIMD dispatch, BLAS L1/L2 | |
| 71 | +| `src/backend/kernels_avx512.rs` | 29KB | AVX-512 intrinsics | |
| 72 | +| `src/simd_avx512.rs` | 39KB | SIMD wrapper types | |
| 73 | +| `src/simd.rs` | 29KB | Public SIMD API | |
| 74 | +| `src/hpc/` | 42KB | 55 HPC extension modules | |
| 75 | +| `src/impl_methods.rs` | 125KB | Core array methods | |
0 commit comments