Skip to content

Commit 4a2d831

Browse files
committed
Add blackboard.md: single-binary architecture inventory
Documents what exists in this repo and how it maps to the polyglot notebook single-binary architecture (Rust transcode plan). https://claude.ai/code/session_01MxwpeMKtXURCsr4SG4yfkX
1 parent 9da4836 commit 4a2d831

1 file changed

Lines changed: 75 additions & 0 deletions

File tree

blackboard.md

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# Blackboard — ndarray
2+
3+
> Single-binary architecture: already Rust. Integrates as `crate::simd` + `crate::linalg`.
4+
5+
## What Exists
6+
7+
Production-grade N-dimensional array library with three-tier SIMD dispatch (AVX-512 → AVX2 → Scalar), pluggable BLAS backends (Native/MKL/OpenBLAS), and 55 HPC extension modules.
8+
9+
## Core Data Structure
10+
11+
```rust
12+
pub struct ArrayBase<S, D, A = <S as RawData>::Elem> {
13+
data: S, // Ownership: Owned, View, ArcArray, CowArray
14+
parts: ArrayPartsSized<A, D>, // ptr + dim + strides
15+
}
16+
17+
// Type aliases
18+
type Array<A, D> = ArrayBase<OwnedRepr<A>, D>; // Owned
19+
type ArrayView<'a, A, D> = ArrayBase<ViewRepr<&'a A>, D>; // Read-only view
20+
```
21+
22+
## SIMD Dispatch
23+
24+
```
25+
LazyLock<Tier> detected once at first call:
26+
AVX-512F → Tier::Avx512 (F32x16, F64x8)
27+
AVX2+FMA → Tier::Avx2 (F32x8, F64x4)
28+
Fallback → Tier::Scalar
29+
```
30+
31+
dispatch! macro generates one-line stubs per function.
32+
33+
## BLAS Operations
34+
35+
### Level 1 (Vector-Vector)
36+
`dot_f32/f64`, `axpy_f32/f64`, `scal_f32/f64`, `nrm2_f32/f64`, `asum_f32/f64`
37+
38+
### Level 2 (Matrix-Vector)
39+
`gemv_f32/f64`, `ger_f32/f64`
40+
41+
### Level 3 (Matrix-Matrix)
42+
`gemm_f32/f64` via `matrixmultiply` crate (Goto BLAS kernel)
43+
44+
## HPC Extensions (`src/hpc/`, 55 modules)
45+
46+
| Module | Purpose |
47+
|---|---|
48+
| `blas_level1/2/3.rs` | BLAS trait extensions |
49+
| `statistics.rs` | median, variance, percentiles |
50+
| `activations.rs` | sigmoid, softmax, relu |
51+
| `fft.rs` | Cooley-Tukey FFT |
52+
| `fingerprint.rs` | 32/256/512-bit containers |
53+
| `cascade.rs` | Hamming distance bands |
54+
| `nars.rs` | NARS reasoning |
55+
| `arrow_bridge.rs` | Apache Arrow integration |
56+
| `clam.rs` | Hierarchical clustering |
57+
58+
## Integration Points for Binary
59+
60+
- lance-graph's BlasGraph calls ndarray for SIMD Hamming distance
61+
- Query result DataFrames use Arrow bridge
62+
- Fingerprint/cascade search for semantic retrieval
63+
64+
## Key Files
65+
66+
| File | Size | Purpose |
67+
|---|---|---|
68+
| `src/lib.rs` | 66KB | ArrayBase definition, exports |
69+
| `src/backend/mod.rs` | 5KB | BlasFloat trait, backend selection |
70+
| `src/backend/native.rs` | 23KB | SIMD dispatch, BLAS L1/L2 |
71+
| `src/backend/kernels_avx512.rs` | 29KB | AVX-512 intrinsics |
72+
| `src/simd_avx512.rs` | 39KB | SIMD wrapper types |
73+
| `src/simd.rs` | 29KB | Public SIMD API |
74+
| `src/hpc/` | 42KB | 55 HPC extension modules |
75+
| `src/impl_methods.rs` | 125KB | Core array methods |

0 commit comments

Comments
 (0)