ndarrow

A zero-copy bridge between Apache Arrow and ndarray. Convert Arrow arrays to ndarray views and back without allocation overhead.

ndarrow is under active development. Public APIs will change until v1. Pin your version.

Install

[dependencies]
ndarrow = "0.0.4"

What It Does

ndarrow lets you move data between Arrow and ndarray with zero allocations on the bridge path:

• Arrow to ndarray: borrow Arrow buffers as ndarray views (O(1), no copy)
• ndarray to Arrow: transfer owned ndarray buffers into Arrow arrays (O(1), ownership move)

This enables Arrow-native systems (DataFusion, Flight, IPC) to leverage ndarray-native numerical libraries (like nabled) without paying for serialization or conversion.

Quick Example

use arrow_array::{Float64Array, FixedSizeListArray};
use ndarrow::{AsNdarray, IntoArrow};
use ndarray::Array1;

// Arrow -> ndarray (zero-copy view)
let arrow_array = Float64Array::from(vec![1.0, 2.0, 3.0, 4.0]);
let view = arrow_array.as_ndarray()?;  // ArrayView1<f64>, no allocation
assert_eq!(view[0], 1.0);

// ndarray -> Arrow (zero-copy ownership transfer)
let result = Array1::from_vec(vec![5.0, 6.0, 7.0, 8.0]);
let arrow_result = result.into_arrow()?;  // PrimitiveArray<Float64>, no allocation

Type Mappings

Arrow Type	ndarray Type	Direction	Copy?
PrimitiveArray<T>	ArrayView1<T>	Arrow -> ndarray	Zero-copy
FixedSizeList<T>(D)	ArrayView2<T> (M, D)	Arrow -> ndarray	Zero-copy
arrow.fixed_shape_tensor	ArrayViewD<T>	Arrow -> ndarray	Zero-copy
arrow.variable_shape_tensor	VariableShapeTensorBatchView<T> (row() -> ArrayViewD<T>)	Arrow -> ndarray	Zero-copy
ndarrow.csr_matrix_batch	CsrMatrixBatchView<T> (row() -> CsrView<T>)	Arrow -> ndarray	Zero-copy
FixedSizeList<ndarrow.complex32>(D)	ArrayView2<Complex32>	Arrow -> ndarray	Zero-copy
FixedSizeList<ndarrow.complex64>(D)	ArrayView2<Complex64>	Arrow -> ndarray	Zero-copy
arrow.fixed_shape_tensor<ndarrow.complex32>	ArrayViewD<Complex32>	Arrow -> ndarray	Zero-copy
arrow.fixed_shape_tensor<ndarrow.complex64>	ArrayViewD<Complex64>	Arrow -> ndarray	Zero-copy
ndarrow.csr_matrix	CsrView<T>	Arrow -> ndarray	Zero-copy
Array1<T>	PrimitiveArray<T>	ndarray -> Arrow	Zero-copy
Array2<T> (M, N)	FixedSizeList<T>(N)	ndarray -> Arrow	Zero-copy*
ArrayD<T>	arrow.fixed_shape_tensor	ndarray -> Arrow	Zero-copy*
Array2<Complex32>	FixedSizeList<ndarrow.complex32>(N)	ndarray -> Arrow	Zero-copy*
Array2<Complex64>	FixedSizeList<ndarrow.complex64>(N)	ndarray -> Arrow	Zero-copy*
ArrayD<Complex32>	arrow.fixed_shape_tensor<ndarrow.complex32>	ndarray -> Arrow	Zero-copy*
ArrayD<Complex64>	arrow.fixed_shape_tensor<ndarrow.complex64>	ndarray -> Arrow	Zero-copy*

* Zero-copy if standard (C-contiguous) layout. Allocates a layout copy otherwise.

Supported Element Types

• f32, f64 (first-class)
• Additional types via the NdarrowElement trait

Null Handling

Null semantics are explicit at the call site:

// Validated: returns Err if nulls present (O(1) check)
let view = array.as_ndarray()?;

// Unchecked: caller guarantees no nulls (zero cost)
let view = unsafe { array.as_ndarray_unchecked() };

// Masked: returns view + validity bitmap (zero allocation)
let (view, mask) = array.as_ndarray_masked();

For numerical object payloads such as FixedSizeList<T>(D) -> ArrayView2<T>, the masked path is row-oriented: it may return an outer row-validity bitmap, but actual inner component nulls are still rejected on validated and masked ingress. Ragged tensor and batched CSR carriers now expose column-level batch views through variable_shape_tensor_batch_view and csr_matrix_batch_view; those views preserve the outer validity bitmap and provide row() / iter() accessors, while variable_shape_tensor_iter_masked and csr_matrix_batch_iter_masked remain convenience wrappers.

Extension Types

ndarrow uses canonical Arrow extension types where they exist:

• arrow.fixed_shape_tensor — fixed-shape multi-dimensional data
• arrow.variable_shape_tensor — variable-shape data (e.g., multi-vectors)

And defines its own for gaps:

• ndarrow.csr_matrix — CSR sparse matrix representation
• ndarrow.complex32 — complex32 scalar element representation
• ndarrow.complex64 — complex64 scalar element representation

Performance Guarantee

Bridge conversions are O(1) regardless of array size. The bridge creates views (pointer + shape) or transfers buffer ownership. It never touches the data. See docs/PERFORMANCE_CONTRACTS.md for the full allocation contract.

Quality Gates

just checks

License

Licensed under the Apache License, Version 2.0.

See LICENSE for details.