Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
af1ec9f
chore: update gitignore and remove dead return-borrowed docs
prestwich Mar 28, 2026
24312c9
bench: add cursor write operation benchmarks
prestwich Mar 28, 2026
44608bb
bench: add reserve, nested_txn, concurrent, and scaling benchmarks
prestwich Mar 28, 2026
dc9c795
test: add proptest_kv, proptest_cursor, proptest_dupsort with migrate…
prestwich Mar 28, 2026
0d1eafa
test: add proptest_dupfixed, proptest_iter, proptest_nested
prestwich Mar 28, 2026
287cfc1
test: complete proptest migration, delete proptest_inputs.rs
prestwich Mar 28, 2026
5c37118
fuzz: add cargo-fuzz setup with 6 fuzz targets
prestwich Mar 28, 2026
7371572
bench: align benchmarks with evmdb for cross-project parity
prestwich Mar 28, 2026
5dfe490
docs: document input validation model, update file layout
prestwich Mar 28, 2026
a48a4f2
docs: document input validation model in README and lib.rs
prestwich Mar 28, 2026
1f4ef67
bench: align cursor traversal benchmarks with evmdb
prestwich Mar 28, 2026
88204dc
bench: add commit cost isolation benchmark
prestwich Mar 28, 2026
965aaf6
bench: mark parity benchmarks as do-not-edit
prestwich Mar 28, 2026
97dd1dd
bench: add quick mode and skip slow benchmarks by default
prestwich Mar 30, 2026
6507b22
bench: deduplicate skip warnings with Once
prestwich Mar 30, 2026
2ac4f42
bench: cold-read parity benchmarks and encoding alignment
prestwich Mar 31, 2026
713e1fd
bench: fix concurrent reader benchmarks for 128-reader parity
prestwich Mar 31, 2026
16ea2cc
bench: add committed parity benchmarks with durable and nosync modes
prestwich Apr 1, 2026
aad9d1f
bench: align parity benchmarks with evmdb counterparts
prestwich Apr 1, 2026
09e8605
fix: rustfmt in scaling bench
prestwich Apr 1, 2026
840312a
bench: remove 4096B from parity value sizes
prestwich Apr 1, 2026
9ac0c51
bench: deterministic cold reads via close→fadvise→reopen
prestwich Apr 1, 2026
7411996
refactor: remove evmdb/parity references from benchmarks
prestwich Apr 2, 2026
9faaf71
chore: bump version to 0.8.2
prestwich Apr 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,8 @@
.DS_Store
Cargo.lock
.idea/
mdbx-sys/target/
mdbx-sys/target/
docs/
fuzz/corpus/
fuzz/artifacts/
fuzz/target/
80 changes: 75 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,31 @@ DUP_SORT/DUP_FIXED methods validate flags at runtime:

- `require_dup_sort()` returns `MdbxError::RequiresDupSort`
- `require_dup_fixed()` returns `MdbxError::RequiresDupFixed`
- `debug_assert_integer_key()` validates key length (4 or 8 bytes) in debug builds

Methods requiring DUP_SORT: `first_dup`, `last_dup`, `next_dup`, `prev_dup`, `get_both`, `get_both_range`
Methods requiring DUP_FIXED: `get_multiple`, `next_multiple`, `prev_multiple`

### Input Validation Model

MDBX's C layer aborts the process (via `cASSERT`) on certain constraint
violations — notably INTEGER_KEY size mismatches and oversized keys/values.
These aborts cannot be caught.

Our validation model (in `src/tx/assertions.rs`):

- **Debug builds:** `debug_assert` checks catch constraint violations in
Rust before they reach FFI. This includes key/value size limits, INTEGER_KEY
length (must be 4 or 8 bytes), INTEGER_DUP length, and append ordering.
- **Release builds:** No checks are performed. Invalid input passes through
to MDBX, which may abort the process.
- **Benchmarks and fuzz targets:** MUST constrain inputs to valid ranges.
Do not feed arbitrary-length keys to INTEGER_KEY databases or oversized
keys/values to any database. The fuzz/bench harness is responsible for
generating valid input, not the library.

This is intentional. The library trusts callers in release mode for
performance. The debug assertions exist to catch bugs during development.

### Error Types

- `MdbxError` - FFI/database errors (in `src/error.rs`)
Expand All @@ -68,6 +88,7 @@ src/
codec.rs - TableObject trait
tx/
mod.rs
assertions.rs - Debug assertions for key/value constraints
cursor.rs - Cursor impl
database.rs - Database struct
sync.rs - Transaction impl
Expand All @@ -76,14 +97,29 @@ src/
sys/
environment.rs - Environment impl
tests/
cursor.rs - Cursor tests
transaction.rs - Transaction tests
environment.rs - Environment tests
cursor.rs - Cursor tests
transaction.rs - Transaction tests
environment.rs - Environment tests
proptest_kv.rs - Property tests: key/value operations
proptest_cursor.rs - Property tests: cursor operations
proptest_dupsort.rs - Property tests: DUPSORT operations
proptest_dupfixed.rs - Property tests: DUPFIXED operations
proptest_iter.rs - Property tests: iterator operations
proptest_nested.rs - Property tests: nested transactions
benches/
cursor.rs - Cursor benchmarks
cursor.rs - Cursor read benchmarks
cursor_write.rs - Cursor write benchmarks (PARITY: evmdb)
transaction.rs - Transaction benchmarks
db_open.rs - Database open benchmarks
reserve.rs - Reserve vs put benchmarks
nested_txn.rs - Nested transaction benchmarks
concurrent.rs - Concurrency benchmarks (PARITY: evmdb)
scaling.rs - Scaling benchmarks (PARITY: evmdb)
deletion.rs - Deletion benchmarks
iter.rs - Iterator benchmarks
utils.rs - Benchmark utilities
fuzz/
fuzz_targets/ - cargo-fuzz targets (FFI/unsafe boundary hardening)
```

## Testing
Expand All @@ -109,3 +145,37 @@ This SHOULD be run alongside local tests and linting, especially for changes tha
- Modify build configuration
- Add new dependencies
- Change platform-specific code

## Parity Benchmarks

Parity-tagged benchmarks (`// PARITY:`) have evmdb equivalents with
identical parameters for cross-project comparison. Both repos use
identical 32-byte binary keys and 128-byte binary values via
`parity_key`/`parity_value` in `benches/utils.rs`.

```bash
# Run all parity benchmarks (native macOS or Linux)
cargo bench --bench cursor_write -- "cursor_write::put::sync|cursor_write::append::sync"
cargo bench --bench cursor -- "cursor::traverse::iter$"
cargo bench --bench concurrent -- "readers_no_writer|readers_one_writer"
cargo bench --bench scaling -- "sequential_get|random_get|full_iteration|append_ordered_put"

# Cold-read parity benchmarks (Linux only — posix_fadvise is a no-op on macOS)
cargo bench --bench scaling -- "cold_random_get|cold_sequential_scan"
```

### Parity bench list

| Bench | File | evmdb counterpart |
|-------|------|-------------------|
| `cursor_write::put::sync` | cursor_write.rs | write_put_100 |
| `cursor_write::append::sync` | cursor_write.rs | write_put_100_sorted |
| `cursor::traverse::iter` | cursor.rs | cursor_seek_first_iterate |
| `readers_no_writer` | concurrent.rs | readers_no_writer |
| `readers_one_writer` | concurrent.rs | readers_with_writer |
| `sequential_get` | scaling.rs | sequential_get |
| `random_get` | scaling.rs | random_get |
| `full_iteration` | scaling.rs | full_iteration |
| `append_ordered_put` | scaling.rs | put_sorted |
| `cold_random_get` | scaling.rs | cold_random_get |
| `cold_sequential_scan` | scaling.rs | cold_sequential_scan |
23 changes: 22 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]
name = "signet-libmdbx"
description = "Idiomatic and safe MDBX wrapper"
version = "0.8.1"
version = "0.8.2"
edition = "2024"
rust-version = "1.92"
license = "MIT OR Apache-2.0"
Expand Down Expand Up @@ -45,6 +45,7 @@ tracing = "0.1.44"

[dev-dependencies]
criterion = "0.8.1"
libc = "0.2"
proptest = "1"
rand = "0.9.2"
tempfile = "3.20.0"
Expand All @@ -68,3 +69,23 @@ harness = false
[[bench]]
name = "deletion"
harness = false

[[bench]]
name = "cursor_write"
harness = false

[[bench]]
name = "reserve"
harness = false

[[bench]]
name = "nested_txn"
harness = false

[[bench]]
name = "concurrent"
harness = false

[[bench]]
name = "scaling"
harness = false
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,24 @@ NOTE: Most of the repo came from [lmdb-rs bindings].
- `sys` - Environment and transaction management.
- `tx` - module contains transactions, cursors, and iterators

## Input Validation

MDBX's C layer **aborts the process** on certain constraint violations,
such as passing an invalid key size to an `INTEGER_KEY` database or
exceeding the maximum key/value size. These aborts cannot be caught.

This crate uses a **debug-only validation model**:

- **Debug builds** (`cfg(debug_assertions)`): Rust-side assertions check
key/value constraints before they reach FFI. Violations panic with a
descriptive message, catching bugs during development.
- **Release builds**: No validation is performed. Invalid input passes
directly to MDBX for maximum performance.

**Callers are responsible for ensuring inputs are valid in release
builds.** The debug assertions exist to catch bugs during development,
not to provide runtime safety guarantees.

## Updating the libmdbx Version

To update the libmdbx version you must clone it and copy the `dist/` folder in
Expand Down
183 changes: 183 additions & 0 deletions benches/concurrent.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
#![allow(missing_docs, dead_code)]
mod utils;

use criterion::{BatchSize, BenchmarkId, Criterion, criterion_group, criterion_main};
use signet_libmdbx::{ObjectLength, WriteFlags};
use std::{
borrow::Cow,
hint::black_box,
sync::{Arc, Barrier},
thread,
};
use utils::{bench_key, bench_value, quick_config, setup_bench_env_with_max_readers};

const N_ROWS: u32 = 1_000;
const READER_COUNTS: &[usize] = &[1, 4, 8, 32, 128];

/// Max readers set high enough for the largest reader count plus criterion
/// overhead threads.
const MAX_READERS: u64 = 256;

fn bench_n_readers_no_writer(c: &mut Criterion) {
let mut group = c.benchmark_group("concurrent::readers_no_writer");

for &n_readers in READER_COUNTS {
let (_dir, env) = setup_bench_env_with_max_readers(N_ROWS, Some(MAX_READERS));
let env = Arc::new(env);
let keys: Arc<Vec<[u8; 32]>> = Arc::new((0..N_ROWS).map(bench_key).collect());
// Open the db handle once — dbi is stable for the environment lifetime.
let db = {
let txn = env.begin_ro_sync().unwrap();
txn.open_db(None).unwrap()
};

group.bench_with_input(
BenchmarkId::from_parameter(n_readers),
&n_readers,
|b, &n_readers| {
b.iter_batched(
|| Arc::new(Barrier::new(n_readers + 1)),
|barrier| {
let handles: Vec<_> = (0..n_readers)
.map(|_| {
let env = Arc::clone(&env);
let keys = Arc::clone(&keys);
let barrier = Arc::clone(&barrier);
thread::spawn(move || {
let txn = env.begin_ro_sync().unwrap();
barrier.wait();
let mut total = 0usize;
for key in keys.iter() {
let val: Cow<'_, [u8]> =
txn.get(db.dbi(), key.as_slice()).unwrap().unwrap();
total += val.len();
}
black_box(total)
})
})
.collect();
barrier.wait();
handles.into_iter().for_each(|h| {
h.join().unwrap();
});
},
BatchSize::PerIteration,
)
},
);
}
group.finish();
}

fn bench_n_readers_one_writer(c: &mut Criterion) {
let mut group = c.benchmark_group("concurrent::readers_one_writer");

for &n_readers in READER_COUNTS {
let (_dir, env) = setup_bench_env_with_max_readers(N_ROWS, Some(MAX_READERS));
let env = Arc::new(env);
let keys: Arc<Vec<[u8; 32]>> = Arc::new((0..N_ROWS).map(bench_key).collect());
// Open the db handle once — dbi is stable for the environment lifetime.
let db = {
let txn = env.begin_ro_sync().unwrap();
txn.open_db(None).unwrap()
};

group.bench_with_input(
BenchmarkId::from_parameter(n_readers),
&n_readers,
|b, &n_readers| {
b.iter_batched(
|| Arc::new(Barrier::new(n_readers + 2)),
|barrier| {
// Spawn readers.
let reader_handles: Vec<_> = (0..n_readers)
.map(|_| {
let env = Arc::clone(&env);
let keys = Arc::clone(&keys);
let barrier = Arc::clone(&barrier);
thread::spawn(move || {
let txn = env.begin_ro_sync().unwrap();
barrier.wait();
let mut total = 0usize;
for key in keys.iter() {
let val: Cow<'_, [u8]> =
txn.get(db.dbi(), key.as_slice()).unwrap().unwrap();
total += val.len();
}
black_box(total)
})
})
.collect();

// Spawn one writer inserting one extra entry.
let writer = {
let env = Arc::clone(&env);
let barrier = Arc::clone(&barrier);
thread::spawn(move || {
barrier.wait();
let txn = env.begin_rw_sync().unwrap();
txn.put(
db,
bench_key(N_ROWS + 1),
bench_value(N_ROWS + 1),
WriteFlags::empty(),
)
.unwrap();
txn.commit().unwrap();
})
};

barrier.wait();
writer.join().unwrap();
reader_handles.into_iter().for_each(|h| {
h.join().unwrap();
});
},
BatchSize::PerIteration,
)
},
);
}
group.finish();
}

/// Single-thread comparison: sync vs unsync transaction creation.
fn bench_single_thread_sync_vs_unsync(c: &mut Criterion) {
let (_dir, env) = setup_bench_env_with_max_readers(N_ROWS, None);
let keys: Arc<Vec<[u8; 32]>> = Arc::new((0..N_ROWS).map(bench_key).collect());

c.bench_function("concurrent::single_thread::sync", |b| {
b.iter(|| {
let txn = env.begin_ro_sync().unwrap();
let db = txn.open_db(None).unwrap();
let mut total = 0usize;
for key in keys.iter() {
total += *txn.get::<ObjectLength>(db.dbi(), key.as_slice()).unwrap().unwrap();
}
black_box(total)
})
});

c.bench_function("concurrent::single_thread::unsync", |b| {
b.iter(|| {
let txn = env.begin_ro_unsync().unwrap();
let db = txn.open_db(None).unwrap();
let mut total = 0usize;
for key in keys.iter() {
total += *txn.get::<ObjectLength>(db.dbi(), key.as_slice()).unwrap().unwrap();
}
black_box(total)
})
});
}

criterion_group! {
name = benches;
config = quick_config();
targets =
bench_n_readers_no_writer,
bench_n_readers_one_writer,
bench_single_thread_sync_vs_unsync,
}

criterion_main!(benches);
Loading