Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
530 changes: 530 additions & 0 deletions .agent/research/sqlite/sqlite-vfs-ltx-redesign.md

Large diffs are not rendered by default.

44 changes: 44 additions & 0 deletions .agent/research/sqlite/v1-baseline-bench.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
{
"capturedAt": "2026-04-16T03:05:23.184Z",
"vfsVersion": "v1",
"source": "examples/sqlite-raw/scripts/benchmark.ts",
"pageSizeBytes": 4096,
"environment": {
"benchmarkHarness": "examples/sqlite-raw wrapper over rivetkit-sqlite-native/examples/v1_baseline_bench.rs",
"rttMs": 0,
"storage": "in-memory SqliteKv benchmark driver exercising the v1 native VFS",
"platform": "linux",
"release": "6.1.0-41-amd64",
"arch": "x64",
"cpuModel": "12th Gen Intel(R) Core(TM) i7-12700KF",
"cpuCount": 20,
"totalMemoryGiB": 62.56
},
"workloads": [
{
"name": "1 MiB insert",
"latencyMs": 3.614,
"roundTrips": 298
},
{
"name": "10 MiB insert",
"latencyMs": 25.892,
"roundTrips": 2606
},
{
"name": "hot-row update",
"latencyMs": 1.036,
"roundTrips": 109
},
{
"name": "cold read",
"latencyMs": 1.729,
"roundTrips": 228
},
{
"name": "mixed read/write",
"latencyMs": 0.824,
"roundTrips": 62
}
]
}
1 change: 1 addition & 0 deletions .claude/scheduled_tasks.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"sessionId":"cb4dbb44-01ef-4eef-91a8-ff5ad2f3e6fe","pid":1948414,"acquiredAt":1776308226245}
9 changes: 9 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,12 +202,16 @@ When the user asks to track something in a note, store it in `.agent/notes/` by

## Architecture

### Deprecated Packages
- `engine/packages/pegboard-runner/` and associated TypeScript "runner" packages (`engine/sdks/typescript/runner`, `rivetkit-typescript/packages/engine-runner/`) and runner workflows are deprecated. All new actor hosting work targets `engine/packages/pegboard-envoy/` exclusively. Do not add features to or fix bugs in the deprecated runner path.

### Monorepo Structure
- This is a Rust workspace-based monorepo for Rivet with the following key packages and components:

- **Core Engine** (`packages/core/engine/`) - Main orchestration service that coordinates all operations
- **Workflow Engine** (`packages/common/gasoline/`) - Handles complex multi-step operations with reliability and observability
- **Pegboard** (`packages/core/pegboard/`) - Actor/server lifecycle management system
- **Pegboard Envoy** (`engine/packages/pegboard-envoy/`) - The active actor-to-engine bridge. All new actor hosting work goes here.
- **Common Packages** (`/packages/common/`) - Foundation utilities, database connections, caching, metrics, logging, health checks, workflow engine core
- **Core Packages** (`/packages/core/`) - Main engine executable, Pegboard actor orchestration, workflow workers
- **Shared Libraries** (`shared/{language}/{package}/`) - Libraries shared between the engine and rivetkit (e.g., `shared/typescript/virtual-websocket/`)
Expand Down Expand Up @@ -279,6 +283,11 @@ let error_with_meta = ApiRateLimited { limit: 100, reset_at: 1234567890 }.build(
**Native SQLite & KV Channel**
- RivetKit SQLite is served by `@rivetkit/rivetkit-native`. Do not reintroduce SQLite-over-KV or WebAssembly SQLite paths in the TypeScript runtime.
- The Rust KV-backed SQLite implementation still lives in `rivetkit-typescript/packages/sqlite-native/src/`. When changing its on-disk or KV layout, update the internal data-channel spec in the same change.
- The native VFS uses the same 4 KiB chunk layout and KV key encoding as the WASM VFS. Data is compatible between backends.
- **The native Rust VFS and the WASM TypeScript VFS must match 1:1.** This includes: KV key layout and encoding, chunk size, PRAGMA settings, VFS callback-to-KV-operation mapping, delete/truncate strategy (both must use `deleteRange`), and journal mode. When changing any VFS behavior in one implementation, update the other. The relevant files are:
- Native: `rivetkit-typescript/packages/sqlite-native/src/vfs.rs`, `kv.rs`
- WASM: `rivetkit-typescript/packages/sqlite-wasm/src/vfs.ts`, `kv.ts`
- SQLite VFS v2 storage keys use literal ASCII path segments under the `0x02` subspace prefix with big-endian numeric suffixes so `scan_prefix` and `BTreeMap` ordering stay numerically correct.
- Full spec: `docs-internal/engine/NATIVE_SQLITE_DATA_CHANNEL.md`

**Inspector HTTP API**
Expand Down
36 changes: 36 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ members = [
"engine/packages/runner-protocol",
"engine/packages/runtime",
"engine/packages/service-manager",
"engine/packages/sqlite-storage",
"engine/packages/telemetry",
"engine/packages/test-deps",
"engine/packages/test-deps-docker",
Expand Down Expand Up @@ -107,6 +108,7 @@ members = [
once_cell = "1.20.2"
opentelemetry-appender-tracing = "0.28.1"
papaya = "0.2.1"
parking_lot = "0.12"
pest_derive = "2.7"
portpicker = "0.1"
prettyplease = "0.2"
Expand Down Expand Up @@ -456,6 +458,9 @@ members = [
[workspace.dependencies.rivet-runtime]
path = "engine/packages/runtime"

[workspace.dependencies.sqlite-storage]
path = "engine/packages/sqlite-storage"

[workspace.dependencies.rivet-service-manager]
path = "engine/packages/service-manager"

Expand Down
Loading
Loading