feat: add persisted sync metadata support by samwillis · Pull Request #1380 · TanStack/db

samwillis · 2026-03-17T15:05:56Z

Summary

add transactional persisted sync metadata support across @tanstack/db, db-sqlite-persisted-collection-core, query-db-collection, and electric-db-collection
persist row-scoped metadata with synced rows and collection-scoped metadata alongside persisted collection state
use that metadata to preserve query ownership/retention across restart and to persist Electric resume state

How To Use

Custom sync implementations

If a collection is running on a persistence layer that supports this feature, the sync function now receives an optional metadata API alongside begin, write, commit, markReady, and truncate.

sync: ({ begin, write, commit, metadata }) => {
  const startupValue = metadata?.collection.get('my-sync:startup')

  begin()
  write({
    type: 'update',
    value: row,
    metadata: { source: 'remote' },
  })
  metadata?.collection.set('my-sync:startup', { ready: true })
  commit()
}

Available operations:

metadata.row.get(key)
metadata.row.set(key, value)
metadata.row.delete(key)
metadata.collection.get(key)
metadata.collection.set(key, value)
metadata.collection.delete(key)
metadata.collection.list(prefix?)

Behavior:

startup reads through get / list are allowed before any transaction is opened
metadata writes are transactional and must happen inside begin() / commit()
write({ metadata }) and metadata.row.set() target the same row metadata store

SQLite persisted collections

persistedCollectionOptions(...) now persists both row metadata and collection metadata when used with db-sqlite-persisted-collection-core.

Typical usage looks like:

const collection = createCollection(
  persistedCollectionOptions({
    ...someCollectionOptions,
    persistence: {
      adapter: sqliteAdapter,
    },
  }),
)

What you get automatically:

row metadata hydrates with persisted rows
collection metadata is loaded during startup before persisted sync work begins
row data, row metadata, and collection metadata commit in the same persisted transaction

Query collections

Query collections can now retain persisted ownership metadata across restarts.

const collection = createCollection(
  persistedCollectionOptions({
    ...queryCollectionOptions({
      id: 'messages',
      queryClient,
      queryKey: ['messages', roomId],
      queryFn: fetchMessages,
      getKey: (row) => row.id,
      persistedGcTime: Number.POSITIVE_INFINITY,
    }),
    persistence: {
      adapter: sqliteAdapter,
    },
  }),
)

What this enables:

row ownership is persisted on rows, so warm starts do not incorrectly delete disjoint pre-hydrated rows
query retention is separate from in-memory TanStack Query gcTime
persistedGcTime can be finite or effectively indefinite for offline-first flows
when a retained query is requested again, reconciliation diffing uses that query's owned-row baseline instead of the whole collection

Electric collections

Electric collections now persist resume metadata at collection scope when wrapped with persisted collection options.

const collection = createCollection(
  persistedCollectionOptions({
    ...electricCollectionOptions({
      id: 'todos',
      getKey: (row) => row.id,
      shapeOptions: {
        url: electricUrl,
        params: { table: 'todos' },
      },
    }),
    persistence: {
      adapter: sqliteAdapter,
    },
  }),
)

What this enables:

persisted startup can reuse a saved Electric offset / handle resume point
resume metadata is committed transactionally with the batch that made it valid
must-refetch/reset paths write a reset marker before reloading so restart does not resume from stale stream state

Test plan

Run pnpm vitest --run packages/db/tests/collection.test.ts
Run pnpm --filter @tanstack/query-db-collection exec vitest run tests/query.test.ts --coverage.enabled false
Run pnpm --filter @tanstack/electric-db-collection exec vitest run tests/electric.test.ts --coverage.enabled false
Run pnpm --filter @tanstack/db-sqlite-persisted-collection-core exec vitest run tests/persisted.test.ts tests/sqlite-core-adapter.test.ts --coverage.enabled false

Notes

The persisted SQLite package tests are functionally green in package-local runs.
There is still noisy stderr in some query tests from expected error-path coverage and existing cleanupQueryIfIdle warnings.

pkg-pr-new · 2026-03-17T15:09:26Z

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/angular-db@1380

@tanstack/db

npm i https://pkg.pr.new/TanStack/db/@tanstack/db@1380

@tanstack/db-browser-wa-sqlite-persisted-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/db-browser-wa-sqlite-persisted-collection@1380

@tanstack/db-ivm

npm i https://pkg.pr.new/TanStack/db/@tanstack/db-ivm@1380

@tanstack/db-react-native-sqlite-persisted-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/db-react-native-sqlite-persisted-collection@1380

@tanstack/db-sqlite-persisted-collection-core

npm i https://pkg.pr.new/TanStack/db/@tanstack/db-sqlite-persisted-collection-core@1380

@tanstack/electric-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/electric-db-collection@1380

@tanstack/offline-transactions

npm i https://pkg.pr.new/TanStack/db/@tanstack/offline-transactions@1380

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/powersync-db-collection@1380

@tanstack/query-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/query-db-collection@1380

@tanstack/react-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/react-db@1380

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/rxdb-db-collection@1380

@tanstack/solid-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/solid-db@1380

@tanstack/svelte-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/svelte-db@1380

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/trailbase-db-collection@1380

@tanstack/vue-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/vue-db@1380

commit: a1a2f2e

github-actions · 2026-03-17T15:10:08Z

Size Change: +504 B (+0.46%)

Total Size: 111 kB

Filename	Size	Change
`./packages/db/dist/esm/collection/state.js`	5.26 kB	+54 B (+1.04%)
`./packages/db/dist/esm/collection/sync.js`	2.88 kB	+450 B (+18.5%)	⚠️

ℹ️ View Unchanged

Filename	Size
`./packages/db/dist/esm/collection/change-events.js`	1.39 kB
`./packages/db/dist/esm/collection/changes.js`	1.38 kB
`./packages/db/dist/esm/collection/cleanup-queue.js`	810 B
`./packages/db/dist/esm/collection/events.js`	434 B
`./packages/db/dist/esm/collection/index.js`	3.69 kB
`./packages/db/dist/esm/collection/indexes.js`	2.35 kB
`./packages/db/dist/esm/collection/lifecycle.js`	1.76 kB
`./packages/db/dist/esm/collection/mutations.js`	2.47 kB
`./packages/db/dist/esm/collection/subscription.js`	3.71 kB
`./packages/db/dist/esm/collection/transaction-metadata.js`	144 B
`./packages/db/dist/esm/deferred.js`	207 B
`./packages/db/dist/esm/errors.js`	4.83 kB
`./packages/db/dist/esm/event-emitter.js`	748 B
`./packages/db/dist/esm/index.js`	2.85 kB
`./packages/db/dist/esm/indexes/auto-index.js`	777 B
`./packages/db/dist/esm/indexes/base-index.js`	766 B
`./packages/db/dist/esm/indexes/btree-index.js`	2.17 kB
`./packages/db/dist/esm/indexes/lazy-index.js`	1.24 kB
`./packages/db/dist/esm/indexes/reverse-index.js`	538 B
`./packages/db/dist/esm/local-only.js`	890 B
`./packages/db/dist/esm/local-storage.js`	2.1 kB
`./packages/db/dist/esm/optimistic-action.js`	359 B
`./packages/db/dist/esm/paced-mutations.js`	496 B
`./packages/db/dist/esm/proxy.js`	3.75 kB
`./packages/db/dist/esm/query/builder/functions.js`	792 B
`./packages/db/dist/esm/query/builder/index.js`	5.15 kB
`./packages/db/dist/esm/query/builder/ref-proxy.js`	1.05 kB
`./packages/db/dist/esm/query/compiler/evaluators.js`	1.62 kB
`./packages/db/dist/esm/query/compiler/expressions.js`	430 B
`./packages/db/dist/esm/query/compiler/group-by.js`	2.69 kB
`./packages/db/dist/esm/query/compiler/index.js`	3.62 kB
`./packages/db/dist/esm/query/compiler/joins.js`	2.11 kB
`./packages/db/dist/esm/query/compiler/order-by.js`	1.5 kB
`./packages/db/dist/esm/query/compiler/select.js`	1.11 kB
`./packages/db/dist/esm/query/effect.js`	4.78 kB
`./packages/db/dist/esm/query/expression-helpers.js`	1.43 kB
`./packages/db/dist/esm/query/ir.js`	784 B
`./packages/db/dist/esm/query/live-query-collection.js`	360 B
`./packages/db/dist/esm/query/live/collection-config-builder.js`	7.63 kB
`./packages/db/dist/esm/query/live/collection-registry.js`	264 B
`./packages/db/dist/esm/query/live/collection-subscriber.js`	1.94 kB
`./packages/db/dist/esm/query/live/internal.js`	145 B
`./packages/db/dist/esm/query/live/utils.js`	1.57 kB
`./packages/db/dist/esm/query/optimizer.js`	2.62 kB
`./packages/db/dist/esm/query/predicate-utils.js`	2.97 kB
`./packages/db/dist/esm/query/query-once.js`	359 B
`./packages/db/dist/esm/query/subset-dedupe.js`	960 B
`./packages/db/dist/esm/scheduler.js`	1.3 kB
`./packages/db/dist/esm/SortedMap.js`	1.3 kB
`./packages/db/dist/esm/strategies/debounceStrategy.js`	247 B
`./packages/db/dist/esm/strategies/queueStrategy.js`	428 B
`./packages/db/dist/esm/strategies/throttleStrategy.js`	246 B
`./packages/db/dist/esm/transactions.js`	2.9 kB
`./packages/db/dist/esm/utils.js`	927 B
`./packages/db/dist/esm/utils/browser-polyfills.js`	304 B
`./packages/db/dist/esm/utils/btree.js`	5.61 kB
`./packages/db/dist/esm/utils/comparison.js`	1.05 kB
`./packages/db/dist/esm/utils/cursor.js`	457 B
`./packages/db/dist/esm/utils/index-optimization.js`	1.54 kB
`./packages/db/dist/esm/utils/type-guards.js`	157 B
`./packages/db/dist/esm/virtual-props.js`	360 B

_{compressed-size-action::db-package-size}

github-actions · 2026-03-17T15:10:49Z

Size Change: 0 B

Total Size: 4.23 kB

ℹ️ View Unchanged

Filename	Size
`./packages/react-db/dist/esm/index.js`	249 B
`./packages/react-db/dist/esm/useLiveInfiniteQuery.js`	1.32 kB
`./packages/react-db/dist/esm/useLiveQuery.js`	1.34 kB
`./packages/react-db/dist/esm/useLiveQueryEffect.js`	355 B
`./packages/react-db/dist/esm/useLiveSuspenseQuery.js`	559 B
`./packages/react-db/dist/esm/usePacedMutations.js`	401 B

_{compressed-size-action::react-db-package-size}

Document the transactional metadata model for persisted collections, including row and collection metadata, query retention, and Electric resume state. Made-with: Cursor

Break the persisted sync metadata RFC into phased implementation docs covering the core API, SQLite integration, query collection, Electric collection, and required invariants tests. Made-with: Cursor

Tighten the RFC and phased plan around startup metadata reads, query-owned reconciliation, cold-row retention cleanup, replay fallback behavior, and Electric reset semantics. Made-with: Cursor

Add transactional row and collection metadata plumbing across core sync state, SQLite persistence, query collections, and Electric resume state so persisted ownership and resume metadata survive restarts. Made-with: Cursor

Buffer persisted metadata writes within wrapper transactions and dedupe concurrent collection setup so warm starts no longer trip missing sync transaction errors or collection registry races. Made-with: Cursor

Drop the RFC and phased implementation plan from the branch while leaving the local working copies in place. Made-with: Cursor

Finish the core persisted metadata follow-through so reloads, retained query ownership, and Electric resume/reset state behave correctly across startup and recovery while clarifying metadata semantics around inserts and cleanup. Made-with: Cursor

Finish the remaining persisted metadata work by adding cold-row retained query cleanup, runtime TTL expiry, stronger Electric resume identity checks, and metadata delta replay for follower recovery while keeping reload fallback for reset-like cases. Made-with: Cursor

Use the persisted JSON encoder for replay payloads so bigint and date values survive applied_tx serialization and package-level SQLite adapter tests pass under the CLI runtime. Made-with: Cursor

samwillis and others added 6 commits March 17, 2026 16:37

feat: add persisted sync metadata RFC

ed79891

Document the transactional metadata model for persisted collections, including row and collection metadata, query retention, and Electric resume state. Made-with: Cursor

feat: add persisted sync metadata implementation plan

7a31dfc

Break the persisted sync metadata RFC into phased implementation docs covering the core API, SQLite integration, query collection, Electric collection, and required invariants tests. Made-with: Cursor

docs: refine persisted sync metadata design docs

a9e43d4

Tighten the RFC and phased plan around startup metadata reads, query-owned reconciliation, cold-row retention cleanup, replay fallback behavior, and Electric reset semantics. Made-with: Cursor

feat: implement persisted sync metadata support

df97fb6

Add transactional row and collection metadata plumbing across core sync state, SQLite persistence, query collections, and Electric resume state so persisted ownership and resume metadata survive restarts. Made-with: Cursor

ci: apply automated fixes

95aaff0

fix: align persisted metadata writes with sync startup

7591ee1

Buffer persisted metadata writes within wrapper transactions and dedupe concurrent collection setup so warm starts no longer trip missing sync transaction errors or collection registry races. Made-with: Cursor

samwillis force-pushed the persisted-sync-metadata branch from fce939f to 7591ee1 Compare March 17, 2026 16:37

samwillis and others added 6 commits March 17, 2026 17:40

chore: remove persisted sync metadata docs from branch

8e32e88

Drop the RFC and phased implementation plan from the branch while leaving the local working copies in place. Made-with: Cursor

ci: apply automated fixes

fbb87e3

fix: encode replay deltas with persisted serializer

9c5c4e6

Use the persisted JSON encoder for replay payloads so bigint and date values survive applied_tx serialization and package-level SQLite adapter tests pass under the CLI runtime. Made-with: Cursor

ci: apply automated fixes

a1a2f2e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add persisted sync metadata support#1380

feat: add persisted sync metadata support#1380
samwillis wants to merge 12 commits intomainfrom
persisted-sync-metadata

samwillis commented Mar 17, 2026 •

edited

Loading

Uh oh!

pkg-pr-new bot commented Mar 17, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 17, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 17, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

samwillis commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How To Use

Custom sync implementations

SQLite persisted collections

Query collections

Electric collections

Test plan

Notes

Uh oh!

pkg-pr-new bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

samwillis commented Mar 17, 2026 •

edited

Loading

pkg-pr-new bot commented Mar 17, 2026 •

edited

Loading

github-actions bot commented Mar 17, 2026 •

edited

Loading

github-actions bot commented Mar 17, 2026 •

edited

Loading