observedFrontier in getStateSnapshot() is stale / lags behind actual vector clock

[flyingrobots]

Summary

getStateSnapshot().observedFrontier does not reflect the actual materialized vector clock. It consistently lags behind by one patch per writer because JoinReducer.join() merges the patch's context (pre-creation state) into observedFrontier, but never incorporates the patch's own dot/tick.

This makes observedFrontier unreliable for consumers who want to answer "what is the current vector clock of the materialized state?"

git-warp version: 1.0.0-alpha.5

Root Cause

In JoinReducer.js:365-368 (and the receipt path at 422-425):

const contextVV = patch.context instanceof Map
  ? patch.context
  : vvDeserialize(patch.context);
state.observedFrontier = vvMerge(state.observedFrontier, contextVV);

The merge uses patch.context — the version vector before the patch was created. The patch's own lamport tick (which advances the writer's counter by 1) is never folded into observedFrontier.

Meanwhile, in patch.methods.js:213, the commit path correctly advances _versionVector via vvIncrement(). But this increment is never propagated back to _cachedState.observedFrontier.

Result: observedFrontier is always one step behind per writer. For a single-writer graph, it shows tick N-1 when the actual state reflects tick N.

Reproduction

import WarpGraph, { GitGraphAdapter } from '@git-stunts/git-warp';
import Plumbing from '@git-stunts/plumbing';

// (assumes a fresh git-inited repo at repoPath)

async function openGraph(repoPath: string): Promise<WarpGraph> {
  const plumbing = Plumbing.createDefault({ cwd: repoPath });
  const persistence = new GitGraphAdapter({ plumbing });
  const graph = await WarpGraph.open({
    persistence,
    graphName: 'test-graph',
    writerId: 'human.probe',
    autoMaterialize: true,
  });
  await graph.syncCoverage();
  await graph.materialize();
  return graph;
}

// --- Step 1: graphA seeds one patch ---
const graphA = await openGraph(repoPath);
const p1 = await graphA.createPatch();
p1.addNode('task:001')
  .setProperty('task:001', 'status', 'INBOX')
  .setProperty('task:001', 'type', 'task');
await p1.commit();

// Expected: { 'human.probe': 1 }
// Actual:   { 'human.probe': 1 }  ← correct (context={} merged, but autoMaterialize replays)
const s1 = await graphA.getStateSnapshot();
console.log('After 1 patch:', Object.fromEntries(s1.observedFrontier));

// --- Step 2: graphB opens, mutates ---
const graphB = await openGraph(repoPath);
const p2 = await graphB.createPatch();
p2.setProperty('task:001', 'status', 'BACKLOG');
await p2.commit();

// Expected: { 'human.probe': 2 }
// Actual:   { 'human.probe': 1 }  ← STALE — patch 2's context was {human.probe:1},
//                                     merge is max(1,1)=1, never reaches 2
const s2 = await graphB.getStateSnapshot();
console.log('After 2 patches (graphB):', Object.fromEntries(s2.observedFrontier));

// --- Step 3: graphA re-syncs ---
await graphA.syncCoverage();
await graphA.materialize();

// Expected: { 'human.probe': 2 }
// Actual:   { 'human.probe': 1 }  ← STALE
const s3 = await graphA.getStateSnapshot();
console.log('After re-materialize (graphA):', Object.fromEntries(s3.observedFrontier));

Output:

After 1 patch: { 'human.probe': 1 }
After 2 patches (graphB): { 'human.probe': 1 }
After re-materialize (graphA): { 'human.probe': 1 }

The tick never advances past 1, regardless of how many patches are committed and materialized.

Additional observations

hasFrontierChanged() and getFrontier() work correctly

These APIs track git commit SHAs and reliably detect mutations. The issue is specific to the lamport-based observedFrontier in WarpStateV5.

Naming confusion

• getStateSnapshot() sounds like "give me the current live state" but returns a clone of internal checkpoint-like state where observedFrontier is stale.
• observedFrontier sounds like "what has been observed" but actually means "the pointwise-max of all patch context vectors I've reduced" — which excludes the patches' own contributions.
• A more accurate name might be causalContextFloor or reducedContextHorizon — it represents the lower bound of what all patches claim to have seen, not the upper bound of what the state contains.

checkpointPolicy counting

checkpointPolicy: { every: N } counts patches per materialization pass, not cumulatively. With incremental materialize (checkpoint exists), only newly-loaded patches count toward the threshold. This means if a graph grows by 1 patch at a time (common in interactive use), the counter resets to 1 on each materialize call and never reaches the threshold.

Suggested fix

In JoinReducer.join(), after merging the context, also fold in the patch's own tick:

state.observedFrontier = vvMerge(state.observedFrontier, contextVV);
// Also incorporate the patch's own contribution
const patchDot = { writerId: patch.writer, counter: patch.lamport };
if ((state.observedFrontier.get(patch.writer) || 0) < patch.lamport) {
  state.observedFrontier.set(patch.writer, patch.lamport);
}

This would make observedFrontier reflect the true high-water mark of all materialized patches, which is what consumers expect.

Context

Discovered while building a TUI status line for XYPH that displays the current WARP graph tick. The status line used getStateSnapshot().observedFrontier to show the current vector clock, but the values were frozen and never advanced despite ongoing mutations.

[flyingrobots]

Failing test case

Drop this into a vitest suite. It passes assertions 1-3 (patches apply, content changes, hasFrontierChanged works). It fails on assertion 4 — observedFrontier never advances past the initial tick.

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { execSync } from 'node:child_process';
import os from 'node:os';
import path from 'node:path';
import fs from 'node:fs';
import WarpGraph, { GitGraphAdapter } from '@git-stunts/git-warp';
import type { PatchSession } from '@git-stunts/git-warp';
import Plumbing from '@git-stunts/plumbing';

describe('observedFrontier should advance after materialize', () => {
  let repoPath: string;
  const graphName = 'frontier-test';
  const writerId = 'human.probe';

  async function openGraph(): Promise<WarpGraph> {
    const plumbing = Plumbing.createDefault({ cwd: repoPath });
    const persistence = new GitGraphAdapter({ plumbing });
    const graph = await WarpGraph.open({
      persistence,
      graphName,
      writerId,
      autoMaterialize: true,
    });
    await graph.syncCoverage();
    await graph.materialize();
    return graph;
  }

  beforeAll(() => {
    repoPath = path.join(os.tmpdir(), `warp-frontier-test-${Date.now()}`);
    fs.mkdirSync(repoPath, { recursive: true });
    execSync('git init', { cwd: repoPath });
    execSync('git config user.email "test@test.dev"', { cwd: repoPath });
    execSync('git config user.name "Test"', { cwd: repoPath });
  });

  afterAll(() => {
    fs.rmSync(repoPath, { recursive: true, force: true });
  });

  it('observedFrontier advances when a second instance commits and the first re-materializes', async () => {
    // --- graphA seeds one patch ---
    const graphA = await openGraph();
    const p1 = (await graphA.createPatch()) as PatchSession;
    p1.addNode('task:001')
      .setProperty('task:001', 'title', 'Seed task')
      .setProperty('task:001', 'status', 'INBOX')
      .setProperty('task:001', 'type', 'task');
    await p1.commit();

    const s1 = await graphA.getStateSnapshot();
    const tick1 = s1!.observedFrontier.get(writerId) ?? 0;
    expect(tick1).toBeGreaterThan(0); // ✅ passes — initial tick is set

    // --- graphB opens fresh, sees seed, commits a mutation ---
    const graphB = await openGraph();
    const p2 = (await graphB.createPatch()) as PatchSession;
    p2.setProperty('task:001', 'status', 'BACKLOG');
    await p2.commit();

    // --- graphA re-syncs and re-materializes ---
    await graphA.syncCoverage();

    // Sanity: hasFrontierChanged() correctly detects the new patch
    const changed = await graphA.hasFrontierChanged();
    expect(changed).toBe(true); // ✅ passes — SHA-based frontier works

    await graphA.materialize();

    // Content is correctly updated
    const props = await graphA.getNodeProps('task:001');
    expect(props?.get('status')).toBe('BACKLOG'); // ✅ passes — mutation visible

    // observedFrontier should reflect the new patch
    const s2 = await graphA.getStateSnapshot();
    const tick2 = s2!.observedFrontier.get(writerId) ?? 0;

    // ❌ FAILS: tick2 === tick1 (both are 1)
    // Expected: tick2 > tick1 (should be 2 after processing the second patch)
    expect(tick2).toBeGreaterThan(tick1);
  });
});

Expected: tick2 > tick1 (should be 2 after materializing the second patch)
Actual: tick2 === tick1 === 1

[flyingrobots]

Closed by #42 (v11.5.0). Fix landed in 1d31fc1 — JoinReducer.join() now folds the patch's own dot into observedFrontier.