Skip to content

docs: update query benchmarks (3.5.0)#684

Merged
carlos-alm merged 4 commits intomainfrom
benchmark/query-v3.5.0-20260330-065335
Mar 30, 2026
Merged

docs: update query benchmarks (3.5.0)#684
carlos-alm merged 4 commits intomainfrom
benchmark/query-v3.5.0-20260330-065335

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

Automated query benchmark update for 3.5.0 from workflow run #521.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 30, 2026

Greptile Summary

This automated PR appends the 3.5.0 query benchmark data to generated/benchmarks/QUERY-BENCHMARKS.md. Because the native Rust engine crashed during graph build (napi-rs serialization bug on InsertNodesDefinition.visibility), only WASM results are recorded; native is null.

Key changes:

  • New row in the summary comparison table for 3.5.0 (WASM-only)
  • "Latest results" section updated to 3.5.0 with the Native (Rust) sub-section removed
  • Mid-query target changed from noTests (3.4.1) to db (3.5.0), correctly reflected in both the detail section and JSON data
  • A new Note (3.5.0) block added inside the established <!-- NOTES_START --> convention, explaining the native crash, the target change, and the resulting non-comparability with prior releases
  • All three representations of the data (summary table, detailed results section, embedded JSON) are internally consistent — every metric and percentage delta was verified

No issues were found. The document follows all precedents set by the 3.4.1 and 3.4.0 notes.

Confidence Score: 5/5

Safe to merge — automated documentation update with consistent data across all representations and proper explanatory notes.

All three data representations (summary table, detailed results section, and embedded JSON) are internally consistent. Percentage deltas are arithmetically correct. The native: null situation is fully explained in the new Note (3.5.0), which follows the established convention used for 3.4.1 and 3.4.0. No code is changed, only generated benchmark documentation. No P0 or P1 issues found.

No files require special attention.

Important Files Changed

Filename Overview
generated/benchmarks/QUERY-BENCHMARKS.md Adds 3.5.0 WASM-only benchmark entry (native: null due to napi-rs crash), updates "Latest results" section, and adds an explanatory note following established convention.

Sequence Diagram

sequenceDiagram
    participant CI as GitHub Actions (Run #521)
    participant NE as Native Engine (Rust)
    participant WE as WASM Engine
    participant BG as Benchmark Generator
    participant DOC as QUERY-BENCHMARKS.md

    CI->>NE: Build graph (insertNodes)
    NE--xCI: CRASH — napi-rs serialization error (InsertNodesDefinition.visibility: null)
    Note over NE: native: null

    CI->>WE: Build graph (insertNodes)
    WE-->>CI: Graph built successfully
    CI->>WE: Run query benchmarks (fnDeps d1/d3/d5, fnImpact, diffImpact)
    WE-->>CI: Results (9.1ms / 9.1ms / 9ms / 3ms / 3.1ms / 3.1ms / 6.4ms)

    CI->>BG: Emit benchmark JSON for v3.5.0
    BG-->>CI: { version: "3.5.0", wasm: {...}, native: null }

    CI->>DOC: Prepend summary table row (wasm only)
    CI->>DOC: Replace "Latest results" section (WASM only, target mid=db)
    CI->>DOC: Prepend Note (3.5.0) to NOTES block
    CI->>DOC: Prepend JSON entry to QUERY_BENCHMARK_DATA block
    DOC-->>CI: PR opened (this PR)
Loading

Reviews (3): Last reviewed commit: "docs: note mid-target change (noTests→db..." | Re-trigger Greptile

| diffImpact latency | 8.1ms |
| diffImpact affected functions | 0 |
| diffImpact affected files | 0 |
**Version:** 3.5.0 | **Date:** 2026-03-30
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Missing native benchmarks — consider adding a note

3.5.0 is the first release since at least 3.3.1 to have "native": null in the benchmark data. Every prior version (3.4.1, 3.4.0, 3.3.1, …) included both a native (Rust) and a WASM row. The document has a precedent of adding a <!-- NOTES_START --> note whenever something unusual affects comparability (see the existing notes for 3.4.1 and 3.4.0).

Without a note, readers who try to compare native query latency across releases will find the series silently broken at 3.5.0. If native benchmarks were intentionally skipped (e.g., the CI step was removed, native engine deprecated, or the runner environment changed), a brief note would preserve the document's self-contained interpretability.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already addressed in the file. Line 61 contains a <!-- NOTES_START --> note for 3.5.0 that explains the native engine crash (InsertNodesDefinition.visibility napi-rs serialization bug), states native: null, and notes that query latencies are not directly comparable to prior versions. This matches the existing convention used for 3.4.1 and 3.4.0 notes.

@carlos-alm
Copy link
Copy Markdown
Contributor

@greptileai

Adds the mid-query target change callout to the 3.5.0 note,
consistent with the convention used in 3.4.1 and 3.4.0 notes.
@carlos-alm
Copy link
Copy Markdown
Contributor

Addressed the P2 suggestion from the Greptile summary: added the mid-query target change callout (noTestsdb) to the 3.5.0 note, consistent with the convention used in the 3.4.1 and 3.4.0 notes. See commit c585b0a.

Regarding CI: the test failures are all caused by the StringExpected / InsertNodesDefinition.visibility napi-rs serialization bug, which is the same root cause being fixed in PR #693. This docs-only PR introduces no code changes — CI will go green once #693 merges to main.

@carlos-alm
Copy link
Copy Markdown
Contributor

@greptileai

@carlos-alm carlos-alm merged commit a0a98c3 into main Mar 30, 2026
13 checks passed
@carlos-alm carlos-alm deleted the benchmark/query-v3.5.0-20260330-065335 branch March 30, 2026 09:41
@github-actions github-actions bot locked and limited conversation to collaborators Mar 30, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant