Skip to content

fix(ope): address CodeRabbit and reviewer feedback on PR #176#197

Merged
freshtonic merged 7 commits intomainfrom
fix/ope-index-type-docs
May 7, 2026
Merged

fix(ope): address CodeRabbit and reviewer feedback on PR #176#197
freshtonic merged 7 commits intomainfrom
fix/ope-index-type-docs

Conversation

@freshtonic
Copy link
Copy Markdown
Contributor

@freshtonic freshtonic commented May 7, 2026

Follow-up to #176 addressing late CodeRabbit and reviewer feedback.

Code changes

compare_ope_cllw_* STRICT cleanup

  • Drop dead outer NULL-handling branches (function inputs a/b short-circuit to NULL via STRICT before the body runs).
  • Keep the inner term-NULL branches — they handle a non-NULL eql_v2_encrypted whose payload lacks the opf / opv field, mirroring compare_ore_block_u64_8_256.
  • Add @note in both compare.sql files explicitly distinguishing the two NULL paths.

OPE helper inlinability

  • Convert ope_cllw_u64_65(eql_v2_encrypted), ope_cllw_var_8(eql_v2_encrypted), both has_ope_* overloads, and order_by_ope from LANGUAGE plpgsql to LANGUAGE sql so the planner can inline them on the sort / order_by hot path.

Doc fixes

  • Type comments: ciphertext length now correctly reads "8 bytes per plaintext byte" (was "per plaintext bit", contradicting the 65-byte result).
  • CLWWCLLW everywhere — matches the ope_cllw_* type names and existing ore_cllw_* docs.
  • @param tags in ope_cllw_*/functions.sql now name the parameter (val) before the type, matching valid Doxygen syntax.
  • docs/reference/sql-support.md:
    • MIN/MAX and COUNT(DISTINCT) rows separate ste_vec-extracted-node requirements from outer-column sibling-index requirements. ope is never emitted by ste_vec extraction; it only applies to the outer column.
    • COUNT(DISTINCT col) outer-column cell lists unique / ore / ope for parity with the MIN/MAX row.

Test changes

Operator parity<, <=, =, >=, >, <> exercised via the eql_v2_encrypted operators on both opf and opv variants (previously only < and > were covered for opf; opv was exercised through compare_* but not via operators).

ORE-vs-OPE precedenceore_wins_over_opf_when_both_present locks in the ORE-before-OPE dispatch order in eql_v2.compare when a payload carries both ob and opf.

NULL-handling parity with ORE — 18 tests covering scenarios where ORE had coverage and OPE had none:

  • {"opf": null} / {"opv": null} payloads dispatch through to HMAC (mirrors compare_hmac_with_null_ore_index).
  • compare_ope_cllw_* returns NULL for both NULL-left and NULL-right operands (STRICT contract codified).
  • ORDER BY × ASC/DESC × NULLS FIRST/NULLS LAST with mixed NULL + opf rows.
  • eql_v2.sort_compare with mixed NULL + opf rows.
  • eql_v2.min / eql_v2.max over OPE-encoded values, including NULL-only → NULL.
  • BETWEEN for both opf and opv variants.

Brittle assertionsconfig_check_rejects_unknown_index no longer matches the exact comma-separated list of valid indexes; checks for ope + the offending token instead, so adding a new valid index won't break the test.

Open review items

  • src/ope_cllw_u64_65/compare.sql (tobyhede, line 30): the dropped outer NULL branches drew the comment "This might have unintended consequences. Let me try and find the history." The branches are dead under STRICT, but their removal diverges from the ORE pattern which keeps both outer (input-NULL) and inner (term-NULL) layers. Awaiting follow-up — happy to restore the outer branches as defense-in-depth if that's preferred.

Summary by CodeRabbit

  • Bug Fixes

    • Corrected NULL handling in comparison functions to align with STRICT semantics; NULL inputs now short-circuit before function execution.
    • Fixed OPE extraction and presence-check behavior for encrypted columns.
  • Documentation

    • Clarified ordered comparison behavior between OPE and ORE index configurations and their operator dispatch rules.
    • Updated comparison function documentation to reflect proper NULL handling and term extraction semantics.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 7, 2026

Review Change Stack

📝 Walkthrough

Walkthrough

This PR refactors OPE (Order-Preserving Encryption) functions from PL/pgSQL to SQL, clarifies STRICT NULL semantics in comparisons, updates documentation from CLWW to CLLW terminology, and adds extensive test coverage for operators, NULL handling, and aggregates across fixed and variable-width OPE encodings.

Changes

OPE Refactoring and NULL Handling

Layer / File(s) Summary
Documentation and Terminology
docs/reference/sql-support.md, src/ope_cllw_u64_65/types.sql
Clarifies ore vs ope operator dispatch behavior; updates ste_vec requirements for COUNT(DISTINCT) and MIN/MAX; standardizes CLWW→CLLW terminology and ciphertext size description across type documentation.
STRICT NULL Semantics
src/ope_cllw_u64_65/compare.sql, src/ope_cllw_var_8/compare.sql
Updated documentation describes STRICT short-circuit semantics for NULL inputs; removed explicit input NULL-handling branches from compare_ope_cllw_u64_65; internal NULL-term checks now scoped to missing opf/opv payloads.
OPE Extraction Functions
src/ope_cllw_u64_65/functions.sql, src/ope_cllw_var_8/functions.sql
Converted ope_cllw_* and has_ope_cllw_* functions from PL/pgSQL to SQL-language: encrypted-column variants delegate via SELECT to JSONB extractors; presence predicates use direct IS NOT NULL checks.
Order-by Integration
src/operators/order_by.sql
Simplified eql_v2.order_by_ope from procedural BEGIN/IF/RETURN to declarative SELECT CASE, preserving fixed-width-then-variable-width selection semantics.
Test Coverage
tests/sqlx/tests/ope_tests.rs
Added 700+ lines covering OPE operators (<=, >=, =, <>), dispatcher precedence, NULL-parity enforcement, STRICT semantics, ORDER BY NULL placement, aggregates (MIN, MAX), and BETWEEN bounds for both opf and opv encodings.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers

  • coderdan
  • tobyhede

Poem

🐰 With SQL so clean and STRICT so true,
We've trimmed the plpgsql fuzz right through,
OPE helpers now dance in elegant lines,
And tests—oh my!—700 shiny designs!
No NULL shall hide, no edge shall slip,
This refactoring's quite the crypto-bunny trip! 🔐

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the pull request as a follow-up addressing feedback from CodeRabbit and reviewers on a previous PR, covering documentation fixes, code refactoring, and test improvements.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/ope-index-type-docs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/ope_cllw_u64_65/compare.sql (1)

30-48: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Clarify whether NULL branches are justified by system invariants or represent incomplete PR implementation

The PR description's claim that STRICT eliminates null-handling branches is imprecise. STRICT short-circuits when function inputs (a and b) are NULL—not when extracted terms (a_term and b_term) are NULL. The latter can legitimately occur: has_ope_cllw_u64_65 checks whether the 'opf' field exists in the encrypted JSONB payload and returns false if absent, leaving the term NULL after lines 30–36.

The NULL branches (lines 38–48) are only dead code if a system invariant guarantees every eql_v2_encrypted value always carries a valid OPE index term. No such invariant is documented in the codebase. Both compare_ope_cllw_u64_65 and compare_ope_cllw_var_8 retain these branches consistently, suggesting intentional defensive logic rather than oversight.

Either confirm that an undocumented invariant exists (and remove both conditional extraction and NULL checks together), or acknowledge that the NULL branches remain necessary.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/ope_cllw_u64_65/compare.sql` around lines 30 - 48, The review points out
that STRICT only covers NULL inputs a/b, not extracted terms, so decide whether
the NULL-handling branches in compare_ope_cllw_u64_65 are removable: either
document and enforce the invariant that every eql_v2_encrypted value has an OPE
term and then remove both the has_ope_cllw_u64_65 checks and the subsequent
a_term/b_term NULL branches (and mirror the same change in
compare_ope_cllw_var_8), or retain the defensive logic as-is; update
code/comments accordingly and reference the functions has_ope_cllw_u64_65,
ope_cllw_u64_65, compare_ope_cllw_u64_65 and compare_ope_cllw_var_8 so reviewers
can verify the chosen approach.
🧹 Nitpick comments (1)
src/ope_cllw_var_8/functions.sql (1)

36-37: 💤 Low value

@param is missing the parameter name val

Lines 36 and 70 document the parameter using the type as a stand-in for the name (@param eql_v2_encrypted …), which is not valid Doxygen syntax. Compare to compare.sql lines 12-13 where the names a and b are explicitly present.

📝 Proposed doc fix
---! `@param` eql_v2_encrypted Encrypted column value
+--! `@param` val eql_v2_encrypted Encrypted column value

Apply the same fix at both line 36 and line 70.

Also applies to: 70-71

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/ope_cllw_var_8/functions.sql` around lines 36 - 37, The docblocks use the
type name instead of the parameter name; update both param annotations so they
include the parameter name "val" before the type (e.g., change the `@param` line
that currently starts with "eql_v2_encrypted" to start with "val
eql_v2_encrypted ...") for the function/documentation referencing
eql_v2.ope_cllw_var_8; apply the same change at the second occurrence around
line 70 so the `@param` syntax matches the valid form used in compare.sql (names
like "a" and "b").
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/reference/sql-support.md`:
- Line 142: The table row for `COUNT(DISTINCT col)` is missing `ope` in the
outer-column requirement; update the docs so the outer-column support list for
`COUNT(DISTINCT col)` includes `ope` alongside `unique` and `ore`. Locate the
row mentioning `COUNT(DISTINCT col)` in sql-support.md and add `ope` to the
outer-column cell so it matches the support matrix for DISTINCT semantics.

---

Outside diff comments:
In `@src/ope_cllw_u64_65/compare.sql`:
- Around line 30-48: The review points out that STRICT only covers NULL inputs
a/b, not extracted terms, so decide whether the NULL-handling branches in
compare_ope_cllw_u64_65 are removable: either document and enforce the invariant
that every eql_v2_encrypted value has an OPE term and then remove both the
has_ope_cllw_u64_65 checks and the subsequent a_term/b_term NULL branches (and
mirror the same change in compare_ope_cllw_var_8), or retain the defensive logic
as-is; update code/comments accordingly and reference the functions
has_ope_cllw_u64_65, ope_cllw_u64_65, compare_ope_cllw_u64_65 and
compare_ope_cllw_var_8 so reviewers can verify the chosen approach.

---

Nitpick comments:
In `@src/ope_cllw_var_8/functions.sql`:
- Around line 36-37: The docblocks use the type name instead of the parameter
name; update both param annotations so they include the parameter name "val"
before the type (e.g., change the `@param` line that currently starts with
"eql_v2_encrypted" to start with "val eql_v2_encrypted ...") for the
function/documentation referencing eql_v2.ope_cllw_var_8; apply the same change
at the second occurrence around line 70 so the `@param` syntax matches the valid
form used in compare.sql (names like "a" and "b").
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ac3e9e50-8bf3-44db-9e12-bd42b28e04e5

📥 Commits

Reviewing files that changed from the base of the PR and between 8ee7af2 and e40b835.

📒 Files selected for processing (10)
  • docs/reference/index-config.md
  • docs/reference/sql-support.md
  • src/ope_cllw_u64_65/compare.sql
  • src/ope_cllw_u64_65/functions.sql
  • src/ope_cllw_u64_65/types.sql
  • src/ope_cllw_var_8/compare.sql
  • src/ope_cllw_var_8/functions.sql
  • src/ope_cllw_var_8/types.sql
  • src/operators/order_by.sql
  • tests/sqlx/tests/ope_tests.rs

Comment thread docs/reference/sql-support.md Outdated
Comment thread src/ope_cllw_u64_65/compare.sql
freshtonic added 3 commits May 7, 2026 13:27
- Drop dead NULL-handling branches from compare_ope_cllw_* (STRICT
  short-circuits NULL inputs before the body runs).
- Fix ciphertext-length wording in OPE type docs ("8 bytes per plaintext
  byte", not "per plaintext bit", which contradicted the 65-byte result).
- Convert simple OPE helpers (extractors, has_*, order_by_ope) from
  LANGUAGE plpgsql to LANGUAGE sql so the planner can inline them on the
  sort/order_by hot path. Per project guideline.
- Spell the cipher consistently as CLLW (matching ope_cllw_* type names
  and the existing ore_cllw_* docs); CLWW was a typo.
- Clarify sql-support.md MIN/MAX and COUNT(DISTINCT) rows so ope is no
  longer implied to be a ste_vec-extracted node term — it only applies
  to the outer column.
- Loosen brittle exact-list assertion in config_check_rejects_unknown_index
  to mention 'ope' and the offending token.
- Add ore_wins_over_opf_when_both_present to lock in the
  ORE-before-OPE dispatch precedence in eql_v2.compare.
- Add operator-level coverage for <=, >=, =, <> on both opf and opv
  variants (previously only < and > were covered for opf, and opv was
  exercised through compare_* but not via operators).
Adds 18 OPE tests covering behaviors that were exercised for ORE but had
no OPE equivalent. The ORE suite duplicates each test across three
ciphertext variants (block / cllw_u64_8 / cllw_var_8); for OPE we only
need one representative per variant per behavior because both opf and
opv use the same primitive (bytea lex compare).

NULL term in payload (mirrors compare_hmac_with_null_ore_index):
- has_opf_false_when_field_is_json_null
- has_opv_false_when_field_is_json_null
- compare_dispatches_through_null_opf_to_hmac
- compare_dispatches_through_null_opv_to_hmac

NULL operands at the comparator (codifies STRICT short-circuit):
- compare_ope_cllw_u64_65_strict_returns_null_for_null_operand
- compare_ope_cllw_var_8_strict_returns_null_for_null_operand

ORDER BY NULLS FIRST/LAST × ASC/DESC on opf-encoded data:
- order_by_asc_nulls_first_with_opf
- order_by_asc_nulls_last_with_opf
- order_by_desc_nulls_first_with_opf
- order_by_desc_nulls_last_with_opf

sort_compare with mixed NULL + opf rows:
- sort_compare_asc_puts_nulls_first_with_opf
- sort_compare_desc_puts_nulls_last_with_opf

MIN / MAX aggregates over OPE-encoded values:
- eql_v2_min_with_opf_finds_minimum
- eql_v2_max_with_opf_finds_maximum
- eql_v2_min_with_opf_null_only_returns_null
- eql_v2_max_with_opf_null_only_returns_null

BETWEEN range filtering:
- between_with_opf_inclusive_bounds
- between_with_opv_inclusive_bounds

All 47 OPE tests pass; full suite (~430 tests) is green.
The earlier sql-support clarification dropped `ope` from the outer-column
index list for COUNT(DISTINCT col). OPE ciphertexts are deterministic
within a column key, so an outer-column `ope` index supports DISTINCT
semantics just like `unique` and `ore`. Restoring it for parity with the
MIN/MAX row.
@freshtonic freshtonic force-pushed the fix/ope-index-type-docs branch from 48aeaa4 to ffb45d0 Compare May 7, 2026 03:27
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/sqlx/tests/ope_tests.rs`:
- Around line 798-808: The test
compare_ope_cllw_var_8_strict_returns_null_for_null_operand only asserts
compare(NULL, x) — add the symmetric assertion to also check compare(x, NULL)
returns NULL by constructing the SQL with the encrypted left operand (use
opv_payload and eql_v2.to_encrypted as in the existing test) and executing a
sqlx::query_scalar to assert the result is_none; update the test
compare_ope_cllw_var_8_strict_returns_null_for_null_operand to include this
second fetch/assert using the same pool and payload so both NULL-left and
NULL-right paths are covered.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ddd41923-fd4a-4a63-8037-22dc6a0e98c9

📥 Commits

Reviewing files that changed from the base of the PR and between e40b835 and 30aa13f.

📒 Files selected for processing (1)
  • tests/sqlx/tests/ope_tests.rs

Comment thread tests/sqlx/tests/ope_tests.rs
freshtonic added a commit that referenced this pull request May 7, 2026
Two doc-only fixes flagged in PR #197 review:

1. compare.sql @note for both u64_65 and var_8: the term-NULL branches
   (a_term IS NULL / b_term IS NULL) are NOT redundant with STRICT.
   STRICT only short-circuits on NULL function inputs (a, b); the term
   branches handle a non-NULL eql_v2_encrypted whose payload simply
   lacks the opf/opv field. Mirrors the defensive pattern in
   compare_ore_block_u64_8_256.

2. functions.sql @param tags for ope_cllw_u64_65 / ope_cllw_var_8 /
   has_ope_cllw_u64_65 / has_ope_cllw_var_8 now include the parameter
   name "val" before the type, matching valid Doxygen syntax (and
   compare.sql which already names a/b correctly).

No code behavior change; mise run docs:validate stays clean.
freshtonic added 2 commits May 7, 2026 13:34
Match the bidirectional NULL coverage of the u64_65 sibling test.
Asserting only compare(NULL, x) leaves the right-NULL STRICT path
unverified; both directions must short-circuit.
Two doc-only fixes flagged in PR #197 review:

1. compare.sql @note for both u64_65 and var_8: the term-NULL branches
   (a_term IS NULL / b_term IS NULL) are NOT redundant with STRICT.
   STRICT only short-circuits on NULL function inputs (a, b); the term
   branches handle a non-NULL eql_v2_encrypted whose payload simply
   lacks the opf/opv field. Mirrors the defensive pattern in
   compare_ore_block_u64_8_256.

2. functions.sql @param tags for ope_cllw_u64_65 / ope_cllw_var_8 /
   has_ope_cllw_u64_65 / has_ope_cllw_var_8 now include the parameter
   name "val" before the type, matching valid Doxygen syntax (and
   compare.sql which already names a/b correctly).

No code behavior change; mise run docs:validate stays clean.
@freshtonic freshtonic force-pushed the fix/ope-index-type-docs branch from e79ab50 to 01f8120 Compare May 7, 2026 03:34
@freshtonic freshtonic requested a review from tobyhede May 7, 2026 05:18
Comment thread docs/reference/sql-support.md Outdated
@coderdan
Copy link
Copy Markdown
Contributor

coderdan commented May 7, 2026

Hey James — flagging an inlining issue that CI is hiding. Splinter is green and the new lint test passes, but the perf goal of the PR (inlinable OPE helpers on the sort/order_by hot path) is silently undone after merge. Wanted to lay out the mechanism in detail since this conflict will keep coming up.

What "inlining" means

PostgreSQL's planner has an optimisation in src/backend/optimizer/util/clauses.c called inline_function. When the planner sees a SQL function in a query, it can substitute the function's body directly into the query before doing anything else. So a query like:

ORDER BY eql_v2.order_by_ope(col)

is rewritten by the planner to:

ORDER BY CASE
  WHEN eql_v2.has_ope_cllw_u64_65(col) THEN (eql_v2.ope_cllw_u64_65(col)).bytes
  WHEN eql_v2.has_ope_cllw_var_8(col)  THEN (eql_v2.ope_cllw_var_8(col)).bytes
END

That matters for two reasons:

  1. Index matching. A btree index on (eql_v2.ope_cllw_u64_65(col)).bytes can only match a query whose ORDER BY literally contains that expression. If order_by_ope is opaque (a function call), the planner can't see through it and the index sits unused.
  2. Constant folding. Inlining lets the planner combine the function body with surrounding query structure, prune dead CASE branches, etc.

What disables inlining

inline_function checks several things on the function's catalog row before inlining. The relevant one (around line 4715 in PG17):

if (... !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ...)
    return NULL;  // bail — won't inline

If pg_proc.proconfig is non-NULL, the planner refuses to inline. proconfig is the column that holds per-function settings — including SET search_path = .... So adding any SET clause to a SQL function turns it from "inlinable" into "opaque function call" forever. Hardcoded; can't be worked around without changing the function definition.

How your PR gets broken

Your source files are correct as written. The damage happens later in the install pipeline. Here's the timeline:

  1. Your source files run (src/ope_cllw_u64_65/functions.sql, etc.). They emit CREATE FUNCTION ... LANGUAGE sql IMMUTABLE STRICT PARALLEL SAFE with no SET clause. At this moment your functions are inlinable.

    eql_v2.order_by_ope(eql_v2_encrypted) | prolang=sql | proconfig=NULL    ✅ inlinable
    
  2. The pinner runs last (tasks/pin_search_path.sql, appended to every release variant by build.sh after all source files). It loops over every function in eql_v2 and runs ALTER FUNCTION ... SET search_path = pg_catalog, extensions, publicmutating the functions you just created.

    eql_v2.order_by_ope(eql_v2_encrypted) | prolang=sql | proconfig={"search_path=..."}    ❌ opaque
    

I simulated the merge state locally (PR branch source files + main's pinner and build.sh) and verified — all 9 OPE helpers (the 7 SQL ones you flipped plus the 2 plpgsql jsonb entry-points) end up with proconfig = {"search_path=..."} after install.

The function still works (returns the right answer); it's just orders of magnitude slower at scale. There's no error, no warning, nothing in CI that fails — splinter is happy because proconfig now has a search_path entry, and lint_phase_1_operators_are_clean only watches =, <>, ~~, ~~*, @>, <@ from #193, not OPE helpers.

That's the silent-undo problem.

Why the pinner exists

Supabase splinter's function_search_path_mutable lint fails any function with proconfig missing a search_path= entry. The two checks read the same column from opposite directions:

Mechanism Wants proconfig to... Reason
PostgreSQL inline_function be NULL inline the body
Splinter function_search_path_mutable contain search_path=... guard against schema injection

Any function where we want both is impossible. The pinner is told which functions to skip via an explicit allowlist (inline_critical_oids filter in the DO block), and those same functions are listed in tasks/test/splinter.sh's allowlist with a justification.

Prioritisation

Stating this explicitly because the conflict will keep coming up: performance wins over splinter compliance. When inlining and function_search_path_mutable conflict, we keep the function inlinable and add an allowlist entry — never the other way round. SET search_path would silently force seq scans on Supabase and any --exclude-operator-family build, which is much worse than a yellow warning in the advisor UI. #198 documents the broader story.

The fix

Your src/ changes are right and shouldn't be touched. The fix is to teach the pinner not to overwrite them. Two places:

1. tasks/pin_search_path.sql — extend inline_critical_oids so the pinner's ALTER FUNCTION loop skips the 7 SQL helpers. The existing block keys on 2-arg signatures; the OPE helpers are 1-arg, so add a structurally distinct block:

-- (encrypted,) overloads
(p.proargtypes[0] = enc_oid AND p.pronargs = 1
  AND p.proname IN (
    'ope_cllw_u64_65', 'ope_cllw_var_8',          -- OPE term extractors (#176)
    'has_ope_cllw_u64_65', 'has_ope_cllw_var_8',  -- OPE term presence checks
    'order_by_ope'                                -- OPE order-by extractor
  ))
-- (jsonb,) overloads
OR (p.proargtypes[0] = jsonb_oid AND p.pronargs = 1
  AND p.proname IN (
    'ope_cllw_u64_65', 'ope_cllw_var_8',
    'has_ope_cllw_u64_65', 'has_ope_cllw_var_8'
  ))

After that change, step 2 above never fires for these functions — proconfig stays NULL, inlining survives.

2. tasks/test/splinter.sh — once the pinner stops pinning these, splinter will start flagging them. Add 5 allowlist entries (splinter dedupes by proname, so one entry per name covers both jsonb/encrypted overloads):

function_search_path_mutable	eql_v2	ope_cllw_u64_65	function	OPE-index inlining (#176): extracts opf bytes; must inline so a btree on (eql_v2.ope_cllw_u64_65(col)).bytes can match.
function_search_path_mutable	eql_v2	ope_cllw_var_8	function	OPE-index inlining: same as ope_cllw_u64_65 for variable-width OPE.
function_search_path_mutable	eql_v2	has_ope_cllw_u64_65	function	OPE-index inlining: presence-check used by order_by_ope's CASE; flipping to LANGUAGE sql is a deliberate perf choice in this PR.
function_search_path_mutable	eql_v2	has_ope_cllw_var_8	function	OPE-index inlining: same as has_ope_cllw_u64_65.
function_search_path_mutable	eql_v2	order_by_ope	function	OPE-index inlining (#176): wraps the CASE that picks ope_cllw_u64_65 vs ope_cllw_var_8; intended for `ORDER BY eql_v2.order_by_ope(col)`.

Commit 1ab0b00 on the #196 branch is the reference shape for this — same problem, same fix, different functions.

Defence-in-depth suggestion (optional)

lint_phase_1_operators_are_clean only watches the Phase 1 operators. Worth either extending it or adding a sibling test that watches order_by_ope + the OPE extractors. Otherwise the same regression can land again the next time someone makes a function inlinable for perf — there's currently nothing in CI that would fail on it.

@coderdan
Copy link
Copy Markdown
Contributor

coderdan commented May 7, 2026

TL;DR on the above: your code is fine. The "pinner" script which is intended to set search paths correctly to satisfy splinter breaks the inlinability of the functions you added.

IMHO this is brittle and we should find an alternative approach or just disable splinter until we can find a better approach.

@coderdan
Copy link
Copy Markdown
Contributor

coderdan commented May 7, 2026

I'll approve now and address the pinning/splinter issue in a separate PR as its not really directly related to your changes.

Copy link
Copy Markdown
Contributor

@coderdan coderdan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just the CLLW vs CLWW thing. Otherwise LGTM and I'll work on a follow up.

Per coderdan: Chenette-Lewi-Weis-Wu = CLWW. The earlier change in this
PR (CLWW -> CLLW everywhere "for consistency with type names") was
wrong-headed — the type names ope_cllw_u64_65 / ope_cllw_var_8 are
themselves typos that already shipped, but the cipher abbreviation in
prose docs should stay CLWW. Spreading the typo to free-text docs makes
the codebase less searchable, not more.

Reverts CLLW -> CLWW only in OPE-context references:
  docs/reference/{index-config,sql-support}.md
  src/ope_cllw_u64_65/{compare,functions,types}.sql
  src/ope_cllw_var_8/{compare,functions,types}.sql
  src/operators/order_by.sql
  tests/sqlx/tests/ope_tests.rs (file-level docstring)

Leaves untouched:
  - The "ORE CLLW variants" / "CLLW ORE" / "CLLW Order-Revealing"
    references — those are the genuine CLLW (Copyless Logarithmic
    Width) ORE cipher used by ore_cllw_u64_8 and ore_cllw_var_8.
  - Type / function / file names (ope_cllw_*) — out of scope to rename.

47/47 OPE tests still pass; docs:validate clean.
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/ope_cllw_var_8/functions.sql (1)

6-16: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Remaining CLWWCLLW doc typos missed by this PR.

Lines 6, 8, and 12 still read "CLWW" while the function/type name is ope_cllw_var_8 (CLLW). The PR objective explicitly lists "Fix CLWW → CLLW naming to match types," so these look like an incomplete fix.

📝 Proposed fix
-–! `@brief` Extract variable-width CLWW OPE index term from JSONB payload
+–! `@brief` Extract variable-width CLLW OPE index term from JSONB payload
 –!
-–! Extracts the variable-width CLWW OPE ciphertext from the 'opv' field of an
+–! Extracts the variable-width CLLW OPE ciphertext from the 'opv' field of an
 –! encrypted data payload. Used internally for range query comparisons.
 –!
 –! `@param` val jsonb encrypted EQL payload
-–! `@return` eql_v2.ope_cllw_var_8 Variable-width CLWW OPE ciphertext
+–! `@return` eql_v2.ope_cllw_var_8 Variable-width CLLW OPE ciphertext

Lines 31, 33, and 37 have the same issue in the second function's doc block.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/ope_cllw_var_8/functions.sql` around lines 6 - 16, The doc comments for
the functions that extract the variable-width OPE term refer to "CLWW" but the
correct acronym and type name is "CLLW"; update all occurrences of "CLWW" to
"CLLW" in the comment blocks for the functions related to ope_cllw_var_8 (the
extract function whose header mentions "Extract variable-width CLWW OPE..." and
the second function's doc block referenced in the review) so the documentation
matches the function/type name eql_v2.ope_cllw_var_8 and related references
(also update any `@see` lines that mistakenly say CLWW).
🧹 Nitpick comments (1)
src/operators/order_by.sql (1)

55-63: Post-install pinner will silently remove inlinability.

The PR discussion already documents this, but it's worth restating concisely for reviewers: tasks/pin_search_path.sql runs after source installation and ALTER FUNCTIONs every function to add SET search_path = ..., which sets pg_proc.proconfig and disables PostgreSQL function inlining. The whole point of converting these helpers to LANGUAGE sql is planner inlining (e.g., index matching on (eql_v2.order_by_ope(col))), but CI won't catch the silent regression.

Two changes are needed before this PR's performance goal survives a production install:

  1. Extend the inline_critical_oids filter in tasks/pin_search_path.sql to exclude order_by_ope, ope_cllw_u64_65, ope_cllw_var_8, has_ope_cllw_u64_65, and has_ope_cllw_var_8 (both encrypted and jsonb overloads).
  2. Add allowlist entries in tasks/test/splinter.sh so the Splinter function_search_path_mutable lint doesn't flag these functions.

A reference implementation for the same fix pattern exists in commit 1ab0b00 on PR #196.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/operators/order_by.sql` around lines 55 - 63, Extend the inlining
whitelist in tasks/pin_search_path.sql by adding exclusions for
eql_v2.order_by_ope and its helper functions eql_v2.ope_cllw_u64_65,
eql_v2.ope_cllw_var_8, eql_v2.has_ope_cllw_u64_65 and eql_v2.has_ope_cllw_var_8
(for both encrypted and jsonb overloads) so they are not ALTERed to set
search_path (preserving LANGUAGE sql inlinability), and add corresponding
allowlist entries in tasks/test/splinter.sh for the Splinter lint key
function_search_path_mutable to avoid CI failures; use the exact symbol names
above to locate the filter and splinter allowlist blocks.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/ope_cllw_var_8/functions.sql`:
- Around line 6-16: The doc comments for the functions that extract the
variable-width OPE term refer to "CLWW" but the correct acronym and type name is
"CLLW"; update all occurrences of "CLWW" to "CLLW" in the comment blocks for the
functions related to ope_cllw_var_8 (the extract function whose header mentions
"Extract variable-width CLWW OPE..." and the second function's doc block
referenced in the review) so the documentation matches the function/type name
eql_v2.ope_cllw_var_8 and related references (also update any `@see` lines that
mistakenly say CLWW).

---

Nitpick comments:
In `@src/operators/order_by.sql`:
- Around line 55-63: Extend the inlining whitelist in tasks/pin_search_path.sql
by adding exclusions for eql_v2.order_by_ope and its helper functions
eql_v2.ope_cllw_u64_65, eql_v2.ope_cllw_var_8, eql_v2.has_ope_cllw_u64_65 and
eql_v2.has_ope_cllw_var_8 (for both encrypted and jsonb overloads) so they are
not ALTERed to set search_path (preserving LANGUAGE sql inlinability), and add
corresponding allowlist entries in tasks/test/splinter.sh for the Splinter lint
key function_search_path_mutable to avoid CI failures; use the exact symbol
names above to locate the filter and splinter allowlist blocks.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: bc58da1e-d2f3-40bd-8afd-505ac5daad91

📥 Commits

Reviewing files that changed from the base of the PR and between 30aa13f and 4efadbd.

📒 Files selected for processing (8)
  • docs/reference/sql-support.md
  • src/ope_cllw_u64_65/compare.sql
  • src/ope_cllw_u64_65/functions.sql
  • src/ope_cllw_u64_65/types.sql
  • src/ope_cllw_var_8/compare.sql
  • src/ope_cllw_var_8/functions.sql
  • src/operators/order_by.sql
  • tests/sqlx/tests/ope_tests.rs
✅ Files skipped from review due to trivial changes (1)
  • docs/reference/sql-support.md
🚧 Files skipped from review as they are similar to previous changes (4)
  • src/ope_cllw_u64_65/compare.sql
  • src/ope_cllw_u64_65/types.sql
  • src/ope_cllw_u64_65/functions.sql
  • src/ope_cllw_var_8/compare.sql

@freshtonic freshtonic merged commit 9893162 into main May 7, 2026
6 checks passed
@freshtonic freshtonic deleted the fix/ope-index-type-docs branch May 7, 2026 06:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants