Skip to content

[phase-31 2/4] Compaction metadata types#6258

Open
g-talbot wants to merge 1 commit intogtt/phase-31-sort-schemafrom
gtt/phase-31-compaction-metadata
Open

[phase-31 2/4] Compaction metadata types#6258
g-talbot wants to merge 1 commit intogtt/phase-31-sort-schemafrom
gtt/phase-31-compaction-metadata

Conversation

@g-talbot
Copy link
Copy Markdown

Summary

Extend MetricsSplitMetadata with 6 compaction fields and update PostgreSQL model types (Phase 31 Metadata Foundation, PR 2 of 4).

Stacks on gtt/phase-31-sort-schema (PR #6242).

What's included

split/metadata.rs:

  • 6 new fields on MetricsSplitMetadata: window_start, window_duration_secs, sort_fields, num_merge_ops, row_keys_proto, zonemap_regexes
  • parquet_files: Vec<String> field (replaces single-file assumption)
  • Builder methods for all new fields
  • TW-1 and TW-2 debug_assert! invariants from ADR-003
  • Backward-compat serde defaults for pre-Phase-31 JSON

split/postgres.rs:

  • Compaction columns added to InsertableMetricsSplit and PgMetricsSplit
  • from_metadata() populates new SQL columns
  • SS-5 debug assertions for JSON/SQL column consistency

schema/fields.rs:

  • ParquetField::from_name() for name-based field lookup (used by sort field resolver)

Verification

  • cargo build -p quickwit-parquet-engine
  • cargo test -p quickwit-parquet-engine -- split::metadata split::postgres ✅ (11 tests)
  • cargo clippy -p quickwit-parquet-engine --all-features --tests

Test plan

  • Pre-Phase-31 JSON backward compatibility test
  • Round-trip test with all compaction fields
  • skip_serializing_if test for optional fields
  • Postgres model round-trip test with compaction columns
  • Clippy clean

(Replaces #6243 which was accidentally auto-merged during a rebase)

🤖 Generated with Claude Code

… model, field lookup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@g-talbot g-talbot force-pushed the gtt/phase-31-compaction-metadata branch from 598de1a to c95095b Compare March 31, 2026 21:50
@g-talbot g-talbot force-pushed the gtt/phase-31-sort-schema branch from 018a265 to 3696bb4 Compare March 31, 2026 21:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant