Skip to content

[phase-31 1/4] Sort schema foundation#6242

Open
g-talbot wants to merge 1 commit intomatthew.kim/metrics-wide-schemafrom
gtt/phase-31-sort-schema
Open

[phase-31 1/4] Sort schema foundation#6242
g-talbot wants to merge 1 commit intomatthew.kim/metrics-wide-schemafrom
gtt/phase-31-sort-schema

Conversation

@g-talbot
Copy link
Copy Markdown

Summary

Sort schema types, parsing, and time-window arithmetic for metrics compaction (Phase 31 Metadata Foundation, PR 1 of 4).

Stacks on matthew.kim/metrics-wide-schema.

What's included

quickwit-proto:

  • Vendored event_store_sortschema.proto (SortSchema, SortColumn, RowKeys, ColumnValue types)
  • SortFieldsError error type for parser/validation errors
  • From<SortFieldsError> impl for MetastoreError

quickwit-parquet-engine:

  • sort_fields/ module:
    • column_type.rs — ColumnTypeId enum matching Go iota values, suffix/name resolution
    • parser.rs — Direct port of Go StringToSchema with V2-only enforcement
    • display.rsschema_to_string / schema_to_string_short (port of Go SchemaToString)
    • equivalence.rs — Schema comparison for compaction (port of Go EquivalentSchemas)
    • validation.rs — Schema validation rules (port of Go ValidateSchema)
    • window.rswindow_start() with rem_euclid for negative timestamp correctness, validate_window_duration() with hour-divisibility constraint
    • tests.rs — 97 tests including Go test port, proptests for window invariants
  • table_config.rsTableConfig with per-product-type default sort fields, serde support

Verification

  • cargo build -p quickwit-parquet-engine
  • cargo test -p quickwit-parquet-engine -- sort_fields table_config ✅ (97 tests)
  • cargo clippy -p quickwit-parquet-engine --all-features --tests ✅ (no new warnings)

Test plan

  • All 97 sort_fields and table_config tests pass
  • Proptest properties verified (window alignment, containment, determinism)
  • Round-trip tests for parse → display → parse
  • Go test suite fully ported
  • Clippy clean

🤖 Generated with Claude Code

@mattmkim mattmkim force-pushed the matthew.kim/metrics-wide-schema branch 5 times, most recently from 0b642de to 7a5979f Compare March 30, 2026 18:45
@fulmicoton-dd
Copy link
Copy Markdown
Collaborator

@mattmkim I only reviewed the changes in the global scope. Can you complete the review?

@g-talbot g-talbot force-pushed the gtt/phase-31-sort-schema branch 8 times, most recently from 8fce718 to 018a265 Compare March 31, 2026 21:40
@g-talbot g-talbot force-pushed the gtt/phase-31-sort-schema branch 3 times, most recently from a90ca2f to d60973e Compare April 1, 2026 11:02
…, window, TableConfig

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@g-talbot g-talbot force-pushed the gtt/phase-31-sort-schema branch from d60973e to cf9484d Compare April 1, 2026 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants