Skip to content

Add getIonDeserializer% and getIonSerializer% elaborators for generic Ion serialization#1095

Draft
keyboardDrummer-bot wants to merge 7 commits intomainfrom
bot/ion-deserializer
Draft

Add getIonDeserializer% and getIonSerializer% elaborators for generic Ion serialization#1095
keyboardDrummer-bot wants to merge 7 commits intomainfrom
bot/ion-deserializer

Conversation

@keyboardDrummer-bot
Copy link
Copy Markdown
Collaborator

Closes #5

Summary

Adds two term-level elaborators for generic Ion serialization/deserialization of Lean types:

  • getIonDeserializer% — generates a function ByteArray → Except Std.Format α that deserializes Ion binary data into values of a given Lean type at compile time.
  • getIonSerializer% — generates Java source files (records, sealed interfaces) with toIon methods that serialize to the same Ion format.

Design

Since Type is erased at runtime in Lean, both elaborators inspect the type's constructors and fields in the Lean environment at elaboration time and produce syntax that is then type-checked normally.

Ion encoding conventions

Lean type Ion encoding
Structures Ion struct with field names as keys
Single-constructor inductives Ion struct with positional keys _0, _1, …
Multi-constructor inductives Ion sexp (ConstructorName arg₁ arg₂ …)

Supported leaf types

Nat, Int, Float, String, Bool, Decimal

Container types

List α → Ion list / java.util.List<T>, Option α → Ion null for none / nullable T

Nested and recursive types

Fields whose types are themselves structures or inductives are handled recursively. For recursive types (e.g., Tree), the generated code uses let rec bindings — the enclosing definition must be marked partial.

Usage

-- Deserializer
def deserializePoint : ByteArray → Except Std.Format Point :=
  getIonDeserializer% Point

-- Java serializer
def pointJava := getIonSerializer% Point "com.example"

Files

  • Strata/Util/IonDeserializer.lean — Runtime helpers and the getIonDeserializer% elaborator
  • Strata/DDM/Integration/Java/Gen.lean — Rewritten to provide getIonSerializer% for Lean types
  • Strata/DDM/Integration/Java/GenDDM.lean — Original DDM-based Java generator (preserved for javaGen CLI)
  • StrataTest/Util/IonDeserializer.lean — Tests for the deserializer
  • StrataTestExtra/DDM/Integration/Java/TestGen.lean — Rewritten tests for the serializer

Testing

  • All existing and new tests pass (583 test jobs).
  • Java compilation and roundtrip tests verify that generated Java code compiles with javac and produces Ion data that getIonDeserializer% can read back correctly.
  • Validated for JVerify compatibility: Switch Laurel AST to getIonSerializer% format jverify#405

Implements a term-level elaborator that inspects Lean inductive and
structure types at compile time and generates a ByteArray → Except
Std.Format α deserializer.

Encoding conventions:
- Structures → Ion structs with field names as keys
- Single-constructor inductives → Ion structs with _0, _1, … keys
- Multi-constructor inductives → Ion sexps (CtorName arg1 arg2 …)
- Supported leaf types: Nat, Int, String, Bool

Closes #5
- Add readFloat runtime helper (accepts Ion float and int values)
- Add Float as a supported leaf type in mkFieldRead and mkIndexRead
- Support nested types: fields that are structures/inductives generate
  readers via let rec bindings in dependency order
- Support recursive types: self-referencing types work when the
  enclosing definition is marked partial
- Add tests for Float (Measurement), nested (Line with Point fields),
  and recursive (Tree) types
- Rewrite Strata/DDM/Integration/Java/Gen.lean to generate Java source
  files from Lean types instead of DDM Dialect values
- New getIonSerializer% term elaborator inspects Lean inductive/structure
  types at compile time and generates:
  - Sealed interfaces for multi-constructor inductives
  - Records for structures and single-constructor inductives
  - Ion serialization matching getIonDeserializer% format:
    - Structures → Ion struct with field name keys
    - Single-ctor inductives → Ion struct with _0, _1, ... keys
    - Multi-ctor inductives → Ion sexp (CtorName arg1 arg2 ...)
- Supported leaf types: Nat, Int, Float, String, Bool
- Nested and recursive types supported automatically
- Remove old javaGen CLI command from StrataMain.lean
- Rewrite tests to use Lean types (Point, Color, Shape, Person, Line, Tree)
- Add Java compilation test and Ion roundtrip test
  (Java serializes → Lean deserializes → verify match)
# Conflicts:
#	StrataMain.lean
#	StrataTestExtra/DDM/Integration/Java/TestGen.lean
#	StrataTestExtra/DDM/Integration/Java/regenerate-testdata.sh
…rializer

- Add readDecimal, readList, readOption runtime helpers to IonDeserializer
- Extend getIonDeserializer% to handle List α, Option α, and Strata.Decimal
- Extend getIonSerializer% to generate Java code for List (java.util.List),
  Option (nullable), and Decimal (java.math.BigDecimal)
- Move old DDM-based Java generator to GenDDM.lean to preserve javaGen CLI
- Remove support for MetaData, MetaDataElem, Core.Expression, and Array
  (no longer in Laurel AST)
- Add tests for new types in StrataTest/Util/IonDeserializer.lean
The javac invocation was inside the HashMap iteration loop, causing it to
compile partial sets of files. With non-deterministic HashMap ordering,
this could fail when a file referencing another type was compiled before
that type's source file was written.

Also added the output directory to -cp so javac can find compiled classes
from other files in the same package, and added cleanup at the end.
The CheckImports linter requires all modules under Strata/ to be
transitively imported by Strata.lean. The new IonDeserializer module
was missing from the import list, causing the lint step to fail in CI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants