Describe
Auron Hudi native scan conversion checks several Hudi scan options and table properties by exact key match.
Spark datasource options are commonly handled case-insensitively. If a Hudi table or query provides mixed-case option keys, Auron may fail to detect important scan properties such as table type, base file format, table path, or time travel options.
For example, a MOR table type option like Hoodie.DataSource.Write.Table.Type=MERGE_ON_READ may not be detected, so Auron could make an unsafe native scan conversion decision.
Describe the solution you'd like
Resolve Hudi scan options and table/catalog properties case-insensitively when detecting native scan support.
The lookup should cover:
- Hudi table type options
- Hudi base file format options
- Hudi table path option
- Hudi time travel options
- Hudi catalog and storage properties
- Hudi table properties loaded from
.hoodie/hoodie.properties
Add unit tests for mixed-case Hudi option keys.
Additional context
This is a compatibility and correctness improvement for Hudi native scan conversion. It does not expand the supported Hudi query types; it makes existing conversion decisions more reliable when option key casing differs.
Describe
Auron Hudi native scan conversion checks several Hudi scan options and table properties by exact key match.
Spark datasource options are commonly handled case-insensitively. If a Hudi table or query provides mixed-case option keys, Auron may fail to detect important scan properties such as table type, base file format, table path, or time travel options.
For example, a MOR table type option like
Hoodie.DataSource.Write.Table.Type=MERGE_ON_READmay not be detected, so Auron could make an unsafe native scan conversion decision.Describe the solution you'd like
Resolve Hudi scan options and table/catalog properties case-insensitively when detecting native scan support.
The lookup should cover:
.hoodie/hoodie.propertiesAdd unit tests for mixed-case Hudi option keys.
Additional context
This is a compatibility and correctness improvement for Hudi native scan conversion. It does not expand the supported Hudi query types; it makes existing conversion decisions more reliable when option key casing differs.