Protobuf transform handling with default values #18792
Description
Affected Version
Druid 33.0.0
Description
There appears to be an issue in the Protobuf transform handling where default-valued fields in non-optional proto definitions are being set to null during ingestion.
Observed behavior:
- When a non-optional proto field (for example, an int32 with a default value of 0) is used inside a transform expression, the resulting value becomes null instead of retaining its default value.
- This behavior does not occur with optional fields. Optional fields behave as expected.
- Because of this, we are forced to wrap non-optional proto fields with functions like
nvl()or similar null-handling expressions to avoid unexpected nulls in the ingestion pipeline.
From code inspection:
- I think this problem should be in convertMessage on /extensions-core/protobuf-extensions/src/main/java/org/apache/druid/data/input/protobuf/ProtobufConverter.java
- It looks like the getAllFields() logic, when a value is set to default, treats default proto values (like 0 for int32) as equivalent to "not set", resulting in the field being mapped to null.
- This appears inconsistent with Protobuf semantics, where default numeric values are valid values and not indicators of missing fields.