fix: Decode avro with negative count and no tags avro issue#865
Open
speeddragon wants to merge 2 commits into
Open
fix: Decode avro with negative count and no tags avro issue#865speeddragon wants to merge 2 commits into
speeddragon wants to merge 2 commits into
Conversation
802b7f1 to
d786bbe
Compare
d786bbe to
16e0b53
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requesting https://arweave.net/raw/sfuxzQEEIFo5w6swrIPNjqUXCkaRm1BiuP5E3tmuNeU causes the header to be parsed incorrectly, leading to issues with non-UTF-8 binaries.
What's happening:
The Apache Avro array encoding has two forms:
The original decode_avro_tags only handled positive counts. The TXID
sfuxzQEEIFo5w6swrIPNjqUXCkaRm1BiuP5E3tmuNeUis a real-world transaction whose tags were encoded using the long-form block (negative count), causing a crash/mismatch on deserialization.Your current branch already has the fix (lines 594–598):
This is standard Avro spec, not a custom encoding — it reads and discards the byte size, then recurses with abs(Count). The test at the bottom of the diff encodes the actual TX binary from that TXID and asserts the two expected tags (IPFS-Hash and Content-Type).
The fix is correct and complete. Nothing else needs to change — it's a missing clause in the existing Avro decoder, not a separate custom decoder.
This includes a second fix, on tx with no tags, like
8cjDy2khfMsc3hrvGp7PrLVYfD_4aYQxEILNSZ0Pv74