[RFC] Add BOLT 12 payer proof primitives#4297
[RFC] Add BOLT 12 payer proof primitives#4297vincenzopalazzo wants to merge 9 commits intolightningdevkit:mainfrom
Conversation
|
👋 Thanks for assigning @jkczyz as a reviewer! |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #4297 +/- ##
==========================================
+ Coverage 85.88% 86.23% +0.34%
==========================================
Files 159 161 +2
Lines 104448 108587 +4139
Branches 104448 108587 +4139
==========================================
+ Hits 89706 93640 +3934
- Misses 12235 12288 +53
- Partials 2507 2659 +152
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
TheBlueMatt
left a comment
TheBlueMatt
left a comment
There was a problem hiding this comment.
A few notes, though I didn't dig into the code at a particularly low level.
vincenzopalazzo
force-pushed
the
macros/proof-of-payment-bolt12-spec
branch
2 times, most recently
from
2324361 to
9f84e19
Compare
Add a Rust CLI tool that generates and verifies test vectors for BOLT 12 payer proofs as specified in lightning/bolts#1295. The tool uses the rust-lightning implementation from lightningdevkit/rust-lightning#4297. Features: - Generate deterministic test vectors with configurable seed - Verify test vectors from JSON files - Support for basic proofs, proofs with notes, and invalid test cases - Uses refund flow for explicit payer key control Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
🔔 1st Reminder Hey @valentinewallace! This PR has been waiting for your review. |
TheBlueMatt
left a comment
TheBlueMatt
left a comment
There was a problem hiding this comment.
Some API comments. I'll review the actual code somewhat later (are we locked on on the spec or is it still in flux at all?), but would be nice to reduce allocations in it first anyway.
|
🔔 2nd Reminder Hey @valentinewallace! This PR has been waiting for your review. |
|
🔔 1st Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 2nd Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 3rd Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 4th Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 5th Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 6th Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 7th Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 8th Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 9th Reminder Hey @jkczyz! This PR has been waiting for your review. |
vincenzopalazzo
force-pushed
the
macros/proof-of-payment-bolt12-spec
branch
5 times, most recently
from
fb8c68c to
9ad5c35
Compare
| fn build_unsigned(self) -> Result<UnsignedPayerProof, PayerProofError> { | ||
| let mut invoice_bytes = Vec::new(); | ||
| self.invoice.write(&mut invoice_bytes).expect("Vec write should not fail"); | ||
| let mut bytes_without_sig = Vec::with_capacity(invoice_bytes.len()); | ||
| for r in TlvStream::new(&invoice_bytes).filter(|r| !SIGNATURE_TYPES.contains(&r.r#type)) { | ||
| bytes_without_sig.extend_from_slice(r.record_bytes); | ||
| } | ||
|
|
||
| let disclosure = | ||
| merkle::compute_selective_disclosure(&bytes_without_sig, &self.included_types)?; |
There was a problem hiding this comment.
Nit: bytes_without_sig includes ALL non-signature TLVs from the invoice, including experimental invoice types (3_000_000_000+) and any unknown types in ranges 1001-999_999_999. These get processed by compute_selective_disclosure and will be omitted by default (since they're not in included_types). This interacts with the trailing_count > 1 validation bug noted on validate_omitted_markers_for_parsing — invoices with experimental types will produce unparseable proofs.
Consider documenting that the TLV stream fed to compute_selective_disclosure should match the set of TLVs that contribute to the invoice's merkle root, and ensuring the producer and consumer agree on the handling of trailing omitted types.
There was a problem hiding this comment.
Not a practical issue -- bytes_without_sig is built from a Bolt12Invoice that was already parsed and validated. Any experimental TLVs present in the invoice are legitimate and should be part of the merkle tree. The include_type() guard (now rejecting types >= 240) prevents them from being included in disclosure, so they become omitted markers/missing_hashes. This is the correct behavior per spec.
|
|
||
| // Build TreeNode vec directly by interleaving included/omitted positions, | ||
| // eliminating the intermediate Vec<bool> from reconstruct_positions_from_records. | ||
| let num_nodes = 1 + included_records.len() + omitted_markers.len(); | ||
| let mut nodes: Vec<TreeNode> = Vec::with_capacity(num_nodes); | ||
|
|
||
| // TLV0 is always omitted |
There was a problem hiding this comment.
The implicit TLV0 node prepended here assumes the original TLV stream starts with type 0 (payer_metadata). This is true for BOLT 12 invoices, but the function's API doesn't enforce or document this invariant.
If compute_selective_disclosure is called on a TLV stream that doesn't contain type 0 (as in the unit test test_selective_disclosure_computation with types [1, 2]), the producer creates a 2-node tree but this reconstruction creates a 3-node tree (with the extra TLV0 node), causing InsufficientMissingHashes because the needs_hash count won't match missing_hashes.
Consider either:
- Adding a
debug_assert!thatomitted_markersandincluded_recordsare consistent with the TLV0 assumption, or - Documenting on both
compute_selective_disclosureandreconstruct_merkle_rootthat the TLV stream MUST start with type 0.
There was a problem hiding this comment.
Same as above — TLV0 is always prepended in payer proof context. The function is private (pub(super)) and only called from TryFrom<Vec<u8>> which always has TLV0. Adding a doc comment noting the invariant.
vincenzopalazzo
force-pushed
the
macros/proof-of-payment-bolt12-spec
branch
from
712d3fc to
64c7f9c
Compare
|
|
||
| // MUST NOT contain signature TLV types | ||
| if SIGNATURE_TYPES.contains(&marker) { | ||
| return Err(crate::ln::msgs::DecodeError::InvalidValue); |
There was a problem hiding this comment.
Nit (performance): The parser allocates two Secp256k1::verification_only() contexts (~0.6 MB each): one inside merkle::verify_signature (line 706) and another here. Consider passing a single context to both verification calls — e.g., by inlining the verify_signature logic or accepting a context parameter:
let secp_ctx = Secp256k1::verification_only();
// Verify invoice signature
let tagged_hash = TaggedHash::from_merkle_root(SIGNATURE_TAG, merkle_root);
let issuer_xonly: XOnlyPublicKey = issuer_signing_pubkey.into();
secp_ctx.verify_schnorr(&invoice_signature, tagged_hash.as_digest(), &issuer_xonly)
.map_err(|_| Bolt12ParseError::Decode(DecodeError::InvalidValue))?;
// Verify payer signature
let message = UnsignedPayerProof::compute_payer_signature_message(...);
secp_ctx.verify_schnorr(&payer_signature, &message, &payer_id.into())
.map_err(|_| Bolt12ParseError::Decode(DecodeError::InvalidValue))?;This halves the heap allocation for signature verification during parsing.
There was a problem hiding this comment.
Valid optimization for a follow-up. The double context allocation matches the pattern used in verify_signature and sign_message throughout the offers module. Refactoring to pass a shared context would touch the merkle::verify_signature API.
| use core::convert::TryFrom; | ||
|
|
||
| // Create a TLV stream with leaf_hashes (type 248) that has invalid length | ||
| // BigSize encoding: values 0-252 are single byte, 253-65535 use 0xFD prefix | ||
| let mut bytes = Vec::new(); | ||
| // TLV type 248 (leaf_hashes) - 248 < 253 so single byte | ||
| bytes.push(0xf8); // type 248 | ||
| bytes.push(0x1f); // length 31 (not multiple of 32!) |
There was a problem hiding this comment.
Bug: derive_payer_signing_keys selects REFUND_IV_BYTES_WITHOUT_METADATA for refund-based invoices (line 1062), but neither integration test exercises the refund path — both use create_offer_builder. If the IV byte selection is wrong for refunds, build_with_derived_key would return KeyDerivationFailed with no test coverage catching it.
Consider adding a test that creates a payer proof from a refund-based invoice to validate this code path.
There was a problem hiding this comment.
Valid point on test coverage. The refund path for derive_payer_signing_keys is tested indirectly through the existing refund/invoice test infrastructure, but adding an explicit payer proof integration test using the refund flow would be good. I'll add that in a follow-up.
vincenzopalazzo
force-pushed
the
macros/proof-of-payment-bolt12-spec
branch
2 times, most recently
from
88ffb7c to
84d2374
Compare
|
Quick update after the latest push. I cherry-picked the follow-up fixes that came out of the Codex/spec review. What changed:
I also updated the external test vectors here: At this point, the real review bugs found around the producer/parser mismatch are fixed on the branch. The one discussion I still consider open is the The other remaining review threads look like follow-up material to me (TLV0 invariant docs, parser shape, secp context reuse), not missing correctness fixes. |
Move the invoice/refund payer key derivation logic into reusable helpers so payer proofs can derive the same signing keys without duplicating the metadata and signer flow.
Add the payer proof types, selective disclosure merkle support, parsing, and tests for constructing and validating BOLT 12 payer proofs from invoices.
Add regression coverage for payer proofs that disclose experimental TLVs above the reserved signature range. The test builds a proof that includes an experimental invoice TLV and asserts that the proof survives a full byte round-trip. This captures the spec-compliance issue found during review against BOLTs PR lightningdevkit#1295, where only 240..=1000 is reserved for signature-related TLVs.
Treat only the 240..=1000 range as reserved for payer-proof and signature TLVs. This updates the inclusion gate, preserves TLV ordering when serializing proofs that contain experimental records, and teaches the parser to accept non-signature invoice TLVs above 1000. The change makes the implementation match the spec text reviewed in BOLTs PR lightningdevkit#1295 while keeping the experimental round-trip test green.
Add a round-trip test for proofs that omit multiple TLVs after the last disclosed field. The writer already emits this shape when trailing fields are withheld, so the new test documents that the parser must accept it as well. This locks in the review finding from the BOLTs PR discussion before the parser change lands.
Add a compile-time and runtime regression test for selectively disclosed fields. The new test constructs a proof that reveals description, issuer, invoice amount, and creation time, then asserts that a parsed proof can expose those values. This captures the API gap found during review: the proof carried the bytes needed for verification, but discarded the disclosed values themselves.
Preserve selected invoice fields inside PayerProof and add accessors for verifiers. The parser and builder now both populate a shared disclosed-field structure so locally produced proofs and parsed proofs expose the same API surface. This keeps selective disclosure useful to callers instead of limiting the proof to merkle reconstruction and signature verification only.
vincenzopalazzo
force-pushed
the
macros/proof-of-payment-bolt12-spec
branch
from
8ca2247 to
44ecb16
Compare
Fix the formatting drift introduced by the payer proof follow-up cherry-picks and update the stale splicing test to use the current RecipientOnionFields::secret_only signature so the CI build matrix compiles again.
Allow payer proof merkle reconstruction to reuse hashes placed at omitted subtree roots instead of descending into already materialized descendants.
vincenzopalazzo
force-pushed
the
macros/proof-of-payment-bolt12-spec
branch
from
c96cef5 to
d5baee8
Compare
5 participants
This is a first draft implementation of the payer proof extension to BOLT 12 as proposed in lightning/bolts#1295. The goal is to get early feedback on the API design before the spec is finalized.
Payer proofs allow proving that a BOLT 12 invoice was paid by demonstrating possession of:
This PR adds the core building blocks:
This is explicitly a PoC to validate the API surface - the spec itself is still being refined. Looking for feedback on:
cc @TheBlueMatt @jkczyz