[RFC] Add BOLT 12 payer proof primitives#4297
Conversation
|
👋 Thanks for assigning @TheBlueMatt as a reviewer! |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #4297 +/- ##
==========================================
+ Coverage 86.95% 87.11% +0.16%
==========================================
Files 161 163 +2
Lines 111668 113777 +2109
Branches 111668 113777 +2109
==========================================
+ Hits 97098 99116 +2018
- Misses 12063 12109 +46
- Partials 2507 2552 +45
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
TheBlueMatt
left a comment
There was a problem hiding this comment.
A few notes, though I didn't dig into the code at a particularly low level.
2324361 to
9f84e19
Compare
Add a Rust CLI tool that generates and verifies test vectors for BOLT 12 payer proofs as specified in lightning/bolts#1295. The tool uses the rust-lightning implementation from lightningdevkit/rust-lightning#4297. Features: - Generate deterministic test vectors with configurable seed - Verify test vectors from JSON files - Support for basic proofs, proofs with notes, and invalid test cases - Uses refund flow for explicit payer key control Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
🔔 1st Reminder Hey @valentinewallace! This PR has been waiting for your review. |
TheBlueMatt
left a comment
There was a problem hiding this comment.
Some API comments. I'll review the actual code somewhat later (are we locked on on the spec or is it still in flux at all?), but would be nice to reduce allocations in it first anyway.
|
🔔 2nd Reminder Hey @valentinewallace! This PR has been waiting for your review. |
|
🔔 1st Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 2nd Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 3rd Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 4th Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 5th Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 6th Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 7th Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 8th Reminder Hey @jkczyz! This PR has been waiting for your review. |
|
🔔 9th Reminder Hey @jkczyz! This PR has been waiting for your review. |
fb8c68c to
9ad5c35
Compare
InvoiceRequest and Refund have payer metadata consisting of an encrypted payment id and, originally, a nonce used to derive the payer signing keys and authenticate any corresponding invoices. The nonce was elided to save space once it was included in the OffersContext of blinded reply paths, but that means verifying a Bolt12Invoice requires state outside the invoice itself. Upcoming payment proofs (lightningdevkit#4297) need the invoice signing keys derivable from the invoice request alone, so include the nonce in the payer metadata again and verify invoices using it rather than the context's nonce. Co-Authored-By: Claude <noreply@anthropic.com>
InvoiceRequest and Refund have payer metadata consisting of an encrypted payment id and, originally, a nonce used to derive the payer signing keys and authenticate any corresponding invoices. The nonce was elided to save space once it was included in the OffersContext of blinded reply paths, but that means verifying a Bolt12Invoice requires state outside the invoice itself. Upcoming payment proofs (lightningdevkit#4297) need the invoice signing keys derivable from the invoice request alone, so include the nonce in the payer metadata again and verify invoices using it rather than the context's nonce. This breaks verification of invoices for invoice requests and refunds with blinded paths created by prior versions, as their payer metadata lacks the nonce; such payments will fail and must be retried with a new payment id. Refunds without blinded paths are unaffected, as their metadata always included the nonce. Co-Authored-By: Claude <noreply@anthropic.com>
93f4104 to
73eea68
Compare
8d543ce to
5aedb03
Compare
| while inc_idx < included_types.len() || mrk_idx < omitted_markers.len() { | ||
| if mrk_idx >= omitted_markers.len() { | ||
| // No more markers, remaining positions are included | ||
| positions.push(true); | ||
| inc_idx += 1; | ||
| } else if inc_idx >= included_types.len() { | ||
| // No more included types, remaining positions are omitted | ||
| positions.push(false); | ||
| prev_marker = omitted_markers[mrk_idx]; | ||
| mrk_idx += 1; | ||
| } else { | ||
| let marker = omitted_markers[mrk_idx]; | ||
| let inc_type = included_types[inc_idx]; | ||
|
|
||
| if marker == next_marker(prev_marker) { | ||
| // Continuation of current run → this position is omitted | ||
| positions.push(false); | ||
| prev_marker = marker; | ||
| mrk_idx += 1; | ||
| } else { | ||
| // Jump detected! An included TLV comes before this marker. | ||
| // After the included type, prev_marker resets to that type, | ||
| // so the marker will be processed as a continuation next iteration. | ||
| positions.push(true); | ||
| prev_marker = inc_type; | ||
| inc_idx += 1; | ||
| // Don't advance mrk_idx - same marker will be continuation next | ||
| } | ||
| } |
There was a problem hiding this comment.
Can't this drift from what's essentially the same loop in reconstruct_merkle_root? We should DRY this up.
There was a problem hiding this comment.
What do you think about this solution? 564d463
|
|
||
| #[inline] | ||
| pub fn do_test<Out: test_logger::Output>(data: &[u8], _out: Out) { | ||
| if let Ok(payer_proof) = PayerProof::try_from(data.to_vec()) { |
There was a problem hiding this comment.
We should also update the invoice fuzzer to build a payer proof, just like how the offer fuzzer builds an invoice request and how the invoice request fuzzer builds an invoice.
There was a problem hiding this comment.
The preimage gate makes this a no-op if I put it in invoice_deser. Building a proof requires SHA256(preimage) == invoice.payment_hash(), and invoice_deser parses arbitrary invoices, so we never hold a matching preimage. prove_payer() returns PreimageMismatch before any of the interesting code runs (build_unsigned doesn't even re-check the invoice signature, the preimage is the only gate), so we'd just be exercising the early error return.
Claude help me find the place where we actually control the payment hash is invoice_request_deser, since it builds the invoice itself via respond_with(paths, payment_hash). If I set payment_hash = SHA256(known_preimage) there, I can chain a full payer proof build+sign right after the invoice is signed, which exercises the selective disclosure, merkle tree, and proof signing for real.
Want me to add it in invoice_request_deser that way? Or do you still want the call in invoice_deser even though it only ever hits PreimageMismatch?
|
🔔 1st Reminder Hey @TheBlueMatt @jkczyz! This PR has been waiting for your review. |
1 similar comment
|
🔔 1st Reminder Hey @TheBlueMatt @jkczyz! This PR has been waiting for your review. |
TheBlueMatt
left a comment
There was a problem hiding this comment.
Didn't get through payer_proof.rs but I'm happy with everything else with these comments addressed.
| let (res, _) = | ||
| claim_payment_along_route(ClaimAlongRouteArgs::new(sender, route, keysend_preimage)); | ||
| assert_eq!(res, Some(PaidBolt12Invoice::StaticInvoice(static_invoice.clone()))); | ||
| assert_eq!(res.as_ref().and_then(|paid| paid.static_invoice()), Some(&static_invoice)); |
There was a problem hiding this comment.
I think the diff in this file can be dropped.
There was a problem hiding this comment.
Done. Dropped the unnecessary diff in this file; the PR diff no longer changes async_payments_tests.rs for this cleanup.
| match self { | ||
| Self::OutboundRoute { | ||
| bolt12_invoice: Some(PaidBolt12Invoice::StaticInvoice(inv)), | ||
| bolt12_invoice: Some(PaidBolt12Invoice::StaticInvoice(invoice)), |
There was a problem hiding this comment.
This looks like unnecessary diff.
There was a problem hiding this comment.
Done. Dropped the unnecessary diff in this file; the PR diff no longer changes channelmanager.rs for this cleanup.
| let (inv, _) = claim_payment_along_route(args); | ||
| assert_eq!(inv, Some(PaidBolt12Invoice::Bolt12Invoice(invoice.clone()))); | ||
| let (paid_invoice, _) = claim_payment_along_route(args); | ||
| assert_eq!(paid_invoice.as_ref().and_then(|paid| paid.bolt12_invoice()), Some(invoice)); |
There was a problem hiding this comment.
Also unnecessary diff.
There was a problem hiding this comment.
Done. Dropped the unnecessary assertion churn at this helper; the remaining diff in this file is needed by the payer-proof round-trip test.
|
|
||
| use crate::blinded_path::{IntroductionNode, NodeIdLookUp}; | ||
| use crate::events::{self, PaidBolt12Invoice, PaymentFailureReason}; | ||
| use crate::events::{self, PaymentFailureReason}; |
There was a problem hiding this comment.
I believe all the diff in this file aside from the test at the end can probably be dropped.
There was a problem hiding this comment.
Done. Dropped the non-test churn; the remaining diff here is the retryable paid-invoice serialization round-trip test.
|
|
||
| let num_omitted_markers = tlv_data | ||
| .iter() | ||
| .filter(|data| !data.is_included && data.tlv_type != PAYER_METADATA_TYPE) |
There was a problem hiding this comment.
nit: honestly this constant is weird (and in compute_omitted_markers). Should we instead just require the higher-level (more BOLT12-aware) code always include PAYER_METADATA_TYPE?
There was a problem hiding this comment.
Good point. I agree the generic merkle code shouldn’t know about PAYER_METADATA_TYPE.
One clarification: in this code “included” means disclosed in the payer proof, and I don’t think we want to disclose the payer metadata TLV. But the BOLT12-aware payer-proof layer should own that policy instead of having merkle.rs special-case type 0.
I’ll rework this so payer-proof construction handles PAYER_METADATA_TYPE as the implicit/always-omitted payer metadata record, while keeping the merkle/selective-disclosure helper generic. Does that match what you had in mind?
| /// the encoding from [`compute_omitted_markers`] and is the single source of truth shared by | ||
| /// merkle-root reconstruction and the position map used in tests, so the two cannot drift. | ||
| /// | ||
| /// `prev_marker` tracks the previous run value: a marker equal to [`next_marker`] of it continues |
There was a problem hiding this comment.
nit: weird to document the function's internal operation in its docs.
There was a problem hiding this comment.
Done. Trimmed the docs so they describe the decoded position map instead of walking through the loop internals.
| fn build_tree_with_disclosure( | ||
| tlv_data: &[TlvMerkleData], branch_tag: &sha256::HashEngine, | ||
| ) -> (sha256::Hash, Vec<sha256::Hash>) { | ||
| debug_assert!(!tlv_data.is_empty(), "TLV stream must contain at least one record"); |
There was a problem hiding this comment.
We'll inf loop in build_tree_dfs anyway, so this isn't gonna do anything.
There was a problem hiding this comment.
Done. Removed the redundant debug_assert; the non-empty TLV invariant is already checked before entering the tree builder.
| ) -> Result<sha256::Hash, SelectiveDisclosureError> { | ||
| debug_assert!({ | ||
| let included_types: BTreeSet<u64> = included_records.iter().map(|r| r.r#type).collect(); | ||
| validate_omitted_markers(omitted_markers, &included_types).is_ok() |
There was a problem hiding this comment.
dont we need to do this and actually fail in case the omitted markers are wrong? IMO we at least need to fail if the omitted markers aren't always the next marker required, cause we want to be really strict about people screwing that up on the generation side.
| // X sits between the previous position and `marker` with `next_marker(X) == marker`. | ||
| if marker != expected_next { | ||
| let mut found = false; | ||
| for inc_type in inc_iter.by_ref() { |
There was a problem hiding this comment.
if we start doing this in non-debug, it might be worth keeping a single iterator of the included types, rather than building anew iterator for each marker - this should be doable because we only care about included types < marker.
| signing_pubkey: PublicKey, tlv_stream: impl core::iter::Iterator<Item = TlvRecord<'a>>, | ||
| secp_ctx: &Secp256k1<T>, | ||
| ) -> Result<Keypair, ()> { | ||
| if metadata.len() < PaymentId::LENGTH { |
There was a problem hiding this comment.
nit: move this into verify_payer_metadata_inner.
There was a problem hiding this comment.
Done. Moved the metadata length check into verify_payer_metadata_inner, so both callers go through the same validation path.
|
🔔 2nd Reminder Hey @jkczyz! This PR has been waiting for your review. |
Move the invoice/refund payer key derivation logic into reusable helpers so payer proofs can derive the same signing keys without duplicating the metadata and signer flow.
Extend the BOLT 12 merkle module with selective-disclosure support: build the full merkle tree from a TLV stream, compute the omitted-TLV markers and the minimal set of missing hashes for omitted subtrees, and reconstruct the merkle root from a partial disclosure. These are the primitives a payer proof is built on. Co-Authored-By: Rusty Russell <rusty@rustcorp.com.au> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-Authored-By: OpenAI Codex <codex@openai.com>
Add the `payer_proof` module: `PayerProof`/`UnsignedPayerProof`, the `PayerProofBuilder` (with selective disclosure and a derived-key path), bech32 `lnp` encoding, and parse-time verification, implementing the payer proof extension to BOLT 12 (lightning/bolts#1295). Also exposes the offer/invoice TLV-type constants and an invoice-bytes accessor used to build proofs, and a `Sha256` `Writeable`/`Readable` impl for the proof hashes. Co-Authored-By: Rusty Russell <rusty@rustcorp.com.au> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-Authored-By: OpenAI Codex <codex@openai.com>
Carry the paid `Bolt12Invoice` through the outbound payment so it survives restarts, and surface it as a `PaidBolt12Invoice` on `Event::PaymentSent` so the payer can build a payer proof. The payer signing key is re-derived from the invoice's own payer metadata, so no extra key material is stored. `PaidBolt12Invoice` now lives in `offers::payer_proof`; existing async payment tests and a test helper are updated to construct it via the new API. Adds an end-to-end test that pays a BOLT 12 offer and builds + verifies a payer proof from the resulting `Event::PaymentSent`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Throw arbitrary bytes at `PayerProof::try_from` to exercise the merkle-root reconstruction and the deserialization path together. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Keep the per-TLV merkle hashing (`merkle_tlv_data`/`TlvMerkleData`) in `merkle.rs` and have the selective-disclosure module import it, instead of duplicating the block into both `root_hash` and `selective_disclosure.rs`. The duplicated copy computed the branch hashes that back every BOLT 12 signature (offers, invoice requests, invoices, static invoices and payer proofs), so two byte-identical copies that must never diverge is a maintenance hazard. Restoring the shared helper re-satisfies the earlier review request to DRY `root_hash` and the disclosure code, while keeping the disclosure types in their own module. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Pushed a small fixup in This finishes the One thing I got wrong in the first split: I moved It autosquashes into |
|
Playing a bit with the code after the last fixup, there is also a closure-free version of this split. Instead of passing the The tradeoff is that closure-free variant (not pushed)--- a/lightning/src/offers/merkle.rs
+++ b/lightning/src/offers/merkle.rs
-/// Per-TLV merkle data shared by [`root_hash`] and the selective-disclosure code: the branch hash
-/// of each non-signature record together with the pieces the disclosure layer needs to rebuild the
-/// tree. Keeping this in one place ensures the signed invoice root and the payer-proof
-/// reconstruction hash identical inputs the exact same way.
-pub(super) struct TlvMerkleData {
+/// The hashes computed for a single non-signature TLV record of the merkle tree. Shared by
+/// [`root_hash`] and the selective-disclosure reconstruction so both hash identical inputs.
+pub(super) struct TlvHashData {
pub(super) tlv_type: u64,
pub(super) nonce_hash: sha256::Hash,
pub(super) per_tlv_hash: sha256::Hash,
- pub(super) is_included: bool,
}
-pub(super) fn merkle_tlv_data<'a, I, F>(
- tlv_stream: I, mut is_included: F,
-) -> (impl Iterator<Item = TlvMerkleData> + 'a, sha256::HashEngine)
+pub(super) fn merkle_tlv_data<'a, I>(
+ tlv_stream: I,
+) -> (impl Iterator<Item = TlvHashData> + 'a, sha256::HashEngine)
where
I: core::iter::Iterator<Item = TlvRecord<'a>> + 'a,
- F: FnMut(u64) -> bool + 'a,
{
// ... nonce_tag / leaf_tag / branch_tag setup unchanged ...
- TlvMerkleData {
- tlv_type: record.r#type,
- nonce_hash,
- per_tlv_hash,
- is_included: is_included(record.r#type),
- }
+ TlvHashData { tlv_type: record.r#type, nonce_hash, per_tlv_hash }
});
(tlv_data, branch_tag)
}
fn root_hash<'a, I: core::iter::Iterator<Item = TlvRecord<'a>> + 'a>(
tlv_stream: I,
) -> sha256::Hash {
- let (tlv_data, branch_tag) = merkle_tlv_data(tlv_stream, |_| false);
+ let (tlv_data, branch_tag) = merkle_tlv_data(tlv_stream);
let mut leaves: Vec<sha256::Hash> = tlv_data.map(|data| data.per_tlv_hash).collect();
--- a/lightning/src/offers/selective_disclosure.rs
+++ b/lightning/src/offers/selective_disclosure.rs
use crate::offers::merkle::{
merkle_tlv_data, tagged_branch_hash_from_engine, tagged_hash_engine, tagged_hash_from_engine,
- TlvMerkleData, TlvRecord,
+ TlvHashData, TlvRecord,
};
+/// Per-TLV merkle data plus whether the record is disclosed in the payer proof.
+struct TlvMerkleData {
+ tlv_type: u64,
+ nonce_hash: sha256::Hash,
+ per_tlv_hash: sha256::Hash,
+ is_included: bool,
+}
+
pub(super) fn compute_selective_disclosure<'a>(
records: impl Iterator<Item = TlvRecord<'a>> + 'a, included_types: &'a BTreeSet<u64>,
) -> SelectiveDisclosure {
- let (tlv_data, branch_tag) =
- merkle_tlv_data(records, |tlv_type| included_types.contains(&tlv_type));
- let tlv_data: Vec<TlvMerkleData> = tlv_data.collect();
+ let (hash_data, branch_tag) = merkle_tlv_data(records);
+ let tlv_data: Vec<TlvMerkleData> = hash_data
+ .map(|TlvHashData { tlv_type, nonce_hash, per_tlv_hash }| TlvMerkleData {
+ tlv_type,
+ nonce_hash,
+ per_tlv_hash,
+ is_included: included_types.contains(&tlv_type),
+ })
+ .collect(); |
|
Hi @vincenzopalazzo, Thanks for your contributions to After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development. You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4297 If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use |
This is a first draft implementation of the payer proof extension to BOLT 12 as proposed in lightning/bolts#1295. The goal is to get early feedback on the API design before the spec is finalized.
Payer proofs allow proving that a BOLT 12 invoice was paid by demonstrating possession of:
This PR adds the core building blocks:
This is explicitly a PoC to validate the API surface - the spec itself is still being refined. Looking for feedback on:
cc @TheBlueMatt @jkczyz