RPC/txoutproof: Support including (and verifying) proofs of wtxid #32844

luke-jr · 2025-06-30T23:05:05Z

Feature requested by @SomberNight for Electrum's Lightning stuff. (Please confirm this does what you need)

Unfortunately WIP because the wtxid merkle root calculation doesn't match and I don't have time to figure out why not yet (maybe after concept ACK? or if anyone else wants to take a look...)

Mostly backward compatible, but a segwit-enabled proof will verify the generation tx in old versions.

DrahtBot · 2025-06-30T23:05:09Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32844.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	Sjors, ajtowns, fjahr

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#33116 (refactor: Convert uint256 to Txid by marcofleon)
#33094 (refactor: unify container presence checks by l0rinc)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

LLM Linter (✨ experimental)

Possible typos and grammar issues:

In RPCResult description: "If verify_witness is true and the proof invalid" -> "If verify_witness is true and the proof is invalid" [missing “is” makes the sentence ungrammatical]

No other typos were found.

^{drahtbot_id_4_m}

src/rpc/txoutproof.cpp

SomberNight · 2025-07-01T00:25:34Z

Feature requested by @SomberNight for Electrum's Lightning stuff. (Please confirm this does what you need)

Wow, thank you for working on this! I was meaning to open an issue to ask for this, but did not even get to that yet.

The request I would like an electrum server, assuming txindex=1, to be able to serve is:

bc.tx.get_merkle_witness(txid), (so given just a txid,)

return a dict containing:

wtxid
block_height
block_hash
pos (index of tx in block)
merkle or wmerkle
- exactly one of the two
- merkle is the Satoshi-SPV proof, in case there is no witness commitment in the block
- wmerkle is the witness-SPV proof otherwise
cb_tx, the generation/coinbase tx
cb_proof, the Satoshi-SPV proof for the generation tx

Obviously the electrum server can act as middleware and transform the response, assuming the necessary fields are available or obtainable.

As far as I can tell, but I haven't tried yet, the RPC response from this PR can be transformed to what I describe above. So the RPC looks to be doing what I need. :)

I think it is non-trivial, so let me detail how an electrum/light client would then use bc.tx.get_merkle_witness(txid):

it is assumed client already has a full raw tx, and wants to verify whether it has been mined
client also has full header chain, proof-of-work verified
client calls bc.tx.get_merkle_witness(txid)
client calculates wtxid of full tx it has, compares it with received wtxid
using cb_tx, cb_proof, block_height, block_hash
- client Satoshi-SPVs the received coinbase tx, against block header merkle_root
client extracts witness_commitment from cb_tx
1. if there is no witness_commitment,
  - client Satoshi-SPVs the txid, using merkle, block_height, block_hash, pos, against block header merkle_root
  - now client has an SPV guarantee that block only contains non-segwit txs, and tx in particular is also non-segwit (wtxid==txid) and it was mined in this block
2. if there is a witness_commitment,
  - client must do witness-SPV, even if tx seems non-segwit (as e.g. malicious electrum server could have stripped away the witness)
  - client does witness-SPV using wmerkle and pos, against witness_commitment
  - client must check that length of wmerkle matches length of cb_proof. This ensures there are the same number of levels in the wtxid merkle tree and in the txid merkle tree. (the two trees should have the same number of leaves, but this seems hard to check as a light client) This is intended as some protection against merkle-tree-construction weaknesses (cf. banning of 64 byte transactions and similar).
now client has an SPV guarantee that wtxid was mined in this block
- or if some check failed, then the client can connect to another server

src/rpc/txoutproof.cpp

Sjors · 2025-07-01T17:37:54Z

Concept ACK. @SomberNight it's still useful if you open an issue to track the reason you need this, just in case this implementation PR doesn't make.

Sjors

Took an initial look at the code, but got a bit confused. Will check again later.

src/consensus/validation.h

src/merkleblock.cpp

src/merkleblock.h

…saction (not a CBlock)

luke-jr · 2025-07-09T03:53:53Z

Reworked this to cover @SomberNight 's requirements and address feedback. Test is now passing.

DrahtBot · 2025-08-13T20:34:09Z

🐙 This pull request conflicts with the target branch and needs rebase.

ajtowns

Transaction proofs are an interoperability feature, I don't think it makes much sense to implement this without also documenting it and providing test vectors via a BIP.

Concept ACK

ajtowns · 2025-10-30T16:57:56Z

src/merkleblock.h

    /** Public only for unit testing */
    CBlockHeader header;
    CPartialMerkleTree txn;
+    /** Only used by subclass WtxidInBlockMerkleProof */


There is no such subclass?

I think all this logic should entirely be in a separate class though. For efficiency probably two paths:

if the generation tx has no witness commitment; provde all (w)txids and the generation tx via the block partial merkle tree.

if the generation tx does have a witness commitment; provide just the generation tx's merkle path back to the block header, and prove all wtxids via a partial merkle tree back to the witness commitment

ajtowns · 2025-10-30T17:02:47Z

src/rpc/txoutproof.cpp

            {"proof", RPCArg::Type::STR_HEX, RPCArg::Optional::NO, "The hex-encoded proof generated by gettxoutproof"},
+            {"options", RPCArg::Type::OBJ_NAMED_PARAMS, RPCArg::Optional::OMITTED, "",
+                {
+                    {"verify_witness", RPCArg::Type::BOOL, RPCArg::Default{false}, "If true, also verifies the associated wtxid/hash of the specified transactions (if included in proof)"},


I think this should probably be automatic, with the current "array of txids" result being stuck behind a -deprecatedrpc option.

I think that makes sense and we could even do the same for gettxoutproof I think (if that wasn't already intended with the comment as well).

ajtowns · 2025-10-30T17:06:21Z

src/rpc/txoutproof.cpp

+                    {RPCResult::Type::STR_HEX, "blockhash", "The block hash this proof links to"},
+                    {RPCResult::Type::NUM, "blockheight", "The height of the block this proof links to"},
+                    {RPCResult::Type::NUM, "confirmations", /*optional=*/true, "Number of blocks (including the one with the transactions) confirming these transactions"},
+                    {RPCResult::Type::NUM, "confirmations_assumed", /*optional=*/true, "The number of unverified blocks confirming these transactions (eg, in an assumed-valid UTXO set)"},


This information (height, confirmations, assumeutxo info) seems better obtained via getblockheader on the blockhash to me.

ajtowns · 2025-10-30T17:09:13Z

src/rpc/txoutproof.cpp

+                if (merkleBlock.m_gentx->GetHash() != vMatch[0]) return res;
+                if (!merkleBlock.m_gentx->IsCoinBase()) return res;
+                witness_commit_outidx = GetWitnessCommitmentIndex(*merkleBlock.m_gentx);
+                if (witness_commit_outidx == NO_WITNESS_COMMITMENT) {


I would have expected this logic to be in merkleblock.cpp as a standard part of verifying a proof (eg so that it could be included in kernel), not in the RPC code.

ajtowns · 2025-10-30T17:10:09Z

src/rpc/txoutproof.cpp

+
+                const CBlockIndex* pindex = chainman.m_blockman.LookupBlockIndex(merkleBlock.header.GetHash());
+                const auto& active_chain = chainman.ActiveChain();
+                if (!pindex || !active_chain.Contains(pindex) || pindex->nTx == 0) {


Being able to easily prove/verify that a tx was included in a reorged block seems fine?

fjahr

Concept ACK

Is there something special about the naming "generation tx" here? It seems to be just an older alias for the coinbase transaction without any other special meaning. I can not recall seeing it used anywhere in the code base or online discourse. If I am not overlooking something it would probably be better to just call it coinbase tx instead which should be less confusing to newer contributors that haven't heard this term before.

fjahr · 2025-10-30T17:57:34Z

test/functional/rpc_txoutproof.py

+        expected_proven['confirmations'] = 1
+        del expected_proven['tx'][0]['txid']
+        del expected_proven['tx'][1]['txid']
+        assert_equal(self.nodes[0].verifytxoutproof(proofres['proof'], verify_witness=True), expected_proven)


Should also cover an invalid proof here

fjahr · 2025-10-30T18:09:52Z

src/rpc/txoutproof.cpp

            {"proof", RPCArg::Type::STR_HEX, RPCArg::Optional::NO, "The hex-encoded proof generated by gettxoutproof"},
+            {"options", RPCArg::Type::OBJ_NAMED_PARAMS, RPCArg::Optional::OMITTED, "",
+                {
+                    {"verify_witness", RPCArg::Type::BOOL, RPCArg::Default{false}, "If true, also verifies the associated wtxid/hash of the specified transactions (if included in proof)"},


I think that makes sense and we could even do the same for gettxoutproof I think (if that wasn't already intended with the comment as well).

DrahtBot added the CI failed label Jul 1, 2025

SomberNight reviewed Jul 1, 2025

View reviewed changes

src/rpc/txoutproof.cpp Outdated Show resolved Hide resolved

SomberNight reviewed Jul 1, 2025

View reviewed changes

src/rpc/txoutproof.cpp Outdated Show resolved Hide resolved

DrahtBot mentioned this pull request Jul 1, 2025

refactor: Txid type safety (parent PR) #32189

Closed

Sjors reviewed Jul 1, 2025

View reviewed changes

Docfix/RPC: txid is no longer the transaction hash

3004f26

luke-jr force-pushed the rpc_gettxoutproof_segwit branch from 3abfd6d to e826ffd Compare July 9, 2025 03:49

luke-jr marked this pull request as ready for review July 9, 2025 03:49

Allow calling GetWitnessCommitmentIndex with just the generation tran…

528a8c9

…saction (not a CBlock)

luke-jr force-pushed the rpc_gettxoutproof_segwit branch 2 times, most recently from e95981b to 4b3a33a Compare July 9, 2025 03:52

RPC/txoutproof: Support new proof format covering witness data

23edd3d

luke-jr force-pushed the rpc_gettxoutproof_segwit branch from 4b3a33a to 23edd3d Compare July 9, 2025 04:01

luke-jr requested review from Sjors and SomberNight July 9, 2025 04:03

DrahtBot removed the CI failed label Jul 9, 2025

This was referenced Jul 30, 2025

refactor: unify container presence checks #33094

Closed

refactor: Convert uint256 to Txid #33116

Merged

DrahtBot added the Needs rebase label Aug 13, 2025

achow101 requested review from ajtowns and fjahr and removed request for SomberNight October 22, 2025 15:29

ajtowns suggested changes Oct 30, 2025

View reviewed changes

fjahr reviewed Oct 30, 2025

View reviewed changes

RPC/txoutproof: Support including (and verifying) proofs of wtxid #32844

Are you sure you want to change the base?

RPC/txoutproof: Support including (and verifying) proofs of wtxid #32844

Conversation

luke-jr commented Jun 30, 2025

Uh oh!

DrahtBot commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage & Benchmarks

Reviews

Conflicts

LLM Linter (✨ experimental)

Uh oh!

Uh oh!

SomberNight commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Sjors commented Jul 1, 2025

Uh oh!

Sjors left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

luke-jr commented Jul 9, 2025

Uh oh!

DrahtBot commented Aug 13, 2025

Uh oh!

ajtowns left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fjahr left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

DrahtBot commented Jun 30, 2025 •

edited

Loading

SomberNight commented Jul 1, 2025 •

edited

Loading

ajtowns left a comment •

edited

Loading