-
Notifications
You must be signed in to change notification settings - Fork 38.9k
Add a "tx output spender" index #24539
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
3f7cdb6 to
aa9f963
Compare
|
Concept ACK. It would be good know more about specific use-cases for this, if you can provide some links or more description. But the implementation seems very simple, so I don't think adding this would be a maintenance burden. |
|
@ ryan, I think the idea is that there is one coin that is owned by more than one entity. There may be several (conflicting) pre-signed txs spending it and as soon as it is spent, all entities with ownership in it want to check which pre-signed tx spent the coin.
|
|
Lightning channels are basically trees of unpublished transactions:
There is also a penalty scheme built into these txs: if you cheat, you need to wait before you can spend your outputs but your channel peer can spend them immediately. And LN is also transitioning to a model where these txs pay minimum fees and can be "bumped" with RBF/CPFP. => it's very useful for LN nodes to detect when transactions have been spent, but also how they've been spent and react accordingly (by extracting info from the spending txs for example), and walk up chains of txs that spend each other. As described above, there are tools in bitcoin core that we could use and would help a bit, and Lightning nodes could also create their own indexes independently or use electrum servers, block explorers, but this would make running LN nodes connected to your own bitcoin node much easier. |
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/24539. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsNo conflicts as of last run. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK - I can see why this could be helpful for lightning, the code is very similar to the txindex code.
Some general thoughts:
getmempooltxspendingprevoutin #24408 takes a list of outpoints,gettxospendertakes a single outpoint. Would it make sense to have the same format here (I'd guess a common use case would be to check both RPCs with the same outpoint(s)?)- Taking this one step further,
getrawtransactionqueries transaction data from the mempool or from blocks (if txindex is enabled) in one RPC. Would it make sense to have something similar for outpoints, instead of two separate RPCs? - I think that the indexed data will not be deleted in case of a reorg (but possibly overwritten if an outpoint would be spent in a different tx as a result of a reorg). That means that a result from
gettxospendercould be stale. Would it make sense to delete entries or should we at least mention this possibility in the rpc doc? - What's the reason for the dependency on -txindex? As far as I can see it's not technical, so would knowing the fact that an outpoint was used in a tx (even if we can't lookup the details of that tx further) be sufficient for some use cases?
aa9f963 to
1b82b7e
Compare
Bitcoin supports batched RPC request so from a functional p.o.v it's equivalent to passing a list of outpoints but it's a bit easier to reason about imho. For
Maybe, let's see where the discussion in #24408 but to me it feels cleaner to have separate calls.
We would almost always want to get the actual tx that spends our outpoint. My reasoning was that relying on |
1b82b7e to
3e97658
Compare
3e97658 to
e021b6b
Compare
|
Concept ACK, nice |
e021b6b to
a6cb2c6
Compare
|
Rebased on #24408 |
|
@sstone Question -- In LDK, when we confirm transactions, we require the transaction's index-in-block to be provided, for short channel ID construction. I'm curious how this API works for you without this information? |
This index would be used to find if and how channels have been spent (after a restart for example) and would be much more efficient that walking down the blockchain from the tip. To compute the short channel Id we simply call |
|
Concept ACK. I like the idea of us providing (useful) optional indexes. I haven't done an in-depth code review yet, but at a high level I feel that this feature would be worthy of a release note? I have reindexed with this enabled and see that the index currently takes about 75GB of space, which could be worth mentioning in such a note so users know what they need before enabling. |
So you need txindex and make 3 RPC calls? ISTM storing the position in this new index would make it much more efficient and more useful to other people. |
I could easily return the block hash along with the spending tx, txindex would not be needed anymore but users would still need to call |
|
🚧 At least one of the CI tasks failed. HintsTry to run the tests locally, according to the documentation. However, a CI failure may still
Leave a comment here, if you need help tracking down a confusing failure. |
bb0cb11 to
c7ba5de
Compare
sedited
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Block hash addition looks good to me. Just needs a small iwyu fix to get the CI to pass again.
src/index/txospenderindex.h
Outdated
| private: | ||
| std::unique_ptr<BaseIndex::DB> m_db; | ||
| std::pair<uint64_t, uint64_t> m_siphash_key; | ||
| bool AllowPrune() const override { return true; } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As the index stores the tx position in disk, how is this correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, fixed in 38f2f53
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool. It would be good to add a test case for it.
acf4fba to
08167df
Compare
Adds an outpoint -> txid index, which can be used to find which transactions spent a given output. We use a composite key with 2 parts (suggested by @romanz): hash(spent outpoint) and tx position, with an empty value. To find the spending tx for a given outpoint, we do a prefix search (prefix being the hash of the provided outpoint), and for all keys that match this prefix we load the tx at the position specified in the key and return it, along with the block hash, if does spend the provided outpoint. To handle reorgs we just erase the keys computed from the removed block. This index is extremely useful for Lightning and more generally for layer-2 protocols that rely on chains of unpublished transactions. If enabled, this index will be used by `gettxspendingprevout` when it does not find a spending transaction in the mempool.
08167df to
38f2f53
Compare
sedited
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re-ACK 38f2f53
This PR adds a new "tx output spender" index, which allows users to query which tx spent a given outpoint with the
gettxspendingprevoutRPC call that was added by #24408.Such an index would be extremely useful for Lightning, and probably for most layer-2 protocols that rely on chains of unpublished transactions.
UPDATE: this PR is ready for review and issues have been addressed:
-txindexanymoreWe use a composite key with 2 parts (suggested by @romanz): hash(spent outpoint) and tx position, with an empty value. Average composite key size is 15 bytes.
The spending tx can optionally be returned by
gettxspendingprevout(even it-txindex is not set).