-
Notifications
You must be signed in to change notification settings - Fork 38.8k
fix assumevalid is ignored during reindex #33854
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33854. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please copy-paste ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. LLM Linter (✨ experimental)Possible typos and grammar issues:
Possible places where named args for integral literals may be used (e.g.
2026-01-09 |
|
🚧 At least one of the CI tasks failed. HintsTry to run the tests locally, according to the documentation. However, a CI failure may still
Leave a comment here, if you need help tracking down a confusing failure. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have you tried this out on signet or mainnet?
I haven't yet, but just from looking at the code I would suspect that we now
- do the -reindex part without connecting any blocks (except genesis here)
- omplete headers-sync with a peer until minchainwork
- Download the first block we don't have on disk from a peer and only then call
ActivateBestChain() - try to connect that block, which will result in the entire chain up to that block being connected by the
msghandthread. This could block the thread hours, in which no other peer would be processed, so existing peers will get disconnected (ping timeout), while net thread will still continually make new connections (that never complete the version handshake because we are too busy connecting blocks).
Also, attempting to reindex while offline wouldn't work anymore if we are below minchainwork.
|
Thanks for trying this out @Eunovo!
Would it be acceptable to emit an error and halt, saying that Not the cleanest perhaps. But we have 2 main usages of
|
I don't know anyone who does
I haven't checked the code yet (am planning on doing that later), but would it be possible to just issue a warning if we're offline and attempt the reindex anyway (especially now that we have proper script verification logging to show if script verification was enabled)? |
8128a5e to
86c3dc7
Compare
I don't think it is particularly important that reindex be able to complete offline. As @hodlinator pointed out:
In both use cases, network access is not an issue.
It is possible to do this, but will this be considered a better solution than automatically doing headers sync?
IBD does not work offline anyway. I'm not sure this will be better to implement, unless there is a use-case that requires |
da3cad6 to
bb65636
Compare
b91f630 to
b59ca76
Compare
I've pushed a solution to this. In the current implementation, the background init thread waits for enough headers to be indexed. When |
This suggestion was meant for the case where we can't do headers sync due to being offline (using zero peers after a certain timeout or something like that, I'm not sure how to best check whether we are offline). I guess the error message could also point out that going online should resolve the situation. |
I see. A message that indicates that the node is waiting for headers sync to finish reindex will be valuable. We probably don't need to check that we are offline at all. |
|
I have checked if it fixes the problem I have noticed a year ago and now that #33336 landed it's even simpler to do a reproducer: for COMMIT in 509dc91db143fe2ebb4e910680aca97ba62233b9 b59ca7602bf9efaad80b3465129685ee1d09a028; do
(echo ""; git fetch -q origin $COMMIT >/dev/null 2>&1 && git checkout -q $COMMIT && git log -1 --pretty='%h %s' || exit 1) && \
cmake -B build >/dev/null 2>&1 && cmake --build build -j$(nproc) >/dev/null 2>&1 && \
killall -9 bitcoind >/dev/null 2>&1; rm -rf demo >/dev/null 2>&1; mkdir -p demo >/dev/null 2>&1 && \
build/bin/bitcoind -datadir=demo -stopatheight=10 >/dev/null 2>&1 && \
build/bin/bitcoind -datadir=demo -stopatheight=10 -reindex >/dev/null 2>&1 && \
build/bin/bitcoind -datadir=demo -stopatheight=10 -reindex-chainstate | grep "script verification"
doneWhich correctly prints: 509dc91db1 Merge bitcoin/bitcoin#33026: test, refactor: Embedded ASMap [1/3]: Selected minor preparatory work
2025-11-24T12:54:28Z Enabling script verification at block #1 (00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048): assumevalid hash not in headers.
b59ca7602b test/reindex: only connect minchainwork chain
2025-11-24T12:59:32Z Disabling script verification at block #1 (00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048).nit: maybe this style of checking via the logs could simplify the testing of the PR Concept ACK |
b59ca76 to
729bf4d
Compare
|
Thanks for the reviews. @l0rinc I added your test as an automated test in 729bf4d |
|
I think minimumchainwork being configurable may violate the underlying security assumption that made me feel comfortable writing assumevalid in the first place. The concern is that attackers (or developers operating under coercion) could target victims with false statements that due to some "network incident" they must start nodes with -assumevalid= (and use DOS attacks to prevent syncing an honest chain). AV security assumption there is that the bad chain needs to have two weeks of work at a contemporary difficulty on top of it. If an attacker is able to do that then there are other at-least-as-serious problems. The idea is that the attacker must either control enough hashpower to cause serious reorgs OR convince the users to actually patch/replace the software and not just reconfigure it. (And of course, if they can convince the user of that-- it's game over as they can do anything). This security assumption is documented in the source code: So I think advising anywhere to change that value is not a good idea and that it was a mistake to make this value configurable on mainnet without clamping it to the hardcoded value... though #10357 doesn't really explain the motivation outside of test harnesses enough for me to say for certain. As it is, however, it does break my security assumptions in a way that concerns me. |
729bf4d to
6eab0e8
Compare
6eab0e8 to
984ccb5
Compare
|
🚧 At least one of the CI tasks failed. HintsTry to run the tests locally, according to the documentation. However, a CI failure may still
Leave a comment here, if you need help tracking down a confusing failure. |
984ccb5 to
d6420a8
Compare
This commit sets the stage for future commits that will change the behavior of ImportBlocks to conditionally ActivateBestChain after reindex
This change will allow headers sync to discover a long enough chain before connecting blocks. `force_activation` is required to preserve `ImportBlocks` behavior in libbitcoinkernel.
The background init thread will wait on this mutex for header tip updates to determin whether to ActivateBestChain in future commits..
This commit removes: - LoadingBlocks() guard that prevent HEADERS sync during reindex, and - adds a LoadingBlocks() guard that prevents requesting blocks during reindex Blocks must not be requested while LoadingBlocks() is `true` because they will be rejected when received and this might lead to the peer being disconnected because the requested block will never be removed from vBlocksInFlight. Note that we cannot connect Blocks while LoadingBlocks() is `true` because this can block the p2p process message thread for a long time if a long chain is being connected in the background.
Future commits will change the LoadingBlocks stage to only end after best_header has at least minchainwork. Mining a block at the tip will no longer be enough to trigger the "unrequested block" scenario.
d6420a8 to
4ced1ae
Compare
During a reindex, the assumevalid setting may be ignored if the chainwork of the best header is below
minimumchainwork. This happens when the previous IBD was stopped before connecting enough blocks to reach the requiredminimumchainwork. The assumevalid block will also be missing from the block index because the assumevalid block is set to match theminimumchainwork. See Issue #31494.As a result,
reindexcan take significantly longer to complete than necessary.This PR addresses the issue by modifying reindex behaviour: if the chainwork of the
best_headeris belowminimumchainwork, the node skips theActivateBestChain()step and proceeds directly to synchronisation.This PR takes a different approach from PR #31615 to keep the validation path for
reindexthe same as the regular IBD validation path.Developers benchmarking IBD with
-reindexmay find this behaviour undesirable. In such cases, it’s recommended to use-reindex-chainstateinstead or ensure that the available blocks have sufficient chainwork.libbitcoinkernelexposes theImportBlocksfunction and expectsreindexto connect blocks, so an attempt was made to preserve the existing behaviour forlibbitcoinkernel.