Skip to content

Conversation

@dexX7
Copy link
Contributor

@dexX7 dexX7 commented Dec 19, 2014

During startup, when adding pending wallet transactions, which spend outputs of
other pending wallet transactions, back to the memory pool, and when they are
added out of order, it appears as if they are orphans with missing inputs.

Those transactions are then rejected and flagged as "conflicting" (= not in the
memory pool, not in the block chain).

To prevent this, transactions are explicitly sorted.

@gmaxwell
Copy link
Contributor

oh erp. Yea, good find.

@gmaxwell
Copy link
Contributor

I don't believe that chrnological order (really nOrderPos order in your patch) always means dependency order, however. Do you have a way to reproduce the behavior you're fixing?

@dexX7
Copy link
Contributor Author

dexX7 commented Dec 20, 2014

I assumed that transactions are not added out of order to the wallet, because it appears as if they have to pass certain checks to be added to the wallet in the first place, and that nOrderPos reflects correct dependency order. If this is questionable, then this approach might not cover all situations.

A simple test to check, if a client adds unconfirmed transactions back to the memory pool in correct order:

  1. Send n transactions
  2. Node should have n transactions in the mempool
  3. Shutdown and restart client
  4. Node should reaccept all transactions and have n transactions in the mempool

As RPC test: https://gist.github.com/dexX7/06df91d1a7f99190d8d6

@laanwj laanwj added the Wallet label Jan 2, 2015
@laanwj
Copy link
Member

laanwj commented Jan 29, 2015

I'd say that transactions generated by the wallet itself, which are the ones useful to insert into the mempool, are always in dependency order by nOrderPos, so this makes sense.

@dexX7 dexX7 force-pushed the reaccept-sorted-wtx branch from 3358e55 to daa3c3d Compare February 11, 2015 03:39
@dexX7
Copy link
Contributor Author

dexX7 commented Feb 11, 2015

@zander: I did not move the lock, but I believe both should work.

I reworded the comment from "chronological order" to "based on initial wallet insertion order". Hope this is more accurate.

During startup, when adding pending wallet transactions, which spend outputs of
other pending wallet transactions, back to the memory pool, and when they are
added out of order, it appears as if they are orphans with missing inputs.

Those transactions are then rejected and flagged as "conflicting" (= not in the
memory pool, not in the block chain).

To prevent this, transactions are explicitly sorted.
@dexX7 dexX7 force-pushed the reaccept-sorted-wtx branch from daa3c3d to e9c3215 Compare March 21, 2015 12:50
@dexX7
Copy link
Contributor Author

dexX7 commented Mar 21, 2015

Rebased (src/wallet.cpp -> src/wallet/wallet.cpp).

Tested with the above linked test a few hundred times, although I can't say in which case nOrderPos might actually not reflect dependency order, and the test only shows it works well under those specific test conditions. It can be used to show the current failure though: sample log output and comparison

@sipa
Copy link
Member

sipa commented Mar 24, 2015

utACK

1 similar comment
@gavinandresen
Copy link
Contributor

utACK

@dexX7 dexX7 mentioned this pull request Apr 17, 2015
@laanwj laanwj merged commit e9c3215 into bitcoin:master Apr 29, 2015
laanwj added a commit that referenced this pull request Apr 29, 2015
e9c3215 [Wallet] sort pending wallet transactions before reaccepting (dexX7)
@willstg
Copy link

willstg commented Jun 13, 2015

As a wallet user who has this issue and a lot of BTC in this status what can be done to get them back?

@dexX7
Copy link
Contributor Author

dexX7 commented Jun 13, 2015

Hey @willstg,

fortunally this is only a local issue, and your BTC are not lost.

Please create a backup of the wallet nevertheless.

Shutdown Bitcoin Core before continuing.

Depending on the OS, the default wallet locations are:

  • On Unix systems: $HOME/.bitcoin/wallet.dat
  • On OS X: $HOME/Library/Application Support/Bitcoin/wallet.dat
  • On Windows: %APPDATA%/Bitcoin/wallet.dat

To clear the conflicted transactions, start bitcoind or bitcoin-qt with the command-line option -zapwallettxes.

Alternatively, zapwallettxes=1 can be added to the bitcoin.conf, but should be removed after the first startup.

zapwallettxes clears the wallet transactions, and then scans the blockchain to find and add only those, which are actually in the chain (the non-conflicted ones).

If you had a bunch of unconfirmed transactions, then it could be that not all, but only some, or maybe even none, are going to be confirmed. This depends on whether other nodes received the transactions (and ultimately the miners).

There is also a chance that other nodes received the unconfirmed transactions, but your local client doesn't know anything about them after zapping, so please wait some time, ideally a few blocks, until you resend the transactions, which didn't go through.

@willstg
Copy link

willstg commented Jun 14, 2015

Thumbs up @dexX7 thanks.

random-zebra added a commit to PIVX-Project/PIVX that referenced this pull request Oct 9, 2019
91b48c7 [Build] Add new merkle files to CMake lists (warrows)
48a3aff [Wallet] Ignore coinbase and zc tx "conflicts" (warrows)
3572354 [Wallet] Fix an error in tx depth computation (warrows)
6928369 [Tests] Enable abandonconflict functional test (warrows)
34cd496 Fix that CWallet::AbandonTransaction would only traverse one level (Ben Woosley)
aba5b75 Fix calculation of balances and available coins. (Alex Morcos)
48d705f Remove stale wallet transactions on initial load. (presstab)
12985ae Flush wallet after abandontransaction (Alex Morcos)
8f87956 [Wallet] sort pending wallet transactions before reaccepting (dexX7)
9c2f445 [Wallet] Call notification signal when a transaction is abandoned (Jonas Schnelli)
778ebf3 Add new rpc call: abandontransaction (Alex Morcos)
0e86c3e Make wallet descendant searching more efficient (Alex Morcos)
d0083a8 Make sure conflicted wallet tx's update balances (Alex Morcos)
6a50e03 [Wallet] Keep track of explicit wallet conflicts instead of using mempool (warrows)
7ccb2b5 [Wallet] Do not flush the wallet in AddToWalletIfInvolvingMe(..) (warrows)
47345be [Refactor] Move wallet functions out of header (warrows)
ab9efb8 [Wallet] Switch to a constant-space Merkle root/branch algorithm (warrows)
5447622 [Wallet] Do not store Merkle branches in the wallet (warrows)

Pull request description:

  This pull request is a happy melting pot of improvements regarding transactions handling. Most of them are backports from bitcoin. I advise reviewers to check the code of the different commits independently to understand them more easily. However, testing is probably better done all at once.
  I am making a single pull request because these changes are all entangled and introducing some of them without others would probably introduce temporary bugs.

  ## Commits details ##
  - 6c3e2ac backport of bitcoin#6550
  - 5304fdf backport of bitcoin#6508
  - c3eeeac simple code move from the header to the cpp file. It contains no functional change.
  - 6cc4d37 backport of bitcoin#4805
  - 10be1db backport of bitcoin#7105
  - 8a34c32 backport of bitcoin#7306
  - 3caf123, 9e17178 and 240f5b4 are the backport for bitcoin#7312
  - ad6d0b1 backport of bitcoin#5511
  - fcc07c3562 backport of bitcoin#9311
  - 5ed5e266794 is an update of #825
  - 392d504 backport of bitcoin#7715
  - 7199f3a backport of bitcoin#13652
  - f09d999 enables and fixes the test from bitcoin#7312
  - 4fd43c5 fixes an oversight in bitcoin#7105 backport

ACKs for top commit:
  random-zebra:
    ACK 91b48c7
  Fuzzbawls:
    ACK 91b48c7

Tree-SHA512: 2628cebe98805b8048b920b51ee26fd4f0c53643d78da9b8cb265aede52dfe1d40c8c19d34293c232c5c35be7f1ab89ff5b4a07073a4b27c371ea70eb8708669
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants