Skip to content

Conversation

@UdjinM6
Copy link
Contributor

@UdjinM6 UdjinM6 commented Feb 28, 2024

I think the expected behaviour of getrawchangeaddress and getnewaddress RPCs is that their failure should not affect keypool in any way. At least that's how legacy wallets work, you can confirm this behaviour by running wallet_keypool.py --legacy-wallet on master with e073f1d applied on top. However running wallet_keypool.py --descriptors on the same commit results in the following failure:

  File "/path/to/bitcoin/test/functional/test_framework/test_framework.py", line 131, in main
    self.run_test()
  File "/path/to/bitcoin/test/functional/wallet_keypool.py", line 114, in run_test
    assert_equal(kp_size_before, kp_size_after)
  File "/path/to/bitcoin/test/functional/test_framework/util.py", line 57, in assert_equal
    raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not([18, 24] == [19, 24])

This happens because we pass nIndex (which is a class member) into GetReservedDestination and since it's passed by reference we get an updated value back, so nIndex won't be equal -1 anymore, no matter if the function failed or succeeded. This means that ReturnDestination (called by dtor of ReserveDestination) will try to return something we did not actually reserve.

The fix is to simply use a temporary variable instead of a class member and only update nIndex when op_address actually has value, basically do it the same way we do for other class members (address and fInternal) already.

@DrahtBot
Copy link
Contributor

DrahtBot commented Feb 28, 2024

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK achow101, josibake

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

@achow101 achow101 added this to the 27.0 milestone Feb 28, 2024
@achow101
Copy link
Member

ACK e073f1d

Nice catch! A consequence of this bug is that the user can get the same change address twice, and that is pretty bad. I think we're going to want this for 27.0, and probably backport to 26.1 and 25.2 too.

@josibake
Copy link
Member

ACK e073f1d

Code reviewed, and also did a bit of poking around to check if this happens any other place nIndex is used. Didn't see anything obvious, and this solves the specific problem with getnewaddress/getrawchangeaddress. Thanks for adding the test!

@DrahtBot DrahtBot removed the request for review from josibake February 29, 2024 16:21
@achow101 achow101 merged commit 22a5ccf into bitcoin:master Feb 29, 2024
achow101 pushed a commit to achow101/bitcoin that referenced this pull request Mar 1, 2024
achow101 pushed a commit to achow101/bitcoin that referenced this pull request Mar 1, 2024
@achow101 achow101 mentioned this pull request Mar 1, 2024
@fanquake
Copy link
Member

fanquake commented Mar 4, 2024

Backported for 25.x in #29531.

luke-jr pushed a commit to luke-jr/bitcoin that referenced this pull request Mar 4, 2024
luke-jr pushed a commit to luke-jr/bitcoin that referenced this pull request Mar 4, 2024
glozow pushed a commit to glozow/bitcoin that referenced this pull request Mar 5, 2024
glozow pushed a commit to glozow/bitcoin that referenced this pull request Mar 5, 2024
@glozow
Copy link
Member

glozow commented Mar 5, 2024

Backported for 26.x in #29509

knst pushed a commit to knst/dash that referenced this pull request Mar 6, 2024
…` failures should not affect keypools for descriptor wallets

e073f1d test: make sure keypool sizes do not change on `getrawchangeaddress`/`getnewaddress` failures (UdjinM6)
367bb7a wallet: Avoid updating `ReserveDestination::nIndex` when `GetReservedDestination` fails (UdjinM6)

Pull request description:

  I think the expected behaviour of `getrawchangeaddress` and `getnewaddress` RPCs is that their failure should not affect keypool in any way. At least that's how legacy wallets work, you can confirm this behaviour by running `wallet_keypool.py --legacy-wallet` on master with e073f1d applied on top. However running `wallet_keypool.py --descriptors` on the same commit results in the following failure:
  ```
    File "/path/to/bitcoin/test/functional/test_framework/test_framework.py", line 131, in main
      self.run_test()
    File "/path/to/bitcoin/test/functional/wallet_keypool.py", line 114, in run_test
      assert_equal(kp_size_before, kp_size_after)
    File "/path/to/bitcoin/test/functional/test_framework/util.py", line 57, in assert_equal
      raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
  AssertionError: not([18, 24] == [19, 24])
  ```

  This happens because we pass `nIndex` (which is a class member) into `GetReservedDestination` and since it's passed by reference we get an updated value back, so `nIndex` won't be equal `-1` anymore, no matter if the function failed or succeeded. This means that `ReturnDestination` (called by dtor of `ReserveDestination`) will try to return something we did not actually reserve.

  The fix is to simply use a temporary variable instead of a class member and only update `nIndex` when `op_address` actually has value, basically do it the same way we do for other class members (`address` and `fInternal`) already.

ACKs for top commit:
  achow101:
    ACK e073f1d
  josibake:
    ACK bitcoin@e073f1d

Tree-SHA512: 1128288a60dd4d8f306ef6f7ac66cdfeae3c9cc35c66ecada2d78fa61ac759f2a757b70fc3976ba8b5081200942b58dfabc184c01ccf911af40ba8c145344651
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Mar 6, 2024
…` failures should not affect keypools for descriptor wallets

e073f1d test: make sure keypool sizes do not change on `getrawchangeaddress`/`getnewaddress` failures (UdjinM6)
367bb7a wallet: Avoid updating `ReserveDestination::nIndex` when `GetReservedDestination` fails (UdjinM6)

Pull request description:

  I think the expected behaviour of `getrawchangeaddress` and `getnewaddress` RPCs is that their failure should not affect keypool in any way. At least that's how legacy wallets work, you can confirm this behaviour by running `wallet_keypool.py --legacy-wallet` on master with e073f1d applied on top. However running `wallet_keypool.py --descriptors` on the same commit results in the following failure:
  ```
    File "/path/to/bitcoin/test/functional/test_framework/test_framework.py", line 131, in main
      self.run_test()
    File "/path/to/bitcoin/test/functional/wallet_keypool.py", line 114, in run_test
      assert_equal(kp_size_before, kp_size_after)
    File "/path/to/bitcoin/test/functional/test_framework/util.py", line 57, in assert_equal
      raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
  AssertionError: not([18, 24] == [19, 24])
  ```

  This happens because we pass `nIndex` (which is a class member) into `GetReservedDestination` and since it's passed by reference we get an updated value back, so `nIndex` won't be equal `-1` anymore, no matter if the function failed or succeeded. This means that `ReturnDestination` (called by dtor of `ReserveDestination`) will try to return something we did not actually reserve.

  The fix is to simply use a temporary variable instead of a class member and only update `nIndex` when `op_address` actually has value, basically do it the same way we do for other class members (`address` and `fInternal`) already.

ACKs for top commit:
  achow101:
    ACK e073f1d
  josibake:
    ACK bitcoin@e073f1d

Tree-SHA512: 1128288a60dd4d8f306ef6f7ac66cdfeae3c9cc35c66ecada2d78fa61ac759f2a757b70fc3976ba8b5081200942b58dfabc184c01ccf911af40ba8c145344651
PastaPastaPasta added a commit to dashpay/dash that referenced this pull request Mar 6, 2024
…itcoin#18787, bitcoin#18805, bitcoin#18888, bitcoin#19502, bitcoin#19077, bitcoin#20125, bitcoin#20153, bitcoin#20198, bitcoin#20262, bitcoin#20266, bitcoin#23608, bitcoin#29510  - native descriptor wallets

6b71f27 Merge bitcoin#29510: wallet: `getrawchangeaddress` and `getnewaddress` failures should not affect keypools for descriptor wallets (Ava Chow)
85fa370 refactor: use Params().ExtCoinType() for descriptor wallets (Konstantin Akimov)
da8e563 fix: skip functional tests which requires BDB if no bdb (see 20267) (Konstantin Akimov)
4ba44fa fix: skip interface_zmq.py which is not ready to work without bdb (Konstantin Akimov)
45fc8a4 fix: autobackup influences an exclusive locks made by SQLite (Konstantin Akimov)
e542cd2 fix: missing changes from bitcoin#21634 (Konstantin Akimov)
2de7aec Merge bitcoin#19502: Bugfix: Wallet: Soft-fail exceptions within ListWalletDir file checks (Samuel Dobson)
c172605 Merge bitcoin#19077: wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets (Samuel Dobson)
2439247 Merge bitcoin#23608: test: fix `feature_rbf.py --descriptors` and add to test runner (fanquake)
f6b3614 fix: descriptor wallets follow-up to merge bitcoin#20202: Make BDB support optional (Konstantin Akimov)
a340ad6 Merge bitcoin#20262: tests: Skip --descriptor tests if sqlite is not compiled (Samuel Dobson)
7d55046 Merge bitcoin#20125: rpc, wallet: Expose database format in getwalletinfo (Samuel Dobson)
343d4b0 fix: descriptor wallets follow-up for bitcoin#20156: Make sqlite support optional (compile-time) (Konstantin Akimov)
fa30777 Merge bitcoin#20198: Show name, format and if uses descriptors in bitcoin-wallet tool (MarcoFalke)
14121ec Merge bitcoin#18888: test: Remove RPCOverloadWrapper boilerplate (MarcoFalke)
b18351e Merge bitcoin#20153: wallet: do not import a descriptor with hardened derivations into a watch-only wallet (Wladimir J. van der Laan)
c995e5d Merge bitcoin#20266: wallet: fix change detection of imported internal descriptors (Wladimir J. van der Laan)
c864582 Merge bitcoin#18787: wallet: descriptor wallet release notes and cleanups (Samuel Dobson)
0949c08 Merge bitcoin#18782: wallet: Make sure no DescriptorScriptPubKeyMan or WalletDescriptor members are left uninitialized after construction (Samuel Dobson)
baa6959 Merge bitcoin#18805: tests: Add missing sync_all to wallet_importdescriptors.py (MarcoFalke)
76e08f9 Merge bitcoin#18027: "PSBT Operations" dialog (Samuel Dobson)
c1b94b6 fix: wallet should be unlocked before generating keys for Descriptor wallet (Konstantin Akimov)
f293c04 Merge bitcoin#16528: Native Descriptor Wallets using DescriptorScriptPubKeyMan (Andrew Chow)
4064334 fix: get receiving address for Descriptor Wallets (Konstantin Akimov)
bdbd0b1 chore: dashification of descriptor implementation in dash (UdjinM6)
b02fc0b fix: counting calculation of internal keys for Descriptor Wallets (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented
  This PR is a batch of backports and related fixes to add a support of native descriptor wallets to Dash Core.

  There're more related backports, but this PR is a minimal package of backports to get descriptor wallets working and unit/functional tests to succeed. To do: bitcoin#20226, bitcoin#21049, bitcoin#18788, bitcoin#20267, bitcoin#19230, bitcoin#19239, bitcoin#19441, bitcoin#19568, bitcoin#19979, bitcoin-core/gui#96, bitcoin#19136, bitcoin#21277, bitcoin#21063, bitcoin#21302, bitcoin#19651, bitcoin#20191, bitcoin#22446 and other.

  Prior work:
   - #5580
   - #5807

  ## What was done?
  backports:
   - bitcoin#16528
   - bitcoin#18027
   - bitcoin#18805
   - bitcoin#18782
   - bitcoin#18787
   - bitcoin#20266
   - bitcoin#20153
   - bitcoin#18888
   - bitcoin#20198
   - bitcoin#20125
   - bitcoin#20262
   - bitcoin#23608
   - bitcoin#19077
   - bitcoin#19502
   - bitcoin#29510

  and extra fixes and missing changes for bitcoin#20156, bitcoin#20202, bitcoin#20267, bitcoin#21634 + fix of auto-backup for sqlite wallets.

  ## How Has This Been Tested?
  There're 2 new functional tests:  `wallet_importdescriptors.py` and `wallet_descriptor.py`
  Beside that many functional tests run twice now: using legacy wallet and descriptor wallets: `wallet_hd.py`, `wallet_basic.py`, `wallet_labels.py`, `wallet_keypool_topup.py`, `wallet_avoidreuse.py`, `rpc_psbt.py`, `wallet_keypool_hd.py`, `rpc_createmultisig.py`, `wallet_encryption.py`.
  With bitcoin#18788 expected to more tests run.

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    Rebase looks good; utACK 6b71f27
  PastaPastaPasta:
    utACK 6b71f27
  UdjinM6:
    utACK 6b71f27
  kwvg:
    utACK 6b71f27

Tree-SHA512: 776c5dfe1eec2b5bebc8d606476cd981c810ac81965b348e78c13e96fff23be500c495ae68c93f669403941c96eccdd3775f2b96572163c34175900e15549b5d
glozow added a commit that referenced this pull request Mar 11, 2024
c68d4d0 [doc] update manual pages for v26.1rc2 (glozow)
bd715bf [build] bump version to v26.1rc2 (glozow)
b6d006d update release notes 26.1 (glozow)
fce992b fuzz: restrict fopencookie usage to Linux & FreeBSD (fanquake)
40c56a4 test: make sure keypool sizes do not change on `getrawchangeaddress`/`getnewaddress` failures (UdjinM6)
7c82b27 wallet: Avoid updating `ReserveDestination::nIndex` when `GetReservedDestination` fails (UdjinM6)
b5419ce p2p: Don't consider blocks mutated if they don't connect to known prev block (Greg Sanders)
0535c25 [test] IsBlockMutated unit tests (dergoegge)
8141498 [validation] Cache merkle root and witness commitment checks (dergoegge)
0c5c596 [test] Add regression test for #27608 (dergoegge)
2473635 [net processing] Don't process mutated blocks (dergoegge)
50c0b61 [validation] Merkle root malleation should be caught by IsBlockMutated (dergoegge)
aff368f [validation] Introduce IsBlockMutated (dergoegge)
076c67c [refactor] Cleanup merkle root checks (dergoegge)
97a1d0a [validation] Isolate merkle root checks (dergoegge)
4ac0eb5 test: Drop `x` modifier in `fsbridge::fopen` call for mingw builds (Hennadii Stepanov)

Pull request description:

  Includes:
  - #29357
  - #29412
  - #29524
  - #29510
  - #29529

  Also does:
  - update to release notes
  - bump to rc2
  - manpages
  - (no changes to bitcoin.conf)

ACKs for top commit:
  achow101:
    ACK c68d4d0

Tree-SHA512: 2f8c3dd705e3f9f33403b3cc17e8006510ff827d7dbd609b09732a1669964e9b001cfecdc63d8d8daeb8f39c652e1e4ad0aac873d44d259c21803de85688ed2b
fanquake added a commit that referenced this pull request Mar 22, 2024
27cfda1 doc: Update release notes for 25.2rc2 (Ava Chow)
daba5e2 doc: Update manpages for 25.2rc2 (Ava Chow)
8a0c980 build: Bump to 25.2rc2 (Ava Chow)
cf7d3a8 p2p: Don't consider blocks mutated if they don't connect to known prev block (Greg Sanders)
3eaaafa [test] IsBlockMutated unit tests (dergoegge)
0667441 [validation] Cache merkle root and witness commitment checks (dergoegge)
de97ecf [test] Add regression test for #27608 (dergoegge)
8cc4b24 [net processing] Don't process mutated blocks (dergoegge)
098f07d [validation] Merkle root malleation should be caught by IsBlockMutated (dergoegge)
8804c36 [validation] Introduce IsBlockMutated (dergoegge)
4f5baac [validation] Isolate merkle root checks (dergoegge)
f93be01 test: make sure keypool sizes do not change on `getrawchangeaddress`/`getnewaddress` failures (UdjinM6)
7c08ccf wallet: Avoid updating `ReserveDestination::nIndex` when `GetReservedDestination` fails (UdjinM6)

Pull request description:

  Backport:

  * #29510
  * #29412
  * #29524

ACKs for top commit:
  glozow:
    utACK 27cfda1

Tree-SHA512: 37feadd65d9ea55c0a92c9d2a6f74f87cafed3bc67f8deeaaafc5b7042f954e55ea34816612e1a49088f4f1906f104e00c7c3bec7affd1c1f48220b57a8769c5
backpacker69 pushed a commit to peercoin/peercoin that referenced this pull request Jun 14, 2024
…Destination` fails

Github-Pull: bitcoin/bitcoin#29510
Rebased-From: 367bb7a80cc71130995672c853d4a6e0134721d6
backpacker69 pushed a commit to peercoin/peercoin that referenced this pull request Jun 14, 2024
…`getnewaddress` failures

Github-Pull: bitcoin/bitcoin#29510
Rebased-From: e073f1dfda7a2a2cb2be9fe2a1d576f122596021
@bitcoin bitcoin locked and limited conversation to collaborators Mar 5, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants