-
Notifications
You must be signed in to change notification settings - Fork 38.7k
test: speedup wallet_backup by whitelisting peers (immediate tx relay) #17121
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
fanquake
merged 1 commit into
bitcoin:master
from
theStack:20191013-test-use_whitelist_in_wallet-backup_to_speedup_mempool_sync
Oct 13, 2019
Merged
test: speedup wallet_backup by whitelisting peers (immediate tx relay) #17121
fanquake
merged 1 commit into
bitcoin:master
from
theStack:20191013-test-use_whitelist_in_wallet-backup_to_speedup_mempool_sync
Oct 13, 2019
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
approaches part of bitcoin#16613 ("Functional test suite bottlenecks") The majority of the test time is spent in sync_mempools() after sending to addresses, i.e. the bottleneck is in relaying transactions. By whitelisting the peers via -whitelist, the inventory is transmissioned immediately rather than on average every 5 seconds, speeding up the test by at least a factor of two: before: $ time ./wallet_backup.py real 2m2.523s user 0m6.093s sys 0m2.454s with this PR: $ time ./wallet_backup_with_whitelist.py real 0m36.570s user 0m5.365s sys 0m1.696s Note that the test is not deterministic (the sendtoaddress RPC in function one_send() is executed with a probability of 50%), hence the times could vary between individual runs.
Member
|
ACK 581c9be, this test is testing the backup behaviour, not the tx relay behaviour |
fanquake
approved these changes
Oct 13, 2019
Member
fanquake
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fanquake
added a commit
that referenced
this pull request
Oct 13, 2019
…diate tx relay) 581c9be test: speedup wallet_backup by whitelisting peers (immediate tx relay) (Sebastian Falbesoner) Pull request description: approaches part of #16613 ("Functional test suite bottlenecks") The majority of the test time is spent in `sync_mempools()` after sending to addresses, i.e. the bottleneck is in relaying transactions. By whitelisting the peers via `-whitelist`, the inventory is transmissioned immediately rather than on average every 5 seconds, speeding up the test by at least a factor of two: before: ``` $ time ./wallet_backup.py real 2m2.523s user 0m6.093s sys 0m2.454s ``` with this PR: ``` $ time ./wallet_backup_with_whitelist.py real 0m36.570s user 0m5.365s sys 0m1.696s ``` Note that the test is not deterministic (the `sendtoaddress` RPC in function `one_send()` is executed with a probability of 50%), hence the times could vary between individual runs. ACKs for top commit: MarcoFalke: ACK 581c9be, this test is testing the backup behaviour, not the tx relay behaviour fanquake: ACK 581c9be Tree-SHA512: d016f39cdb85501e17a74a4c4db5a9f7404baa76fbcc3675a34d3cd7bf03d7a4cb4fd3e5f17cb0597248120bb5ac8b15d3db7663007b76b010902be72954bde0
deadalnix
pushed a commit
to Bitcoin-ABC/bitcoin-abc
that referenced
this pull request
Apr 15, 2020
…diate tx relay)
Summary:
581c9be0d8bff46cd68bd6a3bf72f22d11c09aea test: speedup wallet_backup by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)
Pull request description:
approaches part of #16613 ("Functional test suite bottlenecks")
The majority of the test time is spent in `sync_mempools()` after sending to
addresses, i.e. the bottleneck is in relaying transactions. By whitelisting the
peers via `-whitelist`, the inventory is transmissioned immediately rather than
on average every 5 seconds, speeding up the test by at least a factor of two:
before:
```
$ time ./wallet_backup.py
real 2m2.523s
user 0m6.093s
sys 0m2.454s
```
with this PR:
```
$ time ./wallet_backup_with_whitelist.py
real 0m36.570s
user 0m5.365s
sys 0m1.696s
```
Note that the test is not deterministic (the `sendtoaddress` RPC in function
`one_send()` is executed with a probability of 50%), hence the times could vary
between individual runs.
ACKs for top commit:
MarcoFalke:
ACK 581c9be0d8bff46cd68bd6a3bf72f22d11c09aea, this test is testing the backup behaviour, not the tx relay behaviour
fanquake:
ACK 581c9be0d8bff46cd68bd6a3bf72f22d11c09aea
Tree-SHA512: d016f39cdb85501e17a74a4c4db5a9f7404baa76fbcc3675a34d3cd7bf03d7a4cb4fd3e5f17cb0597248120bb5ac8b15d3db7663007b76b010902be72954bde0
Backport of Core [[bitcoin/bitcoin#17121 | PR17121]]
May be related to this flakiness as well: https://build.bitcoinabc.org/viewLog.html?buildId=35712&buildTypeId=BitcoinABCMasterLinux&tab=buildLog&_focus=1865
Test Plan:
```
test_runner.py wallet_backup
```
On my machine, runtime goes from ~116s to ~36s
Reviewers: #bitcoin_abc, deadalnix
Reviewed By: #bitcoin_abc, deadalnix
Differential Revision: https://reviews.bitcoinabc.org/D5731
maflcko
pushed a commit
that referenced
this pull request
Aug 16, 2020
… speedup) 9e78943 test: speedup p2p_feefilter.py by whitelisting peers (immediate tx relay) (Sebastian Falbesoner) fe3f0cc test: use wait_until for invs matching in p2p_feefilter.py (Sebastian Falbesoner) 6d94192 test: add logging for p2p_feefilter.py (Sebastian Falbesoner) Pull request description: This PR gives some love to the functional test `p2p_feefilter.py` by introducing the following changes: * add missing log messages for the `test_feefilter` subtest (the main one) * deduplicate code by using the utility function `wait_until` (already using the [recently introduced version](12410b1)) instead of a manual condition checking loop with `time.sleep` * improve naming of the function `matchAllInvs` (more expressive name, snake_case) and moving it from global namespace to the connection class `FeefilterConn` * speeding up the test significantly by the good ol' method of activating immediate tx relay (as seen on e.g. #17121, #17124, #17340, #17362, ...): ``` master branch: $ time ./p2p_feefilter.py ... real 0m39.367s user 0m1.227s sys 0m0.571s PR branch: $ time ./p2p_feefilter.py ... real 0m9.386s user 0m1.120s sys 0m0.577s ``` ACKs for top commit: instagibbs: code review ACK 9e78943 jonatack: re-ACK 9e78943 per `git range-diff 3ab2582 ea74a3c 9e78943` Tree-SHA512: fe21c1c5413df9165fea916b5d5f609d3ba33e7b5c3364b38eb824fcc55d9e6abddf27116cbc0b325913d451a73c44542040fb916aec9c46f805c6e12f6f10cf
sidhujag
pushed a commit
to syscoin/syscoin
that referenced
this pull request
Aug 16, 2020
…toring, speedup) 9e78943 test: speedup p2p_feefilter.py by whitelisting peers (immediate tx relay) (Sebastian Falbesoner) fe3f0cc test: use wait_until for invs matching in p2p_feefilter.py (Sebastian Falbesoner) 6d94192 test: add logging for p2p_feefilter.py (Sebastian Falbesoner) Pull request description: This PR gives some love to the functional test `p2p_feefilter.py` by introducing the following changes: * add missing log messages for the `test_feefilter` subtest (the main one) * deduplicate code by using the utility function `wait_until` (already using the [recently introduced version](bitcoin@12410b1)) instead of a manual condition checking loop with `time.sleep` * improve naming of the function `matchAllInvs` (more expressive name, snake_case) and moving it from global namespace to the connection class `FeefilterConn` * speeding up the test significantly by the good ol' method of activating immediate tx relay (as seen on e.g. bitcoin#17121, bitcoin#17124, bitcoin#17340, bitcoin#17362, ...): ``` master branch: $ time ./p2p_feefilter.py ... real 0m39.367s user 0m1.227s sys 0m0.571s PR branch: $ time ./p2p_feefilter.py ... real 0m9.386s user 0m1.120s sys 0m0.577s ``` ACKs for top commit: instagibbs: code review ACK bitcoin@9e78943 jonatack: re-ACK 9e78943 per `git range-diff 3ab2582 ea74a3c 9e78943` Tree-SHA512: fe21c1c5413df9165fea916b5d5f609d3ba33e7b5c3364b38eb824fcc55d9e6abddf27116cbc0b325913d451a73c44542040fb916aec9c46f805c6e12f6f10cf
ftrader
pushed a commit
to bitcoin-cash-node/bitcoin-cash-node
that referenced
this pull request
Aug 17, 2020
…diate tx relay)
Summary:
581c9be0d8bff46cd68bd6a3bf72f22d11c09aea test: speedup wallet_backup by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)
Pull request description:
approaches part of #16613 ("Functional test suite bottlenecks")
The majority of the test time is spent in `sync_mempools()` after sending to
addresses, i.e. the bottleneck is in relaying transactions. By whitelisting the
peers via `-whitelist`, the inventory is transmissioned immediately rather than
on average every 5 seconds, speeding up the test by at least a factor of two:
before:
```
$ time ./wallet_backup.py
real 2m2.523s
user 0m6.093s
sys 0m2.454s
```
with this PR:
```
$ time ./wallet_backup_with_whitelist.py
real 0m36.570s
user 0m5.365s
sys 0m1.696s
```
Note that the test is not deterministic (the `sendtoaddress` RPC in function
`one_send()` is executed with a probability of 50%), hence the times could vary
between individual runs.
ACKs for top commit:
MarcoFalke:
ACK 581c9be0d8bff46cd68bd6a3bf72f22d11c09aea, this test is testing the backup behaviour, not the tx relay behaviour
fanquake:
ACK 581c9be0d8bff46cd68bd6a3bf72f22d11c09aea
Tree-SHA512: d016f39cdb85501e17a74a4c4db5a9f7404baa76fbcc3675a34d3cd7bf03d7a4cb4fd3e5f17cb0597248120bb5ac8b15d3db7663007b76b010902be72954bde0
Backport of Core [[bitcoin/bitcoin#17121 | PR17121]]
May be related to this flakiness as well: https://build.bitcoinabc.org/viewLog.html?buildId=35712&buildTypeId=BitcoinABCMasterLinux&tab=buildLog&_focus=1865
Test Plan:
```
test_runner.py wallet_backup
```
On my machine, runtime goes from ~116s to ~36s
Reviewers: #bitcoin_abc, deadalnix
Reviewed By: #bitcoin_abc, deadalnix
Differential Revision: https://reviews.bitcoinabc.org/D5731
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this pull request
Dec 22, 2021
…s (immediate tx relay) 581c9be test: speedup wallet_backup by whitelisting peers (immediate tx relay) (Sebastian Falbesoner) Pull request description: approaches part of bitcoin#16613 ("Functional test suite bottlenecks") The majority of the test time is spent in `sync_mempools()` after sending to addresses, i.e. the bottleneck is in relaying transactions. By whitelisting the peers via `-whitelist`, the inventory is transmissioned immediately rather than on average every 5 seconds, speeding up the test by at least a factor of two: before: ``` $ time ./wallet_backup.py real 2m2.523s user 0m6.093s sys 0m2.454s ``` with this PR: ``` $ time ./wallet_backup_with_whitelist.py real 0m36.570s user 0m5.365s sys 0m1.696s ``` Note that the test is not deterministic (the `sendtoaddress` RPC in function `one_send()` is executed with a probability of 50%), hence the times could vary between individual runs. ACKs for top commit: MarcoFalke: ACK 581c9be, this test is testing the backup behaviour, not the tx relay behaviour fanquake: ACK 581c9be Tree-SHA512: d016f39cdb85501e17a74a4c4db5a9f7404baa76fbcc3675a34d3cd7bf03d7a4cb4fd3e5f17cb0597248120bb5ac8b15d3db7663007b76b010902be72954bde0
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
approaches part of #16613 ("Functional test suite bottlenecks")
The majority of the test time is spent in
sync_mempools()after sending toaddresses, i.e. the bottleneck is in relaying transactions. By whitelisting the
peers via
-whitelist, the inventory is transmissioned immediately rather thanon average every 5 seconds, speeding up the test by at least a factor of two:
before:
with this PR:
Note that the test is not deterministic (the
sendtoaddressRPC in functionone_send()is executed with a probability of 50%), hence the times could varybetween individual runs.