-
Notifications
You must be signed in to change notification settings - Fork 38.6k
gui: Avoid redundant tx status updates #17905
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
In TransactionTablePriv::index, avoid calling interfaces::Wallet::tryGetTxStatus if the status is up to date as of the most recent NotifyBlockTip notification. Store height from the most recent notification in a new ClientModel::cachedNumBlocks variable in order to check this. This avoids floods of IPC traffic from tryGetTxStatus with bitcoin#10102 when there are a lot of transactions. It might also make the GUI a little more efficient even when there is no IPC.
promag
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK.
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
promag
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code review ACK 2b3d6bf. Restarted the travis job, I think it's an unrelated failure:
interfaces/node.cpp:167:51: runtime error: implicit conversion from type 'unsigned long' of value 13744632839234567870 (64-bit, unsigned) to type 'int64_t' (aka 'long') changed the value to -4702111234474983746 (64-bit, signed)
#0 0x55b40a49492c in interfaces::(anonymous namespace)::NodeImpl::getTotalBytesRecv() /home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-pc-linux-gnu/src/interfaces/node.cpp:167:51
#1 0x55b40a02986f in ClientModel::ClientModel(interfaces::Node&, OptionsModel*, QObject*)::$_0::operator()() const /home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-pc-linux-gnu/src/qt/clientmodel.cpp:48:36
#2 0x55b40a02986f in QtPrivate::FunctorCall<QtPrivate::IndexesList<>, QtPrivate::List<>, void, ClientModel::ClientModel(interfaces::Node&, OptionsModel*, QObject*)::$_0>::call(ClientModel::ClientModel(interfaces::Node&, OptionsModel*, QObject*)::$_0&, void**) /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:130
#3 0x55b40a029404 in void QtPrivate::Functor<ClientModel::ClientModel(interfaces::Node&, OptionsModel*, QObject*)::$_0, 0>::call<QtPrivate::List<>, void>(ClientModel::ClientModel(interfaces::Node&, OptionsModel*, QObject*)::$_0&, void*, void**) /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:240:13
#4 0x55b40a029404 in QtPrivate::QFunctorSlotObject<ClientModel::ClientModel(interfaces::Node&, OptionsModel*, QObject*)::$_0, 0, QtPrivate::List<>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) /usr/include/x86_64-linux-gnu/qt5/QtCore/qobject_impl.h:168
#5 0x7f6de37ee75e in QMetaObject::activate(QObject*, int, int, void**) (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2b975e)
#6 0x7f6de37fb0b6 in QTimer::timeout(QTimer::QPrivateSignal) (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2c60b6)
#7 0x7f6de37fb417 in QTimer::timerEvent(QTimerEvent*) (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2c6417)
#8 0x7f6de37ef16a in QObject::event(QEvent*) (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2ba16a)
#9 0x7f6de287c83b in QApplicationPrivate::notify_helper(QObject*, QEvent*) (/usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5+0x15483b)
#10 0x7f6de2884103 in QApplication::notify(QObject*, QEvent*) (/usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5+0x15c103)
#11 0x7f6de37bf9c7 in QCoreApplication::notifyInternal2(QObject*, QEvent*) (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x28a9c7)
#12 0x7f6de3817e1d in QTimerInfoList::activateTimers() (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2e2e1d)
#13 0x7f6de38185e0 (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2e35e0)
#14 0x7f6dde8dd416 in g_main_context_dispatch (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x4c416)
#15 0x7f6dde8dd64f (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x4c64f)
#16 0x7f6dde8dd6db in g_main_context_iteration (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x4c6db)
#17 0x7f6de381897e in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2e397e)
#18 0x7f6de37bd9f9 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2889f9)
#19 0x7f6de35dc239 in QThread::exec() (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0xa7239)
#20 0x7f6de35e117c (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0xac17c)
#21 0x7f6de41166da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
#22 0x7f6de036988e in clone (/lib/x86_64-linux-gnu/libc.so.6+0x12188e)
SUMMARY: UndefinedBehaviorSanitizer: implicit-integer-sign-change interfaces/node.cpp:167:51 in
FAIL qt/test/test_bitcoin-qt (exit status: 1)
| if(data) | ||
| { | ||
| return createIndex(row, column, priv->index(walletModel->wallet(), row)); | ||
| return createIndex(row, column, data); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well this change alone already improves!
2b3d6bf to
26cde38
Compare
ryanofsky
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated 2b3d6bf -> 26cde38 (pr/ipc-txup.1 -> pr/ipc-txup.2, compare) with suggested changes
Restarted the travis job, I think it's an unrelated failure
Travis failure was just the #17906 failure. Should be fixed now that is merged
|
Concept ACK One remark: Is block height enough as an identifier of the current state? Might this miss a 1-block reorg that ends up in the same height? If so, wouldn't "tip blockhash" be better for this? |
This is true, and if you'd like I could add a followup commit or open a new PR to address this. But the current PR isn't changing anything there because |
promag
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ClientModel gives access to interfaces::Node& and OptionsModel* so maybe take advantage of this and simplify the constructors.
@ryanofsky unless this makes breaking circular dependencies harder. |
26cde38 to
689424a
Compare
ryanofsky
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for suggestions!
Updated 26cde38 -> 689424a (pr/ipc-txup.2 -> pr/ipc-txup.3, compare) implementing suggestions
promag
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code review ACK 689424a.
Just two comments for your consideration.
src/qt/clientmodel.h
Outdated
| // caches for the best header, number of blocks | ||
| mutable std::atomic<int> cachedBestHeaderHeight; | ||
| mutable std::atomic<int64_t> cachedBestHeaderTime; | ||
| mutable std::atomic<int> m_cached_num_blocks; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit
| mutable std::atomic<int> m_cached_num_blocks; | |
| mutable std::atomic<int> m_cached_num_blocks{-1}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| int ClientModel::getNumBlocks() const | ||
| { | ||
| if (m_cached_num_blocks == -1) { | ||
| m_cached_num_blocks = m_node.getNumBlocks(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe just initialize m_cached_num_blocks after subscribeToCoreSignals() and drop the -1 case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re: #17905 (comment)
Maybe just initialize
m_cached_num_blocksaftersubscribeToCoreSignals()and drop the-1case?
It seems a little more fragile to rely on another code path being executed if this is going to be initialized to -1. Also it's more consistent with existing getHeaderTipHeight and getHeaderTipTime to be checking this way
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean this:
m_thread->start();
subscribeToCoreSignals();
+ m_cached_num_blocks = m_node.getNumBlocks();
}
ClientModel::~ClientModel()It's the constructor so it's always executed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re: #17905 (comment)
Thanks again for the review!
I mean this:
Yes, I did understand the suggestion. I like having getNumBlocks implemented the same way as getHeaderTipHeight and getHeaderTipTime, and I like m_cached_num_blocks being touched as few places as possible (initialized to -1 and updated only as needed) and not needing to interact with the more complicated constructor.
I also think this case is similar to #18160 where I am trying to avoid an unnecessary interface method call (in that case interfaces::Wallet::tryGetBalances, in this case interfaces::Node::getNumBlocks), because I see these calls as potentially costly (with #10102 requiring serializing arguments, sending them across a socket and optionally logging them to debug.log, waiting for reply on the socket, deserializing and optionally logging that), but I think you see them more like ordinary method calls.
If other reviewers agree with your suggestion, though, I'm ok with changing this. I don't think it is very significant.
Meta point: When I leave comment on PRs I try be be explicit about (1) What my suggestion is (2) What the benefits of the suggestion are or underlying problem is.
Some reviewers are great at saying what they want but not why they want it, others are great at finding things that should be improved without being specific enough about what kind of change they want to see. In this case, I understand the suggestion but don't understand what's driving it. Readability, avoiding a potential bug, efficiency, or just style?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for not explaining my POV. I'm suggesting to just make ClientModel::getNumBlocks lock free - not sure what would be the first call stack hitting the lock and if it would hurt the event loop, but with my suggestion that wouldn't be a concern. But like you say, it's not very significant so feel free to ignore it, FWIW I did ACK.
689424a to
96cb597
Compare
ryanofsky
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated 689424a -> 96cb597 (pr/ipc-txup.3 -> pr/ipc-txup.4, compare) with suggested change
| int ClientModel::getNumBlocks() const | ||
| { | ||
| if (m_cached_num_blocks == -1) { | ||
| m_cached_num_blocks = m_node.getNumBlocks(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re: #17905 (comment)
Maybe just initialize
m_cached_num_blocksaftersubscribeToCoreSignals()and drop the-1case?
It seems a little more fragile to rely on another code path being executed if this is going to be initialized to -1. Also it's more consistent with existing getHeaderTipHeight and getHeaderTipTime to be checking this way
src/qt/clientmodel.h
Outdated
| // caches for the best header, number of blocks | ||
| mutable std::atomic<int> cachedBestHeaderHeight; | ||
| mutable std::atomic<int64_t> cachedBestHeaderTime; | ||
| mutable std::atomic<int> m_cached_num_blocks; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
|
Code review ACK 96cb597. |
|
Concept ACK. |
hebasto
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK 96cb597
…ceChanged 0933a37 gui: Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged (João Barbosa) Pull request description: Each 250ms the slot `WalletModel::pollBalanceChanged` is called which, at worst case, calls `Wallet::GetBalance`. This is a waste of resources since most of the time there aren't new transactions or new blocks. Fix this by early checking if cache is dirty or not. The actual balance computation can still hang the GUI thread but that is tracked in #16874 and should be fixed with a solution similar to #17135. ACKs for top commit: hebasto: ACK 0933a37, I have not tested the code, but I have reviewed it and it looks OK, I agree it can be merged. jonasschnelli: utACK 0933a37 instagibbs: ACK 0933a37 ryanofsky: Code review ACK 0933a37, but I would prefer (not strongly) for #17905 to be merged first. This PR can be simpler if it is based on #17905, so tryGetBalances can just be left alone instead of changing into to a more complicated tryGetBalancesIfNeeded function, and then getting changed back later when we want to optimize it out. jonatack: ACK 0933a37 based primarily on code review, despite a lot of manual testing with a large 177MB wallet. Tree-SHA512: 18db35bf33a7577666658c8cb0b57308c8474baa5ea95bf1468cd8531a69857d8915584f6ac505874717aa6aabeb1b506ac77630f8acdb6651afab89275e38a1
…llBalanceChanged 0933a37 gui: Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged (João Barbosa) Pull request description: Each 250ms the slot `WalletModel::pollBalanceChanged` is called which, at worst case, calls `Wallet::GetBalance`. This is a waste of resources since most of the time there aren't new transactions or new blocks. Fix this by early checking if cache is dirty or not. The actual balance computation can still hang the GUI thread but that is tracked in bitcoin#16874 and should be fixed with a solution similar to bitcoin#17135. ACKs for top commit: hebasto: ACK 0933a37, I have not tested the code, but I have reviewed it and it looks OK, I agree it can be merged. jonasschnelli: utACK 0933a37 instagibbs: ACK 0933a37 ryanofsky: Code review ACK 0933a37, but I would prefer (not strongly) for bitcoin#17905 to be merged first. This PR can be simpler if it is based on bitcoin#17905, so tryGetBalances can just be left alone instead of changing into to a more complicated tryGetBalancesIfNeeded function, and then getting changed back later when we want to optimize it out. jonatack: ACK 0933a37 based primarily on code review, despite a lot of manual testing with a large 177MB wallet. Tree-SHA512: 18db35bf33a7577666658c8cb0b57308c8474baa5ea95bf1468cd8531a69857d8915584f6ac505874717aa6aabeb1b506ac77630f8acdb6651afab89275e38a1
No change in behavior, this just gives wallet code access to state cached in ClientModel so it can be used in upcoming commits to poll more efficiently. Change suggested by João Barbosa <[email protected]> bitcoin#17905 (review)
|
This PR might be mergeable: has 2 acks, is a pretty limited gui change |
96cb597 gui: Avoid redundant tx status updates (Russell Yanofsky) Pull request description: This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10). In `TransactionTablePriv::index`, avoid calling `interfaces::Wallet::tryGetTxStatus` if the status is up to date as of the most recent `NotifyBlockTip` notification. Store height from the most recent notification in a new `ClientModel::cachedNumBlocks` variable in order to check this. This avoids floods of IPC traffic from `tryGetTxStatus` with bitcoin#10102 when there are a lot of transactions. It might also make the GUI a little more efficient even when there is no IPC. ACKs for top commit: promag: Code review ACK 96cb597. hebasto: ACK 96cb597 Tree-SHA512: fce597bf52a813ad4923110d0a39229ea09e1631e0d580ea18cffb09e58cdbb4b111a40a9a9270ff16d8163cd47b0bd9f1fe7e3a6c7ebb19198f049f8dd1aa46
Tweak of bitcoin#17905 to make gui display of transactions and balances more consistent. This change shouldn't cause visible effects in normal cases, just make GUI wallet code more internally correct and consistent.
Tweak of bitcoin#17905 to make gui display of transactions and balances more consistent. This change shouldn't cause visible effects in normal cases, just make GUI wallet code more internally correct and consistent.
Tweak of bitcoin#17905 to make gui display of transactions and balances more consistent. This change shouldn't cause visible effects in normal cases, just make GUI wallet code more internally correct and consistent.
Switch transaction table to use wallet height not node height Tweak of bitcoin#17905 to make gui display of transactions and balances more consistent. This change shouldn't cause visible effects in normal cases, just make GUI wallet code more internally correct and consistent.
Switch transaction table to use wallet height not node height Tweak of bitcoin#17905 to make gui display of transactions and balances more consistent. This change shouldn't cause visible effects in normal cases, just make GUI wallet code more internally correct and consistent.
This issue is finally fixed in #17993 by @furszy, which cleanly replaces |
…k hash. a06e845 BlockTip struct created and connected to notifyHeaderTip and notifyBlockTip signals. (furszy) 2f86720 Added best block hash to the NotifyHeaderTip and NotifyBlockTip signals. (furszy) Pull request description: Rationale: The height based polling in the GUI is an issue on chain reorgs. Any new tip signal with the same height as the one that it's cached in the model was not triggering the GUI update (interpreting it as the same same block when it could receive a different one). Ending up with bad information presented in the GUI. This PR essentially changes the last cached height to be a last cached block hash. --- Old historical information of this PR. As the tip height is cached and updated via signaling in clientModel, there is no need to continue locking `cs_main` on every balance poll (`m_node.getNumBlocks()` method call). Extra topic: Would suggest to change the `cachedNumBlocks` field inside `walletModel` to a more understandable name, maybe `nLastBalanceUpdateHeight`. And finally, this will have the equal height reorg issue mentioned [here](#17905 (comment)), whatever is presented to fix it, this should use the same flow too. **[Edit - 24/01/2020]** Have added #[17905](#17905 (comment)) comment fix here too. ACKs for top commit: jonasschnelli: utACK a06e845 - it would be great to have QT unit tests (in this case for a reorg) that either automatically inspect the window content based on accessibility and tests for expected values or at least allow for quick manual re-testing (screenshots, automatically create UI situations). hebasto: re-ACK a06e845, suggested style changes implemented since the [previous](#17993 (review)) review. ryanofsky: Code review ACK a06e845. A lot of changes since the last review: rebase after sync_state introduction #18152 and tryGetBalances revert #18587, reverting getLastBlockTime change, fixing spacing and initializations and renaming some variables Tree-SHA512: 835e587a8296df9899cccd7b3e598a5970942b640e432e6a32de0b4eaea5b40f9271258f089ec033595311707b74a0f7187ecf8ed397c713e1153e2714072975
…st block hash. a06e845 BlockTip struct created and connected to notifyHeaderTip and notifyBlockTip signals. (furszy) 2f86720 Added best block hash to the NotifyHeaderTip and NotifyBlockTip signals. (furszy) Pull request description: Rationale: The height based polling in the GUI is an issue on chain reorgs. Any new tip signal with the same height as the one that it's cached in the model was not triggering the GUI update (interpreting it as the same same block when it could receive a different one). Ending up with bad information presented in the GUI. This PR essentially changes the last cached height to be a last cached block hash. --- Old historical information of this PR. As the tip height is cached and updated via signaling in clientModel, there is no need to continue locking `cs_main` on every balance poll (`m_node.getNumBlocks()` method call). Extra topic: Would suggest to change the `cachedNumBlocks` field inside `walletModel` to a more understandable name, maybe `nLastBalanceUpdateHeight`. And finally, this will have the equal height reorg issue mentioned [here](bitcoin#17905 (comment)), whatever is presented to fix it, this should use the same flow too. **[Edit - 24/01/2020]** Have added #[17905](bitcoin#17905 (comment)) comment fix here too. ACKs for top commit: jonasschnelli: utACK a06e845 - it would be great to have QT unit tests (in this case for a reorg) that either automatically inspect the window content based on accessibility and tests for expected values or at least allow for quick manual re-testing (screenshots, automatically create UI situations). hebasto: re-ACK a06e845, suggested style changes implemented since the [previous](bitcoin#17993 (review)) review. ryanofsky: Code review ACK a06e845. A lot of changes since the last review: rebase after sync_state introduction bitcoin#18152 and tryGetBalances revert bitcoin#18587, reverting getLastBlockTime change, fixing spacing and initializations and renaming some variables Tree-SHA512: 835e587a8296df9899cccd7b3e598a5970942b640e432e6a32de0b4eaea5b40f9271258f089ec033595311707b74a0f7187ecf8ed397c713e1153e2714072975
Summary: > In TransactionTablePriv::index, avoid calling > interfaces::Wallet::tryGetTxStatus if the status is up to date as of the most > recent NotifyBlockTip notification. Store height from the most recent > notification in a new ClientModel::cachedNumBlocks variable in order to check > this. > > This avoids floods of IPC traffic from tryGetTxStatus with [[bitcoin/bitcoin#10102 | PR10102]] when there > are a lot of transactions. It might also make the GUI a little more efficient > even when there is no IPC. This is a backport of Core [[bitcoin/bitcoin#17905 | PR17905]] Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Differential Revision: https://reviews.bitcoinabc.org/D8871
Bitcoin Core PR: bitcoin/bitcoin#17905 Pull request description: This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10). In `TransactionTablePriv::index`, avoid calling `interfaces::Wallet::tryGetTxStatus` if the status is up to date as of the most recent `NotifyBlockTip` notification. Store height from the most recent notification in a new `ClientModel::cachedNumBlocks` variable in order to check this. This avoids floods of IPC traffic from `tryGetTxStatus` with bitcoin/bitcoin#10102 when there are a lot of transactions. It might also make the GUI a little more efficient even when there is no IPC.
This PR is part of the process separation project.
In
TransactionTablePriv::index, avoid callinginterfaces::Wallet::tryGetTxStatusif the status is up to date as of the most recentNotifyBlockTipnotification. Store height from the most recent notification in a newClientModel::cachedNumBlocksvariable in order to check this.This avoids floods of IPC traffic from
tryGetTxStatuswith #10102 when there are a lot of transactions. It might also make the GUI a little more efficient even when there is no IPC.