Skip to content

Conversation

@D33r-Gee
Copy link

@D33r-Gee D33r-Gee commented Aug 1, 2025

Add UTXO Snapshot Loading Interface and Progress Notifications

This PR implements a complete interface for loading UTXO snapshots through the GUI, including progress notifications to provide user feedback during the loading process.


Changes

Commit 1: interfaces: expose UTXO snapshot loading in node interface

  • Added a new virtual method loadSnapshot to the node interface for loading UTXO snapshots.
  • Updated the implementation in interfaces.cpp to activate the snapshot using the provided file and metadata.
  • This change enhances the user's ability to load UTXO snapshots through the GUI.

Commit 2: notifications: Implement snapshot load progress notifications

  • Added a new signal SnapshotLoadProgress for snapshot load progress in the UI interface.
  • Updated the notifications interface to include a method for reporting snapshot loading progress.
  • Enhanced the ChainstateManager to report progress during snapshot loading.
  • Implemented the necessary handler in the kernel notifications to relay progress updates.
  • This change improves user feedback during the snapshot loading process.

Commit 3: rpc: refactor loadtxoutset to use snapshot interface

  • Updated the loadtxoutset RPC to call the new snapshot interface instead of directly working with chainstate internals.
  • Added getMetadata and getActivationResult methods to the snapshot interface to provide parity with the legacy RPC.
  • Improves modularity by consolidating snapshot handling through the interface layer.

Technical Details

The implementation includes:

  • UI Interface Enhancement
    Added SnapshotLoadProgress signal to CClientUIInterface with corresponding signal declaration and implementation.

  • Node Interface
    Added loadSnapshot virtual method to the node interface with implementation in interfaces.cpp.

  • Progress Notifications
    Implemented snapshotLoadProgress method in KernelNotifications class to relay progress updates to the UI.

  • Base Interface
    Added snapshotLoadProgress virtual method to the kernel notifications interface.


Benefits

  • User Experience
    Provides real-time feedback during UTXO snapshot loading operations.

  • GUI Integration
    Enables loading UTXO snapshots directly through the Bitcoin Core GUI.

  • Progress Tracking
    Users can monitor the progress of snapshot loading operations.

  • Consistent Interface
    Follows the existing pattern for progress notifications in the codebase.

Testing

  • Daemon build and launch bitcoind then with bitcoin-cli load a snapshot using:
./bitcoin-cli -rpcclienttimeout=0 -signet loadtxoutset /path/to/utxo-snapsshot.dat
  • QT Widgets: build and run this draft PR in legacy gui repo
  • QML version: for a modern look build and run this PR in the newly updated gui-qml repo

@DrahtBot
Copy link
Contributor

DrahtBot commented Aug 1, 2025

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33117.

Reviews

See the guideline for information on the review process.

Type Reviewers
Concept ACK pinheadmz
Approach ACK TheCharlatan

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #bitcoin-core/gui/870 ([DRAFT] Expose AssumeUTXO Load Snapshot Functionality To The GUI by D33r-Gee)
  • #32297 (bitcoin-cli: Add -ipcconnect option by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@Sjors
Copy link
Member

Sjors commented Aug 1, 2025

I plan to test this soon(tm).

@D33r-Gee
Copy link
Author

D33r-Gee commented Aug 1, 2025

I plan to test this soon(tm).

Thanks @Sjors !

@@ -204,6 +205,9 @@ class Node
//! List rpc commands.
virtual std::vector<std::string> listRpcCommands() = 0;

//! Load UTXO Snapshot.
virtual bool loadSnapshot(AutoFile& afile, const node::SnapshotMetadata& metadata, bool in_memory) = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "interfaces: expose UTXO snapshot loading in node interface" (de11e3b89f8aba9a90a3419086807e5fee0f941d)

Note with #10102 if the GUI & node are running in separate processes, it could be a awkward for the gui process to create an AutoFile object and pass it to the node like this. Technically it could probably work: the AutoFile could be passed as a file descriptor or a (path, position) tuple. But it doesn't seem like a clean separation of responsibilities if the GUI process is reading the beginning of the snapshot file and the node process is reading the rest of the file.

Probably current interface is ok, but it might be more ideal if node provided a more generic loadSnapshot method and the GUI didn't directly access the file:

class Node
{
    // ...
    std::unique_ptr<Snapshot> loadSnapshot(const fs::path& path) = 0;
    // ...
};

class Snapshot
{
public:
    virtual ~Snapshot() = default;
    node::SnapshotMetadata getMetadata() = 0;
    bool activate() = 0;
};

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review... will update soon addressing this

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with e79c50c updated the node.h and added snapshot.h to the /interfaces.
Also updated interfaces.cpp with a SnapshotImpl class to process the path and extracts the metada and activate the snapshot.
@ryanofsky let me know if that's what you had in mind?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re: #33117 (comment)

Thanks! I think b464f388197c5c31937d309d817d38c4b4849a91 is an improvement over previous de11e3b89f8aba9a90a3419086807e5fee0f941d because it passes an fs::path argument from the GUI to the node instead of an AutoFile file argument. This is nice because it lets node be fully responsible for parsing the snapshot file instead of relying on the GUI.

One comment on b464f388197c5c31937d309d817d38c4b4849a91 is that it looks like the Snapshot class is not actually necessary in its current form. The reason I suggested adding a Snapshot class above was that I thought GUI wanted to display SnapshotMetadata information before activating the snapshot, and a Snapshot class could provide separate getMetadata() and activate() methods to allow that. But since the Snapshot class only has a single activate() method in b464f388197c5c31937d309d817d38c4b4849a91, it looks like the class is unnecessary, and you could just have a plain Node::activateSnapshot(fs::path) method that returns bool instead of a Snapshot.

Any approach is fine though, so you should choose whatever seems to make the most sense.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking a look!
RE:

The reason I suggested adding a Snapshot class above was that I thought GUI wanted to display SnapshotMetadata information before activating the snapshot, and a Snapshot class could provide separate getMetadata()

I took some time to test various scenarios downstream in the gui-qml project. While we aren't directly utilizing the snapshot data at the moment, I believe your initial instincts about introducing a Snapshot class were well-founded. This abstraction will become increasingly valuable as we begin to expose the dump/generate snapshot functionalities trough the GUI.

@pinheadmz
Copy link
Member

concept ACK, tested the mainnet snapshot load with the QML gui branch built on top of this

@fanquake
Copy link
Member

fanquake commented Aug 6, 2025

https://github.com/bitcoin/bitcoin/actions/runs/16728433539/job/47504424705?pr=33117#step:10:445:

D:\a\bitcoin\bitcoin\src\node\interfaces.cpp(115,40): error C2220: the following warning is treated as an error [D:\a\bitcoin\bitcoin\build\src\bitcoin_node.vcxproj]
D:\a\bitcoin\bitcoin\src\node\interfaces.cpp(115,40): warning C4101: 'e': unreferenced local variable [D:\a\bitcoin\bitcoin\build\src\bitcoin_node.vcxproj]

@D33r-Gee D33r-Gee force-pushed the interface-load-snapshot branch from e79c50c to 8bc4035 Compare August 6, 2025 14:57
@D33r-Gee
Copy link
Author

D33r-Gee commented Aug 7, 2025

with 8bc4035 it has been rebased and the fuzzing CI error has been addressed

Copy link
Member

@Sjors Sjors left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first glance this looks very reasonable. I also tested it in the (current) GUI with bitcoin-core/gui#870

I had a similar concern as @ryanofsky about how to go about passing the snapshot file (path) over the interface. The current approach in 3c8b578836dd0d8438a8aac81a1166df84811562 is the same as how we handle wallet backups (restoreWallet), so probably fine?

return false;
}

auto activation_result{m_chainman.ActivateSnapshot(afile, metadata, m_in_memory)};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 3c8b578836dd0d8438a8aac81a1166df84811562 interfaces: expose UTXO snapshot loading in node interface: a good way to add test coverage to this new interface would be to use it in the loadtxoutset RPC.

For an example, see the various uses of EnsureMining(node);

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Sjors for providing feedback and suggesting to add test coverage via the RPC...

Looking into it now and will update shortly

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a good way to add test coverage to this

refactored loadtxoutset to utilize the new snapshot interface...

@sedited
Copy link
Contributor

sedited commented Sep 8, 2025

Approach ACK

@D33r-Gee D33r-Gee force-pushed the interface-load-snapshot branch from 88b8e88 to 30d9bdb Compare September 15, 2025 16:28
@D33r-Gee
Copy link
Author

With 30d9bdb, the loadtxoutset RPC command has been refactored to use the snapshot interface, as suggested by @Sjors. Additionally, the interface now includes getMetadata and getActivationResult methods to ensure parity with the legacy RPC command.

@D33r-Gee D33r-Gee force-pushed the interface-load-snapshot branch from 30d9bdb to 94d68d1 Compare September 15, 2025 22:02
@D33r-Gee
Copy link
Author

with 94d68d1 RPC Error handling has been restored through the interface

… node interface

- Added a new virtual method `snapshot` to the node interface for loading UTXO snapshots.
- Updated the implementation in `interfaces.cpp` to activate the snapshot using a path.
- Added `snapshot.h` to src/interfaces to process the path and extract the metada
- Added `getMetadata` and `getActivationResults` methods for use by GUI and/or RPC
- Added error handling method and member variable

This change enhances the user ability to load utxo snapshots through the GUI and/or RPC
- Added a new signal for snapshot load progress in the UI interface.
- Updated the notifications interface to include a method for reporting snapshot loading progress.
- Enhanced the ChainstateManager to report progress during snapshot loading.
- Implemented the necessary handler in the kernel notifications to relay progress updates.

This change improves user feedback during the snapshot loading process.
@D33r-Gee D33r-Gee force-pushed the interface-load-snapshot branch from 668f98e to 36ae2ef Compare November 19, 2025 18:24
@D33r-Gee
Copy link
Author

with 36ae2ef rebased over master.

No changes to commits...

…pshot loading

- Updated the loadtxoutset function to use the new node interface for loading UTXO snapshots.
- Replaced direct file handling with snapshot activation through the node interface.
- Enhanced the result object to correctly reference metadata from the snapshot.

This change streamlines the UTXO snapshot loading process and leverages the new interface capabilities.
@D33r-Gee D33r-Gee force-pushed the interface-load-snapshot branch from 36ae2ef to d0c32c2 Compare November 19, 2025 23:22
@D33r-Gee
Copy link
Author

with d0c32c2 fixed RPC command issue that surfaced after rebase

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants