Skip to content

Conversation

@l0rinc
Copy link
Contributor

@l0rinc l0rinc commented Dec 9, 2024

The getblockstats RPC currently overestimates UTXO overhead by treating the fCoinBase bitfield as an additional bool in PER_UTXO_OVERHEAD.
However, fCoinBase and nHeight are stored as bitfields and effectively packed into a single 32-bit value; counting an extra bool in the overhead calculation is unnecessary.

This PR introduces the following changes across three commits:

  • Store fCoinBase as a bool bitfield to reduce implicit conversions at call sites.
  • Use a consistent height/coinbase packing style across Coin serialization, undo serialization, and coinstats hashing (casting nHeight to uint32_t before shifting to avoid signed-promotion UB).
  • Adjust UTXO overhead estimation to match the actual Coin layout and update the related tests accordingly.

@DrahtBot
Copy link
Contributor

DrahtBot commented Dec 9, 2024

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31449.

Reviews

See the guideline for information on the review process.

Type Reviewers
Stale ACK Raimo33

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

No conflicts as of last run.

@l0rinc l0rinc force-pushed the l0rinc/coin-fCoinBase-bool branch from 889b86e to 72ec388 Compare December 9, 2024 18:55
@DrahtBot DrahtBot removed the CI failed label Dec 9, 2024
@l0rinc l0rinc marked this pull request as ready for review December 9, 2024 20:19
// outpoint (needed for the utxo index) + nHeight + fCoinBase
static constexpr size_t PER_UTXO_OVERHEAD = sizeof(COutPoint) + sizeof(uint32_t) + sizeof(bool);
// outpoint (needed for the utxo index) + nHeight|fCoinBase
static constexpr size_t PER_UTXO_OVERHEAD = sizeof(COutPoint) + sizeof(uint32_t);
Copy link
Member

@maflcko maflcko Sep 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't the serialized format using varint compression (or leveldb compression) anyway, so any hardcoded overhead will be wrong here anyway?

I'd say it is fine to leave this as-is, because both this pull and current master are equally "wrong" and in the presence of doubt, it seems better to not change the code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, both are over-estimating, but this is closer to the actual functioning, the previous one explicitly adds the bool which isn't separate.
We can also just fix it by not making this a constexpr but a function of the actual values, but I'd argue the current versions is still better.

@Raimo33
Copy link
Contributor

Raimo33 commented Sep 7, 2025

Code review ACK 61cde55

last commit conflicts with #33184 but I'd argue this PR could be merged first.
My current knowledge prevents me from understanding the changes in test/functional/data/rpc_getblockstats.json and test/functional/rpc_getblockstats.py. Also it seems 433942e could be split up further

The coinbase flag is semantically boolean but was stored as an unsigned bitfield. Store it as a `bool : 1` bitfield to better reflect intent and avoid relying on implicit integral conversions at call sites.

Update users to prefer `Coin::IsCoinBase()` and to pass explicit `true`/`false` values where appropriate.
@l0rinc l0rinc force-pushed the l0rinc/coin-fCoinBase-bool branch from 433942e to cae3b55 Compare January 1, 2026 22:13
@l0rinc
Copy link
Contributor Author

l0rinc commented Jan 1, 2026

l0rinc added 2 commits January 1, 2026 23:21
Serialize `Coin` metadata using the canonical (height << 1) | coinbase packing across `Coin` serialization, undo records, and coinstats hashing.

Cast the 31-bit `nHeight` bitfield to `uint32_t` before shifting to avoid signed promotion undefined behaviour.
`Coin` packs height and the coinbase flag into a single 32-bit value, so `utxo_size_inc(_actual)` should not count an additional boolean.

Update the calculation and adjust the `rpc_getblockstats` test expectations.
@l0rinc l0rinc force-pushed the l0rinc/coin-fCoinBase-bool branch from cae3b55 to 0096d01 Compare January 1, 2026 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants