Skip to content

Manually revert VarInt range check#6182

Merged
apoelstra merged 1 commit into
rust-bitcoin:0.32.xfrom
tcharding:push-knpvuzntvtys
May 24, 2026
Merged

Manually revert VarInt range check#6182
apoelstra merged 1 commit into
rust-bitcoin:0.32.xfrom
tcharding:push-knpvuzntvtys

Conversation

@tcharding

@tcharding tcharding commented May 14, 2026

Copy link
Copy Markdown
Member

Manually revert 91a6e64

Note that we cannot break the API so the new stuff introduced in 0.32.9 has to stay in there. Comment the code as such.

This was a backport of #5697 which added range checking to the VarInt type. On master we did this then realized there are legitimate use cases for the full range of a u64. We then added u64 support to compact size (#5784).

NB: 'compact size' is the new name for varint.

This backport then broke downstream users.

Close: #6138

@github-actions github-actions Bot added the C-bitcoin PRs modifying the bitcoin crate label May 14, 2026
@tcharding tcharding marked this pull request as draft May 14, 2026 04:53
@tcharding tcharding force-pushed the push-knpvuzntvtys branch 2 times, most recently from 6e895ad to c01802b Compare May 14, 2026 05:01
@tcharding tcharding marked this pull request as ready for review May 14, 2026 05:01
@tcharding tcharding force-pushed the push-knpvuzntvtys branch from c01802b to cc4cc0c Compare May 14, 2026 05:05
@tcharding

tcharding commented May 14, 2026

Copy link
Copy Markdown
Member Author

We have another option here. We could break the API (revert), release 0.32.10 then yank 0.32.9.

@bigspider

Copy link
Copy Markdown
Contributor

We have another option here. We could break the API (revert), release 0.32.10 then yank 0.32.9.

I personally think this would indeed be preferable, for the reasons I wrote here:

However, here is why I think that a revert would be acceptable in this case:

  • It's 0.32.9 that made a breaking change in the contract of VarInt. The fact that the code still compiles, to me, makes that worse: in the crate I'm maintaining, it was only discovered in the CI. Had there be no tests covering it, it could very easily reach a release and only be discovered with a production panic. In fact, that could still happen on software wallets depending on the ledger_bitcoin_client if they update rust-bitcoin to 0.32.9 but not ledger_bitcoin_client.
  • Reverting the commit restores to the behavior of 0.32.8, so there is no breaking change unless you're relying on the behavior introduced in 0.32.9. Therefore, I think that not compiling is in fact better in this case, precisely to avoid the kind of runtime bugs that a backwards-compatible fix might cause.

@tcharding

Copy link
Copy Markdown
Member Author

@apoelstra whats your thoughts on this? If we want to revert and re-release I'm inclined to think that we should create a 0.32.10 that is only 0.32.9 and this patch. Then release it and yank 0.32.9. Logiistically that would mean either rebasing 0.32.x (force pushing to public branch is a but racey though) or creating another branch to release 0.32.10 from. Do we merge this patch into 0.32.x branch in its current state (ie on top of whats already unreleased on that branch) and then continue releasing from this branch later on?

@apoelstra

Copy link
Copy Markdown
Member

No, yanking only makes sense if there is a security issue or a bug in a particular version of a crate. It does not give us license to release API-breaking changes in a point release. Only a serious security issue would let us do that.

Comment thread bitcoin/src/consensus/encode.rs Outdated
/// VarInt was encoded in a non-minimal way.
NonMinimalVarInt,
/// VarInt value exceeds the maximum allowed size.
// DO NOT USE. Introduced in 0.32.9, reverted in 0.32.10

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In cc4cc0c:

We should update the public doccomment to indicate that this is unused and has no meaning.

/// the size is encoded as a CompactSize.
///
/// WARNING: This is never used. We introduced range checking in `0.32.9` breaking downstream, it
/// was reverted in `0.32.10` but the public API has to stay.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In cc4cc0c:

This is a run-on sentence.

@tcharding tcharding force-pushed the push-knpvuzntvtys branch from cc4cc0c to a37f76e Compare May 17, 2026 22:56
@tcharding

Copy link
Copy Markdown
Member Author

Fixed and fixed.

@tcharding

tcharding commented May 17, 2026

Copy link
Copy Markdown
Member Author

I think this is going to break a fuzz test somewhere between 0.32.x and master but I'm not exactly sure what or where the fix goes. Just flagging it so we don't forget.

EDIT: Oh no it won't because we don't fuzz types from 0.32 that do not exist on master, VarInt being one such type.

@tcharding

Copy link
Copy Markdown
Member Author

No, yanking only makes sense if there is a security issue or a bug in a particular version of a crate. It does not give us license to release API-breaking changes in a point release. Only a serious security issue would let us do that.

I don't really care but I thought we were accepting that we introduced a bug in 0.32.9 and this PR fixes that bug but because the fix is a breaking change we are leaving ugly as hell code in there. So it seems reasonable to yank 0.32.9 and re-release with just this patch (but without the ugliness in it). Or am I on crack?

@apoelstra

Copy link
Copy Markdown
Member

but because the fix is a breaking change we are leaving ugly as hell code in there.

Yes.

So it seems reasonable to yank 0.32.9 and re-release with just this patch (but without the ugliness in it).

No, removing the ugliness would be a breaking change.

@apoelstra

Copy link
Copy Markdown
Member

Yanking a version or not is a completely separate question from whether we should release breaking changes in a minor release.

@tcharding

Copy link
Copy Markdown
Member Author

I'm not sure exactly where the miscommunication is or confusion.

  1. 0.32.8
  2. 0.32.9 - Introduces Foo
  3. 0.32.10 - Removes Foo (breaking as of 0.32.9 but if that is yanked then non-breaking as of 0.32.8)

@apoelstra

Copy link
Copy Markdown
Member

It's still breaking wrt 0.32.9. Yanking 0.32.9 doesn't change that. It just prevents cargo users from adding a new dependency on 0.32.9.

@tcharding

Copy link
Copy Markdown
Member Author

Oh thanks, I misunderstood yanking. I didn't realize that yank did not remove it from the registry.

This is good to go then. Force push is rebase only.

    
Manually revert 91a6e64

Note that we cannot break the API so the new stuff introduced in
`0.32.9` has to stay in there. Comment the code as such.
    
This was a backport of rust-bitcoin#5697 which added range checking to the
`VarInt` type. On master we did this then realized there are
legitimate use cases for the full range of a `u64`. We then added
`u64` support to compact size (rust-bitcoin#5784).

NB: 'compact size' is the new name for varint.
    
This backport then broke downstream users.
@tcharding tcharding force-pushed the push-knpvuzntvtys branch from a37f76e to 3bcd7c9 Compare May 23, 2026 03:53
@apoelstra

Copy link
Copy Markdown
Member

If we'd caught this within an hour or two of publishing, I'd be fine treating yank as "removes from the registry".

But at 2+ weeks, especially when this latest release contained a bunch of bugfixes, I'd expect it to be pretty widely deployed (though would anybody be using the API that we'd be breaking? kinda doubtful honestly).

@apoelstra apoelstra left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 3bcd7c9; successfully ran local tests

@apoelstra apoelstra merged commit 67b9fbb into rust-bitcoin:0.32.x May 24, 2026
17 of 18 checks passed
@tcharding tcharding deleted the push-knpvuzntvtys branch May 24, 2026 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C-bitcoin PRs modifying the bitcoin crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants