Repository navigation
Discussion: Should Trio never use unbounded dependency pins? #1182
Description
Trio currently sets install_requires as follows:
Lines 81 to 91 in 1de9cae
Note that every one of these dependency declarations specifies no upper bound. (Most have no lower bound either, but let's not focus on that for now.)
When there is no upper bound specified, the following scenario is possible:
- Trio declares that any version, no matter how new, of dependency D is okay. Let's say the newest version of D is currently 19.1.0.
- All goes well for a while. Maybe D stays at the same version. Maybe it publishes some compatible minor-version-bump updates.
- At some point in the future, D publishes version 20.0.0, a backwards-incompatible version.
- All of a sudden, Trio users attempting to
pip install -U trioget a broken set of dependencies due to (1) above. Users must now wait for a new version of Trio to be published that either makes Trio compatible with D v20 or that pins Trio toD < 20, or users must manually declare aD < 20transitive dependency pin themselves, none of which is ideal.
Note that this concern is not just theoretical; I've seen libraries that were working perfectly fine one day become broken the next by the situation above, with no advance warning, and then have to scramble to push out a new release with a correct pin to un-break new installs.
I believe[1] that a library like Trio's goal for any given dependency is to match the widest range of expected-to-be-compatible versions, not just for versions currently available, but for versions that may come out in the future. To achieve this, each dependency declaration can be given an upper bound like D < [next major version, not yet released, and assumed to be incompatible until we've confirmed otherwise once it is released, at which point we can publish a new release of Trio with the updated pin]. Changing from assuming that the next major version of a dependency will be incompatible rather than compatible works much better for semver-compliant dependencies, where that should always be the case. For non-semver-compliant dependencies, the upper bound can be refined based on whatever their versioning strategy is, but in the meantime using this upper bound is probably better than using no upper bound.
As for Dependabot, based on https://dependabot.com/docs/config-file/#version_requirement_updates, it looks like version_requirement_updates=auto ("increase versions if an app, widen ranges if a library") is all that is needed for Dependabot to do the right thing in its PRs. I assume that's the default setting, but it doesn't say. We could add a .dependabot/config.yml to ensure this.
Look forward to hearing if this makes sense. If so, I'm happy to submit a PR that implements this.
Thanks for all the great work on Trio!
[1] I say "believe" rather than "know" because I haven't yet found an authoritative doc that tells Python packagers to do this. The closest I've found so far is https://www.python.org/dev/peps/pep-0592/ which contains, "the obvious solution... will break users who have followed best practices and pinned to a specific version of the project". If anyone knows of better documentation about this somewhere, please link to it here, and if not, does anyone know what the best way would be to get this documented better?
Activity
njsmith commentedon Aug 13, 2019
I think the reason you haven't found authoritative advice is that there is no obvious answer :-).
For cases where you're deploying the code – end-user applications and CI for libraries – then it's easy: if you can, you should pin so you know what you're deploying, and then use a tool like dependabot to keep your pins up to date. But for a library like Trio that gets deployed by lots of people in lots of configurations, it's not as clear.
There's two kinds of errors we could make. Our requirements could allow for an incompatible version to be installed, a "false negative". Then like you say:
Alternatively, if our requirements are too strict and rule out a version that actually would have worked fine – a "false positive" – then this can also break things. Users who need the new features / bugfixes / another package that requires the new version are stuck, and have to wait for a new version of Trio to be published. (And notably, they can't easily take matters into their own hands by explicitly pinning the version they want.)
For future versions, we don't know whether they'll be compatible or not, so we have to make a guess about whether to rule them out or not, and either way we risk being wrong and creating problems for our users.
The idea of semver is that maybe people publishing packages can tell us whether they broke our code. The problem with this is that they don't know what our code does :-). So they can also be wrong both ways: sometimes they'll guess that they broke our code (bump the major version number) when actually they didn't break it, and sometimes they'll guess that they didn't when actually they did. Sometimes this just happens by mistake, which is unavoidable. What makes it more tricky is that even if the dev knows what they did, the effect will be different on different users!
The big question is about percentages: sometimes when you make a change you can be pretty confident that it will break 99%+ of users, or that it will break 0% of users. In that case semver is easy and it tells your users something useful. But what if you have a change that breaks 5% of users? If you don't bump the major version, then that's wrong for 5% of users (tells them that things are fine when actually they're broken). Or if you do bump the major version, then that's wrong for 95% of users (tells them that things are broken when actually they're fine).
And: all of our dependencies are large-ish, widely-used libraries with careful, conservative maintainers, so basically all of their changes break either 0% or 5% of users. That means that semver is basically never useful to them. And AFAIK, none of them actually use semver.
Specifically, our dependencies that we don't maintain ourselves are:
In summary:
So for our situation, I don't think it would be useful to add speculative upper bounds on our requirements.
You might also enjoy this tweet and following conversation: https://twitter.com/vorpalsmith/status/1093346524367474688
m-u-xyz commentedon Aug 13, 2019
I have run into this situation with Home Assistant. They basically force an exact release of all dependencies. While that's not what we are discussing here, the sortedcontainers update would have broken us as easily as it did break hass when my add-on (or rather, one of its dependencies) required the newer version.
Their priorities are different, supportable end-user application vs. widely-used library, so freezing their deps does make sense for them. For Trio, not so much.
jab commentedon Aug 15, 2019
Coincidentally, that sortedcontainers release from a few months back actually broke CI in a library I maintain. You can read more about that in grantjenks/python-sortedcontainers#106 if you like, but good to hear you expect that neither sortedcontainers nor any of Trio's other dependencies will suddenly break Trio in a future release. Thanks for taking the time to write up your thoughts, @njsmith, and please feel free to close this if there's nothing further to discuss.
njsmith commentedon Aug 15, 2019
@jab Oh, I'm sure that sooner or later, some dependency will break Trio! Entropy is inexorable; you don't fix it, you manage it. And this definitely affects choices like what libraries we depend on, motivates the use of dependabot to make sure we notice breakage ASAP, etc. It's just that for the specific strategy of adding speculative pins to rule out future versions based on the version number, I don't think the tradeoffs end up being favorable for us.
Thanks for bringing it up though – this is definitely one of those areas where it's helpful to keep reviewing old strategies and considering new ones, and the tradeoffs here are complicated and worth discussing.