docs: update upgrading.md to highlight how breaking change releases will be handled #3218
Conversation
…be handled going forward by using Semver as well as a categorization system identifying the breaking change scope and upgrade effort.
jasonbahl
added
the
scope: docs
jasonbahl
requested review from
chriszarate,
josephfusco,
justlevine and
moonmeister
moonmeister
left a comment
moonmeister
left a comment
There was a problem hiding this comment.
Really love this. Seems like a great way to unblock some big changes while maintaining stability and clear comms.
One question I have is around backwards compat, deprecation of old things, and beta features (I don't have the right words so might have to describe somethings in too much detail here). Let's say we want to rewrite how X works. Do we simply rewrite X with a new API and release it under a major version? Or do release it under betaX and minor version? Then the next major gets a the new version of X? Or alternatively the next major still has X but it's marked as deprecated, but there's also newX (you can also still use a beta system). then eventually the deprecated API gets dropped on the next MAJOR. This is the idea of "major versions should only remove functionality not add it". If you can't tell I'm a fan of this final method. It's also possible this could be used in relation to impact/effort...meaning some minimal effort items can go straight to major but significant/high impact things go through a staged approach.
However we choose to handle these things, It could be beneficial to clarify not just what versions mean and how we're supporting them, but our overall approach to how we release features in combination with this info.
|
@moonmeister ya, that's a good point. There are a lot of features I think we'll be able to start shipping in a non-breaking way giving folks a time period to opt-in to the change before it becomes default. For example, Custom Scalars are a good candidate for this. Let's say we wanted to add a new We could introduce the Scalars to the codebase, but not actually have any fields resolve to those Scalars unless a user opts-in to that new behavior (i.e. feature flag / experiment). So we can incrementally work on supporting Custom Scalars without any impact on current users, but folks can test by opting-in to Custom Scalar support on either a global basis or a field-by-field basis or some mix. Then, at some point we would flip the switch and custom scalars would be default behavior for fields like For some features, this pattern will work very well. For some features, we might not always be able to use this type of Pattern. For example, one feature that will likely be a breaking change is changing |
|
Yup. I think the key take away is having a cross-over period whenever possible. Where both new and old work. This allows teams(esp. large ones) to do incremental migration whenever possible. Or at least give folks time to migrate while still getting the latest and greatest in other areas. |
justlevine
left a comment
justlevine
left a comment
There was a problem hiding this comment.
Love it! Left some inline feedback.
More generally:
- I think we need to define what's included by the support policy. Is that answering questions in the forums, or patching bugs or maintaining wp compat across multiple versions? Either way, we need to make sure we have the capacity to honor what's promised.
- I think the doc should explicitly call out security releases and how they're handled. Regardless of how "Support Policy" gets defined, I think we should commit that security issues discovered/patched after a breaking
/releasebranch is cut are shipped as a patch and not bundled in the future breaking release. - Release cycles are a fuzzy metric. Whenever possible we should include a minimum days count as well/instead.
As far as the release cycle/cadence goes:
- I think we should commit to a deprecation cycle. It should specify:
- That whenever possible we will deprecate before we break.
- That a deprecation will occur X days/cycles before the break is implemented.
- Similarly, I think we need an additional triage step for when to warrant a breaking change. E.g (some combination of)
- Enough minimum time has passed since the deprecation or no deprecation is possible.
- There's no justifiable way to implement (a subset of) the changes in a non-breaking way. "Justifiable" includes the additional work required.
- There's already a pending breaking release and no "justifiable" reason to parcel it off into a separate breaking release.
- It scores well enough on the
Impact | Effortscale to "justify" that future nonbreaking changes will be gated behind migrating to this breaking version.
| ## Categorizing Breaking Changes | ||
| Breaking changes will be categorized based on two dimensions: **Impact Scope** and **Effort to Upgrade**. | ||
|
|
||
| - **Impact Scope**: | ||
| - Low: Affects few users (niche use cases or specific extensions). | ||
| - Medium: Affects a moderate number of users or common use cases. | ||
| - High: Affects most or all users interacting with the API. | ||
|
|
||
| - **Effort to Upgrade**: | ||
| - Minimal: Requires little to no action to remain compatible. | ||
| - Moderate: Requires small adjustments (e.g., changing function signatures). | ||
| - Significant: Requires refactoring large sections of code or schema. | ||
|
|
||
| Each breaking change will be classified using these dimensions (e.g., "High Impact, Minimal Effort"). | ||
|
|
There was a problem hiding this comment.
Personally I struggle with "subjective" buckets, but I don't have a better alternative.
One factor that I think should be explicitly accounted for is whether the change is to the PHP API (and therefore requires changes to plugins/custom code) vs a change to consumption APIs (schema is the obvious one, but also for example return codes or new headers). Don't think it slides neatly into an Impact | Effort scale, but IMO provides significant more value to people upgrading.
There was a problem hiding this comment.
Ya, it's tough for sure to know the actual level of effort and scope. . .but I think we can usually make decent judgements and users can use that as a baseline to evaluate their own situation.
I do think it makes sense to include another categorization here, as you called out. PHP API vs. Schema / consumption APIs.
I'll think on how to categorize this, but yes, I think that would be beneficial to formalize as it's important information for users to understand when choosing to update a major version.
docs/upgrading.md
Outdated
| WPGraphQL will adopt an approach of **smaller, more frequent breaking changes**. This means: | ||
| - **Incremental Changes**: Breaking changes will be released in smaller increments, allowing users to upgrade more easily and manage the impact with minimal disruption. | ||
| - **Clear Communication**: Each breaking change will be categorized by **Impact Scope** and **Effort to Upgrade**, enabling users to make informed decisions about when to upgrade. | ||
| - **Limited Version Support**: WPGraphQL will support only the last 3 major versions to reduce fragmentation, ensuring that users have clear expectations regarding version support. |
There was a problem hiding this comment.
3 major versions is a huge commitment, we only have 1 major version right now. I cannot envision how we'd absorb the amount of extra work involved in backporting a security fix across 3 breaking versions.
There was a problem hiding this comment.
I was thinking that with the kind of approaches I was suggesting of adding features without breaking changes and then deprecating old systems you could probably reduce the need for supported major versions to 2. Also, just a thought, LTS? e.g. take on the node model.
There was a problem hiding this comment.
@justlevine ya, but I've also not really been shipping breaking changes.
I think realistically if we ship some smaller breaking changes slightly more often, we'll need to support a handful of versions, at least for some period of time. Maybe instead of number of releases it could be period of time.
i.e. something to the tune of "we will support the latest major version and the most prior major version for a period of 6 months" or something to that tune? 🤔
I'm not sure what the answer should be, but I do know there will be a period of time where folks will be split on older versions as they get their projects updated, etc.
There was a problem hiding this comment.
Definitely need to think through this topic a bit more. 🤔
There was a problem hiding this comment.
I was thinking that with the kind of approaches I was suggesting of adding features without breaking changes and then deprecating old systems you could probably reduce the need for supported major versions to 2
Some features are going to be impossible to add in a non-breaking way. . .like updating graphql-php to >15 which drops support for PHP versions below 7.4.
I think we need to do this (upgrade the underlying GraphQL PHP) sooner than later as graphql-php 15 is now pushing 2 years old.
Many other features that might be breaking I do think can be done in more incremental fashion.
Perhaps we prioritize this update and any communication around this update to get to a 2.0, then work on any other potential breaking changes post-2.0 in a more incremental fashion as described, where we can introduce new features that might not be active until you opt-in, and then at a later date they become the defaults.
There was a problem hiding this comment.
Ok, I've updated to state latest 2 versions instead of latest 3. I want to encourage folks to stay updated with the latest and greatest, but I know there will be some periods of fragmentation as folks allocate time to get updated.
Some breaking changes will likely be quick updates for the masses, while some may take more time, and I don't want to fully fragment the community by not offering some support for other versions, but also we don't have a massive team working on this to provide support for a matrix of versions.
There was a problem hiding this comment.
@jasonbahl per my primary feedback, I think first and foremost you need to define what "support" actually means.
There was a problem hiding this comment.
@justlevine how's something to the tune of:
## Support Policy
WPGraphQL will support the **last 2 major versions** to reduce version fragmentation. This means that older versions beyond this window will no longer receive official support, and users will be encouraged to upgrade to the latest version.
### What Does "Support" Mean?
"Support" includes:
- **Bug Fixes**: Active work will be done to resolve bugs and issues reported on supported versions.
- **Security Updates**: General and critical security fixes will be backported to the last 2 major versions, ensuring that these versions remain secure.
- **Compatibility**: Supported versions will be tested against newer versions of WordPress, PHP, and common extensions to ensure continued functionality.
- **Documentation**: Currently our documentation on wpgraphql.com is only setup to provide documentation for the latest version of the plugin. We _may_ explore initiatives to publish documentation for multiple versions.
### Unsupported Versions:
Versions older than the last 2 major versions will no longer receive:
- Bug fixes
- Security updates
- Compatibility testing or guarantees with new versions of WordPress, PHP, or third-party extensions
While older versions may still function, the quality and speed of support will be higher for those using the latest versions.
There was a problem hiding this comment.
Style-wise, love it.
- I'd make it explicit that it's the latest
minor.patchthat's supported for each major, and not everymajor.minor.
Regarding the policy itself:
- the
bug fixesmake me really nervous and that's only considering future bugs over the breaking changes we already know we need (e.g. GraphQL 15, lazy descriptions, custom scalars, uniform hooks), not the confirmed ones already in the tracker. - Similarly, compatibility testing with WordPress we can do, but I don't think it's feasible for PHP or "common extensions".
Ultimately, you know best what resources you have at your disposal, just want to make sure you're committing with eyes open.
There was a problem hiding this comment.
Similarly, compatibility testing with WordPress we can do, but I don't think it's feasible for PHP or "common extensions".
Right now our testing matrix tests new releases against various versions of WordPress and PHP. We don't do testing against common extensions, so perhaps I should remove that. In theory I/we do support common extensions, i.e. if you open an issue or discussion in Github and you're using something like WPGraphQL and Polylang, we will typically help you with your issue.
My intent here was to say that if you're lingering on an old version of WPGraphQL (for whatever reason) we likely won't be supporting your use case with a common extension in the same way we would support you if you were on the latest version.
|
Adding on to @jasonbahl 's response to @moonmeister above: The Experiments API ( #3098 ) will be a way to gate "experimental changes" to gather feedback whether something is mature enough to ship to production. That shouldn't be conflated with a "beta period" preceding a breaking release used to increase and ensure release stability, which is what this document is discussing. We shouldn't ship anything "half baked" to a release ;-). |
|
Good call out. I think a good summary of my original post is: do we want to specify in this doc how changes are rolled out, not just the way they're communicated or does that vary so much with the nature of the breaking change that it must be handled on a case-by-case basis. |
Co-authored-by: Dovid Levine <[email protected]>
Co-authored-by: Dovid Levine <[email protected]>
|
Alright, I've incorporated a lot of feedback from the discussion above. I'm going to merge the document. Feel free to open any PRs or Issues to follow up. Open to feedback, but want to keep moving. Thanks! 🙏🏻 |
Co-authored-by: Dovid Levine <[email protected]>
justlevine
left a comment
justlevine
left a comment
There was a problem hiding this comment.
Looks good! Other than my desire to have "support" clearly defined, I have no other suggestions 🚀
Co-authored-by: Dovid Levine <[email protected]>
|
Code Climate has analyzed commit e7007da and detected 0 issues on this pull request. View more on Code Climate. |
@justlevine check it: #3218 (comment) |
…callout some important tidbits
…3-breaking-change-policy # Conflicts: # docs/upgrading.md
What does this implement/fix? Explain your changes.
This updates the docs at
wpgraphql.com/docs/upgradingto include more details about how breaking changes will be handled going forward.Open to feedback and adjustments (now or in the future) to the content on the page / processes outlined on the page.
Does this close any currently open issues?
closes #3173