test: key plugin version folders by single-digit major#9052
Merged
Conversation
Key each expanded major as a bare major (`versions/<name>@3`) instead of a bounded range (`versions/<name>@>=3.0.0 <4.0.0`). The bare major is the readable folder name and covers each major's latest, including the floor major's, which the range form drops. `*` resolves to the latest major, empty entries throw, and non-consecutive packages (graphql, @redis/client) cover only their floor and top majors.
A bare-major key resolves to that major's newest published version, so a range that ends inside its top major overshoots: microgateway-core `>=2.1 <=3.0.0` keyed `3` installs 3.3.7, which the plugin never supported, so the span comes back `web.request` instead of `microgateway.request`. The top major now keeps the declared range whenever the range stops short of the major's ceiling, so it resolves to the newest version the range still allows; fully-spanned majors and every lower major stay bare single-digit keys.
Contributor
Overall package sizeSelf size: 6.32 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.2.0 | 104.26 kB | 843.44 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
🎉 All green!🧪 All tests passed 🔗 Commit SHA: 9bf46ed | Docs | Datadog PR Page | Give us feedback! |
BenchmarksBenchmark execution time: 2026-06-25 11:43:10 Comparing candidate commit 9bf46ed in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 1952 metrics, 13 unstable metrics.
|
Widening the matrix to every major surfaces versions that install fine but cannot run: `versions/ai@4` resolves to the newest 4.x, which has no VCR cassette, so the test would hit the live API. `brokenVersions` maps a module to the ranges to skip; `withVersions()` drops a matching resolved version and surfaces the reason as a pending test instead of a silent gap. Entries are stop-gaps carrying a TODO so a fixed version drops the entry.
A published manifest that carries a `workspace:` protocol dependency (some monorepo packages publish it raw) was copied verbatim into a generated workspace, so yarn failed with "Couldn't find any versions for X that matches workspace:*". The newly-covered langchain 0.x folders hit this. Fall back to the pinned compatible version instead of the unresolvable spec.
The widened matrix newly tests `@langchain/core@0` (the 0.x latest, 0.3.80), which has no VCR cassette, so the LLMObs langchain specs hit the live API and fail with 401. Same fixture gap as the ai 4.x latest; skip it with a TODO. The 0.1.0 floor and the 1.x latest keep their coverage.
The "should instrument fetch failure" test decides whether the execute span carries the error by comparing the `withVersions` folder key with `version > '2.3.0'`. That is a lexicographic string comparison: it holds while the key is `>=2.3.0`, but a bare-major key such as `2` makes `'2' > '2.3.0'` false, so the test takes the no-error branch even though @apollo/gateway 2.14.0 attaches the error to the execute span via `recordExceptions`. Compare the resolved version (the third `withVersions` argument) with `semver.gt` so the gate stays correct regardless of how the version folder is keyed. (cherry picked from commit fdfd2f5)
The bedrock plugin spec requires versions/@aws-sdk/client-bedrock-runtime@<key> with the `>=3.422.0` key; the single-digit keying renamed that folder to `@3`. Its `before` had already run `agent.load`, so the missing-module throw left the tracer and its Remote Config poll running with no `agent.close`, and the job hung to the 45-minute timeout instead of failing fast (on master the `@>=3.422.0` folder still exists, so it never threw). The LLMObs bedrock spec got this fix; this is the matching APM spec the aws-sdk job actually runs.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 049a756568
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Single-digit version keying renames the folder this spec hard-codes from `express@>=5.0.0` to `express@5`. `existsSync` then returns false for the stale path, so the express 4/5 path-to-regexp dialect suite is silently skipped in the `PLUGINS=express` matrix even though the express 5 workspace is installed.
bengl
approved these changes
Jun 26, 2026
This was referenced Jun 26, 2026
Closed
Merged
Merged
rochdev
pushed a commit
that referenced
this pull request
Jul 2, 2026
Key each expanded major by its bare major (`versions/mongodb@3`) instead of a bounded range (`versions/mongodb@>=3.0.0 <4.0.0`). The bare major reads cleanly as a folder name and covers each major's latest, including the floor major's, which the range form dropped. Follows the shared resolver from #9019. Widening the matrix to every major's latest surfaced several latent failures: 1. A bare-major key resolves to that major's newest version, so a range ending inside its top major overshoots: microgateway-core `>=2.1 <=3.0.0` keyed `3` installed 3.3.7 and the span came back `web.request` instead of `microgateway.request`. The top major keeps the declared range whenever it stops short of the major's ceiling; fully-spanned and lower majors stay bare. 2. `versions/ai@4` and `@langchain/core@0` resolve to versions that have no VCR cassette and would hit the live API (401). A central `brokenVersions` registry drops a matching resolved version and surfaces the reason as a pending test, each entry a stop-gap carrying a TODO. 3. A manifest carrying a `workspace:` protocol dependency was copied verbatim into a generated workspace, so yarn failed with "Couldn't find any versions for X that matches workspace:*". Fall back to the pinned compatible version. 4. The Apollo fetch-failure test gated the error span on `version > '2.3.0'`, a lexicographic compare that breaks once the key is bare (`'2' > '2.3.0'` is false). Compare the resolved version with `semver.gt`. 5. Single-digit keying renames folders that several specs hard-code by range (express, langchain, bedrock runtime, aws-sdk). The bedrock require threw after `agent.load` with no `agent.close`, leaving the Remote Config poll running and hanging the job to the 45-minute timeout; the others silently skipped suites. Point the requires at the renamed folders.
rochdev
pushed a commit
that referenced
this pull request
Jul 2, 2026
Key each expanded major by its bare major (`versions/mongodb@3`) instead of a bounded range (`versions/mongodb@>=3.0.0 <4.0.0`). The bare major reads cleanly as a folder name and covers each major's latest, including the floor major's, which the range form dropped. Follows the shared resolver from #9019. Widening the matrix to every major's latest surfaced several latent failures: 1. A bare-major key resolves to that major's newest version, so a range ending inside its top major overshoots: microgateway-core `>=2.1 <=3.0.0` keyed `3` installed 3.3.7 and the span came back `web.request` instead of `microgateway.request`. The top major keeps the declared range whenever it stops short of the major's ceiling; fully-spanned and lower majors stay bare. 2. `versions/ai@4` and `@langchain/core@0` resolve to versions that have no VCR cassette and would hit the live API (401). A central `brokenVersions` registry drops a matching resolved version and surfaces the reason as a pending test, each entry a stop-gap carrying a TODO. 3. A manifest carrying a `workspace:` protocol dependency was copied verbatim into a generated workspace, so yarn failed with "Couldn't find any versions for X that matches workspace:*". Fall back to the pinned compatible version. 4. The Apollo fetch-failure test gated the error span on `version > '2.3.0'`, a lexicographic compare that breaks once the key is bare (`'2' > '2.3.0'` is false). Compare the resolved version with `semver.gt`. 5. Single-digit keying renames folders that several specs hard-code by range (express, langchain, bedrock runtime, aws-sdk). The bedrock require threw after `agent.load` with no `agent.close`, leaving the Remote Config poll running and hanging the job to the 45-minute timeout; the others silently skipped suites. Point the requires at the renamed folders.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Key plugin version folders by the bare major again (
versions/mongodb@3) ratherthan a bounded range (
versions/mongodb@>=3.0.0 <4.0.0). The bare major readscleanly as a folder name and covers each major's latest, including the floor
major's, which the range form dropped. Follows the shared resolver that landed
in #9019.
Bare keys regressed capped ranges in the original attempt: a key like
3resolves to the major's newest version, so
microgateway-core >=2.1 <=3.0.0installed 3.3.7 and the span came back
web.requestinstead ofmicrogateway.request. The top major now keeps the declared range whenever therange stops short of the major's ceiling (
>=2.1 <=3.0.0,>=4.0.0 <4.3.0),resolving to the newest version the range allows; fully-spanned majors and every
lower major stay bare single-digit keys.
Test plan
yarn test:scriptsruns the resolver unit spec, including the capped-top andsub-major cases.
majors not previously installed (e.g. knex 0.x). Holding for the APM
Integrations matrix to confirm green and to triage any version it surfaces.