Skip to content

fix(core): optimize bun lockfile parser#33623

Merged
leosvelperez merged 3 commits intonrwl:masterfrom
berenddeboer:feat/improve-bun-parser
Dec 4, 2025
Merged

fix(core): optimize bun lockfile parser#33623
leosvelperez merged 3 commits intonrwl:masterfrom
berenddeboer:feat/improve-bun-parser

Conversation

@berenddeboer
Copy link
Copy Markdown
Contributor

@berenddeboer berenddeboer commented Nov 26, 2025

Key optimisations:

  1. Pre-computed PackageIndex - Built once during lockfile parsing:
    • byName: Map from package name → array of versions (O(1) lookup)
    • workspaceNames: Set of workspace package names (O(1) lookup)
    • workspacePaths: Set of workspace paths (O(1) lookup)
    • packagesWithWorkspaceVariants: Set of packages with workspace-specific variants (O(1) lookup)
    • patchedPackages: Set of patched package names (O(1) lookup)
  2. findResolvedVersion: Changed from O(n) scan through all packages to O(1) map lookup + O(k) where k = number of versions for that package (typically 1-3)
  3. isWorkspacePackage: Changed from O(n) scan to O(1) set lookup
  4. hasWorkspaceSpecificVariant: Changed from O(n) scan to O(1) set lookup
  5. isNestedPackageKey: Now uses pre-computed workspace paths/names sets instead of computing them each call

On my 40 project typescript monorepo my time goes from about 30 seconds to 4.5s, a speed-up of 6-7x.

Key Optimizations Made

1. Pre-computed PackageIndex - Built once during lockfile parsing:
   - byName: Map from package name → array of versions (O(1) lookup)
   - workspaceNames: Set of workspace package names (O(1) lookup)
   - workspacePaths: Set of workspace paths (O(1) lookup)
   - packagesWithWorkspaceVariants: Set of packages with workspace-specific variants
(O(1) lookup)
   - patchedPackages: Set of patched package names (O(1) lookup)

2. findResolvedVersion: Changed from O(n) scan through all packages to O(1) map
lookup + O(k) where k = number of versions for that package (typically 1-3)

3. isWorkspacePackage: Changed from O(n) scan to O(1) set lookup

4. hasWorkspaceSpecificVariant: Changed from O(n) scan to O(1) set lookup

5. isNestedPackageKey: Now uses pre-computed workspace paths/names sets instead of
computing them each call
@berenddeboer berenddeboer requested review from a team and meeroslav as code owners November 26, 2025 05:26
@netlify
Copy link
Copy Markdown

netlify bot commented Nov 26, 2025

👷 Deploy request for nx-docs pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 4594bdb

@vercel
Copy link
Copy Markdown

vercel bot commented Nov 26, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Updated (UTC)
nx-dev Ready Ready Preview Dec 4, 2025 8:20am

@berenddeboer berenddeboer changed the title fix(nx): optimize bun lockfile parser with O(1) lookups fix(nx): optimize bun lockfile parser with O(1) lookups: 7x better Nov 26, 2025
const prefix = packageKey.substring(0, lastSlash);

// If the prefix is a workspace path or scoped package pattern
if (workspacePaths.has(prefix) || prefix.startsWith('@')) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing condition in workspace variant detection. The old logic checked prefix.startsWith('@') && prefix.includes('/') but the new code only checks prefix.startsWith('@'). This could incorrectly identify packages as workspace-specific variants.

For example, if a package key is split where the prefix is just @scope (without a slash), it would incorrectly match, whereas the old code would reject it.

Fix:

if (workspacePaths.has(prefix) || (prefix.startsWith('@') && prefix.includes('/'))) {
  packagesWithWorkspaceVariants.add(possiblePackageName);
}
Suggested change
if (workspacePaths.has(prefix) || prefix.startsWith('@')) {
if (workspacePaths.has(prefix) || (prefix.startsWith('@') && prefix.includes('/'))) {

Spotted by Graphite Agent

Fix in Graphite


Is this helpful? React 👍 or 👎 to let us know.

We fixed two related bugs in the workspace variant detection logic:

Bug 1: buildPackageIndex - Missing condition in workspace variant detection

Location: packages/nx/src/plugins/js/lock-file/bun-parser.ts:492

Problem: The check prefix.startsWith('@') was missing the additional condition
prefix.includes('/'). This could incorrectly identify packages as workspace-specific
variants when the prefix is just @scope (without a slash).

Bug 2: isNestedPackageKey - Scoped packages incorrectly filtered as nested

Location: packages/nx/src/plugins/js/lock-file/bun-parser.ts:1200-1205

Problem: The function's final return true would incorrectly filter legitimate scoped
packages like @types/react or scoped aliases like @custom/lodash as nested packages.

Fix: Added a check before the final return true to recognize simple scoped packages
(2-part keys starting with @).
@berenddeboer berenddeboer changed the title fix(nx): optimize bun lockfile parser with O(1) lookups: 7x better fix(nx): optimize bun lockfile parser with O(1) lookups: 7x faster Nov 27, 2025
@yharaskrik
Copy link
Copy Markdown
Contributor

🥺

@berenddeboer berenddeboer changed the title fix(nx): optimize bun lockfile parser with O(1) lookups: 7x faster fix(nx): optimize bun lockfile parser with O(1) lookups: 30s to 4s Nov 30, 2025
@nx-cloud
Copy link
Copy Markdown
Contributor

nx-cloud bot commented Dec 4, 2025

View your CI Pipeline Execution ↗ for commit 4594bdb

Command Status Duration Result
nx affected --targets=lint,test,test-kt,build,e... ✅ Succeeded 18m 22s View ↗
nx run-many -t check-imports check-lock-files c... ✅ Succeeded 3m 2s View ↗
nx-cloud record -- nx-cloud conformance:check ✅ Succeeded 11s View ↗
nx-cloud record -- nx format:check ✅ Succeeded 2s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded <1s View ↗

☁️ Nx Cloud last updated this comment at 2025-12-04 10:38:23 UTC

@leosvelperez leosvelperez self-assigned this Dec 4, 2025
@leosvelperez leosvelperez changed the title fix(nx): optimize bun lockfile parser with O(1) lookups: 30s to 4s fix(core): optimize bun lockfile parser Dec 4, 2025
nx-cloud[bot]

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor

@nx-cloud nx-cloud bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nx Cloud has identified a flaky task in your failed CI:

Since the failure was identified as flaky, the solution is to rerun CI. Because this branch comes from a fork, it is not possible for us to push directly, but you can rerun by pushing an empty commit:

git commit --allow-empty -m "chore: trigger rerun"
git push

Nx Cloud View detailed reasoning in Nx Cloud ↗


🎓 Learn more about Self-Healing CI on nx.dev

Copy link
Copy Markdown
Member

@leosvelperez leosvelperez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

Thanks for the contribution!

@leosvelperez leosvelperez merged commit 8352d40 into nrwl:master Dec 4, 2025
17 of 20 checks passed
barbados-clemens pushed a commit that referenced this pull request Dec 8, 2025
Key optimisations:

1. Pre-computed PackageIndex - Built once during lockfile parsing:
   - byName: Map from package name → array of versions (O(1) lookup)
   - workspaceNames: Set of workspace package names (O(1) lookup)
   - workspacePaths: Set of workspace paths (O(1) lookup)
- packagesWithWorkspaceVariants: Set of packages with workspace-specific
variants (O(1) lookup)
   - patchedPackages: Set of patched package names (O(1) lookup)
2. findResolvedVersion: Changed from O(n) scan through all packages to
O(1) map lookup + O(k) where k = number of versions for that package
(typically 1-3)
3. isWorkspacePackage: Changed from O(n) scan to O(1) set lookup
4. hasWorkspaceSpecificVariant: Changed from O(n) scan to O(1) set
lookup
5. isNestedPackageKey: Now uses pre-computed workspace paths/names sets
instead of computing them each call

On my 40 project typescript monorepo my time goes from about 30 seconds
to 4.5s, a speed-up of 6-7x.

---------

Co-authored-by: Leosvel Pérez Espinosa <[email protected]>
@github-actions
Copy link
Copy Markdown
Contributor

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 10, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants