Skip to content

perf(ext): convert ext/cache, ext/canvas, ext/crypto JS sources to lazy-loaded scripts#33778

Merged
bartlomieju merged 1 commit into
mainfrom
perf/lazy-loaded-cache-canvas-crypto
May 3, 2026
Merged

perf(ext): convert ext/cache, ext/canvas, ext/crypto JS sources to lazy-loaded scripts#33778
bartlomieju merged 1 commit into
mainfrom
perf/lazy-loaded-cache-canvas-crypto

Conversation

@bartlomieju

Copy link
Copy Markdown
Member

Summary

Follow-up to #33760. Converts the ESM JS sources in ext/cache, ext/canvas, and ext/crypto to IIFE-wrapped lazy-loaded scripts (lazy_loaded_js), eliminating ESM module resolution overhead.

  • ext/canvas/02_surface.js - simple re-export from core.ops
  • ext/cache/01_cache.js - ext/fetch dependencies (Request, toInnerRequest, toInnerResponse, getHeader) bridged via internals since ext/fetch is still ESM
  • ext/crypto/00_crypto.js - kKeyObject symbol bridged via internals from ext/node; updated ext/node consumers to use core.loadExtScript()

ext/cron was skipped because 01_cron.ts is TypeScript which cannot be used with lazy_loaded_js (no transpilation for scripts).

Test plan

  • cargo check passes
  • cargo build --bin deno succeeds (snapshot builds)
  • tools/format.js and tools/lint.js --js pass
  • Smoke tested: crypto, crypto.subtle, CryptoKey, caches, CacheStorage, Deno.UnsafeWindowSurface (unstable-webgpu), node:crypto all work correctly
  • CI

@fibibot fibibot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Substance follows the same pattern I approved in #33760. The 12k-line diff is dominated by ext/crypto/00_crypto.js (~11.5k lines) where every line is replaced as the file goes from ESM module to IIFE script — almost-zero net (5684+/5634-). The actual semantic surface is small:

Substance walk

  • ext/crypto/lib.rs: esm = ["00_crypto.js"]lazy_loaded_js = ["00_crypto.js"]. Right shape, mirrors #33760.
  • ext/cache/01_cache.js: ext/fetch dependencies (Request, toInnerRequest, toInnerResponse, getHeader) bridged via internals since ext/fetch is still ESM. Correct workaround pattern — scripts can't import from other extensions, so cross-extension symbols flow through the internals namespace.
  • ext/crypto/00_crypto.js: same wrap. kKeyObject symbol bridge added at ext/node/polyfills/internal/crypto/constants.ts (publishes internals.kKeyObject = kKeyObject) so the now-script-form crypto can read it via internals.kKeyObject.
  • Consumer migration: ext/node/polyfills/crypto.ts:170 and internal/crypto/keys.ts:38 both switch import { ... } from "ext:deno_crypto/00_crypto.js"const { ... } = core.loadExtScript("ext:deno_crypto/00_crypto.js"). runtime/js/90_deno_ns.js does the same for ext:deno_canvas/02_surface.js. These are the exact paths that broke OffscreenCanvas in #33760 before the core.loadExtScript migration — encouraging that they're handled in this PR's initial diff.
  • ext/cron skipped: the body acknowledges 01_cron.ts is TypeScript and lazy_loaded_js doesn't transpile. Right scope decision.

Smaller observations

  • internals.kKeyObject couples crypto.js to node/constants.ts: this is a load-order-sensitive coupling. If node/internal/crypto/constants.ts doesn't run before 00_crypto.js's IIFE evaluates, internals.kKeyObject will be undefined. Worth confirming the bootstrap ordering — and worth a brief comment in constants.ts explaining why internals.kKeyObject = kKeyObject is there ("publishes for ext/crypto's lazy-loaded script") so a future reader doesn't think it's an internals leak to clean up.
  • The // @ts-check and triple-slash references at the top of 00_crypto.js are dropped in the conversion. The IIFE form can't use // @ts-check — that's expected — but it does mean the crypto JS no longer benefits from the TypeScript checker. Worth a tracking issue if this matters for ongoing crypto polyfill work.

CI

26 of 38 relevant test unit / test node_compat / wpt / lint debug shards SUCCESS at this head, with 12 still in flight (macos-aarch64 debug + linux-aarch64 debug + lint debug macos-x86_64 — the slower runners). No failures yet. Given the #33760 precedent had a V8 abort that surfaced only after the build/lint shards had passed, holding COMMENT until the remaining macOS / linux-aarch64 runs land green — particularly the test unit shards which exercised the OffscreenCanvas bug last time. Substance is fine from my side; if the in-flight shards come back clean, this is APPROVE-ready.

@fibibot fibibot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The force-pushed delta is the same lazy_loaded_js conversion I walked previously; substance still tracks the established #33760 / #33779 / #33780 pattern, and the CI signal that was in-flight on my prior pass has now landed clean across the platforms that mattered for the OffscreenCanvas-style hazard:

  • All 12 test unit_node shards pass — the canary for 00_crypto.js consumers like internal/crypto/keys.ts and crypto.ts that now go through core.loadExtScript.
  • All 12 test unit shards pass — Web Crypto + CacheStorage globals from 98_global_scope_shared.js.
  • All 36 test node_compat shards pass (3 shards × 6 platforms × 2 modes) — broadest signal that the internals.kKeyObject bridge resolves correctly across node:crypto entry paths.
  • 11 of 12 test specs shards pass. The lone failure is test specs (2/2) debug macos-aarch64 cancelled at 30m19s — the 30-minute timeout, not a substantive failure. specs (1/2) on the same host passed in 5m30s, and every other macos-aarch64 shard (build / lint / integration / unit / unit_node / node_compat) succeeded. Same flake pattern that hit #33780.

Load-order coupling

The one substance question I left open in my prior pass — does internals.kKeyObject resolve when 00_crypto.js IIFE evaluates from 98_global_scope_shared.js? — checks out by chasing the import graph:

  • 98_global_scope_shared.js ESM imports include import process from "node:process".
  • ext/node/polyfills/process.ts:50 imports node:assert.
  • ext/node/polyfills/assert.ts:21 imports internal/util/comparisons.ts.
  • ext/node/polyfills/internal/util/comparisons.ts:35 imports internal/crypto/constants.ts for kKeyObject.
  • internal/crypto/constants.ts (new in this PR) does internals.kKeyObject = kKeyObject; at module evaluation.

V8 module-evaluation contract: imports run before body. So by the time 98_global_scope_shared.js's body executes the core.loadExtScript("ext:deno_crypto/00_crypto.js") call, the entire process → assert → comparisons → constants chain has already evaluated and internals.kKeyObject is set. The destructure inside the crypto IIFE (const { kKeyObject } = internals;) gets the proper Symbol, not undefined.

The coupling is implicit but real, and worth the doc comment I flagged previously in constants.ts ("published for ext/crypto's lazy-loaded script"). If a future refactor drops the node:assert import chain from process.ts, kKeyObject silently becomes undefined and key[kKeyObject] writes go to property "undefined". Not a blocker for this PR; flagging for a follow-up doc tweak.

Pattern parity check

Bridge pattern matches the established convention from prior PRs in this series:

Bridge Set in Read in
internals.__fetchRequest 98_global_scope_shared.js body before loadExtScript("01_cache.js") 01_cache.js IIFE
internals.__fetchResponse same same
internals.__fetchHeaders same same
internals.kKeyObject internal/crypto/constants.ts (this PR) 00_crypto.js IIFE

The cache bridges are explicit at the call site (read order obvious). The crypto bridge is implicit through the eager-import chain (acceptable, documented above). All four use the internals.* namespace consistent with prior precedent.

Not blocking on the macOS specs flake — pre-existing infrastructure, not introduced by this PR.

bartlomieju added a commit that referenced this pull request May 3, 2026
Follow-up to #33760 and #33779. Converts all 8 ESM JS sources in
`ext/fetch` to IIFE-wrapped lazy-loaded scripts (`lazy_loaded_js`).

- `20_headers.js`, `21_formdata.js`, `22_body.js`, `22_http_client.js`,
`23_request.js`, `23_response.js`, `26_fetch.js`, `27_eventsource.js`
- `26_fetch.js` imports from `ext/telemetry` (TypeScript ESM, can't be
converted yet) -- bridged via `internals.__telemetry` /
`internals.__telemetryUtil` set in the telemetry source files
- All other cross-extension deps use `core.loadExtScript()` (ext/web,
ext/webidl, ext/net are already `lazy_loaded_js`)

This also eliminates the `internals.__fetchRequest/Response/Headers`
bridges from #33778 since `ext/cache` can now use `core.loadExtScript()`
for ext/fetch directly.
@bartlomieju
bartlomieju force-pushed the perf/lazy-loaded-cache-canvas-crypto branch from 24b1177 to ac5c672 Compare May 3, 2026 08:59
@bartlomieju
bartlomieju force-pushed the perf/lazy-loaded-cache-canvas-crypto branch from ac5c672 to 886dc19 Compare May 3, 2026 09:01

@fibibot fibibot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — re-affirming after the rebase. Substance is unchanged in shape but cleaner than the version I approved at 24b11772:

The internals.__fetchRequest / __fetchResponse / __fetchHeaders bridges I walked previously are gone. Now that #33784's ext/fetch lazy_loaded conversion has landed on main, 01_cache.js resolves its cross-extension dependencies directly via core.loadExtScript("ext:deno_fetch/23_request.js") etc., the same pattern every other lazy-loaded script uses. The bridge hack was always a workaround for ext/fetch still being ESM at the time; now there's no need.

What I re-verified at this head:

  • ext/cache/01_cache.js uses core.loadExtScript for Request/RequestPrototype/toInnerRequest, toInnerResponse, getHeader — direct calls, no internals.__* bridge. ✓
  • 98_global_scope_shared.js has zero internals.__fetch* writes — the bridge plumbing the prior version added has been excised. The body just calls core.loadExtScript for each dep. ✓
  • internals.kKeyObject bridge for ext/crypto → ext/node still in place, since ext:deno_node/internal/crypto/constants.ts is still ESM (not lazy_loaded). The same load-order analysis from my prior review still holds: node:processnode:assertinternal/util/comparisonsinternal/crypto/constants.ts runs as part of 98_global_scope_shared.js's ESM imports, before its body's core.loadExtScript("ext:deno_crypto/00_crypto.js") evaluates the IIFE. So internals.kKeyObject is set when crypto's IIFE reads it.
  • Same 11 files, same conversion shape, same JS consumers updated to core.loadExtScript (ext/node/polyfills/crypto.ts, internal/crypto/keys.ts, runtime/js/90_deno_ns.js).

CI

123 pass, 1 pending, 1 skip, 0 fail at this head. The pending shard is the slower macos-aarch64 specs runner — same flake-prone shard that hit my prior review's pass too. All other shards across linux/macos/windows × x86_64 × debug+release pass.

Nothing else to add — this is a strict improvement over the version I previously approved.

@bartlomieju
bartlomieju enabled auto-merge (squash) May 3, 2026 09:56
@bartlomieju
bartlomieju merged commit 9499507 into main May 3, 2026
260 of 262 checks passed
@bartlomieju
bartlomieju deleted the perf/lazy-loaded-cache-canvas-crypto branch May 3, 2026 10:04
bartlomieju added a commit that referenced this pull request May 4, 2026
… scripts (#33817)

- Convert `ext/process/40_process.js` from ESM to IIFE-wrapped
lazy-loaded script (`lazy_loaded_js`)
- Convert `ext/http/00_serve.ts`, `ext/http/01_http.js`,
`ext/http/02_websocket.ts` from ESM to IIFE-wrapped lazy-loaded scripts
- Update all consumers (`runtime/js/90_deno_ns.js`,
`runtime/js/99_main.js`, `ext/node/polyfills/child_process.ts`,
`ext/node/polyfills/internal/child_process.ts`) to use
`core.loadExtScript()` instead of ESM imports
- Replace `import.meta.log` calls with `internals.log` in `00_serve.ts`

Follows the same pattern established in #33778, #33780, #33784, #33800,
and #33801.
bartlomieju added a commit that referenced this pull request May 4, 2026
…ripts (#33818)

## Summary

- Convert `ext/kv/01_db.ts` from ESM to IIFE-wrapped lazy-loaded script
(`lazy_loaded_js`)
- Convert `ext/webgpu/00_init.js` from ESM to IIFE-wrapped lazy-loaded
script (`lazy_loaded_js`)
- Update all consumers (`runtime/js/90_deno_ns.js`,
`runtime/js/98_global_scope_shared.js`,
`runtime/js/98_global_scope_window.js`,
`runtime/js/98_global_scope_worker.js`) to use `core.loadExtScript()`
instead of ESM imports
- Replace `import.meta.log` call with `internals.log` in `01_db.ts`

Follows the same pattern established in #33778, #33780, #33784, #33800,
#33801, and #33817.
bartlomieju added a commit that referenced this pull request May 27, 2026
Resolve KV conflicts after main's lazy-loading refactor (#33778, #33818,
#33966) and host-object deserialize fix (#34380):

- ext/kv/lib.rs: switch `esm` to `lazy_loaded_esm`, drop generics, use
  `Box<dyn DynamicDbHandler>` (matching #33966).
- runtime/js/90_deno_ns.js: load 01_db.ts via `createLazyLoader` (ESM)
  instead of `loadExtScript` (lazy script).
- ext/kv/01_db.ts, impl/*.ts: convert relative imports to `ext:deno_kv/`
  specifiers (cannot-be-a-base scheme rejects relative URLs in lazy ESM).
- impl/sqlite_backend.ts, impl/remote_backend.ts: load now-lazy modules
  (`ext:deno_crypto`, `ext:deno_fetch`, `ext:deno_web/03_abort_signal`)
  via `core.loadExtScript` instead of static imports.
- impl/kv.ts: same for `06_streams.js`; add
  `deserializers: cloneableDeserializers` to KV value/queue payload
  deserialization (port of #34380).
littledivy pushed a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
…and#33784)

Follow-up to denoland#33760 and denoland#33779. Converts all 8 ESM JS sources in
`ext/fetch` to IIFE-wrapped lazy-loaded scripts (`lazy_loaded_js`).

- `20_headers.js`, `21_formdata.js`, `22_body.js`, `22_http_client.js`,
`23_request.js`, `23_response.js`, `26_fetch.js`, `27_eventsource.js`
- `26_fetch.js` imports from `ext/telemetry` (TypeScript ESM, can't be
converted yet) -- bridged via `internals.__telemetry` /
`internals.__telemetryUtil` set in the telemetry source files
- All other cross-extension deps use `core.loadExtScript()` (ext/web,
ext/webidl, ext/net are already `lazy_loaded_js`)

This also eliminates the `internals.__fetchRequest/Response/Headers`
bridges from denoland#33778 since `ext/cache` can now use `core.loadExtScript()`
for ext/fetch directly.
littledivy pushed a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
…zy-loaded scripts (denoland#33778)

Follow-up to denoland#33760. Converts the ESM JS sources in `ext/cache`,
`ext/canvas`, and `ext/crypto` to IIFE-wrapped lazy-loaded scripts
(`lazy_loaded_js`), eliminating ESM module resolution overhead.

- **ext/canvas/02_surface.js** - simple re-export from `core.ops`
- **ext/cache/01_cache.js** - ext/fetch dependencies (Request,
toInnerRequest, toInnerResponse, getHeader) bridged via `internals`
since ext/fetch is still ESM
- **ext/crypto/00_crypto.js** - `kKeyObject` symbol bridged via
`internals` from `ext/node`; updated `ext/node` consumers to use
`core.loadExtScript()`
littledivy pushed a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
… scripts (denoland#33817)

- Convert `ext/process/40_process.js` from ESM to IIFE-wrapped
lazy-loaded script (`lazy_loaded_js`)
- Convert `ext/http/00_serve.ts`, `ext/http/01_http.js`,
`ext/http/02_websocket.ts` from ESM to IIFE-wrapped lazy-loaded scripts
- Update all consumers (`runtime/js/90_deno_ns.js`,
`runtime/js/99_main.js`, `ext/node/polyfills/child_process.ts`,
`ext/node/polyfills/internal/child_process.ts`) to use
`core.loadExtScript()` instead of ESM imports
- Replace `import.meta.log` calls with `internals.log` in `00_serve.ts`

Follows the same pattern established in denoland#33778, denoland#33780, denoland#33784, denoland#33800,
and denoland#33801.
littledivy pushed a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
…ripts (denoland#33818)

## Summary

- Convert `ext/kv/01_db.ts` from ESM to IIFE-wrapped lazy-loaded script
(`lazy_loaded_js`)
- Convert `ext/webgpu/00_init.js` from ESM to IIFE-wrapped lazy-loaded
script (`lazy_loaded_js`)
- Update all consumers (`runtime/js/90_deno_ns.js`,
`runtime/js/98_global_scope_shared.js`,
`runtime/js/98_global_scope_window.js`,
`runtime/js/98_global_scope_worker.js`) to use `core.loadExtScript()`
instead of ESM imports
- Replace `import.meta.log` call with `internals.log` in `01_db.ts`

Follows the same pattern established in denoland#33778, denoland#33780, denoland#33784, denoland#33800,
denoland#33801, and denoland#33817.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants