Skip to content

feat(node/tls): support CA certificate introspection and off-thread loading tests#33708

Merged
littledivy merged 4 commits into
denoland:mainfrom
divybot:claude/test-tls-off-thread-cert-loading
May 3, 2026
Merged

feat(node/tls): support CA certificate introspection and off-thread loading tests#33708
littledivy merged 4 commits into
denoland:mainfrom
divybot:claude/test-tls-off-thread-cert-loading

Conversation

@divybot

@divybot divybot commented Apr 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Enables test-tls-off-thread-cert-loading in node_compat suite.

Test plan

  • cargo test --test node_compat -- test-tls-off-thread-cert-loading

@divybot divybot changed the title implemented Node TLS CA-certificate support needed for the off-thread cert-loading cases, wired feat(node/tls): support CA certificate introspection and off-thread loading tests Apr 29, 2026
… cert-loading cases, wired

Enables tests/node_compat/runner/suite/test/parallel/test-tls-off-thread-cert-loading.js

Co-authored-by: Divy Srivastava <[email protected]>
@divybot
divybot force-pushed the claude/test-tls-off-thread-cert-loading branch from 00d5eec to 14d3245 Compare April 30, 2026 11:39

@fibibot fibibot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Substantial refactor — adds tls.getCACertificates(type) + per-type lazy caches, threads CA-store selection through child_process.fork env, and adds NODE_DEBUG_NATIVE=crypto "off-thread" tracing. Three concrete blockers to address before merge.

Blocker 1 — getCACertificates("bundled") trailing-newline regression

CI is failing on test-tls-get-ca-certificates-bundled.js across linux-aarch64, linux-x86_64, and windows-x86_64 (node_compat shards 2/3 and 3/3):

AssertionError: Expected "actual" to be reference-equal to "expected":
+     '-----END CERTIFICATE-----\n',
-     '-----END CERTIFICATE-----',

The change in ext/node/ops/tls.rs::cert_der_to_pem:

-  format!("-----BEGIN CERTIFICATE-----\n{pem_lines}\n-----END CERTIFICATE-----",)
+  format!("-----BEGIN CERTIFICATE-----\n{pem_lines}\n-----END CERTIFICATE-----\n")

added a trailing \n to every PEM block. tls.rootCertificates works around this by stripping it explicitly:

StringPrototypeReplace(v, new SafeRegExp("\\n$"), ""),

But tls.getCACertificates("bundled") (and "system") returns the raw op output, which still has the trailing \n. The test that passed against the previous format now fails.

Two ways out:

  1. Drop the trailing \n in cert_der_to_pem (matching the old Rust behavior). getPemCertificatesFromFile already adds \n back in the file path, so consistency would be: op returns no trailing \n, file-loader adds it. Then getCACertificates would need a trailing-\n-add for the file path or a strip for the op path — pick one canonical form.
  2. Apply the same trailing-\n strip in getBundledCertificates / getSystemCertificates that ensureLazyRootCertificates already applies.

Option (1) is cleaner — pick "no trailing newline" as the canonical form across all three sources (op, file, frozen export) and remove the \\n$ strip entirely.

Blocker 2 — Permission check removed from CA-cert ops

-pub fn op_get_root_certificates(
-  state: &mut OpState,
-) -> Result<Vec<String>, PermissionCheckError> {
-  state
-    .borrow_mut::<PermissionsContainer>()
-    .check_sys("ca", "node:tls.rootCertificates")?;
+pub fn op_get_root_certificates(state: &mut OpState) -> Vec<String> {

-pub fn op_get_ca_certificates<TSys: ExtNodeSys + 'static>(...) {
-  state.borrow_mut::<PermissionsContainer>().check_sys("ca", "...")?;
+pub fn op_node_get_ca_certificates(...) -> Result<Vec<String>, JsErrorBox> {
+  // no permission check

Both ops previously required --allow-sys=ca (check_sys("ca", ...)); now they're permissionless. This is a user-visible permission-model change for tls.rootCertificates and tls.getCACertificates.

The mitigations I'd want addressed before merging:

  • The extra path still calls readTextFileSync(NODE_EXTRA_CA_CERTS), which is gated by --allow-read. ✓
  • Bundled certs are public Mozilla data — sandbox impact: minimal.
  • System certs read via deno_native_certs::load_native_certs — this opens platform stores (e.g., /etc/ssl/certs/* on Linux, macOS keychain). Even though the data is mostly-public, exposing it to a sandboxed program previously required explicit --allow-sys=ca. Now it doesn't.

If the removal is intentional (say, because CA reading happens at startup before permissions are settable), document it in the PR description and call out the permission-model change explicitly. Otherwise restore check_sys("ca", ...) on both ops.

Blocker 3 — --use-openssl-ca translated to system store

let ca_stores = if use_openssl_ca {
  Some(vec!["system".to_string()])
} else { ... };

Node's --use-openssl-ca selects OpenSSL's built-in CA store, which is OpenSSL's compiled-in certs path (e.g., /etc/ssl/certs/ca-certificates.crt on Debian) — distinct from "system" in the platform-native sense (Windows certmgr, macOS Keychain). Conflating them via DENO_TLS_CA_STORE=system will cause silent behavior differences:

  • On Linux: probably fine since /etc/ssl/certs/... is what load_native_certs() reads anyway.
  • On Windows: --use-openssl-ca should not go to certmgr but to OpenSSL's compiled cert path (typically empty for prebuilt OpenSSL, requiring SSL_CERT_FILE env). Mapping to "system" makes it certmgr.
  • On macOS: similar — Keychain is platform, OpenSSL is its own bundled path.

If deno_native_certs already special-cases the OpenSSL paths via DENO_NODE_USE_OPENSSL_CA=1, document that. Otherwise this is a Node-compat divergence on at least Windows + macOS that won't fail any existing test but will produce wrong CA chains for users running with --use-openssl-ca on those platforms.

Smaller

Windows line endings in getPemCertificatesFromFile

StringPrototypeSplit(content, "-----END CERTIFICATE-----\n"),

Splits on \n only. A Windows-authored PEM file with \r\n line endings would leave a stray \r in each segment, and StringPrototypeTrim(line) === "" only catches the fully empty case — segments with \r would not be empty and would push <garbage>\r-----END CERTIFICATE-----\n into the certs array.

Either normalize line endings first (StringPrototypeReplace(content, new SafeRegExp("\\r\\n", "g"), "\n")) or split on a regex: new SafeRegExp("-----END CERTIFICATE-----\\r?\\n").

new SafeRegExp("\\n$") per-cert

ArrayPrototypePush(
  target,
  StringPrototypeReplace(v, new SafeRegExp("\\n$"), ""),
),

Allocates a fresh regex per iteration in the strip loop. Hoist to a module-level const TRAILING_NL = new SafeRegExp("\\n$") to avoid the per-cert allocation. (And ideally drop the strip entirely once Blocker 1 is fixed.)

op_get_env_no_permission_check for NODE_EXTRA_CA_CERTS

Pre-existing concern in this codebase (other no-permission env reads exist), but worth noting: reading NODE_EXTRA_CA_CERTS env without --allow-env=NODE_EXTRA_CA_CERTS is a permission-model softening for env access. The subsequent readTextFileSync is permission-gated, so the file content is still protected — but the existence and value of the env var is now leakable from sandboxed code.

getCACertificates(type: string = "default") validation order

if (type !== undefined) {
  validateString(type, "type");
}

switch (type) {
  case "default": ...
  default: throw new ERR_INVALID_ARG_VALUE("type", type);
}

validateOneOf got dropped; the switch's default branch handles unknown types, but the previous validateOneOf would throw ERR_INVALID_ARG_VALUE with a specific message about the allowed set. The new error is also ERR_INVALID_ARG_VALUE but without the allowed-set hint. Cosmetic — but worth confirming Node's exact error shape if a regression test depends on it.

Three new tests enrolled

test-tls-off-thread-cert-loading.js (the PR's stated target) plus -disabled.js and -system.js. Good coverage. The system variant exercises DENO_TLS_CA_STORE=system selection — verifies the env-thread-through path through child_process.fork.

Cargo dep added

deno_native_certs added to ext/node's dependencies. Already in the workspace via deno_tls; no version bump.

CI

4 fail (all test-tls-get-ca-certificates-bundled.js), 88 pass, 13 pending. Holding REQUEST_CHANGES primarily on the trailing-newline regression and the permission-check removal — the --use-openssl-ca conflation is at minimum a code-comment ask but likely also a behavior fix on Windows/macOS.

cc @bartlomieju on the permission-model change for the CA-cert ops — that's the user-facing piece worth a maintainer eye.

@fibibot fibibot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three blockers resolved cleanly. Walked the new commit:

Blocker 1 — Trailing-newline regression

-  format!("-----BEGIN CERTIFICATE-----\n{pem_lines}\n-----END CERTIFICATE-----\n")
+  format!("-----BEGIN CERTIFICATE-----\n{pem_lines}\n-----END CERTIFICATE-----")

Plus the file-loader path moved back to Rust (parse_extra_ca_certs), which uses the same canonical "no trailing \n" form. The ensureLazyRootCertificates strip-on-iteration is removed since the source no longer needs stripping — single canonical form across all three sources (op, file, frozen export). ✓

CI confirms: test-tls-get-ca-certificates-bundled.js no longer in the failure list. 0 failures, 54 pass, 13 pending.

Blocker 2 — Permission checks restored

pub fn op_get_root_certificates(state: &mut OpState) -> Result<Vec<String>, PermissionCheckError> {
  state.borrow_mut::<PermissionsContainer>()
    .check_sys("ca", "node:tls.rootCertificates")?;
  ...
}

pub fn op_node_get_ca_certificates<TSys: ExtNodeSys + 'static>(
  state: &mut OpState,
  #[string] kind: String,
) -> Result<Vec<String>, CaCertificatesError> {
  state.borrow_mut::<PermissionsContainer>()
    .check_sys("ca", "node:tls.getCACertificates()")?;
  ...
}

Both ops now require --allow-sys=ca again. ✓ User-visible permission model preserved.

Blocker 3 — --use-openssl-ca no longer conflated with system

-  let ca_stores = if use_openssl_ca {
-    Some(vec!["system".to_string()])
-  } else {
-    match (use_system_ca, use_bundled_ca) {
-      (true, true) => Some(vec!["system".to_string(), "mozilla".to_string()]),
-      ...
-    }
-  };
+  let ca_stores = match (use_system_ca, use_bundled_ca) {
+    (true, true) => Some(vec!["system".to_string(), "mozilla".to_string()]),
+    ...
+  };

The wrong Linux/macOS/Windows mapping is gone. use_openssl_ca still propagates via DENO_NODE_USE_OPENSSL_CA=1 env var (used by emitDefaultCertificatesDebug to suppress the off-thread debug logging), but no longer rewrites the cert source.

Open question: with this fix, --use-openssl-ca no longer changes cert selection — only debug logging. If Node's contract for --use-openssl-ca requires actually using OpenSSL's compiled-in cert path (rather than just suppressing logs), there's a follow-up to wire that through. Worth either:

  • A doc comment on the env var explaining "currently unused beyond debug suppression — follow-up to plumb to native OpenSSL store."
  • Or a TODO referencing the issue / spec link for what --use-openssl-ca should do.

Not blocking — the previous behavior (mapping to system) was strictly worse.

Smaller items now resolved

  • Windows line endings: parse_extra_ca_certs is back in Rust and uses cert.trim() which strips \r\n from segments. ✓
  • new SafeRegExp("\\n$") per-cert: removed (strip no longer needed). ✓
  • validateOneOf restored on getCACertificates: matches Node's error shape. ✓
  • cachedCACertificates restored as the per-type cache.
  • setDefaultCACertificates now correctly invalidates cachedCACertificates keys too — the previous version only cleared lazyRootCertificates, which would have left a stale cachedCACertificates["default"] returning pre-set certs. Subtle but real bug fix.

CI

54 pass, 13 pending, 0 fail. Up from 88 pass / 13 pending / 4 fail before the fix. The four test-tls-get-ca-certificates-bundled.js failures are gone.

Flipping verdict: holding to COMMENT until pending CI lands per the must-complete rule, but substance is now clean. Happy to flip to APPROVE once the remaining shards finish. The --use-openssl-ca-debug-only behavior is the one thing that warrants a comment / TODO / follow-up issue, but not a re-block.

@fibibot fibibot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Single-commit delta since my prior approval-shaped comment ("All three blockers resolved cleanly") at SHA 3af51f6: fix(node/tls): preserve extra CA PEM newlines — re-adds the trailing \n to certs returned by parse_extra_ca_certs (Some(format!("{trimmed}\n-----END CERTIFICATE-----\n"))) and converts a for/delete cache-flush loop to ArrayPrototypeForEach for the prefer-primordials lint. Substance walk for the new commit:

  • The trailing-newline addition is only on parse_extra_ca_certs (the "extra" source). cert_der_to_pem (used by "bundled" and "system") still emits no trailing \n. So getCACertificates("system") returns no-newline strings, getCACertificates("extra") returns trailing-newline strings, and getCACertificates("default") (which can mix both) returns a heterogeneous array. CI is green at this head (135 pass, 1 skip, 0 fail) so existing tests don't catch this mix — but it's worth confirming against Node's actual output format if the goal is full compat. Non-blocking.
  • The delete cachedCACertificates[key] rewrite to ArrayPrototypeForEach(ObjectKeys(...), key => delete ...) is a primordials-rule fix only, no behavior change.

What I verified is still right from the prior review pass

  • Permission checks on both ops (check_sys("ca", ...)) — preserved.
  • --use-openssl-ca no longer conflated with "system" in default cert resolution.
  • Per-type lazy cache (cachedCACertificates) flushed on setDefaultCACertificates so the next read recomputes against the new override.
  • child_process.fork correctly threads DENO_TLS_CA_STORE and the new DENO_NODE_USE_OPENSSL_CA env vars from parent's --use-openssl-ca flag into the child's env.

Note on DENO_NODE_USE_OPENSSL_CA

This is a new env var name introduced by this PR — used only as a parent→child propagation mechanism for --use-openssl-ca across child_process.fork. It's not exposed as user-facing public config (no flags.rs entry, no docs), but it IS a new surface name a user could theoretically read or set. Worth a maintainer's note on whether to formalize it (DENO_* namespace + flags.rs entry + docs) or rename to something more obviously-internal (e.g. DENO_INTERNAL_NODE_USE_OPENSSL_CA) so a future reader doesn't mistake it for public config. cc @bartlomieju.

Not blocking — the substance and CI are clean and the env var is functional as-is.

@littledivy littledivy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@littledivy
littledivy merged commit 09c6782 into denoland:main May 3, 2026
136 checks passed
littledivy added a commit that referenced this pull request May 3, 2026
…eps (#33798)

## Summary
Recent additions of `import` (rather than `core.loadExtScript`) for
`ext:deno_web/08_text_encoding.js` and `ext:deno_io/12_io.js` in:

- `ext/node/polyfills/tls.ts` (#33708)
- `ext/node/polyfills/internal/deps/undici/undici.js` (#33731)

make the snapshot build fail on `main`:

```
thread 'main' panicked at libs/core/runtime/jsruntime.rs:2564:9:
Failed to initialize JsRuntime for snapshotting: ... "Specifier
\"ext:deno_web/08_text_encoding.js\" was not passed as an extension
module and was not included in the snapshot."
```

Those modules aren't snapshot deps for ext/node — every other polyfill
that needs them loads via `core.loadExtScript`. Switch the two new
imports to that pattern, and replace `io.stderr.writeSync(new
TextEncoder().encode(...))` in `tls.ts` with the existing
`core.print(..., true)` helper already used in `process.ts`.

Verified locally with `cargo build -p deno_snapshots`.

Co-authored-by: Divy Srivastava <[email protected]>
littledivy added a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
…eps (denoland#33798)

## Summary
Recent additions of `import` (rather than `core.loadExtScript`) for
`ext:deno_web/08_text_encoding.js` and `ext:deno_io/12_io.js` in:

- `ext/node/polyfills/tls.ts` (denoland#33708)
- `ext/node/polyfills/internal/deps/undici/undici.js` (denoland#33731)

make the snapshot build fail on `main`:

```
thread 'main' panicked at libs/core/runtime/jsruntime.rs:2564:9:
Failed to initialize JsRuntime for snapshotting: ... "Specifier
\"ext:deno_web/08_text_encoding.js\" was not passed as an extension
module and was not included in the snapshot."
```

Those modules aren't snapshot deps for ext/node — every other polyfill
that needs them loads via `core.loadExtScript`. Switch the two new
imports to that pattern, and replace `io.stderr.writeSync(new
TextEncoder().encode(...))` in `tls.ts` with the existing
`core.print(..., true)` helper already used in `process.ts`.

Verified locally with `cargo build -p deno_snapshots`.

Co-authored-by: Divy Srivastava <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants