[8.19] Implement PKI HTTP/2 functional tests (#274000) - #286876
Conversation
Adds Scout-based UI PKI login tests, using http2. Copilot drafted a bogus test suite using the FTR, so I (@legrego) took control of this PR to create what I actually wanted. Resolves elastic#267535 --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: Larry Gregory <[email protected]> Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit 1f1bea5) # Conflicts: # x-pack/platform/plugins/shared/security/tsconfig.json
kibanamachine
left a comment
There was a problem hiding this comment.
Libra found 1 issue.
|
|
||
| export const servers: ScoutServerConfig = { | ||
| ...pkiConfig, | ||
| http2: true, |
There was a problem hiding this comment.
This backport enables HTTP/2 via a configuration field that does not exist in the 8.19 Scout implementation. ScoutServerConfig has no http2 member, and there is no Scout code in this branch that consumes or configures it, so this object fails excess-property type checking and cannot switch Kibana from the inherited HTTP URL to the HTTPS/HTTP2 endpoint used by both new specs. The dependent Scout HTTP/2 support (including TLS server argument and URL rewriting) must be backported as well, or the TLS setup must be expressed using the configuration mechanisms available on 8.19.
In this branch, src/platform/packages/shared/kbn-scout/src/types/server_config.d.ts defines ScoutServerConfig without an http2 property, and the only occurrence of http2 under the Scout source is this line. Meanwhile pkiConfig.servers.kibana still inherits protocol: 'http', while the tests navigate to https://localhost:5620.
💔 Build Failed
Failed CI StepsMetrics [docs]
History |
…uction (#285153) (#286791) # Backport This will backport the following commits from `main` to `8.19`: - [[Security] Fix PKI session invalidation on HTTP/2 stream destruction (#285153)](#285153) <!--- Backport version: 11.0.2 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Jeramy Soucy","email":"[email protected]"},"sourceCommit":{"committedDate":"2026-08-24T09:23:24Z","message":"[Security] Fix PKI session invalidation on HTTP/2 stream destruction (#285153)\n\n## Summary\n\nFixes #258232.\n\nPKI sessions were being unexpectedly invalidated on Kibana 9.x when\n`server.protocol: http2` was active (the new default when `ssl.enabled:\ntrue`). Setting `server.protocol: http1` was the customer workaround.\n\n### Root cause\n\nWhen an HTTP/2 stream is destroyed mid-request — by an `AbortController`\ncancel, browser navigation abort, or any other cause that sends\n`RST_STREAM` — Node.js clears the stream's session reference\n(`stream.session = undefined`). The `Http2ServerRequest.socket` proxy\nthen falls back from the TLS socket to the stream itself, causing every\n`instanceof TLSSocket` check in `KibanaSocket` to return `false`. The\nresult:\n\n- `socket.authorized` → `undefined` (not `true` or `false`)\n- `socket.getPeerCertificate()` → `null`\n\nThe guard in `authenticateViaState` (pki.ts line ~208) was written for\nexactly this \"connection closed\" case:\n\n```ts\n// Before (buggy):\nif (peerCertificate === null && request.socket.authorized) {\n // return non-401 soft fail — session preserved\n}\n```\n\nOn HTTP/1.1 a closed `TLSSocket` keeps `authorized === true`, so the\nguard fired correctly. On HTTP/2, `authorized === undefined` is falsy,\nso the guard was silently skipped. The code then fell through to the\ntoken-invalidation block, revoked the ES access token, and returned\n`notHandled()` — which `authenticate()` converted to\n`Boom.unauthorized()`. `updateSessionValue()` saw the 401 on an owned\nsession and **deleted the server-side session**, logging the user out.\n\n### Fix\n\nChange the guard from a truthy check to `!== false`:\n\n```ts\n// After (fixed):\nif (peerCertificate === null && request.socket.authorized !== false) {\n // return non-401 soft fail — session preserved\n}\n```\n\n`false` = cert was presented but rejected → token invalidation is\ncorrect.\n`undefined` = socket state is unknowable (HTTP/2 stream destroyed) →\ntreat same as `true`, preserve session.\n`true` = socket explicitly authorized → original behaviour unchanged.\n\nAlso improves the diagnostic log in `authenticateViaPeerCertificate` to\ndistinguish `false` (rejected cert) from `undefined` (unknown state),\nmaking production logs actionable.\n\n### Why this is new in 9.x\n\n`http_config.ts` defaults `server.protocol` to `http2` when\n`ssl.enabled: true` — a silent breaking change from 8.x. HTTP/1.1 is\nunaffected because each request has a dedicated `TLSSocket` that keeps\n`authorized === true` as a plain property after the connection closes.\n\n## Test plan\n\n- [x] New regression test in `pki.test.ts`: `does not invalidate token\nor destroy session when socket state is unknown due to HTTP/2 stream\ndestruction` — fails on `main`, passes after fix\n- [x] New test in `pki.test.ts`: `does not handle requests when socket\nauthorization state is unknown due to HTTP/2 stream destruction` —\ncovers `login` and session-less `authenticate`, verifies correct log\nmessage\n- [x] All 40 existing PKI unit tests continue to pass\n\nRun tests:\n```\nnode scripts/jest --config x-pack/platform/plugins/shared/security/jest.config.js --testPathPattern=\"authentication/providers/pki\" --no-coverage\n```\n\n🤖 Generated with [Claude Code](https://claude.com/claude-code)\n\n---\n\n## FTR stress test (regression coverage for RST_STREAM)\n\n`pki.test.ts` unit tests are the authoritative regression lock — they're\ndeterministic and always catch a revert. An additional FTR test is\nincluded to provide end-to-end coverage of the exact failure mode:\n\n\n**`x-pack/platform/test/security_api_integration/tests/pki/pki_http2_stress.ts`**\nRegistered in `pki.http2.stress.config.ts` and added to\n`ftr_platform_stateful_configs.yml`.\n\n### Why supertest can't cover this\n\nThe existing `pki_auth.ts` concurrent-request test (5 streams) uses\nsupertest, which speaks HTTP/1.1. RST_STREAM is an HTTP/2-only signal —\nsupertest has no way to emit it. That is why the bug wasn't caught by\nthe existing concurrent-auth test.\n\n### What the FTR test does\n\n1. Establishes a PKI session via supertest (HTTP/1.1 — simple,\nreliable).\n2. Opens a raw `http2.connect()` connection to Kibana with the same PKI\nclient certificate.\n3. Fires 20 concurrent slow requests against the test endpoints plugin\n(10s hold, giving Kibana time to enter the async session-lookup window).\n4. After 150ms, cancels all 20 streams with `RST_STREAM NGHTTP2_CANCEL`.\n5. Sends a normal (non-cancelled) `GET /internal/security/me` on the\nsame HTTP/2 session.\n6. Asserts `200` + `username === 'first_client'` — the session was NOT\ninvalidated.\n\nBefore the fix: at least one RST_STREAM would destroy the stream during\nthe auth lifecycle, causing `authorized → undefined`, which bypassed the\nguard, invalidated the ES token, and returned 401.\nAfter the fix: all RST_STREAM frames are handled gracefully; the session\nsurvives.\n\n### Stability note\n\nAfter the fix is applied, the test is unconditionally stable — the guard\nchange means RST_STREAM never reaches the token-invalidation path\nregardless of timing. The test cannot produce a false positive (failing\nwhen the fix is present). Before the fix it might occasionally miss the\nnarrow timing window (false negative), but this is acceptable for a\nregression test: it cannot regress silently if the fix is reverted,\nbecause the unit test in `pki.test.ts` will always catch it.\n\n## Release note\nFixes an issue where PKI authentication sessions were unexpectedly\ninvalidated when in-flight requests were cancelled over HTTP/2, logging\nusers out. This affects deployments on server.protocol: http2, which\nbecame the default in 9.0.0 when TLS is enabled.\n\n---------\n\nCo-authored-by: Claude Sonnet 4.6 <[email protected]>\nCo-authored-by: kibanamachine <[email protected]>\nCo-authored-by: Larry Gregory <[email protected]>","sha":"a0520264b3fca9841b7e85f9c4b1996eb038b60e","branchLabelMapping":{"^v9.6.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:Security","backport:all-open","reviewer:scout","v9.6.0"],"title":"[Security] Fix PKI session invalidation on HTTP/2 stream destruction","number":285153,"url":"https://github.com/elastic/kibana/pull/285153","mergeCommit":{"message":"[Security] Fix PKI session invalidation on HTTP/2 stream destruction (#285153)\n\n## Summary\n\nFixes #258232.\n\nPKI sessions were being unexpectedly invalidated on Kibana 9.x when\n`server.protocol: http2` was active (the new default when `ssl.enabled:\ntrue`). Setting `server.protocol: http1` was the customer workaround.\n\n### Root cause\n\nWhen an HTTP/2 stream is destroyed mid-request — by an `AbortController`\ncancel, browser navigation abort, or any other cause that sends\n`RST_STREAM` — Node.js clears the stream's session reference\n(`stream.session = undefined`). The `Http2ServerRequest.socket` proxy\nthen falls back from the TLS socket to the stream itself, causing every\n`instanceof TLSSocket` check in `KibanaSocket` to return `false`. The\nresult:\n\n- `socket.authorized` → `undefined` (not `true` or `false`)\n- `socket.getPeerCertificate()` → `null`\n\nThe guard in `authenticateViaState` (pki.ts line ~208) was written for\nexactly this \"connection closed\" case:\n\n```ts\n// Before (buggy):\nif (peerCertificate === null && request.socket.authorized) {\n // return non-401 soft fail — session preserved\n}\n```\n\nOn HTTP/1.1 a closed `TLSSocket` keeps `authorized === true`, so the\nguard fired correctly. On HTTP/2, `authorized === undefined` is falsy,\nso the guard was silently skipped. The code then fell through to the\ntoken-invalidation block, revoked the ES access token, and returned\n`notHandled()` — which `authenticate()` converted to\n`Boom.unauthorized()`. `updateSessionValue()` saw the 401 on an owned\nsession and **deleted the server-side session**, logging the user out.\n\n### Fix\n\nChange the guard from a truthy check to `!== false`:\n\n```ts\n// After (fixed):\nif (peerCertificate === null && request.socket.authorized !== false) {\n // return non-401 soft fail — session preserved\n}\n```\n\n`false` = cert was presented but rejected → token invalidation is\ncorrect.\n`undefined` = socket state is unknowable (HTTP/2 stream destroyed) →\ntreat same as `true`, preserve session.\n`true` = socket explicitly authorized → original behaviour unchanged.\n\nAlso improves the diagnostic log in `authenticateViaPeerCertificate` to\ndistinguish `false` (rejected cert) from `undefined` (unknown state),\nmaking production logs actionable.\n\n### Why this is new in 9.x\n\n`http_config.ts` defaults `server.protocol` to `http2` when\n`ssl.enabled: true` — a silent breaking change from 8.x. HTTP/1.1 is\nunaffected because each request has a dedicated `TLSSocket` that keeps\n`authorized === true` as a plain property after the connection closes.\n\n## Test plan\n\n- [x] New regression test in `pki.test.ts`: `does not invalidate token\nor destroy session when socket state is unknown due to HTTP/2 stream\ndestruction` — fails on `main`, passes after fix\n- [x] New test in `pki.test.ts`: `does not handle requests when socket\nauthorization state is unknown due to HTTP/2 stream destruction` —\ncovers `login` and session-less `authenticate`, verifies correct log\nmessage\n- [x] All 40 existing PKI unit tests continue to pass\n\nRun tests:\n```\nnode scripts/jest --config x-pack/platform/plugins/shared/security/jest.config.js --testPathPattern=\"authentication/providers/pki\" --no-coverage\n```\n\n🤖 Generated with [Claude Code](https://claude.com/claude-code)\n\n---\n\n## FTR stress test (regression coverage for RST_STREAM)\n\n`pki.test.ts` unit tests are the authoritative regression lock — they're\ndeterministic and always catch a revert. An additional FTR test is\nincluded to provide end-to-end coverage of the exact failure mode:\n\n\n**`x-pack/platform/test/security_api_integration/tests/pki/pki_http2_stress.ts`**\nRegistered in `pki.http2.stress.config.ts` and added to\n`ftr_platform_stateful_configs.yml`.\n\n### Why supertest can't cover this\n\nThe existing `pki_auth.ts` concurrent-request test (5 streams) uses\nsupertest, which speaks HTTP/1.1. RST_STREAM is an HTTP/2-only signal —\nsupertest has no way to emit it. That is why the bug wasn't caught by\nthe existing concurrent-auth test.\n\n### What the FTR test does\n\n1. Establishes a PKI session via supertest (HTTP/1.1 — simple,\nreliable).\n2. Opens a raw `http2.connect()` connection to Kibana with the same PKI\nclient certificate.\n3. Fires 20 concurrent slow requests against the test endpoints plugin\n(10s hold, giving Kibana time to enter the async session-lookup window).\n4. After 150ms, cancels all 20 streams with `RST_STREAM NGHTTP2_CANCEL`.\n5. Sends a normal (non-cancelled) `GET /internal/security/me` on the\nsame HTTP/2 session.\n6. Asserts `200` + `username === 'first_client'` — the session was NOT\ninvalidated.\n\nBefore the fix: at least one RST_STREAM would destroy the stream during\nthe auth lifecycle, causing `authorized → undefined`, which bypassed the\nguard, invalidated the ES token, and returned 401.\nAfter the fix: all RST_STREAM frames are handled gracefully; the session\nsurvives.\n\n### Stability note\n\nAfter the fix is applied, the test is unconditionally stable — the guard\nchange means RST_STREAM never reaches the token-invalidation path\nregardless of timing. The test cannot produce a false positive (failing\nwhen the fix is present). Before the fix it might occasionally miss the\nnarrow timing window (false negative), but this is acceptable for a\nregression test: it cannot regress silently if the fix is reverted,\nbecause the unit test in `pki.test.ts` will always catch it.\n\n## Release note\nFixes an issue where PKI authentication sessions were unexpectedly\ninvalidated when in-flight requests were cancelled over HTTP/2, logging\nusers out. This affects deployments on server.protocol: http2, which\nbecame the default in 9.0.0 when TLS is enabled.\n\n---------\n\nCo-authored-by: Claude Sonnet 4.6 <[email protected]>\nCo-authored-by: kibanamachine <[email protected]>\nCo-authored-by: Larry Gregory <[email protected]>","sha":"a0520264b3fca9841b7e85f9c4b1996eb038b60e"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.6.0","branchLabelMappingKey":"^v9.6.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/285153","number":285153,"mergeCommit":{"message":"[Security] Fix PKI session invalidation on HTTP/2 stream destruction (#285153)\n\n## Summary\n\nFixes #258232.\n\nPKI sessions were being unexpectedly invalidated on Kibana 9.x when\n`server.protocol: http2` was active (the new default when `ssl.enabled:\ntrue`). Setting `server.protocol: http1` was the customer workaround.\n\n### Root cause\n\nWhen an HTTP/2 stream is destroyed mid-request — by an `AbortController`\ncancel, browser navigation abort, or any other cause that sends\n`RST_STREAM` — Node.js clears the stream's session reference\n(`stream.session = undefined`). The `Http2ServerRequest.socket` proxy\nthen falls back from the TLS socket to the stream itself, causing every\n`instanceof TLSSocket` check in `KibanaSocket` to return `false`. The\nresult:\n\n- `socket.authorized` → `undefined` (not `true` or `false`)\n- `socket.getPeerCertificate()` → `null`\n\nThe guard in `authenticateViaState` (pki.ts line ~208) was written for\nexactly this \"connection closed\" case:\n\n```ts\n// Before (buggy):\nif (peerCertificate === null && request.socket.authorized) {\n // return non-401 soft fail — session preserved\n}\n```\n\nOn HTTP/1.1 a closed `TLSSocket` keeps `authorized === true`, so the\nguard fired correctly. On HTTP/2, `authorized === undefined` is falsy,\nso the guard was silently skipped. The code then fell through to the\ntoken-invalidation block, revoked the ES access token, and returned\n`notHandled()` — which `authenticate()` converted to\n`Boom.unauthorized()`. `updateSessionValue()` saw the 401 on an owned\nsession and **deleted the server-side session**, logging the user out.\n\n### Fix\n\nChange the guard from a truthy check to `!== false`:\n\n```ts\n// After (fixed):\nif (peerCertificate === null && request.socket.authorized !== false) {\n // return non-401 soft fail — session preserved\n}\n```\n\n`false` = cert was presented but rejected → token invalidation is\ncorrect.\n`undefined` = socket state is unknowable (HTTP/2 stream destroyed) →\ntreat same as `true`, preserve session.\n`true` = socket explicitly authorized → original behaviour unchanged.\n\nAlso improves the diagnostic log in `authenticateViaPeerCertificate` to\ndistinguish `false` (rejected cert) from `undefined` (unknown state),\nmaking production logs actionable.\n\n### Why this is new in 9.x\n\n`http_config.ts` defaults `server.protocol` to `http2` when\n`ssl.enabled: true` — a silent breaking change from 8.x. HTTP/1.1 is\nunaffected because each request has a dedicated `TLSSocket` that keeps\n`authorized === true` as a plain property after the connection closes.\n\n## Test plan\n\n- [x] New regression test in `pki.test.ts`: `does not invalidate token\nor destroy session when socket state is unknown due to HTTP/2 stream\ndestruction` — fails on `main`, passes after fix\n- [x] New test in `pki.test.ts`: `does not handle requests when socket\nauthorization state is unknown due to HTTP/2 stream destruction` —\ncovers `login` and session-less `authenticate`, verifies correct log\nmessage\n- [x] All 40 existing PKI unit tests continue to pass\n\nRun tests:\n```\nnode scripts/jest --config x-pack/platform/plugins/shared/security/jest.config.js --testPathPattern=\"authentication/providers/pki\" --no-coverage\n```\n\n🤖 Generated with [Claude Code](https://claude.com/claude-code)\n\n---\n\n## FTR stress test (regression coverage for RST_STREAM)\n\n`pki.test.ts` unit tests are the authoritative regression lock — they're\ndeterministic and always catch a revert. An additional FTR test is\nincluded to provide end-to-end coverage of the exact failure mode:\n\n\n**`x-pack/platform/test/security_api_integration/tests/pki/pki_http2_stress.ts`**\nRegistered in `pki.http2.stress.config.ts` and added to\n`ftr_platform_stateful_configs.yml`.\n\n### Why supertest can't cover this\n\nThe existing `pki_auth.ts` concurrent-request test (5 streams) uses\nsupertest, which speaks HTTP/1.1. RST_STREAM is an HTTP/2-only signal —\nsupertest has no way to emit it. That is why the bug wasn't caught by\nthe existing concurrent-auth test.\n\n### What the FTR test does\n\n1. Establishes a PKI session via supertest (HTTP/1.1 — simple,\nreliable).\n2. Opens a raw `http2.connect()` connection to Kibana with the same PKI\nclient certificate.\n3. Fires 20 concurrent slow requests against the test endpoints plugin\n(10s hold, giving Kibana time to enter the async session-lookup window).\n4. After 150ms, cancels all 20 streams with `RST_STREAM NGHTTP2_CANCEL`.\n5. Sends a normal (non-cancelled) `GET /internal/security/me` on the\nsame HTTP/2 session.\n6. Asserts `200` + `username === 'first_client'` — the session was NOT\ninvalidated.\n\nBefore the fix: at least one RST_STREAM would destroy the stream during\nthe auth lifecycle, causing `authorized → undefined`, which bypassed the\nguard, invalidated the ES token, and returned 401.\nAfter the fix: all RST_STREAM frames are handled gracefully; the session\nsurvives.\n\n### Stability note\n\nAfter the fix is applied, the test is unconditionally stable — the guard\nchange means RST_STREAM never reaches the token-invalidation path\nregardless of timing. The test cannot produce a false positive (failing\nwhen the fix is present). Before the fix it might occasionally miss the\nnarrow timing window (false negative), but this is acceptable for a\nregression test: it cannot regress silently if the fix is reverted,\nbecause the unit test in `pki.test.ts` will always catch it.\n\n## Release note\nFixes an issue where PKI authentication sessions were unexpectedly\ninvalidated when in-flight requests were cancelled over HTTP/2, logging\nusers out. This affects deployments on server.protocol: http2, which\nbecame the default in 9.0.0 when TLS is enabled.\n\n---------\n\nCo-authored-by: Claude Sonnet 4.6 <[email protected]>\nCo-authored-by: kibanamachine <[email protected]>\nCo-authored-by: Larry Gregory <[email protected]>","sha":"a0520264b3fca9841b7e85f9c4b1996eb038b60e"}},{"url":"https://github.com/elastic/kibana/pull/286786","number":286786,"branch":"9.4","state":"OPEN"},{"url":"https://github.com/elastic/kibana/pull/286787","number":286787,"branch":"9.5","state":"OPEN"}]}] BACKPORT--> --- ## 8.19 backport adjustments ### Scout test suite replaced with FTR equivalent On `main`, the regression suite lives in `x-pack/platform/plugins/shared/security/test/scout_pki_stress/` and uses Scout's HTTP/2 server configuration (`http2: true` in the server config, landed in #274000). That Scout HTTP/2 plumbing was not backported to 8.19 (`backport:skip` on #274000; backporting it would require ~19 files of shared Scout code). PR #286876 (the Scout HTTP/2 backport) is being closed as a result. The Scout files have been dropped from this backport. The test coverage is preserved via a direct FTR port of the same deterministic test: **`x-pack/platform/test/security_api_integration/tests/pki/pki_http2_stress.ts`** The test logic is identical to the Scout spec — same four steps (login / park inside `PKIAuthenticationProvider.authenticate` via the pre-auth hold / RST until socket is confirmed degraded / release and follow-up). Only the harness glue differs (`retry.tryForTime` replaces `expect.poll`; `tough-cookie` `parseCookie` replaces `findSessionCookie` whose root barrel is absent on 8.19). The pre-auth hold infrastructure in `init_routes.ts` is retained; it is what makes the test deterministic. Note: the `main` PR description's "FTR stress test" section describes a timing-based design (commit `3d344b8d46c5`, which fired 20 slow requests and RST'd them after 150ms). That commit was removed in `24d55c0727ac` in favour of the Scout API test and the description was never updated. The FTR test in this 8.19 backport uses the deterministic pre-auth-hold design, not the timing-based one. `pki.http2.stress.config.ts` registered in `ftr_platform_stateful_configs.yml`. --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]> Co-authored-by: kibanamachine <[email protected]>
|
@jeramysoucy thank you! |
Pull request was closed
Backport
This will backport the following commits from
mainto8.19:Questions ?
Please refer to the Backport tool documentation