Add E2E HTTPS egress tests and reject WebSocket upgrades - #1634

/assign haiyanmeng

The test setup part ended up being bigger than I would have liked. Please let me know if there's something that I can trim down or reuse. I'm re-checking for this myself as well.

Fixes #1017

Let us say Addresses #1017 instead since it does not close all the gaps in #1017.
Fixes #1017 will auto-close the issue after the PR is merged.

ran the MITM fixture related tests; checked it with gvisor + mitm. I couldn't run the micro-vm tests on my machine.

$ E2E_EGRESS_MITM=1 hack/run-e2e-kind.sh \
  ./internal/e2e/suites/egressmitm -v -args --no-color
...
--- PASS: TestActorEgressMITMTrust (62.04s)
...

$ E2E_EGRESS_MITM=1 hack/run-e2e-kind.sh \
  ./internal/e2e/suites/networking -run '^TestActorEgress' -v -args --no-color
...
--- PASS: TestActorEgressHTTPSNonStandardPort (19.29s)
--- PASS: TestActorEgressWebSocket (10.55s)
--- PASS: TestActorEgressSecureWebSocket (19.98s)
...

pr-workflow / e2e-test (pull_request)Failing after 16m

Adhita Selvaraj (@swiftdiaries) , PTAL.

pr-workflow / e2e-test (pull_request)Failing after 16m

Adhita Selvaraj (Adhita Selvaraj (@swiftdiaries)) , PTAL.

Took a look at the failure, it looks like the guest clock doesn't restore the snapshot downtime? I'm not too familiar with the micro-vm test setup and I can't reproduce it :( It looks like a pre-existing failure mode for fresh certs? I'll take some time to look into this and meanwhile, create a separate issue to track, post it on slack to ask for broader help.

Benjamin Elder (@BenTheElder) could I please borrow your expertise a little here? Trying to debug the guestclock not correcting time when a snapshot is restored. I'm not too sure about the setup with Cloud Hypervisor for the micro-vm test path
This was the closest change I could see: #972

re: last commit - trying to see if i can set clocksource=kvm-clock in the micro-vm MITM CI codepath to see if it solves for the clock skew

Tested this on a Linux VM:

Setup:
Kind v1.37.0
Kata 4.0.0
Cloud Hypervisor 53.0

E2E_EGRESS_MITM=1 E2E_SANDBOX_CLASS=microvm \
  hack/run-e2e-kind.sh ./internal/e2e/suites/networking \
  -run '^(TestActorEgressHTTPSNonStandardPort|TestActorEgressWebSocket|TestActorEgressSecureWebSocket)$' \
  -v -args --no-color

--- PASS: TestActorEgressHTTPSNonStandardPort (19.96s)
--- PASS: TestActorEgressWebSocket (12.38s)
--- PASS: TestActorEgressSecureWebSocket (20.58s)
PASS
ok  github.com/agent-substrate/substrate/internal/e2e/suites/networking  53.103s

snapshot, clock, clocksource

golden takeGoldenSnapshotAt: 2026-09-16T05:33:59.957947998Z
host before:                 2026-09-16T05:39:00.283067832Z
guest UTC:                   2026-09-16T05:39:00.320563549Z
host after:                  2026-09-16T05:39:00.388914694Z
current_clocksource:         kvm-clock
available_clocksource:       kvm-clock tsc acpi_pm

Without knowing the clock source in CI, I can't confirm if the last commit is the necessary fix. If it's something else then forcing kvm-clock might break other working tests. So gating this to tests which uses TLS is the one way I could think of to limit blast radius for the change. If we find that this bug occurs in other tests, we can extend coverage and make it standard.

WDYT? haiyanmeng Bowei Du (@bowei)

rebased and fixed some additional gateway checks in CI
cc haiyanmeng

Adhita Selvaraj (@swiftdiaries) , https://github.com/agent-substrate/substrate/blob/main/docs/egress-traffic.md specifies the supported egress traffic for GA.

Currently, the Envoy dataplane supports WebSocket HTTP/1.1 Upgrade. To align with the specification, we can update the PR in the following way: keep the test fixture, modify the Envoy configuration to stop supporting WebSocket HTTP/1.1 Upgrade, and have a negative test verifying that the egress gateway denies the WebSocket HTTP/1.1 Upgrade traffic with 403.

WDYT?

cc Bowei Du (@bowei)

Benjamin Elder (@BenTheElder) CI here is waiting on a maintainer to approve the run, and this PR edits .github/workflows/pr-workflow.yaml, so I'd rather you take a look before it runs. Could you check it's OK to approve?

Thank you Maya Wang (@mayawang) Benjamin Elder (@BenTheElder)
We need #1945 and #1944 to go in first. I've run CI jobs for all three PRs against my fork's main and have passing tests ( per @/haiyanmeng's suggestion to help move things along). Once those two go in, I'll rebase and we can run CI, add this PR to the merge queue?

Wiz Scan Summary

Scanner	Findings
Vulnerabilities	-
Sensitive Data	-
Secrets	-
IaC Misconfigurations	-
SAST Findings	1  3
Software Management Findings	-
Total	1  3

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try the Wiz Code extension for VS Code, JetBrains, or Visual Studio.

I've created a follow-up PR for #1944 here: #2306
This PR is ready for review Benjamin Elder (@BenTheElder)

whats the bound grpc requests commit? what is it for? 7f33812

whats the bound grpc requests commit? what is it for? 7f33812

I saw the wiz comment but I couldn't see the actual vulns so I did a security review pass. I've pulled out the commit, I'll make a separate PR for it after creating an issue to discuss first.

Thanks for the review! Lior Lieberman (@LiorLieberman)

Bowei Du (@bowei) Benjamin Elder (@BenTheElder) could you please take a look when you get a chance?