Skip to content

Commit c10f3d1

Browse files
fix(relay): apply tunnel cleanup mode without a forced deploy
The cleanup mode was read through Config in the Worker's Init, which Alchemy does not compare when deciding whether the Worker changed (alchemy-run/alchemy#1831). Changing it needed a forced deploy, and a forced deploy replaces the Postgres runtime role (alchemy-run/alchemy#1832). Declare the mode as a plain Worker env binding, which Alchemy diffs, and read it from the Worker env at runtime. Default the manual deploy's force input to off and document when force is still needed. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
1 parent 6391be2 commit c10f3d1

5 files changed

Lines changed: 87 additions & 20 deletions

File tree

‎.github/workflows/deploy-relay.yml‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,16 @@ on:
44
push:
55
branches:
66
- main
7-
# Alchemy does not redeploy the Worker when only a Config value read in its
8-
# Init changes (alchemy-run/alchemy#1831), so a changed repository variable
9-
# needs a forced deploy to reach production.
7+
# Redeploys main after a variable or secret change. Tunnel cleanup modes are
8+
# Worker env bindings, so a normal run applies them. Other values the Worker
9+
# reads in Init only apply with force (alchemy-run/alchemy#1831), which also
10+
# replaces the Postgres runtime role (alchemy-run/alchemy#1832).
1011
workflow_dispatch:
1112
inputs:
1213
force:
13-
description: Redeploy every resource, including ones with no detected changes
14+
description: Redeploy every resource. Replaces the Postgres runtime role; only for secrets.
1415
type: boolean
15-
default: true
16+
default: false
1617

1718
permissions:
1819
contents: read

‎docs/operations/release.md‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -140,10 +140,12 @@ Required `production` environment secrets:
140140
- `CLERK_SECRET_KEY`
141141
- `APNS_PRIVATE_KEY`
142142

143-
The relay Worker reads these variables and secrets when it is deployed. Alchemy does not redeploy the
144-
Worker when only one of these values changes ([alchemy-run/alchemy#1831](https://github.com/alchemy-run/alchemy/issues/1831)),
145-
so a push to `main` without relay code changes leaves the old value in place. After changing one, run
146-
the **Deploy T3 Connect relay** workflow manually from `main` with **force** checked.
143+
After changing a variable or secret, run the **Deploy T3 Connect relay** workflow manually from
144+
`main`. The tunnel cleanup modes are Worker `env` bindings, so a normal run applies them. Alchemy does
145+
not redeploy the Worker when only a value read in the Worker's startup code changes
146+
([alchemy-run/alchemy#1831](https://github.com/alchemy-run/alchemy/issues/1831)), so any other
147+
variable or secret needs the run with **force** checked. A forced run also replaces the Postgres
148+
runtime role and its password ([alchemy-run/alchemy#1832](https://github.com/alchemy-run/alchemy/issues/1832)).
147149

148150
The account-scoped repository credentials are consumed by Alchemy while provisioning relay stages; they
149151
are not bound into the relay Worker. The production deployment uses an Axiom personal access token,
@@ -168,7 +170,7 @@ because those builds register recovery and replace a deleted tunnel after wake.
168170
1. Deploy the relay and migration with cleanup `off`.
169171
2. Release the server build and confirm current hosts register recovery. Older hosts stay marked
170172
legacy and are never candidates.
171-
3. Set `dry-run`, run a forced relay deploy, and read the sweep counters (`scanned`, `wouldDelete`,
173+
3. Set `dry-run`, run a relay deploy, and read the sweep counters (`scanned`, `wouldDelete`,
172174
`skippedLegacy`, `skippedOrphan`, `failed`, `truncated`) across several sweeps. Each sweep records
173175
them, and the active `mode`, as `relay.managed_endpoint_reaper.*` attributes on its
174176
`relay.managed_endpoint_reaper.sweep` span in Axiom.
@@ -178,10 +180,10 @@ because those builds register recovery and replace a deleted tunnel after wake.
178180
The job runs every five minutes with a five-minute grace period for tunnels that lost their
179181
connector, so a candidate is usually removed five to ten minutes after it goes down. Tunnels that
180182
never connected wait an hour. One sweep attempts at most 100 deletions, so a backlog takes longer.
181-
Changing `RELAY_TUNNEL_CLEANUP_MODE`, including turning cleanup off during an incident, needs a forced
182-
relay deploy. Confirm the new `mode` on the next sweep span.
183+
Changing `RELAY_TUNNEL_CLEANUP_MODE`, including turning cleanup off during an incident, needs a relay
184+
deploy without force. Confirm the new `mode` on the next sweep span.
183185

184-
To roll back, set cleanup to `off` and run a forced relay deploy before downgrading any host. Keep the
186+
To roll back, set cleanup to `off` and run a relay deploy before downgrading any host. Keep the
185187
recovery endpoints deployed while current server builds are in use. The nullable columns can stay.
186188

187189
### Disposable-host canary
@@ -201,8 +203,8 @@ stage, test Cloudflare account, disposable host, and disposable T3 home. Keep pr
201203
and pause it with `kill -STOP <first-pid>`. Wait until Cloudflare reports it down for over five
202204
minutes.
203205
5. Confirm dry-run counts the first tunnel in `wouldDelete` and the second in `skippedLegacy`.
204-
6. Set cleanup `enabled` on the disposable stage and deploy it with `--force`. Confirm in the test
205-
Cloudflare account that the first tunnel is deleted and the legacy tunnel still exists.
206+
6. Set cleanup `enabled` on the disposable stage and deploy. Confirm in the test Cloudflare account
207+
that the first tunnel is deleted and the legacy tunnel still exists.
206208
7. Resume the first child with `kill -CONT <first-pid>`. Confirm the running server detects the
207209
repeated rejection, requests recovery, and becomes reachable at the same hostname without a
208210
restart.

‎infra/relay/src/Config.test.ts‎

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,11 @@ import { expect, it } from "@effect/vitest";
22
import * as ConfigProvider from "effect/ConfigProvider";
33
import * as Effect from "effect/Effect";
44

5-
import { managedEndpointCleanupModeConfig } from "./Config.ts";
5+
import {
6+
decodeManagedEndpointCleanupModeEnv,
7+
managedEndpointCleanupModeConfig,
8+
managedEndpointCleanupModeEnv,
9+
} from "./Config.ts";
610

711
it.effect.each([
812
{ name: "missing", env: {}, expected: "off" },
@@ -33,3 +37,35 @@ it.effect("rejects an invalid cleanup mode", () =>
3337
expect(error.message).toContain('Expected "off" | "dry-run" | "enabled"');
3438
}),
3539
);
40+
41+
it.effect.each([
42+
{ name: "missing", value: undefined, expected: "off" },
43+
{ name: "blank", value: " ", expected: "off" },
44+
{ name: "dry-run", value: "dry-run", expected: "dry-run" },
45+
{ name: "padded enabled", value: " enabled ", expected: "enabled" },
46+
] as const)("decodes a $name cleanup mode binding as $expected", ({ value, expected }) =>
47+
Effect.gen(function* () {
48+
expect(yield* decodeManagedEndpointCleanupModeEnv(value)).toBe(expected);
49+
}),
50+
);
51+
52+
it.effect("rejects an invalid cleanup mode binding", () =>
53+
Effect.gen(function* () {
54+
const error = yield* Effect.flip(decodeManagedEndpointCleanupModeEnv("delete-everything"));
55+
expect(error._tag).toBe("SchemaError");
56+
}),
57+
);
58+
59+
it.effect("declares the cleanup mode as a plain env binding", () =>
60+
Effect.gen(function* () {
61+
const env = yield* managedEndpointCleanupModeEnv.pipe(
62+
Effect.provideService(
63+
ConfigProvider.ConfigProvider,
64+
ConfigProvider.fromEnv({ env: { RELAY_TUNNEL_CLEANUP_MODE: "dry-run" } }),
65+
),
66+
);
67+
// A plain string is what Alchemy lowers into a `plain_text` binding that
68+
// its Worker diff compares; a Redacted or Config value would not change it.
69+
expect(env).toEqual({ RELAY_TUNNEL_CLEANUP_MODE: "dry-run" });
70+
}),
71+
);

‎infra/relay/src/Config.ts‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,9 @@ export const ManagedEndpointCleanupMode = Schema.Literals(["off", "dry-run", "en
1212
export type ManagedEndpointCleanupMode = typeof ManagedEndpointCleanupMode.Type;
1313
const decodeManagedEndpointCleanupMode = Schema.decodeUnknownEffect(ManagedEndpointCleanupMode);
1414

15-
export const managedEndpointCleanupModeConfig = Config.String("RELAY_TUNNEL_CLEANUP_MODE").pipe(
15+
export const RELAY_TUNNEL_CLEANUP_MODE = "RELAY_TUNNEL_CLEANUP_MODE";
16+
17+
export const managedEndpointCleanupModeConfig = Config.String(RELAY_TUNNEL_CLEANUP_MODE).pipe(
1618
Config.withDefault("off"),
1719
Config.map((value) => value.trim() || "off"),
1820
Config.mapEffect((value) =>
@@ -22,6 +24,24 @@ export const managedEndpointCleanupModeConfig = Config.String("RELAY_TUNNEL_CLEA
2224
),
2325
);
2426

27+
/** Decodes a cleanup mode binding; a missing or blank binding means `off`. */
28+
export const decodeManagedEndpointCleanupModeEnv = (value: unknown) =>
29+
decodeManagedEndpointCleanupMode(
30+
typeof value === "string" && value.trim() !== "" ? value.trim() : "off",
31+
);
32+
33+
/**
34+
* Cleanup modes as plain Worker `env` entries. Alchemy compares declared
35+
* `env` bindings when it decides whether the Worker changed, but not values
36+
* read through `Config` in Init (alchemy-run/alchemy#1831), so a mode read
37+
* only in Init would need a forced deploy to change.
38+
*/
39+
export const managedEndpointCleanupModeEnv = Effect.gen(function* () {
40+
return {
41+
[RELAY_TUNNEL_CLEANUP_MODE]: yield* managedEndpointCleanupModeConfig,
42+
};
43+
});
44+
2545
export interface ApnsCredentials {
2646
readonly teamId: string;
2747
readonly keyId: string;

‎infra/relay/src/worker.ts‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -115,14 +115,15 @@ const ApnsDeliveryJobSigningSecret = Alchemy.makeRandom("ApnsDeliveryJobSigningS
115115
export class Api extends Cloudflare.Worker<Api, {}>()("Api") {}
116116

117117
export const ApiLive = Api.make(
118-
RelayDeploymentConfig.pipe(
119-
Effect.map(({ relayPublicDomain }) => ({
118+
Effect.all([RelayDeploymentConfig, RelayConfiguration.managedEndpointCleanupModeEnv]).pipe(
119+
Effect.map(([{ relayPublicDomain }, cleanupModeEnv]) => ({
120120
main: import.meta.filename,
121121
compatibility: {
122122
date: "2026-05-22",
123123
flags: ["nodejs_compat"],
124124
},
125125
domain: relayPublicDomain,
126+
env: cleanupModeEnv,
126127
})),
127128
Effect.orDie,
128129
),
@@ -182,14 +183,21 @@ export const ApiLive = Api.make(
182183
yield* yield* relayApiZone.zoneId;
183184
const managedEndpointDnsBinding = yield* Cloudflare.DNS.ReadWriteDns(managedEndpointZone);
184185
const managedEndpointZoneName = yield* managedEndpointZone.name;
185-
const managedEndpointCleanupMode = yield* RelayConfiguration.managedEndpointCleanupModeConfig;
186186

187187
//
188188
// 3. Runtime layers and app construction
189189
//
190190
const alchemyRuntimeContext: Alchemy.BaseRuntimeContext = yield* Cloudflare.Worker;
191191

192192
const loadSettings = Effect.gen(function* () {
193+
// Read the cleanup mode from the declared `env` binding, never from a
194+
// Config value captured in Init: only the binding is updated on deploy.
195+
const managedEndpointCleanupMode =
196+
yield* RelayConfiguration.decodeManagedEndpointCleanupModeEnv(
197+
(yield* Cloudflare.Workers.WorkerEnvironment)[
198+
RelayConfiguration.RELAY_TUNNEL_CLEANUP_MODE
199+
],
200+
);
193201
return RelayConfiguration.RelayConfiguration.of({
194202
relayIssuer: relayPublicOrigin,
195203
...(fcmServiceAccount ? { fcmServiceAccount } : {}),

0 commit comments

Comments
 (0)