Skip to content

Commit 9d19828

Browse files
refactor(infra): integrate cache telemetry with native Vite runs
1 parent b2bbd1e commit 9d19828

20 files changed

Lines changed: 568 additions & 302 deletions

‎.github/actions/setup-ci-deps/action.yml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,6 @@ runs:
7070
if: hashFiles('.task-cache.json') != ''
7171
shell: bash
7272
run: |
73-
node infra/task-cache/install.mjs
7473
if [[ "$GITHUB_EVENT_NAME" == push && "$GITHUB_REF" == refs/heads/main ]]; then
7574
echo 'VP_REMOTE_CACHE=read-write' >> "$GITHUB_ENV"
7675
else

‎infra/task-cache/README.md‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
The Worker uses Effect services for request handling, developer authentication, bounded concurrency, deadlines, and cleanup. The pinned [upstream protocol package](https://github.com/voidzero-dev/vite-plus-remote-cache-cloudflare/tree/5b06c2c10d84ea441735ba07c46f6d1b7cb2ff3b) supplies the experimental wire format, streaming upload, GitHub OIDC validation, and atomic D1 publication. Keeping that adapter preserves compatibility with existing cache entries.
66

7-
Vite+ 1.3 includes an experimental remote client even though its TypeScript configuration and CLI help omit it. The small dependency-free launcher supplies `VP_REMOTE_CACHE_URL` and `VP_REMOTE_CACHE`. It leaves projects without `.task-cache.json` alone. Commands keep using the project's installed Vite+ binary.
7+
Vite+ 1.3 includes an experimental remote client even though its TypeScript configuration and CLI help omit it. The native client reads `run.cache.remote.url`, `VP_REMOTE_CACHE_URL`, and `VP_REMOTE_CACHE`. A small patch to the Vite+ JavaScript entrypoint invokes an optional repository hook before and after the native task runner. The global executable and PATH are unchanged. The hook loads saved credentials only when no explicit URL is supplied, and exports native task summaries. Projects without `.task-cache.json` are unaffected.
88

99
## Deploy
1010

@@ -28,12 +28,14 @@ node infra/task-cache/scripts/manage.ts add-writer maria
2828
This updates the local writer registry and writes `~/.config/t3-task-cache/maria.json`. Run `alchemy deploy --stage prod` from `infra/task-cache` to apply the registry, then transfer that file through a private channel. The recipient runs:
2929

3030
```sh
31-
bash infra/task-cache/install.sh /path/to/maria.json
31+
bash infra/task-cache/install.sh
3232
```
3333

34-
The installer adds the launcher to shell startup files after Vite+'s environment setup. Existing shells should run `source ~/.config/t3-task-cache/env` once. Use separate writer names for separate people. Repeating `add-writer` rotates the token; `revoke-writer <name>` removes that writer from the local registry. Deploy again to apply either change. Never commit or attach credential files to an issue or PR.
34+
The installer prompts for the endpoint, developer name, and token (hidden), verifies authentication with an empty telemetry report, and saves credentials with mode `0600`. It patches the project and global fallback JavaScript entrypoints, keeping a backup of each modified entrypoint. Unsupported entrypoint shapes fail before patching. Rerunning setup is idempotent and updates the hook. It also removes the old PATH wrapper integration if present.
3535

36-
CI installs the launcher through `setup-ci-deps`; default-branch push jobs upload with GitHub OIDC, and PR jobs read. Deployment credentials are never needed on contributor machines or CI runners. Reads are public, as in the upstream service. Trusted writers can publish artifacts that developers and CI consume.
36+
Explicit environment settings take precedence: `VP_REMOTE_CACHE_URL` may contain the URL-encoded developer name and token as HTTP Basic credentials, and `VP_REMOTE_CACHE=read-write` enables uploads. These can be supplied by a secret manager instead of saved credentials. Do not put credentials in shell command history or commit them. Use separate writer names for separate people. Repeating `add-writer` rotates the token; `revoke-writer <name>` removes it. Deploy again to apply either change.
37+
38+
CI gets the Vite+ hook through the pinned package patch; default-branch push jobs upload with GitHub OIDC, and PR jobs read. Deployment credentials are never needed on contributor machines or CI runners. Reads are public, as in the upstream service. Trusted writers can publish artifacts that developers and CI consume.
3739

3840
## Verify
3941

@@ -56,10 +58,10 @@ The smoke check uploads a unique fixture, restores its output into a cold second
5658

5759
Alchemy also owns the `t3-task-cache-traces` Axiom dataset, its ingest-only Worker token, a savings dashboard, and a separate retained D1 database for lifetime counters. Deployment needs an Axiom Alchemy profile or `AXIOM_TOKEN`/`AXIOM_ORG_ID` as well as Cloudflare credentials. Contributor machines never receive the Axiom token.
5860

59-
Both `vp run` and its `vpr` shortcut use the launcher. The launcher observes Vite+'s atomic `last-summary.json` files, including nested runs. It reports package/task names, confirmed local and remote hits, avoided task runtime, developer identity, host, OS/architecture, and CI run ID. It does not send commands, arguments, source paths, environment variables, or task output. Developer identity comes from the authenticated cache writer; CI uses GitHub's verified OIDC actor. PR telemetry authorization does not grant artifact uploads. Fork jobs without an OIDC token cannot report.
61+
Both `vp run` and its `vpr` shortcut invoke the hook. The hook observes Vite+'s atomic `last-summary.json` files, including nested runs. It reports package/task names, confirmed local and remote hits, avoided task runtime, developer identity, host, OS/architecture, and CI run ID. It does not send commands, arguments, source paths, environment variables, or task output. Developer identity comes from the authenticated cache writer; CI uses GitHub's verified OIDC actor. PR telemetry authorization does not grant artifact uploads. Fork jobs without an OIDC token cannot report.
6062

61-
Reports spool locally and retry on subsequent commands. Interactive commands flush in a detached process; CI flushes before exiting. Queues retain at most 2,000 reports, each sent with a three-second timeout and a 15-second budget per flush. Stable report IDs and transactional D1 receipts prevent duplicate lifetime increments. Receipts expire after 31 days; accepted reports must be less than 30 days old. Lifetime totals survive raw trace expiration and are re-exported hourly, including inactive developers and hosts. Collection starts when the new launcher is installed; existing cache history is not backfilled.
63+
Reports spool locally and retry on subsequent commands. The completion hook flushes before the native CLI exits, including failed runs. Collection or export failures never change the task exit status. No background service is installed. Queues retain at most 2,000 reports, each sent with a three-second timeout and a 15-second budget per flush. Stable report IDs and transactional D1 receipts prevent duplicate lifetime increments. Receipts expire after 31 days; accepted reports must be less than 30 days old. Lifetime totals survive raw trace expiration and are re-exported hourly, including inactive developers and hosts. Collection starts when the hook is installed; existing cache history is not backfilled.
6264

6365
**Saved time means historical task execution time avoided.** It is useful as a compute proxy, but it is not measured CPU time, money saved, or elapsed developer waiting time. Parallel tasks can overlap, and cache lookup/restore overhead is not subtracted. Failed restores contribute no savings. Older Vite+ versions omit the hit source; their confirmed savings are counted under an explicit unspecified tier. Native summary collection is best-effort: interrupted runs or summaries overwritten before observation can be missing. The experimental Vite+ summary schema is tested here and must be checked when upgrading Vite+.
6466

65-
Host labels use `~/.agents/MACHINE` when present, otherwise the hostname. GitHub-hosted runners use their runner OS/architecture class to avoid a new lifetime series for every ephemeral VM. Set `T3_TASK_CACHE_HOST` to override the label, or `T3_TASK_CACHE_TELEMETRY=off` to opt out while retaining remote caching. Re-run the installer to update an existing machine's launcher.
67+
Host labels use `~/.agents/MACHINE` when present, otherwise the hostname. GitHub-hosted runners use their runner OS/architecture class to avoid a new lifetime series for every ephemeral VM. Set `T3_TASK_CACHE_HOST` to override the label, or `T3_TASK_CACHE_TELEMETRY=off` to opt out while retaining remote caching. Re-run the installer to update an existing machine's hook.

‎infra/task-cache/install.mjs‎

Lines changed: 0 additions & 38 deletions
This file was deleted.

‎infra/task-cache/install.sh‎

Lines changed: 1 addition & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,3 @@
11
#!/usr/bin/env bash
22
set -euo pipefail
3-
4-
# The same installer supports a checkout or a privately transferred rollout payload.
5-
source_dir="${T3_TASK_CACHE_SOURCE_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)}"
6-
export PATH="$HOME/.local/share/vite-plus/bin:$HOME/.vite-plus/bin:$HOME/.local/bin:/opt/homebrew/bin:/usr/local/bin:$PATH"
7-
8-
if [[ $# -gt 0 ]]; then
9-
node "$source_dir/install.mjs" "$1"
10-
else
11-
node "$source_dir/install.mjs"
12-
fi
13-
14-
mkdir -p "$HOME/.config/t3-task-cache"
15-
cat > "$HOME/.config/t3-task-cache/env" <<'EOF'
16-
case "$PATH" in
17-
"$HOME/.local/share/t3-task-cache/bin:"*) ;;
18-
*) export PATH="$HOME/.local/share/t3-task-cache/bin:$PATH" ;;
19-
esac
20-
EOF
21-
22-
line='. "$HOME/.config/t3-task-cache/env"'
23-
for profile in "$HOME/.profile" "$HOME/.bashrc" "$HOME/.bash_profile" "$HOME/.zshenv" "$HOME/.zshrc"; do
24-
# Do not create a bash_profile: doing so would shadow an existing .profile.
25-
if [[ ! -f "$profile" && "$profile" == "$HOME/.bash_profile" ]]; then
26-
continue
27-
fi
28-
if ! grep -Fqx "$line" "$profile" 2>/dev/null; then
29-
printf '\n%s\n' "$line" >> "$profile"
30-
fi
31-
done
32-
33-
. "$HOME/.config/t3-task-cache/env"
34-
printf 'Installed task-cache launcher for %s\n' "$(cat "$HOME/.agents/MACHINE" 2>/dev/null || hostname)"
35-
command -v vp
3+
exec node "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/scripts/install.ts" "$@"

‎infra/task-cache/package.json‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,6 @@
77
"typecheck": "tsc --noEmit",
88
"smoke": "node scripts/smoke.ts",
99
"deploy": "alchemy deploy --stage prod",
10-
"install:launcher": "node install.mjs",
1110
"writers": "node scripts/manage.ts"
1211
},
1312
"dependencies": {
Lines changed: 204 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
/* oxlint-disable t3code/no-global-process-runtime -- Standalone setup must run before workspace dependencies are installed. */
2+
// @effect-diagnostics nodeBuiltinImport:off - standalone bootstrap also runs before dependencies are installed.
3+
// @effect-diagnostics globalConsole:off globalFetch:off globalDate:off - interactive setup and endpoint verification.
4+
import * as NodeCrypto from "node:crypto";
5+
import * as NodeFSP from "node:fs/promises";
6+
import * as NodeOS from "node:os";
7+
import * as NodePath from "node:path";
8+
import * as NodeReadlinePromises from "node:readline/promises";
9+
import { patchNativeCli } from "../src/nativeHook.ts";
10+
11+
const home = NodeOS.homedir();
12+
const root = NodePath.resolve(import.meta.dirname, "../../..");
13+
const state = NodePath.join(home, ".config/t3-task-cache");
14+
const nonInteractive = process.argv.includes("--non-interactive");
15+
const { endpoint: defaultEndpoint } = JSON.parse(
16+
await NodeFSP.readFile(NodePath.join(root, ".task-cache.json"), "utf8"),
17+
);
18+
let credentials: Record<string, { name: string; token: string }> = {};
19+
try {
20+
credentials = JSON.parse(
21+
await NodeFSP.readFile(NodePath.join(state, "credentials.json"), "utf8"),
22+
);
23+
} catch (error) {
24+
if (!(error instanceof Error && "code" in error && error.code === "ENOENT")) throw error;
25+
}
26+
const prompt = NodeReadlinePromises.createInterface({
27+
input: process.stdin,
28+
output: process.stdout,
29+
});
30+
let endpoint = process.env.T3_CACHE_SETUP_URL || defaultEndpoint;
31+
let name = process.env.T3_CACHE_SETUP_NAME || credentials[endpoint]?.name;
32+
let token = process.env.T3_CACHE_SETUP_TOKEN || credentials[endpoint]?.token;
33+
try {
34+
if (!nonInteractive) {
35+
endpoint = (await prompt.question(`Cache URL [${endpoint}]: `)).trim() || endpoint;
36+
name = (await prompt.question(`Developer name${name ? ` [${name}]` : ""}: `)).trim() || name;
37+
prompt.close();
38+
if (!token || endpoint !== defaultEndpoint || name !== credentials[endpoint]?.name) {
39+
if (!process.stdin.isTTY)
40+
throw new Error("Use T3_CACHE_SETUP_TOKEN for non-interactive setup.");
41+
process.stdout.write("Credential token (hidden): ");
42+
process.stdin.setRawMode(true);
43+
process.stdin.resume();
44+
token = await new Promise<string>((resolve, reject) => {
45+
let value = "";
46+
const data = (chunk: Buffer) => {
47+
for (const character of chunk.toString()) {
48+
if (character === "\u0003") {
49+
cleanup();
50+
reject(new Error("Cancelled"));
51+
return;
52+
}
53+
if (character === "\r" || character === "\n") {
54+
cleanup();
55+
resolve(value);
56+
return;
57+
}
58+
if (character === "\u007f") value = value.slice(0, -1);
59+
else value += character;
60+
}
61+
};
62+
const cleanup = () => {
63+
process.stdin.off("data", data);
64+
process.stdin.setRawMode(false);
65+
process.stdin.pause();
66+
process.stdout.write("\n");
67+
};
68+
process.stdin.on("data", data);
69+
});
70+
}
71+
}
72+
} finally {
73+
prompt.close();
74+
}
75+
if (!name || !token || !/^[a-z0-9-]{1,64}$/.test(name) || token.length < 32)
76+
throw new Error("A valid developer name and credential token are required.");
77+
const url = new URL(endpoint);
78+
if (
79+
!["http:", "https:"].includes(url.protocol) ||
80+
url.username ||
81+
url.password ||
82+
url.search ||
83+
url.hash
84+
)
85+
throw new Error("Use a cache endpoint without embedded credentials, query, or fragment.");
86+
endpoint = url.href.replace(/\/$/, "");
87+
const response = await fetch(`${endpoint}/telemetry`, {
88+
method: "POST",
89+
headers: {
90+
Authorization: `Basic ${Buffer.from(`${name}:${token}`).toString("base64")}`,
91+
"Content-Type": "application/json",
92+
},
93+
body: JSON.stringify({
94+
id: NodeCrypto.randomBytes(32).toString("hex"),
95+
recordedAt: Date.now(),
96+
host: NodeOS.hostname(),
97+
platform: NodeOS.platform(),
98+
arch: NodeOS.arch(),
99+
ci: false,
100+
runId: "setup",
101+
tasks: [],
102+
}),
103+
signal: AbortSignal.timeout(10_000),
104+
});
105+
await response.body?.cancel();
106+
if (!response.ok)
107+
throw new Error(`Credential verification failed (${response.status}); nothing was installed.`);
108+
109+
const candidates = [NodePath.join(root, "node_modules/vite-plus/dist/bin.js")];
110+
for (const directory of [
111+
process.env.VITE_PLUS_HOME,
112+
NodePath.join(home, ".vite-plus/current"),
113+
NodePath.join(home, ".local/share/vite-plus/current"),
114+
]) {
115+
if (directory) candidates.push(NodePath.join(directory, "node_modules/vite-plus/dist/bin.js"));
116+
}
117+
for (let index = 0; index < process.argv.length; index++) {
118+
if (process.argv[index] === "--entrypoint") {
119+
const entrypoint = process.argv[++index];
120+
if (!entrypoint) throw new Error("Missing --entrypoint path");
121+
candidates.push(entrypoint);
122+
}
123+
}
124+
const targets = new Map<string, { original: string; patched: string }>();
125+
for (const candidate of candidates) {
126+
try {
127+
const path = await NodeFSP.realpath(candidate);
128+
const original = await NodeFSP.readFile(path, "utf8");
129+
targets.set(path, { original, patched: patchNativeCli(original) });
130+
} catch (error) {
131+
if (!(error instanceof Error && "code" in error && error.code === "ENOENT")) throw error;
132+
}
133+
}
134+
if (!targets.size) throw new Error("Install Vite+ before installing its telemetry hook.");
135+
const hook = await NodeFSP.readFile(
136+
NodePath.join(root, "node_modules/vite-plus/dist/run-hook.js"),
137+
"utf8",
138+
);
139+
const installed = NodePath.join(home, ".local/share/t3-task-cache");
140+
await NodeFSP.mkdir(installed, { recursive: true });
141+
for (const file of ["runHook.ts", "telemetry.ts"])
142+
await NodeFSP.copyFile(
143+
NodePath.join(import.meta.dirname, "../src", file),
144+
NodePath.join(installed, file),
145+
);
146+
const hookConfig = NodePath.join(home, ".config/vite-plus/run-hook.json");
147+
await NodeFSP.mkdir(NodePath.dirname(hookConfig), { recursive: true });
148+
await NodeFSP.writeFile(
149+
hookConfig,
150+
JSON.stringify({
151+
rootMarker: ".task-cache.json",
152+
module: NodePath.join(installed, "runHook.ts"),
153+
}),
154+
);
155+
for (const [path, { original, patched }] of targets) {
156+
if (original !== patched) {
157+
await NodeFSP.writeFile(`${path}.before-task-cache`, original, { flag: "wx" }).catch(
158+
(error) => {
159+
if (error.code !== "EEXIST") throw error;
160+
},
161+
);
162+
await NodeFSP.writeFile(path, patched);
163+
}
164+
await NodeFSP.writeFile(NodePath.join(NodePath.dirname(path), "run-hook.js"), hook);
165+
}
166+
await NodeFSP.mkdir(state, { recursive: true, mode: 0o700 });
167+
credentials[endpoint] = { name, token };
168+
await NodeFSP.writeFile(NodePath.join(state, "credentials.json"), JSON.stringify(credentials), {
169+
mode: 0o600,
170+
});
171+
await NodeFSP.chmod(NodePath.join(state, "credentials.json"), 0o600);
172+
173+
// Remove only the PATH integration installed by the previous task-cache bootstrap.
174+
for (const file of [".profile", ".bashrc", ".bash_profile", ".zshenv", ".zshrc"]) {
175+
const path = NodePath.join(home, file);
176+
try {
177+
const original = await NodeFSP.readFile(path, "utf8");
178+
const cleaned = original
179+
.split("\n")
180+
.filter((line) => line !== '. "$HOME/.config/t3-task-cache/env"')
181+
.join("\n");
182+
if (cleaned !== original) await NodeFSP.writeFile(path, cleaned);
183+
} catch (error) {
184+
if (!(error instanceof Error && "code" in error && error.code === "ENOENT")) throw error;
185+
}
186+
}
187+
for (const file of ["vp", "vpr"]) {
188+
const path = NodePath.join(home, ".local/share/t3-task-cache/bin", file);
189+
try {
190+
if ((await NodeFSP.readlink(path)) === "../vp.ts") await NodeFSP.unlink(path);
191+
} catch (error) {
192+
if (
193+
!(
194+
error instanceof Error &&
195+
"code" in error &&
196+
["ENOENT", "EINVAL"].includes(String(error.code))
197+
)
198+
)
199+
throw error;
200+
}
201+
}
202+
console.log(
203+
`Verified credentials and installed telemetry hooks in ${targets.size} Vite+ entrypoints. Global executables and PATH are unchanged. Open a fresh shell if the old launcher was active.`,
204+
);

0 commit comments

Comments
 (0)