Skip to content

Commit e818ad6

Browse files
perf: index transform cache lazily
## Problem tsx's transform cache is shared across projects and retains entries for seven days. As users work across more projects, unrelated cache files accumulate in the same temporary directory. Every tsx process eagerly read that directory, even when it never needed to transform code. Processes that did transform code then scanned the full listing for every cache lookup. Users therefore paid a growing startup cost based on unrelated past work: with 10,000 unrelated entries, a 1,000-module run increased from 396ms to 717ms. ## Changes tsx now leaves the disk cache untouched until the first transform-cache operation. When the cache is needed, tsx indexes its entries once and reuses that index, so unrelated files no longer make every lookup or new write progressively slower. Existing recovery and cleanup behavior remains protected: expired and duplicate files are still removed, malformed files are ignored, concurrent writers remain supported, and an older valid result is used when the newest entry is corrupt. The benchmark uses isolated seeded caches, leaving the user's real cache untouched. Node 24.15, 1,000 TypeScript modules, 5 runs, minimum wall time: | Unrelated entries | Before | After | | ---: | ---: | ---: | | 0 | 396ms | 378ms | | 1,000 | 433ms | 400ms | | 10,000 | 717ms | 486ms |
1 parent cdcc623 commit e818ad6

6 files changed

Lines changed: 377 additions & 55 deletions

File tree

‎scripts/benchmark/README.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ pnpm benchmark # all scenarios, current Node,
99
pnpm benchmark esm-ts --compare 4.21.0 # one scenario vs a released version
1010
pnpm benchmark hooks-passthrough --node 24.10.0 # async (worker) vs sync hooks (see below)
1111
pnpm benchmark esm-ts --scale # per-module cost + fixed startup tax
12+
pnpm benchmark esm-ts --cache-entries 10000 # warm lookup cost in a large cache
1213
pnpm --silent benchmark --json > results.json # raw per-run data
1314
```
1415

@@ -49,6 +50,7 @@ Chosen from a full sweep. The four defaults cover distinct, high-signal axes wit
4950
| `-s, --specifier` | `esm-ts` import style: `ts`, `js`, `extensionless` | `ts` |
5051
| `-r, --runs` | Timed runs per cell | `5` |
5152
| `--cold` | Clear the tsx transform cache before every run | `false` |
53+
| `--cache-entries` | Seed tsx scenarios with N unrelated cache entries | `0` |
5254
| `--scale` | Sweep module counts 10/100/300/1000; report per-module cost + fixed tax | `false` |
5355
| `--json` | Emit raw per-run results as JSON (stdout) | `false` |
5456

@@ -72,4 +74,4 @@ Scenarios below their `minNodeVersion` (e.g. `native-ts` under 22.18.0) are skip
7274

7375
## Reproducibility
7476

75-
The tsx transform cache is reset before each cell's warmup to avoid stale-file skew (a large `$TMPDIR/tsx-<uid>` degrades warm runs via a linear cache scan). Runs are interleaved after warmup.
77+
The tsx transform cache is reset before each tsx scenario's warmup to avoid stale-file skew. Pass `--cache-entries` to instead create an isolated cache, seed it with unrelated entries, and retain it across warmup and timed runs. This mode measures cache initialization and warm lookup scaling without clearing or changing the user's cache. Node baseline scenarios are unaffected. Runs are interleaved after warmup.

‎scripts/benchmark/index.ts‎

Lines changed: 75 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import path from 'node:path';
2+
import os from 'node:os';
23
import fs from 'node:fs/promises';
34
import { fileURLToPath } from 'node:url';
45
import { cli } from 'cleye';
@@ -62,6 +63,11 @@ const argv = cli({
6263
description: 'Clear the tsx transform cache before every run',
6364
default: false,
6465
},
66+
cacheEntries: {
67+
type: Number,
68+
description: 'Seed an isolated transform cache with unrelated entries',
69+
default: 0,
70+
},
6571
scale: {
6672
type: Boolean,
6773
description: `Sweep module counts ${scaleCounts.join('/')} and report per-module cost + fixed tax`,
@@ -76,9 +82,16 @@ const argv = cli({
7682
});
7783

7884
const {
79-
compare, node: nodeVersions, modules, specifier, runs, cold, scale, json,
85+
compare, node: nodeVersions, modules, specifier, runs, cold, cacheEntries, scale, json,
8086
} = argv.flags;
8187

88+
if (cacheEntries < 0 || !Number.isInteger(cacheEntries)) {
89+
throw new Error('--cache-entries must be a non-negative integer');
90+
}
91+
if (cacheEntries > 0 && cold) {
92+
throw new Error('--cache-entries cannot be combined with --cold');
93+
}
94+
8295
const selectedScenarios = (
8396
argv._.scenarios.length > 0
8497
? argv._.scenarios.map((name) => {
@@ -100,6 +113,45 @@ const clearTransformCache = () => fs.rm(tsxCacheDirectory, {
100113
force: true,
101114
});
102115

116+
const userId = process.geteuid ? process.geteuid() : os.userInfo().username;
117+
const seedCache = async (temporaryDirectory: string) => {
118+
const time = Math.floor(Date.now() / 1e8);
119+
const cacheDirectory = path.join(temporaryDirectory, `tsx-${userId}`);
120+
await fs.mkdir(cacheDirectory, { recursive: true });
121+
122+
// Bound concurrent writes so large cache fixtures don't exhaust file descriptors.
123+
for (let start = 0; start < cacheEntries; start += 500) {
124+
const end = Math.min(start + 500, cacheEntries);
125+
await Promise.all(Array.from(
126+
{ length: end - start },
127+
(_, offset) => fs.writeFile(
128+
path.join(
129+
cacheDirectory,
130+
`${time}-${(start + offset).toString(16).padStart(40, '0')}`,
131+
),
132+
'{}',
133+
),
134+
));
135+
}
136+
137+
// Share seeded files with implementations that use the unscoped cache path.
138+
await fs.symlink(cacheDirectory, path.join(temporaryDirectory, 'tsx'), 'junction');
139+
};
140+
141+
const prepareTransformCache = async (
142+
runsTsx: boolean,
143+
isolatedCacheDirectory?: string,
144+
) => {
145+
if (isolatedCacheDirectory) {
146+
await seedCache(isolatedCacheDirectory);
147+
return;
148+
}
149+
150+
if (runsTsx) {
151+
await clearTransformCache();
152+
}
153+
};
154+
103155
type Row = {
104156
scenario: string;
105157
nodeVersion: string;
@@ -116,20 +168,28 @@ const measureCell = async (
116168
): Promise<RunResult[]> => {
117169
const entryPath = path.join(fixturePath, scenario.entry);
118170
const args = scenario.runner === 'tsx' ? [cliPath, entryPath] : [entryPath];
119-
const usesCache = scenario.runner === 'tsx';
171+
const runsTsx = scenario.runner === 'tsx';
172+
await using cacheFixture = runsTsx && cacheEntries > 0
173+
? await createFixture()
174+
: undefined;
175+
const cacheEnvironment = cacheFixture
176+
? {
177+
TMPDIR: cacheFixture.path,
178+
TEMP: cacheFixture.path,
179+
TMP: cacheFixture.path,
180+
}
181+
: undefined;
120182

121-
// Reset cache before warmup: avoids stale-file skew and cross-cell pollution
122-
if (usesCache) {
123-
await clearTransformCache();
124-
}
125-
await runOnce(node.path, args, fixturePath);
183+
// Isolated mode retains its directory through warmup and timed runs.
184+
await prepareTransformCache(runsTsx, cacheFixture?.path);
185+
await runOnce(node.path, args, fixturePath, cacheEnvironment);
126186

127187
const results: RunResult[] = [];
128188
for (let run = 0; run < runs; run += 1) {
129-
if (usesCache && cold) {
189+
if (runsTsx && cold) {
130190
await clearTransformCache();
131191
}
132-
results.push(await runOnce(node.path, args, fixturePath));
192+
results.push(await runOnce(node.path, args, fixturePath, cacheEnvironment));
133193
}
134194
return results;
135195
};
@@ -216,6 +276,7 @@ if (json) {
216276
meta: {
217277
runs,
218278
cold,
279+
cacheEntries,
219280
scale,
220281
specifier,
221282
},
@@ -227,7 +288,11 @@ if (json) {
227288

228289
function renderTables() {
229290
out();
230-
out(`${runs} runs/cell · ${cold ? 'cold' : 'warm'} cache · specifier=${specifier}`);
291+
out(
292+
`${runs} runs/cell · ${cold ? 'cold' : 'warm'} cache`
293+
+ ` · tsx scenarios: ${cacheEntries} unrelated cache entries`
294+
+ ` · specifier=${specifier}`,
295+
);
231296

232297
for (const scenario of selectedScenarios) {
233298
const scenarioRows = rows.filter(row => row.scenario === scenario.name);

‎scripts/benchmark/utils/run.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,10 +36,12 @@ export const runOnce = async (
3636
nodePath: string,
3737
args: string[],
3838
cwd: string,
39+
env?: NodeJS.ProcessEnv,
3940
): Promise<RunResult> => {
4041
const startTime = performance.now();
4142
const result = await execa(nodePath, args, {
4243
cwd,
44+
env,
4345
reject: false,
4446
});
4547
const wallMs = performance.now() - startTime;

0 commit comments

Comments
 (0)