Skip to content

Delete compilation artifacts once they are no longer needed - #2764

Merged
natebosch merged 28 commits into
masterfrom
cleanup-compilation-artifacts
Sep 29, 2026
Merged

natebosch merged 28 commits into
masterfrom
cleanup-compilation-artifacts

Conversation

@natebosch

@natebosch natebosch commented Sep 19, 2026 •

Copy link
Copy Markdown
Member

Fixes #2710

The runner kept every artifact it compiled until the end of the run: a kernel
file per suite for the default compiler, and a snapshot or bundle per suite for
the exe and cli compilers. On a large test suite that is a lot of temporary
disk space held for no reason, since nothing uses an artifact once its suite has
finished running.

Kernel files

TestCompiler now tracks which per-language-version compiler produced each
kernel file, so release can delete it as soon as its suite is done. The dill
being considered for the incremental cache is kept, since it is still needed,
and is deleted once a larger one supersedes it or the compiler is disposed.

VMPlatform registers the release as a cleanup callback for the suite. The
request to kill the isolate is ordered ahead of it, but Isolate.kill does not
wait for the isolate to exit, so this is best effort; a file that can't be
deleted yet is removed along with the compiler's temp directory when it is
disposed.

exe and cli output

Each suite is compiled into a directory of its own under the runner's temp
directory, which also holds the unix socket. The executable can't be deleted
while it is running, so deleting the directory waits for the test process to
exit. That wait starts as soon as the process starts, so the directory is
deleted whenever the process exits, including before the suite has loaded.

Cleanup is tracked instead of discarded

The per-suite cleanup callbacks were void Function(), so futures from
tear-offs like serverSocket.close were silently dropped, and nothing waited
for any cleanup to finish. They are now FutureOr<void> Function(), and every
cleanup future is recorded on the platform.

Each resource a suite acquires registers its cleanup as soon as it is acquired,
and load runs those callbacks whenever loading fails, so every step shares one
cleanup path. This also covers failures after an isolate is spawned, such as
during --debug setup, which previously left the isolate running.

Cleanup is still not awaited when a suite finishes, so a slow delete can't
delay it. Instead VMPlatform.close waits for all outstanding cleanup before
disposing the compiler and deleting its temp directory. Test processes are
tracked from the moment they start, and any still running at that point are
sent SIGKILL, so a process which ignores SIGTERM, or was never asked to
exit because its suite failed to load, can't hang close or be left running.
A process is not started once the platform is closing.

Every cleanup error is recorded as it happens, and close reports each one as
a warning with its stack trace rather than throwing. Runner.close now deletes
the runner's temp directory in a finally, so it is removed even if closing a
platform fails.

The suite's stream is now closed in a finally, so a failure while waiting for
the debugger after the tests can't leave the suite unfinished.

Load hang when the test process exits early

Loading an exe or cli suite waited for the test process to connect with
nothing else to end the wait. If the process exited first - a crash on
startup, or being killed while the platform closes - the load never finished,
and a graceful shutdown hung on it too. The connection is now raced against
the process exiting, and the load fails with a LoadException naming the exit
code, or returns null if the platform is closing.

Not changed

The small bootstrap .dart files written per test file are still removed when
the platform is closed.

No public API changes; everything here is under lib/src and not exported.

Verification

Measured with a 6-suite project, sampling the temp directory during the run:

before after
kernel files retained 6 1
exe artifacts retained at peak 3 snapshots + 3 dirs 1 dir

The one retained kernel file is the incremental cache candidate, which is
expected.

test_compiler_test.dart gains coverage for both release paths.
compiler_test.dart and signal_test.dart pass.

The load hang has no automated test, since the native bootstrap connects before
any test code runs. It was verified by temporarily making the bootstrap exit
before connecting: the suite now fails to load with "The test process exited
with code 3 before connecting to the test runner.", where previously the run
hung and did not exit on SIGTERM.

Temporary directories were created all over the place - browser
profiles, compiler output, the VM platform's sockets - each with its own
`Directory.systemTemp.createTempSync` call, and nothing deleted most of
them. A run would leave everything it created behind for the OS to clean
up whenever it got around to it.

Create them all under a single `dart_test.` directory instead, so
cleanup is one recursive delete. `Runner.close` does that delete once
every platform has been closed, which also covers a run that was
interrupted with a single Control-C, since that shuts down gracefully.

A second signal skips the graceful shutdown, so handle it directly:
delete the directory synchronously and exit. This requires leaving the
signal subscription active after the first signal, where previously it
was cancelled and the second signal was left to Dart's default handler.
The exit code is the same as the shell reports for a process killed by
the signal, though the process now exits normally with that code rather
than being killed by the signal.

This also replaces `createTempDir`, which returned a path, with
`createTempDirectory`, which returns a `Directory`, removing a
distinction without a difference between the two names.
The runner kept every artifact it compiled until the end of the run: a
kernel file per suite for the default compiler, and a snapshot or bundle
per suite for the `exe` and `cli` compilers. On a large test suite that
is a lot of temporary disk space held for no reason, since nothing uses
an artifact once its suite has finished running.

Track which compiler produced each kernel file so it can be deleted as
soon as its suite is done. The dill being considered for the incremental
cache is kept, since it is still needed, and is deleted once a larger
one supersedes it or the compiler is disposed.

For `exe` and `cli`, give each suite a directory of its own and delete
it once the test process has exited. The process has to be gone first,
since the executable can't be deleted while it is still running.

Fixes #2710
@natebosch
natebosch requested a review from a team as a code owner September 19, 2026 02:21
@github-actions

github-actions Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

PR Health

Changelog Entry ✔️
Package Changed Files

Changes to files need to be accounted for in their respective changelogs.

This check can be disabled by tagging the PR with skip-changelog-check.

@natebosch
natebosch marked this pull request as draft September 19, 2026 02:22
Keep the signal subscription active during graceful shutdown so a second
signal can be handled explicitly in Dart before exiting with the signal
exit code, and cancel the subscription in the finally block once the
runner closes. Add logic to determine the exit code that would have been
used by the Dart VM when it was handling the second signal.
This prepares for an upcoming change which improves temporary directory
cleanup during shutdown.

The `runTests` and `completeShutdown` methods are unused externally, so
remove and inline them. Move the `signalSubscription` global variable to
a local variable in the main method. Drop the global `isShutdown`
variable. Reset `_stdinLines` in `cancelStdinLines` so it is safe to
call more than once.

Closes #1615
Take the signal handling refactor from the base branch, keeping only the
temporary directory cleanup on top of it:

* The signal subscription is now a late final local cancelled by the
  listener itself, so the second signal handler only adds the synchronous
  delete of the temporary directory before exiting.
* Keep the base branch's exit code mapping and its version of the
  'runs tests successfully more than once' runner test, which now calls
  'main' twice instead of the removed 'runTests'.
Stack this on the temporary directory cleanup:

* The per-suite 'exe'/'cli' directory keeps its short name from the base
  branch, since it is part of a unix socket path, while still being a
  'Directory' that is deleted once the test process exits.
* Take the landed frontend server client fake, which communicates the
  completer for each compile through a stream, and respond through it in
  the compile helper for the release tests.
* Keep the changelog entries from both branches.
@natebosch
natebosch changed the base branch from master to temp-directory-cleanup September 24, 2026 00:24
Base automatically changed from temp-directory-cleanup to master September 24, 2026 21:00
Reset `_dillToCacheIsReleased` alongside the `_dillToCache` assignment,
before awaiting the delete of the previous file, so it is clear the new
file is never considered released.

Describe killing the isolate before deleting its kernel file as best
effort, since `Isolate.kill` does not wait for the isolate to exit.

Narrow the `test` changelog entry to compiled artifacts; the small
bootstrap files are still retained per test file until the platform is
closed.
The per-suite cleanup callbacks were typed `void Function()`, so the
futures from tear-offs like `serverSocket.close` and `sink.close` were
silently dropped, and the artifact deletes needed explicit `unawaited`.
Nothing waited for any of it, so cleanup could still be running while
the platform and runner deleted the directories underneath it, and any
error surfaced as an uncaught zone error with no link to its cause.

Type the callbacks as `FutureOr<void> Function()` and record every
future they return, along with those from the load error paths, in a
set owned by the platform. They are still not awaited when a suite
finishes, since a delete which waits for the test process to exit
would otherwise hang the suite if the process ignores SIGTERM. Instead,
`close` waits for them before disposing the compiler and deleting the
temp directory, and first sends SIGKILL to any test process which has
not yet exited so that one can't hang `close` either. Errors are kept
and surface from `close`.

Close the suite's sink in a `finally` so that a failure while waiting
for the debugger after the tests can no longer leave the suite
unfinished.
Deleting a suite's `exe` or `cli` output directory already waited for
the test process to exit, so it did not need to wait for the suite to
finish as well. Start it as soon as the process starts and track it
directly, rather than as a cleanup callback.

The process is now tracked for its whole lifetime, so `close` can kill
it even if its suite never finished loading and never ran the callback
that asks it to exit. Previously such a process was left running, for
instance when `--debug` rejected the `exe` compiler after the process
had already started. The directory is also deleted as soon as the
process exits, even if that is before the suite finishes loading.
Loading an `exe` or `cli` suite waited for the test process to connect
to the runner's socket with nothing else to stop the wait, so if the
process exited first - a crash on startup, or being killed while the
platform closes - the load never finished. Since that load could never
be closed, a graceful shutdown hung on it too; the only way out was a
second signal.

Race the connection against the process exiting, and fail the load with
a `LoadException` naming the exit code if the process wins. If the
platform is closing, return `null` instead, matching how isolate suites
handle a failure during close.

Verified manually by making the native bootstrap exit before connecting:
the suite now fails to load with "The test process exited with code 3
before connecting to the test runner.", where previously the run hung
and did not exit on SIGTERM.
Record every error from tracked cleanup as it happens, and report each
one as a warning with its stack trace once the platform is closed.
Previously `close` rethrew only the first, which failed `Runner.close`
and skipped deleting the runner's temp directory. That delete is now
also in a `finally`, so it happens even if closing fails.

`load` now runs a suite's cleanup callbacks whenever loading fails or
returns `null`, and each resource registers its cleanup as it is
acquired. This replaces the separate error handling after each step,
and covers failures after an isolate is spawned, such as during
`--debug` setup, which previously left the isolate running and its
kernel file unreleased.

Don't start a test process once the platform is closing, and kill one
that started while it closed, since nothing else would.

Share one best-effort delete, `tryDeleteWithRetry`, between the VM
platform and the test compiler.
@natebosch
natebosch marked this pull request as ready for review September 29, 2026 00:23
@natebosch
natebosch merged commit 40a5ad9 into master Sep 29, 2026
71 checks passed
@natebosch
natebosch deleted the cleanup-compilation-artifacts branch September 29, 2026 19:07
copybara-service Bot pushed a commit to dart-lang/sdk that referenced this pull request Oct 7, 2026
Revisions updated by `dart tools/rev_sdk_deps.dart`.

core (https://github.com/dart-lang/core/compare/08e4022..4d80610):
  4d80610a  Thu Oct 1 22:03:38 2026 +0000  dependabot[bot]  Bump the github-actions group with 5 updates (dart-lang/core#1005)

dartdoc (https://github.com/dart-lang/dartdoc/compare/2a040a8..5eb0f2c):
  5eb0f2c7  Thu Oct 1 19:03:48 2026 +0000  dependabot[bot]  Bump github/codeql-action/upload-sarif from 4.37.9 to 4.38.2 in the github-actions group (dart-lang/dartdoc#4291)

http (https://github.com/dart-lang/http/compare/d8e8e9c..585d433):
  585d433  Mon Oct 5 16:12:54 2026 -0700  Brian Quinlan  fix(cupertino_http): fix a websocket memory leak (dart-lang/http#1991)
  337adf3  Mon Oct 5 16:03:25 2026 -0700  Brian Quinlan  fix(cupertino_http): add `autoReleasePool` use to `CupertinoClient` (dart-lang/http#1990)
  47c57df  Tue Sep 29 14:07:01 2026 -0700  Brian Quinlan  fix(ok_http): fix `NetworkOnMainThreadException` (dart-lang/http#1995)
  d8cde5d  Tue Sep 29 14:06:44 2026 -0700  Brian Quinlan  fix test (dart-lang/http#1996)
  31056fe  Tue Sep 29 13:57:20 2026 -0700  Brian Quinlan  chore(cronet_http): prepare to release 1.10.0 (dart-lang/http#1993)
  9bc3a91  Tue Sep 29 13:53:25 2026 -0700  Brian Quinlan  fix(cronet_http): one global reference per `onReadCompleted` (dart-lang/http#1992)
  49ecd36  Mon Sep 28 14:20:26 2026 -0700  Brian Quinlan  fix(conformance): `supportsAbort` not considered in response (dart-lang/http#1994)

native (https://github.com/dart-lang/native/compare/27f7678..a2a6179):
  a2a617910  Tue Oct 6 13:53:17 2026 +0200  Daco Harkes  [jni_flutter] Bump version for publishing (dart-lang/native#3730)
  d3a99d66b  Tue Oct 6 19:46:34 2026 +1100  Liam Appelbe  [infra] Fix ffigen and jni_flutter (dart-lang/native#3724)
  b66d0815a  Tue Oct 6 04:28:35 2026 +0300  Manar Elhabbal  [ffigen] add Support passing C++ classes by value  (dart-lang/native#3677)
  f7ffe9667  Fri Oct 2 11:33:49 2026 +0200  Moritz  [native_toolchain_c] Don't pass /INCLUDE per symbol with a generated .def on Windows (dart-lang/native#3718)
  77a5e4169  Thu Oct 1 10:56:45 2026 +0300  Hassnaa Mohamed  [ffigen] Fix cpp method filtering (dart-lang/native#3696)
  cca065ca6  Thu Oct 1 10:17:12 2026 +1000  Liam Appelbe  [infra] Prepare to publish everything (dart-lang/native#3716)
  f13623ee3  Thu Oct 1 10:16:39 2026 +1000  Liam Appelbe  [jnigen] API docs (dart-lang/native#3715)
  be618790c  Thu Oct 1 10:05:26 2026 +1000  Liam Appelbe  [objective_c] API docs (dart-lang/native#3708)
  a18dd1148  Thu Oct 1 10:04:38 2026 +1000  Liam Appelbe  [ffigen] API docs (dart-lang/native#3707)
  6fd87c3b2  Tue Sep 29 19:01:11 2026 -0700  Brian Quinlan  [jni] Clarify `JBuffer.asUint8List` behavior (dart-lang/native#3704)
  6d5bbcfad  Wed Sep 30 09:35:47 2026 +1000  Liam Appelbe  [jni] API documentation (dart-lang/native#3706)
  fbcbc894b  Wed Sep 30 09:35:34 2026 +1000  Liam Appelbe  [ffigen] Document autorelease issues (dart-lang/native#3705)
  f99449ec9  Tue Sep 29 09:39:27 2026 +1000  Liam Appelbe  [infra] Add swift2objc to the workspace (dart-lang/native#3700)
  31494fa08  Tue Sep 29 09:38:59 2026 +1000  Liam Appelbe  [infra] Add jni_util to the workspace (dart-lang/native#3698)

shelf (https://github.com/dart-lang/shelf/compare/e5c8dc6..b2cb503):
  b2cb503  Wed Sep 30 22:53:07 2026 -0700  Kevin Moore  Resolve shelf.io.connection_info lazily (dart-lang/shelf#542)
  b70fafa  Tue Sep 29 13:37:08 2026 -0700  Kevin Moore  Match parameterless shelf_router routes without running a regex (dart-lang/shelf#544)

test (https://github.com/dart-lang/test/compare/4f92d53..d749b10):
  d749b104  Mon Oct 5 15:42:46 2026 -0700  Nate Bosch  Fail isolate suites when the test isolate exits (dart-lang/test#2775)
  da2157e9  Mon Oct 5 11:12:43 2026 -0700  Nate Bosch  Wait longer for the Chrome test tab to appear (dart-lang/test#2771)
  00bd0c99  Mon Oct 5 18:50:03 2026 +0300  Yusuf İhsan Görgel  Fix broken links in architecture.md (dart-lang/test#2774)
  3be2f547  Thu Oct 1 03:46:29 2026 +0000  dependabot[bot]  Bump the github-actions group with 4 updates (dart-lang/test#2773)
  40a5ad90  Tue Sep 29 12:07:31 2026 -0700  Nate Bosch  Delete compilation artifacts once they are no longer needed (dart-lang/test#2764)
  171ad0e5  Mon Sep 28 17:29:48 2026 -0700  Jacob MacDonald  export additional types from package:test_api/backend.dart (dart-lang/test#2745)
  52b45064  Mon Sep 28 15:04:08 2026 -0700  Nate Bosch  Tag browser and node tests, relax flaky timeouts (dart-lang/test#2770)

tools (https://github.com/dart-lang/tools/compare/d87eaf7..a66d517):
  a66d517d  Tue Oct 6 09:41:32 2026 -0700  Phil Quitslund  [analytics] `DASH__SUPPRESS_ANALYTICS` env handling (dart-lang/tools#2641)
  00c6e4ad  Fri Oct 2 17:17:49 2026 +0530  shivytyahoo  Fix VersionConstraint.intersection throwing on disjoint ranges (dart-lang/tools#2622)
  d523e1f8  Thu Oct 1 11:09:16 2026 +0200  Morgan :)  [unified_analytics] Append to the log file instead of rewriting it (dart-lang/tools#2613)
  6993cbad  Wed Sep 30 14:09:32 2026 +0300  Yusuf İhsan Görgel  [yaml] Improve error messages when mixing list and key-value syntax (dart-lang/tools#2610)
  068119b4  Tue Sep 29 23:21:03 2026 +0200  Lukas Klingsbo  [pubspec_parse] Support tag_pattern for git dependencies (dart-lang/tools#2565)
  7d859ed5  Mon Sep 28 20:28:23 2026 -0700  Kevin Moore  [api_summary] Add expectApiSummaryClean and --write/--check CLI flags (dart-lang/tools#2603)

web (https://github.com/dart-lang/web/compare/ec71a4c..63e6662):
  63e6662  Sat Oct 3 00:23:21 2026 +0000  Natalie Weizenbaum  Add a JSPair type (dart-lang/web#586)
  2d68d90  Fri Oct 2 20:26:48 2026 +0000  Natalie Weizenbaum  Use `js_interop` APIs in `js_interop_gen` (dart-lang/web#585)
  037823c  Thu Oct 1 21:59:21 2026 +0000  Natalie Weizenbaum  Add JSMap, JSSet, and JSSetLike types (dart-lang/web#571)
  b2d1d6b  Thu Oct 1 06:57:03 2026 +0000  dependabot[bot]  Bump the github-actions group with 3 updates (dart-lang/web#584)
  8dc8f75  Wed Sep 30 08:12:18 2026 -0700  Kevin Moore  [js_interop_gen] Split union/intersection AST types and extract TypeTransformer (dart-lang/web#580)
  751ccdf  Wed Sep 30 08:12:18 2026 -0700  Kevin Moore  [js_interop_gen] Decompose `Transformer._transformType` and deduplicate type caching (dart-lang/web#579)
  6f3dd4e  Wed Sep 30 08:12:17 2026 -0700  Kevin Moore  [js_interop_gen] Deduplicate `UnionType` and `IntersectionType` via `UnionOrIntersectionType` (dart-lang/web#578)
  d9f095e  Tue Sep 29 12:17:29 2026 -0700  Kevin Moore  [js_interop_gen] Remove unused JS interop extension types and bindings (dart-lang/web#577)
  07111ee  Tue Sep 29 09:51:36 2026 -0700  Kevin Moore  fix(js_interop_gen): update js_type_supertypes for Dart 3.12 (dart-lang/web#582)

Change-Id: I4532049c45561fe8f8c57713ac8ec2153096fd9a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/560660
Commit-Queue: Nate Bosch <[email protected]>
Reviewed-by: Brian Quinlan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dart test temporary disk usage grows linearly with test file count

2 participants