[2.x] feat: Cache failed compilation to avoid repeated failures by MkDev11 · Pull Request #8490 · sbt/sbt

MkDev11 · 2026-01-12T02:31:10Z

When a compilation fails with CompileFailed, the failure is now cached so that subsequent builds with the same inputs don't re-run the failed compilation. This significantly improves the experience when using BSP clients like Metals that may trigger many compilations in a row.

The implementation:

Adds CachedCompileFailure, CachedProblem, and CachedPosition types to serialize compilation failures
Modifies ActionCache.cache to catch CompileFailed exceptions and store them in the cache with exitCode=1
On cache lookup, checks for cached failures first and re-throws the cached exception if found
Fixes DiskActionCacheStore.put to preserve exitCode from request
Adds unit test to verify cached failure behavior

Fixes #7662

Contribution by Gittensor, see my contribution statistics at https://gittensor.io/miners/details?githubId=94194147

When a compilation fails with CompileFailed, the failure is now cached so that subsequent builds with the same inputs don't re-run the failed compilation. This significantly improves the experience when using BSP clients like Metals that may trigger many compilations in a row. The implementation: - Adds CachedCompileFailure, CachedProblem, and CachedPosition types to serialize compilation failures - Modifies ActionCache.cache to catch CompileFailed exceptions and store them in the cache with exitCode=1 - On cache lookup, checks for cached failures first and re-throws the cached exception if found - Fixes DiskActionCacheStore.put to preserve exitCode from request - Adds unit test to verify cached failure behavior Fixes sbt#7662

util-cache/src/main/scala/sbt/util/ActionCache.scala

util-cache/src/main/scala/sbt/util/CachedCompileFailure.scala

MkDev11 · 2026-01-12T05:04:08Z

all passed!

...${OUT}/value/sha256-8769430c2339a7cbaaa5b190624c7baa20f54d092424131f0055699b423ce6cc/48.json

Address review feedback from eed3si9n: 1. Use existing ProblemFormats, SeverityFormats, and PositionFormats instead of custom CachedProblem and CachedPosition types. This simplifies CachedCompileFailure significantly. 2. Use single cache lookup with exitCode to distinguish success/failure. Avoids 2x network calls when remote cache is configured. Both success and failure use the same input digest. exitCode=0 for success, exitCode=1 for failure.

util-cache/src/main/scala/sbt/util/CachedCompileFailure.scala

eed3si9n · 2026-01-12T06:02:21Z

I think we should keep going with this idea, but one concern I have is that sometimes people use compile to display all the compilation warnings and errors, and I'm not sure cached failure captures all that details.

…band Address review feedback from eed3si9n: 1. Add GenericFailure type to actionresult.contra for binary compatibility. The type uses a 'kind' field to distinguish different failure types (e.g., 'CompileFailed') for future extensibility. 2. CachedCompileFailure now wraps GenericFailure instead of being a standalone case class. 3. Custom JsonFormat provided since xsbti.ProblemFormats doesn't exist in the xsbti package (it's in sbt.internal.util.codec).

MkDev11 · 2026-01-12T06:54:03Z

all passed! please review it again and approve if it's ok

eed3si9n · 2026-01-12T07:32:05Z

I built sbt locally and tried to use it:

sbt:aaa> compile
[info] compiling 2 Scala sources to /private/tmp/aaa/target/out/jvm/scala-3.7.3/aaa/classes ...
[error] -- [E103] Syntax Error: /private/tmp/aaa/src/main/scala/example/A.scala:3:0 ----
[error] 3 |aaclass A
[error]   |^^^^^^^
[error]   |Illegal start of toplevel definition
[error]   |
[error]   | longer explanation available when compiling with `-explain`
[error] -- [E040] Syntax Error: /private/tmp/aaa/src/main/scala/example/Main.scala:5:0 -
[error] 5 |
[error]   |^
[error]   |')' expected, but eof found
[error] two errors found
[error] (Compile / compileIncremental) Compilation failed
[error] elapsed time: 0 s, cache 84%, 1 disk cache hit, 10 symlink cache hits, 2 onsite tasks

sbt:aaa> compile
[error] stack trace is suppressed; run last Compile / compileIncremental for the full output
[error] (Compile / compileIncremental) sbt.util.CachedCompileFailure$$anon$1: Compilation failed
[error] elapsed time: 0 s, cache 100%, 1 cached-failure cache hit, 1 disk cache hit, 11 symlink cache hits

For command-line usage, I don't think this is usable without also relaying the problem information back to the log. Using

scalaVersion := "3.7.3"

the problem list contains Problem structure, and Scala 3 seems to store the pretty-printed error in rendered field to be replayed, but with

scalaVersion := "2.13.18"

the problem list contains Vector(Optional.empty, Optional.empty). So maybe what we should do is inspect the problem list before caching.

Also for Metals usage, where OP originally wanted this feature to be useful, I wonder if we also need to replay the problem to LSP reporter so VS Code would have it. I haven't tested this with Metals yet.

Address eed3si9n's feedback that cached failures weren't displaying errors/warnings on subsequent compile runs. Changes: 1. Add replay() method to CachedCompileFailure that logs problems to the console when a cached failure is thrown. 2. Add formatProblem() that handles both Scala 3 (uses rendered field) and Scala 2.13 (constructs message from position + message). 3. Add hasSufficientInfo() to check if problems have enough info to be useful when replayed (for future use). 4. Call replay() in ActionCache before throwing cached failure.

MkDev11 · 2026-01-12T07:51:47Z

I built sbt locally and tried to use it:
sbt:aaa> compile
[info] compiling 2 Scala sources to /private/tmp/aaa/target/out/jvm/scala-3.7.3/aaa/classes ...
[error] -- [E103] Syntax Error: /private/tmp/aaa/src/main/scala/example/A.scala:3:0 ----
[error] 3 |aaclass A
[error]   |^^^^^^^
[error]   |Illegal start of toplevel definition
[error]   |
[error]   | longer explanation available when compiling with `-explain`
[error] -- [E040] Syntax Error: /private/tmp/aaa/src/main/scala/example/Main.scala:5:0 -
[error] 5 |
[error]   |^
[error]   |')' expected, but eof found
[error] two errors found
[error] (Compile / compileIncremental) Compilation failed
[error] elapsed time: 0 s, cache 84%, 1 disk cache hit, 10 symlink cache hits, 2 onsite tasks
sbt:aaa> compile
[error] stack trace is suppressed; run last Compile / compileIncremental for the full output
[error] (Compile / compileIncremental) sbt.util.CachedCompileFailure$$anon$1: Compilation failed
[error] elapsed time: 0 s, cache 100%, 1 cached-failure cache hit, 1 disk cache hit, 11 symlink cache hits
For command-line usage, I don't think this is usable without also relaying the problem information back to the log. Using
scalaVersion := "3.7.3"
the problem list contains Problem structure, and Scala 3 seems to store the pretty-printed error in rendered field to be replayed, but with
scalaVersion := "2.13.18"
the problem list contains Vector(Optional.empty, Optional.empty). So maybe what we should do is inspect the problem list before caching.

Also for Metals usage, where OP originally wanted this feature to be useful, I wonder if we also need to replay the problem to LSP reporter so VS Code would have it. I haven't tested this with Metals yet.

Thanks for testing! I've added a replay() method that logs the cached problems before throwing the exception. It uses the rendered field when available (Scala 3), and falls back to formatting from position + message for Scala 2.13.

For Metals/LSP, I haven't tested that yet - will look into whether we need to replay to the BSP reporter as well.

MkDev11 · 2026-01-12T07:56:30Z

For Metals/LSP, I think the issue is that when a cached CompileFailed is thrown from ActionCache, it bypasses the BSP reporter notification (sendFailureReport/notifyFailure).

One option would be to wrap the cached compile task with a try-catch that calls the BSP methods on CompileFailed. But I'm not sure if that's the right approach architecturally - would appreciate your guidance on how you'd like this handled.

eed3si9n · 2026-01-12T16:14:20Z

Looking at the code, TaskZero / compileIncremental is uncached, so it's possible that we already have that try catch for us. You can put println temporarily in there to check:

sbt/main/src/main/scala/sbt/Defaults.scala

Lines 2188 to 2192 in add43bd

    
           case Result.Inc(cause) => 
        
             val compileFailed = cause.directCause.collect { case c: CompileFailed => c } 
        
             reporter.sendFailureReport(ci.options.sources) 
        
             bspTask.notifyFailure(compileFailed) 
        
             throw cause

MkDev11 · 2026-01-12T19:19:29Z

I tested with println and confirmed that the cached CachedCompileFailure does flow through the Result.Inc handler in Defaults.scala lines 2188-2192. The compileFailed is correctly extracted and bspTask.notifyFailure() is called.

Debug output on second compile:

[DEBUG] Result.Inc caught: Some(sbt.util.CachedCompileFailure$$anon$1: Compilation failed)
[DEBUG] compileFailed extracted: Some(sbt.util.CachedCompileFailure$$anon$1: Compilation failed)

So the BSP reporter notification should already work for Metals without additional changes. The error replay to the log is also working as shown in the output.

eed3si9n

Thanks!

util-cache/src/main/scala/sbt/util/ActionCache.scala

MkDev11 · 2026-01-12T20:59:51Z

All passed! Can you merge it now?

MkDev11 added 2 commits January 12, 2026 03:29

Merge branch 'develop' into fix/cache-failed-compilation-7662

e60c44b

eed3si9n reviewed Jan 12, 2026

View reviewed changes

util-cache/src/main/scala/sbt/util/ActionCache.scala Outdated Show resolved Hide resolved

eed3si9n reviewed Jan 12, 2026

View reviewed changes

util-cache/src/main/scala/sbt/util/CachedCompileFailure.scala Outdated Show resolved Hide resolved

eed3si9n reviewed Jan 12, 2026

View reviewed changes

...${OUT}/value/sha256-8769430c2339a7cbaaa5b190624c7baa20f54d092424131f0055699b423ce6cc/48.json Outdated Show resolved Hide resolved

MkDev11 force-pushed the fix/cache-failed-compilation-7662 branch from b168702 to 794279d Compare January 12, 2026 05:29

Merge branch 'develop' into fix/cache-failed-compilation-7662

93b4a88

eed3si9n reviewed Jan 12, 2026

View reviewed changes

util-cache/src/main/scala/sbt/util/CachedCompileFailure.scala Outdated Show resolved Hide resolved

eed3si9n changed the title ~~feat: Cache failed compilation to avoid repeated failures~~ [2.x] feat: Cache failed compilation to avoid repeated failures Jan 12, 2026

MkDev11 added 2 commits January 12, 2026 14:26

Merge branch 'develop' into fix/cache-failed-compilation-7662

0802f69

chore: trigger CI

c9a6308

eed3si9n approved these changes Jan 12, 2026

View reviewed changes

util-cache/src/main/scala/sbt/util/ActionCache.scala Outdated Show resolved Hide resolved

refactor: Remove match block for cache storage errors per review

32a4a36

MkDev11 requested a review from eed3si9n January 12, 2026 21:00

eed3si9n merged commit fe6125d into sbt:develop Jan 12, 2026
14 checks passed

Conversation

MkDev11 commented Jan 12, 2026

Uh oh!

Uh oh!

Uh oh!

MkDev11 commented Jan 12, 2026

Uh oh!

Uh oh!

Uh oh!

eed3si9n commented Jan 12, 2026

Uh oh!

MkDev11 commented Jan 12, 2026

Uh oh!

eed3si9n commented Jan 12, 2026

Uh oh!

MkDev11 commented Jan 12, 2026

Uh oh!

MkDev11 commented Jan 12, 2026

Uh oh!

eed3si9n commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MkDev11 commented Jan 12, 2026

Uh oh!

eed3si9n left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MkDev11 commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

eed3si9n commented Jan 12, 2026 •

edited

Loading

MkDev11 commented Jan 12, 2026 •

edited

Loading