Skip to content

[2.x] perf: Cache compiler bridge binary#8377

Merged
eed3si9n merged 1 commit intosbt:developfrom
eed3si9n:wip/compiler-bridge
Nov 22, 2025
Merged

[2.x] perf: Cache compiler bridge binary#8377
eed3si9n merged 1 commit intosbt:developfrom
eed3si9n:wip/compiler-bridge

Conversation

@eed3si9n
Copy link
Copy Markdown
Member

@eed3si9n eed3si9n commented Nov 19, 2025

Ref #8360

Problem

Compiler bridge resolution calls out to Coursier, and it shows up on the profiler.

Solution

This uses sbt 2.x caching to cache the prebuilt compiler bridge binaries to avoid calling Coursier from second time onwards.

Test

Before

2±1s on sbt/sttp running Test/compile.

After

2±0s on sbt/sttp running Test/compile after two warmups.

sbt:sttp> Test/compile
[success] elapsed time: 13 s, cache 100%, 2877 disk cache hits
sbt:sttp> Test/compile
[success] elapsed time: 4 s, cache 100%, 2877 disk cache hits
sbt:sttp> Test/compile
[success] elapsed time: 2 s, cache 100%, 2877 disk cache hits
sbt:sttp> Test/compile
[success] elapsed time: 2 s, cache 100%, 2877 disk cache hits
sbt:sttp> Test/compile
[success] elapsed time: 2 s, cache 100%, 2877 disk cache hits
sbt:sttp> Test/compile
[success] elapsed time: 2 s, cache 100%, 2877 disk cache hits
sbt:sttp> Test/compile
[success] elapsed time: 2 s, cache 100%, 2877 disk cache hits

**Problem**
Compiler bridge resolution calls out to Coursier,
and it shows up on the profiler.

**Solution**
This uses sbt 2.x caching to cache the prebuilt compiler bridge binaries
to avoid calling Coursier from second time onwards.
@eed3si9n eed3si9n force-pushed the wip/compiler-bridge branch from 7aceeb3 to 68b2b7d Compare November 19, 2025 08:26
@eed3si9n eed3si9n merged commit fe6e9b4 into sbt:develop Nov 22, 2025
14 checks passed
@eed3si9n eed3si9n deleted the wip/compiler-bridge branch November 22, 2025 20:30
@azdrojowa123
Copy link
Copy Markdown
Contributor

Hello,
I noticed a small issue/inconsistency with this implementation - when there is a fresh project with no disk cache at all (I manually removed the AC & CAS directories), the target/out/jvm/scala-3.6.2/some-name/compiler-bridge directory contains the actual bridge jar (not a symlink). However, if I remove the target directory and run the scalaCompilerBridgeBin task again, the compiler-bridge directory then contains a symlink to the CAS store.

Because of this, I've been experiencing some flaky tests in the Scala plugin - sometimes the jar is in the project directory, and other times it's in the CAS store. I can try to handle this flakiness, but do you think it's okay that this behavior isn't consistent?

@eed3si9n
Copy link
Copy Markdown
Member Author

eed3si9n commented Jan 7, 2026

In general how is the flakiness manifested? Do you have some code that's following the symlink? If we want this cached, I guess we can try to make symlink even for the first usage?

@azdrojowa123
Copy link
Copy Markdown
Contributor

In the sbt-structure plugin, there are tests that check the compiler bridge jar location from the scalaCompilerBridgeBin task. Sometimes, for the first test in a class, it reports the jar location as something like.../v2/cas/sha256-7ad43c60ac9617fe6bf0e7706bae9dce2492d25bf4a3b7c18eb8e2b7593fbb0e-31929 and other times as .../target/out/jvm/scala-3.6.2/some-name/compiler-bridge/scala3-sbt-bridge-3.6.2.jar.

This likely depends on the test order, but I'm not entirely sure. Either way, this behavior is also reproducible in any project, not just in tests.

I guess we can try to make symlink even for the first usage

If you think it's technically reasonable, then it should fix the problem.

@eed3si9n
Copy link
Copy Markdown
Member Author

eed3si9n commented Jan 7, 2026

I've split this into a new issue - #8445

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants