[2.x] feat: Add dependency lock file support (#2989)#8581
Merged
eed3si9n merged 26 commits intosbt:developfrom Jan 22, 2026
Merged
[2.x] feat: Add dependency lock file support (#2989)#8581eed3si9n merged 26 commits intosbt:developfrom
eed3si9n merged 26 commits intosbt:developfrom
Conversation
eed3si9n
reviewed
Jan 20, 2026
This commit implements dependency lock file functionality for sbt, enabling reproducible builds by locking dependency versions. New features: - dependencyLock task: Generates build.sbt.lock from current resolution - dependencyLockCheck task: Validates lock file matches current dependencies - dependencyLockUpdate task: Forces re-resolution and updates lock file - useDependencyLock setting: When true, uses locked versions during resolution Implementation: - DependencyLock.scala: Data model and JSON serialization using sjsonnew - DependencyLockManager.scala: Read/write/validate utilities - Keys.scala: New task and setting keys - Defaults.scala: Task implementations - LMCoursier.scala: Integration with Coursier's forceVersions Fixes sbt#2989
c35742c to
3198b24
Compare
The forceVersions approach doesn't provide performance benefits since resolution still runs. Per @eed3si9n's feedback, the proper approach requires passing the resolution object to Coursier for fetching only. This commit removes the forceVersions integration and keeps the lock file as a validation/documentation tool. The resolution-skipping feature can be added as a follow-up with proper Coursier integration.
This commit adds the core infrastructure for dependency lock files that can skip Coursier resolution when a valid lock file exists, as specified in issue sbt#2989. New files in lm-coursier: - LockFileData.scala: Data model for lock file (DependencyLock, ConfigurationLock, etc.) - LockFileFormats.scala: JSON serialization using sjson-new - LockFile.scala: Read/write utilities for lock files - BuildClock.scala: SHA-1 computation for dependency validation - ResolutionSerializer.scala: Extract/reconstruct Resolution from lock data - LockFileSpec.scala: Unit tests for lock file serialization Modified: - ResolutionRun.scala: Added resolutionsWithLockFile() method The key feature is the ability to reconstruct Coursier Resolution objects from lock file data, enabling resolution skipping when the lock file is valid (build clock matches).
…skipping - Add lockFile field to CoursierConfiguration (with @SInCE for binary compat) - Modify CoursierDependencyResolution.update to use resolutionsWithLockFile - When lock file is valid, skip resolution and use cached data - When lock file is missing/stale, perform resolution and write new lock file - Update syntax/package.scala to include lockFile parameter
- Add ArtifactLock case class to store artifact URLs for direct fetching - Update DependencyLock to include artifacts field - Add JSON codec for ArtifactLock - Fix BuildClock to include exclusions and strict settings for deterministic hashing - Sort forceVersion entries for deterministic hash computation - Update ResolutionSerializer.extractLockFileData to accept artifact map - Update CoursierDependencyResolution to pass artifact URLs to lock file - Update tests for new artifacts field These fixes ensure the lock file contains enough information for artifact fetching without requiring full resolution.
- Add LockedArtifactsRun to fetch artifacts directly from lock file URLs - Add resolutionsWithLockFileData method to return lock file data for artifact fetching - Modify CoursierDependencyResolution to use locked artifacts when lock file is valid - Fallback to normal artifact fetching if locked artifact fetch fails When a valid lock file exists, both resolution AND artifact discovery are skipped - artifacts are fetched directly from the stored URLs. This provides the full performance benefit of resolution skipping.
- Add lockFile parameter to LMCoursier.coursierConfiguration method - Pass useDependencyLock and dependencyLockFile settings to CoursierConfiguration - When useDependencyLock is true, the lock file path is passed to Coursier - For sbt classifier and compiler bridge tasks, pass None (no lock file)
Test verifies that: - Lock file is created on first update - Subsequent updates use the lock file for resolution skipping - Compile works with locked dependencies
Contributor
Author
|
@eed3si9n CI test failure is none of my changes |
eed3si9n
reviewed
Jan 21, 2026
eed3si9n
reviewed
Jan 21, 2026
lm-coursier/src/main/scala/lmcoursier/internal/LockFileData.scala
Outdated
Show resolved
Hide resolved
eed3si9n
reviewed
Jan 21, 2026
eed3si9n
reviewed
Jan 21, 2026
main/src/main/scala/sbt/internal/librarymanagement/DependencyLock.scala
Outdated
Show resolved
Hide resolved
eed3si9n
reviewed
Jan 21, 2026
eed3si9n
reviewed
Jan 21, 2026
sbt-app/src/sbt-test/dependency-management/dependency-lock/project/build.properties
Outdated
Show resolved
Hide resolved
eed3si9n
reviewed
Jan 21, 2026
eed3si9n
reviewed
Jan 21, 2026
eed3si9n
reviewed
Jan 21, 2026
- Rename lock file to deps.lock - Use baseDirectory instead of ThisBuild for per-project lock files - Remove useDependencyLock setting - lock file used automatically if exists - Generate LockFileFormats with Contraband codec - Add InstantFormats for java.time.Instant serialization - Update scripted tests for new lock file name
eed3si9n
reviewed
Jan 21, 2026
eed3si9n
reviewed
Jan 21, 2026
sbt-app/src/sbt-test/dependency-management/dependency-lock/changes/build-modified.sbt
Show resolved
Hide resolved
eed3si9n
reviewed
Jan 21, 2026
eed3si9n
reviewed
Jan 21, 2026
eed3si9n
reviewed
Jan 21, 2026
- Use manual LockFileFormats instead of Contraband-generated codecs - Remove dependencyLockUpdate task - use dependencyLock to regenerate - Change dependencyLockCheck to return Unit and throw MessageOnlyException - Add csrCacheDirectory to scripted tests - Remove codec generation from lockfile.contra
eed3si9n
reviewed
Jan 21, 2026
eed3si9n
reviewed
Jan 21, 2026
- Remove duplicate lock file types from DependencyLock.scala - Update DependencyLockManager to use lm-coursier LockFileData types - Simplify dependencyLockTask to use unified API
- Pass silently if lock file doesn't exist - Pass silently if lock file is valid - Only throw exception if lock file exists but is stale
- Rename lock file check from build.sbt.lock to deps.lock - Remove dependencyLockUpdate usage (use dependencyLock instead) - dependencyLockCheck passes silently if no lock file or valid - dependencyLockCheck throws exception only if lock file is stale
The lock file is created by dependencyLock, not update.
eed3si9n
reviewed
Jan 21, 2026
lm-coursier/src/main/scala/lmcoursier/CoursierDependencyResolution.scala
Show resolved
Hide resolved
eed3si9n
reviewed
Jan 21, 2026
lm-coursier/src/main/scala/lmcoursier/internal/ResolutionSerializer.scala
Outdated
Show resolved
Hide resolved
- Convert local cache file URLs back to original HTTP URLs for portability - Remove timestamp from LockFileMetadata for repeatable lock files - Remove InstantFormats since timestamp is no longer used
eed3si9n
reviewed
Jan 21, 2026
eed3si9n
reviewed
Jan 21, 2026
sbt-app/src/sbt-test/dependency-management/dependency-lock/project/build.properties
Outdated
Show resolved
Hide resolved
eed3si9n
reviewed
Jan 21, 2026
sbt-app/src/sbt-test/dependency-management/resolution-skip/project/build.properties
Outdated
Show resolved
Hide resolved
eed3si9n
approved these changes
Jan 21, 2026
Member
eed3si9n
left a comment
There was a problem hiding this comment.
This is a cool feature enhancement. Thanks for the contribution!
- Improve cacheFileToOriginalUrl to handle symlinks and use ${CSR_CACHE} fallback
- Add unit tests for URL conversion
- Remove build.properties from scripted tests
Contributor
Author
|
@eed3si9n please review the changes again |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dependency Lock File Support
This PR adds dependency lock file support to sbt 2.x.
What it does
When you run
dependencyLock, sbt generates adeps.lockfile that captures your resolved dependencies. This file can be checked into version control to ensure reproducible builds across different machines and CI environments.New tasks
dependencyLock- Generates the lock file from the current resolutiondependencyLockCheck- Validates the lock file is up-to-date (fails build if stale)How it works
The lock file stores a hash of your declared dependencies and resolvers. When dependencies change, the hash changes, and
dependencyLockCheckwill fail until you regenerate the lock file.If no lock file exists,
dependencyLockCheckpasses silently - this allows gradual adoption.Implementation notes
LockFileDatatypes with the user-facing lock file formatbaseDirectory, notThisBuild)Fixes #2989