Skip to content

[2.x] feat: Add dependency lock file support (#2989)#8581

Merged
eed3si9n merged 26 commits intosbt:developfrom
MkDev11:feature/dependency-lock-file
Jan 22, 2026
Merged

[2.x] feat: Add dependency lock file support (#2989)#8581
eed3si9n merged 26 commits intosbt:developfrom
MkDev11:feature/dependency-lock-file

Conversation

@MkDev11
Copy link
Copy Markdown
Contributor

@MkDev11 MkDev11 commented Jan 20, 2026

Dependency Lock File Support

This PR adds dependency lock file support to sbt 2.x.

What it does

When you run dependencyLock, sbt generates a deps.lock file that captures your resolved dependencies. This file can be checked into version control to ensure reproducible builds across different machines and CI environments.

New tasks

  • dependencyLock - Generates the lock file from the current resolution
  • dependencyLockCheck - Validates the lock file is up-to-date (fails build if stale)

How it works

The lock file stores a hash of your declared dependencies and resolvers. When dependencies change, the hash changes, and dependencyLockCheck will fail until you regenerate the lock file.

If no lock file exists, dependencyLockCheck passes silently - this allows gradual adoption.

Implementation notes

  • Lock file types are generated via Contraband, with manual JSON codecs for Scala 3 compatibility
  • Unified the internal LockFileData types with the user-facing lock file format
  • Lock file is per-project (uses baseDirectory, not ThisBuild)

Fixes #2989

This commit implements dependency lock file functionality for sbt, enabling
reproducible builds by locking dependency versions.

New features:
- dependencyLock task: Generates build.sbt.lock from current resolution
- dependencyLockCheck task: Validates lock file matches current dependencies
- dependencyLockUpdate task: Forces re-resolution and updates lock file
- useDependencyLock setting: When true, uses locked versions during resolution

Implementation:
- DependencyLock.scala: Data model and JSON serialization using sjsonnew
- DependencyLockManager.scala: Read/write/validate utilities
- Keys.scala: New task and setting keys
- Defaults.scala: Task implementations
- LMCoursier.scala: Integration with Coursier's forceVersions

Fixes sbt#2989
@MkDev11 MkDev11 force-pushed the feature/dependency-lock-file branch from c35742c to 3198b24 Compare January 20, 2026 06:27
The forceVersions approach doesn't provide performance benefits since
resolution still runs. Per @eed3si9n's feedback, the proper approach
requires passing the resolution object to Coursier for fetching only.

This commit removes the forceVersions integration and keeps the lock
file as a validation/documentation tool. The resolution-skipping
feature can be added as a follow-up with proper Coursier integration.
This commit adds the core infrastructure for dependency lock files that
can skip Coursier resolution when a valid lock file exists, as specified
in issue sbt#2989.

New files in lm-coursier:
- LockFileData.scala: Data model for lock file (DependencyLock, ConfigurationLock, etc.)
- LockFileFormats.scala: JSON serialization using sjson-new
- LockFile.scala: Read/write utilities for lock files
- BuildClock.scala: SHA-1 computation for dependency validation
- ResolutionSerializer.scala: Extract/reconstruct Resolution from lock data
- LockFileSpec.scala: Unit tests for lock file serialization

Modified:
- ResolutionRun.scala: Added resolutionsWithLockFile() method

The key feature is the ability to reconstruct Coursier Resolution objects
from lock file data, enabling resolution skipping when the lock file is
valid (build clock matches).
…skipping

- Add lockFile field to CoursierConfiguration (with @SInCE for binary compat)
- Modify CoursierDependencyResolution.update to use resolutionsWithLockFile
- When lock file is valid, skip resolution and use cached data
- When lock file is missing/stale, perform resolution and write new lock file
- Update syntax/package.scala to include lockFile parameter
- Add ArtifactLock case class to store artifact URLs for direct fetching
- Update DependencyLock to include artifacts field
- Add JSON codec for ArtifactLock
- Fix BuildClock to include exclusions and strict settings for deterministic hashing
- Sort forceVersion entries for deterministic hash computation
- Update ResolutionSerializer.extractLockFileData to accept artifact map
- Update CoursierDependencyResolution to pass artifact URLs to lock file
- Update tests for new artifacts field

These fixes ensure the lock file contains enough information for
artifact fetching without requiring full resolution.
- Add LockedArtifactsRun to fetch artifacts directly from lock file URLs
- Add resolutionsWithLockFileData method to return lock file data for artifact fetching
- Modify CoursierDependencyResolution to use locked artifacts when lock file is valid
- Fallback to normal artifact fetching if locked artifact fetch fails

When a valid lock file exists, both resolution AND artifact discovery are
skipped - artifacts are fetched directly from the stored URLs. This provides
the full performance benefit of resolution skipping.
- Add lockFile parameter to LMCoursier.coursierConfiguration method
- Pass useDependencyLock and dependencyLockFile settings to CoursierConfiguration
- When useDependencyLock is true, the lock file path is passed to Coursier
- For sbt classifier and compiler bridge tasks, pass None (no lock file)
Test verifies that:
- Lock file is created on first update
- Subsequent updates use the lock file for resolution skipping
- Compile works with locked dependencies
@MkDev11 MkDev11 requested a review from eed3si9n January 21, 2026 00:48
@MkDev11
Copy link
Copy Markdown
Contributor Author

MkDev11 commented Jan 21, 2026

@eed3si9n CI test failure is none of my changes

@eed3si9n eed3si9n changed the title feat: Add dependency lock file support (#2989) [2.x] feat: Add dependency lock file support (#2989) Jan 21, 2026
- Rename lock file to deps.lock
- Use baseDirectory instead of ThisBuild for per-project lock files
- Remove useDependencyLock setting - lock file used automatically if exists
- Generate LockFileFormats with Contraband codec
- Add InstantFormats for java.time.Instant serialization
- Update scripted tests for new lock file name
- Use manual LockFileFormats instead of Contraband-generated codecs
- Remove dependencyLockUpdate task - use dependencyLock to regenerate
- Change dependencyLockCheck to return Unit and throw MessageOnlyException
- Add csrCacheDirectory to scripted tests
- Remove codec generation from lockfile.contra
- Remove duplicate lock file types from DependencyLock.scala
- Update DependencyLockManager to use lm-coursier LockFileData types
- Simplify dependencyLockTask to use unified API
- Pass silently if lock file doesn't exist
- Pass silently if lock file is valid
- Only throw exception if lock file exists but is stale
- Rename lock file check from build.sbt.lock to deps.lock
- Remove dependencyLockUpdate usage (use dependencyLock instead)
- dependencyLockCheck passes silently if no lock file or valid
- dependencyLockCheck throws exception only if lock file is stale
@MkDev11 MkDev11 requested a review from eed3si9n January 21, 2026 05:15
The lock file is created by dependencyLock, not update.
- Convert local cache file URLs back to original HTTP URLs for portability
- Remove timestamp from LockFileMetadata for repeatable lock files
- Remove InstantFormats since timestamp is no longer used
Copy link
Copy Markdown
Member

@eed3si9n eed3si9n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a cool feature enhancement. Thanks for the contribution!

- Improve cacheFileToOriginalUrl to handle symlinks and use ${CSR_CACHE} fallback
- Add unit tests for URL conversion
- Remove build.properties from scripted tests
@MkDev11
Copy link
Copy Markdown
Contributor Author

MkDev11 commented Jan 21, 2026

@eed3si9n please review the changes again

@eed3si9n eed3si9n merged commit 2a5746c into sbt:develop Jan 22, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Split dependency resolution into resolve and downloading (lock dependency)

3 participants