feat: Add cold file tiering support #1042
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
This commit implements the foundation for access-based cold file tiering: 1. Add last_access_time field to FileInfo and ObjectInfo - Track when objects were last accessed for cold detection - Propagate access time through from_file_info conversion 2. Add LAST_ACCESS_TIME metadata constant - New constant for persisting access time in object metadata 3. Create cold_tier_config module - ColdTierConfig struct for cold tier settings - GlobalColdTierConfig for runtime configuration - Environment variable support (RUSTFS_COLD_TIER_*) - Configurable threshold days, target tier, min file size - Access time sampling rate to reduce write amplification 4. Extend ObjectOpts with access-based lifecycle support - Add access_time field for lifecycle evaluation - Add is_cold() method for cold file detection - Add effective_access_time() helper method Closes rustfs#1022
tennisleng
force-pushed
the
feature/cold-file-tiering-1022
branch
from
abfddc3 to
d0c4ac2
Compare
This was referenced Dec 8, 2025
Contributor
Author
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements the foundation for access-based cold file tiering to address issue #1022.
Changes
1. Access Time Tracking (
FileInfoandObjectInfo)last_access_time: Option<OffsetDateTime>field to track when objects are last accessedfrom_file_infoto propagate access time through conversionsCloneimplementation forObjectInfo2. Metadata Constant (
filemeta.rs)LAST_ACCESS_TIMEconstant (x-rustfs-last-access-time) for persisting access time in object metadata3. Cold Tier Configuration Module (
cold_tier_config.rs)New module for configuring cold file tiering:
ColdTierConfigstruct with configurable settingsGlobalColdTierConfigfor runtime configurationRUSTFS_COLD_TIER_ENABLED- Enable/disable cold tieringRUSTFS_COLD_TIER_THRESHOLD_DAYS- Days since last access to mark as cold (default: 30)RUSTFS_COLD_TIER_TARGET- Target tier name for cold filesRUSTFS_COLD_TIER_MIN_FILE_SIZE- Minimum file size to consider (default: 1MB)RUSTFS_COLD_TIER_SAMPLE_RATE- Access time update sampling rate4. Lifecycle Extensions (
lifecycle.rs)access_timefield toObjectOptsfor lifecycle evaluationis_cold(threshold_days)method for cold file detectioneffective_access_time()helper methodTesting
ColdTierConfigandGlobalColdTierConfigFuture Work
This PR provides the foundation. Follow-up PRs can:
Closes #1022