You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 2cd3e9c
Browse filesBrowse the repository at this point in the historyBrowse files
[refactor](fs-spi) Migrate cloud.storage Remote* hierarchy to fs.obj module (#61911)
### What problem does this PR solve?
Issue Number: #61860
Problem Summary: Eliminates duplicate object storage abstraction in the
Cloud mode codebase. Previously, cloud.storage.RemoteBase and its
subclasses (S3Remote, OssRemote, CosRemote, ObsRemote, BosRemote,
AzureRemote, etc.) provided object storage operations independently from
the fs.obj module. This change migrates all functionality into fs.obj
and removes the entire Remote* hierarchy, so all Cloud Stage/Copy
operations now go through ObjFileSystem.
Key changes:
- Add OssObjStorage, CosObjStorage, ObsObjStorage, BosObjStorage as new
ObjStorage implementations with STS token, presigned URL, list, and head
support
- Extend AzureObjStorage and S3ObjStorage with getStsToken,
getPresignedUrl, listObjectsWithPrefix, headObjectWithMeta
- Add ObjFileSystem passthrough methods: getStsToken, getPresignedUrl,
listObjectsWithPrefix, headObjectWithMeta, deleteObjectsByKeys
- Extract RemoteBase.ObjectInfo inner class to standalone
cloud.storage.ObjectInfo
- Rewrite ObjectInfoAdapter: no RemoteBase dependency; uses
ObjFileSystem for STS in ARN flow
- Migrate all callers (StageUtil, CopyLoadPendingTask, CleanCopyJobTask,
CopyIntoAction, CreateStageCommand, CopyIntoInfo) to ObjFileSystem
- Delete RemoteBase, S3Remote, OssRemote, CosRemote, ObsRemote,
BosRemote, TosRemote, AzureRemote, DefaultRemote, MockRemote
- Rewrite CopyLoadPendingTaskTest and StageUtilTest to use
MockUp<ObjFileSystem>
- Fix Maven build cache causing test execution to be skipped
(run-fe-ut.sh)
Co-authored-by: Copilot <[email protected]>
0 commit comments