feat(services/gcs): add configuration aliases for better Arrow object_store compatibility#6526
Merged
Xuanwo merged 3 commits intoapache:mainfrom Aug 25, 2025
Merged
feat(services/gcs): add configuration aliases for better Arrow object_store compatibility#6526Xuanwo merged 3 commits intoapache:mainfrom
Xuanwo merged 3 commits intoapache:mainfrom
Conversation
…_store compatibility This PR adds serde aliases to GcsConfig fields to support multiple configuration key naming conventions, enabling seamless migration from Arrow object_store and other file system libraries. Users can now use either OpenDAL's native field names or Google-prefixed alternatives. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Split large test_config_aliases into 5 focused test functions - Each test function covers aliases for a specific config field - Added missing google_service_account_path alias test - Improved test organization and readability 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Xuanwo
approved these changes
Aug 25, 2025
Member
Xuanwo
left a comment
There was a problem hiding this comment.
Thank you for working on this!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds serde aliases to GcsConfig fields to support multiple configuration key naming conventions, enabling seamless migration from Arrow object_store and other file system libraries. Users can now use either OpenDAL's native field names or Google-prefixed alternatives.
Changes
Added
#[serde(alias = "...")]attributes to 5 key GCS configuration fields, following the exact same patterns used by Apache Arrow's object_store library.Example Usage
Added Aliases
bucketgoogle_bucket,google_bucket_name,bucket_nameservice_accountgoogle_service_account,google_service_account_path,service_account_pathcredentialgoogle_service_account_key,service_account_keycredential_pathgoogle_application_credentialsallow_anonymousgoogle_skip_signature,skip_signatureMissing Configurations (Future Work)
These Arrow object_store configurations are not yet available in OpenDAL and could be added in future PRs:
HTTP Client Configuration
AllowHttpgoogle_allow_http,allow_httpAllowInvalidCertificatesgoogle_allow_invalid_certificates,allow_invalid_certificatesConnectTimeoutgoogle_connect_timeout,connect_timeoutDefaultContentTypegoogle_default_content_type,default_content_typeHttp1Onlygoogle_http1_only,http1_onlyHttp2KeepAliveIntervalgoogle_http2_keep_alive_interval,http2_keep_alive_intervalHttp2KeepAliveTimeoutgoogle_http2_keep_alive_timeout,http2_keep_alive_timeoutHttp2KeepAliveWhileIdlegoogle_http2_keep_alive_while_idle,http2_keep_alive_while_idleHttp2MaxFrameSizegoogle_http2_max_frame_size,http2_max_frame_sizeHttp2Onlygoogle_http2_only,http2_onlyPoolIdleTimeoutgoogle_pool_idle_timeout,pool_idle_timeoutPoolMaxIdlePerHostgoogle_pool_max_idle_per_host,pool_max_idle_per_hostProxyUrlgoogle_proxy_url,proxy_urlProxyCaCertificategoogle_proxy_ca_certificate,proxy_ca_certificateProxyExcludesgoogle_proxy_excludes,proxy_excludesRandomizeAddressesgoogle_randomize_addresses,randomize_addressesTimeoutgoogle_timeout,timeoutUserAgentgoogle_user_agent,user_agentNotes on Missing Configurations
Testing
Backward Compatibility
This change is fully backward compatible. All existing configurations continue to work exactly as before, with the new aliases providing additional flexibility.
Similar to #6524