Skip to content

Comments

feat(services/azblob): Add compatibility aliases for Apache Arrow object_store#6527

Merged
Xuanwo merged 2 commits intoapache:mainfrom
jackye1995:azure-alias
Aug 25, 2025
Merged

feat(services/azblob): Add compatibility aliases for Apache Arrow object_store#6527
Xuanwo merged 2 commits intoapache:mainfrom
jackye1995:azure-alias

Conversation

@jackye1995
Copy link
Contributor

@jackye1995 jackye1995 commented Aug 23, 2025

Summary

Enable seamless migration from Apache Arrow's object_store library by adding serde aliases for Azure Blob storage configuration fields. These aliases follow the same naming conventions used in the object_store Azure builder.

Supported Alias Mappings

OpenDAL Field OpenDAL Type object_store Aliases Notes
container String azure_container_name, container_name Container/bucket name
account_name Option<String> azure_storage_account_name Storage account name
account_key Option<String> azure_storage_account_key, azure_storage_access_key, azure_storage_master_key, access_key, master_key Account key for authentication
sas_token Option<String> azure_storage_sas_key, azure_storage_sas_token, sas_key Shared Access Signature token
endpoint Option<String> azure_storage_endpoint, azure_endpoint Custom endpoint URL

Unsupported Aliases

The following object_store configuration keys are not supported in OpenDAL Azure Blob service:

object_store Key Aliases Reason
client_id azure_storage_client_id, azure_client_id, client_id OAuth client credentials not implemented
client_secret azure_storage_client_secret, azure_client_secret, client_secret OAuth client credentials not implemented
tenant_id azure_storage_tenant_id, azure_storage_authority_id, azure_tenant_id, azure_authority_id, tenant_id, authority_id OAuth tenant authentication not implemented
authority_host azure_storage_authority_host, azure_authority_host, authority_host OAuth authority host not implemented
bearer_token azure_storage_token, bearer_token, token Bearer token authentication not implemented
use_emulator azure_storage_use_emulator, use_emulator Azurite emulator support not implemented
msi_endpoint azure_msi_endpoint, azure_identity_endpoint, identity_endpoint, msi_endpoint Managed identity not implemented
object_id azure_object_id, object_id Managed identity not implemented
msi_resource_id azure_msi_resource_id, msi_resource_id Managed identity not implemented
federated_token_file azure_federated_token_file, federated_token_file Workload identity not implemented
use_azure_cli azure_use_azure_cli, use_azure_cli Azure CLI authentication not implemented
skip_signature azure_skip_signature, skip_signature Public container access not implemented
use_fabric_endpoint azure_use_fabric_endpoint, use_fabric_endpoint Microsoft Fabric endpoint not implemented
disable_tagging azure_disable_tagging, disable_tagging Tagging control not implemented

Missing Configurations

OpenDAL Azure Blob service includes the following configurations that are not present in object_store:

OpenDAL Field Type Purpose
root Option<String> Root path for all operations
encryption_key Option<String> Customer-provided encryption key
encryption_key_sha256 Option<String> SHA256 hash of encryption key
encryption_algorithm Option<String> Encryption algorithm specification
batch_max_operations Option<usize> Maximum operations in batch requests

Usage Example

Users can now migrate from object_store configurations seamlessly:

// object_store style configuration
let config = AzblobConfig {
    azure_container_name: Some("my-container".to_string()),
    azure_storage_account_name: Some("myaccount".to_string()),
    azure_storage_account_key: Some("key".to_string()),
    azure_storage_endpoint: Some("https://myaccount.blob.core.windows.net".to_string()),
    ..Default::default()
};

// OpenDAL native style (still supported)
let config = AzblobConfig {
    container: "my-container".to_string(),
    account_name: Some("myaccount".to_string()),
    account_key: Some("key".to_string()),
    endpoint: Some("https://myaccount.blob.core.windows.net".to_string()),
    ..Default::default()
};

Test Plan

  • Unit tests verify all aliases work correctly through JSON deserialization
  • Clippy checks pass with no warnings
  • Code formatting applied

Closes: N/A
Related: Similar to #6524 for S3 service

🤖 Generated with Claude Code

@jackye1995 @claude
feat(services/azblob): Add compatibility aliases for Apache Arrow obj…
187a3c7 
…ect_store

Enable seamless migration from Apache Arrow's object_store library by adding
serde aliases for Azure Blob storage configuration fields. These aliases
follow the same naming conventions used in object_store.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@jackye1995 jackye1995 requested a review from Xuanwo as a code owner August 23, 2025 04:21
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. releases-note/feat The PR implements a new feature or has a title that begins with "feat" labels Aug 23, 2025
@jackye1995 @claude
test: Split config alias tests into individual functions
46b1ffd 
Split the single test_config_aliases function into 5 focused test functions:
- test_container_name_aliases: Tests container name aliases
- test_account_name_aliases: Tests account name aliases
- test_account_key_aliases: Tests all account key aliases
- test_sas_token_aliases: Tests SAS token aliases
- test_endpoint_aliases: Tests endpoint aliases

This improves test organization and makes it easier to debug specific
alias failures.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Aug 23, 2025
Xuanwo
Xuanwo approved these changes Aug 25, 2025
View reviewed changes
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for working on this!

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Aug 25, 2025
@Xuanwo Xuanwo merged commit 3e18c2b into apache:main Aug 25, 2025
88 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Xuanwo Xuanwo Xuanwo approved these changes

Assignees

No one assigned

Labels

lgtm This PR has been approved by a maintainer releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jackye1995 @Xuanwo