feat(bindings/cpp): enable user-configurable features for storage services by KinomotoMio · Pull Request #6503 · apache/opendal

KinomotoMio · 2025-08-12T02:56:22Z

Which issue does this PR close?

Closes #6250.

Rationale for this change

Currently, the C++ binding hardcodes all storage service features in Cargo.toml, which means:

Users must compile and link all services regardless of their actual needs
Binary size is unnecessarily large
Compilation time is longer than necessary

This PR implements user-configurable features similar to the C binding (#4313), allowing users to specify only the storage services they need at build time.

What changes are included in this PR?

Modified bindings/cpp/Cargo.toml: Removed hardcoded service dependencies, keeping only the essential blocking feature
Enhanced bindings/cpp/CMakeLists.txt:
- Added FEATURES cache variable for user configuration
- Implemented intelligent feature merging logic that combines user-specified services with async support
- Maintains backward compatibility with existing OPENDAL_ENABLE_ASYNC option
Removed obsolete feature handling: Cleaned up the previous hardcoded async feature logic

Are there any user-facing changes?

New option: FEATURES - allows users to specify which OpenDAL storage services to include
Behavior change: By default, no storage services are included (users must explicitly specify needed services)
Backward compatibility: Existing usage of OPENDAL_ENABLE_ASYNC continues to work unchanged
Migration: Users upgrading will need to add -DFEATURES="service1,service2" to their cmake commands to include storage services

Remove hardcoded service dependencies to enable user-configurable features. Only keep 'blocking' feature as it's required for C++ bindings. Related to apache#6250

Related to apache#6250

…builds Related to apache#6250

KinomotoMio · 2025-08-12T02:57:40Z

@asukaminato0721

asukaminato0721 · 2025-08-12T02:59:27Z

refer to here

opendal/.github/workflows/test_behavior_binding_c.yml

Line 66 in 542df66

run: cargo build --features="opendal/${{ matrix.cases.feature }}"

asukaminato0721 · 2025-08-12T03:24:44Z

opendal/.github/actions/test_behavior_binding_cpp/action.yaml

Lines 47 to 53 in 542df66

    
                           cmake \ 
        
                             -DCMAKE_BUILD_TYPE=Release \ 
        
                             -DOPENDAL_ENABLE_TESTING=ON \ 
        
                             -DOPENDAL_DEV=ON \ 
        
                             -DOPENDAL_ENABLE_ASYNC=ON \ 
        
                             -DCMAKE_CXX_STANDARD=20 \ 
        
                             ..

Maybe you need to add the feature config here.

Related to apache#6250

asukaminato0721 · 2025-08-12T11:40:43Z

/home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/rocksdb-0.21.0/src/ffi_util.rs:75:
(.text._ZN7rocksdb2db21DBCommon$LT$T$C$D$GT$3get17h65f01d16dcc18acaE+0x52): undefined reference to 
`rocksdb_get_pinned'

can't understand...

asukaminato0721 · 2025-08-12T11:43:24Z

monoiofs

==7264==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 608 byte(s) in 38 object(s) allocated from:
    #0 0x55c2f57a72e3 in malloc (/home/runner/work/opendal/opendal/bindings/cpp/build/opendal_cpp_test+0x1a82e3) (BuildId: a9450bbf7a6088b8d60b7ed549bd88dfc165fab6)

Related to apache#6250

Alluxio doesn't support file overwrite without explicit overwrite option. Related to apache#6250

Alluxio has different list behavior that includes unexpected paths. Related to apache#6250

Related to apache#6250

asukaminato0721 · 2025-08-13T05:38:00Z

seems only memory leak now.

bindings/cpp/tests/behavior/delete_test.cpp

Xuanwo · 2025-08-14T07:44:32Z

seems only memory leak now.

Thank you for the work, I think we can disable cpp test on those two services for now.

Xuanwo

Overall LGTM, thank you for your work!

bindings/cpp/CMakeLists.txt

bindings/cpp/README.md

Related to apache#6250

…ation test Related to apache#6250

…gFile test Related to apache#6250

…ior test Related to apache#6250

…rFile test Related to apache#6250

Related to apache#6250

bindings/cpp/tests/behavior/list_test.cpp

KinomotoMio added 3 commits August 12, 2025 09:04

feat(bindings/cpp): remove hardcoded service features from Cargo.toml

51915d8

Remove hardcoded service dependencies to enable user-configurable features. Only keep 'blocking' feature as it's required for C++ bindings. Related to apache#6250

feat(bindings/cpp): add FEATURES cache variable to CMakeLists.txt

6e51ab6

Related to apache#6250

feat(bindings/cpp): implement feature merging logic for configurable …

3e8a425

…builds Related to apache#6250

KinomotoMio added 3 commits August 12, 2025 16:57

docs(bindings/cpp): add FEATURES option documentation

440b44a

Related to apache#6250

ci(bindings/cpp): add FEATURES support to behavior test action

5d25d05

Related to apache#6250

ci(bindings/cpp): add FEATURES parameter to all build configurations

966b7d6

Related to apache#6250

KinomotoMio added 13 commits August 13, 2025 06:06

Merge branch 'main' into feat/cpp-binding-configurable-features

0747c74

feat(bindings/cpp): add conditional RocksDB linking support

77f774f

Related to apache#6250

feat(bindings/cpp): add conditional HDFS linking support

9aa6d91

Related to apache#6250

test(bindings/cpp): skip ReadAfterModification test for alluxio

8baf43c

Alluxio doesn't support file overwrite without explicit overwrite option. Related to apache#6250

test(bindings/cpp): skip OverwriteExistingFile test for alluxio

86c5bed

Alluxio doesn't support file overwrite without explicit overwrite option. Related to apache#6250

test(bindings/cpp): skip WriteAppendBehavior test for alluxio

6b3c0db

Alluxio doesn't support file overwrite without explicit overwrite option. Related to apache#6250

test(bindings/cpp): skip DeleteSpecialCharFile test for alluxio

b5e84cf

Alluxio doesn't support file overwrite without explicit overwrite option. Related to apache#6250

test(bindings/cpp): skip ListEmptyDirectory test for alluxio

015a1e6

Alluxio has different list behavior that includes unexpected paths. Related to apache#6250

test(bindings/cpp): skip ListDirectoryWithFiles test for alluxio

a8f76cf

Alluxio has different list behavior that includes unexpected paths. Related to apache#6250

test(bindings/cpp): skip ListNestedDirectories test for alluxio

b89e431

Alluxio has different list behavior that includes unexpected paths. Related to apache#6250

test(bindings/cpp): skip ListManyFiles test for alluxio

7281dd0

Alluxio has different list behavior that includes unexpected paths. Related to apache#6250

test(bindings/cpp): skip ListSpecialCharNames test for alluxio

c8fa9d0

Alluxio has different list behavior that includes unexpected paths. Related to apache#6250

test(bindings/cpp): skip ListerIterator test for alluxio

6129ffd

Alluxio has different list behavior that includes unexpected paths. Related to apache#6250

KinomotoMio force-pushed the feat/cpp-binding-configurable-features branch from efb120e to 6129ffd Compare August 13, 2025 04:40

fix(bindings/cpp): improve HDFS library detection using HADOOP_HOME

4a5b670

Related to apache#6250

KinomotoMio mentioned this pull request Aug 14, 2025

bug: Memory leak detected in cpp binding CI (LeakSanitizer) #6505

Closed

1 task

asukaminato0721 reviewed Aug 14, 2025

View reviewed changes

bindings/cpp/tests/behavior/delete_test.cpp Outdated Show resolved Hide resolved

Xuanwo reviewed Aug 14, 2025

View reviewed changes

bindings/cpp/CMakeLists.txt Outdated Show resolved Hide resolved

bindings/cpp/CMakeLists.txt Outdated Show resolved Hide resolved

bindings/cpp/README.md Outdated Show resolved Hide resolved

KinomotoMio added 12 commits August 15, 2025 00:34

refactor(bindings/cpp): rename FEATURES to OPENDAL_FEATURES

f334964

Related to apache#6250

docs(bindings/cpp): update README to reflect OPENDAL_FEATURES rename

9f318ff

Related to apache#6250

test(bindings/cpp): use adaptive logic for ListEmptyDirectory test

44d493d

Related to apache#6250

test(bindings/cpp): use adaptive logic for ListDirectoryWithFiles test

d4a01d8

Related to apache#6250

test(bindings/cpp): use adaptive logic for ListNestedDirectories test

12b86f7

Related to apache#6250

test(bindings/cpp): use adaptive logic for ListManyFiles test

ec5132b

Related to apache#6250

test(bindings/cpp): use adaptive logic for ListSpecialCharNames test

37899ad

Related to apache#6250

test(bindings/cpp): use adaptive logic for ListerIterator test

467672c

Related to apache#6250

test(bindings/cpp): use exception-based approach for ReadAfterModific…

2501c54

…ation test Related to apache#6250

test(bindings/cpp): use exception-based approach for OverwriteExistin…

73add42

…gFile test Related to apache#6250

test(bindings/cpp): use exception-based approach for WriteAppendBehav…

082d50d

…ior test Related to apache#6250

test(bindings/cpp): use exception-based approach for DeleteSpecialCha…

ae941db

…rFile test Related to apache#6250

KinomotoMio force-pushed the feat/cpp-binding-configurable-features branch from dce280e to ae941db Compare August 14, 2025 21:50

refactor(bindings/cpp): remove manual RocksDB and HDFS linking

f57c2c6

Related to apache#6250

KinomotoMio marked this pull request as ready for review August 14, 2025 22:23

dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. releases-note/feat The PR implements a new feature or has a title that begins with "feat" labels Aug 14, 2025

asukaminato0721 reviewed Aug 14, 2025

View reviewed changes

bindings/cpp/tests/behavior/list_test.cpp Outdated Show resolved Hide resolved

asukaminato0721 reviewed Aug 14, 2025

View reviewed changes

bindings/cpp/tests/behavior/list_test.cpp Outdated Show resolved Hide resolved

Apply suggestions from code review

64d24dc

asukaminato0721 self-requested a review August 14, 2025 23:41

asukaminato0721 approved these changes Aug 14, 2025

View reviewed changes

asukaminato0721 merged commit 08880ef into apache:main Aug 14, 2025
64 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

feat(bindings/cpp): enable user-configurable features for storage services#6503

feat(bindings/cpp): enable user-configurable features for storage services#6503
asukaminato0721 merged 34 commits intoapache:mainfrom
KinomotoMio:feat/cpp-binding-configurable-features

KinomotoMio commented Aug 12, 2025 •

edited

Loading

Uh oh!

KinomotoMio commented Aug 12, 2025

Uh oh!

asukaminato0721 commented Aug 12, 2025

Uh oh!

asukaminato0721 commented Aug 12, 2025

Uh oh!

asukaminato0721 commented Aug 12, 2025 •

edited

Loading

Uh oh!

asukaminato0721 commented Aug 12, 2025

Uh oh!

asukaminato0721 commented Aug 13, 2025

Uh oh!

Uh oh!

Xuanwo commented Aug 14, 2025

Uh oh!

Xuanwo left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

Conversation

KinomotoMio commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

Uh oh!

KinomotoMio commented Aug 12, 2025

Uh oh!

asukaminato0721 commented Aug 12, 2025

Uh oh!

asukaminato0721 commented Aug 12, 2025

Uh oh!

asukaminato0721 commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asukaminato0721 commented Aug 12, 2025

Uh oh!

asukaminato0721 commented Aug 13, 2025

Uh oh!

Uh oh!

Xuanwo commented Aug 14, 2025

Uh oh!

Xuanwo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

KinomotoMio commented Aug 12, 2025 •

edited

Loading

asukaminato0721 commented Aug 12, 2025 •

edited

Loading