Skip to content

Comments

fix(services/ipmfs): fix Ipmfs behavior tests#5969

Merged
Xuanwo merged 2 commits intoapache:mainfrom
miroim:split-ipmfs-core
Apr 6, 2025
Merged

fix(services/ipmfs): fix Ipmfs behavior tests#5969
Xuanwo merged 2 commits intoapache:mainfrom
miroim:split-ipmfs-core

Conversation

@miroim
Copy link
Member

@miroim miroim commented Apr 6, 2025

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Improve error handling and directory listing behavior in lister to pass behavior tests.

Are there any user-facing changes?

No

@miroim miroim requested a review from Xuanwo as a code owner April 6, 2025 11:13
@dosubot dosubot bot added size:S This PR changes 10-29 lines, ignoring generated files. releases-note/fix The PR fixes a bug or has a title that begins with "fix" services/ipmfs labels Apr 6, 2025
@miroim
Copy link
Member Author

miroim commented Apr 6, 2025

Behavior tests log:

Before: 
cargo test behavior --features tests,services-ipmfs -- --show-output

   Compiling opendal v0.53.0 (E:\A_work\OSS\opendal\core)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 15.28s
     Running unittests src\lib.rs (target\debug\deps\opendal-bcda60b7dff17fe0.exe)

running 0 tests

successes:

successes:

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 132 filtered out; finished in 0.00s

     Running tests\behavior\main.rs (target\debug\deps\behavior-3751b30d6263eaa2.exe)

running 101 tests
test behavior::test_delete_empty_dir                          ... ok
test behavior::test_delete_stream                             ... ok
test behavior::test_delete_with_not_existing_version          ... ok
test behavior::test_batch_delete                              ... ok
test behavior::test_delete_with_version                       ... ok
test behavior::test_batch_delete_with_version                 ... ok
test behavior::test_list_with_start_after                     ... ok
test behavior::test_delete_not_existing                       ... ok
test behavior::test_list_non_exist_dir                        ... FAILED
test behavior::test_list_non_exist_dir_with_recursive         ... FAILED
test behavior::test_check                                     ... ok
test behavior::test_list_files_with_versions                  ... ok
test behavior::test_list_with_versions_and_limit              ... ok
test behavior::test_list_with_versions_and_start_after        ... ok
test behavior::test_list_files_with_deleted                   ... ok

thread '<unnamed>' panicked at tests\behavior\async_list.rs:158:5:
assertion `left == right` failed: only return the dir itself, but found: {}
  left: 0
 right: 1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test behavior::test_list_empty_dir                            ... FAILED
test behavior::test_delete_file                               ... ok
test behavior::test_list_sub_dir                              ... ok
test behavior::test_reader_with_if_match                      ... ok
test behavior::test_reader_with_if_none_match                 ... ok
test behavior::test_reader_with_if_modified_since             ... ok
test behavior::test_reader_with_if_unmodified_since           ... ok
test behavior::test_list_prefix                               ... ok
test behavior::test_read_with_if_match                        ... ok
test behavior::test_read_with_if_none_match                   ... ok
test behavior::test_read_with_if_modified_since               ... ok
test behavior::test_read_with_if_unmodified_since             ... ok
test behavior::test_read_with_dir_path                        ... ok
test behavior::test_read_not_exist                            ... ok
test behavior::test_read_with_override_cache_control          ... ok
test behavior::test_read_with_override_content_disposition    ... ok
test behavior::test_read_with_override_content_type           ... ok
test behavior::test_read_with_version                         ... ok
test behavior::test_read_with_not_existing_version            ... ok

thread '<unnamed>' panicked at tests\behavior\async_list.rs:288:5:
assertion `left == right` failed: parent should got 2 entry
  left: 1
 right: 2
test behavior::test_list_nested_dir                           ... FAILED
test behavior::test_stat_dir                                  ... ok
test behavior::test_stat_nested_parent_dir                    ... ok
test behavior::test_remove_one_file                           ... ok
test behavior::test_list_file_with_recursive                  ... ok
test behavior::test_stat_not_exist                            ... ok
test behavior::test_stat_with_if_match                        ... ok
test behavior::test_stat_with_if_none_match                   ... ok
test behavior::test_stat_with_if_modified_since               ... ok
test behavior::test_stat_with_if_unmodified_since             ... ok
test behavior::test_stat_with_override_cache_control          ... ok
test behavior::test_stat_with_override_content_disposition    ... ok
test behavior::test_stat_with_override_content_type           ... ok
test behavior::test_stat_root                                 ... ok
test behavior::test_stat_with_version                         ... ok
test behavior::stat_with_not_existing_version                 ... ok
test behavior::test_list_dir_with_file_path                   ... ok
test behavior::test_write_with_empty_content                  ... ok
test behavior::test_write_with_dir_path                       ... ok
test behavior::test_delete_with_special_chars                 ... ok
test behavior::test_write_with_cache_control                  ... ok
test behavior::test_write_with_content_type                   ... ok
test behavior::test_write_with_content_disposition            ... ok
test behavior::test_write_with_content_encoding               ... ok
test behavior::test_write_with_if_none_match                  ... ok
test behavior::test_write_with_if_not_exists                  ... ok
test behavior::test_write_with_if_match                       ... ok
test behavior::test_write_with_user_metadata                  ... ok
test behavior::test_list_dir                                  ... ok
test behavior::test_writer_write                              ... ok
test behavior::test_read_range                                ... ok
test behavior::test_writer_write_with_concurrent              ... ok
test behavior::test_writer_sink                               ... ok
test behavior::test_writer_sink_with_concurrent               ... ok
test behavior::test_stat_file                                 ... ok
test behavior::test_writer_abort_with_concurrent              ... ok
test behavior::test_writer_futures_copy                       ... ok
test behavior::test_writer_futures_copy_with_concurrent       ... ok
test behavior::test_writer_return_metadata                    ... ok
test behavior::test_read_full                                 ... ok
test behavior::test_write_only                                ... ok
test behavior::test_reader                                    ... ok

thread '<unnamed>' panicked at tests\behavior\async_list.rs:471:5:
assertion `left == right` failed
  left: ["x/", "x/x/", "x/x/x/", "x/x/x/y", "x/x/y", "x/y", "x/yy"]
 right: ["x/", "x/x/", "x/x/x/", "x/x/x/x/", "x/x/x/y", "x/x/y", "x/y", "x/yy"]
test behavior::test_list_dir_with_recursive                   ... FAILED
test behavior::test_blocking_read_not_exist                   ... ok

thread '<unnamed>' panicked at tests\behavior\async_list.rs:507:5:
assertion `left == right` failed
  left: ["x/", "x/x/", "x/x/x/", "x/x/x/y", "x/x/y", "x/y", "x/yy"]
 right: ["x/", "x/x/", "x/x/x/", "x/x/x/x/", "x/x/x/y", "x/x/y", "x/y", "x/yy"]
test behavior::test_list_dir_with_recursive_no_trailing_slash ... FAILED
test behavior::test_blocking_stat_dir                         ... ok
test behavior::test_read_with_special_chars                   ... ok
test behavior::test_blocking_stat_not_exist                   ... ok
test behavior::test_remove_all                                ... ok
test behavior::test_writer_abort                              ... ok

thread '<unnamed>' panicked at tests\behavior\async_list.rs:140:5:
assertion `left == right` failed
  left: ["test_list_rich_dir/file-0", "test_list_rich_dir/file-1", "test_list_rich_dir/file-10", "test_list_rich_dir/file-2", "test_list_rich_dir/file-3", "test_list_rich_dir/file-4", "test_list_rich_dir/file-5", "test_list_rich_dir/file-6", "test_list_rich_dir/file-7", "test_list_rich_dir/file-8", "test_list_rich_dir/file-9"]
 right: ["test_list_rich_dir/", "test_list_rich_dir/file-0", "test_list_rich_dir/file-1", "test_list_rich_dir/file-10", "test_list_rich_dir/file-2", "test_list_rich_dir/file-3", "test_list_rich_dir/file-4", "test_list_rich_dir/file-5", "test_list_rich_dir/file-6", "test_list_rich_dir/file-7", "test_list_rich_dir/file-8", "test_list_rich_dir/file-9"]
test behavior::test_list_rich_dir                             ... FAILED
test behavior::test_stat_not_cleaned_path                     ... ok
test behavior::test_write_returns_metadata                    ... ok
test behavior::test_stat_with_special_chars                   ... ok
test behavior::test_blocking_read_range                       ... ok
test behavior::test_write_with_special_chars                  ... ok
test behavior::test_blocking_delete_file                      ... ok
test behavior::test_blocking_stat_file                        ... ok
test behavior::test_blocking_remove_one_file                  ... ok
test behavior::test_blocking_write_returns_metadata           ... ok
test behavior::test_blocking_stat_with_special_chars          ... ok
test behavior::test_blocking_write_with_dir_path              ... ok
test behavior::test_blocking_write_file                       ... ok
test behavior::test_blocking_read_full                        ... ok
test behavior::test_list_root_with_recursive                  ... ok
test behavior::test_blocking_write_with_special_chars         ... ok
test behavior::test_writer_write_with_overwrite               ... ok

failures:

---- behavior::test_list_non_exist_dir ----
NotFound (persistent) at list, context: { uri: http://127.0.0.1:5001/api/v0/files/ls?arg=/9de0e8df-f40d-4e3f-942e-96c6cf092425/&long=true, response: Parts { status: 500, version: HTTP/1.1, headers: {"connection": "close", "transfer-encoding": "chunked", "trailer": "X-Stream-Error", "access-control-allow-headers": "X-Stream-Output, X-Chunked-Output, X-Content-Length", "access-control-expose-headers": "X-Stream-Output, X-Chunked-Output, X-Content-Length", "content-type": "application/json", "date": "Sun, 06 Apr 2025 11:18:18 GMT", "server": "kubo/0.34.1", "vary": "Origin"} }, service: ipmfs, path: 9de0e8df-f40d-4e3f-942e-96c6cf092425/, listed: 0 } => IpfsError { message: "file does not exist", code: 0, ty: "error" }

---- behavior::test_list_non_exist_dir_with_recursive ----
NotFound (persistent) at list, context: { uri: http://127.0.0.1:5001/api/v0/files/ls?arg=/697352b2-8360-4acb-b175-001f8ce044c3/&long=true, response: Parts { status: 500, version: HTTP/1.1, headers: {"connection": "close", "transfer-encoding": "chunked", "trailer": "X-Stream-Error", "access-control-allow-headers": "X-Stream-Output, X-Chunked-Output, X-Content-Length", "access-control-expose-headers": "X-Stream-Output, X-Chunked-Output, X-Content-Length", "content-type": "application/json", "date": "Sun, 06 Apr 2025 11:18:18 GMT", "server": "kubo/0.34.1", "vary": "Origin"} }, service: ipmfs, path: 697352b2-8360-4acb-b175-001f8ce044c3/, listed: 0 } => IpfsError { message: "file does not exist", code: 0, ty: "error" }

---- behavior::test_list_empty_dir ----
test panicked: assertion `left == right` failed: only return the dir itself, but found: {}     
  left: 0
 right: 1

---- behavior::test_list_nested_dir ----
test panicked: assertion `left == right` failed: parent should got 2 entry
  left: 1
 right: 2

---- behavior::test_list_dir_with_recursive ----
test panicked: assertion `left == right` failed
  left: ["x/", "x/x/", "x/x/x/", "x/x/x/y", "x/x/y", "x/y", "x/yy"]
 right: ["x/", "x/x/", "x/x/x/", "x/x/x/x/", "x/x/x/y", "x/x/y", "x/y", "x/yy"]

---- behavior::test_list_dir_with_recursive_no_trailing_slash ----
test panicked: assertion `left == right` failed
  left: ["x/", "x/x/", "x/x/x/", "x/x/x/y", "x/x/y", "x/y", "x/yy"]
 right: ["x/", "x/x/", "x/x/x/", "x/x/x/x/", "x/x/x/y", "x/x/y", "x/y", "x/yy"]

---- behavior::test_list_rich_dir ----
test panicked: assertion `left == right` failed
  left: ["test_list_rich_dir/file-0", "test_list_rich_dir/file-1", "test_list_rich_dir/file-10", "test_list_rich_dir/file-2", "test_list_rich_dir/file-3", "test_list_rich_dir/file-4", "test_list_rich_dir/file-5", "test_list_rich_dir/file-6", "test_list_rich_dir/file-7", "test_list_rich_dir/file-8", "test_list_rich_dir/file-9"]
 right: ["test_list_rich_dir/", "test_list_rich_dir/file-0", "test_list_rich_dir/file-1", "test_list_rich_dir/file-10", "test_list_rich_dir/file-2", "test_list_rich_dir/file-3", "test_list_rich_dir/file-4", "test_list_rich_dir/file-5", "test_list_rich_dir/file-6", "test_list_rich_dir/file-7", "test_list_rich_dir/file-8", "test_list_rich_dir/file-9"]


failures:
    behavior::test_list_non_exist_dir
    behavior::test_list_non_exist_dir_with_recursive
    behavior::test_list_empty_dir
    behavior::test_list_nested_dir
    behavior::test_list_dir_with_recursive
    behavior::test_list_dir_with_recursive_no_trailing_slash
    behavior::test_list_rich_dir

test result: FAILED. 94 passed; 7 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.86s
After: 
cargo test behavior --features tests,services-ipmfs -- --show-output

    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.81s
     Running unittests src\lib.rs (target\debug\deps\opendal-bcda60b7dff17fe0.exe)

running 0 tests

successes:

successes:

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 132 filtered out; finished in 0.00s

     Running tests\behavior\main.rs (target\debug\deps\behavior-3751b30d6263eaa2.exe)

running 101 tests
test behavior::test_delete_empty_dir                          ... ok
test behavior::test_delete_stream                             ... ok
test behavior::test_delete_with_version                       ... ok
test behavior::test_batch_delete                              ... ok
test behavior::test_delete_with_not_existing_version          ... ok
test behavior::test_batch_delete_with_version                 ... ok
test behavior::test_list_with_start_after                     ... ok
test behavior::test_list_non_exist_dir                        ... ok
test behavior::test_delete_not_existing                       ... ok
test behavior::test_check                                     ... ok
test behavior::test_list_non_exist_dir_with_recursive         ... ok
test behavior::test_list_files_with_versions                  ... ok
test behavior::test_list_with_versions_and_limit              ... ok
test behavior::test_list_with_versions_and_start_after        ... ok
test behavior::test_list_files_with_deleted                   ... ok
test behavior::test_list_sub_dir                              ... ok
test behavior::test_delete_file                               ... ok
test behavior::test_delete_with_special_chars                 ... ok
test behavior::test_reader_with_if_match                      ... ok
test behavior::test_reader_with_if_none_match                 ... ok
test behavior::test_reader_with_if_modified_since             ... ok
test behavior::test_reader_with_if_unmodified_since           ... ok
test behavior::test_read_not_exist                            ... ok
test behavior::test_read_with_if_match                        ... ok
test behavior::test_read_with_if_none_match                   ... ok
test behavior::test_read_with_if_modified_since               ... ok
test behavior::test_read_with_if_unmodified_since             ... ok
test behavior::test_read_with_dir_path                        ... ok
test behavior::test_list_file_with_recursive                  ... ok
test behavior::test_read_with_override_cache_control          ... ok
test behavior::test_read_with_override_content_disposition    ... ok
test behavior::test_read_with_override_content_type           ... ok
test behavior::test_read_with_version                         ... ok
test behavior::test_read_with_not_existing_version            ... ok
test behavior::test_list_dir                                  ... ok
test behavior::test_stat_dir                                  ... ok
test behavior::test_stat_nested_parent_dir                    ... ok
test behavior::test_list_dir_with_file_path                   ... ok
test behavior::test_read_full                                 ... ok
test behavior::test_stat_not_exist                            ... ok
test behavior::test_stat_with_if_match                        ... ok
test behavior::test_stat_with_if_none_match                   ... ok
test behavior::test_stat_with_if_modified_since               ... ok
test behavior::test_stat_with_if_unmodified_since             ... ok
test behavior::test_stat_with_override_cache_control          ... ok
test behavior::test_stat_with_override_content_disposition    ... ok
test behavior::test_stat_with_override_content_type           ... ok
test behavior::test_stat_root                                 ... ok
test behavior::test_stat_with_version                         ... ok
test behavior::stat_with_not_existing_version                 ... ok
test behavior::test_read_with_special_chars                   ... ok
test behavior::test_write_with_empty_content                  ... ok
test behavior::test_write_with_dir_path                       ... ok
test behavior::test_reader                                    ... ok
test behavior::test_write_with_cache_control                  ... ok
test behavior::test_write_with_content_type                   ... ok
test behavior::test_write_with_content_disposition            ... ok
test behavior::test_write_with_content_encoding               ... ok
test behavior::test_write_with_if_none_match                  ... ok
test behavior::test_write_with_if_not_exists                  ... ok
test behavior::test_write_with_if_match                       ... ok
test behavior::test_write_with_user_metadata                  ... ok
test behavior::test_list_nested_dir                           ... ok
test behavior::test_writer_write                              ... ok
test behavior::test_list_prefix                               ... ok
test behavior::test_writer_write_with_concurrent              ... ok
test behavior::test_writer_sink                               ... ok
test behavior::test_writer_sink_with_concurrent               ... ok
test behavior::test_read_range                                ... ok
test behavior::test_stat_not_cleaned_path                     ... ok
test behavior::test_writer_futures_copy                       ... ok
test behavior::test_writer_futures_copy_with_concurrent       ... ok
test behavior::test_writer_return_metadata                    ... ok
test behavior::test_writer_abort                              ... ok
test behavior::test_remove_one_file                           ... ok
test behavior::test_writer_abort_with_concurrent              ... ok
test behavior::test_stat_file                                 ... ok
test behavior::test_list_dir_with_recursive                   ... ok
test behavior::test_blocking_read_not_exist                   ... ok
test behavior::test_blocking_stat_dir                         ... ok
test behavior::test_list_dir_with_recursive_no_trailing_slash ... ok
test behavior::test_blocking_stat_not_exist                   ... ok
test behavior::test_stat_with_special_chars                   ... ok
test behavior::test_write_with_special_chars                  ... ok
test behavior::test_write_returns_metadata                    ... ok
test behavior::test_remove_all                                ... ok
test behavior::test_write_only                                ... ok
test behavior::test_blocking_write_file                       ... ok
test behavior::test_blocking_read_range                       ... ok
test behavior::test_blocking_write_returns_metadata           ... ok
test behavior::test_blocking_delete_file                      ... ok
test behavior::test_blocking_write_with_dir_path              ... ok
test behavior::test_blocking_remove_one_file                  ... ok
test behavior::test_blocking_stat_file                        ... ok
test behavior::test_blocking_stat_with_special_chars          ... ok
test behavior::test_list_rich_dir                             ... ok
test behavior::test_blocking_write_with_special_chars         ... ok
test behavior::test_list_root_with_recursive                  ... ok
test behavior::test_list_empty_dir                            ... ok
test behavior::test_blocking_read_full                        ... ok
test behavior::test_writer_write_with_overwrite               ... ok

test result: ok. 101 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.81s

Xuanwo
Xuanwo approved these changes Apr 6, 2025
View reviewed changes
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @miroim for the fix!

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Apr 6, 2025
@Xuanwo Xuanwo merged commit 6d2a09e into apache:main Apr 6, 2025
91 checks passed
@miroim miroim deleted the split-ipmfs-core branch April 6, 2025 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Xuanwo Xuanwo Xuanwo approved these changes

Assignees

No one assigned

Labels

lgtm This PR has been approved by a maintainer releases-note/fix The PR fixes a bug or has a title that begins with "fix" services/ipmfs size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@miroim @Xuanwo