feat(services/onedrive): List dir shows metadata by erickguan · Pull Request #5632 · apache/opendal

erickguan · 2025-02-16T21:22:50Z

Which issue does this PR close?

Part of #4746.

What changes are included in this PR?

Supports OneDrive lister with metadata.

meteorgan

The test workflow in .github/services is missing, we should add one.

meteorgan · 2025-02-20T12:26:44Z

core/src/services/onedrive/lister.rs

-                    oio::Entry::new(&normalized_path, Metadata::new(EntryMode::FILE))
-                }
+            let mut meta = Metadata::new(entry_mode)
+                .with_last_modified(


we have some time functions in chrono_utils.rs, like: parse_datetime_from_rfc2822. Maybe we could reuse them.

Thanks for the tip, added.

Xuanwo · 2025-02-21T02:09:29Z

The test workflow in .github/services is missing, we should add one.

I remembered that we failed to set up a OneDrive test.

erickguan · 2025-02-21T22:08:19Z

I generated a token from a personal test account with Files.ReadWrite. This workflow run.

Additional permissions are useful for API Graph explorer which I recommend:

DelegatedPermissionGrant.ReadWrite.All
Directory.Read.All

What do you want me next? These actions look sensible:

Merge this PR as is.
Set up an OpenDAL test account.
Fix bugs in a new PR.

meteorgan · 2025-02-22T09:37:41Z

I generated a token from a personal test account with Files.ReadWrite. This workflow run.

Additional permissions are useful for API Graph explorer which I recommend:

DelegatedPermissionGrant.ReadWrite.All

Directory.Read.All

What do you want me next? These actions look sensible:

Merge this PR as is.

Set up an OpenDAL test account.

Fix bugs in a new PR.

Since we don't have a test workflow for this, have you run the tests locally ? You can use the command OPENDAL_TEST=onedrive cargo test behavior --features tests,services-onedrive -- --show-output. if all the tests pass, I think we can proceed with merging this PR.

erickguan · 2025-02-25T20:56:54Z

Rebased and ready.

OPENDAL_TEST=onedrive cargo test behavior --features tests,services-onedrive -- --show-output

   Compiling opendal v0.52.0 (/home/erickg/Dev/opendal/core)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 11.49s
     Running unittests src/lib.rs (target/debug/deps/opendal-9022af4dd162386a)

running 0 tests

successes:

successes:

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 134 filtered out; finished in 0.00s

     Running tests/behavior/main.rs (target/debug/deps/behavior-c8073076fdb6f2e9)

running 102 tests
test behavior::test_delete_with_version                       ... ok
test behavior::test_delete_with_not_existing_version          ... ok
test behavior::test_list_with_start_after                     ... ok
test behavior::test_list_with_versions_and_start_after        ... ok
test behavior::test_reader_with_if_modified_since             ... ok
test behavior::test_reader_with_if_unmodified_since           ... ok
test behavior::test_reader_with_if_none_match                 ... ok
test behavior::test_read_with_if_match                        ... ok
test behavior::test_read_with_if_none_match                   ... ok
test behavior::test_read_with_if_modified_since               ... ok
test behavior::test_read_with_if_unmodified_since             ... ok
test behavior::test_list_files_with_versions                  ... ok
test behavior::test_list_files_with_deleted                   ... ok
test behavior::test_list_with_versions_and_limit              ... ok
test behavior::test_reader_with_if_match                      ... ok
test behavior::test_read_with_override_content_type           ... ok
test behavior::test_read_with_version                         ... ok
test behavior::test_read_with_not_existing_version            ... ok
test behavior::test_read_with_override_content_disposition    ... ok
test behavior::test_read_with_override_cache_control          ... ok
test behavior::test_read_not_exist                            ... ok
test behavior::test_check                                     ... ok
test behavior::test_list_non_exist_dir                        ... ok
test behavior::test_stat_with_if_match                        ... ok
test behavior::test_stat_with_if_none_match                   ... ok
test behavior::test_stat_with_if_modified_since               ... ok
test behavior::test_stat_with_if_unmodified_since             ... ok
test behavior::test_stat_with_override_cache_control          ... ok
test behavior::test_stat_with_override_content_disposition    ... ok
test behavior::test_stat_with_override_content_type           ... ok
test behavior::test_stat_root                                 ... ok
test behavior::test_stat_with_version                         ... ok
test behavior::stat_with_not_existing_version                 ... ok
test behavior::test_list_root_with_recursive                  ... ok
test behavior::test_write_with_empty_content                  ... ok
test behavior::test_write_with_dir_path                       ... ok
test behavior::test_delete_not_existing                       ... ok
test behavior::test_write_with_cache_control                  ... ok
test behavior::test_write_with_content_type                   ... ok
test behavior::test_write_with_content_disposition            ... ok
test behavior::test_write_with_content_encoding               ... ok
test behavior::test_write_with_if_none_match                  ... ok
test behavior::test_write_with_if_not_exists                  ... ok
test behavior::test_write_with_if_match                       ... ok
test behavior::test_write_with_user_metadata                  ... ok
test behavior::test_read_with_dir_path                        ... ok
test behavior::test_writer_write                              ... ok
test behavior::test_stat_not_exist                            ... ok
test behavior::test_writer_write_with_concurrent              ... ok
test behavior::test_writer_sink                               ... ok
test behavior::test_writer_sink_with_concurrent               ... ok
test behavior::test_writer_abort                              ... ok
test behavior::test_create_dir                                ... ok
test behavior::test_writer_futures_copy                       ... ok
test behavior::test_writer_futures_copy_with_concurrent       ... ok
test behavior::test_writer_return_metadata                    ... ok
test behavior::test_stat_dir                                  ... ok
test behavior::test_delete_empty_dir                          ... ok
test behavior::test_writer_abort_with_concurrent              ... ok
test behavior::test_create_dir_existing                       ... ok
test behavior::test_blocking_create_dir                       ... ok
test behavior::test_list_sub_dir                              ... ok
test behavior::test_blocking_read_not_exist                   ... ok
test behavior::test_delete_file                               ... ok
test behavior::test_stat_with_special_chars                   ... ok
test behavior::test_read_range                                ... ok
test behavior::test_delete_with_special_chars                 ... ok
test behavior::test_write_returns_metadata                    ... ok
test behavior::test_blocking_create_dir_existing              ... ok
test behavior::test_list_prefix                               ... ok
test behavior::test_stat_nested_parent_dir                    ... ok
test behavior::test_blocking_stat_not_exist                   ... ok
test behavior::test_blocking_write_with_dir_path              ... ok
test behavior::test_write_only                                ... ok
test behavior::test_blocking_stat_dir                         ... ok
test behavior::test_list_dir_with_file_path                   ... ok
test behavior::test_read_full                                 ... ok
test behavior::test_stat_file                                 ... ok
test behavior::test_list_file_with_recursive                  ... ok
test behavior::test_write_with_special_chars                  ... ok
test behavior::test_stat_not_cleaned_path                     ... ok
test behavior::test_read_with_special_chars                   ... ok
test behavior::test_list_dir                                  ... ok
test behavior::test_blocking_delete_file                      ... ok
test behavior::test_blocking_write_file                       ... ok
test behavior::test_blocking_stat_with_special_chars          ... ok
test behavior::test_blocking_remove_one_file                  ... ok
test behavior::test_blocking_read_range                       ... ok
test behavior::test_blocking_stat_file                        ... ok
test behavior::test_reader                                    ... ok
test behavior::test_blocking_write_with_special_chars         ... ok
test behavior::test_remove_one_file                           ... ok
test behavior::test_blocking_write_returns_metadata           ... ok
test behavior::test_blocking_read_full                        ... ok
test behavior::test_list_dir_with_recursive_no_trailing_slash ... ok
test behavior::test_list_nested_dir                           ... ok
test behavior::test_writer_write_with_overwrite               ... ok
test behavior::test_list_dir_with_recursive                   ... ok
test behavior::test_list_empty_dir                            ... ok
test behavior::test_remove_all                                ... ok
test behavior::test_list_rich_dir                             ... ok
test behavior::test_delete_stream                             ... ok

test result: ok. 102 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 38.88s

meteorgan · 2025-02-26T04:41:52Z

LGTM. Thanks for working on this. @erickguan

erickguan · 2025-02-26T08:54:12Z

@meteorgan happy to help!

Xuanwo

Thank you @erickguan for working on this and thank you @meteorgan for the review.

Xuanwo · 2025-02-26T10:18:09Z

By the way, @erickguan would you like to help fix the conflicts?

erickguan · 2025-02-26T11:38:12Z

Sure, I will rebase it and ping you.

Most my test folders have 0 in size, but one has 3. I didn't dig into it.

erickguan · 2025-03-01T22:21:12Z

Rebased @Xuanwo

Xuanwo · 2025-03-03T05:34:23Z

Thank you @erickguan for this!

erickguan requested a review from Xuanwo as a code owner February 16, 2025 21:22

github-actions bot added the releases-note/feat The PR implements a new feature or has a title that begins with "feat" label Feb 16, 2025

meteorgan reviewed Feb 20, 2025

View reviewed changes

This was referenced Feb 23, 2025

fix(services/onedrive): fix OneDrive behavior tests #5652

Merged

Set up behavior tests for OneDrive #5653

Closed

erickguan requested a review from meteorgan February 25, 2025 20:57

meteorgan approved these changes Feb 26, 2025

View reviewed changes

Xuanwo approved these changes Feb 26, 2025

View reviewed changes

feat(services/onedrive): List dir shows metadata

014a80f

Most my test folders have 0 in size, but one has 3. I didn't dig into it.

Xuanwo merged commit f15ce89 into apache:main Mar 3, 2025
244 checks passed

erickguan deleted the onedrive-metadata branch March 10, 2025 09:48

Comments

Conversation

erickguan commented Feb 16, 2025

Which issue does this PR close?

What changes are included in this PR?

Uh oh!

meteorgan left a comment

Choose a reason for hiding this comment

Uh oh!

meteorgan Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

erickguan Feb 21, 2025

Choose a reason for hiding this comment

Uh oh!

Xuanwo commented Feb 21, 2025

Uh oh!

erickguan commented Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

meteorgan commented Feb 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

erickguan commented Feb 25, 2025

Uh oh!

meteorgan commented Feb 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

erickguan commented Feb 26, 2025

Uh oh!

Xuanwo left a comment

Choose a reason for hiding this comment

Uh oh!

Xuanwo commented Feb 26, 2025

Uh oh!

erickguan commented Feb 26, 2025

Uh oh!

erickguan commented Mar 1, 2025

Uh oh!

Xuanwo commented Mar 3, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

erickguan commented Feb 21, 2025 •

edited

Loading

meteorgan commented Feb 22, 2025 •

edited

Loading

meteorgan commented Feb 26, 2025 •

edited

Loading