Skip to content

Comments

feat(bindings/python): Enhance Stat, Lister, Metadata & Entry #6232

Merged
Xuanwo merged 17 commits intoapache:mainfrom
chitralverma:py-update-lister-entry
Jun 9, 2025
Merged

feat(bindings/python): Enhance Stat, Lister, Metadata & Entry #6232
Xuanwo merged 17 commits intoapache:mainfrom
chitralverma:py-update-lister-entry

Conversation

@chitralverma
Copy link
Contributor

@chitralverma chitralverma commented May 29, 2025

Which issue does this PR close?

None

Rationale for this change

Lister, Metadata & Entry in python bindings had some missing capabilities, this PR adds them.

What changes are included in this PR?

  • Add Metadata to Entry which was missing
  • Add missing keys to Metadata
  • Support all ListOptions for list & scan
  • Simplify scan as list with recursive = True
  • Support all StatOptions for stat
  • Lister on local fs did not return most of the metadata, fixed that in sync with stat

Are there any user-facing changes?

Yes, new things are added to Entry, Metadata and list/ scan/ stat functions.

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label May 29, 2025
@chitralverma chitralverma marked this pull request as draft May 29, 2025 15:19
@chitralverma chitralverma changed the title draft: feat(bindings/python): Enhance Lister, Metadata & Entry feat(bindings/python): Enhance Lister, Metadata & Entry May 29, 2025
@dosubot dosubot bot added the releases-note/feat The PR implements a new feature or has a title that begins with "feat" label May 29, 2025
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM!

@chitralverma
Copy link
Contributor Author

@Xuanwo @asukaminato0721 all your suggestions have been included now,.
Please let me know if there's anything else pending before merging this.

@chitralverma chitralverma force-pushed the py-update-lister-entry branch from 691afbb to 50255f4 Compare June 3, 2025 07:10
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Jun 3, 2025
@chitralverma chitralverma changed the title feat(bindings/python): Enhance Lister, Metadata & Entry feat(bindings/python): Enhance Stat, Lister, Metadata & Entry Jun 3, 2025
@chitralverma
Copy link
Contributor Author

Added options to stat as well and updated PR description.
Updated with the deprecated decorator.

@chitralverma
Copy link
Contributor Author

@Xuanwo @asukaminato0721 Please let me know if there's anything else pending before merging this.
I think the PR is ready now.

@chitralverma chitralverma requested a review from Zheaoli June 4, 2025 15:54
(&rel_path, EntryMode::Unknown)
};

let meta = tokio::fs::metadata(self.root.join(path))
Copy link
Member

@Xuanwo Xuanwo Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, please don't change this. This will make the list run several times slower than before. Users who need content_length should call stat themselves. There is no guarantee that the list can retrieve metadata for all services.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, i understand, let me revert this.

There is no guarantee that the list can retrieve metadata for all services.

may be this can be a note somewhere in core on the Lister/ list() ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may be this can be a note somewhere in core on the Lister/ list() ?

Yes, I agree that we should document that list does not return the complete metadata for a file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raised #6272

@asukaminato0721 asukaminato0721 self-requested a review June 4, 2025 17:27
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @chitralverma for working on this and thank you @asukaminato0721 for the review!

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jun 9, 2025
@Xuanwo Xuanwo merged commit cdb9871 into apache:main Jun 9, 2025
63 of 64 checks passed
@chitralverma chitralverma deleted the py-update-lister-entry branch June 9, 2025 11:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bindings/python lgtm This PR has been approved by a maintainer releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants