Skip to content

Comments

cls/rgw: fix rgw list-object-versions key-marker behavior to match s3#60280

Closed
samakshd wants to merge 1 commit intoceph:mainfrom
samakshd:list_object_versions_backup
Closed

cls/rgw: fix rgw list-object-versions key-marker behavior to match s3#60280
samakshd wants to merge 1 commit intoceph:mainfrom
samakshd:list_object_versions_backup

Conversation

@samakshd
Copy link

The key-marker parameter in the list-object-versions operation is inclusive in RGW, while it's exclusive in S3. This commit updates RGW's behavior to make the key-marker parameter exclusive, ensuring consistent behavior between RGW and S3.

Fixes: https://tracker.ceph.com/issues/68055

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows
  • jenkins test rook e2e

The key-marker parameter in the list-object-versions operation is inclusive in RGW, while it's exclusive in S3. This commit updates RGW's behavior to make the key-marker parameter exclusive, ensuring consistent behavior between RGW and S3.

Fixes: https://tracker.ceph.com/issues/68055

Signed-off-by: Samaksh Dhingra <[email protected]>
@cbodley
Copy link
Contributor

cbodley commented Dec 16, 2024

passed qa in https://pulpito.ceph.com/cbodley-2024-12-13_17:43:49-rgw-wip-65369-distro-default-smithi/

new test cases from ceph/s3-tests#594 passed:

2024-12-13T20:40:03.346 INFO:teuthology.orchestra.run.smithi083.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data_versioning_key_marker PASSED [ 38%]
2024-12-13T20:40:04.957 INFO:teuthology.orchestra.run.smithi083.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data_versioning_version_id_marker PASSED [ 38%]

@ivancich
Copy link
Member

@samakshd, thanks for noticing that! Have you looked into why this extra check is even necessary since the functions are all coded as exclusive?

rgw_bucket_list has local variables named start_after_omap_key and start_after_entry_key, which are named to show that they're intended to be exclusive.

That in turn calls get_obj_vals.

And ultimately this is called, and the parameter is named start_after, implying it's exclusive:

extern int cls_cxx_map_get_vals(cls_method_context_t hctx,  
                                const std::string& start_after,  
                                const std::string& filter_prefix, 
                                uint64_t max_to_get, 
                                std::map<std::string, ceph::buffer::list> *vals,
                                bool *more);  

If there's a logic error in that chain, I would prefer a solution that fixed it rather than checking each entry returned.

So I'm wondering if you traced that through and what ideas you might have towards that. Thanks!

Copy link
Member

@ivancich ivancich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer a fix that resolved an apparent logic error rather than checking each value.

@ivancich
Copy link
Member

Also, I want to apologize for the very late review, @samakshd!

@yuriw
Copy link
Contributor

yuriw commented Jan 16, 2025

jenkins test make check

@ceph-jenkins
Copy link
Collaborator

Can one of the admins verify this patch?

@yuriw yuriw removed the TESTED label Jan 17, 2025
@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@github-actions github-actions bot added the stale label Mar 18, 2025
@github-actions
Copy link

This pull request has been automatically closed because there has been no activity for 90 days. Please feel free to reopen this pull request (or open a new one) if the proposed change is still appropriate. Thank you for your contribution!

@github-actions github-actions bot closed this Apr 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants