squid: client,mds: case-insensitive directory trees#62095
Merged
batrick merged 73 commits intoceph:squidfrom Mar 19, 2025
Merged
squid: client,mds: case-insensitive directory trees#62095batrick merged 73 commits intoceph:squidfrom
batrick merged 73 commits intoceph:squidfrom
Conversation
Member
Author
|
This PR is under test in https://tracker.ceph.com/issues/70271. |
anthonyeleven
approved these changes
Mar 3, 2025
1938840 to
69b4548
Compare
14 tasks
This was referenced Mar 5, 2025
14 tasks
7bd6cc4 to
8e8173d
Compare
As distutils is removed from python 3.12 ceph can no longer use the simple conversion function once located in that module. Add our own trivial replacement function. Signed-off-by: John Mulligan <[email protected]> (cherry picked from commit 51516ba)
When passing an empty string to filepath, it would test if the first character is '/' which is an invalid access through std::string_view. Signed-off-by: Patrick Donnelly <[email protected]> Fixes: https://tracker.ceph.com/issues/66373 (cherry picked from commit e62fce4)
This gets converted in messenger douts. Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit da50c6e)
Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit d6a5656)
Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit 8e88fb0)
Ancillary change: do not do client upkeep after map processing. Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit 2810ed0)
Convert ms_dispatch to ms_dispatch2 to enable indicating that a map message is acknowledged and instead of processed (or deliberately not processed). Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit f060ee8)
This avoids messages like:
2025-02-18T05:31:17.738+0000 7f5206546640 0 ms_deliver_dispatch: unhandled message 0x5632d05f0700 fsmap(e 9) from mon.0 v2:172.21.3.230:40412/0
Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit 56cb057)
Instead of marking the message as handled, give another component (or Client) a chance to process. Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit 616fdd1)
At the time this construction seemed safe since the caller should have a reference but it could conveivably be the only ref. We don't want the ref count to reach 0. Additionally, catch errors so this callback is genuinely noexcept. Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit 395263c)
For some newer versions of cython, it appears it requires explicitly specifying noexcept but old versions of Cython 0.29.Z do not understand that attribute. See: https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html#exception-values-and-noexcept Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit 90ac740)
invoking cmake with -DWITH_LIBCEPHFS=OFF fails to configure the client target:
CMake Error at src/client/CMakeLists.txt:13 (target_link_libraries):
Target "client" links to:
Boost::locale
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
because the client target is not conditional on WITH_LIBCEPHFS in src/CMakeLists.txt:
add_subdirectory(client)
if(WITH_LIBCEPHFS)
because client is also needed for ceph-fuse, make the client and its
dependencies depend on WITH_LIBCEPHFS OR WITH_FUSE
Signed-off-by: Casey Bodley <[email protected]>
(cherry picked from commit 70eb1d6)
Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit a33faa7)
Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit d62a379)
Unfortunately, it's not easy to refactor this test into a shared method without setting up an explicit test class which has been avoided up to this point. So I'm going to just copy the code. Sorry. Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit 28bdb8e)
In particular: there's no reason to do a getcwd after chdir. Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit 3bfd614)
This was missed in the path_walk refactor. readdir is not the only way to "get" dentry names. Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit cda2a6e)
libcephfs semantics require that the old path to the cwd be returned when getcwd is encounters an unlinked directory in the current working directory. Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit c7c5089)
When we wrap a dname after a path walk, we cannot do another path walk. Originally this method was setup to permit that but with (upcoming) encryption it cannot work. The dname from the path_walk is in the encrypted namespace so we cannot do another lookup in a second path_walk. Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit af8d267)
Fixes: https://tracker.ceph.com/issues/70409 Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit db27845)
Fixes: 9fbf37a Fixes: https://tracker.ceph.com/issues/70409 Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit 5fc94be) Conflicts: src/mds/journal.cc: code movement caused by referent inode changeset
For inspection from `dump tree` MDS command. Signed-off-by: Patrick Donnelly <[email protected]> (cherry picked from commit 7ad1937)
8e8173d to
82ef5fb
Compare
Member
Author
|
squid dependencies have been merged; will merge this when jenkins tests pass |
Member
Author
|
jenkins test dashboard cephadm |
1 similar comment
Member
Author
|
jenkins test dashboard cephadm |
Member
Author
Doesn't look related... |
Contributor
|
Failing tests are not related, thanks |
vshankar
approved these changes
Mar 19, 2025
Contributor
vshankar
left a comment
There was a problem hiding this comment.
LGTM once QA run is reviewed.
Member
Author
|
anoopcs9
added a commit
to anoopcs9/ceph
that referenced
this pull request
Sep 23, 2025
man fchownat(2)[1] says the following:
. . .
AT_EMPTY_PATH (since Linux 2.6.39)
If pathname is an empty string, operate on the file referred to by
dirfd (which may have been obtained using the open(2) O_PATH flag).
In this case, dirfd can refer to any type of file, not just a
directory. If dirfd is AT_FDCWD, the call operates on the current
working directory.
. . .
Look out for an empty pathname and use the relative fd's inode in the
presence of AT_EMPTY_PATH flag before calling internal _setattr().
Fixes: https://tracker.ceph.com/issues/68189
Review with: git show -w
[1] https://www.man7.org/linux/man-pages/man2/fchownat.2.html
Signed-off-by: Anoop C S <[email protected]>
(cherry picked from commit 829f388)
Conflicts:
src/client/Client.cc
- path_walk() refactor from ceph#62095
included the required core changes.
anoopcs9
added a commit
to anoopcs9/ceph
that referenced
this pull request
Sep 23, 2025
man statx(2)[1] says the following:
. . .
AT_EMPTY_PATH
If pathname is an empty string, operate on the file referred to by
dirfd (which may have been obtained using the open(2) O_PATH flag).
In this case, dirfd can refer to any type of file, not just a
directory.
If dirfd is AT_FDCWD, the call operates on the current working
directory.
. . .
Look out for an empty pathname and use the relative fd's inode in the
presence of AT_EMPTY_PATH flag before calling internal _getattr().
Fixes: https://tracker.ceph.com/issues/68189
Review with: git show -w
[1] https://www.man7.org/linux/man-pages/man2/statx.2.html
Signed-off-by: Anoop C S <[email protected]>
(cherry picked from commit edd7fe7)
Conflicts:
src/client/Client.cc
- path_walk() refactor from ceph#62095
included the required core changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dependencies:
backport trackers:
backport of #60746
parent tracker: https://tracker.ceph.com/issues/66373
this backport was staged using ceph-backport.sh version 16.0.0.6848
find the latest version at https://github.com/ceph/ceph/blob/main/src/script/ceph-backport.sh