Skip to content

Comments

squid: client,mds: case-insensitive directory trees#62095

Merged
batrick merged 73 commits intoceph:squidfrom
batrick:wip-70259-squid
Mar 19, 2025
Merged

squid: client,mds: case-insensitive directory trees#62095
batrick merged 73 commits intoceph:squidfrom
batrick:wip-70259-squid

Conversation

@batrick
Copy link
Member Author

batrick commented Mar 3, 2025

phlogistonjohn and others added 5 commits March 17, 2025 15:43
As distutils is removed from python 3.12 ceph can no longer use the
simple conversion function once located in that module. Add our own
trivial replacement function.

Signed-off-by: John Mulligan <[email protected]>
(cherry picked from commit 51516ba)
When passing an empty string to filepath, it would test if the first character
is '/' which is an invalid access through std::string_view.

Signed-off-by: Patrick Donnelly <[email protected]>
Fixes: https://tracker.ceph.com/issues/66373
(cherry picked from commit e62fce4)
This gets converted in messenger douts.

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit da50c6e)
Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit d6a5656)
Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit 8e88fb0)
batrick and others added 17 commits March 17, 2025 15:43
Ancillary change: do not do client upkeep after map processing.

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit 2810ed0)
Convert ms_dispatch to ms_dispatch2 to enable indicating that a map message is
acknowledged and instead of processed (or deliberately not processed).

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit f060ee8)
This avoids messages like:

    2025-02-18T05:31:17.738+0000 7f5206546640  0 ms_deliver_dispatch: unhandled message 0x5632d05f0700 fsmap(e 9) from mon.0 v2:172.21.3.230:40412/0

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit 56cb057)
Instead of marking the message as handled, give another component (or Client) a
chance to process.

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit 616fdd1)
At the time this construction seemed safe since the caller should have a
reference but it could conveivably be the only ref. We don't want the ref count
to reach 0.

Additionally, catch errors so this callback is genuinely noexcept.

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit 395263c)
For some newer versions of cython, it appears it requires explicitly specifying
noexcept but old versions of Cython 0.29.Z do not understand that attribute.

See: https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html#exception-values-and-noexcept

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit 90ac740)
invoking cmake with -DWITH_LIBCEPHFS=OFF fails to configure the client target:

CMake Error at src/client/CMakeLists.txt:13 (target_link_libraries):
  Target "client" links to:

    Boost::locale

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

because the client target is not conditional on WITH_LIBCEPHFS in src/CMakeLists.txt:

add_subdirectory(client)

if(WITH_LIBCEPHFS)

because client is also needed for ceph-fuse, make the client and its
dependencies depend on WITH_LIBCEPHFS OR WITH_FUSE

Signed-off-by: Casey Bodley <[email protected]>
(cherry picked from commit 70eb1d6)
Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit a33faa7)
Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit d62a379)
Unfortunately, it's not easy to refactor this test into a shared method without
setting up an explicit test class which has been avoided up to this point. So
I'm going to just copy the code. Sorry.

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit 28bdb8e)
In particular: there's no reason to do a getcwd after chdir.

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit 3bfd614)
This was missed in the path_walk refactor. readdir is not the only way to "get"
dentry names.

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit cda2a6e)
libcephfs semantics require that the old path to the cwd be returned when
getcwd is encounters an unlinked directory in the current working directory.

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit c7c5089)
When we wrap a dname after a path walk, we cannot do another path walk.
Originally this method was setup to permit that but with (upcoming) encryption
it cannot work. The dname from the path_walk is in the encrypted namespace so
we cannot do another lookup in a second path_walk.

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit af8d267)
Fixes: 9fbf37a
Fixes: https://tracker.ceph.com/issues/70409
Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit 5fc94be)

Conflicts:
	src/mds/journal.cc: code movement caused by referent inode changeset
For inspection from `dump tree` MDS command.

Signed-off-by: Patrick Donnelly <[email protected]>
(cherry picked from commit 7ad1937)
@batrick
Copy link
Member Author

batrick commented Mar 17, 2025

squid dependencies have been merged; will merge this when jenkins tests pass

@batrick
Copy link
Member Author

batrick commented Mar 17, 2025

jenkins test dashboard cephadm

1 similar comment
@batrick
Copy link
Member Author

batrick commented Mar 18, 2025

jenkins test dashboard cephadm

@batrick
Copy link
Member Author

batrick commented Mar 18, 2025


�[0m�[0m
�[0m  Cluster expansion host addition�[0m
  �[32m  ✓�[0m�[90m Add hosts (example #1)�[0m�[31m (55484ms)�[0m
  �[32m  ✓�[0m�[90m Add hosts (example #2)�[0m�[31m (63932ms)�[0m
  �[31m  1) Remove hosts (example #1)�[0m
  �[31m  2) Remove hosts (example #2)�[0m
  �[31m  3) Add hosts using pattern 'ceph-node-[01-02]'�[0m
  �[32m  ✓�[0m�[90m Add existing host and verify it failed�[0m�[31m (10480ms)�[0m
  �[32m  ✓�[0m�[90m Add and remove labels on host (example #1)�[0m�[31m (19712ms)�[0m


�[92m �[0m�[32m 4 passing�[0m�[90m (9m)�[0m
�[31m  3 failing�[0m

Doesn't look related...

@afreen23
Copy link
Contributor

Failing tests are not related, thanks

Copy link
Contributor

@vshankar vshankar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once QA run is reviewed.

@batrick
Copy link
Member Author

batrick commented Mar 19, 2025

@batrick batrick merged commit bacae92 into ceph:squid Mar 19, 2025
9 of 10 checks passed
@batrick batrick deleted the wip-70259-squid branch March 19, 2025 12:09
anoopcs9 added a commit to anoopcs9/ceph that referenced this pull request Sep 23, 2025
man fchownat(2)[1] says the following:
. . .
AT_EMPTY_PATH (since Linux 2.6.39)
    If pathname is an empty string, operate on the file referred to by
    dirfd (which may have been obtained using the open(2) O_PATH flag).
    In this case, dirfd can refer to any type of file, not just a
    directory. If dirfd is AT_FDCWD, the call operates on the current
    working directory.
. . .

Look out for an empty pathname and use the relative fd's inode in the
presence of AT_EMPTY_PATH flag before calling internal _setattr().

Fixes: https://tracker.ceph.com/issues/68189

Review with: git show -w

[1] https://www.man7.org/linux/man-pages/man2/fchownat.2.html

Signed-off-by: Anoop C S <[email protected]>
(cherry picked from commit 829f388)

Conflicts:
        src/client/Client.cc
- path_walk() refactor from ceph#62095
  included the required core changes.
anoopcs9 added a commit to anoopcs9/ceph that referenced this pull request Sep 23, 2025
man statx(2)[1] says the following:
. . .
AT_EMPTY_PATH
    If pathname is an empty string, operate on the file referred to by
    dirfd (which may have been obtained using the open(2) O_PATH flag).
    In this case, dirfd can refer to any type of file, not just a
    directory.

    If dirfd is AT_FDCWD, the call operates on the current working
    directory.
. . .

Look out for an empty pathname and use the relative fd's inode in the
presence of AT_EMPTY_PATH flag before calling internal _getattr().

Fixes: https://tracker.ceph.com/issues/68189

Review with: git show -w

[1] https://www.man7.org/linux/man-pages/man2/statx.2.html

Signed-off-by: Anoop C S <[email protected]>
(cherry picked from commit edd7fe7)

Conflicts:
        src/client/Client.cc
- path_walk() refactor from ceph#62095
  included the required core changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants