Project

General

Profile

Actions
Copy link

Tasks #68798

closed

Tasks #63293: Implement fscrypt in libcephfs and cephfs-fuse

fscrypt unlock hangs

Added by Christopher Hoffman over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Christopher Hoffman
Category:
-
Target version:
-
% Done:

0%

Reviewed:
Affected Versions:
Component(FS):
Labels (FS):
Pull request ID:
Tags (freeform):
Merge Commit:
Fixed In:
Released In:
Upkeep Timestamp:

Description

fscrypt unlock <dir> hangs.

It hangs during test_fscrypt suite setUp:

python3 ../qa/tasks/vstart_runner.py tasks.cephfs.test_fscrypt.TestFSCryptRMW --debug --interactive

The issue is present after a few tests run and several setUp/tearDown.

2024-10-31 17:20:23,913.913 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /cephfs/ceph/build/asok/client.1.272850.asok
2024-10-31 17:20:23,913.913 INFO:__main__:> sudo ./bin/ceph --admin-daemon /cephfs/ceph/build/asok/client.1.272850.asok status
2024-10-31 17:20:24,067.067 INFO:__main__:> stat --file-system '--printf=%T
' -- /tmp/tmp5qb_ditx/mnt.1
2024-10-31 17:20:24,069.069 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/tmp5qb_ditx/mnt.1
2024-10-31 17:20:24,070.070 INFO:__main__:> sudo chmod 1777 /tmp/tmp5qb_ditx/mnt.1
2024-10-31 17:20:24,078.078 INFO:__main__:> /cephfs/ceph/build/teuthology/venv-teuth/bin/stdin-killer --timeout=900 -- bash -c 'sudo fscrypt --help'
2024-10-31 17:20:24,128.128 INFO:__main__:> /cephfs/ceph/build/teuthology/venv-teuth/bin/stdin-killer --timeout=900 -- bash -c 'sudo fscrypt setup --help'
2024-10-31 17:20:24,178.178 INFO:__main__:> /cephfs/ceph/build/teuthology/venv-teuth/bin/stdin-killer --timeout=900 -- bash -c 'sudo fscrypt setup --force --quiet'
2024-10-31 17:20:28,234.234 INFO:__main__:> /cephfs/ceph/build/teuthology/venv-teuth/bin/stdin-killer --timeout=900 -- bash -c 'sudo fscrypt status'
2024-10-31 17:20:28,288.288 INFO:__main__:> /cephfs/ceph/build/teuthology/venv-teuth/bin/stdin-killer --timeout=900 -- bash -c 'sudo fscrypt setup --quiet /tmp/tmp5qb_ditx/mnt.0'
2024-10-31 17:20:28,632.632 INFO:__main__:> /cephfs/ceph/build/teuthology/venv-teuth/bin/stdin-killer --timeout=900 -- bash -c 'sudo fscrypt status'
2024-10-31 17:20:28,746.746 INFO:__main__:> /cephfs/ceph/build/teuthology/venv-teuth/bin/stdin-killer --timeout=900 -- bash -c 'sudo dd if=/dev/urandom of=/tmp/key bs=32 count=1'
2024-10-31 17:20:28,796.796 INFO:__main__:> /cephfs/ceph/build/teuthology/venv-teuth/bin/stdin-killer --timeout=900 -- bash -c 'mkdir -p dir/'
2024-10-31 17:20:28,844.844 INFO:__main__:> /cephfs/ceph/build/teuthology/venv-teuth/bin/stdin-killer --timeout=900 -- bash -c 'sudo fscrypt encrypt --quiet --source=raw_key --name=sUlMTZRR --no-recovery --skip-unlock --key=/tmp/key dir/'
2024-10-31 17:20:29,598.598 INFO:__main__:> /cephfs/ceph/build/teuthology/venv-teuth/bin/stdin-killer --timeout=900 -- bash -c 'sudo fscrypt unlock --quiet --key=/tmp/key dir/'
2024-10-31 17:20:29,682.682 INFO:__main__:> /cephfs/ceph/build/teuthology/venv-teuth/bin/stdin-killer --timeout=900 -- bash -c 'sudo fscrypt unlock --quiet --key=/tmp/key dir/'

Related issues 1 (0 open — 1 closed)

Related to CephFS - Bug #68831: fix LibCephFS.DoubleChmod testResolvedChristopher Hoffman

Actions
Actions
Copy link
 #1

Updated by Christopher Hoffman over 1 year ago

  • Related to Bug #68831: fix LibCephFS.DoubleChmod test added
Actions
Copy link
 #2

Updated by Christopher Hoffman over 1 year ago

Running into the same issue as https://tracker.ceph.com/issues/68831

There's caps that are still present:

2024-11-07T19:50:11.611+0000 7f605e08e680  1 client.6850 dump_inode: DISCONNECTED inode 0x100000001fb #0x100000001fb ref 2 0x100000001fb.head(faked_ino=0 nref=2 ll_ref=0 cap_refs={4=0,1024=3,2048=0,4096=0,8192=0} open={1=0,3=0} mode=100664 size=65536/0 nlink=1 btime=2024-11-07T19:47:30.425545+0000 mtime=2024-11-07T19:48:09.923923+0000 ctime=2024-11-07T19:48:09.923923+0000 change_attr=65551 caps=- objectset[0x100000001fb ts 0/0 objects 0 dirty_or_tx 0] 0x7f5ff800d380)
2024-11-07T19:50:11.611+0000 7f605e08e680  2 client.6850 cache still has 0+1 items, waiting (for caps to release?)

Actions
Copy link
 #3

Updated by Christopher Hoffman over 1 year ago

  • Status changed from New to Resolved
  • Assignee set to Christopher Hoffman

This is resolved. CEPH_CAP_FILE_CACHE cap was not being released. This problem was introduced during a rebase against main.

commit 1b3615e17a2fc091e120edf0128030aa1bffdaf1 (HEAD -> wip-fscrypt, origin/wip-fscrypt)
Author: Christopher Hoffman <[email protected]>
Date:   Fri Nov 8 14:00:43 2024 +0000

    client: Ensure file_cache cap is released.

    Fixes: https://tracker.ceph.com/issues/68798
    Fixes: https://tracker.ceph.com/issues/68831
    Signed-off-by: Christopher Hoffman <[email protected]>

diff --git a/src/client/Client.cc b/src/client/Client.cc
index 3741a879d41..dfd47b027f1 100644
--- a/src/client/Client.cc
+++ b/src/client/Client.cc
@@ -11447,6 +11447,8 @@ int Client::_read_async(Fh *f, uint64_t off, uint64_t len, bufferlist *bl,
     client_lock.lock();
        ldout(cct, 10) << "aahere7" <<dendl;
     put_cap_ref(in, CEPH_CAP_FILE_CACHE);
+  } else {
+    put_cap_ref(in, CEPH_CAP_FILE_CACHE);
   }

   if (r >= 0) {
@@ -11462,8 +11464,6 @@ int Client::_read_async(Fh *f, uint64_t off, uint64_t len, bufferlist *bl,
     r = bl->length();

     update_read_io_size(bl->length());
-  } else {
-    put_cap_ref(in, CEPH_CAP_FILE_CACHE);
   }
Actions
Copy link

Also available in: Atom PDF