libcephfs: FSCrypt userspace implementation#61137
Merged
Conversation
Member
|
sorry for the noise but i accidentally added the `wip-nia-testinglabel which i was intending to add on another. removed it btw |
3eadca1 to
75546bf
Compare
|
This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved |
027f917 to
f4dd0af
Compare
f4dd0af to
09bb9f7
Compare
|
This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved |
4bb7c3d to
8341cca
Compare
Signed-off-by: Christopher Hoffman <[email protected]>
fcopyfile() reads 1 MiB of data every time but when a fragment smaller than 1 MiB is left, it still reads 1 MiB of data, causing to never meet the condition of "off == size". This leads to an infinity loop which continues to write until CephFS becomes full. Resolves: rhbz#2379716 Fixes: https://tracker.ceph.com/issues/72238 Signed-off-by: Rishabh Dave <[email protected]>
Signed-off-by: Rishabh Dave <[email protected]>
Move "directory empty" logic to new method "_is_empty_directory". Future logic will not be a one-liner, so let's keep it separate. Resolves: rhbz#2376757 Signed-off-by: Marcus Watts <[email protected]> (cherry picked from commit f0cf85a595876165e2c0eb2ca584e97f44971f9d)
Better check for empty direcotry. It turns out in->dirstat contains a count of files and subdirectories from a directory, so all we have to do is make sure that's valid. Resolves: rhbz#2376757 Signed-off-by: Marcus Watts <[email protected]> (cherry picked from commit ba233f2dda3cf30c67b653b065e9ed47d42cb9d6)
After each fscrypt unit test clean up after each unit test. Signed-off-by: Christopher Hoffman <[email protected]>
Add fscrypt dummy encryption to client. This will allow for mounting a cephfs volume without providing any fscrypt information. This will allow for more straightforward setup for development and test suites. Signed-off-by: Christopher Hoffman <[email protected]>
Signed-off-by: Christopher Hoffman <[email protected]>
Signed-off-by: Christopher Hoffman <[email protected]>
Fix warnings/errors in ceph API tests that are present in FSCrypt.cc
src/client/FSCrypt.cc:90:6: error: variable 'olen' set but not used [-Werror,-Wunused-but-set-variable]
90 | int olen = 0;
| ^
src/client/FSCrypt.cc:91:6: error: variable 'line' set but not used [-Werror,-Wunused-but-set-variable]
91 | int line = 0;
| ^
src/client/FSCrypt.cc:945:2: error: is this the way to do it? [-Werror,-W#warnings]
945 | #warning is this the way to do it?
Signed-off-by: Christopher Hoffman <[email protected]>
Fix warnings/errors in ceph API tests that are present in various files
that were introduced by fscrypt feature
src/client/FSCrypt.cc:90:6: error: variable 'olen' set but not used [-Werror,-Wunused-but-set-variable]
90 | int olen = 0;
| ^
src/client/FSCrypt.cc:91:6: error: variable 'line' set but not used [-Werror,-Wunused-but-set-variable]
91 | int line = 0;
| ^
src/client/FSCrypt.cc:945:2: error: is this the way to do it? [-Werror,-W#warnings]
945 | #warning is this the way to do it?
src/client/Client.cc:11850:2: error: read holes [-Werror,-W#warnings]
11850 | #warning read holes
| ^
src/client/Client.cc:11855:2: error: implement file read here [-Werror,-W#warnings]
11855 | #warning implement file read here
| ^
src/client/Inode.cc:847:2: error: need to make sure that we do not skip entire subtree somehow [-Werror,-W#warnings]
847 | #warning need to make sure that we do not skip entire subtree somehow
| ^
Signed-off-by: Christopher Hoffman <[email protected]>
When calling get encryption policy via ioctl, do not display the hex str of inbuf. This buffer is not used/uninitialized so no need to display. Signed-off-by: Christopher Hoffman <[email protected]>
Signed-off-by: Christopher Hoffman <[email protected]>
ALL caps are now needed for when setting fscrypt policy Signed-off-by: Christopher Hoffman <[email protected]>
The libcephfs API call add_fscrypt_key exposes an internal fscrypt data structure. This is because a hash keyid (of the master key) is used for calls such as remove_fscrypt_key. Instead of using this structure, use a char array to obtain keyid. Fixes: https://tracker.ceph.com/issues/63293 Signed-off-by: Christopher Hoffman <[email protected]>
Once a volume/filesystem is set to use fscrypt encryption, layout cannot be changed. The configurable ec_profile, sets layout and will fail. Fixes: https://tracker.ceph.com/issues/73461 Signed-off-by: Christopher Hoffman <[email protected]>
A max io size can currently be up to INT_MAX. If it is greater, then clamp the size to INT_MAX. This conflicts with fscrypt io operations. An fscrypt, op needs to read a whole fscrypt block. The size of fscrypt block size is 4K, INT_MAX % 4K is not equal to 0. Therefore, get the nearest multiple of 4K to INT_MAX that does not go over. In the fscrypt case, this value will be used for clamping max io size. Fixes: https://tracker.ceph.com/issues/73346 Signed-off-by: Christopher Hoffman <[email protected]>
When testing large io sizes and clamping that io, consider fscrypt max io size. This max io size should be a multiple of 4K (fscrypt block size), but not to exceed INT_MAX. Signed-off-by: Christopher Hoffman <[email protected]>
The commit 154b867 reverted part of the fix seen in 2b74598. This commit will reapply any missing changes of the fix. Fixes: https://tracker.ceph.com/issues/73416 Signed-off-by: Christopher Hoffman <[email protected]>
Fix various python bindings and linting issues that arose from libcephfs fscrypt testing in pipeline. Signed-off-by: Christopher Hoffman <[email protected]>
Do not build client with fscrypt on non-linux distros. Fixes: https://tracker.ceph.com/issues/70324 Signed-off-by: Christopher Hoffman <[email protected]>
Signed-off-by: Christopher Hoffman <[email protected]>
Contributor
Author
|
/config check ok |
0fc7547 to
6ec36b4
Compare
vshankar
approved these changes
Nov 6, 2025
|
ideepika
pushed a commit
to ideepika/ceph
that referenced
this pull request
Nov 25, 2025
Removed ifdef for a failure we encountered during rebase against case sensitive feature -ceph#61137 (comment) Add debug dout when entering WriteEncMgr::read -ceph#61137 (comment) Remove FILE_RD mark_caps_dirty -ceph#61137 (comment) Add comment to various lines -ceph#61137 (comment) -ceph#61137 (comment) -ceph#61137 (comment) During write_success mark FILE_WR as dirty -ceph#61137 (comment) Signed-off-by: Christopher Hoffman <[email protected]>
JoshuaGabriel
pushed a commit
to JoshuaGabriel/ceph
that referenced
this pull request
Jan 7, 2026
Removed ifdef for a failure we encountered during rebase against case sensitive feature -ceph#61137 (comment) Add debug dout when entering WriteEncMgr::read -ceph#61137 (comment) Remove FILE_RD mark_caps_dirty -ceph#61137 (comment) Add comment to various lines -ceph#61137 (comment) -ceph#61137 (comment) -ceph#61137 (comment) During write_success mark FILE_WR as dirty -ceph#61137 (comment) Signed-off-by: Christopher Hoffman <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds fscrypt support to CephFS userspace.
Development was done in two phases. This PR will include work done in both phases.
Phase 1: Core development
Code: #52172
Functionality added:
Phase 2: Making true to fscrypt spec and feature hardening
Functionality added:
Some areas reviewers should focus on:
What’s left:
Few minor commentsAdd new capability to test existing fs workunit tests on top of fscrypt enabled dirs (create trackers for issues, fix after this PR)Implement fscrypt in libcephfs and cephfs-fuse tracker: https://tracker.ceph.com/issues/63293
Contribution Guidelines
To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an
xbetween the brackets:[x]. Spaces and capitalization matter when checking off items this way.Checklist
Show available Jenkins commands
jenkins retest this pleasejenkins test classic perfjenkins test crimson perfjenkins test signedjenkins test make checkjenkins test make check arm64jenkins test submodulesjenkins test dashboardjenkins test dashboard cephadmjenkins test apijenkins test docsjenkins render docsjenkins test ceph-volume alljenkins test ceph-volume toxjenkins test windowsjenkins test rook e2e