Skip to content

Comments

[DNM] libcephfs: fscrypt support#52172

Closed
yehudasa wants to merge 61 commits intoceph:mainfrom
yehudasa:wip-libcephfs-fscrypt
Closed

[DNM] libcephfs: fscrypt support#52172
yehudasa wants to merge 61 commits intoceph:mainfrom
yehudasa:wip-libcephfs-fscrypt

Conversation

@yehudasa
Copy link
Member

@yehudasa yehudasa commented Jun 23, 2023 •
edited
Loading

This implements fscrypt support in libcephfs, and the cephfs userspace client that is interoperable with the cephfs kernel client (see: https://lwn.net/Articles/889912/).

Note that this is still in heavy development.

Currently implemented:

  • Key management infrastructure
  • Can use [a modified] fscrypt utility [1] with fuse mounted filesystem
  • Supports "lock", and "unlock", as well as other required operations
  • Listing encrypted directories
  • Encrypted files read
  • Encrypted files write
  • libcephfs control API

TODO:

  • Currently cannot select encryption types (uses defaults)
  • Add libcephfs API to control crypto keys
  • Long file names support
  • Cleanly deal with the read caps needed when writing data
  • read-modify race detection when writing files
  • symlinks implementation
  • snapshots
  • async API
  • restrict certain operations to only happen when directory is unlocked (e.g., read file, create snapshots, etc.)
  • Probably more filesystem operations coverage (e.g., where there are multiple implementations of different functionalities, etc.)
  • testing

We currently only support the default encryption types that the fscrypt control tool sets. The code does handle selection of encryption types dynamically, and gracefully handles the case where it cannot support an encryption type. Since the fscrypt tool does not make it easy to set encryption types, and since libopenssl does not support every type/mode out of the box, we should consider in the future which encryption types we want to support, however this is beyond the scope of this PR.

[1] fscrypt utility is modified due to use of ioctls that are not well formed (a problem with fuse), see https://github.com/yehudasa/fscrypt/tree/wip-ceph-fuse

Contribution Guidelines

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

@yehudasa yehudasa added feature cephfs Ceph File System labels Jun 23, 2023
@yehudasa yehudasa requested review from a team as code owners June 23, 2023 09:39
@yehudasa yehudasa force-pushed the wip-libcephfs-fscrypt branch from 719cf7e to 6f64325 Compare June 23, 2023 09:42
@github-actions
Copy link

github-actions bot commented Jun 23, 2023

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@yehudasa yehudasa force-pushed the wip-libcephfs-fscrypt branch from 6f64325 to 803b42e Compare June 23, 2023 09:49
@yehudasa yehudasa removed the request for review from a team June 23, 2023 12:52
@yehudasa yehudasa force-pushed the wip-libcephfs-fscrypt branch from 24fd517 to b8c0bb8 Compare July 5, 2023 08:28
@lxbsz lxbsz requested a review from a team July 27, 2023 05:02
@lxbsz lxbsz self-assigned this Jul 27, 2023
@lxbsz
Copy link
Member

lxbsz commented Jul 27, 2023 •
edited
Loading

@yehudasa Please enable the qa test cases for fscrypt for libephfs at the same time and make sure all the test pass before we merging it.

@yehudasa yehudasa force-pushed the wip-libcephfs-fscrypt branch from 70617d0 to ebf3ead Compare September 8, 2023 08:26
@github-actions github-actions bot added the tests label Sep 8, 2023
@yehudasa yehudasa force-pushed the wip-libcephfs-fscrypt branch from ebf3ead to 1c54d80 Compare September 8, 2023 10:48
@yehudasa
fscypt: userspace client support for add key ioctl (initial)
ce08eb7 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
common: ceph_crypto: define HMACSHA512
c84cc1f 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: hkdf calc function
2be2db7 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: calculate key identifier when adding key
5206076 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: initial work on key repo
2580c73 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: get policy ioctl returns data stored on inode
8d9ab2b 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: basic support for FS_IOC_GET_ENCRYPTION_KEY_STATUS
8ae7c76 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: support key removal
10247d1 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: basic remove key ioctl support
a43fa0b 
Signed-off-by: Yehuda Sadeh <[email protected]>
yehudasa and others added 9 commits September 27, 2023 05:08
@yehudasa
fscrypt: switch to ldout
592ef59 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: conditional caps change
59e90a3 
when opening encrypted files for write, also get the read cap

Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: handle truncate call
ca59217 
round physical size, and update fscrypt_file to reflect requested size

Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: return read only FS only if not unlocked
836c8d4 
The previous behavior was returning it unconditionally

Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: readdir fixes
75cf533 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
test/libcephfs: add fscrypt test wrapper
922b44e 
encrypt a directory, and execute regular tests on it

Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: async read support
7554e59 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: async write support
5147e0f 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
client: indentation fixes
8fc5199 
Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa yehudasa force-pushed the wip-libcephfs-fscrypt branch from c81c97d to 8fc5199 Compare September 27, 2023 09:12
@yehudasa
test/client: add ceph_test_client_fscrypt
aa89bf7 
Add more fscrypt unitest coverage, specifically it also tests the
nonblocking api.

Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
test/ceph_test_client: nonblocking, adjust test parameters
9a21e07 
to trigger more fscrypt related cases

Signed-off-by: Yehuda Sadeh <[email protected]>
@yehudasa
fscrypt: add missing file
00b4884 
Signed-off-by: Yehuda Sadeh <[email protected]>
@github-actions
Copy link

github-actions bot commented Dec 10, 2023

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@github-actions github-actions bot added the stale label Dec 10, 2023
@yehudasa yehudasa removed the stale label Dec 12, 2023
@github-actions
Copy link

github-actions bot commented Feb 10, 2024

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@github-actions github-actions bot added the stale label Feb 10, 2024
@github-actions
Copy link

github-actions bot commented Mar 11, 2024

This pull request has been automatically closed because there has been no activity for 90 days. Please feel free to reopen this pull request (or open a new one) if the proposed change is still appropriate. Thank you for your contribution!

@github-actions github-actions bot closed this Mar 11, 2024
@yehudasa yehudasa reopened this Mar 11, 2024
@yehudasa
Copy link
Member Author

yehudasa commented Mar 11, 2024

@gregsfortytwo is there a new PR that replaces this one?

@gregsfortytwo
Copy link
Member

gregsfortytwo commented Mar 11, 2024

@gregsfortytwo is there a new PR that replaces this one?

I know @chrisphoffman has been doing work on top of this at https://github.com/chrisphoffman/ceph/commits/wip-fscrypt/, but I don't think we have another PR for it?

@github-actions github-actions bot removed the stale label Mar 11, 2024
@chrisphoffman
Copy link
Contributor

chrisphoffman commented Mar 12, 2024

@gregsfortytwo is there a new PR that replaces this one?

I know @chrisphoffman has been doing work on top of this at https://github.com/chrisphoffman/ceph/commits/wip-fscrypt/, but I don't think we have another PR for it?

That branch isn't ready for PR yet, the plan is to create one soon.

@github-actions
Copy link

github-actions bot commented May 11, 2024

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@github-actions github-actions bot added the stale label May 11, 2024
@github-actions
Copy link

github-actions bot commented Jun 10, 2024

This pull request has been automatically closed because there has been no activity for 90 days. Please feel free to reopen this pull request (or open a new one) if the proposed change is still appropriate. Thank you for your contribution!

@github-actions github-actions bot closed this Jun 10, 2024
@chrisphoffman chrisphoffman mentioned this pull request Dec 18, 2024
Merged
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@lxbsz lxbsz

Labels

build/ops cephfs Ceph File System common core feature needs-rebase stale tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@yehudasa @lxbsz @gregsfortytwo @chrisphoffman