Skip to content

Add SSHMaster::Connection::trySetBufferSize#10765

Merged
Ericson2314 merged 2 commits intoNixOS:masterfrom
obsidiansystems:ssh-pipe-size-method
Feb 17, 2025
Merged

Add SSHMaster::Connection::trySetBufferSize#10765
Ericson2314 merged 2 commits intoNixOS:masterfrom
obsidiansystems:ssh-pipe-size-method

Conversation

@Ericson2314
Copy link
Member

Motivation

It is unused in Nix currently, but will be used in Hydra. This reflects what Hydra does in NixOS/hydra#1387.

Context

We may probably to use it more widely for better SSH store performance, but this needs to be subject to more testing before we do that.

Priorities and Process

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

@Ericson2314 Ericson2314 requested a review from thufschmitt as a code owner May 23, 2024 15:57
@Ericson2314
Copy link
Member Author

Oh wat, segfault in CI in ambient Nix (i.e. nothing to do with this PR)

/Users/runner/work/_temp/8e2e66c1-0beb-427b-912b-9e63f8be4be1.sh: line 1:  4963 Segmentation fault: 11  nix --experimental-features 'nix-command flakes' flake check -L

@roberth
Copy link
Member

roberth commented May 24, 2024

segfault in CI in ambient Nix (i.e. nothing to do with this PR)

/Users/runner/work/_temp/8e2e66c1-0beb-427b-912b-9e63f8be4be1.sh: line 1:  4963 Segmentation fault: 11  nix --experimental-features 'nix-command flakes' flake check -L

Is that on Darwin? Can you get a stack trace for it?

@vcunat
Copy link
Member

vcunat commented May 24, 2024

We didn't post here, but the fcntl calls are wrong. Setter takes an int, not a pointer. And getter takes nothing, returns value. EDIT: that's how man fcntl.2 reads to me, at least.

@Ericson2314 Ericson2314 force-pushed the ssh-pipe-size-method branch from bbeb2fc to 05be135 Compare May 26, 2024 15:00
@vcunat
Copy link
Member

vcunat commented May 26, 2024

Nit: why design the interface with size_t when we're only able to implement int?

EDIT: especially note that the (current) implementation silently overflows to non-sensical values.

@Ericson2314
Copy link
Member Author

Ericson2314 commented May 27, 2024

@vcunat ah I could assert, or that + change type to unsigned int. I just didn't want to accept negative numbers.

@roberth
Copy link
Member

roberth commented May 27, 2024

Could you add a test? We can't just rely on types here, unfortunately.
You could detect the buffer size by filling a pipe in non blocking mode, without reading it.

@vcunat
Copy link
Member

vcunat commented May 27, 2024

Accepting negative numbers? Well, if you do an unsigned int or size_t and pass a too large value, you can end up passing a negative number to the syscall. So if you don't have code somehow guarding that, I'd say it's better to make the real type visible in the API already.

But perhaps more generally: the function returns the resulting buffer size (or negative error code), so perhaps it would make sense to check the return value? Overriding buffer size surely won't be a very important error, but perhaps some debug-level logs in case the value differs from what was passed as desired size? (that could even cover these negative cases by itself)

EDIT: I don't know the customs of Nix codebase, and I only really use pure C, so... take my comments with a grain of salt.

@Ericson2314 Ericson2314 marked this pull request as draft May 27, 2024 14:19
It is unused in Nix currently, but will be used in Hydra. This reflects
what Hydra does in NixOS/hydra#1387.

We may probably to use it more widely for better SSH store performance,
but this needs to be subject to more testing before we do that.
@Ericson2314
Copy link
Member Author

Ericson2314 commented Feb 17, 2025

Yes, IMO as, one needs to assert either way to check it is within [0, INT_MAX] neither int nor size_t is better than the other. I went with size_t because I think it is more evocative and suitable for a portable interface --- someday, this should be implemented on platforms other than Linux.

@Ericson2314 Ericson2314 marked this pull request as ready for review February 17, 2025 00:47
Exposed for Hydra. We could make it fancier but with (a) new store
settings (b) switch to `ssh-ng://` both in the works, it doesn't seem
worth it.
@github-actions github-actions bot added the store Issues and pull requests concerning the Nix store label Feb 17, 2025
Comment on lines +253 to +256
fcntl(in.get(), F_SETPIPE_SZ, pipesize);
fcntl(out.get(), F_SETPIPE_SZ, pipesize);
#endif
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If any errors occur, might they be worth logging?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'll do this in the future --- e.g. if we're just going to ssh-ng:// anyways this is temporary.

@Ericson2314 Ericson2314 merged commit d1b9324 into NixOS:master Feb 17, 2025
12 checks passed
@Ericson2314 Ericson2314 deleted the ssh-pipe-size-method branch February 17, 2025 16:55
Ericson2314 added a commit that referenced this pull request Feb 17, 2025
…0765

Add `SSHMaster::Connection::trySetBufferSize` (backport #10765)
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2025-02-17-meeting-minutes-213-214/60813/1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

store Issues and pull requests concerning the Nix store

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants