check_by_ssh: OpenSSH 10.1 Post-Quantum Cryptography Warning Breakage #2147
Description
Starting with the upcoming version 10.1, OpenSSH will report big warnings if the opposite server does not support post-quantum cryptography, https://www.openssh.com/pq.html.
Yesterday, I got a report from someone using current OpenSSH w/ check_by_ssh and Icinga. Their monitored server did not support PQ, so ssh(1) printed the warning text on stderr, but check_by_ssh only reported the first line and exits with UNKNOWN.
Personally also using an unstable OpenSSH version on my monitoring server, I was able to reproduce this with a server not supporting any PQ KexAlgorithms.
Using ssh(1) shows the mentioned warning on stderr (2>/dev/null hides it), but executes the command and exits with 0.
$ ssh [email protected] '/usr/lib/nagios/plugins/check_disk -c 10% -w 20% /'
** WARNING: connection is not using a post-quantum key exchange algorithm.
** This session may be vulnerable to "store now, decrypt later" attacks.
** The server may need to be upgraded. See https://openssh.com/pq.html
DISK OK - free space: / 7183MiB (54% inode=91%);| /=6355419136B;11723079680;13188464640;0;14653849600
$ echo $?
0
When using check_by_ssh, only the first line of the warning text from stderr is shown and the commands exits with 3. However, the actual command was executed, but it's output was completely ignored due to the presence of something on stderr.
monitoring-plugins/plugins/check_by_ssh.c
Lines 113 to 127 in f69aba3
$ /usr/local/libexec/nagios/check_by_ssh -C '/usr/lib/nagios/plugins/check_disk -c 10% -w 20% /' -H example.com -l icinga
Remote command execution failed: ** WARNING: connection is not using a post-quantum key exchange algorithm.
$ echo $?
3
As stated in the linked "OpenSSH Post-Quantum Cryptography" document, the new ssh_config(5) option WarnWeakCrypto was introduced. Passing -o WarnWeakCrypto=no to check_by_ssh mitigates this issue.
$ /usr/local/libexec/nagios/check_by_ssh -C '/usr/lib/nagios/plugins/check_disk -c 10% -w 20% /' -H example.com -l icinga -o WarnWeakCrypto=no
DISK OK - free space: / 7183MiB (54% inode=91%);| /=6355419136B;11723079680;13188464640;0;14653849600
$ echo $?
0
FTR, it is also possible to mitigate this via check_by_ssh's --skip-stderr option.
Since OpenSSH 10.1 will be released soon and will slowly be shipped to downstream distributions, this will lead to issues within the next months if the monitoring server is up to date, but has to monitor more ancient machines.