TLDR: The new --output=pipe option redirects stdout through a pipe instead of to /dev/null. On Linux, it's backed by splice(), so its overhead very competitive.

Benchmarks

The first benchmark is cat large, which has its output handled by --output=....
The other benchmarks are baselines:

• cat large >/dev/null is effectively --output=null
• cat large | cat >/dev/null is effectively --output=pipe on non-Linux (without splice())
• cat large | pv -q >/dev/null is effectively --output=pipe on Linux (with splice())

--output=null (the default)

Basically the same as >/dev/null:

--output=pipe

cat large is within 2-3% of | pv -q >/dev/null, which makes sense as both are using splice().
These methods are about 40% faster than | cat >/dev/null, which uses read()/write() through a pipe.

--output=/tmp/file

Writing to tmpfs is ~10% faster than writing to /dev/null through a pipe! But slower than splice(), which makes sense.

This is awesome - thank you very much! Also for the benchmarks which clearly show that this is working correctly.

I have a couple of questions before I proceed with the review:

• Should we add --output=terminal and then treat --show-output as an alias for --output=terminal?
• Concerning the Linux-implementation of --output=pipe using splice: Could that possibly cause any confusion among users? Don't get me wrong - it's very cool that we can do this. But imagine someone is comparing benchmark results on Linux and MacOS and cannot explain the difference. I mean, we would expect OS-specific differences anyway in the implementation of a pipe. And the user would have to type --output=pipe explicitly. But I wonder if it's worth mentioning that in the documentation at least? Or do we want to add something like --output=pipe and --output=pipe_splice? (I'd rather not)

This is awesome - thank you very much! Also for the benchmarks which clearly show that this is working correctly.

You're welcome!

I have a couple of questions before I proceed with the review:

• Should we add --output=terminal and then treat --show-output as an alias for --output=terminal?

Yeah I think that makes sense for consistency. Maybe --output=show/shared/inherit or something? It's not necessarily a terminal if hyperfine is redirected.

I also think we should eventually add --output=pty (what I described here), so that benchmarks will think isatty(stdout) without flooding the current terminal with output.

Which reminds me, I think --output=<FILE> should require a path separator in <FILE>, so that we can add --output=word backwards compatibly.

• Concerning the Linux-implementation of --output=pipe using splice: Could that possibly cause any confusion among users? Don't get me wrong - it's very cool that we can do this. But imagine someone is comparing benchmark results on Linux and MacOS and cannot explain the difference. I mean, we would expect OS-specific differences anyway in the implementation of a pipe. And the user would have to type --output=pipe explicitly. But I wonder if it's worth mentioning that in the documentation at least? Or do we want to add something like --output=pipe and --output=pipe_splice? (I'd rather not)

I don't see the benefit of a separate --output=pipe_splice -- if it's available, you should pretty much always use it over pipe, so why not make it the default? There's so much that's different between macOS and Linux anyway that a direct comparison is probably not informative IMO.

Yeah I think that makes sense for consistency. Maybe --output=show/shared/inherit or something? It's not necessarily a terminal if hyperfine is redirected.

Good point. I like --output=inherit which seems to be common terminology (e.g. being used in Rust or Python).

I also think we should eventually add --output=pty (what I described here), so that benchmarks will think isatty(stdout) without flooding the current terminal with output.

I remember that comment. Yes, absolutely agreed.

Which reminds me, I think --output=<FILE> should require a path separator in <FILE>, so that we can add --output=word backwards compatibly.

👍 I was thinking about this edge case ("what if a user wants to print to a file called null") but didn't post a comment because it seemed too nitpicky. But simply disallowing paths without a separator sounds like a good idea.

I don't see the benefit of a separate --output=pipe_splice -- if it's available, you should pretty much always use it over pipe, so why not make it the default? There's so much that's different between macOS and Linux anyway that a direct comparison is probably not informative IMO.

Okay.

Yeah I think that makes sense for consistency. Maybe --output=show/shared/inherit or something? It's not necessarily a terminal if hyperfine is redirected.

Good point. I like --output=inherit which seems to be common terminology (e.g. being used in Rust or Python).

Done.

Which reminds me, I think --output=<FILE> should require a path separator in <FILE>, so that we can add --output=word backwards compatibly.

+1 I was thinking about this edge case ("what if a user wants to print to a file called null") but didn't post a comment because it seemed too nitpicky. But simply disallowing paths without a separator sounds like a good idea.

Done.