net: arrange zero-copy of os.File and TCPConn to UnixConn #58808
Description
Background
For the most commonly used stream-oriented/stream-like file descriptor types that can benefit the most from Linux zero-copy tech: TCP socket, Unix domain socket, and regular file, Go currently supports these cases of zero-copy:
- From TCP socket to TCP socket
- From Unix socket to TCP socket
- From file to TCP socket
- From TCP socket to file
- From Unix socket to file
- From file to file
from which the zero-copy supports whose destinations are Unix domain sockets are missing.
Since io.Copy(dst Writer, src Reader) leverages the src.(WriterTo).WriteTo or dst.(ReaderFrom).ReadFrom to try zero-copy for data transfer internally, and net.UnixConn has already implemented net.PacketConn that has WriteTo and ReadFrom methods with different function signatures, which means there is no way for us to achieve the zero-copy by implementing these two methods for net.UnixConn, thus, the zero-copy case unix-socket-to-unix-socket is off the table.
Proposal
As mentioned above, unix-socket-to-unix-socket is unlikely to accomplish, fortunately we can still support zero-copy of tcp-socket-to-unix-socket and file-to-unix-socket, therefore, I propose supporting zero-copy with destinations of Unix sockets by implementing io.WriterTo interface on os.File and net.TCPConn.
Once the implementation of this proposal is done, it will make the zero-copy technique support in Go more complete.
API changes
There will be one new method for net.TCPConn and os.File: WriteTo(io.Writer).
Benefit
Performance improvement
Benchmarks
goos: linux goarch: amd64 pkg: net cpu: DO-Premium-Intel │ old │ new │ │ sec/op │ sec/op vs base │ Splice/tcp-to-unix/1024-4 3.560µ ± 24% 5.165µ ± 27% +45.10% (p=0.001 n=10) Splice/tcp-to-unix/2048-4 4.402µ ± 21% 5.295µ ± 10% +20.29% (p=0.007 n=10) Splice/tcp-to-unix/4096-4 6.161µ ± 11% 5.535µ ± 19% ~ (p=0.105 n=10) Splice/tcp-to-unix/8192-4 7.009µ ± 22% 7.557µ ± 15% ~ (p=0.218 n=10) Splice/tcp-to-unix/16384-4 11.466µ ± 25% 7.228µ ± 25% -36.96% (p=0.000 n=10) Splice/tcp-to-unix/32768-4 19.262µ ± 34% 9.707µ ± 14% -49.60% (p=0.000 n=10) Splice/tcp-to-unix/65536-4 41.19µ ± 18% 19.77µ ± 16% -52.01% (p=0.000 n=10) Splice/tcp-to-unix/131072-4 85.08µ ± 10% 30.50µ ± 24% -64.15% (p=0.000 n=10) Splice/tcp-to-unix/262144-4 197.60µ ± 19% 55.84µ ± 12% -71.74% (p=0.000 n=10) Splice/tcp-to-unix/524288-4 345.8µ ± 16% 144.4µ ± 20% -58.24% (p=0.000 n=10) Splice/tcp-to-unix/1048576-4 696.2µ ± 12% 265.1µ ± 9% -61.93% (p=0.000 n=10) geomean 30.94µ 18.80µ -39.26%│ old │ new │ │ B/s │ B/s vs base │Splice/tcp-to-unix/1024-4 274.5Mi ± 20% 189.1Mi ± 21% -31.12% (p=0.001 n=10)
Splice/tcp-to-unix/2048-4 443.7Mi ± 20% 369.1Mi ± 9% -16.81% (p=0.007 n=10)
Splice/tcp-to-unix/4096-4 634.1Mi ± 13% 705.8Mi ± 24% ~ (p=0.105 n=10)
Splice/tcp-to-unix/8192-4 1.090Gi ± 20% 1.010Gi ± 15% ~ (p=0.218 n=10)
Splice/tcp-to-unix/16384-4 1.333Gi ± 20% 2.111Gi ± 20% +58.41% (p=0.000 n=10)
Splice/tcp-to-unix/32768-4 1.584Gi ± 25% 3.144Gi ± 12% +98.44% (p=0.000 n=10)
Splice/tcp-to-unix/65536-4 1.482Gi ± 15% 3.113Gi ± 19% +110.04% (p=0.000 n=10)
Splice/tcp-to-unix/131072-4 1.435Gi ± 11% 4.003Gi ± 19% +179.00% (p=0.000 n=10)
Splice/tcp-to-unix/262144-4 1.236Gi ± 16% 4.372Gi ± 10% +253.65% (p=0.000 n=10)
Splice/tcp-to-unix/524288-4 1.415Gi ± 14% 3.384Gi ± 25% +139.18% (p=0.000 n=10)
Splice/tcp-to-unix/1048576-4 1.403Gi ± 13% 3.691Gi ± 10% +163.08% (p=0.000 n=10)
geomean 1010.4Mi 1.625Gi +64.73%│ old │ new │ │ B/op │ B/op vs base │Splice/tcp-to-unix/1024-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/2048-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/4096-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/8192-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/16384-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/32768-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/65536-4 1.000 ± ? 0.000 ± 0% -100.00% (p=0.011 n=10)
Splice/tcp-to-unix/131072-4 2.000 ± 50% 0.000 ± 0% -100.00% (p=0.000 n=10)
Splice/tcp-to-unix/262144-4 5.000 ± 20% 0.000 ± 0% -100.00% (p=0.000 n=10)
Splice/tcp-to-unix/524288-4 9.000 ± 22% 0.000 ± 0% -100.00% (p=0.000 n=10)
Splice/tcp-to-unix/1048576-4 17.50 ± 20% 0.00 ± 0% -100.00% (p=0.000 n=10)
geomean ² ? ² ³
¹ all samples are equal
² summaries must be >0 to compute geomean
³ ratios must be >0 to compute geomean│ old │ new │ │ allocs/op │ allocs/op vs base │
Splice/tcp-to-unix/1024-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/2048-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/4096-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/8192-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/16384-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/32768-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/65536-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/131072-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/262144-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/524288-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/1048576-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
geomean ² +0.00% ²
¹ all samples are equal
² summaries must be >0 to compute geomean