Skip to content

Commit 4cc176c

Browse files
authored
zstd: Add true concurrent stream encoding (#1136)
* zstd: Add true concurrent encoding ```go // WithConcurrentBlocks enables job-based parallel compression for streams. // When enabled and concurrent > 1, input is split into large sections (jobs) // that are compressed simultaneously by multiple goroutines. // Each non-first job receives an overlap prefix from the previous job for match context. // Output is flushed in order, producing a valid single-frame zstd stream. // // This is incompatible with dictionary encoding. // Cannot be changed with ResetWithOptions. func WithConcurrentBlocks(b bool) EOption { ``` Example: | Level | 1 thread | 4 threads | 16 threads | 1T ratio | 16T ratio | |---------|:----------:|:------------------:|:-------------------:|:--------:|:---------:| | fastest | 783 MB/s | 2950 MB/s (3.8×) | 6939 MB/s (8.9×) | 12.24% | 12.26% | | default | 728 MB/s | 2533 MB/s (3.5×) | 5340 MB/s (7.3×) | 10.67% | 10.68% | | better | 434 MB/s | 1105 MB/s (2.5×) | 2206 MB/s (5.1×) | 9.14% | 9.21% | | best | 129 MB/s | 367 MB/s (2.8×) | 884 MB/s (6.8×) | 8.48% | 8.63% | * Tweaks * Just disable when using dictionary encoding. * Fix Close nil-writer check ignoring concurrent buffer Close() checked s.w == nil before the e.o.concurrentBlocks branch. In concurrent mode, Write fills js.filling (not s.filling), so the guard len(s.filling) == 0 && s.nInput == 0 would incorrectly return nil even with pending data. Fix: moved the concurrentBlocks branch before the nil-writer check, and added a matching nil-writer guard inside closeJobs() that checks js.filling instead of s.filling. * Clean up reuse * Share encoder channel with single-shot.
1 parent 4705fcb commit 4cc176c

11 files changed

Lines changed: 2191 additions & 22 deletions

‎zstd/README.md‎

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,14 +75,47 @@ The above is fine for big encodes. However, whenever possible try to *reuse* the
7575
To reuse the encoder, you can use the `Reset(io.Writer)` function to change to another output.
7676
This will allow the encoder to reuse all resources and avoid wasteful allocations.
7777

78-
Currently stream encoding has 'light' concurrency, meaning up to 2 goroutines can be working on part
79-
of a stream. This is independent of the `WithEncoderConcurrency(n)`, but that is likely to change
78+
By default, stream encoding has 'light' concurrency, meaning up to 2 goroutines can be working on part
79+
of a stream. This is independent of the `WithEncoderConcurrency(n)`, but that is likely to change
8080
in the future. So if you want to limit concurrency for future updates, specify the concurrency
8181
you would like.
8282

8383
If you would like stream encoding to be done without spawning async goroutines, use `WithEncoderConcurrency(1)`
8484
which will compress input as each block is completed, blocking on writes until each has completed.
8585

86+
#### Parallel Stream Compression
87+
88+
For maximum throughput on large streams, use `WithConcurrentBlocks(true)` together with
89+
`WithEncoderConcurrency(n)` where n is the number of CPU cores you want to use.
90+
This splits the input into large sections (jobs) that are compressed simultaneously by multiple goroutines,
91+
similar to how the C zstd library does multithreaded compression.
92+
93+
```Go
94+
enc, err := zstd.NewWriter(out,
95+
zstd.WithEncoderLevel(zstd.SpeedDefault),
96+
zstd.WithEncoderConcurrency(runtime.GOMAXPROCS(0)),
97+
zstd.WithConcurrentBlocks(true),
98+
)
99+
```
100+
101+
Each non-first job receives an overlap prefix from the previous job for match context,
102+
so compression ratio is only marginally affected. Output is flushed in order,
103+
producing a valid single-frame zstd stream.
104+
105+
Benchmark on 1.8GB GOB stream (AMD Ryzen 9 9950X):
106+
107+
| Level | 1 thread | 4 threads | 16 threads | 1T ratio | 16T ratio |
108+
|---------|:----------:|:------------------:|:-------------------:|:--------:|:---------:|
109+
| fastest | 783 MB/s | 2950 MB/s (3.8×) | 6939 MB/s (8.9×) | 12.24% | 12.26% |
110+
| default | 728 MB/s | 2533 MB/s (3.5×) | 5340 MB/s (7.3×) | 10.67% | 10.68% |
111+
| better | 434 MB/s | 1105 MB/s (2.5×) | 2206 MB/s (5.1×) | 9.14% | 9.21% |
112+
| best | 129 MB/s | 367 MB/s (2.8×) | 884 MB/s (6.8×) | 8.48% | 8.63% |
113+
114+
Notes:
115+
* Not compatible with dictionary encoding.
116+
* `Flush()` dispatches the current partial job, so latency-sensitive callers can force output.
117+
* `EncodeAll` is unaffected — it uses its own concurrency via the encoder pool.
118+
86119
You can specify your desired compression level using `WithEncoderLevel()` option. Currently only pre-defined
87120
compression settings can be specified.
88121

‎zstd/enc_base.go‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,34 @@ func (e *fastBase) matchlen(s, t int32, src []byte) int32 {
128128
return int32(matchLen(src[s:], src[t:]))
129129
}
130130

131+
// resetBasePrefix resets the encoder state and loads prefix as initial history.
132+
// This is used for parallel job encoding where non-first jobs need overlap context.
133+
// Rep offsets are set to defaults [1,4,8] (invalidated, matching C behavior).
134+
func (e *fastBase) resetBasePrefix(prefix []byte) {
135+
if e.blk == nil {
136+
e.blk = &blockEnc{lowMem: e.lowMem}
137+
e.blk.init()
138+
} else {
139+
e.blk.reset(nil)
140+
}
141+
e.blk.initNewEncode()
142+
if e.crc == nil {
143+
e.crc = xxhash.New()
144+
} else {
145+
e.crc.Reset()
146+
}
147+
e.blk.dictLitEnc = nil
148+
e.ensureHist(len(prefix) + maxCompressedBlockSize)
149+
// Bump cur so old table entries fall outside the window.
150+
// When cur >= bufferReset, leave it; the first Encode call
151+
// will shift/clear tables, preserving valid prefix entries.
152+
if e.cur < e.bufferReset {
153+
e.cur += e.maxMatchOff + int32(len(e.hist))
154+
}
155+
e.hist = e.hist[:0]
156+
e.hist = append(e.hist, prefix...)
157+
}
158+
131159
// Reset the encoding table.
132160
func (e *fastBase) resetBase(d *dict, singleBlock bool) {
133161
if e.blk == nil {

‎zstd/enc_best.go‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -551,3 +551,18 @@ func (e *bestFastEncoder) Reset(d *dict, singleBlock bool) {
551551
// Reset table to initial state
552552
copy(e.table[:], e.dictTable)
553553
}
554+
555+
func (e *bestFastEncoder) ResetPrefix(prefix []byte) {
556+
e.resetBasePrefix(prefix)
557+
if len(prefix) < 8 {
558+
return
559+
}
560+
end := e.cur + int32(len(prefix)) - 8
561+
for i := e.cur; i < end; i++ {
562+
cv := load6432(prefix, i-e.cur)
563+
h := hashLen(cv, bestLongTableBits, bestLongLen)
564+
e.longTable[h] = prevEntry{offset: i, prev: e.longTable[h].offset}
565+
h0 := hashLen(cv, bestShortTableBits, bestShortLen)
566+
e.table[h0] = prevEntry{offset: i, prev: e.table[h0].offset}
567+
}
568+
}

‎zstd/enc_better.go‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1096,6 +1096,20 @@ func (e *betterFastEncoder) Reset(d *dict, singleBlock bool) {
10961096
}
10971097
}
10981098

1099+
func (e *betterFastEncoder) ResetPrefix(prefix []byte) {
1100+
e.resetBasePrefix(prefix)
1101+
if len(prefix) < 8 {
1102+
return
1103+
}
1104+
end := e.cur + int32(len(prefix)) - 8
1105+
for i := e.cur; i < end; i += 2 {
1106+
cv := load6432(prefix, i-e.cur)
1107+
h := hashLen(cv, betterLongTableBits, betterLongLen)
1108+
e.longTable[h] = prevEntry{offset: i, prev: e.longTable[h].offset}
1109+
e.table[hashLen(cv>>8, betterShortTableBits, betterShortLen)] = tableEntry{val: uint32(cv >> 8), offset: i + 1}
1110+
}
1111+
}
1112+
10991113
// ResetDict will reset and set a dictionary if not nil
11001114
func (e *betterFastEncoderDict) Reset(d *dict, singleBlock bool) {
11011115
e.resetBase(d, singleBlock)
@@ -1229,6 +1243,10 @@ func (e *betterFastEncoderDict) Reset(d *dict, singleBlock bool) {
12291243
e.allDirty = false
12301244
}
12311245

1246+
func (e *betterFastEncoderDict) ResetPrefix([]byte) {
1247+
panic("ResetPrefix not supported for dict encoders")
1248+
}
1249+
12321250
func (e *betterFastEncoderDict) markLongShardDirty(entryNum uint32) {
12331251
e.longTableShardDirty[entryNum/betterLongTableShardSize] = true
12341252
}

‎zstd/enc_dfast.go‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1037,6 +1037,18 @@ func (e *doubleFastEncoder) Reset(d *dict, singleBlock bool) {
10371037
}
10381038
}
10391039

1040+
func (e *doubleFastEncoder) ResetPrefix(prefix []byte) {
1041+
e.fastEncoder.ResetPrefix(prefix)
1042+
if len(prefix) < 8 {
1043+
return
1044+
}
1045+
end := e.cur + int32(len(prefix)) - 8
1046+
for i := e.cur + 1; i < end; i += 2 {
1047+
cv := load6432(prefix, i-e.cur)
1048+
e.longTable[hashLen(cv, dFastLongTableBits, dFastLongLen)] = tableEntry{val: uint32(cv), offset: i}
1049+
}
1050+
}
1051+
10401052
// ResetDict will reset and set a dictionary if not nil
10411053
func (e *doubleFastEncoderDict) Reset(d *dict, singleBlock bool) {
10421054
allDirty := e.allDirty
@@ -1102,6 +1114,10 @@ func (e *doubleFastEncoderDict) Reset(d *dict, singleBlock bool) {
11021114
}
11031115
}
11041116

1117+
func (e *doubleFastEncoderDict) ResetPrefix([]byte) {
1118+
panic("ResetPrefix not supported for dict encoders")
1119+
}
1120+
11051121
func (e *doubleFastEncoderDict) markLongShardDirty(entryNum uint32) {
11061122
e.longTableShardDirty[entryNum/dLongTableShardSize] = true
11071123
}

‎zstd/enc_fast.go‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -797,6 +797,19 @@ func (e *fastEncoder) Reset(d *dict, singleBlock bool) {
797797
}
798798
}
799799

800+
func (e *fastEncoder) ResetPrefix(prefix []byte) {
801+
e.resetBasePrefix(prefix)
802+
if len(prefix) < 8 {
803+
return
804+
}
805+
end := e.cur + int32(len(prefix)) - 8
806+
// Index every 4th
807+
for i := e.cur + 1; i < end; i += 4 {
808+
cv := load6432(prefix, i-e.cur)
809+
e.table[hashLen(cv, tableBits, tableFastHashLen)] = tableEntry{val: uint32(cv), offset: i}
810+
}
811+
}
812+
800813
// ResetDict will reset and set a dictionary if not nil
801814
func (e *fastEncoderDict) Reset(d *dict, singleBlock bool) {
802815
e.resetBase(d, singleBlock)
@@ -866,6 +879,10 @@ func (e *fastEncoderDict) Reset(d *dict, singleBlock bool) {
866879
e.allDirty = false
867880
}
868881

882+
func (e *fastEncoderDict) ResetPrefix([]byte) {
883+
panic("ResetPrefix not supported for dict encoders")
884+
}
885+
869886
func (e *fastEncoderDict) markAllShardsDirty() {
870887
e.allDirty = true
871888
}

0 commit comments

Comments
 (0)