panic: too many compression settings #5260
Description
I encountered this error after upgrading Pebble from v2.0.6 to #crl-release-25.3 and changing the format major from FormatSyntheticPrefixSuffix to FormatValueSeparation. The app crashes several times a day after it occurs. I'm not sure when exactly it happens and what exactly causes it yet - I will do more testing - but wanted to put this here as a reminder.
panic: too many compression settings
panic: Release called on a BufferPool with in-use buffers
goroutine 11177 [running]:
github.com/cockroachdb/pebble/sstable/block.(*BufferPool).Release(0xc0001fd7f0)
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/sstable/block/buffer_pool.go:177 +0xc5
panic({0x10a3a9400?, 0x10a655710?})
/usr/local/Cellar/go/1.24.4/libexec/src/runtime/panic.go:792 +0x132
github.com/cockroachdb/pebble/sstable/block.(*CompressionStats).add(...)
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/sstable/block/compressor.go:139
github.com/cockroachdb/pebble/sstable/block.(*Compressor).UncompressedBlock(...)
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/sstable/block/compressor.go:102
github.com/cockroachdb/pebble/sstable/block.CopyAndChecksum(0xc0014d8c18, {0xc0055d0040?, 0x35, 0xc000e7f448?}, 0x0?, 0xc000fa8928, 0xc000fa8910)
/Users/pacx/go/pkg/mod/github.com/c:349 +0x225
github.com/cockroachdb/pebble/sstable/blob.(*FileWriter).Close(0xc000fa8788)
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/sstable/blob/blob.go:288 +0x1a5
github.com/cockroachdb/pebble.(*writeNewBlobFiles).FinishOutput(0xc001814b60)
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/value_separation.go:303 +0x59
github.com/cockroachdb/pebble/internal/compact.(*Runner).WriteTable(0xc0014d95b0, {0x296c8, 0x2, {0x0, 0x0, {0x0, 0x0}, 0x0, {0x0, 0x0}, ...}}, ...)
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/internal/compact/run.go:232 +0x2cd
github.com/cockroachdb/pebble.(*DB).compactAndWrite(0xc000258008, 0x117, 0xc0001fd608, {0x0, 0x0, 0x0}, 0x9, {0x10a66acf8, 0xc001814b60})
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/compaction.go:3407 +0x988
github.com/cockroachdb/pebble.(*DB).runCompaction(0xc000258008, 0x117, 0xc0001fd608)
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/compaction.go:3232 +0x44c
github.com/cockroachdb/pebble.(*DB).compact1(0xc000258008, 0x117, 0xc0001fd608)
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/compaction.go:2686 +0x105
github.com/cockroachdb/pebble.(*tableCompaction).Execute(0xc0001fd608, 0x117, 0xc000258008)
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/compaction.go:380 +0x45
github.com/cockroachdb/pebble.(*DB).compact.func1.1(0xc000258008, {0x10a678600, 0xc0001fd608}, 0x0)
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/compaction.go:2560 +0xd0
github.com/cockroachdb/pebble.(*DB).compact.func1({0x10a66a908?, 0xc00335bd70?})
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/compaction.go:2584 +0x32
runtime/pprof.Do({0x10a66a760?, 0x10b2dbe80?}, {{0xc001ddb280?, 0x0?, 0x0?}}, 0xc0035dff78)
/usr/local/Cellar/go/1.24.4/libexec/src/runtime/pprof/runtime.go:51 +0x8c
github.com/cockroachdb/pebble.(*DB).compact(0xc000258008, {0x10a678600, 0xc0001fd608}, 0x0)
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/compaction.go:2554 +0x10c
github.com/cockroachdb/pebble.(*DB).runPickedCompaction.func1()
/Users/pacx/go/pkg/mod/github.com/:2083 +0x25
created by github.com/cockroachdb/pebble.(*DB).runPickedCompaction in goroutine 11020
/Users/pacx/go/pkg/mod/github.com/cockroachdb/[email protected]/compaction.go:2082 +0x132
exit status 2
Here are the options I’m using to open the DB:
func GetDBOptions() *pebble.Options {
opts := &pebble.Options{
FormatMajorVersion: pebble.FormatValueSeparation,
LBaseMaxBytes: 64 << 20, // 64 MB
MemTableSize: 64 << 20, // 64 MB
MemTableStopWritesThreshold: 4,
}
opts.Experimental.ValueSeparationPolicy = func() pebble.ValueSeparationPolicy {
return pebble.ValueSeparationPolicy{
Enabled: true,
MinimumSize: 1 << 10, // 1kb
TargetGarbageRatio: 100,
RewriteMinimumAge: 5 * time.Minute,
MaxBlobReferenceDepth: 10,
}
}
opts.ApplyCompressionSettings(func() pebble.DBCompressionSettings {
return pebble.DBCompressionGood
})
return opts
}Jira issue: PEBBLE-1159