Skip to content

Panic in grpccompression/snappy #15237

Description

@droidnxs

Component(s)

No response

What happened?

Describe the bug
After updating to release 0.151.0, we started seeing panics in our cluster from the snappy grpccompressor. This is likely introduced by this commit https://github.com/open-telemetry/opentelemetry-collector/pull/15168/changes#diff-b0b7ab96aa0e37e4e25fbdfbf838004a08f20163279a3e660fa555517824a9d4

The snappy compressor stores sync.Once on a pooled object and returns the object to the pool from inside once.Do, allowing reuse to reset the same Once before the first caller finished unwinding.

What did you expect to see?
No panics.

What did you see instead?

fatal error: sync: unlock of unlocked mutex

goroutine 356 [running]:
internal/sync.fatal(...)
    runtime/panic.go:1056 +0x18
internal/sync.(*Mutex).unlockSlow(0xc0011d4154, 0xffffffff)
    internal/sync/mutex.go:204 +0x35
internal/sync.(*Mutex).Unlock(...)
    internal/sync/mutex.go:198
sync.(*Mutex).Unlock(...)
    sync/mutex.go:65
sync.(*Once).doSlow(0x0?, 0x0?)
    sync/once.go:80 +0xe4
sync.(*Once).Do(...)
    sync/once.go:69
go.opentelemetry.io/collector/internal/grpccompression/snappy.(*writer).Close(0xc0011d4140)
    go.opentelemetry.io/collector@v0.151.0/internal/grpccompression/snappy/snappy.go:81 +0x9c
google.golang.org/grpc.compress({0xc0007160c0, 0x1, 0x1}, {0x0, 0x0}, ...)
    google.golang.org/grpc@v1.80.0/rpc_util.go:840 +0x543
google.golang.org/grpc.prepareMsg(...)
    google.golang.org/grpc@v1.80.0/stream.go:1896 +0x132

Collector version

0.151.0

Environment information

Environment

Kubernetes with Docker/amd64 image
Compiler(if manually compiled): 1.24.8

OpenTelemetry Collector configuration

Log output

Additional context

No response

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingneeds triageNew item requiring triage

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions