utils: sync, Make zlib compression pluggable via x/plugin#2012
Conversation
There was a problem hiding this comment.
Pull request overview
This PR makes go-git’s zlib compression/decompression implementation pluggable by introducing ZlibReader, ZlibWriter, and ZlibProvider in utils/sync, with compress/zlib remaining the default provider. It updates internal pooling/factory usage and adjusts call sites to depend on the new interfaces instead of stdlib concrete types.
Changes:
- Add
ZlibProvider+SetZlibProviderand route pooled reader/writer construction through the active provider. - Introduce
NewZlibWriterfor non-pooled, long-lived writer usage (e.g., packfile encoder). - Update packfile/objfile writers and add tests + documentation for the new extension point.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| utils/sync/zlib.go | Introduces provider + interfaces and switches pools/factories to use the provider. |
| utils/sync/zlib_test.go | Adds tests validating provider swapping and pooled/non-pooled paths. |
| plumbing/format/packfile/encoder.go | Switches encoder’s long-lived zlib writer to sync.NewZlibWriter. |
| plumbing/format/objfile/writer.go | Switches pooled objfile writer to sync.ZlibWriter + pool put-back. |
| EXTENDING.md | Documents how to register an alternate zlib provider and notes the breaking change. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
380dec7 to
d4d5371
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
utils/sync/zlib.go:220
- PutZlibWriter now accepts an interface, so callers can accidentally pass a typed-nil (e.g., (*zlib.Writer)(nil)). The
w == nilcheck won't catch that, and a typed-nil would be stored in the pool, causing a panic on the next Get/Reset. Consider adding a typed-nil guard (e.g., via reflect.ValueOf(w).IsNil when applicable) or otherwise ensuring nil concrete values can’t enter the pool.
func PutZlibWriter(w ZlibWriter) {
if w == nil {
return
}
zlibWriter.Put(w)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
fac0eac to
5d64a44
Compare
Introduce ZlibReader, ZlibWriter, and ZlibProvider interfaces in utils/sync together with SetZlibProvider for registering a non-default implementation. The stdlib compress/zlib is wired as the default via an internal wrapper, so existing behavior is unchanged. Users who want faster decompression can register an alternate provider (for example github.com/klauspost/compress/zlib) without go-git taking a direct dependency on it. The existing sync.Pool lifecycle for readers and writers is preserved; the pool's New funcs now construct instances via the active provider instead of calling compress/zlib directly. A new NewZlibWriter factory hands out non-pooled writers for long-lived callers (packfile.Encoder holds one for the lifetime of an encode and resets it per entry). Breaking: GetZlibWriter now returns sync.ZlibWriter (an interface) instead of *zlib.Writer, and PutZlibWriter accepts the interface. The method set is Write, Close, Reset(io.Writer), and Flush. After this change, compress/zlib is imported in exactly one file in the repository (utils/sync/zlib.go). Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
- SetZlibProvider now panics on a nil provider rather than storing it and deferring the failure to the first pool Get (a confusing nil dereference deep in call sites). - The zlib reader pool's New func panics with context if the provider returns an error constructing the seed reader. A conforming provider must accept the minimal valid stream used to seed the pool, so an error here is a programmer error that should fail loudly. - ZlibReader and ZlibWriter godoc now spell out the behavioral contract go-git relies on: Close must not close the underlying writer, Reset after Close is required (packfile.Encoder reuses the writer across entries), and Reset on a pool-rented reader reseeds it for the next rental. - The EXTENDING.md klauspost example snippet now imports "io" so the copy/paste compiles as written. - Added a test covering the nil-provider panic. Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
- Export the stdlib provider as StdlibZlibProvider (previously the unexported stdlibZlibProvider). The zlib_test helper stdlibWrapper was a verbatim re-implementation of the unexported type; using the exported default deletes the duplication and gives downstream users a supported delegation target. - Handle the zlib.Resetter type assertion in the stdlib provider with comma-ok and a descriptive error instead of an implicit panic. The klauspost snippet in EXTENDING.md is updated to the same safer pattern. - Prune the method-level godoc on ZlibReader.Reset and ZlibWriter's Reset/Flush that restated the method signatures; fold the contract into the type-level comment block. Add a note that Reset runs on the per-object hot path during packfile scan/encode and should be O(1) so custom providers do not regress throughput. - Move the v6 breaking-change note above the example in EXTENDING.md so readers upgrading see it before the tutorial code. - Add paralleltest-compliant annotations to the zlib tests: t.Parallel() on the two that do not touch global state; //nolint:paralleltest // modifies global zlib provider state on the three that swap the provider (matching the convention used in x/plugin/plugin_test.go). Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
TestDefaultProviderRoundTripWithStdlib was marked t.Parallel() on the reasoning that every tracking provider in the suite wraps stdlib, so the round-trip is stdlib-compatible regardless of what a concurrent test has installed. That is correctness-by-coincidence: the test name promises a default-provider check, but under parallelism with another test swapping the provider the assertion is really about whichever provider happens to be active. Swap to an explicit save-and-restore of StdlibZlibProvider and drop t.Parallel() (with the existing //nolint:paralleltest rationale) so the test always exercises the intended path and does not become flaky if a future test installs a non-stdlib-compatible provider. Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
Addresses PR feedback from @pjbgf to align zlib pluggability with the x/plugin registration pattern already used by ObjectSigner and ConfigLoader. Replaces the bespoke SetZlibProvider / atomic.Pointer mechanism in utils/sync with plugin.Register(plugin.Zlib(), factory). The Reader, Writer, and Provider interfaces live in x/plugin/zlib (the leaf package) and are aliased from x/plugin, which avoids an import cycle and lets *xzlib.Stdlib satisfy plugin.ZlibProvider directly without an adapter. utils/sync resolves the provider lazily via sync.OnceValue on the first pool miss. Removes SetZlibProvider, StdlibZlibProvider, and activeZlibProvider from utils/sync. utils/sync.ZlibReader and ZlibWriter remain as type aliases so callers in packfile/encoder.go and objfile/writer.go need no changes. Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
golangci-lint v2.11.4 (CI-pinned) flags the //nolint:revive directive on Zlib() as unused, failing validate-lint. Unlike ConfigLoader() and ObjectSigner() where revive still fires on the unexported key return, ZlibProvider is a type alias so revive resolves through it and never complains — leaving the directive unused. Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
Name the concrete interface in the type-assertion failure message (plugin.ZlibReader) rather than the stdlib Resetter shorthand, since that is what the assertion actually checks against. Also handle plugin.Register's error by panicking in init — silent _ = could hide a frozen registry or nil factory misuse, which is hard to debug post-hoc. Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
Signed-off-by: Paulo Gomes <[email protected]>
Signed-off-by: Paulo Gomes <[email protected]>
5d64a44 to
d742bc3
Compare
| ```go | ||
| import ( |
There was a problem hiding this comment.
As a follow-up, we could add this implementation to https://github.com/go-git/x.
pjbgf
left a comment
There was a problem hiding this comment.
@stiak thanks for working on this! 🙇
I've rebased and made a few additional changes to your original PR:
- Added a new validation primitive so that plugins can be rejected at registration time. This further decreases the chances of an invalid zlib plugin causing a panic at runtime.
- In the unlike case of the zlib not being registered, falls back to default.
- A few nits around documentation.
|
Brilliant, thanks @pjbgf ! |
Adds a pluggable zlib compression path to go-git by wiring it into the existing
x/pluginregistry. The stdlibcompress/zlibis registered as the default provider duringx/plugin's init, so existing behaviour is unchanged — but applications can now register an alternative (for examplegithub.com/klauspost/compress/zlib) without go-git taking a direct dependency on it.See EXTENDING.md for the full example.
Registration follows the same freeze-on-first-use lifecycle as other plugins: once go-git has resolved the provider (on the first pool miss or NewZlibWriter call) subsequent plugin.Register calls return plugin.ErrFrozen and the active provider is kept.