Skip to content

utils: sync, Make zlib compression pluggable via x/plugin#2012

Merged
pjbgf merged 9 commits into
go-git:mainfrom
stiak:add-pluggable-compression-library
Apr 25, 2026
Merged

utils: sync, Make zlib compression pluggable via x/plugin#2012
pjbgf merged 9 commits into
go-git:mainfrom
stiak:add-pluggable-compression-library

Conversation

@stiak
Copy link
Copy Markdown
Contributor

@stiak stiak commented Apr 22, 2026

Adds a pluggable zlib compression path to go-git by wiring it into the existing x/plugin registry. The stdlib compress/zlib is registered as the default provider during x/plugin's init, so existing behaviour is unchanged — but applications can now register an alternative (for example github.com/klauspost/compress/zlib) without go-git taking a direct dependency on it.

See EXTENDING.md for the full example.

Registration follows the same freeze-on-first-use lifecycle as other plugins: once go-git has resolved the provider (on the first pool miss or NewZlibWriter call) subsequent plugin.Register calls return plugin.ErrFrozen and the active provider is kept.

Copilot AI review requested due to automatic review settings April 22, 2026 00:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes go-git’s zlib compression/decompression implementation pluggable by introducing ZlibReader, ZlibWriter, and ZlibProvider in utils/sync, with compress/zlib remaining the default provider. It updates internal pooling/factory usage and adjusts call sites to depend on the new interfaces instead of stdlib concrete types.

Changes:

  • Add ZlibProvider + SetZlibProvider and route pooled reader/writer construction through the active provider.
  • Introduce NewZlibWriter for non-pooled, long-lived writer usage (e.g., packfile encoder).
  • Update packfile/objfile writers and add tests + documentation for the new extension point.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
utils/sync/zlib.go Introduces provider + interfaces and switches pools/factories to use the provider.
utils/sync/zlib_test.go Adds tests validating provider swapping and pooled/non-pooled paths.
plumbing/format/packfile/encoder.go Switches encoder’s long-lived zlib writer to sync.NewZlibWriter.
plumbing/format/objfile/writer.go Switches pooled objfile writer to sync.ZlibWriter + pool put-back.
EXTENDING.md Documents how to register an alternate zlib provider and notes the breaking change.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread utils/sync/zlib.go Outdated
Comment thread utils/sync/zlib.go Outdated
Comment thread utils/sync/zlib.go Outdated
Comment thread utils/sync/zlib.go Outdated
Comment thread EXTENDING.md
Comment thread utils/sync/zlib_test.go
@stiak stiak force-pushed the add-pluggable-compression-library branch from 380dec7 to d4d5371 Compare April 22, 2026 00:17
@stiak stiak requested a review from Copilot April 22, 2026 01:26
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

utils/sync/zlib.go:220

  • PutZlibWriter now accepts an interface, so callers can accidentally pass a typed-nil (e.g., (*zlib.Writer)(nil)). The w == nil check won't catch that, and a typed-nil would be stored in the pool, causing a panic on the next Get/Reset. Consider adding a typed-nil guard (e.g., via reflect.ValueOf(w).IsNil when applicable) or otherwise ensuring nil concrete values can’t enter the pool.
func PutZlibWriter(w ZlibWriter) {
	if w == nil {
		return
	}
	zlibWriter.Put(w)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread utils/sync/zlib_test.go Outdated
Copy link
Copy Markdown
Member

@pjbgf pjbgf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stiak thanks for proposing this: I 100% agree this will be a nice addition to the project.

Let's just align the implementation of the factor/provider registration with the x/plugin.

Comment thread utils/sync/zlib.go Outdated
Comment thread utils/sync/zlib.go Outdated
@stiak stiak force-pushed the add-pluggable-compression-library branch from fac0eac to 5d64a44 Compare April 23, 2026 01:16
@stiak stiak changed the title utils: sync, Make zlib compression pluggable via ZlibProvider utils: sync, Make zlib compression pluggable via x/plugins Apr 23, 2026
@stiak stiak changed the title utils: sync, Make zlib compression pluggable via x/plugins utils: sync, Make zlib compression pluggable via x/plugin Apr 23, 2026
@pjbgf pjbgf added the breaking label Apr 23, 2026
stiak and others added 9 commits April 25, 2026 21:16
Introduce ZlibReader, ZlibWriter, and ZlibProvider interfaces in
utils/sync together with SetZlibProvider for registering a non-default
implementation. The stdlib compress/zlib is wired as the default via
an internal wrapper, so existing behavior is unchanged. Users who want
faster decompression can register an alternate provider (for example
github.com/klauspost/compress/zlib) without go-git taking a direct
dependency on it.

The existing sync.Pool lifecycle for readers and writers is preserved;
the pool's New funcs now construct instances via the active provider
instead of calling compress/zlib directly. A new NewZlibWriter factory
hands out non-pooled writers for long-lived callers (packfile.Encoder
holds one for the lifetime of an encode and resets it per entry).

Breaking: GetZlibWriter now returns sync.ZlibWriter (an interface)
instead of *zlib.Writer, and PutZlibWriter accepts the interface. The
method set is Write, Close, Reset(io.Writer), and Flush.

After this change, compress/zlib is imported in exactly one file in
the repository (utils/sync/zlib.go).

Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
- SetZlibProvider now panics on a nil provider rather than storing it
  and deferring the failure to the first pool Get (a confusing nil
  dereference deep in call sites).

- The zlib reader pool's New func panics with context if the provider
  returns an error constructing the seed reader. A conforming provider
  must accept the minimal valid stream used to seed the pool, so an
  error here is a programmer error that should fail loudly.

- ZlibReader and ZlibWriter godoc now spell out the behavioral
  contract go-git relies on: Close must not close the underlying
  writer, Reset after Close is required (packfile.Encoder reuses the
  writer across entries), and Reset on a pool-rented reader reseeds
  it for the next rental.

- The EXTENDING.md klauspost example snippet now imports "io" so the
  copy/paste compiles as written.

- Added a test covering the nil-provider panic.

Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
- Export the stdlib provider as StdlibZlibProvider (previously the
  unexported stdlibZlibProvider). The zlib_test helper stdlibWrapper
  was a verbatim re-implementation of the unexported type; using the
  exported default deletes the duplication and gives downstream users
  a supported delegation target.

- Handle the zlib.Resetter type assertion in the stdlib provider with
  comma-ok and a descriptive error instead of an implicit panic. The
  klauspost snippet in EXTENDING.md is updated to the same safer
  pattern.

- Prune the method-level godoc on ZlibReader.Reset and ZlibWriter's
  Reset/Flush that restated the method signatures; fold the contract
  into the type-level comment block. Add a note that Reset runs on
  the per-object hot path during packfile scan/encode and should be
  O(1) so custom providers do not regress throughput.

- Move the v6 breaking-change note above the example in EXTENDING.md
  so readers upgrading see it before the tutorial code.

- Add paralleltest-compliant annotations to the zlib tests:
  t.Parallel() on the two that do not touch global state;
  //nolint:paralleltest // modifies global zlib provider state on
  the three that swap the provider (matching the convention used in
  x/plugin/plugin_test.go).

Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
TestDefaultProviderRoundTripWithStdlib was marked t.Parallel() on the
reasoning that every tracking provider in the suite wraps stdlib, so
the round-trip is stdlib-compatible regardless of what a concurrent
test has installed. That is correctness-by-coincidence: the test name
promises a default-provider check, but under parallelism with another
test swapping the provider the assertion is really about whichever
provider happens to be active.

Swap to an explicit save-and-restore of StdlibZlibProvider and drop
t.Parallel() (with the existing //nolint:paralleltest rationale) so
the test always exercises the intended path and does not become
flaky if a future test installs a non-stdlib-compatible provider.

Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
Addresses PR feedback from @pjbgf to align zlib pluggability with
the x/plugin registration pattern already used by ObjectSigner and
ConfigLoader. Replaces the bespoke SetZlibProvider / atomic.Pointer
mechanism in utils/sync with plugin.Register(plugin.Zlib(), factory).

The Reader, Writer, and Provider interfaces live in x/plugin/zlib
(the leaf package) and are aliased from x/plugin, which avoids an
import cycle and lets *xzlib.Stdlib satisfy plugin.ZlibProvider
directly without an adapter. utils/sync resolves the provider lazily
via sync.OnceValue on the first pool miss.

Removes SetZlibProvider, StdlibZlibProvider, and activeZlibProvider
from utils/sync. utils/sync.ZlibReader and ZlibWriter remain as type
aliases so callers in packfile/encoder.go and objfile/writer.go need
no changes.

Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
golangci-lint v2.11.4 (CI-pinned) flags the //nolint:revive directive
on Zlib() as unused, failing validate-lint. Unlike ConfigLoader() and
ObjectSigner() where revive still fires on the unexported key return,
ZlibProvider is a type alias so revive resolves through it and never
complains — leaving the directive unused.

Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
Name the concrete interface in the type-assertion failure message
(plugin.ZlibReader) rather than the stdlib Resetter shorthand, since
that is what the assertion actually checks against. Also handle
plugin.Register's error by panicking in init — silent _ = could hide
a frozen registry or nil factory misuse, which is hard to debug
post-hoc.

Assisted-by: Claude Opus 4.7 (1M context) <[email protected]>
@pjbgf pjbgf force-pushed the add-pluggable-compression-library branch from 5d64a44 to d742bc3 Compare April 25, 2026 20:17
Comment thread EXTENDING.md
Comment on lines +86 to +87
```go
import (
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

@pjbgf pjbgf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stiak thanks for working on this! 🙇

I've rebased and made a few additional changes to your original PR:

  • Added a new validation primitive so that plugins can be rejected at registration time. This further decreases the chances of an invalid zlib plugin causing a panic at runtime.
  • In the unlike case of the zlib not being registered, falls back to default.
  • A few nits around documentation.

@pjbgf pjbgf merged commit c50c742 into go-git:main Apr 25, 2026
23 of 24 checks passed
@stiak
Copy link
Copy Markdown
Contributor Author

stiak commented Apr 27, 2026

Brilliant, thanks @pjbgf !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants