Skip to content

storage: dotgit, fix hasIncomingObjects race#2131

Merged
pjbgf merged 1 commit into
go-git:mainfrom
hiddeco:dotgit-incoming-race
May 17, 2026
Merged

storage: dotgit, fix hasIncomingObjects race#2131
pjbgf merged 1 commit into
go-git:mainfrom
hiddeco:dotgit-incoming-race

Conversation

@hiddeco

@hiddeco hiddeco commented May 16, 2026

Copy link
Copy Markdown
Member

No description provided.

DotGit.hasIncomingObjects lazily caches the result of an
incoming-dir scan in two struct fields (incomingChecked and
incomingDirName) without any synchronisation. Two goroutines
calling Object on the same *DotGit race on those fields; the
race detector reliably flags it under -race, with both a
read/write race on incomingChecked and a write/write race on
the cached name.

The bug was introduced in c7a4011 ("storage/dotgit: search
for incoming dir only once", 2018-08-25). *DotGit was
effectively single-threaded at the time, so the race stayed
latent for seven years. It surfaces under modern
parallel-reader patterns that funnel through getFromUnpacked
-> dir.Object -> hasIncomingObjects on cold startup.

Replace the two cache fields with a sync.Once plus the
cached name. sync.Once.Do provides the happens-before
guarantee the lazy init needs, so readers stay lock-free
after the first call. The loop body is preserved verbatim to
keep the existing selection behaviour (last lexical match
when multiple incoming directories coexist, which can happen
under concurrent git-receive-pack invocations).

Test: TestDotGit_HasIncomingObjects_NoRace launches 32
goroutines that all call Object against a fresh DotGit.
Clean under "go test ./storage/filesystem/dotgit/ -race
-count=10". Verified before this change that the same test
fails with the race detector on the buggy code.

Assisted-by: Claude Opus 4.7
Signed-off-by: Hidde Beydals <[email protected]>
@hiddeco hiddeco force-pushed the dotgit-incoming-race branch from bdd9daf to 8ab1a10 Compare May 16, 2026 14:00
// race on the cached fields.
func (d *DotGit) hasIncomingObjects() bool {
if !d.incomingChecked {
d.incomingOnce.Do(func() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing code seems to imply that incoming dirs would happen at most once within the lifetime of the dotgit instance, in which case the use of sync.Once here is appropriate.

My take is that that is not the case, instead incomingDirName should be an array instead, and to ensure we process each dir at most once, we could use golang.org/x/sync/singleflight instead.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing assumptions would be largely correct for CLI's, but maybe would not be the case for long running processes. I'll go ahead and merge this as fix the current race condition. But the long term of this feature may need to change as a follow-up.

@pjbgf pjbgf left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hiddeco Thanks for working on this. 🙇

@pjbgf pjbgf merged commit 7069092 into go-git:main May 17, 2026
17 checks passed
@AriehSchneier

Copy link
Copy Markdown
Contributor

@pjbgf That was part of this PR, which also has some other race fixes:
#2094

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants