storage: dotgit, fix hasIncomingObjects race#2131
Merged
Merged
Conversation
DotGit.hasIncomingObjects lazily caches the result of an incoming-dir scan in two struct fields (incomingChecked and incomingDirName) without any synchronisation. Two goroutines calling Object on the same *DotGit race on those fields; the race detector reliably flags it under -race, with both a read/write race on incomingChecked and a write/write race on the cached name. The bug was introduced in c7a4011 ("storage/dotgit: search for incoming dir only once", 2018-08-25). *DotGit was effectively single-threaded at the time, so the race stayed latent for seven years. It surfaces under modern parallel-reader patterns that funnel through getFromUnpacked -> dir.Object -> hasIncomingObjects on cold startup. Replace the two cache fields with a sync.Once plus the cached name. sync.Once.Do provides the happens-before guarantee the lazy init needs, so readers stay lock-free after the first call. The loop body is preserved verbatim to keep the existing selection behaviour (last lexical match when multiple incoming directories coexist, which can happen under concurrent git-receive-pack invocations). Test: TestDotGit_HasIncomingObjects_NoRace launches 32 goroutines that all call Object against a fresh DotGit. Clean under "go test ./storage/filesystem/dotgit/ -race -count=10". Verified before this change that the same test fails with the race detector on the buggy code. Assisted-by: Claude Opus 4.7 Signed-off-by: Hidde Beydals <[email protected]>
bdd9daf to
8ab1a10
Compare
pjbgf
reviewed
May 17, 2026
| // race on the cached fields. | ||
| func (d *DotGit) hasIncomingObjects() bool { | ||
| if !d.incomingChecked { | ||
| d.incomingOnce.Do(func() { |
Member
There was a problem hiding this comment.
The existing code seems to imply that incoming dirs would happen at most once within the lifetime of the dotgit instance, in which case the use of sync.Once here is appropriate.
My take is that that is not the case, instead incomingDirName should be an array instead, and to ensure we process each dir at most once, we could use golang.org/x/sync/singleflight instead.
Member
There was a problem hiding this comment.
The existing assumptions would be largely correct for CLI's, but maybe would not be the case for long running processes. I'll go ahead and merge this as fix the current race condition. But the long term of this feature may need to change as a follow-up.
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.