fix(seaweedfs): stop the master crashing on the crowded volume map - #5308

What this PR does

Backports seaweedfs/seaweedfs#11216 ("topology: fix fatal concurrent map read/write on VolumeLayout.crowded", released in 4.46) as a patch on the in-tree 4.45 image, next to reader-cache-prefetch-window.diff.

In 4.45, (*VolumeLayout).SetVolumeCrowded mutates the crowded map under accessLock.RLock(), while (*VolumeLayout).GetWritableVolumeCount reads the same map under RLock() on the Assign path. Two read-lock holders, one writing and one reading the map, make the Go runtime abort the master with fatal error: concurrent map read and map write. This cannot be recovered, so the process exits and S3 requests that need an assign fail until a master is back. The fix takes the write lock in SetVolumeCrowded; every other mutation of crowded already holds it.

patches/volume-layout-crowded-lock.diff is the upstream squash commit 5d5ea18287cfbf9e274f4b78be40a0ae220cb55d (files under weed/topology/), regenerated against the 4.45 source so it applies without offset. Its content is unchanged, including the regression test TestSetVolumeCrowdedNoRaceWithGetWritableVolumeCount, which the Dockerfile's existing go test step runs on every build.

Seen in production

On a production platform running Cozystack v1.7.0 (in-tree SeaweedFS 4.45), with a dedicated S3 cluster used as the Velero backup target:

• Since the upgrade to 4.45 on 2026-11-05, all 3 masters crash every night with exit code 2 under the backup load, between 01:00 and 03:00 UTC: 2 crashes on 2026-11-06, about 23 on 2026-11-13, 33 to 43 restarts per master over 8 days.
• The stack is always the same:

  fatal error: concurrent map read and map write
  ...
  github.com/seaweedfs/seaweedfs/weed/topology.(*VolumeLayout).GetWritableVolumeCount(...)
      weed/topology/volume_layout.go:834
  github.com/seaweedfs/seaweedfs/weed/server.(*MasterServer).Assign(...)
      weed/server/master_grpc_server_assign.go:97
  

• Clients see HTTP 500 InternalError from the S3 gateway, and the nightly backups end Failed or PartiallyFailed (4 Failed and 3 PartiallyFailed on the night of 2026-11-12 to 13).

Upstream issue: seaweedfs/seaweedfs#11211.

Testing

• git apply --check of both patches against the 4.45 source archive (SOURCE_SHA256 in the Dockerfile matches).
• Regression test on 4.45 with only the test applied, CGO_ENABLED=0 as in the image build: fatal error: concurrent map writes, FAIL. With the fix: go test ./weed/topology/ passes, and the new test passes under -race too.
• docker buildx build images/seaweedfs --platform linux/amd64 builds, running the patched packages' tests.
• make test in packages/system/seaweedfs: 107 helm unit tests, the naming-guard check and the image/chart version check pass.

Lifecycle

The patch is dropped once the vendored chart reaches 4.46 or later, for example with #4879 (4.48 bump). From that release on it no longer applies, and make update-image fails until it is removed, as the Dockerfile header says. This PR is kept small so that it does not wait for #4879.

Backport

This is a fix with no CRD, migration or API change, and release-1.7 carries the same images/seaweedfs/ as main, so the backport batch should take it onto release-1.7 with a clean cherry-pick for the next v1.7.x. Clusters on v1.7 are the ones that crash today.

Downstream repositories

• No downstream repository is affected by this change
• cozystack/website - follow-up:
• cozystack/terraform-provider-cozystack - follow-up:
• cozystack/ansible-cozystack - follow-up:
• cozystack/ccp - follow-up:
• cozystack/talm - follow-up:
• cozystack/cozyhr - follow-up:
• cozystack/cozy-proxy - follow-up:
• cozystack/cozystack-telemetry-server - follow-up:
• cozystack/external-apps-example - follow-up:
• cozystack/examples - follow-up:
• cozystack/community - follow-up:

The diff touches only packages/system/seaweedfs/images/seaweedfs/ (a Go patch and a Dockerfile comment): no package added or renamed, no values, schema, CRD, hack/ or installer change, so no entry of the trigger map applies.

Release note

fix(seaweedfs): stop SeaweedFS masters crashing with "concurrent map read and map write" under write load, which made S3 answer 500 (backport of seaweedfs/seaweedfs#11216 onto 4.45).

Summary by CodeRabbit

• Bug Fixes
  • Improved reliability when updating crowded-volume state concurrently with writable-volume queries.
• Documentation
  • Added notes about the upstream fix and when the local patch can be removed.