Skip to content

DEP-268: doc: switch local kind guide to RustFS - #39125

Merged
jubrad merged 2 commits into
MaterializeInc:mainfrom
jubrad:justin/dep-268-self-managed-local-kind-guide-hits-minio-issue
Sep 24, 2026
Merged

jubrad merged 2 commits into
MaterializeInc:mainfrom
jubrad:justin/dep-268-self-managed-local-kind-guide-hits-minio-issue

Conversation

@jubrad

@jubrad jubrad commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

https://linear.app/materializeinc/issue/DEP-268/self-managed-local-kind-guide-hits-minio-issue

Issue: The Self-Managed "Install locally on kind" guide curls misc/helm-charts/testing/minio.yaml from the release tag. Every published copy references the minio/minio Docker Hub image, which has been deleted, so the guide fails.

Solution:

  • Add misc/helm-charts/testing/rustfs.yaml: a RustFS Deployment and Service, plus a Job that creates the bucket. It mirrors the kind test backends in materialize-terraform-self-managed.
  • The guide fetches rustfs.yaml from main, since no release tag contains it. It also tells users to set persist_backend_url to RustFS, because released materialize.yaml files still point at MinIO.
  • workflow_documentation_defaults in test/orchestratord/mzcompose.py now follows the updated guide. It uses RustFS and overrides the persist URL for every version, so the MinIO image rewrite hack is removed.

Testing: workflow_documentation_defaults exercises the new guide flow end to end in CI. I also followed the updated guide manually on a local kind cluster against v26.42.0, using the branch copy of rustfs.yaml. Materialize came up, a materialized view returned correct results, and persist data landed in the RustFS bucket. The sample kubectl get all output in the guide is taken from that run.

Gotchas:

  • The guide's refs/heads/main/.../rustfs.yaml URL returns 404 until this merges, so the docs must not be published before then.
  • testing/materialize.yaml deliberately stays on MinIO. The other orchestratord workflows build their secret from it and still deploy minio.yaml.

Tips for reviewer

There are two commits here, one to add rustfs and another to update the example kubectl output which now reflects rustfs and multiple balancerd pods.

🤖 Generated with Claude Code

jubrad and others added 2 commits September 24, 2026 13:05
The "Install locally on kind" guide downloads minio.yaml from release
tags, and every published copy references the `minio/minio` Docker Hub
image, which has been deleted. Add a RustFS manifest to
misc/helm-charts/testing, fetch it from main in the guide, and have
users point `persist_backend_url` at RustFS. The orchestratord
documentation workflow follows the guide accordingly.

Closes: DEP-268

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Captured from a run of the guide against v26.42.0. The operator now runs
two replicas behind a Service, and balancerd runs two replicas.

DEP-268

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@jubrad
jubrad marked this pull request as ready for review September 24, 2026 20:14
@jubrad
jubrad requested a review from a team as a code owner September 24, 2026 20:14
@jubrad
jubrad requested a review from Alphadelta14 September 24, 2026 20:15
@jubrad
jubrad merged commit d37cfaa into MaterializeInc:main Sep 24, 2026
14 checks passed
@jubrad
jubrad deleted the justin/dep-268-self-managed-local-kind-guide-hits-minio-issue branch September 24, 2026 20:18
@bosconi bosconi mentioned this pull request Sep 25, 2026
@ggevay

ggevay commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

@jubrad, wait, shouldn't we have merged #39115 before this PR? If a user follows the current guide, and then loads enough data to have a part >8 MB, then we crashloop. (My Claude reproed this, following the currently live guide.) Maybe we should instead point to our own MinIO build (materialize/minio:v26.42.0) until a version that has #39115 is released?

ggevay added a commit that referenced this pull request Sep 25, 2026
Follow-up to #39125
([DEP-268](https://linear.app/materializeinc/issue/DEP-268)).

**Issue:**
- Released versions of persist cannot read multipart blobs from RustFS,
because RustFS omits `x-amz-mp-parts-count`. #39115 fixes this, but no
release has it yet. Following the current local kind guide on v26.42.0,
any batch part over 8 MiB crashes environmentd or crash-loops a cluster
with `Invalid Parquet file. Corrupt footer`.
- `misc/helm-charts/testing/minio.yaml` pulls `quay.io/minio/minio`,
which no longer allows anonymous pulls, so the orchestratord nightly
workflows cannot start MinIO.

**Solution:**
- `testing/minio.yaml` runs `materialize/minio:v26.42.0`, our own MinIO
build, which is published under every release tag. The image has no
`mc`, so a Job creates the buckets instead of the postStart hook. The
service name, credentials, and buckets are unchanged, so the
`persist_backend_url` in released `materialize.yaml` files keeps
working.
- The guide fetches `minio.yaml` from `main` and no longer asks users to
edit `persist_backend_url`.
- `workflow_documentation_defaults` follows the guide again.

**Gotchas:**
- `rustfs.yaml` stays in `misc/helm-charts/testing` for the RustFS move
once a release contains #39115.
- The orchestratord workflows run only in Nightly, so they need a
Nightly trigger on this PR.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
@bosconi bosconi mentioned this pull request Oct 2, 2026
maheshwarip added a commit that referenced this pull request Oct 2, 2026
The v26.44.0 release-note train, one commit per snapshot. Dates in RC
snapshots are provisional until the release ships; the final snapshot
sets the real dates.

**rc.1** adds the `## v26.44.0` section — 6 Improvements, 3 Agent
Skills, 4 Bug Fixes — and the `v26.44` row in the self-managed operator
compatibility table. No Features section: nothing in this snapshot's 144
in-scope PRs rises to one. The Agent Skills section is back after five
empty windows — the window `(2026-09-23T16:49:13Z,
2026-09-25T02:18:32Z]` holds 4 PRs and all 4 are published.

**final** sets the real dates and adds no new bullets. Nothing shipped
on the materialize side between `v26.44.0-rc.1` and the `v26.44.0` tag:
`GET /compare/v26.44.0-rc.1...v26.44.0` is `ahead_by: 1`, and that one
commit is the version bump, so every `rc.1` classification and all four
verified flag defaults still hold at the ship tag. This train has only
one RC (no `v26.44.0-rc.2` tag exists). Its two changes are:

- **Real dates**: Cloud **2026-09-30**, Self-Managed **2026-10-01**,
from the `v26.44.0` tag's commit date (`2026-09-30T10:50:59Z`) and `+1
day` — one day earlier than the provisional pair rc.1 carried, exactly
as rc.1 predicted. Both date lines in `_index.md` and the `Release
date:` in the YAML are updated.
- **One cross-snapshot rewrite, not a new bullet**:
[agent-skills#74](MaterializeInc/agent-skills#74)
finishes the `mz-` rename that
[#68](MaterializeInc/agent-skills#68) left
`materialize-docs` out of. Rather than publishing a second Agent Skills
bullet saying skills were renamed to `mz-`, `materialize-docs` is now
`mz-docs` is folded into the existing **`mz-` skill names** bullet as
another example. The rc.1 snapshot directory is left as the record of
what was reviewed at the time.

The final commit preserves the two Features a human editor added to this
branch — **SCIM group-to-role mapping is generally available** and
**Operational dashboards for Self-Managed** — and leaves the
Improvements and Bug Fixes sections untouched.

⚠️ **Reviewer checks**

- **The hedged-reads entry has an open testing caveat — still open, and
the one entry to cut if it stays open.** #39110 flips
`persist_blob_hedged_get_enabled` to `true` in the code default, which
is what carries the capability to Self-Managed — Cloud has served it on
through LaunchDarkly since 2026-09-24. But the PR states that real GCS
and Azure object stores are not yet tested (CI covers MinIO and Azurite)
and that testing under
[PER-120](https://linear.app/materializeinc/issue/PER-120) "should
finish before this reaches self-managed users." Nothing in the `final`
increment resolves this, and Linear was not reachable from that run.
**If [PER-120](https://linear.app/materializeinc/issue/PER-120) has not
closed, cut this entry before merging.** It is the only conditional
entry in the release.
- **The balancerd connection-limit entry changes the meaning of
something already published.** `balancerd_max_connections` shipped an
entry in v26.42.0 rc.2 describing over-limit connections as "rejected
with an error". #38854 removes that: the `TOO_MANY_CONNECTIONS` response
and the HTTPS 503 are gone, and a client over the limit now sees libpq's
"server closed the connection unexpectedly". Anyone who built alerting
on the v26.42.0 wording needs this.
- **Two PRs are excluded as already published by the v26.43.0 train** —
#38615 and #39029. Both merged to `main` inside the rc.1 window and were
cherry-picked onto the v26.43 release branch, where that train's rc.2
snapshot published them. Accounted for in `omitted.md` under "Already
published in an earlier release".
- **This branch will need a rebase if the v26.43.0 docs PR merges
first.** `_index.md` here still jumps from `## v26.44.0` to `##
v26.42.0` — the v26.43.0 section is absent because that train's PR
([#38928](#38928)) has
not merged. Whichever of the two merges second needs the rebase to get
the sections in version order.
- **Resolved:** rc.1's provisional dates were flagged as very likely one
day high. They were — the `final` snapshot corrects them down by one
day, on the tag date.

**Inclusion rate is 9.7% (14/144) at rc.1**, inside the usual 5–15%
band, and **20% (1/5) at `final`**, which reads high only because the
denominator is five. The rc.1 denominator is dominated by dependency
automation — 91 of the 142 materialize PRs are dependency bumps, 65 of
them a single Dependabot fan-out of the
`materialize-terraform-self-managed` modules across five
`test/terraform` directories. Strip those and the materialize side is 10
included out of 49.

Snapshot drafts (with PR links) in mz-skills:
- rc.1:
[release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.44.0/rc.1/release_notes.md)
/
[omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.44.0/rc.1/omitted.md)
- final:
[release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.44.0/final/release_notes.md)
/
[omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.44.0/final/omitted.md)

**Borderline PRs omitted:** rc.1: 7 — [review borderline
calls](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.44.0/rc.1/omitted.md#borderline),
final: 0 —
[none](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.44.0/final/omitted.md#borderline).

The rc.1 borderline calls are unchanged by seeing the whole release,
since `final` adds no materialize PRs. Two are still worth a reviewer's
eye: #39125 (the local kind install guide moving to RustFS) — the
documented Self-Managed local-kind install **fails today** for every
published tag, because their `minio.yaml` references a Docker Hub
repository MinIO deleted, and this is the likeliest restoration; and
#39034/#39066 (the catalog half of the Console cluster-utilization
memory and swap columns) — **if that Console PR shipped in this release,
it belongs in the notes as a Console improvement.** Three further items
are gated on flags (`enable_upsert_v2`,
`enable_compute_index_peek_offload`, `enable_unified_cluster`).

---------

Co-authored-by: Claude <[email protected]>
Co-authored-by: Valerie Chiang <[email protected]>
Co-authored-by: Pranshu Maheshwari <[email protected]>
Co-authored-by: Pranshu Maheshwari <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants