Repository navigation
Conversation
The "Install locally on kind" guide downloads minio.yaml from release tags, and every published copy references the `minio/minio` Docker Hub image, which has been deleted. Add a RustFS manifest to misc/helm-charts/testing, fetch it from main in the guide, and have users point `persist_backend_url` at RustFS. The orchestratord documentation workflow follows the guide accordingly. Closes: DEP-268 Co-Authored-By: Claude Opus 5.5 <[email protected]>
Captured from a run of the guide against v26.42.0. The operator now runs two replicas behind a Service, and balancerd runs two replicas. DEP-268 Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Alphadelta14
approved these changes
Sep 24, 2026
jubrad
deleted the
justin/dep-268-self-managed-local-kind-guide-hits-minio-issue
branch
September 24, 2026 20:18
Contributor
|
@jubrad, wait, shouldn't we have merged #39115 before this PR? If a user follows the current guide, and then loads enough data to have a part >8 MB, then we crashloop. (My Claude reproed this, following the currently live guide.) Maybe we should instead point to our own MinIO build (materialize/minio:v26.42.0) until a version that has #39115 is released? |
ggevay
added a commit
that referenced
this pull request
Sep 25, 2026
Follow-up to #39125 ([DEP-268](https://linear.app/materializeinc/issue/DEP-268)). **Issue:** - Released versions of persist cannot read multipart blobs from RustFS, because RustFS omits `x-amz-mp-parts-count`. #39115 fixes this, but no release has it yet. Following the current local kind guide on v26.42.0, any batch part over 8 MiB crashes environmentd or crash-loops a cluster with `Invalid Parquet file. Corrupt footer`. - `misc/helm-charts/testing/minio.yaml` pulls `quay.io/minio/minio`, which no longer allows anonymous pulls, so the orchestratord nightly workflows cannot start MinIO. **Solution:** - `testing/minio.yaml` runs `materialize/minio:v26.42.0`, our own MinIO build, which is published under every release tag. The image has no `mc`, so a Job creates the buckets instead of the postStart hook. The service name, credentials, and buckets are unchanged, so the `persist_backend_url` in released `materialize.yaml` files keeps working. - The guide fetches `minio.yaml` from `main` and no longer asks users to edit `persist_backend_url`. - `workflow_documentation_defaults` follows the guide again. **Gotchas:** - `rustfs.yaml` stays in `misc/helm-charts/testing` for the RustFS move once a release contains #39115. - The orchestratord workflows run only in Nightly, so they need a Nightly trigger on this PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
maheshwarip
added a commit
that referenced
this pull request
Oct 2, 2026
The v26.44.0 release-note train, one commit per snapshot. Dates in RC snapshots are provisional until the release ships; the final snapshot sets the real dates. **rc.1** adds the `## v26.44.0` section — 6 Improvements, 3 Agent Skills, 4 Bug Fixes — and the `v26.44` row in the self-managed operator compatibility table. No Features section: nothing in this snapshot's 144 in-scope PRs rises to one. The Agent Skills section is back after five empty windows — the window `(2026-09-23T16:49:13Z, 2026-09-25T02:18:32Z]` holds 4 PRs and all 4 are published. **final** sets the real dates and adds no new bullets. Nothing shipped on the materialize side between `v26.44.0-rc.1` and the `v26.44.0` tag: `GET /compare/v26.44.0-rc.1...v26.44.0` is `ahead_by: 1`, and that one commit is the version bump, so every `rc.1` classification and all four verified flag defaults still hold at the ship tag. This train has only one RC (no `v26.44.0-rc.2` tag exists). Its two changes are: - **Real dates**: Cloud **2026-09-30**, Self-Managed **2026-10-01**, from the `v26.44.0` tag's commit date (`2026-09-30T10:50:59Z`) and `+1 day` — one day earlier than the provisional pair rc.1 carried, exactly as rc.1 predicted. Both date lines in `_index.md` and the `Release date:` in the YAML are updated. - **One cross-snapshot rewrite, not a new bullet**: [agent-skills#74](MaterializeInc/agent-skills#74) finishes the `mz-` rename that [#68](MaterializeInc/agent-skills#68) left `materialize-docs` out of. Rather than publishing a second Agent Skills bullet saying skills were renamed to `mz-`, `materialize-docs` is now `mz-docs` is folded into the existing **`mz-` skill names** bullet as another example. The rc.1 snapshot directory is left as the record of what was reviewed at the time. The final commit preserves the two Features a human editor added to this branch — **SCIM group-to-role mapping is generally available** and **Operational dashboards for Self-Managed** — and leaves the Improvements and Bug Fixes sections untouched.⚠️ **Reviewer checks** - **The hedged-reads entry has an open testing caveat — still open, and the one entry to cut if it stays open.** #39110 flips `persist_blob_hedged_get_enabled` to `true` in the code default, which is what carries the capability to Self-Managed — Cloud has served it on through LaunchDarkly since 2026-09-24. But the PR states that real GCS and Azure object stores are not yet tested (CI covers MinIO and Azurite) and that testing under [PER-120](https://linear.app/materializeinc/issue/PER-120) "should finish before this reaches self-managed users." Nothing in the `final` increment resolves this, and Linear was not reachable from that run. **If [PER-120](https://linear.app/materializeinc/issue/PER-120) has not closed, cut this entry before merging.** It is the only conditional entry in the release. - **The balancerd connection-limit entry changes the meaning of something already published.** `balancerd_max_connections` shipped an entry in v26.42.0 rc.2 describing over-limit connections as "rejected with an error". #38854 removes that: the `TOO_MANY_CONNECTIONS` response and the HTTPS 503 are gone, and a client over the limit now sees libpq's "server closed the connection unexpectedly". Anyone who built alerting on the v26.42.0 wording needs this. - **Two PRs are excluded as already published by the v26.43.0 train** — #38615 and #39029. Both merged to `main` inside the rc.1 window and were cherry-picked onto the v26.43 release branch, where that train's rc.2 snapshot published them. Accounted for in `omitted.md` under "Already published in an earlier release". - **This branch will need a rebase if the v26.43.0 docs PR merges first.** `_index.md` here still jumps from `## v26.44.0` to `## v26.42.0` — the v26.43.0 section is absent because that train's PR ([#38928](#38928)) has not merged. Whichever of the two merges second needs the rebase to get the sections in version order. - **Resolved:** rc.1's provisional dates were flagged as very likely one day high. They were — the `final` snapshot corrects them down by one day, on the tag date. **Inclusion rate is 9.7% (14/144) at rc.1**, inside the usual 5–15% band, and **20% (1/5) at `final`**, which reads high only because the denominator is five. The rc.1 denominator is dominated by dependency automation — 91 of the 142 materialize PRs are dependency bumps, 65 of them a single Dependabot fan-out of the `materialize-terraform-self-managed` modules across five `test/terraform` directories. Strip those and the materialize side is 10 included out of 49. Snapshot drafts (with PR links) in mz-skills: - rc.1: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.44.0/rc.1/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.44.0/rc.1/omitted.md) - final: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.44.0/final/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.44.0/final/omitted.md) **Borderline PRs omitted:** rc.1: 7 — [review borderline calls](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.44.0/rc.1/omitted.md#borderline), final: 0 — [none](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.44.0/final/omitted.md#borderline). The rc.1 borderline calls are unchanged by seeing the whole release, since `final` adds no materialize PRs. Two are still worth a reviewer's eye: #39125 (the local kind install guide moving to RustFS) — the documented Self-Managed local-kind install **fails today** for every published tag, because their `minio.yaml` references a Docker Hub repository MinIO deleted, and this is the likeliest restoration; and #39034/#39066 (the catalog half of the Console cluster-utilization memory and swap columns) — **if that Console PR shipped in this release, it belongs in the notes as a Console improvement.** Three further items are gated on flags (`enable_upsert_v2`, `enable_compute_index_peek_offload`, `enable_unified_cluster`). --------- Co-authored-by: Claude <[email protected]> Co-authored-by: Valerie Chiang <[email protected]> Co-authored-by: Pranshu Maheshwari <[email protected]> Co-authored-by: Pranshu Maheshwari <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://linear.app/materializeinc/issue/DEP-268/self-managed-local-kind-guide-hits-minio-issue
Issue: The Self-Managed "Install locally on kind" guide curls
misc/helm-charts/testing/minio.yamlfrom the release tag. Every published copy references theminio/minioDocker Hub image, which has been deleted, so the guide fails.Solution:
misc/helm-charts/testing/rustfs.yaml: a RustFS Deployment and Service, plus a Job that creates the bucket. It mirrors the kind test backends inmaterialize-terraform-self-managed.rustfs.yamlfrommain, since no release tag contains it. It also tells users to setpersist_backend_urlto RustFS, because releasedmaterialize.yamlfiles still point at MinIO.workflow_documentation_defaultsintest/orchestratord/mzcompose.pynow follows the updated guide. It uses RustFS and overrides the persist URL for every version, so the MinIO image rewrite hack is removed.Testing:
workflow_documentation_defaultsexercises the new guide flow end to end in CI. I also followed the updated guide manually on a local kind cluster against v26.42.0, using the branch copy ofrustfs.yaml. Materialize came up, a materialized view returned correct results, and persist data landed in the RustFS bucket. The samplekubectl get alloutput in the guide is taken from that run.Gotchas:
refs/heads/main/.../rustfs.yamlURL returns 404 until this merges, so the docs must not be published before then.testing/materialize.yamldeliberately stays on MinIO. The other orchestratord workflows build their secret from it and still deployminio.yaml.Tips for reviewer
There are two commits here, one to add rustfs and another to update the example kubectl output which now reflects rustfs and multiple balancerd pods.
🤖 Generated with Claude Code