Repository navigation
Convert mz_materialized_view_refresh_strategies to a builtin mv - #39597
Merged
SangJunBak merged 4 commits intoOct 14, 2026
Merged
Conversation
Contributor
Author
|
@ggevay Added you as a reviewer since you seemed like the best to review it! I know we discussed evaluating the expression in a sql function but it gets trickier since we can also have expressions that evaluate functions (e.g. |
ggevay
approved these changes
Oct 8, 2026
| DROP MATERIALIZED VIEW mv_retract | ||
|
|
||
| query I | ||
| SELECT count(*) FROM mz_internal.mz_materialized_view_refresh_strategies WHERE materialized_view_id LIKE 'u%' |
Contributor
There was a problem hiding this comment.
These counts only look at user views, so no PR-CI test checks the rows of the new builtin_strategies branch: catalog_server_explain.slt only pins its plan, and test/cluster's check_introspection runs only in Nightly. Dropping that branch, or losing a single builtin row, still passes this file. mz_history_retention_strategies.slt already has the same check for its relation. This version passes on the PR's CI image and fails when flipped to 1:
# Every materialized view, builtin or user, has a refresh strategy row.
query I
SELECT count(*)
FROM mz_catalog.mz_materialized_views mv
FULL OUTER JOIN mz_internal.mz_materialized_view_refresh_strategies rs ON mv.id = rs.materialized_view_id
WHERE mv.id IS NULL OR rs.materialized_view_id IS NULL
----
0
|
|
||
| /// Rejects times that `mz_materialized_view_refresh_strategies` could not pack as a | ||
| /// `timestamptz`, whose range is far smaller than `mz_timestamp`'s. | ||
| /// Rejects times that are too large. |
Contributor
There was a problem hiding this comment.
Nit: the deleted packer was why this check was added, but the bound still matters. With replica expiration on, compute rounds a dataflow's expiration up to the next refresh and logs it through mz_ore::now::to_datetime (compute_state.rs:811), which panics past chrono's range, the same range this check enforces. So it keeps REFRESH AT times out of that panic. Could the doc say so? The comment at materialized_views.slt:765 ("which is how these get packed") is stale too.
Comment on lines
+538
to
+539
| // - aligned_at (timestamptz) -> aligned_to_expression (text) | ||
| // - at (timestamptz)-> at_expression (text) |
Contributor
There was a problem hiding this comment.
Nit: aligned_at should be aligned_to (also in the description).
Suggested change
| // - aligned_at (timestamptz) -> aligned_to_expression (text) | |
| // - at (timestamptz)-> at_expression (text) | |
| // - aligned_to (timestamptz) -> aligned_to_expression (text) | |
| // - at (timestamptz) -> at_expression (text) |
SangJunBak
force-pushed
the
jun/sql-702/mv-refresh-strat
branch
from
October 9, 2026 15:29
ea833b7 to
1a4d69a
Compare
SangJunBak
force-pushed
the
jun/sql-702/mv-refresh-strat
branch
from
October 9, 2026 15:34
1a4d69a to
5330841
Compare
Contributor
QA LLM Review1. MEDIUM -- In-place upgrade from any pre-merge dev build crash-loops environmentd
Retyping DetailsReproduced with docker, keeping the same
The earlier conversions ( There are three ways out. Keep the desc evolvable, for example by leaving the old columns in place and appending the text columns as new nullable trailing columns. Land a fix for the same-version and forced dev paths before this PR. Or have the conversion allocate a fresh shard when the desc is incompatible, instead of carrying the table's mapping over. |
Contributor
As mentioned above, this is an instance of
Btw. fixing the underlying issues seems not too hard (but also not urgent), see https://linear.app/materializeinc/issue/SQL-663/dev-to-dev-upgrades-at-the-same-version-skip-all-builtin-schema#comment-f76ac321 |
SangJunBak
force-pushed
the
jun/sql-702/mv-refresh-strat
branch
from
October 13, 2026 17:38
5330841 to
8bdffff
Compare
For materialized views, we parse the REFRESH values from it such that we can populate `mz_materialized_view_refresh_strategies` in the future. We take the full stable ast string, meaning we don't evaluate expressions stored in the `create_sql` of a statement. This is because we would need the planner and optimizer to evaluate it, something too heavyweight for a SQL function. The consequence of this is we can't reliably report back timestamptz values `mz_materialized_view_refresh_strategies` and return back the full expression string instead. Given we'll be deprecating this feature in the future and `mz_materialized_view_refresh_strategies`'s schema isn't stable, this is acceptable.
…a materialized view We use parse_catalog_create_sql to get the refresh strategy information from the previous commit.
…gies as a materialized view
SangJunBak
force-pushed
the
jun/sql-702/mv-refresh-strat
branch
from
October 14, 2026 08:57
8bdffff to
2321960
Compare
maheshwarip
added a commit
that referenced
this pull request
Oct 23, 2026
Release notes for the **v26.47.0** train. One commit was added per snapshot; the `final` snapshot sets the real dates. **All six snapshots (`rc.1`–`rc.5`, `final`) are now applied — this branch is complete and ready for review.** - **Content**: 2 Feature sections (`TRY_CAST`, `EXCLUDE ALL NOT NULL CONSTRAINTS`), 7 Improvements, 13 Bug Fixes — `rc.1` contributed all of it except the top four Bug Fixes: `rc.2` added two, and `rc.3` and `rc.4` one each. `rc.5` added no new entries; it amended `rc.4`'s probe-port Bug Fix instead (see below). `final` adds nothing at all — its tracking issue holds only the unassociated `release: bump to version v26.47.0` commit ([`bed23494d`](bed2349)), and the `v26.47.0-rc.5` and `v26.47.0` tags are 14h28m apart with that bump the only thing between them. Its commit is empty and exists to record on the branch that the snapshot was applied. - **Dates (final, no longer provisional)**: Cloud **2026-10-22**, Self-Managed **2026-10-23**. Every RC estimated this same pair from the `v26.47.0-rc.1` tag, and the real dates confirmed it — the five RCs compressed into one week rather than pushing the train out — so no published date text changed in the `final` commit.⚠️ One thing worth a last look before merge: **2026-10-22 is a Wednesday**, one day off the usual Thursday Cloud date (2026-10-23 is the normal Friday Self-Managed day). That is the supplied value and it stands, but if the release owner expected Thursday 2026-10-23 / Friday 2026-10-24, this branch is where to correct it. - **Operator compatibility**: row added for `v26.47`, release date `2026-10-23`. - **Branch base**: `main`, whose newest section is `## v26.45.1` — v26.46.0's section is still on its own unmerged branch (#39690). Expect a top-of-list conflict when both merge; `v26.47.0` goes above `v26.46.0`. ✅ **Corrections already on this branch** — a `final` verification pass over the whole assembled section confirmed these and deliberately left them alone: - **`rc.1` named a syntax that does not exist.** It published "`CREATE TABLE ... FROM SOURCE` now accepts an `ALL COLUMNS NULLABLE` option", taking the name from [#39680](#39680 title and body. At the `v26.47.0` tag the option is `EXCLUDE ALL NOT NULL CONSTRAINTS` (`TableFromSourceOptionName::ExcludeAllNotNullConstraints`), and `NULLABLE` is not a lexer keyword at all, so the published syntax would have been a parse error for anyone who copied it. A human editor caught this and replaced the bullet with the `{#v26.47-exclude-not-null-constraints}` feature section now on the branch. The `rc.1` snapshot directory in mz-skills still carries the wrong wording — snapshot directories are the record of what was reviewed at the time and are left untouched. - **`rc.5`'s cross-snapshot rewrite is applied once**, including the "Deployments using password, SASL, or OIDC authentication have no separate internal SQL port and continue to be probed on the external one" clause. - **PR citations are stripped from all entries** as a deliberate house-style edit, matching how published sections read. PR links remain available in the mz-skills snapshot drafts linked below.⚠️ **Open questions for the release owner** — none of these were resolved by `final`; this branch is the last place to act on them - **Was any `rc.4` artifact handed to a customer or used for an upgrade rehearsal?** This decides a published entry. [#40036](#40036) (published as a Bug Fix by `rc.4`) made the operator probe `environmentd`'s internal SQL port unconditionally, which crashloops deployments using Password, SASL, or OIDC authentication — they have no internal SQL port. [#40046](#40046) fixed that roughly seven hours later, inside this same unreleased train, so no shipped build (`v26.46.1` probes the external port unconditionally) could have hit it. It is therefore recorded as **Borderline** rather than published as a Bug Fix of its own, and its substance folded into `rc.4`'s bullet as a scoping clause. If an `rc.4` build did reach anyone, the crashloop *was* reachable and [#40046](#40046) should be published in its own right. Note the author wrote an explicit positive release-note line for it, which is why this is flagged rather than quietly dropped. - **GA announcements — the largest judgment call in this train.** Three docs PRs remove preview markers: [#39831](#39831) (source versioning, `CREATE TABLE ... FROM SOURCE`), [#39836](#39836) (replacement materialized views), [#39830](#39830) (Kafka sink headers, noted as "GA in v26.47"). None changes code, so all three were held; drafted wording for each is in `omitted.md`. Which are actually GA in v26.47.0 and should be announced? Note v26.46.0 published Kafka sink message attribution as *private preview* — if the header goes GA here, this release should say so rather than repeat the preview framing. - **GCS blob read throughput** ([#39906](#39906), held in `rc.2`): restores the read throughput GCS deployments lost when [#39351](#39351) — already shipped — forced HTTP/2 and collapsed per-request connections into one. It is selected by `persist_blob_http_pool`, verified to default to `off` at the `v26.47.0-rc.2` tag, and the author notes "the default should eventually be 'GCS-only'". **Is `persist_blob_http_pool` being set to `gcs` for v26.47.0?** If so this is a clear win to publish for GCP users, who carry the regression today; drafted wording is in rc.2's `omitted.md`. - **`SUBSCRIBE` soft assertions** ([#37611](#37611), [#39764](#39764), both held): drafted as Bug Fixes, then pulled back. Both failures come from `soft_assert_or_log!`, which panics only where soft assertions are enabled and otherwise logs; neither PR says what a release build does. Are soft assertions on in the environments customers run? If yes, both are publishable as drafted in `omitted.md`. - **Console Freshness page** ([#39340](#39340), held): a complete new Monitoring > Freshness page, gated behind `freshness-page-CNS164`, off in both Cloud and self-managed, and the LaunchDarkly flag is not yet created. If it is turned on for v26.47.0, this likely deserves a Feature heading. - **PostgreSQL sources ingesting during a snapshot** ([#39419](#39419), [#39479](#39479), held): adding a large table to an existing source would no longer stall the tables already hydrated — but `storage_persist_sink_description_lookahead` is `Duration::ZERO` at the tag. Publish if the lookahead is being enabled. - **Coalesced materialized-view sink batches** ([#37591](#37591), held): measures ~67% fewer blob writes and ~54% smaller state diffs, at ~10 ms added p50 latency on small writes, but `compute_mv_sink_shared_batch_max_bytes` is `0` at the tag. Publish (with the latency cost stated) if the staged rollout reaches customers in this release. - **MCP 2026-07-28 conformance checks** ([#39694](#39694), [#39695](#39695), held): `enable_mcp_protocol_2026_07_28` is `false` at the tag. If the revision is being served, these belong together as one Improvement. - **Refresh strategy columns** ([#39597](#39597), published as an Improvement): this renames `aligned_to` → `aligned_to_expression` and `at` → `at_expression` and changes their type. It is a breaking change for anyone selecting them, in a schema documented as unstable — it may warrant more prominence than a bullet. - **Retained input after `ALTER SINK ... SET FROM`** (from `rc.3`'s [#37653](#37653)): the PR records a pre-existing gap it does not fix — after `SET FROM`, the *old* upstream relation stays held, and therefore un-compactable, until the sink is dropped or `environmentd` restarts. It is not something this release changed, so it is out of these notes, but it is a real footgun for the blue/green pattern that `sql/alter-sink.md` recommends `SET FROM` for. Should the `ALTER SINK` page or the blue/green guide call it out? - **Operator upgrade shows pending changes** (from `rc.4`'s [#40036](#40036)): the probe-port change alters the generated `environmentd` StatefulSet, so after upgrading the operator, existing environments report pending changes and pick up the new port on their next rollout — they are **not** restarted on their own. That is an upgrade consequence rather than something the release changed, so it is out of the notes, but a Self-Managed operator who cannot account for the pending-changes state may read it as a problem. The v26.47 upgrade guide looks like the right venue. Snapshot drafts (with PR links) in mz-skills: - rc.1: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.1/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.1/omitted.md) - rc.2: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.2/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.2/omitted.md) - rc.3: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.3/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.3/omitted.md) - rc.4: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.4/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.4/omitted.md) - rc.5: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.5/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.5/omitted.md) - final: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/final/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/final/omitted.md) **Borderline PRs omitted:** rc.1: 14 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.1/omitted.md#borderline), rc.2: 2 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.2/omitted.md#borderline), rc.3: 0 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.3/omitted.md#borderline), rc.4: 0 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.4/omitted.md#borderline), rc.5: 1 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.5/omitted.md#borderline), final: 0 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/final/omitted.md#borderline) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude <[email protected]> Co-authored-by: Valerie Chiang <[email protected]> Co-authored-by: Pranshu Maheshwari <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mz_materialized_view_refresh_strategiesto a builtin materialized view. Read commit messages for more detailsaligned_atandatfrom timestamptz to string values.Motivation
Closes SQL-701
Verification
mz_materialized_view_refresh_strategiesalready. Added a retraction test case