Repository navigation
[PER-136] For GCS, use a pool of HTTP/2 clients to restore blob read throughput to HTTP/1.1 levels - #39906
Merged
Conversation
ublubu
force-pushed
the
db-1310-fix-b
branch
2 times, most recently
from
October 16, 2026 20:39
bd3a803 to
24fac87
Compare
mtabebe
reviewed
Oct 19, 2026
mtabebe
left a comment
Contributor
There was a problem hiding this comment.
Some minor comments to be addressed before merge. And a bigger architectural question for a longer follow up
ublubu
force-pushed
the
db-1310-fix-b
branch
7 times, most recently
from
October 19, 2026 19:39
65a2ed6 to
612abdd
Compare
Claude: simplify 1, count in-flight requests with Arc clones instead of an atomic and a drop guard Claude: simplify 2, pick returns the member, one Pool constructor shared with the tests size-hint pass-through fix in-flight request counter tests (for failed requests) match any googleapis.com endpoint, e.g. regional and private endpoints explain where the "100 max concurrent streams" comes from
Claude: responding to other Claude's review of the spec Claude: reviews itself Claude: blob-read-bench option for the pooled client Claude: simplify blob-read bench, pass 1 Claude: simplify blob-read bench, pass 2
Claude: simplify GCS client pool setting (pass 1) persist: default persist_blob_http_pool to off, and treat unrecognized values as off
Claude: min/max "requests in flight" gauges Claude: simplify http pool metrics Claude: metrics only track one pool
Claude: simplify nightly GCP HTTP pool check
Claude: simplify http pool Getter and test helper Claude: second round of http pool simplifications
ublubu
marked this pull request as ready for review
October 19, 2026 19:42
mtabebe
approved these changes
Oct 19, 2026
mtabebe
left a comment
Contributor
There was a problem hiding this comment.
LGTM thanks for the tweaks. (I focused on the new changes)
| } | ||
| } | ||
|
|
||
| fn http_pool_metrics(registry: &MetricsRegistry) -> HttpPoolMetrics { |
bosconi
added a commit
that referenced
this pull request
Oct 20, 2026
Publish #39906 as a v26.46.1 bug fix instead of holding it back. - The regression it fixes shipped in v26.45.0. #39351 moved the AWS SDK HTTP client from hyper 0.14 / hyper-tls, which offered no ALPN protocols, to the SDK's rustls connector, which offers h2. GCS accepts h2, so every concurrent persist read on a GCS-backed deployment shares one HTTP/2 connection, capped at GCS's 100 concurrent streams. - Only Self-Managed GCS deployments are affected. Cloud runs on AWS, and S3 endpoints still negotiate HTTP/1.1. - The fix ships with persist_blob_http_pool defaulting to 'off', so affected users need to know it exists and how to turn it on. The entry gives both paths: ALTER SYSTEM SET as mz_system, or the system parameters ConfigMap. The setting takes effect without a restart. - The entry says "Fixed reduced" rather than "Restored": the v26.46.1 GCP validation measured a 1.35x-1.59x gain over 'off', not a return to pre-v26.45.0 throughput. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
maheshwarip
added a commit
that referenced
this pull request
Oct 23, 2026
Release notes for the **v26.47.0** train. One commit was added per snapshot; the `final` snapshot sets the real dates. **All six snapshots (`rc.1`–`rc.5`, `final`) are now applied — this branch is complete and ready for review.** - **Content**: 2 Feature sections (`TRY_CAST`, `EXCLUDE ALL NOT NULL CONSTRAINTS`), 7 Improvements, 13 Bug Fixes — `rc.1` contributed all of it except the top four Bug Fixes: `rc.2` added two, and `rc.3` and `rc.4` one each. `rc.5` added no new entries; it amended `rc.4`'s probe-port Bug Fix instead (see below). `final` adds nothing at all — its tracking issue holds only the unassociated `release: bump to version v26.47.0` commit ([`bed23494d`](bed2349)), and the `v26.47.0-rc.5` and `v26.47.0` tags are 14h28m apart with that bump the only thing between them. Its commit is empty and exists to record on the branch that the snapshot was applied. - **Dates (final, no longer provisional)**: Cloud **2026-10-22**, Self-Managed **2026-10-23**. Every RC estimated this same pair from the `v26.47.0-rc.1` tag, and the real dates confirmed it — the five RCs compressed into one week rather than pushing the train out — so no published date text changed in the `final` commit.⚠️ One thing worth a last look before merge: **2026-10-22 is a Wednesday**, one day off the usual Thursday Cloud date (2026-10-23 is the normal Friday Self-Managed day). That is the supplied value and it stands, but if the release owner expected Thursday 2026-10-23 / Friday 2026-10-24, this branch is where to correct it. - **Operator compatibility**: row added for `v26.47`, release date `2026-10-23`. - **Branch base**: `main`, whose newest section is `## v26.45.1` — v26.46.0's section is still on its own unmerged branch (#39690). Expect a top-of-list conflict when both merge; `v26.47.0` goes above `v26.46.0`. ✅ **Corrections already on this branch** — a `final` verification pass over the whole assembled section confirmed these and deliberately left them alone: - **`rc.1` named a syntax that does not exist.** It published "`CREATE TABLE ... FROM SOURCE` now accepts an `ALL COLUMNS NULLABLE` option", taking the name from [#39680](#39680 title and body. At the `v26.47.0` tag the option is `EXCLUDE ALL NOT NULL CONSTRAINTS` (`TableFromSourceOptionName::ExcludeAllNotNullConstraints`), and `NULLABLE` is not a lexer keyword at all, so the published syntax would have been a parse error for anyone who copied it. A human editor caught this and replaced the bullet with the `{#v26.47-exclude-not-null-constraints}` feature section now on the branch. The `rc.1` snapshot directory in mz-skills still carries the wrong wording — snapshot directories are the record of what was reviewed at the time and are left untouched. - **`rc.5`'s cross-snapshot rewrite is applied once**, including the "Deployments using password, SASL, or OIDC authentication have no separate internal SQL port and continue to be probed on the external one" clause. - **PR citations are stripped from all entries** as a deliberate house-style edit, matching how published sections read. PR links remain available in the mz-skills snapshot drafts linked below.⚠️ **Open questions for the release owner** — none of these were resolved by `final`; this branch is the last place to act on them - **Was any `rc.4` artifact handed to a customer or used for an upgrade rehearsal?** This decides a published entry. [#40036](#40036) (published as a Bug Fix by `rc.4`) made the operator probe `environmentd`'s internal SQL port unconditionally, which crashloops deployments using Password, SASL, or OIDC authentication — they have no internal SQL port. [#40046](#40046) fixed that roughly seven hours later, inside this same unreleased train, so no shipped build (`v26.46.1` probes the external port unconditionally) could have hit it. It is therefore recorded as **Borderline** rather than published as a Bug Fix of its own, and its substance folded into `rc.4`'s bullet as a scoping clause. If an `rc.4` build did reach anyone, the crashloop *was* reachable and [#40046](#40046) should be published in its own right. Note the author wrote an explicit positive release-note line for it, which is why this is flagged rather than quietly dropped. - **GA announcements — the largest judgment call in this train.** Three docs PRs remove preview markers: [#39831](#39831) (source versioning, `CREATE TABLE ... FROM SOURCE`), [#39836](#39836) (replacement materialized views), [#39830](#39830) (Kafka sink headers, noted as "GA in v26.47"). None changes code, so all three were held; drafted wording for each is in `omitted.md`. Which are actually GA in v26.47.0 and should be announced? Note v26.46.0 published Kafka sink message attribution as *private preview* — if the header goes GA here, this release should say so rather than repeat the preview framing. - **GCS blob read throughput** ([#39906](#39906), held in `rc.2`): restores the read throughput GCS deployments lost when [#39351](#39351) — already shipped — forced HTTP/2 and collapsed per-request connections into one. It is selected by `persist_blob_http_pool`, verified to default to `off` at the `v26.47.0-rc.2` tag, and the author notes "the default should eventually be 'GCS-only'". **Is `persist_blob_http_pool` being set to `gcs` for v26.47.0?** If so this is a clear win to publish for GCP users, who carry the regression today; drafted wording is in rc.2's `omitted.md`. - **`SUBSCRIBE` soft assertions** ([#37611](#37611), [#39764](#39764), both held): drafted as Bug Fixes, then pulled back. Both failures come from `soft_assert_or_log!`, which panics only where soft assertions are enabled and otherwise logs; neither PR says what a release build does. Are soft assertions on in the environments customers run? If yes, both are publishable as drafted in `omitted.md`. - **Console Freshness page** ([#39340](#39340), held): a complete new Monitoring > Freshness page, gated behind `freshness-page-CNS164`, off in both Cloud and self-managed, and the LaunchDarkly flag is not yet created. If it is turned on for v26.47.0, this likely deserves a Feature heading. - **PostgreSQL sources ingesting during a snapshot** ([#39419](#39419), [#39479](#39479), held): adding a large table to an existing source would no longer stall the tables already hydrated — but `storage_persist_sink_description_lookahead` is `Duration::ZERO` at the tag. Publish if the lookahead is being enabled. - **Coalesced materialized-view sink batches** ([#37591](#37591), held): measures ~67% fewer blob writes and ~54% smaller state diffs, at ~10 ms added p50 latency on small writes, but `compute_mv_sink_shared_batch_max_bytes` is `0` at the tag. Publish (with the latency cost stated) if the staged rollout reaches customers in this release. - **MCP 2026-07-28 conformance checks** ([#39694](#39694), [#39695](#39695), held): `enable_mcp_protocol_2026_07_28` is `false` at the tag. If the revision is being served, these belong together as one Improvement. - **Refresh strategy columns** ([#39597](#39597), published as an Improvement): this renames `aligned_to` → `aligned_to_expression` and `at` → `at_expression` and changes their type. It is a breaking change for anyone selecting them, in a schema documented as unstable — it may warrant more prominence than a bullet. - **Retained input after `ALTER SINK ... SET FROM`** (from `rc.3`'s [#37653](#37653)): the PR records a pre-existing gap it does not fix — after `SET FROM`, the *old* upstream relation stays held, and therefore un-compactable, until the sink is dropped or `environmentd` restarts. It is not something this release changed, so it is out of these notes, but it is a real footgun for the blue/green pattern that `sql/alter-sink.md` recommends `SET FROM` for. Should the `ALTER SINK` page or the blue/green guide call it out? - **Operator upgrade shows pending changes** (from `rc.4`'s [#40036](#40036)): the probe-port change alters the generated `environmentd` StatefulSet, so after upgrading the operator, existing environments report pending changes and pick up the new port on their next rollout — they are **not** restarted on their own. That is an upgrade consequence rather than something the release changed, so it is out of the notes, but a Self-Managed operator who cannot account for the pending-changes state may read it as a problem. The v26.47 upgrade guide looks like the right venue. Snapshot drafts (with PR links) in mz-skills: - rc.1: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.1/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.1/omitted.md) - rc.2: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.2/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.2/omitted.md) - rc.3: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.3/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.3/omitted.md) - rc.4: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.4/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.4/omitted.md) - rc.5: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.5/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.5/omitted.md) - final: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/final/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/final/omitted.md) **Borderline PRs omitted:** rc.1: 14 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.1/omitted.md#borderline), rc.2: 2 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.2/omitted.md#borderline), rc.3: 0 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.3/omitted.md#borderline), rc.4: 0 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.4/omitted.md#borderline), rc.5: 1 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/rc.5/omitted.md#borderline), final: 0 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.47.0/final/omitted.md#borderline) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude <[email protected]> Co-authored-by: Valerie Chiang <[email protected]> Co-authored-by: Pranshu Maheshwari <[email protected]>
val-materialize
added a commit
that referenced
this pull request
Oct 30, 2026
Release notes for the **v26.48.0** train. One commit is added per snapshot; the `final` snapshot sets the real dates. **Applied: `rc.1`, `rc.2`, `rc.3`, `final` — the train is complete.** - **Content**: 3 Feature sections (upsert hydration; Kafka sink headers GA; MySQL `EXCLUDE ALL NOT NULL CONSTRAINTS`), 9 Improvements, 2 Guides, 4 Bug Fixes. No **Agent Skills** section — every agent-skills PR in all four windows was omitted. - **Scope**: 123 PRs across the four snapshots — rc.1 114 (14 included), rc.2 5 (0 included), rc.3 1 (1 included), final 3 (0 included). - **Dates — now real, not provisional**: Cloud **2026-10-29**, Self-Managed **2026-10-30**. The `rc.1`-anchored provisional estimate was right, so `final` left both files unchanged. The `v26.48.0` tag's commit date is 2026-10-28T18:35:05Z, a Wednesday; the first Thursday on or after it is 2026-10-29, with Self-Managed the day after. - **Operator compatibility**: row for `v26.48`, release date `2026-10-30`. Unchanged by rc.2, rc.3, and final. - **Branch base**: `main`, whose newest published section was `## v26.46.1` when this branch was cut; v26.47.0 and v26.47.1 have since merged in. `v26.48.0` stays at the top of the list. ### final increment — empty, and verified empty `final`'s commit is empty. The tracking issue ([release#627](MaterializeInc/release#627)) lists exactly one item, the unassociated `4edd60a8d` "release: bump to version v26.48.0", which the release-process-automation rule omits. The three agent-skills PRs in the window ([#118](MaterializeInc/agent-skills#118), [#119](MaterializeInc/agent-skills#119), [#120](MaterializeInc/agent-skills#120)) were all omitted. This was **verified rather than assumed**, because a zero-PR snapshot and a failed fetch produce identical output: `compare/v26.48.0-rc.3...v26.48.0` returns `total_commits: 1`, and that commit is the version bump. Nothing merged onto the release branch in the 42 h 49 m between the `rc.3` tag and the ship tag. Across the train, every agent-skills PR from [#111](MaterializeInc/agent-skills#111) to [#120](MaterializeInc/agent-skills#120) falls in exactly one window, with no gaps or overlaps.⚠️ **One decision is still open for the release owner — see "Upsert v2" below.** Every flag this train held open was re-read in source at the `v26.48.0` ship tag rather than inherited from the RC snapshots: | Flag | Default at `v26.48.0` | Source | Held since | Disposition | | -- | -- | -- | -- | -- | | `enable_upsert_v2` | **`false`** | `src/storage-types/src/dyncfgs.rs:505-509` | rc.2 |⚠️ premise contradicted — see below | | `enable_upsert_chunked_stash` | **`false`** | `src/storage-types/src/dyncfgs.rs:469-475` | rc.2 | same | | `enable_upsert_paged_spill` | **`false`** | `src/storage-types/src/dyncfgs.rs:450-455` | rc.2 | same | | `enable_mcp_protocol_2026_07_28` | **`false`** | `src/adapter-dyncfgs/src/dyncfgs.rs:295-299` | rc.1 | hold stands — still omitted | | `cluster-details-page-cns-173` | n/a (LaunchDarkly) | no source default | rc.1 | hold stands — still omitted | **The rc.2 hold on [#40189](#40189) rests on a premise this branch now contradicts.** rc.2 omitted it at Borderline on exactly one ground — every measurement is taken on a path `enable_upsert_v2` keeps off by default — and asked whether any environment actually runs upsert v2. The branch answers that without anyone feeding the answer back: the human-added **"Improved hydration times and memory efficiency for upsert sources"** section states *"This change has been rolled out to all Materialize Cloud environments and applies to all existing and new clusters."* The three flags still read `false` at the ship tag, which is exactly the source-default-versus-LaunchDarkly divergence the rollout-lever rule covers: the published docs call wins, and the sole stated reason for the omit has fallen away. - **Not already covered.** The published section is about **hydration** — building a source from scratch, 206 min → 108–113 min, 542 GiB → 183–208 GiB. #40189 is about **catch-up after a backlog**, a different operation with its own motivating incident ([CPU-320](https://linear.app/materializeinc/issue/CPU-320): a source that *"could not catch up on any replica size after a producer rate increase"*). A customer who read the hydration section would learn nothing about catch-up. - **Suggested Improvements bullet, to add only if you confirm** — it was *not* applied in the `final` commit: > **Faster catch-up for upsert sources with a large backlog**: An upsert source working through a backlog now drains one timestamp per round instead of merging and re-pushing the whole backlog each round, so catch-up time grows linearly rather than quadratically with the backlog; draining a 100-timestamp tail fell from 6.5 to 0.15 seconds, and a 400-timestamp tail drains in 0.67 seconds. - **Caveat that must travel with the decision**: the baseline in those timings is the *paged* flavor on `main`, not the configuration Cloud ran before this rollout — so they are not a clean before/after for a given customer. Dropping the two timings and keeping the linear-versus-quadratic claim holds regardless of baseline. - **Two weaker links, stated rather than buried**: that the rolled-out configuration is the one #40189 was measured on is *inferred from the published section's prose* (its "buffer pool … serve sequential work from disk" reads as chunked stash plus paged spill), not confirmed by anyone. And the author declared "Release notes: None", naming `enable_upsert_v2` — so publishing this would be an **override** of an author signal, justified by your own later rollout statement. - **Third consecutive train** to hold this flag: `v26.44.0-rc.1` ([#39041](#39041)), `v26.45.0-rc.1` ([#39044](#39044)), `v26.48.0-rc.2` (#40189). If the rollout statement is accurate, the earlier two are worth revisiting rather than left to age out. **The other two holds stand, corroborated.** `enable_mcp_protocol_2026_07_28` still reads `false` at the ship tag and the five MCP PRs rc.1 held appear nowhere in the section; neither does the Console cluster-details page stack. In both cases you had the whole train in view and did not promote them — the confirmation rc.1 asked for, arriving in the negative. **No duplicates, re-checked at ship.** The seven PRs rc.1 flagged as already covered by v26.47.0 are still absent, the `## v26.47.1` section is untouched, and rc.3's entry ([#40065](#40065)) appears exactly once. **Two notes on human edits to this branch**, both recorded as fact rather than objection: - `524adb8` removed the rc.1 feature section **"Native Google Cloud Storage for persist"** ([#39983](#39983)) outright. Flagging it only because a published entry disappearing is easy to miss in a long branch — if that was deliberate, nothing to do. - Two rc.1 Improvements were promoted to their own feature sections with docs links and a code sample (Kafka sink headers [#39908](#39908); MySQL `EXCLUDE ALL NOT NULL CONSTRAINTS` [#39873](#39873)), and a **Guides** section was added that is not in the skill's template. The lesson for next time: an option whose flag flipped to `true` in the same release more likely warrants a feature section than the Improvements bullet the rubric suggests by default. **Carried forward from rc.3 and still open**: the `K8s recovery: envd on failing node` nightly — the job that sank [#40029](#40029), the first attempt at what #40065 re-lands — **was not checked**; it needs Buildkite, outside this run. The entry itself was verified correct in source at both the rc.3 and ship tags, so this is a CI-confidence question, not a notes-accuracy one. ### rc.3 increment One Improvement, for #40065: a read-only `environmentd` that reboots to pick up DDL changes during zero-downtime upgrade catch-up now restarts in place rather than exiting its container, so Kubernetes no longer applies crash-restart backoff. Not inherited from rc.1 — rc.1 correctly omitted the first attempt (#40029) together with its revert (#40061) as a no-op on `main`; rc.3 re-lands the behavior in narrowed form, verified as a real change by comparing `entrypoint.sh` at the `v26.47.0` tag (no retry loop) against `v26.48.0-rc.3` (re-exec on exit 75). The draft's closing "so zero-downtime upgrades catch up faster" was cut: no cited PR measures an upgrade duration, so the entry claims only the mechanism. - **Not a Bug Fix.** 0dt upgrades completed on v26.47.0; the catch-up reboots merely collected backoff delay. Existing behavior improved, so Improvements. - **Residual, deliberately unpublished.** A stale read-only process in a replaced pod still restarts in place on these halts until the kubelet kills its container, competing with its replacement for the generation's replicas. The `MZ_RESTARTED_IN_PLACE` marker check keeps it from booting writable, so it cannot fence the leader — the failure #40061 reverted. Worth watching if 0dt reports look odd after this ships. ### rc.2 increment — empty, by design rc.2's commit is empty: no entry was published and the provisional dates did not move. It exists so later runs can see from the commit subject that rc.2 was applied. Its single materialize PR, #40189, is the open item discussed under the `final` increment above. ### rc.1 — standing warnings⚠️ **Seven PRs in rc.1's tracking issue were already covered by the v26.47.0 train, and none is republished here.** v26.47.0 took five RCs whose later ones were cherry-picks onto the release branch; the same PRs appear in this issue because they are also what landed on `main` for v26.48.0. Four of the seven **shipped to Cloud in v26.47.0 on 2026-10-22** — #37653, #39910, #39927, #40036 — so republishing any of them would put duplicate text in front of customers eight days later. The other three (#39903, #39906, #40046) were held at Borderline in v26.47.0 and are held again here, each reason re-checked rather than inherited. The "snapshots are disjoint by construction" guarantee holds *within* a train, not across trains, so this was caught mechanically by grepping every PR number in the issue against the v26.47.0 snapshot directories. All seven are listed under their own heading in `rc.1/omitted.md`. **Open questions for the release owner** - **Upsert v2** (#40189, rc.2) — see the `final` increment above. The rollout statement on this branch has changed the picture since rc.2; this is the one item that needs a decision before merge. - **MCP protocol revision 2026-07-28** (#39698, #39702, #39703, #39759, #39761) — the stack is complete and `enable_mcp_protocol_2026_07_28` still defaults to **`false`** at the ship tag. Omitted, with the absence of any MCP text on this branch taken as confirmation. Drafted wording remains in `rc.1/omitted.md` if the flag flips. - **Redesigned cluster details page in the Console** (#39446, #39507, #39508, #39510, #39607) — omitted for the same reason. Note that the Improvements section does carry **"New cluster details page on Self-Managed"**, added by hand, which links the v26.47 section rather than re-describing the page. - **Schema projection on the Console's Freshness page** (#40031) — a genuine new way to diagnose lag, but the author states the whole Freshness page "is currently behind a feature flag that is only enabled for certain customers in impersonation mode". **Is the page generally reachable yet?** This is the same page v26.47.0 held for the same reason. - **Time zone data 2026e timing** (#40045) — the author flags the Canadian zone changes as effective **2026-11-01**, three days after the Cloud date. Tight, but it holds. *Resolved since rc.1: the Kafka sink headers framing question — the GA section on this branch answers it.* **Verification notes** — rc.1's Step 5 pass cut one drafted entry and rewrote four: - **Cut**: a Bug Fix drafted for #40032 ("halt with exit code 168 when a live reader loses its lease"). The PR's own Problem section is about alerting — `mz_persist_lease_timeout_read` cannot distinguish a dead reader from a stuck one — and the follow-up it names is a Cloud alerting change. Nothing a customer observes changes. - **tzdata** (#40045): the draft said the Canadian zones move to "permanent daylight saving time", the author's prose. The PR's data table shows a permanent **−05/EST** and **−06/CST** — summer *offsets* kept year-round, but with another zone's standard-time *abbreviations*, which is what a user sees from `AT TIME ZONE`. Reworded to what the table supports. - **Hydration memory** (#39923): scoped to the consolidation step and the measured outcome (~628 GiB peak, no restart, against every pre-change run exiting on the 849 GiB limit). The PR explicitly says the fix "covers this consolidation path, not the sink's whole footprint"; [CPU-317](https://linear.app/materializeinc/issue/CPU-317) tracks the rest. - **Kafka clients** (#40026): "55 to 24" is measured on a 4-worker, 11-source setup. The CPU throttling and degraded freshness are observed symptoms of the *old* behavior at a few hundred sources, not a measured post-change result at that scale, so the entry states the count as the outcome and the throttling as what the count drove. - **`environmentd` accept backlog** (#40035): the author's hedge is kept — "less likely to be restarted", not eliminated, because the PR says "This makes the queue fill later, not never." **Judgment calls a reviewer may want to revisit** — #39605 is filed as an **Improvement**, not a Bug Fix: nothing was incorrect, an unbounded operation gained a bound and a knob. #40101 is filed as a **Bug Fix** rather than omitted under the internal-logging rule, because credentials reaching the logs is a security defect with a user-visible remedy; only credentials a user inlined rather than wrapped in a `SECRET` were affected. Snapshot drafts (with PR links) in mz-skills: - rc.1: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.48.0/rc.1/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.48.0/rc.1/omitted.md) - rc.2: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.48.0/rc.2/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.48.0/rc.2/omitted.md) - rc.3: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.48.0/rc.3/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.48.0/rc.3/omitted.md) - final: [release_notes.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.48.0/final/release_notes.md) / [omitted.md](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.48.0/final/omitted.md) **Borderline PRs omitted:** rc.1: 12 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.48.0/rc.1/omitted.md#borderline), rc.2: 1 — [omitted.md#borderline](https://github.com/MaterializeInc/mz-skills/blob/main/data/mz-release-notes/v26.48.0/rc.2/omitted.md#borderline), rc.3: 0, final: 0 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude <[email protected]> Co-authored-by: Valerie Chiang <[email protected]> Co-authored-by: Pranshu Maheshwari <[email protected]> Co-authored-by: Pranshu Maheshwari <[email protected]>
alex-hunt-materialize
added a commit
that referenced
this pull request
Nov 10, 2026
`GcsBlob::get` streamed each blob over one HTTP/1.1 connection (the SDK forces HTTP/1.1, so the HTTP/2 pool from #39906 doesn't apply). It now reads the first 8 MiB to learn the size and generation, then fetches the remaining ranges concurrently, pinned to that generation. The SDK skips checksum validation for ranged reads, so `get` checks the total length and verifies the object's CRC32C itself, hashing each chunk as it arrives. This adds a direct `crc32c` dependency, already in `Cargo.lock` transitively. 128 MiB, n2-standard-16 in us-east1-b, median of 10: | | get | set | |---|---|---| | `GcsBlob`, single stream (before) | ~520 ms | ~880 ms | | `GcsBlob`, chunked (this PR) | ~125 ms | unchanged | | `S3Blob` over GCS S3 API, pool on | ~620 ms | ~450 ms | Writes remain slower than `S3Blob` and are left for a follow-up. Tests: `gcs_blob_get_chunk_boundaries` covers empty objects (GCS rejects ranges on them with 416, which `get` maps to an empty result) and sizes around chunk boundaries. `gcs_blob_throughput` is an ignored bench producing the numbers above. The two commits are easier to review individually: the first changes `get`, the second adds the bench. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#39351 enabled (forced) HTTP/2 for GCS clients, which created only one (shared) HTTP/2 connection where the previous behavior was to create an HTTP/1.1 connection for each concurrent request.
The AWS SDK does not give us granular control over this behavior. We can only supply our own HTTP client for the SDK to use.
I evaluated two approaches:
1. Implement an HTTP client that forces HTTP/1.1. Rejected.
https://github.com/ublubu/materialize/blob/d4caeee2c46024212597a53f7a82ae6036680625/src/aws-util/src/http_client.rs
This duplicates the AWS SDK's own HTTP/1.1 code so we can omit all the HTTP/2 bits.
2. Wrap the AWS SDK's HTTP client into a client pool. This PR.
The code simpler and we aren't maintaining a duplicate of the SDK's internals.
Intentional simplifications made in this PR:
System Flag: