Skip to content

fix(tsdb): chunk overflow on ooo query#18692

Merged
krajorama merged 4 commits into
mainfrom
krajo/fix-chunk-overlap-query
May 14, 2026
Merged

fix(tsdb): chunk overflow on ooo query#18692
krajorama merged 4 commits into
mainfrom
krajo/fix-chunk-overlap-query

Conversation

@krajorama

@krajorama krajorama commented May 14, 2026

Copy link
Copy Markdown
Member

Protect against and fix overflow of chunks with more than 2^16-1 samples in case we're recoding chunks due to for example in-order and ooo samples overlap during compaction or query. This is very unlikely to happen, but we should still fix it I think.

Which issue(s) does the PR fix:

Release notes for end users (ALL commits must be considered).

Reviewers should verify clarity and quality.

[BUGFIX] TSDB: panic with native histograms during query of overlapping chunks.

Protect against and fix overflow of chunks with more than 2^16-1 samples
in case we're recoding chunks due to for example in-order and ooo samples
overlap during compaction or query.

Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
krajorama force-pushed the krajo/fix-chunk-overlap-query branch from 9170f36 to d9a29f3 Compare May 14, 2026 11:29
@krajorama
krajorama marked this pull request as ready for review May 14, 2026 11:30
@krajorama
krajorama requested review from a team, bwplotka, codesome and jesusvazquez as code owners May 14, 2026 11:30

@bwplotka bwplotka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread tsdb/chunkenc/chunk.go
Comment thread tsdb/querier.go
Comment thread tsdb/querier_test.go Outdated
Comment thread tsdb/querier_test.go Outdated

@ywwg ywwg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small comments and a big question :)

Comment thread tsdb/chunkenc/float_histogram.go Outdated
Comment thread tsdb/chunkenc/histogram.go Outdated
Comment thread tsdb/chunkenc/xor2.go
krajorama added 3 commits May 14, 2026 15:40
Signed-off-by: György Krajcsovits <[email protected]>
Signed-off-by: György Krajcsovits <[email protected]>
Mention that the implementation may panic to protect against overflow.

Signed-off-by: György Krajcsovits <[email protected]>

@ywwg ywwg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you!

@krajorama
krajorama merged commit 8a9f4ff into main May 14, 2026
35 checks passed
@krajorama
krajorama deleted the krajo/fix-chunk-overlap-query branch May 14, 2026 20:22
aknuds1 added a commit to aknuds1/prometheus that referenced this pull request May 15, 2026
The oversize-chunk trigger introduced in prometheus#18692 was implemented as a
closure defined inside the per-sample loop in
populateChunksFromIterable and invoked once at the `if` condition.
Replace it with a plain conditional and hoist
`len(currentChunk.Bytes())` out of the switch so the two encoding
cases don't repeat the same expression. The new shape preserves the
original `||` short-circuit: the size check is only evaluated when
neither the encoding nor the start-timestamp capability forces a new
chunk, which also keeps `currentChunk` non-nil at the point of read.

`gcflags=-m=2` reports the closure body inlined and the symbol table
shows no separate `func1` symbol, yet benchstat shows a measurable
speedup. The most likely explanation: the closure body inlines, but
the `funcval` struct (capturing `currentChunk` and `currentValueType`)
is still stack-constructed each iteration — invisible to escape
analysis, but a real per-iteration cost in a hot loop.

Benchmark, `go test -count=6 -benchmem -bench=BenchmarkQuerierSelectWithOutOfOrder -benchtime=5s -run=^$ ./tsdb/`,
Intel Xeon Platinum 8280 @ 2.70 GHz (linux/amd64), 1M-series head,
query selectivity varies:

                                          │   main      │           optimized                │
                                          │   sec/op    │    sec/op    vs base               │
Head/1of1000000-16                          301.5m ± 4%   257.0m ±  4%  -14.74% (p=0.002 n=6)
Head/10of1000000-16                         305.6m ± 3%   260.4m ±  2%  -14.80% (p=0.002 n=6)
Head/100of1000000-16                        303.9m ± 2%   259.7m ±  2%  -14.54% (p=0.002 n=6)
Head/1000of1000000-16                       303.8m ± 2%   267.0m ±  2%  -12.13% (p=0.002 n=6)
Head/10000of1000000-16                      318.1m ± 1%   278.9m ±  8%  -12.33% (p=0.002 n=6)
Head/100000of1000000-16                     364.1m ± 7%   352.8m ±  4%        ~ (p=0.065 n=6)
Head/1000000of1000000-16                     1.115 ± 2%    1.089 ± 26%        ~ (p=0.394 n=6)
geomean                                     377.8m        337.3m        -10.71%

allocs/op and B/op unchanged. The two largest-selectivity cases trend
faster but are dominated by the per-sample append cost so the relative
delta is smaller and lost in variance.

`TestChunkQuerier_OverlappingInOrderAndOOOChunks` continues to
exercise the overflow path.

```release-notes
[PERF] TSDB: Speed up query-time chunk reconstruction when in-order and out-of-order chunks overlap.
```

Signed-off-by: Arve Knudsen <[email protected]>
aknuds1 added a commit to aknuds1/prometheus that referenced this pull request May 15, 2026
The oversize-chunk trigger introduced in prometheus#18692 was implemented as a
closure defined inside the per-sample loop in
populateChunksFromIterable and invoked once at the `if` condition.
Replace it with a plain conditional and hoist
`len(currentChunk.Bytes())` out of the switch so the two encoding
cases don't repeat the same expression. The new shape preserves the
original `||` short-circuit: the size check is only evaluated when
neither the encoding nor the start-timestamp capability forces a new
chunk, which also keeps `currentChunk` non-nil at the point of read.

`gcflags=-m=2` reports the closure body inlined and the symbol table
shows no separate `func1` symbol, yet benchstat shows a measurable
speedup. The most likely explanation: the closure body inlines, but
the `funcval` struct (capturing `currentChunk` and `currentValueType`)
is still stack-constructed each iteration — invisible to escape
analysis, but a real per-iteration cost in a hot loop.

Benchmark, `go test -count=6 -benchmem -bench=BenchmarkQuerierSelectWithOutOfOrder -benchtime=5s -run=^$ ./tsdb/`,
Intel Xeon Platinum 8280 @ 2.70 GHz (linux/amd64), 1M-series head,
query selectivity varies:

                                          │   main      │           optimized                │
                                          │   sec/op    │    sec/op    vs base               │
Head/1of1000000-16                          301.5m ± 4%   257.0m ±  4%  -14.74% (p=0.002 n=6)
Head/10of1000000-16                         305.6m ± 3%   260.4m ±  2%  -14.80% (p=0.002 n=6)
Head/100of1000000-16                        303.9m ± 2%   259.7m ±  2%  -14.54% (p=0.002 n=6)
Head/1000of1000000-16                       303.8m ± 2%   267.0m ±  2%  -12.13% (p=0.002 n=6)
Head/10000of1000000-16                      318.1m ± 1%   278.9m ±  8%  -12.33% (p=0.002 n=6)
Head/100000of1000000-16                     364.1m ± 7%   352.8m ±  4%        ~ (p=0.065 n=6)
Head/1000000of1000000-16                     1.115 ± 2%    1.089 ± 26%        ~ (p=0.394 n=6)
geomean                                     377.8m        337.3m        -10.71%

allocs/op and B/op unchanged. The two largest-selectivity cases trend
faster but are dominated by the per-sample append cost so the relative
delta is smaller and lost in variance.

`TestChunkQuerier_OverlappingInOrderAndOOOChunks` continues to
exercise the overflow path.

```release-notes
NONE
```

Signed-off-by: Arve Knudsen <[email protected]>
aknuds1 added a commit that referenced this pull request May 15, 2026
…#18699)

The oversize-chunk trigger introduced in #18692 was implemented as a
closure defined inside the per-sample loop in
populateChunksFromIterable and invoked once at the `if` condition.
Replace it with a plain conditional and hoist
`len(currentChunk.Bytes())` out of the switch so the two encoding
cases don't repeat the same expression. The new shape preserves the
original `||` short-circuit: the size check is only evaluated when
neither the encoding nor the start-timestamp capability forces a new
chunk, which also keeps `currentChunk` non-nil at the point of read.

`gcflags=-m=2` reports the closure body inlined and the symbol table
shows no separate `func1` symbol, yet benchstat shows a measurable
speedup. The most likely explanation: the closure body inlines, but
the `funcval` struct (capturing `currentChunk` and `currentValueType`)
is still stack-constructed each iteration — invisible to escape
analysis, but a real per-iteration cost in a hot loop.

Benchmark, `go test -count=6 -benchmem -bench=BenchmarkQuerierSelectWithOutOfOrder -benchtime=5s -run=^$ ./tsdb/`,
Intel Xeon Platinum 8280 @ 2.70 GHz (linux/amd64), 1M-series head,
query selectivity varies:

                                          │   main      │           optimized                │
                                          │   sec/op    │    sec/op    vs base               │
Head/1of1000000-16                          301.5m ± 4%   257.0m ±  4%  -14.74% (p=0.002 n=6)
Head/10of1000000-16                         305.6m ± 3%   260.4m ±  2%  -14.80% (p=0.002 n=6)
Head/100of1000000-16                        303.9m ± 2%   259.7m ±  2%  -14.54% (p=0.002 n=6)
Head/1000of1000000-16                       303.8m ± 2%   267.0m ±  2%  -12.13% (p=0.002 n=6)
Head/10000of1000000-16                      318.1m ± 1%   278.9m ±  8%  -12.33% (p=0.002 n=6)
Head/100000of1000000-16                     364.1m ± 7%   352.8m ±  4%        ~ (p=0.065 n=6)
Head/1000000of1000000-16                     1.115 ± 2%    1.089 ± 26%        ~ (p=0.394 n=6)
geomean                                     377.8m        337.3m        -10.71%

allocs/op and B/op unchanged. The two largest-selectivity cases trend
faster but are dominated by the per-sample append cost so the relative
delta is smaller and lost in variance.

`TestChunkQuerier_OverlappingInOrderAndOOOChunks` continues to
exercise the overflow path.

```release-notes
NONE
```

Signed-off-by: Arve Knudsen <[email protected]>
eleboucher pushed a commit to eleboucher/homelab that referenced this pull request May 28, 2026
…➔ v3.12.0) (#730)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [quay.io/prometheus/prometheus](https://github.com/prometheus/prometheus) | minor | `v3.11.3` → `v3.12.0` |

---

### Release Notes

<details>
<summary>prometheus/prometheus (quay.io/prometheus/prometheus)</summary>

### [`v3.12.0`](https://github.com/prometheus/prometheus/releases/tag/v3.12.0): 3.12.0 / 2026-05-28

[Compare Source](prometheus/prometheus@v3.11.3...v3.12.0)

- \[SECURITY] Remote-write: Reject snappy-compressed requests whose declared decoded length exceeds the 32MB. Thanks to [@&#8203;hibrian827](https://github.com/hibrian827) for reporting it. [#&#8203;18642](prometheus/prometheus#18642)
- \[SECURITY] STACKIT SD: Fix secrets being exposed in plaintext via `/-/config` endpoint. Thanks to [@&#8203;August829](https://github.com/August829) and [@&#8203;Phaxma](https://github.com/Phaxma) for reporting. GHSA-39j6-789q-qxvh [#&#8203;18649](prometheus/prometheus#18649)
- \[CHANGE] TSDB/Agent: Adds Start Timestamp field to all WAL Histogram samples in memory; used `st-storage` flag is enabled. [#&#8203;18221](prometheus/prometheus#18221)
- \[FEATURE] API: Add `/api/v1/status/self_metrics` endpoint returning the current state of the Prometheus server's own metrics about itself as JSON. [#&#8203;18411](prometheus/prometheus#18411)
- \[FEATURE] Discovery: Add DigitalOcean Managed Databases service discovery [#&#8203;18287](prometheus/prometheus#18287)
- \[FEATURE] Prometheus: Add support for the aix/ppc64 compilation target [#&#8203;18321](prometheus/prometheus#18321)
- \[FEATURE] Discovery: Add Outscale VM service discovery (`outscale_sd_configs`) for discovering scrape targets from the Outscale Cloud API. [#&#8203;18139](prometheus/prometheus#18139)
- \[FEATURE] PromQL: Emit a warning when `sort`, `sort_by_label` or `sort_by_label_desc` is used within range (matrix) queries, as these functions do not have effect in that context. [#&#8203;18498](prometheus/prometheus#18498)
- \[FEATURE] PromQL: Add `start()`, `end()`, `range()`, and `step()` experimental functions [#&#8203;17877](prometheus/prometheus#17877)
- \[FEATURE] PromQL: Update `resets()` function to consider start timestamp resets. Hidden behind `use-start-timestamps` feature flag. [#&#8203;18627](prometheus/prometheus#18627)
- \[FEATURE] Prometheus: Promote auto-reload-config as stable [#&#8203;18620](prometheus/prometheus#18620)
- \[FEATURE] TSDB/Agent: Add `CheckpointFromInMemorySeries` option to `agent.DB` that enables checkpoint based on in-memory series. [#&#8203;17948](prometheus/prometheus#17948)
- \[FEATURE] UI: Add a web interface for deleting time series and cleaning tombstones, accessible from the Status menu. [#&#8203;18390](prometheus/prometheus#18390)
- \[FEATURE] PromQL: Use start timestamps for `rate()`, `irate(), and `increase()`calculations, behind a feature flag`use-start-timestamps`. Doesn't work together with extended range selectors `anchored`and`smoothed\`. [#&#8203;18344](prometheus/prometheus#18344)
- \[FEATURE] Scrape: Added a feature flag `st-synthesis` which synthesizes unknown STs for scraped cumulative metrics. Useful when Remote Writing 2.0 with delta or Otel-based backends. [#&#8203;18279](prometheus/prometheus#18279)
- \[FEATURE] promqltest: support `@st` annotation in `load` blocks to specify per-sample start timestamps. [#&#8203;18360](prometheus/prometheus#18360)
- \[ENHANCEMENT] API: reject concurrent fgprof profiles. [#&#8203;18651](prometheus/prometheus#18651)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field to ECS/MSK/RDS/Elasticache. [#&#8203;18579](prometheus/prometheus#18579)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field. [#&#8203;17171](prometheus/prometheus#17171)
- \[ENHANCEMENT] Discovery: Propagate SD target updates faster by introducing dynamic backoff interval instead of static 5s interval for throttling. [#&#8203;18187](prometheus/prometheus#18187)
- \[ENHANCEMENT] Promtool: Add `--header` flag to `query instant` command, matching existing `query range` behaviour. [#&#8203;18418](prometheus/prometheus#18418)
- \[ENHANCEMENT]: AWS SD: Allows EC2 service discovery to discover IPv6 addresses to communicate with target endpoints. The private IPv4 address remains the default when both IPv4 and IPv6 addresses are present. [#&#8203;16088](prometheus/prometheus#16088)
- \[PERF] TSDB: Make head chunk lookup in range queries constant time instead of quadratic time [#&#8203;18302](prometheus/prometheus#18302)
- \[PERF] TSDB: Skip entire stripes in mmapHeadChunks when no series need mmapping, reducing CPU utilization significantly at production-relevant scales. [#&#8203;18541](prometheus/prometheus#18541)
- \[PERF] TSDB: Skip clean series during periodic head chunk mmap using cached head chunk count [#&#8203;18272](prometheus/prometheus#18272)
- \[PERF] PromQL: Address FloatHistogram.KahanAdd performance regression on Go 1.26. [#&#8203;18568](prometheus/prometheus#18568)
- \[BUGFIX] PromQL: Fix `info()` function incorrectly handling negated `__name__` matchers [#&#8203;17932](prometheus/prometheus#17932)
- \[BUGFIX] API: Return duration expressions in `/parse_ast`. [#&#8203;18624](prometheus/prometheus#18624)
- \[BUGFIX] API: correctly document formats accepted for duration query request parameters (step, timeout and lookback delta) in OpenAPI spec [#&#8203;18305](prometheus/prometheus#18305)
- \[BUGFIX] Scrape: AppenderV2 now tracks staleness even when OOO/duplicate series errors happen similar to AppenderV1 [#&#8203;18567](prometheus/prometheus#18567)
- \[BUGFIX] Config: Validate remote\_write queue\_config fields at load time to prevent runtime panic and silent misconfiguration. [#&#8203;18209](prometheus/prometheus#18209)
- \[BUGFIX] Discovery/Consul: Add `health_filter` for Health API filtering, fixing breakage when using Catalog-only fields like `ServiceTags` in `filter`. [#&#8203;18479](prometheus/prometheus#18479) [#&#8203;18499](prometheus/prometheus#18499)
- \[BUGFIX] OTLP: limit decompressed body size for gzip-encoded OTLP write requests. [#&#8203;18408](prometheus/prometheus#18408)
- \[BUGFIX] PromQL: Fix `smoothed` rate/increase returning zero instead of no result when all data falls strictly after the query range. [#&#8203;18523](prometheus/prometheus#18523)
- \[BUGFIX] PromQL: Fix metric name not being dropped when last\_over\_time or first\_over\_time is applied to subqueries containing name-dropping functions like abs(). [#&#8203;18409](prometheus/prometheus#18409)
- \[BUGFIX] PromQL: Fix missing warning when mixing exponential and custom-bucket histograms in stats queries. [#&#8203;18660](prometheus/prometheus#18660)
- \[BUGFIX] PromQL: Fix parsing of `range()` keyword in duration expressions such as `foo[5m+range()]`. [#&#8203;18623](prometheus/prometheus#18623)
- \[BUGFIX] PromQL: Fix smoothed vector selector returning no results in binary operations when the `@` modifier is used. [#&#8203;18531](prometheus/prometheus#18531)
- \[BUGFIX] PromQL: Reject NaN, infinite, and out-of-range duration expressions instead of silently producing an out-of-range time.Duration. [#&#8203;18639](prometheus/prometheus#18639)
- \[BUGFIX] Scrape: Fix panic when scraping malformed native histograms. [#&#8203;18414](prometheus/prometheus#18414)
- \[BUGFIX] Scrape: fix panic when scraping a target exposing a summary with no quantiles via the protobuf format. [#&#8203;18382](prometheus/prometheus#18382)
- \[BUGFIX] Scrape: fix scrape failure log file occasionally not applied after a configuration reload. [#&#8203;18421](prometheus/prometheus#18421)
- \[BUGFIX] TSDB: Allow retention percentage with new data path. [#&#8203;18628](prometheus/prometheus#18628)
- \[BUGFIX] TSDB: Preserve decimal precision in percentage-based retention [#&#8203;18374](prometheus/prometheus#18374)
- \[BUGFIX] TSDB: fix prometheus\_tsdb\_head\_chunks going negative after WAL replay [#&#8203;18401](prometheus/prometheus#18401)
- \[BUGFIX] TSDB: panic with native histograms during query of overlapping chunks. [#&#8203;18692](prometheus/prometheus#18692)
- \[BUGFIX] Tracing: fix startup failure for insecure OTLP HTTP tracing [#&#8203;18469](prometheus/prometheus#18469)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#&#8203;18658](prometheus/prometheus#18658)
- \[BUGFIX] UI: Improve Y-axis tick label precision for graph values over small ranges. [#&#8203;18682](prometheus/prometheus#18682)
- \[BUGFIX] `prometheus_sd_refresh*` and `prometheus_sd_discovered_targets` metrics for specific scrape jobs are deleted when the scrape job is removed. [#&#8203;17614](prometheus/prometheus#17614)
- \[BUGFIX] Remote: fixed validation for received RW2 requests when parsing metadata unit symbols. This fixes a case when request would cause (recovered) handler panic. [#&#8203;18641](prometheus/prometheus#18641)
- \[BUGFIX] TSDB/Agent: fix race in agent appender where concurrent appends for the same label set could produce duplicate in-memory series and duplicate WAL records. [#&#8203;18292](prometheus/prometheus#18292)
- \[BUGFIX] Config: Update `--enable-feature`  flag description and sort feature names. [#&#8203;18487](prometheus/prometheus#18487)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEwMS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZS9jb250YWluZXIiLCJ0eXBlL21pbm9yIl19-->

Reviewed-on: https://git.erwanleboucher.dev/eleboucher/homelab/pulls/730
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jun 6, 2026
##### [\`v3.12.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.12.0)

This release contains security fixes, new features (especially around PromQL and Service Discovery), performance improvements in TSDB, Start Timestamp improvements and numerous bug fixes.

Thanks to all contributors!

#### Key Highlights

- **Security**: Two security vulnerabilities have been addressed: a denial of service in remote-write (snappy decompression limit) and a secret exposure leak in STACKIT service discovery.
- **PromQL & Metadata**: Several features and bug fixes related to the experimental "start timestamps" support, including updates to `rate()`, `irate()`, `increase()`, and `resets()`. New experimental functions `start()`, `end()`, `range()`, and `step()` are introduced.
- **TSDB Performance**: Optimizations in head chunk lookup (constant time) and mmap operations to reduce CPU usage.
- **Service Discovery**: Added support for DigitalOcean Managed Databases and Outscale VM, along with improvements to AWS SD (IPv6 support for EC2, external ID support).
- **UI**: Added a web interface for deleting time series and cleaning tombstones.

#### Changelog

- \[SECURITY] Remote: Reject snappy-compressed received requests via Remote Write whose declared decoded length exceeds the 32MB. Thanks to [@hibrian827](https://github.com/hibrian827) for reporting it. [#18642](prometheus/prometheus#18642)
- \[SECURITY] STACKIT SD: Fix secrets being exposed in plaintext via `/-/config` endpoint. Thanks to [@August829](https://github.com/August829) and [@Phaxma](https://github.com/Phaxma) for reporting. GHSA-39j6-789q-qxvh [#18649](prometheus/prometheus#18649)
- \[CHANGE] TSDB/Agent: Adds Start Timestamp field to all WAL Histogram samples in memory; used `st-storage` flag is enabled. [#18221](prometheus/prometheus#18221)
- \[FEATURE] API: Add `/api/v1/status/self_metrics` endpoint returning the current state of the Prometheus server's own metrics about itself as JSON. [#18411](prometheus/prometheus#18411)
- \[FEATURE] Discovery: Add DigitalOcean Managed Databases service discovery [#18287](prometheus/prometheus#18287)
- \[FEATURE] Prometheus: Add support for the aix/ppc64 compilation target [#18321](prometheus/prometheus#18321)
- \[FEATURE] Discovery: Add Outscale VM service discovery (`outscale_sd_configs`) for discovering scrape targets from the Outscale Cloud API. [#18139](prometheus/prometheus#18139)
- \[FEATURE] PromQL: Emit a warning when `sort`, `sort_by_label` or `sort_by_label_desc` is used within range (matrix) queries, as these functions do not have effect in that context. [#18498](prometheus/prometheus#18498)
- \[FEATURE] PromQL: Add `start()`, `end()`, `range()`, and `step()` experimental functions [#17877](prometheus/prometheus#17877)
- \[FEATURE] PromQL: Update `resets()` function to consider start timestamp resets. Hidden behind `use-start-timestamps` feature flag. [#18627](prometheus/prometheus#18627)
- \[FEATURE] Prometheus: Promote auto-reload-config as stable [#18620](prometheus/prometheus#18620)
- \[FEATURE] TSDB/Agent: Add `CheckpointFromInMemorySeries` option to `agent.DB` that enables checkpoint based on in-memory series. [#17948](prometheus/prometheus#17948)
- \[FEATURE] UI: Add a web interface for deleting time series and cleaning tombstones, accessible from the Status menu. [#18390](prometheus/prometheus#18390)
- \[FEATURE] PromQL: Use start timestamps for `rate()`, `irate()`, and `increase()` calculations, behind a feature flag `use-start-timestamps`. Doesn't work together with extended range selectors `anchored` and `smoothed`. [#18344](prometheus/prometheus#18344)
- \[FEATURE] Scrape: Added a feature flag `st-synthesis` which synthesizes unknown STs for scraped cumulative metrics. Useful when Remote Writing 2.0 with delta or Otel-based backends. [#18279](prometheus/prometheus#18279)
- \[FEATURE] promqltest: support `@st` annotation in `load` blocks to specify per-sample start timestamps. [#18360](prometheus/prometheus#18360)
- \[ENHANCEMENT] API: reject concurrent fgprof profiles. [#18651](prometheus/prometheus#18651)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field to ECS/MSK/RDS/Elasticache. [#18579](prometheus/prometheus#18579)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field. [#17171](prometheus/prometheus#17171)
- \[ENHANCEMENT] Discovery: Propagate SD target updates faster by introducing dynamic backoff interval instead of static 5s interval for throttling. [#18187](prometheus/prometheus#18187)
- \[ENHANCEMENT] Promtool: Add `--header` flag to `query instant` command, matching existing `query range` behaviour. [#18418](prometheus/prometheus#18418)
- \[ENHANCEMENT]: AWS SD: Allows EC2 service discovery to discover IPv6 addresses to communicate with target endpoints. The private IPv4 address remains the default when both IPv4 and IPv6 addresses are present. [#16088](prometheus/prometheus#16088)
- \[PERF] TSDB: Make head chunk lookup in range queries constant time instead of quadratic time [#18302](prometheus/prometheus#18302)
- \[PERF] TSDB: Skip entire stripes in mmapHeadChunks when no series need mmapping, reducing CPU utilization significantly at production-relevant scales. [#18541](prometheus/prometheus#18541)
- \[PERF] TSDB: Skip clean series during periodic head chunk mmap using cached head chunk count [#18272](prometheus/prometheus#18272)
- \[PERF] PromQL: Address FloatHistogram.KahanAdd performance regression on Go 1.26. [#18568](prometheus/prometheus#18568)
- \[BUGFIX] PromQL: Fix `info()` function incorrectly handling negated `__name__` matchers [#17932](prometheus/prometheus#17932)
- \[BUGFIX] API: Return duration expressions in `/parse_ast`. [#18624](prometheus/prometheus#18624)
- \[BUGFIX] API: correctly document formats accepted for duration query request parameters (step, timeout and lookback delta) in OpenAPI spec [#18305](prometheus/prometheus#18305)
- \[BUGFIX] Scrape: AppenderV2 now tracks staleness even when OOO/duplicate series errors happen similar to AppenderV1 [#18567](prometheus/prometheus#18567)
- \[BUGFIX] Config: Validate remote\_write queue\_config fields at load time to prevent runtime panic and silent misconfiguration. [#18209](prometheus/prometheus#18209)
- \[BUGFIX] Discovery/Consul: Add `health_filter` for Health API filtering, fixing breakage when using Catalog-only fields like `ServiceTags` in `filter`. [#18479](prometheus/prometheus#18479) [#18499](prometheus/prometheus#18499)
- \[BUGFIX] OTLP: limit decompressed body size for gzip-encoded OTLP write requests. [#18408](prometheus/prometheus#18408)
- \[BUGFIX] PromQL: Fix `smoothed` rate/increase returning zero instead of no result when all data falls strictly after the query range. [#18523](prometheus/prometheus#18523)
- \[BUGFIX] PromQL: Fix metric name not being dropped when last\_over\_time or first\_over\_time is applied to subqueries containing name-dropping functions like abs(). [#18409](prometheus/prometheus#18409)
- \[BUGFIX] PromQL: Fix missing warning when mixing exponential and custom-bucket histograms in stats queries. [#18660](prometheus/prometheus#18660)
- \[BUGFIX] PromQL: Fix parsing of `range()` keyword in duration expressions such as `foo[5m+range()]`. [#18623](prometheus/prometheus#18623)
- \[BUGFIX] PromQL: Fix smoothed vector selector returning no results in binary operations when the `@` modifier is used. [#18531](prometheus/prometheus#18531)
- \[BUGFIX] PromQL: Reject NaN, infinite, and out-of-range duration expressions instead of silently producing an out-of-range time.Duration. [#18639](prometheus/prometheus#18639)
- \[BUGFIX] Scrape: Fix panic when scraping malformed native histograms. [#18414](prometheus/prometheus#18414)
- \[BUGFIX] Scrape: fix panic when scraping a target exposing a summary with no quantiles via the protobuf format. [#18382](prometheus/prometheus#18382)
- \[BUGFIX] Scrape: fix scrape failure log file occasionally not applied after a configuration reload. [#18421](prometheus/prometheus#18421)
- \[BUGFIX] TSDB: Allow retention percentage with new data path. [#18628](prometheus/prometheus#18628)
- \[BUGFIX] TSDB: Preserve decimal precision in percentage-based retention [#18374](prometheus/prometheus#18374)
- \[BUGFIX] TSDB: fix prometheus\_tsdb\_head\_chunks going negative after WAL replay [#18401](prometheus/prometheus#18401)
- \[BUGFIX] TSDB: panic with native histograms during query of overlapping chunks. [#18692](prometheus/prometheus#18692)
- \[BUGFIX] Tracing: fix startup failure for insecure OTLP HTTP tracing [#18469](prometheus/prometheus#18469)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] UI: Improve Y-axis tick label precision for graph values over small ranges. [#18682](prometheus/prometheus#18682)
- \[BUGFIX] `prometheus_sd_refresh*` and `prometheus_sd_discovered_targets` metrics for specific scrape jobs are deleted when the scrape job is removed. [#17614](prometheus/prometheus#17614)
- \[BUGFIX] Remote: fixed validation for received RW2 requests when parsing metadata unit symbols. This fixes a case when request would cause (recovered) handler panic. [#18641](prometheus/prometheus#18641)
- \[BUGFIX] TSDB/Agent: fix race in agent appender where concurrent appends for the same label set could produce duplicate in-memory series and duplicate WAL records. [#18292](prometheus/prometheus#18292)
- \[BUGFIX] Config: Update `--enable-feature`  flag description and sort feature names. [#18487](prometheus/prometheus#18487)
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jun 6, 2026
##### [\`v3.12.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.12.0)

This release contains security fixes, new features (especially around PromQL and Service Discovery), performance improvements in TSDB, Start Timestamp improvements and numerous bug fixes.

Thanks to all contributors!

#### Key Highlights

- **Security**: Two security vulnerabilities have been addressed: a denial of service in remote-write (snappy decompression limit) and a secret exposure leak in STACKIT service discovery.
- **PromQL & Metadata**: Several features and bug fixes related to the experimental "start timestamps" support, including updates to `rate()`, `irate()`, `increase()`, and `resets()`. New experimental functions `start()`, `end()`, `range()`, and `step()` are introduced.
- **TSDB Performance**: Optimizations in head chunk lookup (constant time) and mmap operations to reduce CPU usage.
- **Service Discovery**: Added support for DigitalOcean Managed Databases and Outscale VM, along with improvements to AWS SD (IPv6 support for EC2, external ID support).
- **UI**: Added a web interface for deleting time series and cleaning tombstones.

#### Changelog

- \[SECURITY] Remote: Reject snappy-compressed received requests via Remote Write whose declared decoded length exceeds the 32MB. Thanks to [@hibrian827](https://github.com/hibrian827) for reporting it. [#18642](prometheus/prometheus#18642)
- \[SECURITY] STACKIT SD: Fix secrets being exposed in plaintext via `/-/config` endpoint. Thanks to [@August829](https://github.com/August829) and [@Phaxma](https://github.com/Phaxma) for reporting. GHSA-39j6-789q-qxvh [#18649](prometheus/prometheus#18649)
- \[CHANGE] TSDB/Agent: Adds Start Timestamp field to all WAL Histogram samples in memory; used `st-storage` flag is enabled. [#18221](prometheus/prometheus#18221)
- \[FEATURE] API: Add `/api/v1/status/self_metrics` endpoint returning the current state of the Prometheus server's own metrics about itself as JSON. [#18411](prometheus/prometheus#18411)
- \[FEATURE] Discovery: Add DigitalOcean Managed Databases service discovery [#18287](prometheus/prometheus#18287)
- \[FEATURE] Prometheus: Add support for the aix/ppc64 compilation target [#18321](prometheus/prometheus#18321)
- \[FEATURE] Discovery: Add Outscale VM service discovery (`outscale_sd_configs`) for discovering scrape targets from the Outscale Cloud API. [#18139](prometheus/prometheus#18139)
- \[FEATURE] PromQL: Emit a warning when `sort`, `sort_by_label` or `sort_by_label_desc` is used within range (matrix) queries, as these functions do not have effect in that context. [#18498](prometheus/prometheus#18498)
- \[FEATURE] PromQL: Add `start()`, `end()`, `range()`, and `step()` experimental functions [#17877](prometheus/prometheus#17877)
- \[FEATURE] PromQL: Update `resets()` function to consider start timestamp resets. Hidden behind `use-start-timestamps` feature flag. [#18627](prometheus/prometheus#18627)
- \[FEATURE] Prometheus: Promote auto-reload-config as stable [#18620](prometheus/prometheus#18620)
- \[FEATURE] TSDB/Agent: Add `CheckpointFromInMemorySeries` option to `agent.DB` that enables checkpoint based on in-memory series. [#17948](prometheus/prometheus#17948)
- \[FEATURE] UI: Add a web interface for deleting time series and cleaning tombstones, accessible from the Status menu. [#18390](prometheus/prometheus#18390)
- \[FEATURE] PromQL: Use start timestamps for `rate()`, `irate()`, and `increase()` calculations, behind a feature flag `use-start-timestamps`. Doesn't work together with extended range selectors `anchored` and `smoothed`. [#18344](prometheus/prometheus#18344)
- \[FEATURE] Scrape: Added a feature flag `st-synthesis` which synthesizes unknown STs for scraped cumulative metrics. Useful when Remote Writing 2.0 with delta or Otel-based backends. [#18279](prometheus/prometheus#18279)
- \[FEATURE] promqltest: support `@st` annotation in `load` blocks to specify per-sample start timestamps. [#18360](prometheus/prometheus#18360)
- \[ENHANCEMENT] API: reject concurrent fgprof profiles. [#18651](prometheus/prometheus#18651)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field to ECS/MSK/RDS/Elasticache. [#18579](prometheus/prometheus#18579)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field. [#17171](prometheus/prometheus#17171)
- \[ENHANCEMENT] Discovery: Propagate SD target updates faster by introducing dynamic backoff interval instead of static 5s interval for throttling. [#18187](prometheus/prometheus#18187)
- \[ENHANCEMENT] Promtool: Add `--header` flag to `query instant` command, matching existing `query range` behaviour. [#18418](prometheus/prometheus#18418)
- \[ENHANCEMENT]: AWS SD: Allows EC2 service discovery to discover IPv6 addresses to communicate with target endpoints. The private IPv4 address remains the default when both IPv4 and IPv6 addresses are present. [#16088](prometheus/prometheus#16088)
- \[PERF] TSDB: Make head chunk lookup in range queries constant time instead of quadratic time [#18302](prometheus/prometheus#18302)
- \[PERF] TSDB: Skip entire stripes in mmapHeadChunks when no series need mmapping, reducing CPU utilization significantly at production-relevant scales. [#18541](prometheus/prometheus#18541)
- \[PERF] TSDB: Skip clean series during periodic head chunk mmap using cached head chunk count [#18272](prometheus/prometheus#18272)
- \[PERF] PromQL: Address FloatHistogram.KahanAdd performance regression on Go 1.26. [#18568](prometheus/prometheus#18568)
- \[BUGFIX] PromQL: Fix `info()` function incorrectly handling negated `__name__` matchers [#17932](prometheus/prometheus#17932)
- \[BUGFIX] API: Return duration expressions in `/parse_ast`. [#18624](prometheus/prometheus#18624)
- \[BUGFIX] API: correctly document formats accepted for duration query request parameters (step, timeout and lookback delta) in OpenAPI spec [#18305](prometheus/prometheus#18305)
- \[BUGFIX] Scrape: AppenderV2 now tracks staleness even when OOO/duplicate series errors happen similar to AppenderV1 [#18567](prometheus/prometheus#18567)
- \[BUGFIX] Config: Validate remote\_write queue\_config fields at load time to prevent runtime panic and silent misconfiguration. [#18209](prometheus/prometheus#18209)
- \[BUGFIX] Discovery/Consul: Add `health_filter` for Health API filtering, fixing breakage when using Catalog-only fields like `ServiceTags` in `filter`. [#18479](prometheus/prometheus#18479) [#18499](prometheus/prometheus#18499)
- \[BUGFIX] OTLP: limit decompressed body size for gzip-encoded OTLP write requests. [#18408](prometheus/prometheus#18408)
- \[BUGFIX] PromQL: Fix `smoothed` rate/increase returning zero instead of no result when all data falls strictly after the query range. [#18523](prometheus/prometheus#18523)
- \[BUGFIX] PromQL: Fix metric name not being dropped when last\_over\_time or first\_over\_time is applied to subqueries containing name-dropping functions like abs(). [#18409](prometheus/prometheus#18409)
- \[BUGFIX] PromQL: Fix missing warning when mixing exponential and custom-bucket histograms in stats queries. [#18660](prometheus/prometheus#18660)
- \[BUGFIX] PromQL: Fix parsing of `range()` keyword in duration expressions such as `foo[5m+range()]`. [#18623](prometheus/prometheus#18623)
- \[BUGFIX] PromQL: Fix smoothed vector selector returning no results in binary operations when the `@` modifier is used. [#18531](prometheus/prometheus#18531)
- \[BUGFIX] PromQL: Reject NaN, infinite, and out-of-range duration expressions instead of silently producing an out-of-range time.Duration. [#18639](prometheus/prometheus#18639)
- \[BUGFIX] Scrape: Fix panic when scraping malformed native histograms. [#18414](prometheus/prometheus#18414)
- \[BUGFIX] Scrape: fix panic when scraping a target exposing a summary with no quantiles via the protobuf format. [#18382](prometheus/prometheus#18382)
- \[BUGFIX] Scrape: fix scrape failure log file occasionally not applied after a configuration reload. [#18421](prometheus/prometheus#18421)
- \[BUGFIX] TSDB: Allow retention percentage with new data path. [#18628](prometheus/prometheus#18628)
- \[BUGFIX] TSDB: Preserve decimal precision in percentage-based retention [#18374](prometheus/prometheus#18374)
- \[BUGFIX] TSDB: fix prometheus\_tsdb\_head\_chunks going negative after WAL replay [#18401](prometheus/prometheus#18401)
- \[BUGFIX] TSDB: panic with native histograms during query of overlapping chunks. [#18692](prometheus/prometheus#18692)
- \[BUGFIX] Tracing: fix startup failure for insecure OTLP HTTP tracing [#18469](prometheus/prometheus#18469)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] UI: Improve Y-axis tick label precision for graph values over small ranges. [#18682](prometheus/prometheus#18682)
- \[BUGFIX] `prometheus_sd_refresh*` and `prometheus_sd_discovered_targets` metrics for specific scrape jobs are deleted when the scrape job is removed. [#17614](prometheus/prometheus#17614)
- \[BUGFIX] Remote: fixed validation for received RW2 requests when parsing metadata unit symbols. This fixes a case when request would cause (recovered) handler panic. [#18641](prometheus/prometheus#18641)
- \[BUGFIX] TSDB/Agent: fix race in agent appender where concurrent appends for the same label set could produce duplicate in-memory series and duplicate WAL records. [#18292](prometheus/prometheus#18292)
- \[BUGFIX] Config: Update `--enable-feature`  flag description and sort feature names. [#18487](prometheus/prometheus#18487)
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jun 6, 2026
##### [\`v3.12.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.12.0)

This release contains security fixes, new features (especially around PromQL and Service Discovery), performance improvements in TSDB, Start Timestamp improvements and numerous bug fixes.

Thanks to all contributors!

#### Key Highlights

- **Security**: Two security vulnerabilities have been addressed: a denial of service in remote-write (snappy decompression limit) and a secret exposure leak in STACKIT service discovery.
- **PromQL & Metadata**: Several features and bug fixes related to the experimental "start timestamps" support, including updates to `rate()`, `irate()`, `increase()`, and `resets()`. New experimental functions `start()`, `end()`, `range()`, and `step()` are introduced.
- **TSDB Performance**: Optimizations in head chunk lookup (constant time) and mmap operations to reduce CPU usage.
- **Service Discovery**: Added support for DigitalOcean Managed Databases and Outscale VM, along with improvements to AWS SD (IPv6 support for EC2, external ID support).
- **UI**: Added a web interface for deleting time series and cleaning tombstones.

#### Changelog

- \[SECURITY] Remote: Reject snappy-compressed received requests via Remote Write whose declared decoded length exceeds the 32MB. Thanks to [@hibrian827](https://github.com/hibrian827) for reporting it. [#18642](prometheus/prometheus#18642)
- \[SECURITY] STACKIT SD: Fix secrets being exposed in plaintext via `/-/config` endpoint. Thanks to [@August829](https://github.com/August829) and [@Phaxma](https://github.com/Phaxma) for reporting. GHSA-39j6-789q-qxvh [#18649](prometheus/prometheus#18649)
- \[CHANGE] TSDB/Agent: Adds Start Timestamp field to all WAL Histogram samples in memory; used `st-storage` flag is enabled. [#18221](prometheus/prometheus#18221)
- \[FEATURE] API: Add `/api/v1/status/self_metrics` endpoint returning the current state of the Prometheus server's own metrics about itself as JSON. [#18411](prometheus/prometheus#18411)
- \[FEATURE] Discovery: Add DigitalOcean Managed Databases service discovery [#18287](prometheus/prometheus#18287)
- \[FEATURE] Prometheus: Add support for the aix/ppc64 compilation target [#18321](prometheus/prometheus#18321)
- \[FEATURE] Discovery: Add Outscale VM service discovery (`outscale_sd_configs`) for discovering scrape targets from the Outscale Cloud API. [#18139](prometheus/prometheus#18139)
- \[FEATURE] PromQL: Emit a warning when `sort`, `sort_by_label` or `sort_by_label_desc` is used within range (matrix) queries, as these functions do not have effect in that context. [#18498](prometheus/prometheus#18498)
- \[FEATURE] PromQL: Add `start()`, `end()`, `range()`, and `step()` experimental functions [#17877](prometheus/prometheus#17877)
- \[FEATURE] PromQL: Update `resets()` function to consider start timestamp resets. Hidden behind `use-start-timestamps` feature flag. [#18627](prometheus/prometheus#18627)
- \[FEATURE] Prometheus: Promote auto-reload-config as stable [#18620](prometheus/prometheus#18620)
- \[FEATURE] TSDB/Agent: Add `CheckpointFromInMemorySeries` option to `agent.DB` that enables checkpoint based on in-memory series. [#17948](prometheus/prometheus#17948)
- \[FEATURE] UI: Add a web interface for deleting time series and cleaning tombstones, accessible from the Status menu. [#18390](prometheus/prometheus#18390)
- \[FEATURE] PromQL: Use start timestamps for `rate()`, `irate()`, and `increase()` calculations, behind a feature flag `use-start-timestamps`. Doesn't work together with extended range selectors `anchored` and `smoothed`. [#18344](prometheus/prometheus#18344)
- \[FEATURE] Scrape: Added a feature flag `st-synthesis` which synthesizes unknown STs for scraped cumulative metrics. Useful when Remote Writing 2.0 with delta or Otel-based backends. [#18279](prometheus/prometheus#18279)
- \[FEATURE] promqltest: support `@st` annotation in `load` blocks to specify per-sample start timestamps. [#18360](prometheus/prometheus#18360)
- \[ENHANCEMENT] API: reject concurrent fgprof profiles. [#18651](prometheus/prometheus#18651)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field to ECS/MSK/RDS/Elasticache. [#18579](prometheus/prometheus#18579)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field. [#17171](prometheus/prometheus#17171)
- \[ENHANCEMENT] Discovery: Propagate SD target updates faster by introducing dynamic backoff interval instead of static 5s interval for throttling. [#18187](prometheus/prometheus#18187)
- \[ENHANCEMENT] Promtool: Add `--header` flag to `query instant` command, matching existing `query range` behaviour. [#18418](prometheus/prometheus#18418)
- \[ENHANCEMENT]: AWS SD: Allows EC2 service discovery to discover IPv6 addresses to communicate with target endpoints. The private IPv4 address remains the default when both IPv4 and IPv6 addresses are present. [#16088](prometheus/prometheus#16088)
- \[PERF] TSDB: Make head chunk lookup in range queries constant time instead of quadratic time [#18302](prometheus/prometheus#18302)
- \[PERF] TSDB: Skip entire stripes in mmapHeadChunks when no series need mmapping, reducing CPU utilization significantly at production-relevant scales. [#18541](prometheus/prometheus#18541)
- \[PERF] TSDB: Skip clean series during periodic head chunk mmap using cached head chunk count [#18272](prometheus/prometheus#18272)
- \[PERF] PromQL: Address FloatHistogram.KahanAdd performance regression on Go 1.26. [#18568](prometheus/prometheus#18568)
- \[BUGFIX] PromQL: Fix `info()` function incorrectly handling negated `__name__` matchers [#17932](prometheus/prometheus#17932)
- \[BUGFIX] API: Return duration expressions in `/parse_ast`. [#18624](prometheus/prometheus#18624)
- \[BUGFIX] API: correctly document formats accepted for duration query request parameters (step, timeout and lookback delta) in OpenAPI spec [#18305](prometheus/prometheus#18305)
- \[BUGFIX] Scrape: AppenderV2 now tracks staleness even when OOO/duplicate series errors happen similar to AppenderV1 [#18567](prometheus/prometheus#18567)
- \[BUGFIX] Config: Validate remote\_write queue\_config fields at load time to prevent runtime panic and silent misconfiguration. [#18209](prometheus/prometheus#18209)
- \[BUGFIX] Discovery/Consul: Add `health_filter` for Health API filtering, fixing breakage when using Catalog-only fields like `ServiceTags` in `filter`. [#18479](prometheus/prometheus#18479) [#18499](prometheus/prometheus#18499)
- \[BUGFIX] OTLP: limit decompressed body size for gzip-encoded OTLP write requests. [#18408](prometheus/prometheus#18408)
- \[BUGFIX] PromQL: Fix `smoothed` rate/increase returning zero instead of no result when all data falls strictly after the query range. [#18523](prometheus/prometheus#18523)
- \[BUGFIX] PromQL: Fix metric name not being dropped when last\_over\_time or first\_over\_time is applied to subqueries containing name-dropping functions like abs(). [#18409](prometheus/prometheus#18409)
- \[BUGFIX] PromQL: Fix missing warning when mixing exponential and custom-bucket histograms in stats queries. [#18660](prometheus/prometheus#18660)
- \[BUGFIX] PromQL: Fix parsing of `range()` keyword in duration expressions such as `foo[5m+range()]`. [#18623](prometheus/prometheus#18623)
- \[BUGFIX] PromQL: Fix smoothed vector selector returning no results in binary operations when the `@` modifier is used. [#18531](prometheus/prometheus#18531)
- \[BUGFIX] PromQL: Reject NaN, infinite, and out-of-range duration expressions instead of silently producing an out-of-range time.Duration. [#18639](prometheus/prometheus#18639)
- \[BUGFIX] Scrape: Fix panic when scraping malformed native histograms. [#18414](prometheus/prometheus#18414)
- \[BUGFIX] Scrape: fix panic when scraping a target exposing a summary with no quantiles via the protobuf format. [#18382](prometheus/prometheus#18382)
- \[BUGFIX] Scrape: fix scrape failure log file occasionally not applied after a configuration reload. [#18421](prometheus/prometheus#18421)
- \[BUGFIX] TSDB: Allow retention percentage with new data path. [#18628](prometheus/prometheus#18628)
- \[BUGFIX] TSDB: Preserve decimal precision in percentage-based retention [#18374](prometheus/prometheus#18374)
- \[BUGFIX] TSDB: fix prometheus\_tsdb\_head\_chunks going negative after WAL replay [#18401](prometheus/prometheus#18401)
- \[BUGFIX] TSDB: panic with native histograms during query of overlapping chunks. [#18692](prometheus/prometheus#18692)
- \[BUGFIX] Tracing: fix startup failure for insecure OTLP HTTP tracing [#18469](prometheus/prometheus#18469)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] UI: Improve Y-axis tick label precision for graph values over small ranges. [#18682](prometheus/prometheus#18682)
- \[BUGFIX] `prometheus_sd_refresh*` and `prometheus_sd_discovered_targets` metrics for specific scrape jobs are deleted when the scrape job is removed. [#17614](prometheus/prometheus#17614)
- \[BUGFIX] Remote: fixed validation for received RW2 requests when parsing metadata unit symbols. This fixes a case when request would cause (recovered) handler panic. [#18641](prometheus/prometheus#18641)
- \[BUGFIX] TSDB/Agent: fix race in agent appender where concurrent appends for the same label set could produce duplicate in-memory series and duplicate WAL records. [#18292](prometheus/prometheus#18292)
- \[BUGFIX] Config: Update `--enable-feature`  flag description and sort feature names. [#18487](prometheus/prometheus#18487)
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jun 6, 2026
##### [\`v3.12.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.12.0)

This release contains security fixes, new features (especially around PromQL and Service Discovery), performance improvements in TSDB, Start Timestamp improvements and numerous bug fixes.

Thanks to all contributors!

#### Key Highlights

- **Security**: Two security vulnerabilities have been addressed: a denial of service in remote-write (snappy decompression limit) and a secret exposure leak in STACKIT service discovery.
- **PromQL & Metadata**: Several features and bug fixes related to the experimental "start timestamps" support, including updates to `rate()`, `irate()`, `increase()`, and `resets()`. New experimental functions `start()`, `end()`, `range()`, and `step()` are introduced.
- **TSDB Performance**: Optimizations in head chunk lookup (constant time) and mmap operations to reduce CPU usage.
- **Service Discovery**: Added support for DigitalOcean Managed Databases and Outscale VM, along with improvements to AWS SD (IPv6 support for EC2, external ID support).
- **UI**: Added a web interface for deleting time series and cleaning tombstones.

#### Changelog

- \[SECURITY] Remote: Reject snappy-compressed received requests via Remote Write whose declared decoded length exceeds the 32MB. Thanks to [@hibrian827](https://github.com/hibrian827) for reporting it. [#18642](prometheus/prometheus#18642)
- \[SECURITY] STACKIT SD: Fix secrets being exposed in plaintext via `/-/config` endpoint. Thanks to [@August829](https://github.com/August829) and [@Phaxma](https://github.com/Phaxma) for reporting. GHSA-39j6-789q-qxvh [#18649](prometheus/prometheus#18649)
- \[CHANGE] TSDB/Agent: Adds Start Timestamp field to all WAL Histogram samples in memory; used `st-storage` flag is enabled. [#18221](prometheus/prometheus#18221)
- \[FEATURE] API: Add `/api/v1/status/self_metrics` endpoint returning the current state of the Prometheus server's own metrics about itself as JSON. [#18411](prometheus/prometheus#18411)
- \[FEATURE] Discovery: Add DigitalOcean Managed Databases service discovery [#18287](prometheus/prometheus#18287)
- \[FEATURE] Prometheus: Add support for the aix/ppc64 compilation target [#18321](prometheus/prometheus#18321)
- \[FEATURE] Discovery: Add Outscale VM service discovery (`outscale_sd_configs`) for discovering scrape targets from the Outscale Cloud API. [#18139](prometheus/prometheus#18139)
- \[FEATURE] PromQL: Emit a warning when `sort`, `sort_by_label` or `sort_by_label_desc` is used within range (matrix) queries, as these functions do not have effect in that context. [#18498](prometheus/prometheus#18498)
- \[FEATURE] PromQL: Add `start()`, `end()`, `range()`, and `step()` experimental functions [#17877](prometheus/prometheus#17877)
- \[FEATURE] PromQL: Update `resets()` function to consider start timestamp resets. Hidden behind `use-start-timestamps` feature flag. [#18627](prometheus/prometheus#18627)
- \[FEATURE] Prometheus: Promote auto-reload-config as stable [#18620](prometheus/prometheus#18620)
- \[FEATURE] TSDB/Agent: Add `CheckpointFromInMemorySeries` option to `agent.DB` that enables checkpoint based on in-memory series. [#17948](prometheus/prometheus#17948)
- \[FEATURE] UI: Add a web interface for deleting time series and cleaning tombstones, accessible from the Status menu. [#18390](prometheus/prometheus#18390)
- \[FEATURE] PromQL: Use start timestamps for `rate()`, `irate()`, and `increase()` calculations, behind a feature flag `use-start-timestamps`. Doesn't work together with extended range selectors `anchored` and `smoothed`. [#18344](prometheus/prometheus#18344)
- \[FEATURE] Scrape: Added a feature flag `st-synthesis` which synthesizes unknown STs for scraped cumulative metrics. Useful when Remote Writing 2.0 with delta or Otel-based backends. [#18279](prometheus/prometheus#18279)
- \[FEATURE] promqltest: support `@st` annotation in `load` blocks to specify per-sample start timestamps. [#18360](prometheus/prometheus#18360)
- \[ENHANCEMENT] API: reject concurrent fgprof profiles. [#18651](prometheus/prometheus#18651)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field to ECS/MSK/RDS/Elasticache. [#18579](prometheus/prometheus#18579)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field. [#17171](prometheus/prometheus#17171)
- \[ENHANCEMENT] Discovery: Propagate SD target updates faster by introducing dynamic backoff interval instead of static 5s interval for throttling. [#18187](prometheus/prometheus#18187)
- \[ENHANCEMENT] Promtool: Add `--header` flag to `query instant` command, matching existing `query range` behaviour. [#18418](prometheus/prometheus#18418)
- \[ENHANCEMENT]: AWS SD: Allows EC2 service discovery to discover IPv6 addresses to communicate with target endpoints. The private IPv4 address remains the default when both IPv4 and IPv6 addresses are present. [#16088](prometheus/prometheus#16088)
- \[PERF] TSDB: Make head chunk lookup in range queries constant time instead of quadratic time [#18302](prometheus/prometheus#18302)
- \[PERF] TSDB: Skip entire stripes in mmapHeadChunks when no series need mmapping, reducing CPU utilization significantly at production-relevant scales. [#18541](prometheus/prometheus#18541)
- \[PERF] TSDB: Skip clean series during periodic head chunk mmap using cached head chunk count [#18272](prometheus/prometheus#18272)
- \[PERF] PromQL: Address FloatHistogram.KahanAdd performance regression on Go 1.26. [#18568](prometheus/prometheus#18568)
- \[BUGFIX] PromQL: Fix `info()` function incorrectly handling negated `__name__` matchers [#17932](prometheus/prometheus#17932)
- \[BUGFIX] API: Return duration expressions in `/parse_ast`. [#18624](prometheus/prometheus#18624)
- \[BUGFIX] API: correctly document formats accepted for duration query request parameters (step, timeout and lookback delta) in OpenAPI spec [#18305](prometheus/prometheus#18305)
- \[BUGFIX] Scrape: AppenderV2 now tracks staleness even when OOO/duplicate series errors happen similar to AppenderV1 [#18567](prometheus/prometheus#18567)
- \[BUGFIX] Config: Validate remote\_write queue\_config fields at load time to prevent runtime panic and silent misconfiguration. [#18209](prometheus/prometheus#18209)
- \[BUGFIX] Discovery/Consul: Add `health_filter` for Health API filtering, fixing breakage when using Catalog-only fields like `ServiceTags` in `filter`. [#18479](prometheus/prometheus#18479) [#18499](prometheus/prometheus#18499)
- \[BUGFIX] OTLP: limit decompressed body size for gzip-encoded OTLP write requests. [#18408](prometheus/prometheus#18408)
- \[BUGFIX] PromQL: Fix `smoothed` rate/increase returning zero instead of no result when all data falls strictly after the query range. [#18523](prometheus/prometheus#18523)
- \[BUGFIX] PromQL: Fix metric name not being dropped when last\_over\_time or first\_over\_time is applied to subqueries containing name-dropping functions like abs(). [#18409](prometheus/prometheus#18409)
- \[BUGFIX] PromQL: Fix missing warning when mixing exponential and custom-bucket histograms in stats queries. [#18660](prometheus/prometheus#18660)
- \[BUGFIX] PromQL: Fix parsing of `range()` keyword in duration expressions such as `foo[5m+range()]`. [#18623](prometheus/prometheus#18623)
- \[BUGFIX] PromQL: Fix smoothed vector selector returning no results in binary operations when the `@` modifier is used. [#18531](prometheus/prometheus#18531)
- \[BUGFIX] PromQL: Reject NaN, infinite, and out-of-range duration expressions instead of silently producing an out-of-range time.Duration. [#18639](prometheus/prometheus#18639)
- \[BUGFIX] Scrape: Fix panic when scraping malformed native histograms. [#18414](prometheus/prometheus#18414)
- \[BUGFIX] Scrape: fix panic when scraping a target exposing a summary with no quantiles via the protobuf format. [#18382](prometheus/prometheus#18382)
- \[BUGFIX] Scrape: fix scrape failure log file occasionally not applied after a configuration reload. [#18421](prometheus/prometheus#18421)
- \[BUGFIX] TSDB: Allow retention percentage with new data path. [#18628](prometheus/prometheus#18628)
- \[BUGFIX] TSDB: Preserve decimal precision in percentage-based retention [#18374](prometheus/prometheus#18374)
- \[BUGFIX] TSDB: fix prometheus\_tsdb\_head\_chunks going negative after WAL replay [#18401](prometheus/prometheus#18401)
- \[BUGFIX] TSDB: panic with native histograms during query of overlapping chunks. [#18692](prometheus/prometheus#18692)
- \[BUGFIX] Tracing: fix startup failure for insecure OTLP HTTP tracing [#18469](prometheus/prometheus#18469)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] UI: Improve Y-axis tick label precision for graph values over small ranges. [#18682](prometheus/prometheus#18682)
- \[BUGFIX] `prometheus_sd_refresh*` and `prometheus_sd_discovered_targets` metrics for specific scrape jobs are deleted when the scrape job is removed. [#17614](prometheus/prometheus#17614)
- \[BUGFIX] Remote: fixed validation for received RW2 requests when parsing metadata unit symbols. This fixes a case when request would cause (recovered) handler panic. [#18641](prometheus/prometheus#18641)
- \[BUGFIX] TSDB/Agent: fix race in agent appender where concurrent appends for the same label set could produce duplicate in-memory series and duplicate WAL records. [#18292](prometheus/prometheus#18292)
- \[BUGFIX] Config: Update `--enable-feature`  flag description and sort feature names. [#18487](prometheus/prometheus#18487)
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jun 6, 2026
##### [\`v3.12.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.12.0)

This release contains security fixes, new features (especially around PromQL and Service Discovery), performance improvements in TSDB, Start Timestamp improvements and numerous bug fixes.

Thanks to all contributors!

#### Key Highlights

- **Security**: Two security vulnerabilities have been addressed: a denial of service in remote-write (snappy decompression limit) and a secret exposure leak in STACKIT service discovery.
- **PromQL & Metadata**: Several features and bug fixes related to the experimental "start timestamps" support, including updates to `rate()`, `irate()`, `increase()`, and `resets()`. New experimental functions `start()`, `end()`, `range()`, and `step()` are introduced.
- **TSDB Performance**: Optimizations in head chunk lookup (constant time) and mmap operations to reduce CPU usage.
- **Service Discovery**: Added support for DigitalOcean Managed Databases and Outscale VM, along with improvements to AWS SD (IPv6 support for EC2, external ID support).
- **UI**: Added a web interface for deleting time series and cleaning tombstones.

#### Changelog

- \[SECURITY] Remote: Reject snappy-compressed received requests via Remote Write whose declared decoded length exceeds the 32MB. Thanks to [@hibrian827](https://github.com/hibrian827) for reporting it. [#18642](prometheus/prometheus#18642)
- \[SECURITY] STACKIT SD: Fix secrets being exposed in plaintext via `/-/config` endpoint. Thanks to [@August829](https://github.com/August829) and [@Phaxma](https://github.com/Phaxma) for reporting. GHSA-39j6-789q-qxvh [#18649](prometheus/prometheus#18649)
- \[CHANGE] TSDB/Agent: Adds Start Timestamp field to all WAL Histogram samples in memory; used `st-storage` flag is enabled. [#18221](prometheus/prometheus#18221)
- \[FEATURE] API: Add `/api/v1/status/self_metrics` endpoint returning the current state of the Prometheus server's own metrics about itself as JSON. [#18411](prometheus/prometheus#18411)
- \[FEATURE] Discovery: Add DigitalOcean Managed Databases service discovery [#18287](prometheus/prometheus#18287)
- \[FEATURE] Prometheus: Add support for the aix/ppc64 compilation target [#18321](prometheus/prometheus#18321)
- \[FEATURE] Discovery: Add Outscale VM service discovery (`outscale_sd_configs`) for discovering scrape targets from the Outscale Cloud API. [#18139](prometheus/prometheus#18139)
- \[FEATURE] PromQL: Emit a warning when `sort`, `sort_by_label` or `sort_by_label_desc` is used within range (matrix) queries, as these functions do not have effect in that context. [#18498](prometheus/prometheus#18498)
- \[FEATURE] PromQL: Add `start()`, `end()`, `range()`, and `step()` experimental functions [#17877](prometheus/prometheus#17877)
- \[FEATURE] PromQL: Update `resets()` function to consider start timestamp resets. Hidden behind `use-start-timestamps` feature flag. [#18627](prometheus/prometheus#18627)
- \[FEATURE] Prometheus: Promote auto-reload-config as stable [#18620](prometheus/prometheus#18620)
- \[FEATURE] TSDB/Agent: Add `CheckpointFromInMemorySeries` option to `agent.DB` that enables checkpoint based on in-memory series. [#17948](prometheus/prometheus#17948)
- \[FEATURE] UI: Add a web interface for deleting time series and cleaning tombstones, accessible from the Status menu. [#18390](prometheus/prometheus#18390)
- \[FEATURE] PromQL: Use start timestamps for `rate()`, `irate()`, and `increase()` calculations, behind a feature flag `use-start-timestamps`. Doesn't work together with extended range selectors `anchored` and `smoothed`. [#18344](prometheus/prometheus#18344)
- \[FEATURE] Scrape: Added a feature flag `st-synthesis` which synthesizes unknown STs for scraped cumulative metrics. Useful when Remote Writing 2.0 with delta or Otel-based backends. [#18279](prometheus/prometheus#18279)
- \[FEATURE] promqltest: support `@st` annotation in `load` blocks to specify per-sample start timestamps. [#18360](prometheus/prometheus#18360)
- \[ENHANCEMENT] API: reject concurrent fgprof profiles. [#18651](prometheus/prometheus#18651)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field to ECS/MSK/RDS/Elasticache. [#18579](prometheus/prometheus#18579)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field. [#17171](prometheus/prometheus#17171)
- \[ENHANCEMENT] Discovery: Propagate SD target updates faster by introducing dynamic backoff interval instead of static 5s interval for throttling. [#18187](prometheus/prometheus#18187)
- \[ENHANCEMENT] Promtool: Add `--header` flag to `query instant` command, matching existing `query range` behaviour. [#18418](prometheus/prometheus#18418)
- \[ENHANCEMENT]: AWS SD: Allows EC2 service discovery to discover IPv6 addresses to communicate with target endpoints. The private IPv4 address remains the default when both IPv4 and IPv6 addresses are present. [#16088](prometheus/prometheus#16088)
- \[PERF] TSDB: Make head chunk lookup in range queries constant time instead of quadratic time [#18302](prometheus/prometheus#18302)
- \[PERF] TSDB: Skip entire stripes in mmapHeadChunks when no series need mmapping, reducing CPU utilization significantly at production-relevant scales. [#18541](prometheus/prometheus#18541)
- \[PERF] TSDB: Skip clean series during periodic head chunk mmap using cached head chunk count [#18272](prometheus/prometheus#18272)
- \[PERF] PromQL: Address FloatHistogram.KahanAdd performance regression on Go 1.26. [#18568](prometheus/prometheus#18568)
- \[BUGFIX] PromQL: Fix `info()` function incorrectly handling negated `__name__` matchers [#17932](prometheus/prometheus#17932)
- \[BUGFIX] API: Return duration expressions in `/parse_ast`. [#18624](prometheus/prometheus#18624)
- \[BUGFIX] API: correctly document formats accepted for duration query request parameters (step, timeout and lookback delta) in OpenAPI spec [#18305](prometheus/prometheus#18305)
- \[BUGFIX] Scrape: AppenderV2 now tracks staleness even when OOO/duplicate series errors happen similar to AppenderV1 [#18567](prometheus/prometheus#18567)
- \[BUGFIX] Config: Validate remote\_write queue\_config fields at load time to prevent runtime panic and silent misconfiguration. [#18209](prometheus/prometheus#18209)
- \[BUGFIX] Discovery/Consul: Add `health_filter` for Health API filtering, fixing breakage when using Catalog-only fields like `ServiceTags` in `filter`. [#18479](prometheus/prometheus#18479) [#18499](prometheus/prometheus#18499)
- \[BUGFIX] OTLP: limit decompressed body size for gzip-encoded OTLP write requests. [#18408](prometheus/prometheus#18408)
- \[BUGFIX] PromQL: Fix `smoothed` rate/increase returning zero instead of no result when all data falls strictly after the query range. [#18523](prometheus/prometheus#18523)
- \[BUGFIX] PromQL: Fix metric name not being dropped when last\_over\_time or first\_over\_time is applied to subqueries containing name-dropping functions like abs(). [#18409](prometheus/prometheus#18409)
- \[BUGFIX] PromQL: Fix missing warning when mixing exponential and custom-bucket histograms in stats queries. [#18660](prometheus/prometheus#18660)
- \[BUGFIX] PromQL: Fix parsing of `range()` keyword in duration expressions such as `foo[5m+range()]`. [#18623](prometheus/prometheus#18623)
- \[BUGFIX] PromQL: Fix smoothed vector selector returning no results in binary operations when the `@` modifier is used. [#18531](prometheus/prometheus#18531)
- \[BUGFIX] PromQL: Reject NaN, infinite, and out-of-range duration expressions instead of silently producing an out-of-range time.Duration. [#18639](prometheus/prometheus#18639)
- \[BUGFIX] Scrape: Fix panic when scraping malformed native histograms. [#18414](prometheus/prometheus#18414)
- \[BUGFIX] Scrape: fix panic when scraping a target exposing a summary with no quantiles via the protobuf format. [#18382](prometheus/prometheus#18382)
- \[BUGFIX] Scrape: fix scrape failure log file occasionally not applied after a configuration reload. [#18421](prometheus/prometheus#18421)
- \[BUGFIX] TSDB: Allow retention percentage with new data path. [#18628](prometheus/prometheus#18628)
- \[BUGFIX] TSDB: Preserve decimal precision in percentage-based retention [#18374](prometheus/prometheus#18374)
- \[BUGFIX] TSDB: fix prometheus\_tsdb\_head\_chunks going negative after WAL replay [#18401](prometheus/prometheus#18401)
- \[BUGFIX] TSDB: panic with native histograms during query of overlapping chunks. [#18692](prometheus/prometheus#18692)
- \[BUGFIX] Tracing: fix startup failure for insecure OTLP HTTP tracing [#18469](prometheus/prometheus#18469)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] UI: Improve Y-axis tick label precision for graph values over small ranges. [#18682](prometheus/prometheus#18682)
- \[BUGFIX] `prometheus_sd_refresh*` and `prometheus_sd_discovered_targets` metrics for specific scrape jobs are deleted when the scrape job is removed. [#17614](prometheus/prometheus#17614)
- \[BUGFIX] Remote: fixed validation for received RW2 requests when parsing metadata unit symbols. This fixes a case when request would cause (recovered) handler panic. [#18641](prometheus/prometheus#18641)
- \[BUGFIX] TSDB/Agent: fix race in agent appender where concurrent appends for the same label set could produce duplicate in-memory series and duplicate WAL records. [#18292](prometheus/prometheus#18292)
- \[BUGFIX] Config: Update `--enable-feature`  flag description and sort feature names. [#18487](prometheus/prometheus#18487)
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jun 6, 2026
##### [\`v3.12.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.12.0)

This release contains security fixes, new features (especially around PromQL and Service Discovery), performance improvements in TSDB, Start Timestamp improvements and numerous bug fixes.

Thanks to all contributors!

#### Key Highlights

- **Security**: Two security vulnerabilities have been addressed: a denial of service in remote-write (snappy decompression limit) and a secret exposure leak in STACKIT service discovery.
- **PromQL & Metadata**: Several features and bug fixes related to the experimental "start timestamps" support, including updates to `rate()`, `irate()`, `increase()`, and `resets()`. New experimental functions `start()`, `end()`, `range()`, and `step()` are introduced.
- **TSDB Performance**: Optimizations in head chunk lookup (constant time) and mmap operations to reduce CPU usage.
- **Service Discovery**: Added support for DigitalOcean Managed Databases and Outscale VM, along with improvements to AWS SD (IPv6 support for EC2, external ID support).
- **UI**: Added a web interface for deleting time series and cleaning tombstones.

#### Changelog

- \[SECURITY] Remote: Reject snappy-compressed received requests via Remote Write whose declared decoded length exceeds the 32MB. Thanks to [@hibrian827](https://github.com/hibrian827) for reporting it. [#18642](prometheus/prometheus#18642)
- \[SECURITY] STACKIT SD: Fix secrets being exposed in plaintext via `/-/config` endpoint. Thanks to [@August829](https://github.com/August829) and [@Phaxma](https://github.com/Phaxma) for reporting. GHSA-39j6-789q-qxvh [#18649](prometheus/prometheus#18649)
- \[CHANGE] TSDB/Agent: Adds Start Timestamp field to all WAL Histogram samples in memory; used `st-storage` flag is enabled. [#18221](prometheus/prometheus#18221)
- \[FEATURE] API: Add `/api/v1/status/self_metrics` endpoint returning the current state of the Prometheus server's own metrics about itself as JSON. [#18411](prometheus/prometheus#18411)
- \[FEATURE] Discovery: Add DigitalOcean Managed Databases service discovery [#18287](prometheus/prometheus#18287)
- \[FEATURE] Prometheus: Add support for the aix/ppc64 compilation target [#18321](prometheus/prometheus#18321)
- \[FEATURE] Discovery: Add Outscale VM service discovery (`outscale_sd_configs`) for discovering scrape targets from the Outscale Cloud API. [#18139](prometheus/prometheus#18139)
- \[FEATURE] PromQL: Emit a warning when `sort`, `sort_by_label` or `sort_by_label_desc` is used within range (matrix) queries, as these functions do not have effect in that context. [#18498](prometheus/prometheus#18498)
- \[FEATURE] PromQL: Add `start()`, `end()`, `range()`, and `step()` experimental functions [#17877](prometheus/prometheus#17877)
- \[FEATURE] PromQL: Update `resets()` function to consider start timestamp resets. Hidden behind `use-start-timestamps` feature flag. [#18627](prometheus/prometheus#18627)
- \[FEATURE] Prometheus: Promote auto-reload-config as stable [#18620](prometheus/prometheus#18620)
- \[FEATURE] TSDB/Agent: Add `CheckpointFromInMemorySeries` option to `agent.DB` that enables checkpoint based on in-memory series. [#17948](prometheus/prometheus#17948)
- \[FEATURE] UI: Add a web interface for deleting time series and cleaning tombstones, accessible from the Status menu. [#18390](prometheus/prometheus#18390)
- \[FEATURE] PromQL: Use start timestamps for `rate()`, `irate()`, and `increase()` calculations, behind a feature flag `use-start-timestamps`. Doesn't work together with extended range selectors `anchored` and `smoothed`. [#18344](prometheus/prometheus#18344)
- \[FEATURE] Scrape: Added a feature flag `st-synthesis` which synthesizes unknown STs for scraped cumulative metrics. Useful when Remote Writing 2.0 with delta or Otel-based backends. [#18279](prometheus/prometheus#18279)
- \[FEATURE] promqltest: support `@st` annotation in `load` blocks to specify per-sample start timestamps. [#18360](prometheus/prometheus#18360)
- \[ENHANCEMENT] API: reject concurrent fgprof profiles. [#18651](prometheus/prometheus#18651)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field to ECS/MSK/RDS/Elasticache. [#18579](prometheus/prometheus#18579)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field. [#17171](prometheus/prometheus#17171)
- \[ENHANCEMENT] Discovery: Propagate SD target updates faster by introducing dynamic backoff interval instead of static 5s interval for throttling. [#18187](prometheus/prometheus#18187)
- \[ENHANCEMENT] Promtool: Add `--header` flag to `query instant` command, matching existing `query range` behaviour. [#18418](prometheus/prometheus#18418)
- \[ENHANCEMENT]: AWS SD: Allows EC2 service discovery to discover IPv6 addresses to communicate with target endpoints. The private IPv4 address remains the default when both IPv4 and IPv6 addresses are present. [#16088](prometheus/prometheus#16088)
- \[PERF] TSDB: Make head chunk lookup in range queries constant time instead of quadratic time [#18302](prometheus/prometheus#18302)
- \[PERF] TSDB: Skip entire stripes in mmapHeadChunks when no series need mmapping, reducing CPU utilization significantly at production-relevant scales. [#18541](prometheus/prometheus#18541)
- \[PERF] TSDB: Skip clean series during periodic head chunk mmap using cached head chunk count [#18272](prometheus/prometheus#18272)
- \[PERF] PromQL: Address FloatHistogram.KahanAdd performance regression on Go 1.26. [#18568](prometheus/prometheus#18568)
- \[BUGFIX] PromQL: Fix `info()` function incorrectly handling negated `__name__` matchers [#17932](prometheus/prometheus#17932)
- \[BUGFIX] API: Return duration expressions in `/parse_ast`. [#18624](prometheus/prometheus#18624)
- \[BUGFIX] API: correctly document formats accepted for duration query request parameters (step, timeout and lookback delta) in OpenAPI spec [#18305](prometheus/prometheus#18305)
- \[BUGFIX] Scrape: AppenderV2 now tracks staleness even when OOO/duplicate series errors happen similar to AppenderV1 [#18567](prometheus/prometheus#18567)
- \[BUGFIX] Config: Validate remote\_write queue\_config fields at load time to prevent runtime panic and silent misconfiguration. [#18209](prometheus/prometheus#18209)
- \[BUGFIX] Discovery/Consul: Add `health_filter` for Health API filtering, fixing breakage when using Catalog-only fields like `ServiceTags` in `filter`. [#18479](prometheus/prometheus#18479) [#18499](prometheus/prometheus#18499)
- \[BUGFIX] OTLP: limit decompressed body size for gzip-encoded OTLP write requests. [#18408](prometheus/prometheus#18408)
- \[BUGFIX] PromQL: Fix `smoothed` rate/increase returning zero instead of no result when all data falls strictly after the query range. [#18523](prometheus/prometheus#18523)
- \[BUGFIX] PromQL: Fix metric name not being dropped when last\_over\_time or first\_over\_time is applied to subqueries containing name-dropping functions like abs(). [#18409](prometheus/prometheus#18409)
- \[BUGFIX] PromQL: Fix missing warning when mixing exponential and custom-bucket histograms in stats queries. [#18660](prometheus/prometheus#18660)
- \[BUGFIX] PromQL: Fix parsing of `range()` keyword in duration expressions such as `foo[5m+range()]`. [#18623](prometheus/prometheus#18623)
- \[BUGFIX] PromQL: Fix smoothed vector selector returning no results in binary operations when the `@` modifier is used. [#18531](prometheus/prometheus#18531)
- \[BUGFIX] PromQL: Reject NaN, infinite, and out-of-range duration expressions instead of silently producing an out-of-range time.Duration. [#18639](prometheus/prometheus#18639)
- \[BUGFIX] Scrape: Fix panic when scraping malformed native histograms. [#18414](prometheus/prometheus#18414)
- \[BUGFIX] Scrape: fix panic when scraping a target exposing a summary with no quantiles via the protobuf format. [#18382](prometheus/prometheus#18382)
- \[BUGFIX] Scrape: fix scrape failure log file occasionally not applied after a configuration reload. [#18421](prometheus/prometheus#18421)
- \[BUGFIX] TSDB: Allow retention percentage with new data path. [#18628](prometheus/prometheus#18628)
- \[BUGFIX] TSDB: Preserve decimal precision in percentage-based retention [#18374](prometheus/prometheus#18374)
- \[BUGFIX] TSDB: fix prometheus\_tsdb\_head\_chunks going negative after WAL replay [#18401](prometheus/prometheus#18401)
- \[BUGFIX] TSDB: panic with native histograms during query of overlapping chunks. [#18692](prometheus/prometheus#18692)
- \[BUGFIX] Tracing: fix startup failure for insecure OTLP HTTP tracing [#18469](prometheus/prometheus#18469)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] UI: Improve Y-axis tick label precision for graph values over small ranges. [#18682](prometheus/prometheus#18682)
- \[BUGFIX] `prometheus_sd_refresh*` and `prometheus_sd_discovered_targets` metrics for specific scrape jobs are deleted when the scrape job is removed. [#17614](prometheus/prometheus#17614)
- \[BUGFIX] Remote: fixed validation for received RW2 requests when parsing metadata unit symbols. This fixes a case when request would cause (recovered) handler panic. [#18641](prometheus/prometheus#18641)
- \[BUGFIX] TSDB/Agent: fix race in agent appender where concurrent appends for the same label set could produce duplicate in-memory series and duplicate WAL records. [#18292](prometheus/prometheus#18292)
- \[BUGFIX] Config: Update `--enable-feature`  flag description and sort feature names. [#18487](prometheus/prometheus#18487)
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jun 6, 2026
##### [\`v3.12.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.12.0)

This release contains security fixes, new features (especially around PromQL and Service Discovery), performance improvements in TSDB, Start Timestamp improvements and numerous bug fixes.

Thanks to all contributors!

#### Key Highlights

- **Security**: Two security vulnerabilities have been addressed: a denial of service in remote-write (snappy decompression limit) and a secret exposure leak in STACKIT service discovery.
- **PromQL & Metadata**: Several features and bug fixes related to the experimental "start timestamps" support, including updates to `rate()`, `irate()`, `increase()`, and `resets()`. New experimental functions `start()`, `end()`, `range()`, and `step()` are introduced.
- **TSDB Performance**: Optimizations in head chunk lookup (constant time) and mmap operations to reduce CPU usage.
- **Service Discovery**: Added support for DigitalOcean Managed Databases and Outscale VM, along with improvements to AWS SD (IPv6 support for EC2, external ID support).
- **UI**: Added a web interface for deleting time series and cleaning tombstones.

#### Changelog

- \[SECURITY] Remote: Reject snappy-compressed received requests via Remote Write whose declared decoded length exceeds the 32MB. Thanks to [@hibrian827](https://github.com/hibrian827) for reporting it. [#18642](prometheus/prometheus#18642)
- \[SECURITY] STACKIT SD: Fix secrets being exposed in plaintext via `/-/config` endpoint. Thanks to [@August829](https://github.com/August829) and [@Phaxma](https://github.com/Phaxma) for reporting. GHSA-39j6-789q-qxvh [#18649](prometheus/prometheus#18649)
- \[CHANGE] TSDB/Agent: Adds Start Timestamp field to all WAL Histogram samples in memory; used `st-storage` flag is enabled. [#18221](prometheus/prometheus#18221)
- \[FEATURE] API: Add `/api/v1/status/self_metrics` endpoint returning the current state of the Prometheus server's own metrics about itself as JSON. [#18411](prometheus/prometheus#18411)
- \[FEATURE] Discovery: Add DigitalOcean Managed Databases service discovery [#18287](prometheus/prometheus#18287)
- \[FEATURE] Prometheus: Add support for the aix/ppc64 compilation target [#18321](prometheus/prometheus#18321)
- \[FEATURE] Discovery: Add Outscale VM service discovery (`outscale_sd_configs`) for discovering scrape targets from the Outscale Cloud API. [#18139](prometheus/prometheus#18139)
- \[FEATURE] PromQL: Emit a warning when `sort`, `sort_by_label` or `sort_by_label_desc` is used within range (matrix) queries, as these functions do not have effect in that context. [#18498](prometheus/prometheus#18498)
- \[FEATURE] PromQL: Add `start()`, `end()`, `range()`, and `step()` experimental functions [#17877](prometheus/prometheus#17877)
- \[FEATURE] PromQL: Update `resets()` function to consider start timestamp resets. Hidden behind `use-start-timestamps` feature flag. [#18627](prometheus/prometheus#18627)
- \[FEATURE] Prometheus: Promote auto-reload-config as stable [#18620](prometheus/prometheus#18620)
- \[FEATURE] TSDB/Agent: Add `CheckpointFromInMemorySeries` option to `agent.DB` that enables checkpoint based on in-memory series. [#17948](prometheus/prometheus#17948)
- \[FEATURE] UI: Add a web interface for deleting time series and cleaning tombstones, accessible from the Status menu. [#18390](prometheus/prometheus#18390)
- \[FEATURE] PromQL: Use start timestamps for `rate()`, `irate()`, and `increase()` calculations, behind a feature flag `use-start-timestamps`. Doesn't work together with extended range selectors `anchored` and `smoothed`. [#18344](prometheus/prometheus#18344)
- \[FEATURE] Scrape: Added a feature flag `st-synthesis` which synthesizes unknown STs for scraped cumulative metrics. Useful when Remote Writing 2.0 with delta or Otel-based backends. [#18279](prometheus/prometheus#18279)
- \[FEATURE] promqltest: support `@st` annotation in `load` blocks to specify per-sample start timestamps. [#18360](prometheus/prometheus#18360)
- \[ENHANCEMENT] API: reject concurrent fgprof profiles. [#18651](prometheus/prometheus#18651)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field to ECS/MSK/RDS/Elasticache. [#18579](prometheus/prometheus#18579)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field. [#17171](prometheus/prometheus#17171)
- \[ENHANCEMENT] Discovery: Propagate SD target updates faster by introducing dynamic backoff interval instead of static 5s interval for throttling. [#18187](prometheus/prometheus#18187)
- \[ENHANCEMENT] Promtool: Add `--header` flag to `query instant` command, matching existing `query range` behaviour. [#18418](prometheus/prometheus#18418)
- \[ENHANCEMENT]: AWS SD: Allows EC2 service discovery to discover IPv6 addresses to communicate with target endpoints. The private IPv4 address remains the default when both IPv4 and IPv6 addresses are present. [#16088](prometheus/prometheus#16088)
- \[PERF] TSDB: Make head chunk lookup in range queries constant time instead of quadratic time [#18302](prometheus/prometheus#18302)
- \[PERF] TSDB: Skip entire stripes in mmapHeadChunks when no series need mmapping, reducing CPU utilization significantly at production-relevant scales. [#18541](prometheus/prometheus#18541)
- \[PERF] TSDB: Skip clean series during periodic head chunk mmap using cached head chunk count [#18272](prometheus/prometheus#18272)
- \[PERF] PromQL: Address FloatHistogram.KahanAdd performance regression on Go 1.26. [#18568](prometheus/prometheus#18568)
- \[BUGFIX] PromQL: Fix `info()` function incorrectly handling negated `__name__` matchers [#17932](prometheus/prometheus#17932)
- \[BUGFIX] API: Return duration expressions in `/parse_ast`. [#18624](prometheus/prometheus#18624)
- \[BUGFIX] API: correctly document formats accepted for duration query request parameters (step, timeout and lookback delta) in OpenAPI spec [#18305](prometheus/prometheus#18305)
- \[BUGFIX] Scrape: AppenderV2 now tracks staleness even when OOO/duplicate series errors happen similar to AppenderV1 [#18567](prometheus/prometheus#18567)
- \[BUGFIX] Config: Validate remote\_write queue\_config fields at load time to prevent runtime panic and silent misconfiguration. [#18209](prometheus/prometheus#18209)
- \[BUGFIX] Discovery/Consul: Add `health_filter` for Health API filtering, fixing breakage when using Catalog-only fields like `ServiceTags` in `filter`. [#18479](prometheus/prometheus#18479) [#18499](prometheus/prometheus#18499)
- \[BUGFIX] OTLP: limit decompressed body size for gzip-encoded OTLP write requests. [#18408](prometheus/prometheus#18408)
- \[BUGFIX] PromQL: Fix `smoothed` rate/increase returning zero instead of no result when all data falls strictly after the query range. [#18523](prometheus/prometheus#18523)
- \[BUGFIX] PromQL: Fix metric name not being dropped when last\_over\_time or first\_over\_time is applied to subqueries containing name-dropping functions like abs(). [#18409](prometheus/prometheus#18409)
- \[BUGFIX] PromQL: Fix missing warning when mixing exponential and custom-bucket histograms in stats queries. [#18660](prometheus/prometheus#18660)
- \[BUGFIX] PromQL: Fix parsing of `range()` keyword in duration expressions such as `foo[5m+range()]`. [#18623](prometheus/prometheus#18623)
- \[BUGFIX] PromQL: Fix smoothed vector selector returning no results in binary operations when the `@` modifier is used. [#18531](prometheus/prometheus#18531)
- \[BUGFIX] PromQL: Reject NaN, infinite, and out-of-range duration expressions instead of silently producing an out-of-range time.Duration. [#18639](prometheus/prometheus#18639)
- \[BUGFIX] Scrape: Fix panic when scraping malformed native histograms. [#18414](prometheus/prometheus#18414)
- \[BUGFIX] Scrape: fix panic when scraping a target exposing a summary with no quantiles via the protobuf format. [#18382](prometheus/prometheus#18382)
- \[BUGFIX] Scrape: fix scrape failure log file occasionally not applied after a configuration reload. [#18421](prometheus/prometheus#18421)
- \[BUGFIX] TSDB: Allow retention percentage with new data path. [#18628](prometheus/prometheus#18628)
- \[BUGFIX] TSDB: Preserve decimal precision in percentage-based retention [#18374](prometheus/prometheus#18374)
- \[BUGFIX] TSDB: fix prometheus\_tsdb\_head\_chunks going negative after WAL replay [#18401](prometheus/prometheus#18401)
- \[BUGFIX] TSDB: panic with native histograms during query of overlapping chunks. [#18692](prometheus/prometheus#18692)
- \[BUGFIX] Tracing: fix startup failure for insecure OTLP HTTP tracing [#18469](prometheus/prometheus#18469)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] UI: Improve Y-axis tick label precision for graph values over small ranges. [#18682](prometheus/prometheus#18682)
- \[BUGFIX] `prometheus_sd_refresh*` and `prometheus_sd_discovered_targets` metrics for specific scrape jobs are deleted when the scrape job is removed. [#17614](prometheus/prometheus#17614)
- \[BUGFIX] Remote: fixed validation for received RW2 requests when parsing metadata unit symbols. This fixes a case when request would cause (recovered) handler panic. [#18641](prometheus/prometheus#18641)
- \[BUGFIX] TSDB/Agent: fix race in agent appender where concurrent appends for the same label set could produce duplicate in-memory series and duplicate WAL records. [#18292](prometheus/prometheus#18292)
- \[BUGFIX] Config: Update `--enable-feature`  flag description and sort feature names. [#18487](prometheus/prometheus#18487)
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jun 6, 2026
##### [\`v3.12.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.12.0)

This release contains security fixes, new features (especially around PromQL and Service Discovery), performance improvements in TSDB, Start Timestamp improvements and numerous bug fixes.

Thanks to all contributors!

#### Key Highlights

- **Security**: Two security vulnerabilities have been addressed: a denial of service in remote-write (snappy decompression limit) and a secret exposure leak in STACKIT service discovery.
- **PromQL & Metadata**: Several features and bug fixes related to the experimental "start timestamps" support, including updates to `rate()`, `irate()`, `increase()`, and `resets()`. New experimental functions `start()`, `end()`, `range()`, and `step()` are introduced.
- **TSDB Performance**: Optimizations in head chunk lookup (constant time) and mmap operations to reduce CPU usage.
- **Service Discovery**: Added support for DigitalOcean Managed Databases and Outscale VM, along with improvements to AWS SD (IPv6 support for EC2, external ID support).
- **UI**: Added a web interface for deleting time series and cleaning tombstones.

#### Changelog

- \[SECURITY] Remote: Reject snappy-compressed received requests via Remote Write whose declared decoded length exceeds the 32MB. Thanks to [@hibrian827](https://github.com/hibrian827) for reporting it. [#18642](prometheus/prometheus#18642)
- \[SECURITY] STACKIT SD: Fix secrets being exposed in plaintext via `/-/config` endpoint. Thanks to [@August829](https://github.com/August829) and [@Phaxma](https://github.com/Phaxma) for reporting. GHSA-39j6-789q-qxvh [#18649](prometheus/prometheus#18649)
- \[CHANGE] TSDB/Agent: Adds Start Timestamp field to all WAL Histogram samples in memory; used `st-storage` flag is enabled. [#18221](prometheus/prometheus#18221)
- \[FEATURE] API: Add `/api/v1/status/self_metrics` endpoint returning the current state of the Prometheus server's own metrics about itself as JSON. [#18411](prometheus/prometheus#18411)
- \[FEATURE] Discovery: Add DigitalOcean Managed Databases service discovery [#18287](prometheus/prometheus#18287)
- \[FEATURE] Prometheus: Add support for the aix/ppc64 compilation target [#18321](prometheus/prometheus#18321)
- \[FEATURE] Discovery: Add Outscale VM service discovery (`outscale_sd_configs`) for discovering scrape targets from the Outscale Cloud API. [#18139](prometheus/prometheus#18139)
- \[FEATURE] PromQL: Emit a warning when `sort`, `sort_by_label` or `sort_by_label_desc` is used within range (matrix) queries, as these functions do not have effect in that context. [#18498](prometheus/prometheus#18498)
- \[FEATURE] PromQL: Add `start()`, `end()`, `range()`, and `step()` experimental functions [#17877](prometheus/prometheus#17877)
- \[FEATURE] PromQL: Update `resets()` function to consider start timestamp resets. Hidden behind `use-start-timestamps` feature flag. [#18627](prometheus/prometheus#18627)
- \[FEATURE] Prometheus: Promote auto-reload-config as stable [#18620](prometheus/prometheus#18620)
- \[FEATURE] TSDB/Agent: Add `CheckpointFromInMemorySeries` option to `agent.DB` that enables checkpoint based on in-memory series. [#17948](prometheus/prometheus#17948)
- \[FEATURE] UI: Add a web interface for deleting time series and cleaning tombstones, accessible from the Status menu. [#18390](prometheus/prometheus#18390)
- \[FEATURE] PromQL: Use start timestamps for `rate()`, `irate()`, and `increase()` calculations, behind a feature flag `use-start-timestamps`. Doesn't work together with extended range selectors `anchored` and `smoothed`. [#18344](prometheus/prometheus#18344)
- \[FEATURE] Scrape: Added a feature flag `st-synthesis` which synthesizes unknown STs for scraped cumulative metrics. Useful when Remote Writing 2.0 with delta or Otel-based backends. [#18279](prometheus/prometheus#18279)
- \[FEATURE] promqltest: support `@st` annotation in `load` blocks to specify per-sample start timestamps. [#18360](prometheus/prometheus#18360)
- \[ENHANCEMENT] API: reject concurrent fgprof profiles. [#18651](prometheus/prometheus#18651)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field to ECS/MSK/RDS/Elasticache. [#18579](prometheus/prometheus#18579)
- \[ENHANCEMENT] AWS SD: Add optional `external_id` field. [#17171](prometheus/prometheus#17171)
- \[ENHANCEMENT] Discovery: Propagate SD target updates faster by introducing dynamic backoff interval instead of static 5s interval for throttling. [#18187](prometheus/prometheus#18187)
- \[ENHANCEMENT] Promtool: Add `--header` flag to `query instant` command, matching existing `query range` behaviour. [#18418](prometheus/prometheus#18418)
- \[ENHANCEMENT]: AWS SD: Allows EC2 service discovery to discover IPv6 addresses to communicate with target endpoints. The private IPv4 address remains the default when both IPv4 and IPv6 addresses are present. [#16088](prometheus/prometheus#16088)
- \[PERF] TSDB: Make head chunk lookup in range queries constant time instead of quadratic time [#18302](prometheus/prometheus#18302)
- \[PERF] TSDB: Skip entire stripes in mmapHeadChunks when no series need mmapping, reducing CPU utilization significantly at production-relevant scales. [#18541](prometheus/prometheus#18541)
- \[PERF] TSDB: Skip clean series during periodic head chunk mmap using cached head chunk count [#18272](prometheus/prometheus#18272)
- \[PERF] PromQL: Address FloatHistogram.KahanAdd performance regression on Go 1.26. [#18568](prometheus/prometheus#18568)
- \[BUGFIX] PromQL: Fix `info()` function incorrectly handling negated `__name__` matchers [#17932](prometheus/prometheus#17932)
- \[BUGFIX] API: Return duration expressions in `/parse_ast`. [#18624](prometheus/prometheus#18624)
- \[BUGFIX] API: correctly document formats accepted for duration query request parameters (step, timeout and lookback delta) in OpenAPI spec [#18305](prometheus/prometheus#18305)
- \[BUGFIX] Scrape: AppenderV2 now tracks staleness even when OOO/duplicate series errors happen similar to AppenderV1 [#18567](prometheus/prometheus#18567)
- \[BUGFIX] Config: Validate remote\_write queue\_config fields at load time to prevent runtime panic and silent misconfiguration. [#18209](prometheus/prometheus#18209)
- \[BUGFIX] Discovery/Consul: Add `health_filter` for Health API filtering, fixing breakage when using Catalog-only fields like `ServiceTags` in `filter`. [#18479](prometheus/prometheus#18479) [#18499](prometheus/prometheus#18499)
- \[BUGFIX] OTLP: limit decompressed body size for gzip-encoded OTLP write requests. [#18408](prometheus/prometheus#18408)
- \[BUGFIX] PromQL: Fix `smoothed` rate/increase returning zero instead of no result when all data falls strictly after the query range. [#18523](prometheus/prometheus#18523)
- \[BUGFIX] PromQL: Fix metric name not being dropped when last\_over\_time or first\_over\_time is applied to subqueries containing name-dropping functions like abs(). [#18409](prometheus/prometheus#18409)
- \[BUGFIX] PromQL: Fix missing warning when mixing exponential and custom-bucket histograms in stats queries. [#18660](prometheus/prometheus#18660)
- \[BUGFIX] PromQL: Fix parsing of `range()` keyword in duration expressions such as `foo[5m+range()]`. [#18623](prometheus/prometheus#18623)
- \[BUGFIX] PromQL: Fix smoothed vector selector returning no results in binary operations when the `@` modifier is used. [#18531](prometheus/prometheus#18531)
- \[BUGFIX] PromQL: Reject NaN, infinite, and out-of-range duration expressions instead of silently producing an out-of-range time.Duration. [#18639](prometheus/prometheus#18639)
- \[BUGFIX] Scrape: Fix panic when scraping malformed native histograms. [#18414](prometheus/prometheus#18414)
- \[BUGFIX] Scrape: fix panic when scraping a target exposing a summary with no quantiles via the protobuf format. [#18382](prometheus/prometheus#18382)
- \[BUGFIX] Scrape: fix scrape failure log file occasionally not applied after a configuration reload. [#18421](prometheus/prometheus#18421)
- \[BUGFIX] TSDB: Allow retention percentage with new data path. [#18628](prometheus/prometheus#18628)
- \[BUGFIX] TSDB: Preserve decimal precision in percentage-based retention [#18374](prometheus/prometheus#18374)
- \[BUGFIX] TSDB: fix prometheus\_tsdb\_head\_chunks going negative after WAL replay [#18401](prometheus/prometheus#18401)
- \[BUGFIX] TSDB: panic with native histograms during query of overlapping chunks. [#18692](prometheus/prometheus#18692)
- \[BUGFIX] Tracing: fix startup failure for insecure OTLP HTTP tracing [#18469](prometheus/prometheus#18469)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] UI: Improve Y-axis tick label precision for graph values over small ranges. [#18682](prometheus/prometheus#18682)
- \[BUGFIX] `prometheus_sd_refresh*` and `prometheus_sd_discovered_targets` metrics for specific scrape jobs are deleted when the scrape job is removed. [#17614](prometheus/prometheus#17614)
- \[BUGFIX] Remote: fixed validation for received RW2 requests when parsing metadata unit symbols. This fixes a case when request would cause (recovered) handler panic. [#18641](prometheus/prometheus#18641)
- \[BUGFIX] TSDB/Agent: fix race in agent appender where concurrent appends for the same label set could produce duplicate in-memory series and duplicate WAL records. [#18292](prometheus/prometheus#18292)
- \[BUGFIX] Config: Update `--enable-feature`  flag description and sort feature names. [#18487](prometheus/prometheus#18487)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants