Skip to content

remote_write : Add Certificate support for ingesting data into an Azure Monitor Workspace#18217

Merged
bwplotka merged 4 commits into
prometheus:mainfrom
bragi92:main
Jun 8, 2026
Merged

remote_write : Add Certificate support for ingesting data into an Azure Monitor Workspace#18217
bwplotka merged 4 commits into
prometheus:mainfrom
bragi92:main

Conversation

@bragi92

@bragi92 bragi92 commented Mar 2, 2026

Copy link
Copy Markdown
Contributor

Which issue(s) does the PR fix: Fixes #17751

Does this PR introduce a user-facing change?

[FEATURE] remote_write : Add Certificate support for ingesting data into an Azure Monitor Workspace

Changes

  • Adds a new azuread.certificate configuration block for remote_write, allowing users to authenticate to Azure AD using a client certificate and private key instead of a client secret.
  • Implements certificate credential support in azuread.go using azidentity.NewClientCertificateCredential.
  • Adds a shared util/certutil package for certificate parsing, supporting both PEM and PKCS#12 formats, using the standard golang.org/x/crypto/pkcs12 library (no third-party dependencies).
  • Maintains full backwards compatibility with existing client secret, managed identity, and workload identity auth paths.
  • Validates that only one Azure AD auth method is configured at a time.

Configuration Example

remote_write:
  - url: <metrics ingestion endpoint for azure monitor workspace>
    azuread:
      cloud: AzurePublic
      certificate:
        client_id: 
        tenant_id: 
        certificate_path: /etc/prometheus/certs/app.crt
        certificate_key_path: /etc/prometheus/certs/app.key

Testing

I've done some manual validation too for this by running the binary in my local setup and using a self-signed cert uploaded into an App Registrion in Azure. I see data flowing correctly into an Azure Monitor Workspace after I gave 'Monitoring Metrics Publisher' role on the system assigned DCR (Data Collector Rule) for the Azure Monitor Workspace.

image

* Initial plan

* Add Azure AD certificate-based authentication support

Co-authored-by: bragi92 <[email protected]>

* Update documentation for certificate-based authentication

Co-authored-by: bragi92 <[email protected]>

* Address code review feedback - improve error messages and format detection

Co-authored-by: bragi92 <[email protected]>

* Replace third-party go-pkcs12 with official golang.org/x/crypto/pkcs12

Co-authored-by: bragi92 <[email protected]>

* Extract certificate parsing to common util/certutil package

Co-authored-by: bragi92 <[email protected]>

* Fix linting errors: import ordering and comment formatting

Co-authored-by: bragi92 <[email protected]>

---------

Co-authored-by: copilot-swe-agent[bot] <[email protected]>
Co-authored-by: bragi92 <[email protected]>
@bragi92 bragi92 changed the title Add Azure AD certificate-based authentication for remote write [FEATURE] remote: add Azure AD certificate-based authentication support for remote write clients Mar 2, 2026
@bragi92 bragi92 changed the title [FEATURE] remote: add Azure AD certificate-based authentication support for remote write clients [FEATURE] remote_write: add Azure AD certificate-based authentication support for remote write clients Mar 2, 2026
@bragi92 bragi92 changed the title [FEATURE] remote_write: add Azure AD certificate-based authentication support for remote write clients remote_write azure auth: add Azure AD certificate-based authentication support Mar 2, 2026
@bragi92 bragi92 changed the title remote_write azure auth: add Azure AD certificate-based authentication support remote_write : Add Certificate support for ingesting data into an Azure Monitor Workspace Mar 2, 2026
@krajorama

Copy link
Copy Markdown
Member

Hello from the bug scrub!

We'll let the remote write people say if this is ok.

@bragi92

bragi92 commented Mar 3, 2026

Copy link
Copy Markdown
Contributor Author

The failing job in the workflow run was due to a test timeout: panic: test timed out after 10m0s while running TestDiskFillingUpAfterDisablingOOO/sample=custom_buckets_float_histogram/appV2=false.

I don't think this is related to my changes, can someone who has access try re-running it? I don't see an option to do it on my end.

Also tagging @bwplotka to help with the review if you're available. Thanks!

@bwplotka bwplotka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Do we really need certutil.go? It feels odds we need to implement so much for this 🤔

Example use azidentity function https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#pkg-overview

Comment thread storage/remote/azuread/azuread.go Outdated
Address review feedback: use azidentity.ParseCertificates instead of
the custom util/certutil package for certificate parsing. The azidentity
package already handles PEM and PKCS#12 formats natively.

- Remove util/certutil package entirely
- Use azidentity.ParseCertificates in newCertificateTokenCredential
- Revert golang.org/x/crypto to indirect dependency

Agent-Logs-Url: https://github.com/bragi92/prometheus/sessions/efc3d6c5-9927-4d5b-8aa9-afe94b659c6e

Co-authored-by: copilot-swe-agent[bot] <[email protected]>
Co-authored-by: bragi92 <[email protected]>
@bragi92

bragi92 commented May 11, 2026

Copy link
Copy Markdown
Contributor Author

Bumping this up, Please review when you have some time. @bwplotka / @cstyan / @tomwilkie / @alexgreenbank

@bragi92

bragi92 commented May 28, 2026

Copy link
Copy Markdown
Contributor Author

@krajorama, @bwplotka Will you be able to help move this along? Is there anything I can do to get this merged in.

@bwplotka bwplotka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - you can help us review and maintain some of this code, so we can get back quicker (:

Looks OK, just one important suggestion, otherwise LGTM, thanks!

Comment thread storage/remote/azuread/azuread.go Outdated
Co-authored-by: Bartlomiej Plotka <[email protected]>
Signed-off-by: bragi92 <[email protected]>
@bragi92

bragi92 commented Jun 1, 2026

Copy link
Copy Markdown
Contributor Author

Yes - you can help us review and maintain some of this code, so we can get back quicker (:

Looks OK, just one important suggestion, otherwise LGTM, thanks!

Nice catch @bwplotka Yes, I've applied the suggestion. Thank you!

And yes, I'd be happy to help maintain some of this code. I'm fairly comfortable with the remote write path, especially the Azure side: I'm on the Microsoft team that works on managed Prometheus ingestion, and enabling OSS Prometheus to ingest into Azure via remote write is exactly what we set out to support here.

The previous commit changed CertificatePassword from string to config_util.Secret but did not add the corresponding import. The CI build for this PR alone passed only because GitHub builds the merge of the PR with upstream main, which already imports config_util (introduced in upstream commit 5ccebcd for ClientSecret). Add the import so the PR's azuread.go is self-consistent.

Co-authored-by: Copilot <[email protected]>
Signed-off-by: Kaveesh Dubey (from Dev Box) <[email protected]>
@bwplotka
bwplotka merged commit d0db9b6 into prometheus:main Jun 8, 2026
35 checks passed
@bwplotka

bwplotka commented Jun 8, 2026

Copy link
Copy Markdown
Member

Ack, I will start by pinging you on RW PRs, if that's ok!

@bwplotka

bwplotka commented Jun 8, 2026

Copy link
Copy Markdown
Member

Are you on Slack @bragi92?

I'd be also curious, if there's a way to have some help to setup e2e tests for Azure SD changes or have help maintaining Azure SD in general.

@bragi92

bragi92 commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

I created an account on slack and pinged you on it @bwplotka. Also, I'll investigate the Azure SD side of this and see if I can setup some e2e tests after validating it on my end.

eleboucher pushed a commit to eleboucher/homelab that referenced this pull request Jul 1, 2026
…➔ v3.13.0) (#1360)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [quay.io/prometheus/prometheus](https://github.com/prometheus/prometheus) | minor | `v3.12.0` → `v3.13.0` |

---

### Release Notes

<details>
<summary>prometheus/prometheus (quay.io/prometheus/prometheus)</summary>

### [`v3.13.0`](https://github.com/prometheus/prometheus/releases/tag/v3.13.0): 3.13.0 / 2026-07-01

[Compare Source](prometheus/prometheus@v3.12.0...v3.13.0)

This is a Long Term Support [LTS](https://prometheus.io/docs/introduction/release-cycle/) release.

- \[SECURITY] UI: Bump `sanitize-html` to fix a cross-site scripting vulnerability (CVE-2026-44990). [#&#8203;18697](prometheus/prometheus#18697)
- \[CHANGE] UI: Third-party npm dependency licenses are now embedded in the Prometheus binary and served at `/assets/third-party-licenses.txt`, replacing the `npm_licenses.tar.bz2` archive previously shipped in release tarballs and container images. [#&#8203;18997](prometheus/prometheus#18997)
- \[CHANGE] API: Use SHA-256 instead of SHA-1 to generate rule group pagination tokens. [#&#8203;18927](prometheus/prometheus#18927)
- \[CHANGE] HTTP clients: Credentials (Authorization header, basic auth, bearer token, OAuth2, configured headers) are no longer forwarded when following a redirect to a different host; affects scraping, remote read/write, alerting, and service discovery. Via prometheus/common v0.69.0 (CVE-2025-4673 CVE-2023-45289). [#&#8203;18949](prometheus/prometheus#18949)
- \[CHANGE] promtool: Relative file paths in the file passed to `--http.config.file` are now resolved relative to that config file's directory instead of its parent directory. Via prometheus/common v0.69.0. [#&#8203;18949](prometheus/prometheus#18949)
- \[CHANGE] PromQL: Rename the `min()` and `max()` duration-expression functions (experimental feature flag `experimental-duration-expr`) to `min_of()` and `max_of()` to avoid confusion with the `min` and `max` aggregate operators. [#&#8203;18687](prometheus/prometheus#18687)
- \[FEATURE] API: Add experimental search endpoints to search metric names, label names, and label values. [#&#8203;18573](prometheus/prometheus#18573)
- \[FEATURE] Discovery/AWS: Add ability to filter RDS instances. [#&#8203;18859](prometheus/prometheus#18859)
- \[FEATURE] PromQL: Add `min_of(a, b)` and `max_of(a, b)` scalar experimental functions, returning the smaller or larger of two scalar values. [#&#8203;18687](prometheus/prometheus#18687)
- \[FEATURE] PromQL: Add support for smoothed/anchored rate with native histograms. [#&#8203;18564](prometheus/prometheus#18564)
- \[FEATURE] PromQL: Expose per-query `samplesRead` (and `samplesReadPerStep` with `stats=all` and the `promql-per-step-stats` feature flag) in the query stats response, and add the `prometheus_engine_query_samples_read_total` engine counter. `samplesRead` reflects storage I/O distinct from `totalQueryableSamples`, which counts samples loaded into the evaluator (and so over-counts when a sample is reused across multiple range-vector windows). [#&#8203;18081](prometheus/prometheus#18081)
- \[FEATURE] Scrape: Add `__convert_classic_histograms_to_nhcb__` internal label to allow per-target override of `convert_classic_histograms_to_nhcb` scrape configuration via relabeling. [#&#8203;18840](prometheus/prometheus#18840)
- \[FEATURE] TSDB: Add `storage.tsdb.chunk_encoding.floats` configuration field to select float chunk encoding (`xor` or `xor2`) at runtime, independently of the `--enable-feature=xor2-encoding` flag. [#&#8203;18769](prometheus/prometheus#18769)
- \[FEATURE] remote\_write: Add Certificate support for ingesting data into an Azure Monitor Workspace. [#&#8203;18217](prometheus/prometheus#18217)
- \[FEATURE] Scrape: Add `__always_scrape_classic_histograms__` and `__scrape_native_histograms__` internal labels to allow per-target override of the `always_scrape_classic_histograms` and `scrape_native_histograms` scrape configuration via relabeling. [#&#8203;18929](prometheus/prometheus#18929)
- \[ENHANCEMENT] Release: Container images are now also published to the GitHub Container Registry (ghcr.io). [#&#8203;18791](prometheus/prometheus#18791)
- \[ENHANCEMENT] PromQL: Prettify `fill_left(x) fill_right(x)` as `fill(x)` when both fill values are equal. [#&#8203;18851](prometheus/prometheus#18851)
- \[ENHANCEMENT] UI: Improve autocompletion after closing a function bracket. [#&#8203;18894](prometheus/prometheus#18894)
- \[PERF] Labels: Add case-insensitive prefix matching to speed up evaluation of long case-insensitive regular expressions (up to \~2x faster). [#&#8203;18540](prometheus/prometheus#18540)
- \[PERF] TSDB: Reduce per-sample overhead in chunk population, speeding up affected queries by \~12-15% in benchmarks. [#&#8203;18699](prometheus/prometheus#18699)
- \[PERF] TSDB: Eliminate unnecessary heap allocations in the V2 histogram WAL decoder, reducing allocations by up to 50% and memory by up to 10% for deployments using native histograms with created-timestamp storage enabled (`--enable-feature=created-timestamp-zero-ingestion`). [#&#8203;18813](prometheus/prometheus#18813)
- \[BUGFIX] Discovery/AWS: Fix failure when processing an AWS RDS cluster without instances. [#&#8203;18845](prometheus/prometheus#18845)
- \[BUGFIX] Fix race condition in initTime that could cause ErrOutOfBounds. [#&#8203;18629](prometheus/prometheus#18629)
- \[BUGFIX] PromQL: A range query whose `end` was not aligned to `step` caused subqueries inside it to evaluate past the parent's last actual step, inflating `peakSamples` in the query stats and against the `query.max-samples` limit, and wasting storage I/O reading samples that were never used in the result. [#&#8203;18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: A range query containing an at-modifier-unsafe function over a range-vector with an `@` modifier (e.g. `predict_linear(metric[60s] @&#8203; T, X)`) silently under-counted `totalQueryableSamples` for steps after step 0. [#&#8203;18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: Fix `fill_left`/`fill_right` producing missing samples in range queries when using `group_left`/`group_right`. [#&#8203;18850](prometheus/prometheus#18850)
- \[BUGFIX] PromQL: Fix for resets() and changes() in anchored range extenders with histograms. [#&#8203;18906](prometheus/prometheus#18906)
- \[BUGFIX] PromQL: Fix panic on `1[5m] smoothed` and similar expressions when extended range selectors are enabled. [#&#8203;18764](prometheus/prometheus#18764)
- \[BUGFIX] PromQL: Fix panic when a `smoothed` instant vector selector produces no samples for a series. [#&#8203;18943](prometheus/prometheus#18943)
- \[BUGFIX] PromQL: Fix panic when using a parenthesised plain number as an offset (e.g. `foo offset -(5)`). [#&#8203;18768](prometheus/prometheus#18768)
- \[BUGFIX] promtool: Fix panic when parsing exposition text containing empty braces `{}`. Via prometheus/common v0.69.0. [#&#8203;18949](prometheus/prometheus#18949)
- \[BUGFIX] Promtool: Fix `check healthy` and `check ready` when `--url` ends with a trailing slash. [#&#8203;18854](prometheus/prometheus#18854)
- \[BUGFIX] Rules: Close PromQL query after each rule evaluation to ensure resources are released. [#&#8203;18733](prometheus/prometheus#18733)
- \[BUGFIX] Scaleway SD: Resolve VPC/IPAM-only instances that have no legacy `private_ip` or `public_ip` field, but do have private NICs attached. [#&#8203;18772](prometheus/prometheus#18772)
- \[BUGFIX] TSDB: Do not leak head series when an integer histogram append is rejected (e.g. out-of-order). [#&#8203;18838](prometheus/prometheus#18838)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#&#8203;18658](prometheus/prometheus#18658)
- \[BUGFIX] TSDB: Fix chunk snapshot encoding for EncXOR2 chunks, preventing corruption on TSDB restart when EncXOR2-encoded series were present. [#&#8203;18739](prometheus/prometheus#18739)
- \[BUGFIX] TSDB: Store a millisecond timestamp (not a WAL segment number) in walExpiries when a series is evicted via CompactStaleHead/CompactSelectedSeries, so the series's label record is correctly retained in the next WAL checkpoint and replays cleanly. [#&#8203;18847](prometheus/prometheus#18847)
- \[BUGFIX] TSDB: Prevent loss of samples at the chunk-range boundary when CompactSelectedSeries (and CompactStaleHead) evict the series — the per-slice compaction loop now runs one more iteration so the boundary timestamp is captured in a block before the in-memory copy is removed. [#&#8203;18849](prometheus/prometheus#18849)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEwMS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZS9jb250YWluZXIiLCJ0eXBlL21pbm9yIl19-->

Reviewed-on: https://git.erwanleboucher.dev/eleboucher/homelab/pulls/1360
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jul 7, 2026
##### [\`v3.13.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.13.0)

This is a Long Term Support [LTS](https://prometheus.io/docs/introduction/release-cycle/) release.

- \[SECURITY] UI: Bump `sanitize-html` to fix a cross-site scripting vulnerability (CVE-2026-44990). [#18697](prometheus/prometheus#18697)
- \[CHANGE] UI: Third-party npm dependency licenses are now embedded in the Prometheus binary and served at `/assets/third-party-licenses.txt`, replacing the `npm_licenses.tar.bz2` archive previously shipped in release tarballs and container images. [#18997](prometheus/prometheus#18997)
- \[CHANGE] API: Use SHA-256 instead of SHA-1 to generate rule group pagination tokens. [#18927](prometheus/prometheus#18927)
- \[CHANGE] HTTP clients: Credentials (Authorization header, basic auth, bearer token, OAuth2, configured headers) are no longer forwarded when following a redirect to a different host; affects scraping, remote read/write, alerting, and service discovery. Via prometheus/common v0.69.0 (CVE-2025-4673 CVE-2023-45289). [#18949](prometheus/prometheus#18949)
- \[CHANGE] promtool: Relative file paths in the file passed to `--http.config.file` are now resolved relative to that config file's directory instead of its parent directory. Via prometheus/common v0.69.0. [#18949](prometheus/prometheus#18949)
- \[CHANGE] PromQL: Rename the `min()` and `max()` duration-expression functions (experimental feature flag `experimental-duration-expr`) to `min_of()` and `max_of()` to avoid confusion with the `min` and `max` aggregate operators. [#18687](prometheus/prometheus#18687)
- \[FEATURE] API: Add experimental search endpoints to search metric names, label names, and label values. [#18573](prometheus/prometheus#18573)
- \[FEATURE] Discovery/AWS: Add ability to filter RDS instances. [#18859](prometheus/prometheus#18859)
- \[FEATURE] PromQL: Add `min_of(a, b)` and `max_of(a, b)` scalar experimental functions, returning the smaller or larger of two scalar values. [#18687](prometheus/prometheus#18687)
- \[FEATURE] PromQL: Add support for smoothed/anchored rate with native histograms. [#18564](prometheus/prometheus#18564)
- \[FEATURE] PromQL: Expose per-query `samplesRead` (and `samplesReadPerStep` with `stats=all` and the `promql-per-step-stats` feature flag) in the query stats response, and add the `prometheus_engine_query_samples_read_total` engine counter. `samplesRead` reflects storage I/O distinct from `totalQueryableSamples`, which counts samples loaded into the evaluator (and so over-counts when a sample is reused across multiple range-vector windows). [#18081](prometheus/prometheus#18081)
- \[FEATURE] Scrape: Add `__convert_classic_histograms_to_nhcb__` internal label to allow per-target override of `convert_classic_histograms_to_nhcb` scrape configuration via relabeling. [#18840](prometheus/prometheus#18840)
- \[FEATURE] TSDB: Add `storage.tsdb.chunk_encoding.floats` configuration field to select float chunk encoding (`xor` or `xor2`) at runtime, independently of the `--enable-feature=xor2-encoding` flag. [#18769](prometheus/prometheus#18769)
- \[FEATURE] remote\_write: Add Certificate support for ingesting data into an Azure Monitor Workspace. [#18217](prometheus/prometheus#18217)
- \[FEATURE] Scrape: Add `__always_scrape_classic_histograms__` and `__scrape_native_histograms__` internal labels to allow per-target override of the `always_scrape_classic_histograms` and `scrape_native_histograms` scrape configuration via relabeling. [#18929](prometheus/prometheus#18929)
- \[ENHANCEMENT] Release: Container images are now also published to the GitHub Container Registry (ghcr.io). [#18791](prometheus/prometheus#18791)
- \[ENHANCEMENT] PromQL: Prettify `fill_left(x) fill_right(x)` as `fill(x)` when both fill values are equal. [#18851](prometheus/prometheus#18851)
- \[ENHANCEMENT] UI: Improve autocompletion after closing a function bracket. [#18894](prometheus/prometheus#18894)
- \[PERF] Labels: Add case-insensitive prefix matching to speed up evaluation of long case-insensitive regular expressions (up to \~2x faster). [#18540](prometheus/prometheus#18540)
- \[PERF] TSDB: Reduce per-sample overhead in chunk population, speeding up affected queries by \~12-15% in benchmarks. [#18699](prometheus/prometheus#18699)
- \[PERF] TSDB: Eliminate unnecessary heap allocations in the V2 histogram WAL decoder, reducing allocations by up to 50% and memory by up to 10% for deployments using native histograms with created-timestamp storage enabled (`--enable-feature=created-timestamp-zero-ingestion`). [#18813](prometheus/prometheus#18813)
- \[BUGFIX] Discovery/AWS: Fix failure when processing an AWS RDS cluster without instances. [#18845](prometheus/prometheus#18845)
- \[BUGFIX] Fix race condition in initTime that could cause ErrOutOfBounds. [#18629](prometheus/prometheus#18629)
- \[BUGFIX] PromQL: A range query whose `end` was not aligned to `step` caused subqueries inside it to evaluate past the parent's last actual step, inflating `peakSamples` in the query stats and against the `query.max-samples` limit, and wasting storage I/O reading samples that were never used in the result. [#18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: A range query containing an at-modifier-unsafe function over a range-vector with an `@` modifier (e.g. `predict_linear(metric[60s] @ T, X)`) silently under-counted `totalQueryableSamples` for steps after step 0. [#18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: Fix `fill_left`/`fill_right` producing missing samples in range queries when using `group_left`/`group_right`. [#18850](prometheus/prometheus#18850)
- \[BUGFIX] PromQL: Fix for resets() and changes() in anchored range extenders with histograms. [#18906](prometheus/prometheus#18906)
- \[BUGFIX] PromQL: Fix panic on `1[5m] smoothed` and similar expressions when extended range selectors are enabled. [#18764](prometheus/prometheus#18764)
- \[BUGFIX] PromQL: Fix panic when a `smoothed` instant vector selector produces no samples for a series. [#18943](prometheus/prometheus#18943)
- \[BUGFIX] PromQL: Fix panic when using a parenthesised plain number as an offset (e.g. `foo offset -(5)`). [#18768](prometheus/prometheus#18768)
- \[BUGFIX] promtool: Fix panic when parsing exposition text containing empty braces `{}`. Via prometheus/common v0.69.0. [#18949](prometheus/prometheus#18949)
- \[BUGFIX] Promtool: Fix `check healthy` and `check ready` when `--url` ends with a trailing slash. [#18854](prometheus/prometheus#18854)
- \[BUGFIX] Rules: Close PromQL query after each rule evaluation to ensure resources are released. [#18733](prometheus/prometheus#18733)
- \[BUGFIX] Scaleway SD: Resolve VPC/IPAM-only instances that have no legacy `private_ip` or `public_ip` field, but do have private NICs attached. [#18772](prometheus/prometheus#18772)
- \[BUGFIX] TSDB: Do not leak head series when an integer histogram append is rejected (e.g. out-of-order). [#18838](prometheus/prometheus#18838)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] TSDB: Fix chunk snapshot encoding for EncXOR2 chunks, preventing corruption on TSDB restart when EncXOR2-encoded series were present. [#18739](prometheus/prometheus#18739)
- \[BUGFIX] TSDB: Store a millisecond timestamp (not a WAL segment number) in walExpiries when a series is evicted via CompactStaleHead/CompactSelectedSeries, so the series's label record is correctly retained in the next WAL checkpoint and replays cleanly. [#18847](prometheus/prometheus#18847)
- \[BUGFIX] TSDB: Prevent loss of samples at the chunk-range boundary when CompactSelectedSeries (and CompactStaleHead) evict the series — the per-slice compaction loop now runs one more iteration so the boundary timestamp is captured in a block before the in-memory copy is removed. [#18849](prometheus/prometheus#18849)
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jul 7, 2026
##### [\`v3.13.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.13.0)

This is a Long Term Support [LTS](https://prometheus.io/docs/introduction/release-cycle/) release.

- \[SECURITY] UI: Bump `sanitize-html` to fix a cross-site scripting vulnerability (CVE-2026-44990). [#18697](prometheus/prometheus#18697)
- \[CHANGE] UI: Third-party npm dependency licenses are now embedded in the Prometheus binary and served at `/assets/third-party-licenses.txt`, replacing the `npm_licenses.tar.bz2` archive previously shipped in release tarballs and container images. [#18997](prometheus/prometheus#18997)
- \[CHANGE] API: Use SHA-256 instead of SHA-1 to generate rule group pagination tokens. [#18927](prometheus/prometheus#18927)
- \[CHANGE] HTTP clients: Credentials (Authorization header, basic auth, bearer token, OAuth2, configured headers) are no longer forwarded when following a redirect to a different host; affects scraping, remote read/write, alerting, and service discovery. Via prometheus/common v0.69.0 (CVE-2025-4673 CVE-2023-45289). [#18949](prometheus/prometheus#18949)
- \[CHANGE] promtool: Relative file paths in the file passed to `--http.config.file` are now resolved relative to that config file's directory instead of its parent directory. Via prometheus/common v0.69.0. [#18949](prometheus/prometheus#18949)
- \[CHANGE] PromQL: Rename the `min()` and `max()` duration-expression functions (experimental feature flag `experimental-duration-expr`) to `min_of()` and `max_of()` to avoid confusion with the `min` and `max` aggregate operators. [#18687](prometheus/prometheus#18687)
- \[FEATURE] API: Add experimental search endpoints to search metric names, label names, and label values. [#18573](prometheus/prometheus#18573)
- \[FEATURE] Discovery/AWS: Add ability to filter RDS instances. [#18859](prometheus/prometheus#18859)
- \[FEATURE] PromQL: Add `min_of(a, b)` and `max_of(a, b)` scalar experimental functions, returning the smaller or larger of two scalar values. [#18687](prometheus/prometheus#18687)
- \[FEATURE] PromQL: Add support for smoothed/anchored rate with native histograms. [#18564](prometheus/prometheus#18564)
- \[FEATURE] PromQL: Expose per-query `samplesRead` (and `samplesReadPerStep` with `stats=all` and the `promql-per-step-stats` feature flag) in the query stats response, and add the `prometheus_engine_query_samples_read_total` engine counter. `samplesRead` reflects storage I/O distinct from `totalQueryableSamples`, which counts samples loaded into the evaluator (and so over-counts when a sample is reused across multiple range-vector windows). [#18081](prometheus/prometheus#18081)
- \[FEATURE] Scrape: Add `__convert_classic_histograms_to_nhcb__` internal label to allow per-target override of `convert_classic_histograms_to_nhcb` scrape configuration via relabeling. [#18840](prometheus/prometheus#18840)
- \[FEATURE] TSDB: Add `storage.tsdb.chunk_encoding.floats` configuration field to select float chunk encoding (`xor` or `xor2`) at runtime, independently of the `--enable-feature=xor2-encoding` flag. [#18769](prometheus/prometheus#18769)
- \[FEATURE] remote\_write: Add Certificate support for ingesting data into an Azure Monitor Workspace. [#18217](prometheus/prometheus#18217)
- \[FEATURE] Scrape: Add `__always_scrape_classic_histograms__` and `__scrape_native_histograms__` internal labels to allow per-target override of the `always_scrape_classic_histograms` and `scrape_native_histograms` scrape configuration via relabeling. [#18929](prometheus/prometheus#18929)
- \[ENHANCEMENT] Release: Container images are now also published to the GitHub Container Registry (ghcr.io). [#18791](prometheus/prometheus#18791)
- \[ENHANCEMENT] PromQL: Prettify `fill_left(x) fill_right(x)` as `fill(x)` when both fill values are equal. [#18851](prometheus/prometheus#18851)
- \[ENHANCEMENT] UI: Improve autocompletion after closing a function bracket. [#18894](prometheus/prometheus#18894)
- \[PERF] Labels: Add case-insensitive prefix matching to speed up evaluation of long case-insensitive regular expressions (up to \~2x faster). [#18540](prometheus/prometheus#18540)
- \[PERF] TSDB: Reduce per-sample overhead in chunk population, speeding up affected queries by \~12-15% in benchmarks. [#18699](prometheus/prometheus#18699)
- \[PERF] TSDB: Eliminate unnecessary heap allocations in the V2 histogram WAL decoder, reducing allocations by up to 50% and memory by up to 10% for deployments using native histograms with created-timestamp storage enabled (`--enable-feature=created-timestamp-zero-ingestion`). [#18813](prometheus/prometheus#18813)
- \[BUGFIX] Discovery/AWS: Fix failure when processing an AWS RDS cluster without instances. [#18845](prometheus/prometheus#18845)
- \[BUGFIX] Fix race condition in initTime that could cause ErrOutOfBounds. [#18629](prometheus/prometheus#18629)
- \[BUGFIX] PromQL: A range query whose `end` was not aligned to `step` caused subqueries inside it to evaluate past the parent's last actual step, inflating `peakSamples` in the query stats and against the `query.max-samples` limit, and wasting storage I/O reading samples that were never used in the result. [#18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: A range query containing an at-modifier-unsafe function over a range-vector with an `@` modifier (e.g. `predict_linear(metric[60s] @ T, X)`) silently under-counted `totalQueryableSamples` for steps after step 0. [#18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: Fix `fill_left`/`fill_right` producing missing samples in range queries when using `group_left`/`group_right`. [#18850](prometheus/prometheus#18850)
- \[BUGFIX] PromQL: Fix for resets() and changes() in anchored range extenders with histograms. [#18906](prometheus/prometheus#18906)
- \[BUGFIX] PromQL: Fix panic on `1[5m] smoothed` and similar expressions when extended range selectors are enabled. [#18764](prometheus/prometheus#18764)
- \[BUGFIX] PromQL: Fix panic when a `smoothed` instant vector selector produces no samples for a series. [#18943](prometheus/prometheus#18943)
- \[BUGFIX] PromQL: Fix panic when using a parenthesised plain number as an offset (e.g. `foo offset -(5)`). [#18768](prometheus/prometheus#18768)
- \[BUGFIX] promtool: Fix panic when parsing exposition text containing empty braces `{}`. Via prometheus/common v0.69.0. [#18949](prometheus/prometheus#18949)
- \[BUGFIX] Promtool: Fix `check healthy` and `check ready` when `--url` ends with a trailing slash. [#18854](prometheus/prometheus#18854)
- \[BUGFIX] Rules: Close PromQL query after each rule evaluation to ensure resources are released. [#18733](prometheus/prometheus#18733)
- \[BUGFIX] Scaleway SD: Resolve VPC/IPAM-only instances that have no legacy `private_ip` or `public_ip` field, but do have private NICs attached. [#18772](prometheus/prometheus#18772)
- \[BUGFIX] TSDB: Do not leak head series when an integer histogram append is rejected (e.g. out-of-order). [#18838](prometheus/prometheus#18838)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] TSDB: Fix chunk snapshot encoding for EncXOR2 chunks, preventing corruption on TSDB restart when EncXOR2-encoded series were present. [#18739](prometheus/prometheus#18739)
- \[BUGFIX] TSDB: Store a millisecond timestamp (not a WAL segment number) in walExpiries when a series is evicted via CompactStaleHead/CompactSelectedSeries, so the series's label record is correctly retained in the next WAL checkpoint and replays cleanly. [#18847](prometheus/prometheus#18847)
- \[BUGFIX] TSDB: Prevent loss of samples at the chunk-range boundary when CompactSelectedSeries (and CompactStaleHead) evict the series — the per-slice compaction loop now runs one more iteration so the boundary timestamp is captured in a block before the in-memory copy is removed. [#18849](prometheus/prometheus#18849)
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jul 7, 2026
##### [\`v3.13.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.13.0)

This is a Long Term Support [LTS](https://prometheus.io/docs/introduction/release-cycle/) release.

- \[SECURITY] UI: Bump `sanitize-html` to fix a cross-site scripting vulnerability (CVE-2026-44990). [#18697](prometheus/prometheus#18697)
- \[CHANGE] UI: Third-party npm dependency licenses are now embedded in the Prometheus binary and served at `/assets/third-party-licenses.txt`, replacing the `npm_licenses.tar.bz2` archive previously shipped in release tarballs and container images. [#18997](prometheus/prometheus#18997)
- \[CHANGE] API: Use SHA-256 instead of SHA-1 to generate rule group pagination tokens. [#18927](prometheus/prometheus#18927)
- \[CHANGE] HTTP clients: Credentials (Authorization header, basic auth, bearer token, OAuth2, configured headers) are no longer forwarded when following a redirect to a different host; affects scraping, remote read/write, alerting, and service discovery. Via prometheus/common v0.69.0 (CVE-2025-4673 CVE-2023-45289). [#18949](prometheus/prometheus#18949)
- \[CHANGE] promtool: Relative file paths in the file passed to `--http.config.file` are now resolved relative to that config file's directory instead of its parent directory. Via prometheus/common v0.69.0. [#18949](prometheus/prometheus#18949)
- \[CHANGE] PromQL: Rename the `min()` and `max()` duration-expression functions (experimental feature flag `experimental-duration-expr`) to `min_of()` and `max_of()` to avoid confusion with the `min` and `max` aggregate operators. [#18687](prometheus/prometheus#18687)
- \[FEATURE] API: Add experimental search endpoints to search metric names, label names, and label values. [#18573](prometheus/prometheus#18573)
- \[FEATURE] Discovery/AWS: Add ability to filter RDS instances. [#18859](prometheus/prometheus#18859)
- \[FEATURE] PromQL: Add `min_of(a, b)` and `max_of(a, b)` scalar experimental functions, returning the smaller or larger of two scalar values. [#18687](prometheus/prometheus#18687)
- \[FEATURE] PromQL: Add support for smoothed/anchored rate with native histograms. [#18564](prometheus/prometheus#18564)
- \[FEATURE] PromQL: Expose per-query `samplesRead` (and `samplesReadPerStep` with `stats=all` and the `promql-per-step-stats` feature flag) in the query stats response, and add the `prometheus_engine_query_samples_read_total` engine counter. `samplesRead` reflects storage I/O distinct from `totalQueryableSamples`, which counts samples loaded into the evaluator (and so over-counts when a sample is reused across multiple range-vector windows). [#18081](prometheus/prometheus#18081)
- \[FEATURE] Scrape: Add `__convert_classic_histograms_to_nhcb__` internal label to allow per-target override of `convert_classic_histograms_to_nhcb` scrape configuration via relabeling. [#18840](prometheus/prometheus#18840)
- \[FEATURE] TSDB: Add `storage.tsdb.chunk_encoding.floats` configuration field to select float chunk encoding (`xor` or `xor2`) at runtime, independently of the `--enable-feature=xor2-encoding` flag. [#18769](prometheus/prometheus#18769)
- \[FEATURE] remote\_write: Add Certificate support for ingesting data into an Azure Monitor Workspace. [#18217](prometheus/prometheus#18217)
- \[FEATURE] Scrape: Add `__always_scrape_classic_histograms__` and `__scrape_native_histograms__` internal labels to allow per-target override of the `always_scrape_classic_histograms` and `scrape_native_histograms` scrape configuration via relabeling. [#18929](prometheus/prometheus#18929)
- \[ENHANCEMENT] Release: Container images are now also published to the GitHub Container Registry (ghcr.io). [#18791](prometheus/prometheus#18791)
- \[ENHANCEMENT] PromQL: Prettify `fill_left(x) fill_right(x)` as `fill(x)` when both fill values are equal. [#18851](prometheus/prometheus#18851)
- \[ENHANCEMENT] UI: Improve autocompletion after closing a function bracket. [#18894](prometheus/prometheus#18894)
- \[PERF] Labels: Add case-insensitive prefix matching to speed up evaluation of long case-insensitive regular expressions (up to \~2x faster). [#18540](prometheus/prometheus#18540)
- \[PERF] TSDB: Reduce per-sample overhead in chunk population, speeding up affected queries by \~12-15% in benchmarks. [#18699](prometheus/prometheus#18699)
- \[PERF] TSDB: Eliminate unnecessary heap allocations in the V2 histogram WAL decoder, reducing allocations by up to 50% and memory by up to 10% for deployments using native histograms with created-timestamp storage enabled (`--enable-feature=created-timestamp-zero-ingestion`). [#18813](prometheus/prometheus#18813)
- \[BUGFIX] Discovery/AWS: Fix failure when processing an AWS RDS cluster without instances. [#18845](prometheus/prometheus#18845)
- \[BUGFIX] Fix race condition in initTime that could cause ErrOutOfBounds. [#18629](prometheus/prometheus#18629)
- \[BUGFIX] PromQL: A range query whose `end` was not aligned to `step` caused subqueries inside it to evaluate past the parent's last actual step, inflating `peakSamples` in the query stats and against the `query.max-samples` limit, and wasting storage I/O reading samples that were never used in the result. [#18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: A range query containing an at-modifier-unsafe function over a range-vector with an `@` modifier (e.g. `predict_linear(metric[60s] @ T, X)`) silently under-counted `totalQueryableSamples` for steps after step 0. [#18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: Fix `fill_left`/`fill_right` producing missing samples in range queries when using `group_left`/`group_right`. [#18850](prometheus/prometheus#18850)
- \[BUGFIX] PromQL: Fix for resets() and changes() in anchored range extenders with histograms. [#18906](prometheus/prometheus#18906)
- \[BUGFIX] PromQL: Fix panic on `1[5m] smoothed` and similar expressions when extended range selectors are enabled. [#18764](prometheus/prometheus#18764)
- \[BUGFIX] PromQL: Fix panic when a `smoothed` instant vector selector produces no samples for a series. [#18943](prometheus/prometheus#18943)
- \[BUGFIX] PromQL: Fix panic when using a parenthesised plain number as an offset (e.g. `foo offset -(5)`). [#18768](prometheus/prometheus#18768)
- \[BUGFIX] promtool: Fix panic when parsing exposition text containing empty braces `{}`. Via prometheus/common v0.69.0. [#18949](prometheus/prometheus#18949)
- \[BUGFIX] Promtool: Fix `check healthy` and `check ready` when `--url` ends with a trailing slash. [#18854](prometheus/prometheus#18854)
- \[BUGFIX] Rules: Close PromQL query after each rule evaluation to ensure resources are released. [#18733](prometheus/prometheus#18733)
- \[BUGFIX] Scaleway SD: Resolve VPC/IPAM-only instances that have no legacy `private_ip` or `public_ip` field, but do have private NICs attached. [#18772](prometheus/prometheus#18772)
- \[BUGFIX] TSDB: Do not leak head series when an integer histogram append is rejected (e.g. out-of-order). [#18838](prometheus/prometheus#18838)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] TSDB: Fix chunk snapshot encoding for EncXOR2 chunks, preventing corruption on TSDB restart when EncXOR2-encoded series were present. [#18739](prometheus/prometheus#18739)
- \[BUGFIX] TSDB: Store a millisecond timestamp (not a WAL segment number) in walExpiries when a series is evicted via CompactStaleHead/CompactSelectedSeries, so the series's label record is correctly retained in the next WAL checkpoint and replays cleanly. [#18847](prometheus/prometheus#18847)
- \[BUGFIX] TSDB: Prevent loss of samples at the chunk-range boundary when CompactSelectedSeries (and CompactStaleHead) evict the series — the per-slice compaction loop now runs one more iteration so the boundary timestamp is captured in a block before the in-memory copy is removed. [#18849](prometheus/prometheus#18849)
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jul 7, 2026
##### [\`v3.13.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.13.0)

This is a Long Term Support [LTS](https://prometheus.io/docs/introduction/release-cycle/) release.

- \[SECURITY] UI: Bump `sanitize-html` to fix a cross-site scripting vulnerability (CVE-2026-44990). [#18697](prometheus/prometheus#18697)
- \[CHANGE] UI: Third-party npm dependency licenses are now embedded in the Prometheus binary and served at `/assets/third-party-licenses.txt`, replacing the `npm_licenses.tar.bz2` archive previously shipped in release tarballs and container images. [#18997](prometheus/prometheus#18997)
- \[CHANGE] API: Use SHA-256 instead of SHA-1 to generate rule group pagination tokens. [#18927](prometheus/prometheus#18927)
- \[CHANGE] HTTP clients: Credentials (Authorization header, basic auth, bearer token, OAuth2, configured headers) are no longer forwarded when following a redirect to a different host; affects scraping, remote read/write, alerting, and service discovery. Via prometheus/common v0.69.0 (CVE-2025-4673 CVE-2023-45289). [#18949](prometheus/prometheus#18949)
- \[CHANGE] promtool: Relative file paths in the file passed to `--http.config.file` are now resolved relative to that config file's directory instead of its parent directory. Via prometheus/common v0.69.0. [#18949](prometheus/prometheus#18949)
- \[CHANGE] PromQL: Rename the `min()` and `max()` duration-expression functions (experimental feature flag `experimental-duration-expr`) to `min_of()` and `max_of()` to avoid confusion with the `min` and `max` aggregate operators. [#18687](prometheus/prometheus#18687)
- \[FEATURE] API: Add experimental search endpoints to search metric names, label names, and label values. [#18573](prometheus/prometheus#18573)
- \[FEATURE] Discovery/AWS: Add ability to filter RDS instances. [#18859](prometheus/prometheus#18859)
- \[FEATURE] PromQL: Add `min_of(a, b)` and `max_of(a, b)` scalar experimental functions, returning the smaller or larger of two scalar values. [#18687](prometheus/prometheus#18687)
- \[FEATURE] PromQL: Add support for smoothed/anchored rate with native histograms. [#18564](prometheus/prometheus#18564)
- \[FEATURE] PromQL: Expose per-query `samplesRead` (and `samplesReadPerStep` with `stats=all` and the `promql-per-step-stats` feature flag) in the query stats response, and add the `prometheus_engine_query_samples_read_total` engine counter. `samplesRead` reflects storage I/O distinct from `totalQueryableSamples`, which counts samples loaded into the evaluator (and so over-counts when a sample is reused across multiple range-vector windows). [#18081](prometheus/prometheus#18081)
- \[FEATURE] Scrape: Add `__convert_classic_histograms_to_nhcb__` internal label to allow per-target override of `convert_classic_histograms_to_nhcb` scrape configuration via relabeling. [#18840](prometheus/prometheus#18840)
- \[FEATURE] TSDB: Add `storage.tsdb.chunk_encoding.floats` configuration field to select float chunk encoding (`xor` or `xor2`) at runtime, independently of the `--enable-feature=xor2-encoding` flag. [#18769](prometheus/prometheus#18769)
- \[FEATURE] remote\_write: Add Certificate support for ingesting data into an Azure Monitor Workspace. [#18217](prometheus/prometheus#18217)
- \[FEATURE] Scrape: Add `__always_scrape_classic_histograms__` and `__scrape_native_histograms__` internal labels to allow per-target override of the `always_scrape_classic_histograms` and `scrape_native_histograms` scrape configuration via relabeling. [#18929](prometheus/prometheus#18929)
- \[ENHANCEMENT] Release: Container images are now also published to the GitHub Container Registry (ghcr.io). [#18791](prometheus/prometheus#18791)
- \[ENHANCEMENT] PromQL: Prettify `fill_left(x) fill_right(x)` as `fill(x)` when both fill values are equal. [#18851](prometheus/prometheus#18851)
- \[ENHANCEMENT] UI: Improve autocompletion after closing a function bracket. [#18894](prometheus/prometheus#18894)
- \[PERF] Labels: Add case-insensitive prefix matching to speed up evaluation of long case-insensitive regular expressions (up to \~2x faster). [#18540](prometheus/prometheus#18540)
- \[PERF] TSDB: Reduce per-sample overhead in chunk population, speeding up affected queries by \~12-15% in benchmarks. [#18699](prometheus/prometheus#18699)
- \[PERF] TSDB: Eliminate unnecessary heap allocations in the V2 histogram WAL decoder, reducing allocations by up to 50% and memory by up to 10% for deployments using native histograms with created-timestamp storage enabled (`--enable-feature=created-timestamp-zero-ingestion`). [#18813](prometheus/prometheus#18813)
- \[BUGFIX] Discovery/AWS: Fix failure when processing an AWS RDS cluster without instances. [#18845](prometheus/prometheus#18845)
- \[BUGFIX] Fix race condition in initTime that could cause ErrOutOfBounds. [#18629](prometheus/prometheus#18629)
- \[BUGFIX] PromQL: A range query whose `end` was not aligned to `step` caused subqueries inside it to evaluate past the parent's last actual step, inflating `peakSamples` in the query stats and against the `query.max-samples` limit, and wasting storage I/O reading samples that were never used in the result. [#18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: A range query containing an at-modifier-unsafe function over a range-vector with an `@` modifier (e.g. `predict_linear(metric[60s] @ T, X)`) silently under-counted `totalQueryableSamples` for steps after step 0. [#18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: Fix `fill_left`/`fill_right` producing missing samples in range queries when using `group_left`/`group_right`. [#18850](prometheus/prometheus#18850)
- \[BUGFIX] PromQL: Fix for resets() and changes() in anchored range extenders with histograms. [#18906](prometheus/prometheus#18906)
- \[BUGFIX] PromQL: Fix panic on `1[5m] smoothed` and similar expressions when extended range selectors are enabled. [#18764](prometheus/prometheus#18764)
- \[BUGFIX] PromQL: Fix panic when a `smoothed` instant vector selector produces no samples for a series. [#18943](prometheus/prometheus#18943)
- \[BUGFIX] PromQL: Fix panic when using a parenthesised plain number as an offset (e.g. `foo offset -(5)`). [#18768](prometheus/prometheus#18768)
- \[BUGFIX] promtool: Fix panic when parsing exposition text containing empty braces `{}`. Via prometheus/common v0.69.0. [#18949](prometheus/prometheus#18949)
- \[BUGFIX] Promtool: Fix `check healthy` and `check ready` when `--url` ends with a trailing slash. [#18854](prometheus/prometheus#18854)
- \[BUGFIX] Rules: Close PromQL query after each rule evaluation to ensure resources are released. [#18733](prometheus/prometheus#18733)
- \[BUGFIX] Scaleway SD: Resolve VPC/IPAM-only instances that have no legacy `private_ip` or `public_ip` field, but do have private NICs attached. [#18772](prometheus/prometheus#18772)
- \[BUGFIX] TSDB: Do not leak head series when an integer histogram append is rejected (e.g. out-of-order). [#18838](prometheus/prometheus#18838)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] TSDB: Fix chunk snapshot encoding for EncXOR2 chunks, preventing corruption on TSDB restart when EncXOR2-encoded series were present. [#18739](prometheus/prometheus#18739)
- \[BUGFIX] TSDB: Store a millisecond timestamp (not a WAL segment number) in walExpiries when a series is evicted via CompactStaleHead/CompactSelectedSeries, so the series's label record is correctly retained in the next WAL checkpoint and replays cleanly. [#18847](prometheus/prometheus#18847)
- \[BUGFIX] TSDB: Prevent loss of samples at the chunk-range boundary when CompactSelectedSeries (and CompactStaleHead) evict the series — the per-slice compaction loop now runs one more iteration so the boundary timestamp is captured in a block before the in-memory copy is removed. [#18849](prometheus/prometheus#18849)
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jul 7, 2026
##### [\`v3.13.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.13.0)

This is a Long Term Support [LTS](https://prometheus.io/docs/introduction/release-cycle/) release.

- \[SECURITY] UI: Bump `sanitize-html` to fix a cross-site scripting vulnerability (CVE-2026-44990). [#18697](prometheus/prometheus#18697)
- \[CHANGE] UI: Third-party npm dependency licenses are now embedded in the Prometheus binary and served at `/assets/third-party-licenses.txt`, replacing the `npm_licenses.tar.bz2` archive previously shipped in release tarballs and container images. [#18997](prometheus/prometheus#18997)
- \[CHANGE] API: Use SHA-256 instead of SHA-1 to generate rule group pagination tokens. [#18927](prometheus/prometheus#18927)
- \[CHANGE] HTTP clients: Credentials (Authorization header, basic auth, bearer token, OAuth2, configured headers) are no longer forwarded when following a redirect to a different host; affects scraping, remote read/write, alerting, and service discovery. Via prometheus/common v0.69.0 (CVE-2025-4673 CVE-2023-45289). [#18949](prometheus/prometheus#18949)
- \[CHANGE] promtool: Relative file paths in the file passed to `--http.config.file` are now resolved relative to that config file's directory instead of its parent directory. Via prometheus/common v0.69.0. [#18949](prometheus/prometheus#18949)
- \[CHANGE] PromQL: Rename the `min()` and `max()` duration-expression functions (experimental feature flag `experimental-duration-expr`) to `min_of()` and `max_of()` to avoid confusion with the `min` and `max` aggregate operators. [#18687](prometheus/prometheus#18687)
- \[FEATURE] API: Add experimental search endpoints to search metric names, label names, and label values. [#18573](prometheus/prometheus#18573)
- \[FEATURE] Discovery/AWS: Add ability to filter RDS instances. [#18859](prometheus/prometheus#18859)
- \[FEATURE] PromQL: Add `min_of(a, b)` and `max_of(a, b)` scalar experimental functions, returning the smaller or larger of two scalar values. [#18687](prometheus/prometheus#18687)
- \[FEATURE] PromQL: Add support for smoothed/anchored rate with native histograms. [#18564](prometheus/prometheus#18564)
- \[FEATURE] PromQL: Expose per-query `samplesRead` (and `samplesReadPerStep` with `stats=all` and the `promql-per-step-stats` feature flag) in the query stats response, and add the `prometheus_engine_query_samples_read_total` engine counter. `samplesRead` reflects storage I/O distinct from `totalQueryableSamples`, which counts samples loaded into the evaluator (and so over-counts when a sample is reused across multiple range-vector windows). [#18081](prometheus/prometheus#18081)
- \[FEATURE] Scrape: Add `__convert_classic_histograms_to_nhcb__` internal label to allow per-target override of `convert_classic_histograms_to_nhcb` scrape configuration via relabeling. [#18840](prometheus/prometheus#18840)
- \[FEATURE] TSDB: Add `storage.tsdb.chunk_encoding.floats` configuration field to select float chunk encoding (`xor` or `xor2`) at runtime, independently of the `--enable-feature=xor2-encoding` flag. [#18769](prometheus/prometheus#18769)
- \[FEATURE] remote\_write: Add Certificate support for ingesting data into an Azure Monitor Workspace. [#18217](prometheus/prometheus#18217)
- \[FEATURE] Scrape: Add `__always_scrape_classic_histograms__` and `__scrape_native_histograms__` internal labels to allow per-target override of the `always_scrape_classic_histograms` and `scrape_native_histograms` scrape configuration via relabeling. [#18929](prometheus/prometheus#18929)
- \[ENHANCEMENT] Release: Container images are now also published to the GitHub Container Registry (ghcr.io). [#18791](prometheus/prometheus#18791)
- \[ENHANCEMENT] PromQL: Prettify `fill_left(x) fill_right(x)` as `fill(x)` when both fill values are equal. [#18851](prometheus/prometheus#18851)
- \[ENHANCEMENT] UI: Improve autocompletion after closing a function bracket. [#18894](prometheus/prometheus#18894)
- \[PERF] Labels: Add case-insensitive prefix matching to speed up evaluation of long case-insensitive regular expressions (up to \~2x faster). [#18540](prometheus/prometheus#18540)
- \[PERF] TSDB: Reduce per-sample overhead in chunk population, speeding up affected queries by \~12-15% in benchmarks. [#18699](prometheus/prometheus#18699)
- \[PERF] TSDB: Eliminate unnecessary heap allocations in the V2 histogram WAL decoder, reducing allocations by up to 50% and memory by up to 10% for deployments using native histograms with created-timestamp storage enabled (`--enable-feature=created-timestamp-zero-ingestion`). [#18813](prometheus/prometheus#18813)
- \[BUGFIX] Discovery/AWS: Fix failure when processing an AWS RDS cluster without instances. [#18845](prometheus/prometheus#18845)
- \[BUGFIX] Fix race condition in initTime that could cause ErrOutOfBounds. [#18629](prometheus/prometheus#18629)
- \[BUGFIX] PromQL: A range query whose `end` was not aligned to `step` caused subqueries inside it to evaluate past the parent's last actual step, inflating `peakSamples` in the query stats and against the `query.max-samples` limit, and wasting storage I/O reading samples that were never used in the result. [#18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: A range query containing an at-modifier-unsafe function over a range-vector with an `@` modifier (e.g. `predict_linear(metric[60s] @ T, X)`) silently under-counted `totalQueryableSamples` for steps after step 0. [#18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: Fix `fill_left`/`fill_right` producing missing samples in range queries when using `group_left`/`group_right`. [#18850](prometheus/prometheus#18850)
- \[BUGFIX] PromQL: Fix for resets() and changes() in anchored range extenders with histograms. [#18906](prometheus/prometheus#18906)
- \[BUGFIX] PromQL: Fix panic on `1[5m] smoothed` and similar expressions when extended range selectors are enabled. [#18764](prometheus/prometheus#18764)
- \[BUGFIX] PromQL: Fix panic when a `smoothed` instant vector selector produces no samples for a series. [#18943](prometheus/prometheus#18943)
- \[BUGFIX] PromQL: Fix panic when using a parenthesised plain number as an offset (e.g. `foo offset -(5)`). [#18768](prometheus/prometheus#18768)
- \[BUGFIX] promtool: Fix panic when parsing exposition text containing empty braces `{}`. Via prometheus/common v0.69.0. [#18949](prometheus/prometheus#18949)
- \[BUGFIX] Promtool: Fix `check healthy` and `check ready` when `--url` ends with a trailing slash. [#18854](prometheus/prometheus#18854)
- \[BUGFIX] Rules: Close PromQL query after each rule evaluation to ensure resources are released. [#18733](prometheus/prometheus#18733)
- \[BUGFIX] Scaleway SD: Resolve VPC/IPAM-only instances that have no legacy `private_ip` or `public_ip` field, but do have private NICs attached. [#18772](prometheus/prometheus#18772)
- \[BUGFIX] TSDB: Do not leak head series when an integer histogram append is rejected (e.g. out-of-order). [#18838](prometheus/prometheus#18838)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] TSDB: Fix chunk snapshot encoding for EncXOR2 chunks, preventing corruption on TSDB restart when EncXOR2-encoded series were present. [#18739](prometheus/prometheus#18739)
- \[BUGFIX] TSDB: Store a millisecond timestamp (not a WAL segment number) in walExpiries when a series is evicted via CompactStaleHead/CompactSelectedSeries, so the series's label record is correctly retained in the next WAL checkpoint and replays cleanly. [#18847](prometheus/prometheus#18847)
- \[BUGFIX] TSDB: Prevent loss of samples at the chunk-range boundary when CompactSelectedSeries (and CompactStaleHead) evict the series — the per-slice compaction loop now runs one more iteration so the boundary timestamp is captured in a block before the in-memory copy is removed. [#18849](prometheus/prometheus#18849)
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jul 7, 2026
##### [\`v3.13.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.13.0)

This is a Long Term Support [LTS](https://prometheus.io/docs/introduction/release-cycle/) release.

- \[SECURITY] UI: Bump `sanitize-html` to fix a cross-site scripting vulnerability (CVE-2026-44990). [#18697](prometheus/prometheus#18697)
- \[CHANGE] UI: Third-party npm dependency licenses are now embedded in the Prometheus binary and served at `/assets/third-party-licenses.txt`, replacing the `npm_licenses.tar.bz2` archive previously shipped in release tarballs and container images. [#18997](prometheus/prometheus#18997)
- \[CHANGE] API: Use SHA-256 instead of SHA-1 to generate rule group pagination tokens. [#18927](prometheus/prometheus#18927)
- \[CHANGE] HTTP clients: Credentials (Authorization header, basic auth, bearer token, OAuth2, configured headers) are no longer forwarded when following a redirect to a different host; affects scraping, remote read/write, alerting, and service discovery. Via prometheus/common v0.69.0 (CVE-2025-4673 CVE-2023-45289). [#18949](prometheus/prometheus#18949)
- \[CHANGE] promtool: Relative file paths in the file passed to `--http.config.file` are now resolved relative to that config file's directory instead of its parent directory. Via prometheus/common v0.69.0. [#18949](prometheus/prometheus#18949)
- \[CHANGE] PromQL: Rename the `min()` and `max()` duration-expression functions (experimental feature flag `experimental-duration-expr`) to `min_of()` and `max_of()` to avoid confusion with the `min` and `max` aggregate operators. [#18687](prometheus/prometheus#18687)
- \[FEATURE] API: Add experimental search endpoints to search metric names, label names, and label values. [#18573](prometheus/prometheus#18573)
- \[FEATURE] Discovery/AWS: Add ability to filter RDS instances. [#18859](prometheus/prometheus#18859)
- \[FEATURE] PromQL: Add `min_of(a, b)` and `max_of(a, b)` scalar experimental functions, returning the smaller or larger of two scalar values. [#18687](prometheus/prometheus#18687)
- \[FEATURE] PromQL: Add support for smoothed/anchored rate with native histograms. [#18564](prometheus/prometheus#18564)
- \[FEATURE] PromQL: Expose per-query `samplesRead` (and `samplesReadPerStep` with `stats=all` and the `promql-per-step-stats` feature flag) in the query stats response, and add the `prometheus_engine_query_samples_read_total` engine counter. `samplesRead` reflects storage I/O distinct from `totalQueryableSamples`, which counts samples loaded into the evaluator (and so over-counts when a sample is reused across multiple range-vector windows). [#18081](prometheus/prometheus#18081)
- \[FEATURE] Scrape: Add `__convert_classic_histograms_to_nhcb__` internal label to allow per-target override of `convert_classic_histograms_to_nhcb` scrape configuration via relabeling. [#18840](prometheus/prometheus#18840)
- \[FEATURE] TSDB: Add `storage.tsdb.chunk_encoding.floats` configuration field to select float chunk encoding (`xor` or `xor2`) at runtime, independently of the `--enable-feature=xor2-encoding` flag. [#18769](prometheus/prometheus#18769)
- \[FEATURE] remote\_write: Add Certificate support for ingesting data into an Azure Monitor Workspace. [#18217](prometheus/prometheus#18217)
- \[FEATURE] Scrape: Add `__always_scrape_classic_histograms__` and `__scrape_native_histograms__` internal labels to allow per-target override of the `always_scrape_classic_histograms` and `scrape_native_histograms` scrape configuration via relabeling. [#18929](prometheus/prometheus#18929)
- \[ENHANCEMENT] Release: Container images are now also published to the GitHub Container Registry (ghcr.io). [#18791](prometheus/prometheus#18791)
- \[ENHANCEMENT] PromQL: Prettify `fill_left(x) fill_right(x)` as `fill(x)` when both fill values are equal. [#18851](prometheus/prometheus#18851)
- \[ENHANCEMENT] UI: Improve autocompletion after closing a function bracket. [#18894](prometheus/prometheus#18894)
- \[PERF] Labels: Add case-insensitive prefix matching to speed up evaluation of long case-insensitive regular expressions (up to \~2x faster). [#18540](prometheus/prometheus#18540)
- \[PERF] TSDB: Reduce per-sample overhead in chunk population, speeding up affected queries by \~12-15% in benchmarks. [#18699](prometheus/prometheus#18699)
- \[PERF] TSDB: Eliminate unnecessary heap allocations in the V2 histogram WAL decoder, reducing allocations by up to 50% and memory by up to 10% for deployments using native histograms with created-timestamp storage enabled (`--enable-feature=created-timestamp-zero-ingestion`). [#18813](prometheus/prometheus#18813)
- \[BUGFIX] Discovery/AWS: Fix failure when processing an AWS RDS cluster without instances. [#18845](prometheus/prometheus#18845)
- \[BUGFIX] Fix race condition in initTime that could cause ErrOutOfBounds. [#18629](prometheus/prometheus#18629)
- \[BUGFIX] PromQL: A range query whose `end` was not aligned to `step` caused subqueries inside it to evaluate past the parent's last actual step, inflating `peakSamples` in the query stats and against the `query.max-samples` limit, and wasting storage I/O reading samples that were never used in the result. [#18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: A range query containing an at-modifier-unsafe function over a range-vector with an `@` modifier (e.g. `predict_linear(metric[60s] @ T, X)`) silently under-counted `totalQueryableSamples` for steps after step 0. [#18081](prometheus/prometheus#18081)
- \[BUGFIX] PromQL: Fix `fill_left`/`fill_right` producing missing samples in range queries when using `group_left`/`group_right`. [#18850](prometheus/prometheus#18850)
- \[BUGFIX] PromQL: Fix for resets() and changes() in anchored range extenders with histograms. [#18906](prometheus/prometheus#18906)
- \[BUGFIX] PromQL: Fix panic on `1[5m] smoothed` and similar expressions when extended range selectors are enabled. [#18764](prometheus/prometheus#18764)
- \[BUGFIX] PromQL: Fix panic when a `smoothed` instant vector selector produces no samples for a series. [#18943](prometheus/prometheus#18943)
- \[BUGFIX] PromQL: Fix panic when using a parenthesised plain number as an offset (e.g. `foo offset -(5)`). [#18768](prometheus/prometheus#18768)
- \[BUGFIX] promtool: Fix panic when parsing exposition text containing empty braces `{}`. Via prometheus/common v0.69.0. [#18949](prometheus/prometheus#18949)
- \[BUGFIX] Promtool: Fix `check healthy` and `check ready` when `--url` ends with a trailing slash. [#18854](prometheus/prometheus#18854)
- \[BUGFIX] Rules: Close PromQL query after each rule evaluation to ensure resources are released. [#18733](prometheus/prometheus#18733)
- \[BUGFIX] Scaleway SD: Resolve VPC/IPAM-only instances that have no legacy `private_ip` or `public_ip` field, but do have private NICs attached. [#18772](prometheus/prometheus#18772)
- \[BUGFIX] TSDB: Do not leak head series when an integer histogram append is rejected (e.g. out-of-order). [#18838](prometheus/prometheus#18838)
- \[BUGFIX] UI: Escape label values offered by PromQL autocomplete. [#18658](prometheus/prometheus#18658)
- \[BUGFIX] TSDB: Fix chunk snapshot encoding for EncXOR2 chunks, preventing corruption on TSDB restart when EncXOR2-encoded series were present. [#18739](prometheus/prometheus#18739)
- \[BUGFIX] TSDB: Store a millisecond timestamp (not a WAL segment number) in walExpiries when a series is evicted via CompactStaleHead/CompactSelectedSeries, so the series's label record is correctly retained in the next WAL checkpoint and replays cleanly. [#18847](prometheus/prometheus#18847)
- \[BUGFIX] TSDB: Prevent loss of samples at the chunk-range boundary when CompactSelectedSeries (and CompactStaleHead) evict the series — the per-slice compaction loop now runs one more iteration so the boundary timestamp is captured in a block before the in-memory copy is removed. [#18849](prometheus/prometheus#18849)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Certificate-Based OAuth Support for Azure AD Remote Write

4 participants