Skip to content

model/labels: improve performance of regex matchers like .*-.*-.*#17707

Merged
bboreham merged 4 commits intoprometheus:mainfrom
charleskorn:charleskorn/optimise-chained-regexp-contains
Jan 8, 2026
Merged

model/labels: improve performance of regex matchers like .*-.*-.*#17707
bboreham merged 4 commits intoprometheus:mainfrom
charleskorn:charleskorn/optimise-chained-regexp-contains

Conversation

@charleskorn
Copy link
Copy Markdown
Contributor

@charleskorn charleskorn commented Dec 17, 2025

Which issue(s) does the PR fix:

#14173 introduced an optimisation to better handle regex patterns like .*-.*-.*. It identifies strings the pattern cannot possibly match (because they do not contain all of the literal values) and returns false from MatchString early.

However, if the string does contain all literal values, then the Go regex engine is used to confirm that the string does match the pattern. But this is not necessary in the case where the start and end of the pattern is .* and everything in between is either a literal or .*: if the string contains all of the literals in order, then it matches the pattern, and invoking Go's regex engine to confirm this is unnecessary and quite slow.

So this PR introduces a fast path for patterns like this.

I think we could apply something similar to patterns that don't start or end with .*, but this would be a slightly more complex change, and I wanted to keep this PR as small as possible while I get my head around FastRegexMatcher.

The existing FastRegexMatcher benchmark didn't cover the case where a pattern like .*-.*-.* matches the string, so I added a specific benchmark for this case:

goos: darwin
goarch: arm64
pkg: github.com/prometheus/prometheus/model/labels
cpu: Apple M1 Pro
                                        │   main.txt   │               pr.txt               │
                                        │    sec/op    │   sec/op     vs base               │
FastRegexMatcher_ConcatenatedPattern-10   3327.5n ± 1%   126.3n ± 2%  -96.20% (p=0.002 n=6)

                                        │  main.txt  │            pr.txt             │
                                        │    B/op    │    B/op     vs base           │
FastRegexMatcher_ConcatenatedPattern-10   0.000 ± 0%   0.000 ± 0%  ~ (p=1.000 n=6) ¹
¹ all samples are equal

                                        │  main.txt  │            pr.txt             │
                                        │ allocs/op  │ allocs/op   vs base           │
FastRegexMatcher_ConcatenatedPattern-10   0.000 ± 0%   0.000 ± 0%  ~ (p=1.000 n=6) ¹
¹ all samples are equal

Does this PR introduce a user-facing change?

[PERF] Improve performance of regex matchers like `.*-.*-.*`

@charleskorn charleskorn force-pushed the charleskorn/optimise-chained-regexp-contains branch from 3ad2a3a to cb5120e Compare December 17, 2025 04:23
@krajorama krajorama requested a review from bboreham December 17, 2025 13:32
Copy link
Copy Markdown
Member

@bboreham bboreham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Broadly fine; one question about the structure.

Also could you post your benchmark results for the cases added to BenchmarkFastRegexMatcher ?

@charleskorn charleskorn force-pushed the charleskorn/optimise-chained-regexp-contains branch from a4dc256 to db9cc23 Compare January 8, 2026 01:40
@charleskorn
Copy link
Copy Markdown
Contributor Author

Also could you post your benchmark results for the cases added to BenchmarkFastRegexMatcher ?

That benchmark doesn't show much impact given most of the random test strings will not match the patterns, so they don't benefit from this optimisation:

goos: darwin
goarch: arm64
pkg: github.com/prometheus/prometheus/model/labels
cpu: Apple M1 Pro
                                                        │   before.txt   │              after.txt              │
                                                        │     sec/op     │    sec/op     vs base               │
FastRegexMatcher/.*-.*-.*-.*-.*-10                         192.8n ±   0%   192.2n ±  0%   -0.31% (p=0.041 n=6)
FastRegexMatcher/.+-.*-.*-.*-.+-10                         192.1n ±   0%   192.2n ±  0%        ~ (p=0.372 n=6)
FastRegexMatcher/-.*-.*-.*-.*-10                           101.4n ±   1%   101.2n ±  0%        ~ (p=0.054 n=6)
FastRegexMatcher/.*-.*-.*-.*--10                           119.0n ±   0%   119.3n ±  0%   +0.21% (p=0.024 n=6)

I added the BenchmarkFastRegexMatcher_ConcatenatedPattern benchmark for this reason (see PR description for results).

Copy link
Copy Markdown
Member

@bboreham bboreham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that seems better to me.

@bboreham bboreham merged commit a919e6d into prometheus:main Jan 8, 2026
28 checks passed
56quarters added a commit to 56quarters/prometheus that referenced this pull request Jan 9, 2026
This change fixes an issue introduced in prometheus#17707. When a regex
with a wildcard, literal, and final wildcard surounded by a
capture group was parsed - the capture group was not removed
first preventing `optimizeConcatRegex` from running.

Found via fuzz testing.

Signed-off-by: Nick Pillitteri <[email protected]>
56quarters added a commit to 56quarters/prometheus that referenced this pull request Jan 9, 2026
This change fixes an issue introduced in prometheus#17707. When a regex
with a wildcard, literal, and final wildcard surounded by a
capture group was parsed - the capture group was not removed
first preventing `optimizeConcatRegex` from running.

Found via fuzz testing.

Signed-off-by: Nick Pillitteri <[email protected]>
bboreham pushed a commit that referenced this pull request Jan 12, 2026
This change fixes an issue introduced in #17707. When a regex
with a wildcard, literal, and final wildcard surounded by a
capture group was parsed - the capture group was not removed
first preventing `optimizeConcatRegex` from running.

Found via fuzz testing.

Signed-off-by: Nick Pillitteri <[email protected]>
charleskorn added a commit to grafana/mimir-prometheus that referenced this pull request Jan 15, 2026
renovate bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Mar 12, 2026
##### [\`v3.10.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.10.0)

Prometheus now offers a distroless Docker image variant alongside the default
busybox image. The distroless variant provides enhanced security with a minimal
base image, uses UID/GID 65532 (nonroot) instead of nobody, and removes the
VOLUME declaration. Both variants are available with `-busybox` and `-distroless`
tag suffixes (e.g., `prom/prometheus:latest-busybox`, `prom/prometheus:latest-distroless`).
The busybox image remains the default with no suffix for backwards compatibility
(e.g., `prom/prometheus:latest` points to the busybox variant).

For users migrating existing **named** volumes from the busybox image to the distroless variant, the ownership can be adjusted with:

```
docker run --rm -v prometheus-data:/prometheus alpine chown -R 65532:65532 /prometheus
```

Then, the container can be started with the old volume with:

```
docker run -v prometheus-data:/prometheus prom/prometheus:latest-distroless
```

User migrating from bind mounts might need to ajust permissions too, depending on their setup.

- \[CHANGE] Alerting: Add `alertmanager` dimension to following metrics: `prometheus_notifications_dropped_total`, `prometheus_notifications_queue_capacity`, `prometheus_notifications_queue_length`. [#16355](prometheus/prometheus#16355)
- \[CHANGE] UI: Hide expanded alert annotations by default, enabling more information density on the `/alerts` page. [#17611](prometheus/prometheus#17611)
- \[FEATURE] AWS SD: Add MSK Role. [#17600](prometheus/prometheus#17600)
- \[FEATURE] PromQL: Add `fill()` / `fill_left()` / `fill_right()` binop modifiers for specifying default values for missing series. [#17644](prometheus/prometheus#17644)
- \[FEATURE] Web: Add OpenAPI 3.2 specification for the HTTP API at `/api/v1/openapi.yaml`. [#17825](prometheus/prometheus#17825)
- \[FEATURE] Dockerfile: Add distroless image variant using UID/GID 65532 and no VOLUME declaration. Busybox image remains default. [#17876](prometheus/prometheus#17876)
- \[FEATURE] Web: Add on-demand wall time profiling under `<URL>/debug/pprof/fgprof`. [#18027](prometheus/prometheus#18027)
- \[ENHANCEMENT] PromQL: Add more detail to histogram quantile monotonicity info annotations. [#15578](prometheus/prometheus#15578)
- \[ENHANCEMENT] Alerting: Independent alertmanager sendloops. [#16355](prometheus/prometheus#16355)
- \[ENHANCEMENT] TSDB: Experimental support for early compaction of stale series in the memory with configurable threshold `stale_series_compaction_threshold` in the config file. [#16929](prometheus/prometheus#16929)
- \[ENHANCEMENT] Service Discovery: Service discoveries are now removable from the Prometheus binary through the Go build tag `remove_all_sd` and individual service discoveries can be re-added with the build tags `enable_<sd name>_sd`. Users can build a custom Prometheus with only the necessary SDs for a smaller binary size. [#17736](prometheus/prometheus#17736)
- \[ENHANCEMENT] Promtool: Support promql syntax features `promql-duration-expr` and `promql-extended-range-selectors`. [#17926](prometheus/prometheus#17926)
- \[PERF] PromQL: Avoid unnecessary label extraction in PromQL functions. [#17676](prometheus/prometheus#17676)
- \[PERF] PromQL: Improve performance of regex matchers like `.*-.*-.*`. [#17707](prometheus/prometheus#17707)
- \[PERF] OTLP: Add label caching for OTLP-to-Prometheus conversion to reduce allocations and improve latency. [#17860](prometheus/prometheus#17860)
- \[PERF] API: Compute `/api/v1/targets/relabel_steps` in a single pass instead of re-running relabeling for each prefix. [#17969](prometheus/prometheus#17969)
- \[PERF] tsdb: Optimize LabelValues intersection performance for matchers. [#18069](prometheus/prometheus#18069)
- \[BUGFIX] PromQL: Prevent query strings containing only UTF-8 continuation bytes from crashing Prometheus. [#17735](prometheus/prometheus#17735)
- \[BUGFIX] Web: Fix missing `X-Prometheus-Stopping` header for `/-/ready` endpoint in `NotReady` state. [#17795](prometheus/prometheus#17795)
- \[BUGFIX] PromQL: Fix PromQL `info()` function returning empty results when filtering by a label that exists on both the input metric and `target_info`. [#17817](prometheus/prometheus#17817)
- \[BUGFIX] TSDB: Fix a bug during exemplar buffer grow/shrink that could cause exemplars to be incorrectly discarded. [#17863](prometheus/prometheus#17863)
- \[BUGFIX] UI: Fix broken graph display after page reload, due to broken Y axis min encoding/decoding. [#17869](prometheus/prometheus#17869)
- \[BUGFIX] TSDB: Fix memory leaks in buffer pools by clearing reference fields (Labels, Histogram pointers, metadata strings) before returning buffers to pools. [#17879](prometheus/prometheus#17879)
- \[BUGFIX] PromQL: info function: fix series without identifying labels not being returned. [#17898](prometheus/prometheus#17898)
- \[BUGFIX] OTLP: Filter `__name__` from OTLP attributes to prevent duplicate labels. [#17917](prometheus/prometheus#17917)
- \[BUGFIX] TSDB: Fix division by zero when computing stale series ratio with empty head. [#17952](prometheus/prometheus#17952)
- \[BUGFIX] OTLP: Fix potential silent data loss for sum metrics. [#17954](prometheus/prometheus#17954)
- \[BUGFIX] PromQL: Fix smoothed interpolation across counter resets. [#17988](prometheus/prometheus#17988)
- \[BUGFIX] PromQL: Fix panic with `@` modifier on empty ranges. [#18020](prometheus/prometheus#18020)
- \[BUGFIX] PromQL: Fix `avg_over_time` for a single native histogram. [#18058](prometheus/prometheus#18058)
renovate bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Mar 13, 2026
##### [\`v3.10.0\`](https://github.com/prometheus/prometheus/releases/tag/v3.10.0)

Prometheus now offers a distroless Docker image variant alongside the default
busybox image. The distroless variant provides enhanced security with a minimal
base image, uses UID/GID 65532 (nonroot) instead of nobody, and removes the
VOLUME declaration. Both variants are available with `-busybox` and `-distroless`
tag suffixes (e.g., `prom/prometheus:latest-busybox`, `prom/prometheus:latest-distroless`).
The busybox image remains the default with no suffix for backwards compatibility
(e.g., `prom/prometheus:latest` points to the busybox variant).

For users migrating existing **named** volumes from the busybox image to the distroless variant, the ownership can be adjusted with:

```
docker run --rm -v prometheus-data:/prometheus alpine chown -R 65532:65532 /prometheus
```

Then, the container can be started with the old volume with:

```
docker run -v prometheus-data:/prometheus prom/prometheus:latest-distroless
```

User migrating from bind mounts might need to ajust permissions too, depending on their setup.

- \[CHANGE] Alerting: Add `alertmanager` dimension to following metrics: `prometheus_notifications_dropped_total`, `prometheus_notifications_queue_capacity`, `prometheus_notifications_queue_length`. [#16355](prometheus/prometheus#16355)
- \[CHANGE] UI: Hide expanded alert annotations by default, enabling more information density on the `/alerts` page. [#17611](prometheus/prometheus#17611)
- \[FEATURE] AWS SD: Add MSK Role. [#17600](prometheus/prometheus#17600)
- \[FEATURE] PromQL: Add `fill()` / `fill_left()` / `fill_right()` binop modifiers for specifying default values for missing series. [#17644](prometheus/prometheus#17644)
- \[FEATURE] Web: Add OpenAPI 3.2 specification for the HTTP API at `/api/v1/openapi.yaml`. [#17825](prometheus/prometheus#17825)
- \[FEATURE] Dockerfile: Add distroless image variant using UID/GID 65532 and no VOLUME declaration. Busybox image remains default. [#17876](prometheus/prometheus#17876)
- \[FEATURE] Web: Add on-demand wall time profiling under `<URL>/debug/pprof/fgprof`. [#18027](prometheus/prometheus#18027)
- \[ENHANCEMENT] PromQL: Add more detail to histogram quantile monotonicity info annotations. [#15578](prometheus/prometheus#15578)
- \[ENHANCEMENT] Alerting: Independent alertmanager sendloops. [#16355](prometheus/prometheus#16355)
- \[ENHANCEMENT] TSDB: Experimental support for early compaction of stale series in the memory with configurable threshold `stale_series_compaction_threshold` in the config file. [#16929](prometheus/prometheus#16929)
- \[ENHANCEMENT] Service Discovery: Service discoveries are now removable from the Prometheus binary through the Go build tag `remove_all_sd` and individual service discoveries can be re-added with the build tags `enable_<sd name>_sd`. Users can build a custom Prometheus with only the necessary SDs for a smaller binary size. [#17736](prometheus/prometheus#17736)
- \[ENHANCEMENT] Promtool: Support promql syntax features `promql-duration-expr` and `promql-extended-range-selectors`. [#17926](prometheus/prometheus#17926)
- \[PERF] PromQL: Avoid unnecessary label extraction in PromQL functions. [#17676](prometheus/prometheus#17676)
- \[PERF] PromQL: Improve performance of regex matchers like `.*-.*-.*`. [#17707](prometheus/prometheus#17707)
- \[PERF] OTLP: Add label caching for OTLP-to-Prometheus conversion to reduce allocations and improve latency. [#17860](prometheus/prometheus#17860)
- \[PERF] API: Compute `/api/v1/targets/relabel_steps` in a single pass instead of re-running relabeling for each prefix. [#17969](prometheus/prometheus#17969)
- \[PERF] tsdb: Optimize LabelValues intersection performance for matchers. [#18069](prometheus/prometheus#18069)
- \[BUGFIX] PromQL: Prevent query strings containing only UTF-8 continuation bytes from crashing Prometheus. [#17735](prometheus/prometheus#17735)
- \[BUGFIX] Web: Fix missing `X-Prometheus-Stopping` header for `/-/ready` endpoint in `NotReady` state. [#17795](prometheus/prometheus#17795)
- \[BUGFIX] PromQL: Fix PromQL `info()` function returning empty results when filtering by a label that exists on both the input metric and `target_info`. [#17817](prometheus/prometheus#17817)
- \[BUGFIX] TSDB: Fix a bug during exemplar buffer grow/shrink that could cause exemplars to be incorrectly discarded. [#17863](prometheus/prometheus#17863)
- \[BUGFIX] UI: Fix broken graph display after page reload, due to broken Y axis min encoding/decoding. [#17869](prometheus/prometheus#17869)
- \[BUGFIX] TSDB: Fix memory leaks in buffer pools by clearing reference fields (Labels, Histogram pointers, metadata strings) before returning buffers to pools. [#17879](prometheus/prometheus#17879)
- \[BUGFIX] PromQL: info function: fix series without identifying labels not being returned. [#17898](prometheus/prometheus#17898)
- \[BUGFIX] OTLP: Filter `__name__` from OTLP attributes to prevent duplicate labels. [#17917](prometheus/prometheus#17917)
- \[BUGFIX] TSDB: Fix division by zero when computing stale series ratio with empty head. [#17952](prometheus/prometheus#17952)
- \[BUGFIX] OTLP: Fix potential silent data loss for sum metrics. [#17954](prometheus/prometheus#17954)
- \[BUGFIX] PromQL: Fix smoothed interpolation across counter resets. [#17988](prometheus/prometheus#17988)
- \[BUGFIX] PromQL: Fix panic with `@` modifier on empty ranges. [#18020](prometheus/prometheus#18020)
- \[BUGFIX] PromQL: Fix `avg_over_time` for a single native histogram. [#18058](prometheus/prometheus#18058)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants