-
Notifications
You must be signed in to change notification settings - Fork 715
fix: metrics can't search parquet in wal #8896
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Failed to generate code suggestions for PR |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Overview
Greptile Summary
This PR fixes metrics parquet file searching in WAL by properly handling partition time levels (daily vs hourly), and refactors cache control from negative no_cache to positive use_cache semantics.
Key Changes:
- Fixed WAL parquet search to respect
partition_time_levelsetting, correctly aligning time ranges to either daily or hourly boundaries based on stream configuration - Refactored cache control from
no_cachetouse_cachethroughout the codebase for clearer intent - Added cache update logic that allows queries with caching disabled to still update existing cache entries
- Implemented frontend support for
window.use_cacheoverride to control query caching behavior during development/debugging
Issue Found:
- Comment on line 100 in
src/service/promql/search/mod.rsincorrectly describes the logic (says "use_cache is true" when it should say "use_cache is false")
Confidence Score: 4/5
- This PR is safe to merge with one minor documentation fix needed
- The changes are well-structured and address the stated issues. The partition time level fix is correct and properly threaded through the codebase. The cache semantics refactor is consistent across all files. Only issue is an incorrect comment that doesn't affect functionality
- src/service/promql/search/mod.rs needs comment correction on line 100
Important Files Changed
File Analysis
| Filename | Score | Overview |
|---|---|---|
| src/service/search/grpc/wal.rs | 5/5 | Added partition_time_level parameter to correctly handle daily vs hourly partitioning when searching parquet files in WAL for metrics |
| src/service/promql/search/mod.rs | 4/5 | Changed from no_cache to use_cache semantics, added cache update logic; comment on line 100 needs correction |
| src/service/promql/search/cache/mod.rs | 5/5 | Added update parameter to set() function to control whether cache should be updated even when already covered |
| web/src/services/search.ts | 5/5 | Added support for window.use_cache override in frontend to control query caching behavior |
Sequence Diagram
sequenceDiagram
participant Client as Web Client
participant HTTP as HTTP Handler
participant Search as PromQL Search
participant Cache as Cache Service
participant Cluster as Cluster RPC
participant WAL as WAL Search
participant Parquet as Parquet Files
Client->>HTTP: query_range(query, use_cache)
HTTP->>Search: search_in_cluster(req)
Note over Search: Check if cache_disabled<br/>(!use_cache || !cache_enabled)
alt Cache enabled and use_cache=true
Search->>Cache: get(query, start, end)
Cache-->>Search: cached data (if exists)
end
Search->>Cluster: distribute query to nodes
Cluster->>WAL: search_parquet(query)
Note over WAL: Get partition_time_level<br/>from stream settings
alt Daily Partitioning
WAL->>WAL: Align to DAY_MICRO_SECS
else Hourly Partitioning
WAL->>WAL: Align to HOUR_MICRO_SECS
end
WAL->>Parquet: scan files in time range
Parquet-->>WAL: record batches
WAL-->>Cluster: results
Cluster-->>Search: aggregated results
alt Cache enabled
Search->>Cache: set(query, results, update=!cache_disabled)
Note over Cache: Update cache if needed
end
Search-->>HTTP: query results
HTTP-->>Client: response
12 files reviewed, 1 comment
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 182 | 168 | 0 | 13 | 1 | 92% | 2m 39s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 121 | 110 | 0 | 10 | 1 | 91% | 2m 32s |
Testdino Test Results
|
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 365 | 339 | 0 | 19 | 7 | 93% | 4m 38s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 365 | 343 | 0 | 19 | 3 | 94% | 4m 38s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 365 | 343 | 0 | 19 | 3 | 94% | 4m 40s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 365 | 343 | 0 | 19 | 3 | 94% | 4m 39s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 365 | 344 | 0 | 19 | 2 | 94% | 5m 2s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 366 | 344 | 0 | 19 | 3 | 94% | 4m 39s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 366 | 345 | 0 | 19 | 2 | 94% | 4m 39s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 366 | 345 | 0 | 19 | 2 | 94% | 4m 39s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 366 | 344 | 0 | 19 | 3 | 94% | 4m 38s |
window.use_cache=falsefor metrics to skip query cache