Skip to content

Conversation

@hengfeiyang
Copy link
Contributor

@hengfeiyang hengfeiyang commented Oct 28, 2025

  • fixed: can't search parquet in wal for metrics
  • feat: support window.use_cache=false for metrics to skip query cache

@github-actions github-actions bot added the ☢️ Bug Something isn't working label Oct 28, 2025
@github-actions
Copy link
Contributor

Failed to generate code suggestions for PR

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Greptile Summary

This PR fixes metrics parquet file searching in WAL by properly handling partition time levels (daily vs hourly), and refactors cache control from negative no_cache to positive use_cache semantics.

Key Changes:

  • Fixed WAL parquet search to respect partition_time_level setting, correctly aligning time ranges to either daily or hourly boundaries based on stream configuration
  • Refactored cache control from no_cache to use_cache throughout the codebase for clearer intent
  • Added cache update logic that allows queries with caching disabled to still update existing cache entries
  • Implemented frontend support for window.use_cache override to control query caching behavior during development/debugging

Issue Found:

  • Comment on line 100 in src/service/promql/search/mod.rs incorrectly describes the logic (says "use_cache is true" when it should say "use_cache is false")

Confidence Score: 4/5

  • This PR is safe to merge with one minor documentation fix needed
  • The changes are well-structured and address the stated issues. The partition time level fix is correct and properly threaded through the codebase. The cache semantics refactor is consistent across all files. Only issue is an incorrect comment that doesn't affect functionality
  • src/service/promql/search/mod.rs needs comment correction on line 100

Important Files Changed

File Analysis

Filename Score Overview
src/service/search/grpc/wal.rs 5/5 Added partition_time_level parameter to correctly handle daily vs hourly partitioning when searching parquet files in WAL for metrics
src/service/promql/search/mod.rs 4/5 Changed from no_cache to use_cache semantics, added cache update logic; comment on line 100 needs correction
src/service/promql/search/cache/mod.rs 5/5 Added update parameter to set() function to control whether cache should be updated even when already covered
web/src/services/search.ts 5/5 Added support for window.use_cache override in frontend to control query caching behavior

Sequence Diagram

sequenceDiagram
    participant Client as Web Client
    participant HTTP as HTTP Handler
    participant Search as PromQL Search
    participant Cache as Cache Service
    participant Cluster as Cluster RPC
    participant WAL as WAL Search
    participant Parquet as Parquet Files

    Client->>HTTP: query_range(query, use_cache)
    HTTP->>Search: search_in_cluster(req)
    
    Note over Search: Check if cache_disabled<br/>(!use_cache || !cache_enabled)
    
    alt Cache enabled and use_cache=true
        Search->>Cache: get(query, start, end)
        Cache-->>Search: cached data (if exists)
    end
    
    Search->>Cluster: distribute query to nodes
    Cluster->>WAL: search_parquet(query)
    
    Note over WAL: Get partition_time_level<br/>from stream settings
    
    alt Daily Partitioning
        WAL->>WAL: Align to DAY_MICRO_SECS
    else Hourly Partitioning
        WAL->>WAL: Align to HOUR_MICRO_SECS
    end
    
    WAL->>Parquet: scan files in time range
    Parquet-->>WAL: record batches
    WAL-->>Cluster: results
    Cluster-->>Search: aggregated results
    
    alt Cache enabled
        Search->>Cache: set(query, results, update=!cache_disabled)
        Note over Cache: Update cache if needed
    end
    
    Search-->>HTTP: query results
    HTTP-->>Client: response
Loading

12 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

@testdino-playwright-reporter
Copy link

⚠️ Test Run Unstable


Author: hengfeiyang | Branch: feat/metrics-cache | Commit: 41be0ab

Testdino Test Results

Status Total Passed Failed Skipped Flaky Pass Rate Duration
All tests passed 182 168 0 13 1 92% 2m 39s

View Detailed Results

@testdino-playwright-reporter
Copy link

⚠️ Test Run Unstable


Author: hengfeiyang | Branch: feat/metrics-cache | Commit: 41be0ab

Testdino Test Results

Status Total Passed Failed Skipped Flaky Pass Rate Duration
All tests passed 121 110 0 10 1 91% 2m 32s

View Detailed Results

@testdino-playwright-reporter
Copy link

Testdino Test Results

Status Total Passed Failed Skipped Flaky Pass Rate Duration
All tests passed 8 8 0 0 0 100% 2m 36s

View Detailed Results

@testdino-playwright-reporter
Copy link

⚠️ Test Run Unstable


Author: hengfeiyang | Branch: feat/metrics-cache | Commit: 60b3cac

Testdino Test Results

Status Total Passed Failed Skipped Flaky Pass Rate Duration
All tests passed 365 339 0 19 7 93% 4m 38s

View Detailed Results

@testdino-playwright-reporter
Copy link

⚠️ Test Run Unstable


Author: hengfeiyang | Branch: feat/metrics-cache | Commit: 5bf18a8

Testdino Test Results

Status Total Passed Failed Skipped Flaky Pass Rate Duration
All tests passed 365 343 0 19 3 94% 4m 38s

View Detailed Results

@testdino-playwright-reporter
Copy link

⚠️ Test Run Unstable


Author: hengfeiyang | Branch: feat/metrics-cache | Commit: ddb1cf9

Testdino Test Results

Status Total Passed Failed Skipped Flaky Pass Rate Duration
All tests passed 365 343 0 19 3 94% 4m 40s

View Detailed Results

@testdino-playwright-reporter
Copy link

⚠️ Test Run Unstable


Author: hengfeiyang | Branch: feat/metrics-cache | Commit: ddb1cf9

Testdino Test Results

Status Total Passed Failed Skipped Flaky Pass Rate Duration
All tests passed 365 343 0 19 3 94% 4m 39s

View Detailed Results

@testdino-playwright-reporter
Copy link

⚠️ Test Run Unstable


Author: yaga-simha | Branch: feat/metrics-cache | Commit: 2eaa7ff

Testdino Test Results

Status Total Passed Failed Skipped Flaky Pass Rate Duration
All tests passed 365 344 0 19 2 94% 5m 2s

View Detailed Results

@testdino-playwright-reporter
Copy link

⚠️ Test Run Unstable


Author: hengfeiyang | Branch: feat/metrics-cache | Commit: 520544a

Testdino Test Results

Status Total Passed Failed Skipped Flaky Pass Rate Duration
All tests passed 366 344 0 19 3 94% 4m 39s

View Detailed Results

@testdino-playwright-reporter
Copy link

⚠️ Test Run Unstable


Author: hengfeiyang | Branch: feat/metrics-cache | Commit: 520544a

Testdino Test Results

Status Total Passed Failed Skipped Flaky Pass Rate Duration
All tests passed 366 345 0 19 2 94% 4m 39s

View Detailed Results

@testdino-playwright-reporter
Copy link

⚠️ Test Run Unstable


Author: hengfeiyang | Branch: feat/metrics-cache | Commit: 8bd579c

Testdino Test Results

Status Total Passed Failed Skipped Flaky Pass Rate Duration
All tests passed 366 345 0 19 2 94% 4m 39s

View Detailed Results

@hengfeiyang hengfeiyang merged commit f56b2b8 into main Oct 29, 2025
12 of 15 checks passed
@hengfeiyang hengfeiyang deleted the feat/metrics-cache branch October 29, 2025 05:37
@testdino-playwright-reporter
Copy link

⚠️ Test Run Unstable


Author: openobserve | Branch: feat/metrics-cache | Commit: 8ed5061

Testdino Test Results

Status Total Passed Failed Skipped Flaky Pass Rate Duration
All tests passed 366 344 0 19 3 94% 4m 38s

View Detailed Results

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

☢️ Bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants