fix: Update query recommendations BG job interval and update BG job strategy to use DB triggers to ensure singleton like behaviour across the cluster. #8817

yaga-simha · 2025-10-15T09:17:05Z

User description

PR Type

Fix

Description

Default config values of query recommendations generate far too few updates. Current change will generate a recommendation every one hour for the last one hour's usage.

PR Type

Bug fix, Documentation

Description

Correct interval unit for recommendations
Clarify duration unit with comment
Keep hourly analysis intent intact

Diagram Walkthrough

flowchart LR
  cfg["config.rs Limits"] -- "add unit comments" --> dur["ZO_QUERY_RECOMMENDATION_DURATION"]
  cfg -- "fix interval units" --> intv["ZO_QUERY_RECOMMENDATION_INTERVAL"]

File Walkthrough

Relevant files

Bug fix

config.rs `Fix recommendation interval default and document units` src/config/src/config.rs Added unit comment to `ZO_QUERY_RECOMMENDATION_DURATION` (microseconds). Changed `ZO_QUERY_RECOMMENDATION_INTERVAL` default from 3600000000 to 3600 (seconds). Added unit comment to `ZO_QUERY_RECOMMENDATION_INTERVAL` (seconds).	+2/-2

github-actions · 2025-10-15T09:17:55Z

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review Unit Consistency Verify that downstream code interpreting `query_recommendation_duration` expects microseconds and that `query_recommendation_analysis_interval` expects seconds, to avoid time window or scheduling mismatches after this change. #[env_config(name = "ZO_QUERY_RECOMMENDATION_DURATION", default = 3600000000)] // microseconds pub query_recommendation_duration: i64, #[env_config(name = "ZO_QUERY_RECOMMENDATION_INTERVAL", default = 3600)] // seconds pub query_recommendation_analysis_interval: i64, Default Values Confirm the new defaults align with intended 1-hour behavior in production (duration=1h, interval=1h) and do not inadvertently increase load; check any rate limiting or scheduler overlap handling. // Default Config: Run Query Recommendation Analysis for last one hour for every hour #[env_config(name = "ZO_QUERY_RECOMMENDATION_DURATION", default = 3600000000)] // microseconds pub query_recommendation_duration: i64, #[env_config(name = "ZO_QUERY_RECOMMENDATION_INTERVAL", default = 3600)] // seconds pub query_recommendation_analysis_interval: i64,

github-actions · 2025-10-15T09:18:09Z

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Clarify and enforce duration units The comment says microseconds, but the default value (3,600,000,000) equals one hour in microseconds, which is correct only if downstream expects microseconds. Verify the unit expected by `query_recommendation_duration`; if the code elsewhere interprets this as milliseconds or seconds, this will cause a 1,000x or 1,000,000x mismatch. Make the unit explicit in the field name to prevent misuse. src/config/src/config.rs [1282-1283] -#[env_config(name = "ZO_QUERY_RECOMMENDATION_DURATION", default = 3600000000)] // microseconds -pub query_recommendation_duration: i64, +/// Duration in microseconds +#[env_config(name = "ZO_QUERY_RECOMMENDATION_DURATION", default = 3600000000)] +pub query_recommendation_duration_us: i64, Suggestion importance[1-10]: 6 __ Why: Identifies potential unit mismatch risk and proposes clearer naming; however, it asks to verify external expectations and renames a public field, which may be breaking and not strictly necessary if units are already consistent.	Low
Possible issue	Align interval and duration units The interval is now in seconds, while the duration above is labeled microseconds; mixed units can cause scheduling errors if combined arithmetically. Align units or convert explicitly to avoid off-by-1e6 bugs. src/config/src/config.rs [1284-1285] -#[env_config(name = "ZO_QUERY_RECOMMENDATION_INTERVAL", default = 3600)] // seconds -pub query_recommendation_analysis_interval: i64, +/// Interval in seconds +#[env_config(name = "ZO_QUERY_RECOMMENDATION_INTERVAL", default = 3600)] +pub query_recommendation_analysis_interval_s: i64, Suggestion importance[1-10]: 6 __ Why: Correctly notes mixed units between duration (microseconds) and interval (seconds) that could cause arithmetic errors; the fix improves clarity but is moderate impact and may require broader refactors.	Low

greptile-apps

Greptile Overview

Summary

Fixes a critical configuration bug where the query recommendation background job interval was incorrectly set to 3600000000 (interpreted as ~1000 hours) instead of 3600 seconds (1 hour).

Key Changes:

Changed query_recommendation_analysis_interval from 3600000000 to 3600
Added clarifying unit comments: // microseconds for duration and // seconds for interval
Aligns with the intended behavior described in the inline comment: "Run Query Recommendation Analysis for last one hour for every hour"

This fix ensures the background job runs hourly as intended, rather than once every ~41.7 days, significantly improving the frequency of query optimization recommendations.

Confidence Score: 5/5

This PR is safe to merge with no risk - it fixes a critical configuration bug
The change is a simple configuration fix that corrects an obvious unit mismatch. The interval value of 3600000000 was clearly incorrect (would run every ~1000 hours), and the fix to 3600 seconds aligns perfectly with the comment stating the job should run every hour. The added unit comments improve code clarity and prevent future confusion.
No files require special attention

Important Files Changed

File Analysis

Filename	Score	Overview
src/config/src/config.rs	5/5	Fixed query recommendation interval from 1000 hours to 1 hour, added clarifying unit comments

Sequence Diagram

sequenceDiagram
    participant BG as Background Job Scheduler
    participant QRS as QueryRecommendationService
    participant CFG as Config (config.rs)
    participant ENG as OptimizerContext Engine
    
    Note over CFG: Configuration Values<br/>duration: 3600000000 μs (1 hour)<br/>interval: 3600 s (1 hour)
    
    BG->>QRS: Initialize with config values
    QRS->>CFG: Read query_recommendation_analysis_interval (3600s)
    QRS->>CFG: Read query_recommendation_duration (3600000000μs)
    CFG-->>QRS: Return config values
    
    loop Every 1 hour (interval)
        QRS->>ENG: Trigger analysis
        Note over ENG: Analyze last 1 hour<br/>of query usage data
        ENG->>ENG: Generate recommendations
        ENG-->>QRS: Return recommendations
        QRS->>QRS: Wait for next interval (3600s)
    end

_{1 file reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

testdino-playwright-reporter · 2025-10-15T09:40:26Z

⚠️ Test Run Unstable

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: `b92515e`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	364	334	0	19	11	92%	4m 40s

View Detailed Results

testdino-playwright-reporter · 2025-10-15T10:56:25Z

⚠️ Test Run Unstable

Author: `Simha G` | Branch: `fix/query_recommendations_constants` | Commit: `2cd76d4`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	364	336	0	19	9	92%	4m 39s

View Detailed Results

testdino-playwright-reporter · 2025-10-15T12:20:15Z

⚠️ Test Run Unstable

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: `e856887`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	364	331	0	19	14	91%	4m 39s

View Detailed Results

testdino-playwright-reporter · 2025-10-15T13:42:59Z

⚠️ Test Run Unstable

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: `e8140d3`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	364	334	0	19	11	92%	4m 39s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T11:08:46Z

⚠️ Test Run Unstable

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: `8cbceae`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	365	342	0	19	4	94%	4m 38s

View Detailed Results

…singleton execution can be guaranteed

testdino-playwright-reporter · 2025-10-21T11:44:27Z

⚠️ Test Run Unstable

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: `edb166e`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	365	343	0	19	3	94%	4m 40s

View Detailed Results

testdino-playwright-reporter · 2025-10-22T10:18:12Z

⚠️ Test Run Unstable

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: d8f4998

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	69	64	0	4	1	93%	2m 39s

View Detailed Results

testdino-playwright-reporter · 2025-10-22T11:30:06Z

⚠️ Test Run Unstable

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: `b8e5d78`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	365	344	0	19	2	94%	4m 39s

View Detailed Results

…trategy to use DB triggers to ensure singleton like behaviour across the cluster. (#8817) ### **User description** # PR Type Fix # Description Default config values of query recommendations generate far too few updates. Current change will generate a recommendation every one hour for the last one hour's usage. ___ ### **PR Type** Bug fix, Documentation ___ ### **Description** - Correct interval unit for recommendations - Clarify duration unit with comment - Keep hourly analysis intent intact ___ ### Diagram Walkthrough ```mermaid flowchart LR cfg["config.rs Limits"] -- "add unit comments" --> dur["ZO_QUERY_RECOMMENDATION_DURATION"] cfg -- "fix interval units" --> intv["ZO_QUERY_RECOMMENDATION_INTERVAL"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><table> <tr> <td> <details> <summary><strong>config.rs</strong><dd><code>Fix recommendation interval default and document units</code>      </dd></summary> <hr> src/config/src/config.rs <ul><li>Added unit comment to <code>ZO_QUERY_RECOMMENDATION_DURATION</code> (microseconds).<br> <li> Changed <code>ZO_QUERY_RECOMMENDATION_INTERVAL</code> default from 3600000000 to <br>3600 (seconds).<br> <li> Added unit comment to <code>ZO_QUERY_RECOMMENDATION_INTERVAL</code> (seconds).</ul> </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/8817/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+2/-2</a>      </td> </tr> </table></td></tr></tr></tbody></table> </details> ___

github-actions bot added the Invalid PR Title label Oct 15, 2025

yaga-simha changed the title ~~Update query recommendations BG job interval to 1hr and add comments~~ fix: Update query recommendations BG job interval to 1hr and add comments Oct 15, 2025

github-actions bot added ✏️ Feature 📒 Documentation 🧹 Updates Review effort 2/5 labels Oct 15, 2025

greptile-apps bot reviewed Oct 15, 2025

View reviewed changes

github-actions bot added ☢️ Bug Something isn't working and removed Invalid PR Title labels Oct 15, 2025

yaga-simha self-assigned this Oct 15, 2025

yaga-simha force-pushed the fix/query_recommendations_constants branch from 2cd76d4 to e856887 Compare October 15, 2025 11:56

yaga-simha added 2 commits October 21, 2025 15:55

Update query recommendations BG job interval to 1hr and add comments

4c26b1e

Make query recommendations service limited to alert manager node

b0d8984

yaga-simha force-pushed the fix/query_recommendations_constants branch from e8140d3 to 8cbceae Compare October 21, 2025 10:45

Move query recommendations from jobs to triggers so that distributed …

edb166e

…singleton execution can be guaranteed

yaga-simha force-pushed the fix/query_recommendations_constants branch from 8cbceae to edb166e Compare October 21, 2025 11:22

yaga-simha changed the title ~~fix: Update query recommendations BG job interval to 1hr and add comments~~ fix: Update query recommendations BG job interval and update BG job strategy to use DB triggers to ensure singleton like behaviour across the cluster. Oct 21, 2025

Merge branch 'main' into fix/query_recommendations_constants

d8f4998

hengfeiyang approved these changes Oct 22, 2025

View reviewed changes

Merge branch 'main' into fix/query_recommendations_constants

b8e5d78

yaga-simha merged commit 07e7637 into main Oct 22, 2025
32 checks passed

yaga-simha deleted the fix/query_recommendations_constants branch October 22, 2025 11:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: Update query recommendations BG job interval and update BG job strategy to use DB triggers to ensure singleton like behaviour across the cluster. #8817

fix: Update query recommendations BG job interval and update BG job strategy to use DB triggers to ensure singleton like behaviour across the cluster. #8817

Uh oh!

yaga-simha commented Oct 15, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

greptile-apps bot left a comment

Uh oh!

testdino-playwright-reporter bot commented Oct 15, 2025

Uh oh!

testdino-playwright-reporter bot commented Oct 15, 2025

Uh oh!

testdino-playwright-reporter bot commented Oct 15, 2025

Uh oh!

testdino-playwright-reporter bot commented Oct 15, 2025

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

Uh oh!

testdino-playwright-reporter bot commented Oct 22, 2025

Uh oh!

testdino-playwright-reporter bot commented Oct 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fix: Update query recommendations BG job interval and update BG job strategy to use DB triggers to ensure singleton like behaviour across the cluster. #8817

fix: Update query recommendations BG job interval and update BG job strategy to use DB triggers to ensure singleton like behaviour across the cluster. #8817

Uh oh!

Conversation

yaga-simha commented Oct 15, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

User description

PR Type

Description

PR Type

Description

Diagram Walkthrough

File Walkthrough

Uh oh!

github-actions bot commented Oct 15, 2025

PR Reviewer Guide 🔍

Uh oh!

github-actions bot commented Oct 15, 2025

PR Code Suggestions ✨

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Greptile Overview

Summary

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

testdino-playwright-reporter bot commented Oct 15, 2025

⚠️ Test Run Unstable

Author: yaga-simha | Branch: fix/query_recommendations_constants | Commit: b92515e

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 15, 2025

⚠️ Test Run Unstable

Author: Simha G | Branch: fix/query_recommendations_constants | Commit: 2cd76d4

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 15, 2025

⚠️ Test Run Unstable

Author: yaga-simha | Branch: fix/query_recommendations_constants | Commit: e856887

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 15, 2025

⚠️ Test Run Unstable

Author: yaga-simha | Branch: fix/query_recommendations_constants | Commit: e8140d3

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

⚠️ Test Run Unstable

Author: yaga-simha | Branch: fix/query_recommendations_constants | Commit: 8cbceae

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

⚠️ Test Run Unstable

Author: yaga-simha | Branch: fix/query_recommendations_constants | Commit: edb166e

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 22, 2025

⚠️ Test Run Unstable

Author: yaga-simha | Branch: fix/query_recommendations_constants | Commit: d8f4998

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 22, 2025

⚠️ Test Run Unstable

Author: yaga-simha | Branch: fix/query_recommendations_constants | Commit: b8e5d78

Testdino Test Results

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yaga-simha commented Oct 15, 2025 •

edited by github-actions bot

Loading

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: `b92515e`

Author: `Simha G` | Branch: `fix/query_recommendations_constants` | Commit: `2cd76d4`

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: `e856887`

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: `e8140d3`

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: `8cbceae`

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: `edb166e`

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: d8f4998

Author: `yaga-simha` | Branch: `fix/query_recommendations_constants` | Commit: `b8e5d78`