-
Notifications
You must be signed in to change notification settings - Fork 715
feat: UI explain query #8831
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: UI explain query #8831
Conversation
Add a comprehensive query plan visualization feature that allows users to view and analyze DataFusion query execution plans directly from the logs page SQL editor. Changes: - Add QueryPlanDialog component with EXPLAIN and EXPLAIN ANALYZE support - Add Explain button to logs search toolbar (lightbulb icon) - Display prettified query plans with proper indentation and structure - Show execution metrics (time, memory, rows) with ANALYZE option - Add i18n translations for query plan UI elements - Enable only when SQL mode is active and query is present The dialog opens maximized and provides: - EXPLAIN query plan visualization with formatted output - Analyze button to run EXPLAIN ANALYZE for execution metrics - Real-time plan parsing and formatting - Metrics summary display (execution time, memory, row counts) - Full-screen view for complex query plans Technical implementation: - Reuses existing Quasar dialog patterns for consistency - Integrates with existing search service API - Parses DataFusion plan format with enhanced visual separation - Supports both logical and physical plan views
Fix issues with the query plan explanation feature: 1. Dialog Size & Positioning: - Remove fullscreen/maximized mode - Set fixed width (900px, max 90vw) for centered panel display - Add proper max-height (85vh) with scrollable content - Better matches existing dialog patterns in the app 2. Complete Query Construction: - Use buildSearch() from useSearchStream composable - Include all query parameters: time range, streams, filters, pagination - Preserve regions, clusters, and other enterprise parameters - Wrap complete SQL with EXPLAIN/EXPLAIN ANALYZE instead of simplified query 3. Props Refactoring: - Accept full searchObj instead of individual sqlQuery and orgIdentifier - Access store.state.selectedOrganization.identifier directly - Import and use useSearchStream composable for query building 4. API Integration: - Use same search infrastructure as main query execution - Include traceparent for distributed tracing - Better error handling with response.data.message extraction 5. Styling Improvements: - Flexible layout with proper overflow handling - Max-height constraints for scrollable areas - Plan card with 500px max-height for long plans The dialog now displays query plans using the exact same query context as the "Run Query" button, ensuring accurate execution plan representation.
- Parse DataFusion plan_type field to separate logical_plan and physical_plan - Add Quasar tabs component with "Logical Plan" and "Physical Plan" tabs - Update parsePlans() to extract plans based on plan_type from hits array - Add fallback parsing for combined plan text - Update state management with logicalPlan and physicalPlan refs - Add i18n strings for noLogicalPlan and noPhysicalPlan messages - Improve plan display with proper scrolling in tab panels
- Add support for phase field in addition to plan_type - EXPLAIN uses plan_type, EXPLAIN ANALYZE uses phase - Check both fields when parsing query plans
- EXPLAIN ANALYZE returns numeric phase field (0, 1) not string plan_type - Combine all phases into single physical plan display - Add message explaining EXPLAIN ANALYZE only shows physical plan - Sort phases by number before combining - EXPLAIN (without ANALYZE) continues to work with plan_type field
- Switch from searchService to streamingSearch to use _search_stream endpoint - Add parseSSEResponse function to handle Server-Sent Events format - Use traceId parameter instead of traceparent header for streaming API - Handle EXPLAIN ANALYZE numeric phase field (0, 1, 2...) correctly - Combine all EXPLAIN ANALYZE phases into single physical plan view The streaming API returns responses in SSE format with event and data lines that need to be parsed to extract the actual JSON results. This fix ensures both EXPLAIN and EXPLAIN ANALYZE queries work correctly with the streaming backend.
Major improvements to EXPLAIN and EXPLAIN ANALYZE functionality: **New Components:** - QueryPlanTree: Hierarchical tree visualization with expand/collapse - QueryPlanNode: Recursive node component with tree connectors - CollapsibleProjection: Expandable field lists (collapses > 5 fields) - MetricsSummaryCard: Aggregate metrics display (time, rows, memory, operators) **Parser Utilities (queryPlanParser.ts):** - parseQueryPlanTree: Parse indentation-based plan into tree structure - calculateSummaryMetrics: Aggregate metrics with RepartitionExec handling - Parallel execution: MAX time, SUM rows - Sequential execution: SUM time, pass through rows - collapseProjections: Auto-collapse long field lists (threshold: 5) - formatTime/formatMemory: Human-readable metric formatting **QueryPlanDialog Updates:** - Two-mode rendering: - EXPLAIN: Logical/Physical tabs with tree view - EXPLAIN ANALYZE: Single view (no tabs) with summary card + tree - Tree visualization replaces plain text for better readability - Inline metrics display on each operator node - Visual tree connectors (├─, └─, │) for hierarchy **UI/UX Improvements:** - Reduced cognitive load through progressive disclosure - Collapsible projections prevent long field lists from obscuring plan - Metrics displayed inline with operators for easy scanning - Tree structure shows parent-child relationships clearly - No color coding (user interprets metrics objectively) **More Actions Menu:** - Added "Explain Query" option to toolbar dropdown - Shows when SQL mode enabled and query not empty - Quick access to query plan analysis **i18n:** - executionSummary, logicalPlan, physicalPlan - totalTime, totalRows, peakMemory, operatorCount
Changes: - Removed operatorCount from SummaryMetrics interface - Removed operator count metric card from MetricsSummaryCard (now 3 metrics) - Added parentPrefix indentation to QueryPlanNode template - Fixed CSS to preserve whitespace with white-space: pre - Added tree-indent styling for proper branch visualization - Adjusted spacing and gaps for cleaner tree structure This ensures multi-branch query plans (e.g., with multiple CTEs or joins) display proper tree indentation showing parent-child relationships clearly.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Explore these optional code suggestions:
|
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 364 | 344 | 0 | 19 | 1 | 95% | 4m 40s |
### **PR Type**
Bug fix, Enhancement
___
### **Description**
- Cap series per query for PromQL
- Split SQL series limit across y-axes
- Prevent multi y-axis performance regressions
___
### Diagram Walkthrough
```mermaid
flowchart LR
Config["Dashboard config: max_dashboard_series"]
PromQL["convertPromQLData: per-query cap"]
SQL["convertSQLData: per-axis cap with breakdown"]
Performance["Reduced series -> better performance"]
Config -- "read max limit" --> PromQL
Config -- "read max/top_results" --> SQL
PromQL -- "divide by active queries" --> Performance
SQL -- "divide by yAxisKeys when breakdowns" --> Performance
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>convertPromQLData.ts</strong><dd><code>Per-query series
limiting for PromQL results</code>
</dd></summary>
<hr>
web/src/utils/dashboard/convertPromQLData.ts
<ul><li>Replace global series cap with per-query cap.<br> <li> Compute
active query count and divide max limit.<br> <li> Slice each query's
results by per-query limit.<br> <li> Remove decremental cross-query
limiting logic.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8835/files#diff-68d70ba6c622911d48cdb567940dc404a4a6468b86c6d24fb54e5f1d8aa5ca2a">+10/-4</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Bug fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>convertSQLData.ts</strong><dd><code>Per-axis series cap
for SQL with breakdowns</code>
</dd></summary>
<hr>
web/src/utils/dashboard/convertSQLData.ts
<ul><li>Introduce adaptive limit for multi y-axis charts.<br> <li>
Divide series limit by y-axis count when breakdowns present.<br> <li>
Preserve top_results and max_dashboard_series constraints.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8835/files#diff-88c0e865040a20b2c8bb7d77e7f6bd187a5b07c1645c120223bc9d59603752c3">+8/-1</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
### **PR Type**
Bug fix
___
### **Description**
- Guard histogram interval must be positive
- Prevent misaligned cache time adjustments
- Skip alignment when interval invalid
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["write_results cache alignment"] -- "is_aggregate && interval present" --> B["validate interval > 0"]
B -- "true" --> C["align start/end to interval"]
B -- "false" --> D["skip alignment/adjustment"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Bug
fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Add positive-interval guards
to histogram alignment</code>
</dd></summary>
<hr>
src/service/search/cache/mod.rs
<ul><li>Add <code>interval > 0</code> check before histogram
alignment.<br> <li> Guard end-time cache adjustment with positive
interval.<br> <li> Prevent microsecond conversion for zero/invalid
intervals.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8838/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+5/-1</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
---------
Signed-off-by: loaki07 <[email protected]>
Adds a config variable to control the length of hash UI changes to accomodate Hash pattern policy <img width="720" height="410" alt="image" src="https://github.com/user-attachments/assets/b99e28ec-2b7d-431e-9606-7b855df62f8d" /> <img width="2037" height="1112" alt="image (3)" src="https://github.com/user-attachments/assets/67432a01-c598-4b0a-830b-137a4cd04bf9" /> --------- Co-authored-by: Shrinath Rao <[email protected]>
### **User description** resolve #8832 ___ ### **PR Type** Bug fix ___ ### **Description** - Fallback to default org settings if missing - Handle missing DB key without error - Cache default settings to reduce queries - Preserve free trial expiry logic ___ ### Diagram Walkthrough ```mermaid flowchart LR A["Request org settings"] --> B{"Cache hit?"} B -- "yes" --> C["Return cached + set trial expiry"] B -- "no" --> D{"DB get key"} D -- "found" --> E["Deserialize JSON"] D -- "not found" --> F["Use OrganizationSetting::default()"] D -- "other error" --> G["Propagate error"] E --> H["Cache settings"] F --> H H --> C ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><table> <tr> <td> <details> <summary><strong>organization.rs</strong><dd><code>Default and cache org settings when DB key missing</code> </dd></summary> <hr> src/service/db/organization.rs <ul><li>Replace direct DB get with match handling.<br> <li> Default to <code>OrganizationSetting::default()</code> on missing key.<br> <li> Cache settings even when defaulted.<br> <li> Maintain <code>free_trial_expiry</code> assignment on returns.</ul> </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/8833/files#diff-ce65cea926a09721a8fea14d07fe188bc2b887218e95d74a8bffefb54001da50">+11/-3</a> </td> </tr> </table></td></tr></tr></tbody></table> </details> ___ Co-authored-by: Hengfei Yang <[email protected]>
- Implement OAuth 2.0 Authorization Server Metadata endpoint (RFC 8414) - Add JWT token validation for MCP endpoints with dynamic clients - Add `WWW-Authenticate` header support for unauthorized responses - Add `AuthTokensExt::has_expired()` method for token validation - Refactor base64 decode to accept generic byte inputs - Update authentication flow to allow MCP users without DB records - Clippy delinting
### **PR Type**
Enhancement, Tests
___
### **Description**
- Add filter config to variable creation
- Improve save/close stability with waits
- Add helper to await dependent API calls
- New e2e test for filtered variables
___
### Diagram Walkthrough
```mermaid
flowchart LR
varsJS["dashboard-variables.js"]
testJS["dashboard-streaming.spec.js"]
feature["Filter config for variables"]
stability["Stability and wait improvements"]
helper["Dependent API calls wait helper"]
e2e["E2E test for filtered variables"]
varsJS -- "implement" --> feature
varsJS -- "add" --> stability
varsJS -- "introduce" --> helper
testJS -- "use" --> feature
testJS -- "validate" --> helper
testJS -- "verify streaming behavior" --> e2e
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>dashboard-variables.js</strong><dd><code>Variable
creation gains filter support and waits</code>
</dd></summary>
<hr>
tests/ui-testing/pages/dashboardPages/dashboard-variables.js
<ul><li>Extend addDashboardVariable with filterConfig param.<br> <li>
Add robust waits and save/close handling.<br> <li> Implement
waitForDependentVariablesToLoad helper.<br> <li> Improve selectors and
option picking with visibility checks.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8767/files#diff-67d38d39bb67f4a0cbfa473ee81ad50b1fffebc40c635417e58905f3c865a6e2">+96/-19</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Tests</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>dashboard-streaming.spec.js</strong><dd><code>Streaming
test validates filtered variable dependencies</code>
</dd></summary>
<hr>
tests/ui-testing/playwright-tests/dashboards/dashboard-streaming.spec.js
<ul><li>Add test for variables with filter chains.<br> <li> Track
streaming values API responses.<br> <li> Assert dependent calls and
statuses.<br> <li> Create panel and verify variable-driven
filtering.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8767/files#diff-92f44296937f53cb448371d1c61510c8c910fbecab83b02b8e89490d5832be19">+203/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
Failed to generate code suggestions for PR |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Overview
Greptile Summary
This PR adds a comprehensive query plan visualization feature to the logs UI, allowing users to view EXPLAIN and EXPLAIN ANALYZE results for their SQL queries. The implementation includes a full-screen dialog with a split view showing the SQL query and execution plans, a parser for DataFusion query plan output, and interactive tree components with expand/collapse functionality.
Key Changes:
- New
QueryPlanDialogcomponent with EXPLAIN and EXPLAIN ANALYZE support - Parser utilities for DataFusion plans with metrics extraction and hierarchical phase nesting
- Tree visualization components with collapsible nodes and inline metrics display
- Integration into SearchBar toolbar and dropdown menu (SQL mode only)
- i18n strings for all UI labels
Critical Issue Found:
QueryPlanDialog.vue:583-587returns undefined variables (showVerbose,hasVerboseData,verboseLogicalPlan,verbosePhysicalPlan,verboseAnalyzePlan) that are never defined in the setup function, which will cause runtime errors
Confidence Score: 2/5
- This PR has a critical runtime error that will break the dialog component
- Score reflects undefined variables being returned from the setup function in QueryPlanDialog.vue (lines 583-587), which will cause immediate runtime errors when the component is used. The rest of the implementation appears well-structured with good separation of concerns, but this critical bug prevents safe merging.
- Pay close attention to
web/src/components/QueryPlanDialog.vue- remove undefined variables from the return statement (lines 583-587)
Important Files Changed
File Analysis
| Filename | Score | Overview |
|---|---|---|
| web/src/utils/queryPlanParser.ts | 4/5 | Parser and utilities for DataFusion query plans with time/memory formatting and metrics calculation |
| web/src/components/QueryPlanDialog.vue | 2/5 | Dialog component with undefined variables returned in setup function (showVerbose, hasVerboseData, verboseLogicalPlan, verbosePhysicalPlan, verboseAnalyzePlan) |
| web/src/plugins/logs/SearchBar.vue | 5/5 | Added Explain button to toolbar and menu with proper SQL mode checking and query validation |
Sequence Diagram
sequenceDiagram
participant User
participant SearchBar
participant QueryPlanDialog
participant StreamingSearch
participant Backend
participant Parser
participant Tree
User->>SearchBar: Click Explain button
SearchBar->>QueryPlanDialog: Open dialog (showExplainDialog=true)
QueryPlanDialog->>QueryPlanDialog: buildSearch() to get query
QueryPlanDialog->>QueryPlanDialog: Wrap SQL with EXPLAIN
QueryPlanDialog->>StreamingSearch: search(EXPLAIN query)
StreamingSearch->>Backend: POST streaming search request
Backend-->>StreamingSearch: SSE response with plans
StreamingSearch-->>QueryPlanDialog: SSE data
QueryPlanDialog->>QueryPlanDialog: parseSSEResponse()
QueryPlanDialog->>QueryPlanDialog: parsePlans() - extract logical/physical
QueryPlanDialog->>Parser: parseQueryPlanTree(logicalPlan)
Parser-->>QueryPlanDialog: logicalPlanTree
QueryPlanDialog->>Parser: parseQueryPlanTree(physicalPlan)
Parser-->>QueryPlanDialog: physicalPlanTree
QueryPlanDialog->>Tree: Render QueryPlanTree components
Tree-->>User: Display logical/physical plans
alt User clicks Analyze
User->>QueryPlanDialog: Click Analyze button
QueryPlanDialog->>QueryPlanDialog: Wrap SQL with EXPLAIN ANALYZE
QueryPlanDialog->>StreamingSearch: search(EXPLAIN ANALYZE query)
StreamingSearch->>Backend: POST streaming search request
Backend-->>StreamingSearch: SSE response with execution plans
StreamingSearch-->>QueryPlanDialog: SSE data (phase 0 & 1)
QueryPlanDialog->>QueryPlanDialog: parsePlans(data, isAnalyze=true)
QueryPlanDialog->>Parser: parseQueryPlanTree(phase0)
Parser-->>QueryPlanDialog: phase0Tree
QueryPlanDialog->>QueryPlanDialog: findRemoteExecNode(phase0Tree)
QueryPlanDialog->>Parser: parseQueryPlanTree(phase1)
Parser-->>QueryPlanDialog: phase1Children
QueryPlanDialog->>QueryPlanDialog: Merge phase1 under RemoteExec
QueryPlanDialog->>Parser: calculateSummaryMetrics(analyzePlan)
Parser-->>QueryPlanDialog: summaryMetrics
QueryPlanDialog->>Tree: Render with metrics
Tree-->>User: Display execution plan with metrics
end
68 files reviewed, 1 comment
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 365 | 345 | 0 | 19 | 1 | 95% | 4m 39s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 41 | 38 | 0 | 2 | 1 | 93% | 1m 17s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 366 | 340 | 0 | 19 | 7 | 93% | 4m 38s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 366 | 342 | 0 | 19 | 5 | 93% | 4m 46s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 366 | 345 | 0 | 19 | 2 | 94% | 4m 40s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 366 | 346 | 0 | 19 | 1 | 95% | 4m 39s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 366 | 344 | 0 | 19 | 3 | 94% | 4m 39s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 366 | 344 | 0 | 19 | 3 | 94% | 4m 39s |
PR Type
Enhancement, Documentation
Description
Add query plan dialog with EXPLAIN/ANALYZE
Parse, render plan tree with metrics
Integrate Explain button in logs toolbar/menu
Add i18n strings for new UI labels
Diagram Walkthrough
File Walkthrough
7 files
Add parser and metrics utilities for DataFusion plansNew dialog to fetch, parse, and display plansCollapsible projection field list rendererSummary card for time, rows, memory metricsTree node component with metrics and collapseRender plan as expandable ASCII treeAdd Explain button and dialog wiring in toolbar/menu1 files
Add i18n strings for query plan UI