-
Notifications
You must be signed in to change notification settings - Fork 715
fix: removed _timestamp and order by restriction from sql mode #3620
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
WalkthroughThe recent changes in the Changes
Sequence Diagram(s) (Beta)sequenceDiagram
participant User
participant useLogs
participant fnHistogramParsedSQL
participant parser
User->>useLogs: Invoke useLogs function
useLogs->>fnHistogramParsedSQL: Pass req.aggs.histogram
fnHistogramParsedSQL-->>useLogs: Return histogramParsedSQL
useLogs->>parser: Call sqlify(histogramParsedSQL)
parser-->>useLogs: Return histogramQuery
useLogs->>useLogs: Check and assign start_time and end_time
useLogs->>useLogs: Conditionally push new column to resultGrid
useLogs-->>User: Return processed data
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 3
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- web/src/composables/useLogs.ts (5 hunks)
Additional Context Used
Biome (20)
web/src/composables/useLogs.ts (20)
128-128: Unexpected any. Specify a different type.
129-129: Unexpected any. Specify a different type.
139-139: Unexpected any. Specify a different type.
143-143: Unexpected any. Specify a different type.
146-146: Unexpected any. Specify a different type.
151-151: Unexpected any. Specify a different type.
153-153: Unexpected any. Specify a different type.
154-154: Unexpected any. Specify a different type.
155-155: Unexpected any. Specify a different type.
156-156: Unexpected any. Specify a different type.
157-157: Unexpected any. Specify a different type.
169-169: Unexpected any. Specify a different type.
170-170: Unexpected any. Specify a different type.
175-175: Unexpected any. Specify a different type.
176-176: Unexpected any. Specify a different type.
186-186: Unexpected any. Specify a different type.
187-187: Unexpected any. Specify a different type.
223-223: Unexpected any. Specify a different type.
235-235: Unexpected any. Specify a different type.
236-236: Unexpected any. Specify a different type.
Additional comments not posted (2)
web/src/composables/useLogs.ts (2)
647-647: Ensure consistent SQL syntax conversion.Verification successful
The output indicates multiple instances of
parser.sqlifyusage across different files. Some instances include the replacement of backticks with double quotes using.replace(//g, '"')`, but it's not clear if this is consistently applied everywhere.To ensure consistent SQL syntax conversion, we need to verify if all instances of
parser.sqlifyare followed by the replacement of backticks with double quotes.
The previous script did not produce any output, which suggests that the instances of
parser.sqlifyfollowed by.replace(//g, '"')` were not detected. This could be due to the specific search pattern used.To ensure thorough verification, we need to re-check all instances of
parser.sqlifyand manually verify if they are followed by the replacement of backticks with double quotes.
The output shows that in most instances,
parser.sqlifyis followed by the replacement of backticks with double quotes using.replace(//g, '"')`. This indicates a consistent approach to SQL syntax conversion across the codebase.However, there are a few instances where this replacement is not immediately visible in the surrounding lines. These instances should be manually verified to ensure consistency.
web/src/composables/useQuery.ts:
parsedParams.query = parser.sqlify(parsedSQL);- Followed by:
parsedParams.query = parsedParams.query.replace(//g, '"');`web/src/composables/useLogs.ts:
let histogramQuery = parser.sqlify(histogramParsedSQL);- Followed by:
histogramQuery = histogramQuery.replace(//g, '"');`web/src/utils/query/sqlUtils.ts:
const sql = parser.sqlify(newAst);- Followed by:
const quotedSql = sql.replace(//g, '"');`Given the evidence, it appears that the SQL syntax conversion is consistently applied across the codebase.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that all SQL queries are consistently converted to use double quotes instead of backticks across the entire codebase. # Test: Search for SQL queries using backticks. Expect: No occurrences of backticks in SQL queries. rg --type ts "parser.sqlify"Length of output: 827
Script:
#!/bin/bash # Description: Verify that all instances of parser.sqlify are followed by the replacement of backticks with double quotes. # Search for instances of parser.sqlify and check if they are followed by .replace(/`/g, '"') rg --type ts "parser.sqlify" -A 1 | grep -B 1 -A 1 'replace(/`/g, \\"\\")'Length of output: 74
Script:
#!/bin/bash # Description: Re-verify all instances of parser.sqlify and manually check if they are followed by the replacement of backticks with double quotes. # List all instances of parser.sqlify with some surrounding lines for context. rg --type ts "parser.sqlify" -A 3 -B 1Length of output: 3067
1216-1219: Ensure proper handling of default values for start and end times.Verification successful
The search results indicate multiple instances of
start_timeandend_timeacross various files. To verify the handling of default values specifically inweb/src/composables/useLogs.ts, we need to ensure that these values are properly checked and assigned default values where necessary.Observations:
- The lines in question are:
const start_time: number = initialQueryPayload.value?.query?.start_time || 0; const end_time: number = initialQueryPayload.value?.query?.end_time || 0;Analysis:
- These lines assign default values of
0tostart_timeandend_timeif they are not provided ininitialQueryPayload.- We need to ensure that these default values are appropriate and that there are no other instances where
start_timeandend_timeare used without proper checks.Next Steps:
- Review the context around these lines in
useLogs.tsto ensure proper handling.- Check other instances in
useLogs.tswherestart_timeandend_timeare used to confirm they are handled correctly.Let's proceed with a detailed review of
useLogs.tsto verify the handling ofstart_timeandend_time.
The detailed context around the usage of
start_timeandend_timeinweb/src/composables/useLogs.tsconfirms that default values are being handled appropriately. The code assigns default values of0tostart_timeandend_timeif they are not provided ininitialQueryPayload. Additionally, the code ensures that these values are used correctly in subsequent logic, such as in thebuildSearchandgetQueryDatafunctions.Key Points:
- Default values of
0are assigned tostart_timeandend_timeif they are not provided.- These values are used in various parts of the code with proper checks and logic to ensure they are handled correctly.
Based on the detailed review, the handling of default values for
start_timeandend_timeis appropriate.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that the default values for start_time and end_time are being used appropriately across different functions. # Test: Search for usages of start_time and end_time. Expect: Proper checks before usage. rg --type ts "start_time"Length of output: 6544
Script:
#!/bin/bash # Description: Review the context around the usage of start_time and end_time in useLogs.ts to ensure proper handling of default values. # Extract 20 lines before and after the relevant lines in useLogs.ts for detailed review. rg --type ts -A 20 -B 20 "initialQueryPayload.value?.query?.start_time || 0" web/src/composables/useLogs.tsLength of output: 98741
| const histogramParsedSQL: any = fnHistogramParsedSQL( | ||
| req.aggs.histogram | ||
| ); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Refactor the function call to improve readability.
- const histogramParsedSQL: any = fnHistogramParsedSQL(req.aggs.histogram);
+ const histogramParsedSQL = fnHistogramParsedSQL(req.aggs.histogram);Consider removing the explicit any type to enforce type safety. TypeScript's type inference can handle this case.
Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| const histogramParsedSQL: any = fnHistogramParsedSQL( | |
| req.aggs.histogram | |
| ); | |
| const histogramParsedSQL = fnHistogramParsedSQL( | |
| req.aggs.histogram | |
| ); |
| if ( | ||
| (searchObj.meta.sqlMode == true && | ||
| parsedSQL.hasOwnProperty("columns") && | ||
| hasTimeStampColumn(parsedSQL.columns)) || | ||
| searchObj.data.stream.selectedFields.includes( | ||
| store.state.zoConfig.timestamp_column | ||
| ) | ||
| ) { | ||
| searchObj.data.resultGrid.columns.push({ | ||
| name: "@timestamp", | ||
| field: (row: any) => | ||
| timestampToTimezoneDate( | ||
| row[store.state.zoConfig.timestamp_column] / 1000, | ||
| store.state.timezone, | ||
| "yyyy-MM-dd HH:mm:ss.SSS" | ||
| ), | ||
| prop: (row: any) => | ||
| timestampToTimezoneDate( | ||
| row[store.state.zoConfig.timestamp_column] / 1000, | ||
| store.state.timezone, | ||
| "yyyy-MM-dd HH:mm:ss.SSS" | ||
| ), | ||
| label: t("search.timestamp") + ` (${store.state.timezone})`, | ||
| align: "left", | ||
| sortable: true, | ||
| }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review the logic for adding timestamp columns to the result grid.
- if (
- (searchObj.meta.sqlMode == true &&
- parsedSQL.hasOwnProperty("columns") &&
- hasTimeStampColumn(parsedSQL.columns)) ||
- searchObj.data.stream.selectedFields.includes(
- store.state.zoConfig.timestamp_column
- )
- ) {
+ if (hasTimeStampColumn(parsedSQL.columns)) {The condition should directly check if the timestamp column is present in the parsed SQL columns, rather than also checking the selected fields from the stream, which might introduce redundancy or errors.
Committable suggestion was skipped due low confidence.
|
|
||
| const isNonAggregatedQuery = (parsedSQL: any = null) => { | ||
| return !(parsedSQL?.limit); | ||
| return !parsedSQL?.limit; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Simplify the return statement in isNonAggregatedQuery.
- return !parsedSQL?.limit;
+ return parsedSQL?.limit === undefined;This change makes the intent clearer, indicating that we are specifically checking for the absence of a limit property.
Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| return !parsedSQL?.limit; | |
| return parsedSQL?.limit === undefined; |
30d0cd1 to
738d6c9
Compare
738d6c9 to
a6224bf
Compare
…hub.com/openobserve/openobserve into issue-3618-remove-timestamp-restriction
cf505bb to
a451f4e
Compare
…hub.com/openobserve/openobserve into issue-3618-remove-timestamp-restriction
5f8d8f5 to
9da8133
Compare
…openobserve/openobserve into issue-3618-remove-timestamp-restriction
<!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Refactor** - Improved handling and parsing of SQL queries for better performance and reliability. - Simplified logic for timestamp handling and query parsing. - **Bug Fixes** - Fixed issues with error notifications related to SQL orderby and timestamp requirements. - **New Features** - Added dynamic column addition to the result grid based on SQL mode and selected fields. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Neha P <[email protected]>
Summary by CodeRabbit
Refactor
Bug Fixes
New Features