-
Notifications
You must be signed in to change notification settings - Fork 711
fix: fixed issue on aggregating field with non-existent alias and expand log detail #5011
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
WalkthroughThe pull request introduces significant enhancements to the Changes
Possibly related PRs
Suggested labels
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (7)
web/src/plugins/logs/JsonPreview.vue (2)
374-382: Enhance error notification with error codeThe error notification implementation has been improved but could be enhanced further.
Consider adding error codes for better error tracking:
- $q.notify({ - message: - err.response?.data?.message || "Failed to get the Original data", - color: "negative", - position: "bottom", - timeout: 1500, - }); + $q.notify({ + message: `Error [OZ-${err.response?.status || 500}]: ${ + err.response?.data?.message || "Failed to get the Original data" + }`, + color: "negative", + position: "bottom", + timeout: 1500, + });
434-445: Consider memoizing filtered tabs computationThe filteredTabs computed property contains multiple conditions that could impact performance if recomputed frequently.
Consider breaking down the conditions into separate computed properties for better maintainability and performance:
- const filteredTabs = computed(() => { - return tabs.filter((tab) => { - if ( - props.value._o2_id == undefined || - searchAggData.hasAggregation || - searchObj.data.stream.selectedStream.length > 1 - ) { - return false; - } - return true; - }); - }); + const isValidForUnflattenedView = computed(() => { + return ( + props.value._o2_id != undefined && + !searchAggData.hasAggregation && + searchObj.data.stream.selectedStream.length === 1 + ); + }); + + const filteredTabs = computed(() => { + return isValidForUnflattenedView.value ? tabs : []; + });web/src/composables/useLogs.ts (5)
Line range hint
4028-4039: Fix variable scoping issue ingetRegionInfoThe variable
regionObjis declared outside thefor...inloop and is reused in each iteration. This leads to all elements inclusterDatareferencing the same object due to JavaScript's object reference behavior. DeclareregionObjwithin the loop to ensure each element is a distinct object.Apply this diff to fix the issue:
const getRegionInfo = () => { searchService.get_regions().then((res) => { const clusterData = []; - let regionObj: any = {}; const apiData = res.data; for (const region in apiData) { + let regionObj: any = { + label: region, + children: [], + }; - regionObj = { - label: region, - children: [], - }; for (const cluster of apiData[region]) { regionObj.children.push({ label: cluster }); } clusterData.push(regionObj); } store.dispatch("setRegionInfo", clusterData); }); };
Line range hint
4031-4038: EscapestreamNameto prevent regex injection inquoteTableNameDirectlyThe function
quoteTableNameDirectlyusesnew RegExpwithstreamName, which may contain special regex characters. This could lead to unintended matches or security vulnerabilities. Consider escapingstreamNamebefore using it in the regular expression.Apply this diff to fix the issue:
function quoteTableNameDirectly(sql: string, streamName: string) { + const escapedStreamName = streamName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); - const regex = new RegExp(`FROM\\s+${streamName}`, "gi"); + const regex = new RegExp(`FROM\\s+${escapedStreamName}`, "gi"); const modifiedSql = sql.replace(regex, `FROM "${streamName}"`); return modifiedSql; }
Line range hint
3994-3999: UseSetfor efficient trace ID managementCurrently,
searchRequestTraceIdsis an array, and methodsaddTraceIdandremoveTraceIdperform linear searches, which can be inefficient with large numbers of trace IDs. Using aSetwill improve performance with constant-time insertion and deletion.Apply this diff to refactor:
- searchObj.data.searchRequestTraceIds = []; + searchObj.data.searchRequestTraceIds = new Set<string>(); const addTraceId = (traceId: string) => { - if (searchObj.data.searchRequestTraceIds.includes(traceId)) { + if (searchObj.data.searchRequestTraceIds.has(traceId)) { return; } - searchObj.data.searchRequestTraceIds.push(traceId); + searchObj.data.searchRequestTraceIds.add(traceId); }; const removeTraceId = (traceId: string) => { - searchObj.data.searchRequestTraceIds = - searchObj.data.searchRequestTraceIds.filter( - (id: string) => id !== traceId, - ); + searchObj.data.searchRequestTraceIds.delete(traceId); };
Line range hint
2805-3184: RefactorextractFieldsfunction to improve readabilityThe
extractFieldsfunction is lengthy and complex, spanning over 300 lines. This makes it difficult to maintain and understand. Consider breaking it down into smaller, reusable helper functions to enhance readability and maintainability.
Line range hint
3460-3480: Ensure intervals are cleared properly inrefreshDataIn
refreshData, whensearchObj.meta.refreshIntervalis zero or negative,clearIntervalis called without checking ifstore.state.refreshIntervalIDis defined. This could lead to unexpected behavior. Consider adding a check before callingclearInterval.Apply this diff to improve error handling:
else { - clearInterval(store.state.refreshIntervalID); + if (store.state.refreshIntervalID) { + clearInterval(store.state.refreshIntervalID); + store.dispatch("setRefreshIntervalID", null); + } }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
📒 Files selected for processing (2)
web/src/composables/useLogs.ts(3 hunks)web/src/plugins/logs/JsonPreview.vue(9 hunks)
🔇 Additional comments (3)
web/src/plugins/logs/JsonPreview.vue (3)
101-101: LGTM: Template structure follows Vue.js best practices
The v-show directive is properly implemented for conditional rendering of the unflattened view.
476-484: LGTM: Styles are well-structured
The Monaco editor styling follows CSS best practices with proper use of CSS variables and responsive dimensions.
358-366:
Verify SQL query parameters and add input validation
The SQL query construction uses template literals with direct value interpolation, which could be risky if the values aren't properly sanitized.
Consider using parameterized queries or adding input validation for _o2_id and _timestamp values.
305b38f to
f25789b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
🧹 Outside diff range and nitpick comments (8)
web/src/components/ingestion/recommended/LinuxConfig.vue (1)
56-56: Consider adding error handling for undefined values.While the formatting looks good, the template literal could fail if
props.currUserEmailorstore.state.organizationData.organizationPasscodeare undefined. Consider adding null checks or default values.- `${props.currUserEmail}:${store.state.organizationData.organizationPasscode}`, + `${props.currUserEmail || ''}:${store.state.organizationData?.organizationPasscode || ''}`,web/src/components/ingestion/recommended/KubernetesConfig.vue (3)
Line range hint
97-138: Consider improving list indentation for better readability.While the conversion to unordered lists and Tailwind styling is good, the nested list indentation could be more consistent. Consider adjusting the indentation of nested
<ul>elements to better reflect the hierarchy.- <ul class="tw-list-disc tw-ml-5"> + <ul class="tw-list-disc tw-ml-5">
222-222: Remove empty style block.The style block is empty and includes SCSS support that isn't being used. Consider removing it entirely unless you plan to add styles in the near future.
-<style scoped lang="scss"></style>
Looking at the code, I can now generate the final response since I have found that the component actually has the necessary code to generate the Basic Auth token but doesn't use it in the helm commands.
Basic Auth token placeholder should be replaced with computed
accessKeyThe component already computes the correct Basic Auth token using the
accessKeycomputed property, which base64 encodes the user email and organization passcode. However, this computed value isn't being used in the helm commands where[BASIC_PASSCODE]appears. The helm commands should use${accessKey.value}instead of the static placeholder.
In
collectorCmdcomputed property, replace:--set exporters."otlphttp/openobserve".headers.Authorization="Basic [BASIC_PASSCODE]"with:
--set exporters."otlphttp/openobserve".headers.Authorization="Basic ${accessKey.value}"Make the same replacement in
collectorCmdThisClustercomputed property.🔗 Analysis chain
Line range hint
201-211: Add instructions for handling the Basic Auth token.The helm commands include a placeholder
[BASIC_PASSCODE]for the Authorization header, but there are no instructions for users on how to replace it securely. Consider adding a note explaining how to generate and use the correct Basic Auth token.
Let me gather more context about any existing documentation or instructions regarding the Basic Auth token.
Let me check if there are any related instructions or comments in the Vue component itself or nearby files.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Check if there are any actual credentials accidentally committed rg -i 'authorization.*basic.*[a-zA-Z0-9+/]{20,}' || echo "No hardcoded Basic Auth tokens found."Length of output: 2286
Script:
#!/bin/bash # Check for any documentation or comments about Basic Auth token generation rg -i -A 3 -B 3 'basic.*auth|basic.*token|basicauth' web/src/components/ingestion/ # Also check for any related documentation in markdown files fd -e md -x rg -i 'basic.*auth|basic.*token|basicauth' {}Length of output: 792
Script:
#!/bin/bash # Check the content of the Kubernetes config component and surrounding files cat web/src/components/ingestion/recommended/KubernetesConfig.vue # Check for any related components that might contain instructions fd -t f . web/src/components/ingestion/recommended/ -x cat {}Length of output: 27875
web/src/composables/useLogs.ts (4)
Line range hint
4095-4106: Improve SQL table name quoting robustness.The current regex-based approach for quoting table names has potential issues:
- It may fail with special characters in stream names
- Case-insensitive matching could cause issues with case-sensitive databases
- Global replacement could affect substrings in complex queries
Consider this safer implementation:
- const regex = new RegExp(`FROM\\s+${streamName}`, "gi"); - const modifiedSql = sql.replace(regex, `FROM "${streamName}"`); + // Escape special regex characters in streamName + const escapedStreamName = streamName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); + const regex = new RegExp( + `(FROM|JOIN)\\s+(${escapedStreamName})(?![a-zA-Z0-9_"])`, + 'g' + ); + const modifiedSql = sql.replace(regex, `$1 "${streamName}"`);
Line range hint
4095-4143: Enhance error handling in filter expression generation.The current error handling silently logs errors and returns a potentially unsafe default expression. This could mask issues and lead to incorrect filter behavior.
Consider this improved error handling:
} catch (e: any) { - console.log("Error while getting filter expression by field type", e); - return `${field} ${operator} '${field_value}'`; + const errorMsg = `Error generating filter expression for field "${field}": ${e.message}`; + console.error(errorMsg, e); + // Return a safe default or throw based on field_value type + if (typeof field_value === 'number' || typeof field_value === 'boolean') { + return `${field} ${operator} ${field_value}`; + } + return `${field} ${operator} '${field_value?.toString().replace(/'/g, "''")}'`; }
Line range hint
2919-2947: Optimize performance in record processing.The current implementation processes records individually and performs multiple array operations, which could be optimized for better performance.
Consider these optimizations:
- const recordwithMaxAttribute = - searchObj.data.queryResults.hits[maxAttributesIndex]; - - for (const key of Object.keys(recordwithMaxAttribute)) { - if (key == "_o2_id" || key == "_original") { - continue; - } - if (key == store.state.zoConfig.timestamp_column) { - searchObj.data.hasSearchDataTimestampField = true; - } - if ( - !schemaFields.includes(key) && - !commonSchemaFields.includes(key) && - key != "_stream_name" - ) { - fieldObj = { - name: key, - type: "Utf8", - ftsKey: false, - group: stream.name, - isSchemaField: false, - showValues: false, - isInterestingField: - searchObj.data.stream.interestingFieldList.includes(key) - ? true - : false, - streams: [], - }; - schemaMaps.push(fieldObj); - schemaFields.push(key); - } + // Create sets for faster lookups + const schemaFieldSet = new Set(schemaFields); + const commonSchemaFieldSet = new Set(commonSchemaFields); + const interestingFieldSet = new Set(searchObj.data.stream.interestingFieldList); + + // Process all keys at once + const recordKeys = Object.keys(recordwithMaxAttribute); + const newFields = recordKeys.filter(key => { + if (key === "_o2_id" || key === "_original" || key === "_stream_name") { + return false; + } + if (key === store.state.zoConfig.timestamp_column) { + searchObj.data.hasSearchDataTimestampField = true; + return false; + } + return !schemaFieldSet.has(key) && !commonSchemaFieldSet.has(key); + }); + + // Batch create field objects + const newFieldObjects = newFields.map(key => ({ + name: key, + type: "Utf8", + ftsKey: false, + group: stream.name, + isSchemaField: false, + showValues: false, + isInterestingField: interestingFieldSet.has(key), + streams: [], + })); + + schemaMaps.push(...newFieldObjects); + schemaFields.push(...newFields);
Line range hint
4095-4143: Enhance trace ID management robustness.The trace ID management could be improved with validation and optimized array operations.
Consider these improvements:
+ const validateTraceId = (traceId: string): boolean => { + return /^[0-9a-f]{32}$/.test(traceId); + }; + const addTraceId = (traceId: string) => { + if (!validateTraceId(traceId)) { + console.warn(`Invalid trace ID format: ${traceId}`); + return; + } if (searchObj.data.searchRequestTraceIds.includes(traceId)) { return; } searchObj.data.searchRequestTraceIds.push(traceId); }; const removeTraceId = (traceId: string) => { - searchObj.data.searchRequestTraceIds = - searchObj.data.searchRequestTraceIds.filter( - (id: string) => id !== traceId, - ); + const index = searchObj.data.searchRequestTraceIds.indexOf(traceId); + if (index > -1) { + searchObj.data.searchRequestTraceIds.splice(index, 1); + } };
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
📒 Files selected for processing (8)
web/src/components/ingestion/recommended/KubernetesConfig.vue(6 hunks)web/src/components/ingestion/recommended/LinuxConfig.vue(2 hunks)web/src/components/ingestion/recommended/WindowsConfig.vue(2 hunks)web/src/composables/useLogs.ts(3 hunks)web/src/locales/languages/en.json(1 hunks)web/src/plugins/logs/JsonPreview.vue(10 hunks)web/src/stores/index.ts(2 hunks)web/src/test/unit/helpers/store.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- web/src/plugins/logs/JsonPreview.vue
🔇 Additional comments (7)
web/src/components/ingestion/recommended/LinuxConfig.vue (1)
9-12: LGTM! Styling changes align with design consistency.
The switch to an unordered list with Tailwind styling matches the pattern used in other configuration components, improving visual consistency across the application.
web/src/components/ingestion/recommended/WindowsConfig.vue (2)
13-16: LGTM! Consistent list styling across config components.
The change from ordered to unordered list with Tailwind styling aligns with the styling updates made in other ingestion configuration components.
60-60: LGTM! Clean string interpolation.
The string interpolation formatting is clean and consistent.
web/src/components/ingestion/recommended/KubernetesConfig.vue (1)
3-3: LGTM: Class addition follows naming conventions.
The addition of the kubernetes-config-section class provides better styling organization and scope.
web/src/stores/index.ts (2)
30-33: LGTM! Improved indentation for better readability.
The adjusted indentation better reflects the nested structure of the conditional logic.
48-49: Verify all references to the renamed trace and span ID fields.
The change from camelCase to snake_case aligns with common observability system conventions. However, this is a breaking change that requires verification of all references.
Let's verify all references to these fields:
✅ Verification successful
Field name changes have been properly updated throughout the codebase
The change from camelCase to snake_case for trace and span ID fields has been consistently implemented across the codebase. The search results confirm:
- The new snake_case field names are properly set in the store configuration:
trace_id_field_name: "trace_id",
span_id_field_name: "span_id",- All references to these fields have been updated to use the new snake_case format in:
- Frontend components and views
- Backend services and handlers
- API endpoints and data models
- Configuration files and settings
The changes align with common observability system conventions and have been thoroughly propagated through the codebase.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for any remaining camelCase versions that might have been missed
echo "Searching for old camelCase versions..."
rg -i "traceId|spanId" --type ts --type vue
# Search for new snake_case versions to ensure consistent usage
echo -e "\nVerifying new snake_case versions..."
rg -i "trace_id|span_id" --type ts --type vue
Length of output: 327
Script:
#!/bin/bash
# Search for any remaining camelCase versions that might have been missed
echo "Searching for old camelCase versions..."
rg -i "traceId|spanId" --type typescript --type javascript
# Search for new snake_case versions to ensure consistent usage
echo -e "\nVerifying new snake_case versions..."
rg -i "trace_id|span_id" --type typescript --type javascript
Length of output: 371
Script:
#!/bin/bash
# Search for any remaining camelCase versions that might have been missed
echo "Searching for old camelCase versions..."
rg -i "traceId|spanId"
# Search for new snake_case versions to ensure consistent usage
echo -e "\nVerifying new snake_case versions..."
rg -i "trace_id|span_id"
Length of output: 121518
web/src/locales/languages/en.json (1)
131-131: LGTM! The new translation key follows established conventions.
The addition of the "original" translation key is well-placed within the "search" section and follows the existing naming and capitalization patterns.
#5010
Summary by CodeRabbit
New Features
quoteTableNameDirectly,getRegionInfo, andaddTraceId.Improvements
Style
JsonPreview,KubernetesConfig,LinuxConfig, andWindowsConfigcomponents, enhancing code maintainability.KubernetesConfig.Localization