Skip to content

Conversation

@haohuaijin
Copy link
Collaborator

@haohuaijin haohuaijin commented Aug 29, 2025

@github-actions
Copy link
Contributor

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Logging Format

The new log message mixes implicit and explicit formatting: it embeds {use_inverted_index} in the string literal while also passing arguments separately. Ensure consistent usage to avoid unexpected output.

log::info!(
    "[trace_id {}] flight->search: use_inverted_index {use_inverted_index}, index_condition {:?}",
    query.trace_id,
    index_condition,
);
Potential Panic

use_inverted_index computation unwraps index_condition after checking is_some(); this is safe but brittle. Consider avoiding unwrap by pattern matching to reduce risk if future refactors change conditions.

let use_inverted_index = query.use_inverted_index
    && index_condition.is_some()
    && !index_condition.as_ref().unwrap().is_condition_all();
Filter Back Clearing

After tantivy_search, index_condition is set to None when !is_add_filter_back. Confirm this preserves previous semantics, especially for negated queries now that negation-specific handling was removed.

// set index_condition to None, means we do not need to add filter back
if !is_add_filter_back {
    index_condition = None;
    fst_fields = vec![];
}

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Summary

This PR removes special handling for negative index search conditions across the OpenObserve search pipeline. The changes primarily affect two key files in the search service:

In src/service/search/index.rs: The PR eliminates methods that identified and segregated negative conditions from positive ones. Specifically, it removes:

  • is_negated_condition() methods from both IndexCondition and Condition structs that detected negated conditions (NOT, NOT IN, !=)
  • split_condition_by_negated() method that separated conditions into positive and negative groups for different processing paths
  • Associated unit tests for this functionality

In src/service/search/grpc/storage.rs: The PR simplifies the inverted index decision logic by removing complex condition splitting and recombining functions:

  • Removes check_inverted_index() function that split conditions into positive/negative parts
  • Removes generate_add_filter_back_condition() function that combined different condition types
  • Replaces the complex logic with a straightforward inline check for inverted index usage
  • Eliminates extensive test coverage for the removed functions

The refactoring consolidates the search logic to treat all conditions uniformly, whether they are positive (=, IN, contains) or negative (!=, NOT IN, not_contains). This aligns with modern search engine capabilities where negative conditions can be processed efficiently without requiring separate code paths. The change reduces technical debt and simplifies the codebase by removing the artificial distinction between condition types at the index level, allowing the underlying Tantivy search engine to handle all condition types through its standard query processing.

Confidence score: 4/5

  • This PR appears safe to merge with careful review of the search functionality
  • Score reflects solid refactoring that removes complexity without introducing obvious bugs
  • Pay close attention to search performance and correctness testing to ensure negative conditions still work properly

2 files reviewed, no comments

Edit Code Review Bot Settings | Greptile

@github-actions
Copy link
Contributor

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Remove risky unwrap usage

Avoid unwrapping index_condition after an is_some() check; concurrent mutation in
this function makes the unwrap brittle and can panic. Use pattern matching to safely
access the reference without multiple Option probes.

src/service/search/grpc/storage.rs [132-134]

-let use_inverted_index = query.use_inverted_index
-    && index_condition.is_some()
-    && !index_condition.as_ref().unwrap().is_condition_all();
+let use_inverted_index = if query.use_inverted_index {
+    match index_condition.as_ref() {
+        Some(cond) => !cond.is_condition_all(),
+        None => false,
+    }
+} else {
+    false
+};
Suggestion importance[1-10]: 6

__

Why: Replacing is_some() + unwrap() with a single match is safer and clearer; while current code won’t panic given no mutation between checks, the refactor avoids brittleness and repeated probes.

Low
General
Log post-filter-back state

Clarify the lifecycle of index_condition: it is cloned for tantivy_search and then
possibly set to None. Add a debug log after mutation to ensure downstream consumers
don't rely on a moved/cleared value and to aid diagnosing mismatches between index
filtering and filter-back.

src/service/search/grpc/storage.rs [143-152]

 if use_inverted_index {
     (idx_took, is_add_filter_back, ..) = tantivy_search(
         query.clone(),
         &mut files,
         index_condition.clone(),
         idx_optimize_rule,
     )
     .await?;
 }
 
-// set index_condition to None, means we do not need to add filter back
 if !is_add_filter_back {
     index_condition = None;
     fst_fields = vec![];
 }
 
+log::debug!(
+    "[trace_id {}] storage: finalized add-filter-back={is_add_filter_back}, index_condition={:?}",
+    query.trace_id,
+    &index_condition
+);
+
Suggestion importance[1-10]: 4

__

Why: Adding a debug log after potentially clearing index_condition can aid diagnostics without changing behavior; improvement is minor and optional for functionality.

Low

@haohuaijin haohuaijin requested a review from hengfeiyang August 29, 2025 14:37
@hengfeiyang hengfeiyang merged commit 9d1f672 into main Aug 30, 2025
28 checks passed
@hengfeiyang hengfeiyang deleted the remove-special-for-not-index branch August 30, 2025 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants