Implement Mutable FST Index #8861
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #8861 +/- ##
============================================
+ Coverage 69.70% 69.74% +0.04%
- Complexity 4668 4990 +322
============================================
Files 1741 1804 +63
Lines 91521 93883 +2362
Branches 13691 13981 +290
============================================
+ Hits 63793 65478 +1685
- Misses 23289 23869 +580
- Partials 4439 4536 +97
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Jackie-Jiang
added
feature
release-notes
Jackie-Jiang
left a comment
Jackie-Jiang
left a comment
There was a problem hiding this comment.
Good job! Only minor comments
Please add some description to the PR, and add a release-note section
| public class NativeMutableFSTIndex implements MutableTextIndex { | ||
| private final String _column; | ||
| private final MutableFST _fst; | ||
| private int _dictId; |
There was a problem hiding this comment.
(minor)
Move this after the final variables for readability
|
|
||
| @Override | ||
| public MutableRoaringBitmap getDocIds(String searchQuery) { | ||
| return null; |
There was a problem hiding this comment.
Let's throw an unsupported exception with some error message for debugging purpose
| invertedIndexColumns.contains(column) ? indexProvider.newInvertedIndex(context.forInvertedIndex()) : null; | ||
|
|
||
| MutableTextIndex fstIndex = null; | ||
| //FST Index |
There was a problem hiding this comment.
(nit) Add a space after the //
There was a problem hiding this comment.
We can remove the TODO on line 246
|
|
||
| if (results != null) { | ||
| for (int result : results) { | ||
| assertEquals(resultMap.contains(result), true); |
| MutableTextIndex fstIndex = null; | ||
| //FST Index | ||
| if (fstIndexColumns.contains(column)) { | ||
| if (_fieldConfigList != null) { |
There was a problem hiding this comment.
nit: you can combine these nested if statement into one like:
if (_fieldConfigList != null && fstIndexColumns.contains(column))
|
|
||
| nativeQuery = ".*ed"; | ||
| resultList = Arrays.asList(6); | ||
| testSelectionResults(nativeQuery, 1, resultList); |
4 participants
This PR implements a real time FST index, allowing fields containing tokenized documents to be directly stored in the FST index and queried in real time, with no flush requirement.
This can be enabled by specifying the following property in the properties section of the FST index, being created on a real time table:
"properties":[{"fstType":"native"}]