Repository navigation
Commit 511a8a0
authored
[fix](be) Normalize SNII prefix syntax before analysis (apache#66874)
### What problem does this PR solve?
Issue Number: None
Related PR: apache#66052
Problem Summary:
`SEARCH ... PREFIX` values retain their trailing DSL `*` marker in FE.
SNII classifies a custom keyword analyzer as analyzed, but that analyzer
preserves the marker as a literal byte. Consequently, a predicate such
as `fail*` sent `fail*` to the phrase-prefix reader instead of the
indexed prefix `fail` and returned no rows, while V3 matched the
expected documents.
This PR strips exactly the trailing DSL marker before SNII phrase-prefix
analysis. Non-analyzed keyword fields continue to use `WILDCARD_QUERY`
with the marker intact, ordinary WILDCARD clauses are unchanged, and the
V3 path is unchanged. The change is query-only: it does not alter index
writes or the on-disk format, so existing SNII indexes are fixed after
upgrade without rebuilding.
TDD evidence:
- RED: a real custom keyword tokenizer/analyzer preserved `fail*`;
`FunctionSearch::build_leaf_query` sent `fail*` and produced an empty
bitmap instead of `{0, 2}`.
- GREEN: the same path sends `fail`, retains
`MATCH_PHRASE_PREFIX_QUERY`, and produces `{0, 2}`.
- All 98 `FunctionSearchTest` cases pass.
- `./build.sh --be -j 192` passes with ASAN.
- `build-support/check-format.sh` passes.
### Release note
Fix SEARCH PREFIX queries on SNII indexes that use a custom keyword
analyzer.1 parent 74b4418 commit 511a8a0
2 files changed
Lines changed: 91 additions & 17 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
860 | 860 | | |
861 | 861 | | |
862 | 862 | | |
863 | | - | |
864 | | - | |
865 | | - | |
866 | | - | |
867 | | - | |
868 | | - | |
869 | | - | |
870 | | - | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
871 | 867 | | |
872 | 868 | | |
873 | 869 | | |
| |||
903 | 899 | | |
904 | 900 | | |
905 | 901 | | |
906 | | - | |
907 | | - | |
908 | | - | |
909 | | - | |
910 | | - | |
911 | | - | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
912 | 913 | | |
913 | 914 | | |
914 | 915 | | |
| |||
925 | 926 | | |
926 | 927 | | |
927 | 928 | | |
928 | | - | |
929 | | - | |
| 929 | + | |
930 | 930 | | |
931 | | - | |
| 931 | + | |
932 | 932 | | |
933 | 933 | | |
934 | 934 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| 45 | + | |
45 | 46 | | |
46 | 47 | | |
47 | 48 | | |
| |||
2824 | 2825 | | |
2825 | 2826 | | |
2826 | 2827 | | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
| 2837 | + | |
| 2838 | + | |
| 2839 | + | |
| 2840 | + | |
| 2841 | + | |
| 2842 | + | |
| 2843 | + | |
| 2844 | + | |
| 2845 | + | |
| 2846 | + | |
| 2847 | + | |
| 2848 | + | |
| 2849 | + | |
| 2850 | + | |
| 2851 | + | |
| 2852 | + | |
| 2853 | + | |
| 2854 | + | |
| 2855 | + | |
| 2856 | + | |
| 2857 | + | |
| 2858 | + | |
| 2859 | + | |
| 2860 | + | |
| 2861 | + | |
| 2862 | + | |
| 2863 | + | |
| 2864 | + | |
| 2865 | + | |
| 2866 | + | |
| 2867 | + | |
| 2868 | + | |
| 2869 | + | |
| 2870 | + | |
| 2871 | + | |
| 2872 | + | |
| 2873 | + | |
| 2874 | + | |
| 2875 | + | |
| 2876 | + | |
| 2877 | + | |
| 2878 | + | |
| 2879 | + | |
| 2880 | + | |
| 2881 | + | |
| 2882 | + | |
| 2883 | + | |
| 2884 | + | |
| 2885 | + | |
| 2886 | + | |
| 2887 | + | |
| 2888 | + | |
| 2889 | + | |
| 2890 | + | |
| 2891 | + | |
| 2892 | + | |
| 2893 | + | |
| 2894 | + | |
| 2895 | + | |
| 2896 | + | |
| 2897 | + | |
| 2898 | + | |
| 2899 | + | |
| 2900 | + | |
2827 | 2901 | | |
2828 | 2902 | | |
2829 | 2903 | | |
| |||
0 commit comments