You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 9a48f81
Browse filesBrowse the repository at this point in the historyBrowse files
[fix](be) Fall back from expensive Hyperscan bounded repeats (apache#66788)
Problem Summary:
Hyperscan can spend excessive time and memory compiling bounded
repetitions with large limits. Doris now detects regex quantifiers above
50 before every direct Hyperscan compiler entry point. LIKE and REGEXP
paths with a compatible fallback use RE2; compiler paths without a
fallback reject the pattern before Hyperscan compilation.
The guard follows ClickHouse SlowWithHyperscanChecker and searches the
raw regular expression. This intentionally uses the same conservative
best-effort classification instead of maintaining a partial Hyperscan
grammar parser.
The new `enable_hyperscan_fallback` session variable controls
strictness. It defaults to `true`, preserving fallback behavior. When
set to `false`, Doris returns the interception or Hyperscan compilation
error instead of falling back.
Normal query execution and request-scoped synchronous Stream Load
propagate this option through `TQueryOptions`. Asynchronous Broker Load
and Routine Load session inheritance are outside this PR scope and
retain their historical option defaults.
### Release note
Add `enable_hyperscan_fallback`. It defaults to `true`; setting it to
`false` returns an error when Hyperscan cannot compile or safely process
the regular expression on supported request-scoped paths.
### Check List (For Author)
- Test:
- Targeted BE unit tests for LIKE/REGEXP, MATCH_REGEXP, multi-match, and
inverted-index regexp v1/v2
- `DORIS_THIRDPARTY=$PWD/thirdparty FE_UT_PARALLEL=48 ./run-fe-ut.sh
--run
org.apache.doris.nereids.load.NereidsLoadScanProviderTest,org.apache.doris.qe.SessionVariablesTest`
(25 passed)
- `DORIS_THIRDPARTY=$PWD/thirdparty ./build.sh --fe -j 48`
- BE clang-format and clang-tidy
- Behavior changed: Yes. Expensive bounded repeats are intercepted
before Hyperscan compilation, and request-scoped sessions may disable
the RE2 fallback. The default remains fallback enabled.
- Does this need documentation: No
0 commit comments