feat: validate regular expressions#300
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds post-parse and post-optimization validation for assembled regular expressions to prevent generating patterns that are incompatible with ModSecurity/PCRE 8-bit mode (e.g., rejecting large \x{...} code points and multi-byte literals inside character classes).
Changes:
- Introduces a new
regex/validationpackage with codepoint and character-class validation plus unit tests. - Hooks validation into the regex assembler (pre- and post-processing) and into
crs-toolchain regex format. - Simplifies several scanners by removing redundant
scanner.Split(bufio.ScanLines)calls and updates parser API/tests to reflect the newParsesignature.
Reviewed changes
Copilot reviewed 16 out of 17 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| util/renumber_tests.go | Removes redundant scanner split setup. |
| regex/validation/regex_validation.go | Adds core regex validation logic and custom scanner split funcs. |
| regex/validation/regex_validation_test.go | Adds unit tests for validation behavior. |
| regex/parser/parser.go | Changes Parse API and adjusts write handling. |
| regex/parser/parser_test.go | Updates parser tests for new Parse signature. |
| regex/parser/include_with_definition_test.go | Updates parser tests for new Parse signature. |
| regex/parser/include_test.go | Updates parser tests for new Parse signature. |
| regex/parser/include_multiple_test.go | Updates parser tests for new Parse signature. |
| regex/parser/include_except_test.go | Updates parser tests for new Parse signature. |
| regex/parser/include_except_builder.go | Removes redundant scanner split setup. |
| regex/parser/definition_test.go | Updates parser tests for new Parse signature. |
| regex/operators/assembler.go | Adds pre/post validation around assembly and propagates final-pass errors. |
| regex/operators/assembler_test.go | Adds regression tests for invalid escapes / validation failures. |
| cmd/regex/format/format.go | Runs validation when formatting regex-assembly files. |
| chore/update_copyright/update_copyright.go | Removes redundant scanner split setup. |
| go.mod | Updates indirect dependencies. |
| go.sum | Updates dependency checksums. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Member
|
Maybe creating a new type interface for validation and then register the validations into a list, one list for pre, another for post. That way you just perform |
theseion
force-pushed
the
validate-regex
branch
2 times, most recently
from
April 5, 2026 07:21
d48d807 to
4215177
Compare
Contributor
Author
|
I've added a |
We don't have full control over what rassemble-go and regexp generate.
This can lead to unexpected situations, where valid input regex becomes
invalid. For example `[\s\S]+` will be expanded to `[\x00-\x{10FFFF}]`
by regexp, but ModSecurity can only handle single byte escape codes (up
to `\x{ff}`).
This change introduces validation logic for regular expressions after
initial parsing and after final optimization.
Currently, two validations have been implemented:
- valid Unicode hex escapes (for ModSecurity (PCRE 8-bit))
- no multi-byte characters in character classes (in a character class
without UTF-8 support enabled, a multibyte character is essentially a
logical disjunction of the individual bytes, instead of a sequence,
which can lead to unexpected results)
Fixes #290
fzipi
reviewed
Apr 5, 2026
fzipi
approved these changes
Apr 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We don't have full control over what rassemble-go and regexp generate. This can lead to unexpected situations, where valid input regex becomes invalid. For example
[\s\S]+will be expanded to[\x00-\x{10FFFF}]by regexp, but ModSecurity can only handle single byte escape codes (up to\x{ff}).This change introduces validation logic for regular expressions after initial parsing and after final optimization.
Currently, two validations have been implemented:
Fixes #290