Skip to content

Cross-engine parity: pyatr silently drops rules the TypeScript engine accepts (variable-width lookbehind, \u{...}), plus generate-sigma.py loses case-insensitivity #331

Open

Description

@markbriers

Hi Adam,

I was cross-checking last week's new rules against the sigma-ai runtime side and ran into three related regex-portability problems. They all come back to the same thing, which is that the TypeScript engine and the published Python engine (pyatr) do not agree on what a valid pattern is, and a handful of rules fall down the gap. None of it shows up in the TypeScript test run, which is probably why it has gone unnoticed. All line references are against main as of today.

1. Variable-width lookbehind: fine in the TS engine, dead in pyatr

Two of the newer rules use a variable-width lookbehind: ATR-2026-02100 (privilege-escalation, cypher-destructive-admin-query-injection) and ATR-2026-02304 (context-exfiltration, preapproved-domain-encoded-path-exfil). 02100 carries this, for instance:

(?<!\b(?:you will learn that|you will learn|for example|this query|note that|for instance|such as|explains that)\s)\b(?:MATCH|CALL)\b[\s\S]{0,40}\bDETACH\s+DELETE\b

V8's RegExp is happy with a variable-width lookbehind, so this compiles and runs in the TypeScript engine. Python's re is not:

>>> import re
>>> re.compile(r'(?<!\b(?:you will learn that|for example)\s)\bDETACH\s+DELETE\b')
re.error: look-behind requires fixed-width pattern

pyatr compiles with re.compile (engine.py, _compile_regex), and when a pattern fails to compile the on-the-fly path in _evaluate swallows the error:

try:
    regex = _compile_regex(pattern_str)
    return bool(regex.search(normalized)) or bool(regex.search(raw))
except re.error:
    return False

So under pyatr these two rules never match anything, including their own true_positives, and they do it silently. Running either through the test_runner reports the true-positive cases as not_triggered rather than surfacing a compile error.

2. \u{...} escapes: same split

Four of the emoji-payload prompt-injection rules use ECMAScript-style \u{...} ranges: ATR-2026-02004, ATR-2026-02010, ATR-2026-02015 and ATR-2026-02018. For example 02018:

[\u{1F1E6}-\u{1F1FF}]{4,}

The TS engine copes with these because it adds the u flag when it sees \u{ or \p{ (engine.ts, around line 1481: flags = pattern.includes('\u{') || pattern.includes('\p{') ? 'iu' : 'i'). Python re does not understand \u{...} at all:

>>> re.compile(r'[\u{1F1E6}-\u{1F1FF}]{4,}')
re.error: incomplete escape \u at position 1

Same outcome as above, so these four are dead in pyatr, silently, and failing their own positives there. They are also the least portable on the Sigma side, since pySigma and RE2 reject \u{...} as well. Rewriting the ranges with the eight-digit form, for example [\U0001F1E6-\U0001F1FF], compiles cleanly in Python and, with the u flag already set, keeps the TS engine working too.

3. generate-sigma.py loses case-insensitivity (surfaced by ATR-2026-02005)

This one lives in the converter rather than the rules. Both engines run case-insensitively no matter what: pyatr forces re.IGNORECASE in _compile_regex, and the TS engine always includes i in its flags. Plenty of rules quite reasonably rely on that implicit behaviour rather than carrying an inline (?i).

generate-sigma.py, though, only emits the Sigma |re|i modifier when the source pattern starts with (?i); otherwise it emits a case-sensitive |re. So any rule that leans on the engine's implicit case-insensitivity gets exported as case-sensitive Sigma and can quietly stop matching.

02005 is the clean example. Its pattern is lowercase and has no (?i):

\b(actors?|schauspieler\w*)\b[\s\S]{0,250}\b(...|bleiben\s+immer|verharren\s+stets)\b

but its own German true-positive is "... zwei Schauspieler ...". Inside ATR that matches fine, because IGNORECASE is forced. The exported Sigma rule is case-sensitive, so it fails to match its own documented positive.

The blast radius is not tiny. Of the 768 rules, 221 have no inline (?i) on any condition, and 214 of those contain lowercase letters, so they are all being exported case-sensitive while the engines treat them case-insensitive. The simplest fix is for generate-sigma.py to always emit |re|i, or to normalise every source pattern to a leading (?i) before emitting, so that the export matches engine semantics.

Suggestions

For 1 and 2, the smallest per-rule fix is to make the patterns compile under Python re as well as V8: rewrite \u{XXXX} as \U0000XXXX, and rewrite the two lookbehind rules to a fixed-width lookbehind or, better, to an explicit filter-and-negate form. Separately, I would suggest pyatr not swallow re.error silently. A warning at load or evaluate time would have made all six of these visible straight away rather than looking like ordinary non-matches. It might also be worth running the rule test_cases through pyatr in CI and not just the TypeScript engine, which would catch the true-positive regressions automatically.

Happy to send PRs for any of this if it helps.

Cheers,
Mark

Activity

  1. dc-tw commented on Sep 8, 2026

    @dc-tw
    Collaborator

    One additional, independent compile error remains in ATR-2026-00290, even with the proposed Python fallback in #554:

    (?:(?:\w+\s+){40,})\1{3,}

    The pattern references \1, but contains no capturing group. Python re and the fallback regex backend both reject it as an invalid group reference. JavaScript only appears to accept it without the u flag because \1 is interpreted as a legacy octal/control-character escape, so it does not implement the described repetition check there either.

    The first three conditions detect the instruction to repeat something forever. This fourth condition appears to have a different purpose: detecting repetition that has already accumulated in the output. The common observable behavior is unbounded repetition consuming tokens/time, while the rule's stated security objective is divergence into memorized training data. I think we should confirm which boundary the rule is meant to cover before changing it.

    Possible fixes:

    1. Detect one token repeated at least 40 times

      \b(\w{1,30})(?:[\s,]+\1){39,}\b

      This best matches the current description, with bounded token length, but it detects repetitive output rather than the divergence/exfiltration transition itself.

    2. Make the apparent original expression syntactically valid

      ((?:\w+\s+){40,})\1{3,}

      This is the smallest textual repair, but its meaning is a 40-word block repeated four times—not one word repeated 40+ times—and it is likely more expensive than intended.

    3. Detect repeated substrings more generally, similar in spirit to a repeated-token detector, for example:

      (.{10,100}?)\1{2,}

      This covers more repetition shapes, but has the largest false-positive and regex-cost risk and still does not prove training-data extraction.

    4. Remove the fourth condition and keep this rule focused on the explicit repeat-forever instruction. All five current true positives are already covered by the first three conditions. If repeated-output monitoring is desired, it may deserve a separate streaming/output rule with its own positives, negatives, and performance limits.

    My preference is 1 if detecting an ongoing repetition loop is intentional, or 4 if this rule is intended only to detect the attack instruction. I would avoid 2 and 3 unless the broader/long-block semantics are confirmed. Which behavior was intended here?

  2. eeee2345 commented on Sep 22, 2026

    @eeee2345
    Contributor

    @markbriers Seventy days is far too long for a report this precise, and that is on us. Here is where all three items stand, re-measured against main today rather than recalled.

    Item 2, the \u{...} escapes, is fixed. Commit 35847f4 on 2026-08-23 removed them, and there are now no \u{ escapes left anywhere in the rule set. One difference from your suggestion is worth flagging rather than glossing: instead of the eight-digit \U0001F1E6 form, the patterns were rewritten to literal characters, because \U0001... is Python-only and Go wants \x{1F1E6}, so literals are the one spelling all three engines accept. Your four turned out to be part of a larger set than you found: sixteen conditions across ten rules.

    Item 3, generate-sigma.py, is fixed, by @dc-tw in commit aaec966 on 2026-08-26. The emitter now defaults to re|i and normalises a leading (?i) into the modifier instead of only honouring the explicit prefix, with case-sensitive export demoted to an explicit per-condition opt-out. There is a test_generate_sigma.py guarding it.

    Item 1 is still open, and it is the one that matters. On main today there are 3,443 regex conditions, of which 12 fail to compile under Python re. Split the way it should be: 11 are variable-width lookbehind across four rules, ATR-2026-00442, 02100, 02300 and 02304, and the twelfth is a different defect, a backreference in ATR-2026-00290. Your two, 02100 and 02304, are in the lookbehind set. All five of these rules landed before you filed on 2026-07-14, the oldest on 2026-04-20, so the list was already longer than the two you caught rather than having drifted since.

    pyatr 0.3.0 is the published version and still swallows re.error exactly as you quoted.

    PR #554 from @dc-tw implements your do-not-swallow-silently suggestion, warning once per failing condition and not retrying it, plus a regex-module fallback. I re-ran it against the current rule set: four of the five compile under that fallback. ATR-2026-00290 fails under both, because its fourth condition backreferences \1 while the pattern contains no capturing group. That is a rule bug rather than an engine one. He set out the options in this thread and asked which behaviour that condition was meant to cover, and that answer is mine to give rather than his to guess, so it will come from me here.

    Your last suggestion is the one we did not act on and should have. pyatr's tests only run inside publish-python.yml at publish time, not per-PR, so nothing compares the two engines on every change. That absence is the root cause of this whole class of problem landing and staying invisible, more than any individual pattern is.

    The order we are taking it in: #554 for the engine-side warnings and fallback; then a per-PR job that runs every rule's test_cases through pyatr and fails on a compile error or a missed true positive; then 00290 and whatever pattern rewrites that gate surfaces.

    You offered PRs. The per-PR parity job is the one I would most like to come from you, since you have the clearest picture of where the two engines diverge. If you would rather not, say so and we will build it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions