You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Exported Python for Regular Expression and Substring Search matches non-ASCII text differently from the engine #8989
The exported Python for Regular Expression and Substring Search disagrees with the engine on non-ASCII text.
Regular Expression: Java's Pattern ignores case only for ASCII letters unless UNICODE_CASE is set, and its \w, \d and \s match only ASCII. Python's re is Unicode by default. So with Case Insensitive on, "é" matches "É" in the export and not in the engine, and with it off, \w+ matches "é" and \d matches "٣" in the export only.
Substring Search: with Case Sensitive off, the engine compares toLowerCase of both sides, and pandas' str.contains(case=False) compares upper(). So "ss" matches "ß", "ς" matches "σ" and "ff" matches "FF" in the export only, and "i̇" matches "İ" in the engine only.
How to reproduce?
Feed a table with a string column holding "É" into Regular Expression with regex "é" and Case Insensitive on. The engine keeps no row.
Export the workflow as Python and run it on the same table. It keeps the row.
Likewise, Substring Search with substring "ss", Case Sensitive off, on a row holding "ß": the engine keeps no row, the export keeps it.
Activity
fix(workflow-operator): match non-ASCII text in Regex and Substring S…
feat(workflow-operator): match non-ASCII text in Regex and Substring …