You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(analyzer): Support delimited and ASR comma-grouped Turkish National IDs (TCKN) in TrNationalIdRecognizer #2451
Is your feature request related to a problem? Please describe. TrNationalIdRecognizer (TR_NATIONAL_ID) currently defines a replacement_pairs parameter in __init__ and invokes EntityRecognizer.sanitize_value(pattern_text, self.replacement_pairs) in validate_result, indicating an intent to support formatted/delimited Turkish National ID numbers (TCKN). However, there are two issues in the current implementation:
Delimited and ASR comma-grouped TCKNs are missed (replacement_pairs is dead code):self.replacement_pairs defaults to [], and PATTERNS only contains r"\b[1-9][0-9]{10}\b" (11 contiguous digits). As a result, TCKNs formatted with spaces (100 000 001 46), hyphens (100-000-001-46), or comma-separated digit groups produced by Speech-to-Text / ASR models during speaker pauses (100, 000, 001, 46) never match the regex and never reach validate_result.
Dummy repeated-digit sequences pass the checksum: Mathematical placeholder sequences where the first 10 digits are identical and the 11th digit is 0 (e.g., 11111111110, 22222222220, ..., 99999999990) satisfy the NVI modulo-10 equations ((5d * 7 - 4d) % 10 == d and 10d % 10 == 0) and are currently promoted to 1.0 confidence because invalidate_result is not implemented.
Describe the solution you'd like
Enhance TrNationalIdRecognizer in presidio_analyzer/predefined_recognizers/country_specific/turkey/tr_national_id_recognizer.py:
Set default replacement_pairs to [("-", ""), (" ", ""), (",", "")] when replacement_pairs is None (while still respecting an explicit replacement_pairs=[]).
Add a delimited pattern supporting Turkish TCKN digit groupings (3-3-3-2, 3-2-2-2-2, and 4-4-3) separated by spaces, hyphens, or ASR pause commas (", " / ","), bounded with negative lookbehind/lookahead assertions so partial slices of longer delimited number sequences are not matched.
Implement invalidate_result (and validation checks) to reject inconsistent/mixed delimiters (e.g., 100-000 001-46) and dummy repeated-digit sequences (11111111110–99999999990).
Add unit and YAML configuration-path tests (RecognizerRegistryProvider + AnalyzerEngine) in tests/test_tr_national_id_recognizer.py.
Describe alternatives you've considered
Relying on custom user patterns per deployment, which leaves the built-in replacement_pairs parameter in TrNationalIdRecognizer non-functional out of the box and fails to filter dummy repeated-digit TCKNs.
Additional context
In Turkish call-center ASR (e.g. Whisper) transcripts and formatted documents, 11-digit TCKNs are commonly grouped as 3-3-3-2 (100 000 001 46, 100-000-001-46, or 100, 000, 001, 46). Supporting these groupings while enforcing delimiter consistency and boundary checks prevents both false negatives on speech transcripts and false positives on unrelated numeric sequences.
Activity