-
Notifications
You must be signed in to change notification settings - Fork 715
feat: add hash replace feature in SDR #8803
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Explore these optional code suggestions:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Overview
Summary
This PR introduces a new Hash policy option for regex pattern matching in SDR (Sensitive Data Redaction), allowing matched values to be replaced with searchable hash prefixes instead of redacting or dropping them.
Key Changes:
- Added
ZO_RE_PATTERN_HASH_LENGTHconfiguration variable (default: 12, range: 12-64 characters) with validation - Extended
PatternPolicyenum withHashvariant alongside existingRedactandDropFieldoptions - Updated pattern test endpoint to accept optional
policyparameter, maintaining backward compatibility by defaulting to "Redact" - Added UI radio button for Hash policy with description "Replace with searchable hash"
- Properly threaded policy parameter from frontend through API to backend handler
Architecture:
The implementation follows a clean layered approach: config validation at startup, enum extension in the data model, optional parameter handling in the API with sensible defaults, and UI updates. The actual hashing logic resides in the enterprise PatternManager component (not visible in this PR).
Confidence Score: 5/5
- This PR is safe to merge with minimal risk
- The changes are well-structured and follow defensive coding practices: config validation with clear bounds, backward-compatible API changes with sensible defaults, proper enum extension with serialization support, and clean UI integration. No breaking changes are introduced.
- No files require special attention
Important Files Changed
File Analysis
| Filename | Score | Overview |
|---|---|---|
| src/config/src/config.rs | 5/5 | Added RePattern config struct with hash_length parameter (default 12, range 12-64) and validation logic |
| src/handler/http/request/re_pattern/mod.rs | 5/5 | Added optional policy parameter to test endpoint, defaults to "Redact" for backward compatibility |
| src/infra/src/table/re_pattern_stream_map.rs | 5/5 | Added Hash variant to PatternPolicy enum with proper serialization support |
| web/src/components/logstream/AssociatedRegexPatterns.vue | 5/5 | Added Hash radio button option to UI with description "Replace with searchable hash", passes policy to test function |
| web/src/services/regex_pattern.ts | 5/5 | Added optional policy parameter to test function, conditionally includes it in payload |
Sequence Diagram
sequenceDiagram
participant User
participant UI as Vue Component
participant API as regex_pattern.ts
participant Handler as re_pattern/mod.rs
participant PM as PatternManager (Enterprise)
participant Config as Config System
Note over Config: ZO_RE_PATTERN_HASH_LENGTH<br/>default: 12, range: 12-64
User->>UI: Select Hash policy
User->>UI: Enter test pattern & records
UI->>API: test(org, pattern, records, "Hash")
API->>Handler: POST /api/{org}/re_patterns/test
Note over API: payload: {pattern, test_records, policy: "Hash"}
Handler->>Handler: Parse policy (default "Redact")
Handler->>Handler: Convert to PatternPolicy::Hash
Handler->>PM: test_pattern(pattern, input, Hash)
Note over PM: Uses config.re_pattern.hash_length<br/>to generate hash prefix
PM-->>Handler: Hashed result
Handler-->>API: PatternTestResponse
API-->>UI: Display hashed output
UI-->>User: Show result with searchable hash
5 files reviewed, no comments
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 364 | 335 | 0 | 19 | 10 | 92% | 5m 1s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 363 | 330 | 0 | 19 | 14 | 91% | 5m 0s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 364 | 331 | 0 | 19 | 14 | 91% | 5m 0s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 364 | 337 | 0 | 19 | 8 | 93% | 4m 37s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 364 | 338 | 0 | 19 | 7 | 93% | 4m 40s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 364 | 335 | 0 | 19 | 10 | 92% | 4m 55s |
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 364 | 334 | 0 | 19 | 11 | 92% | 5m 12s |
Testdino Test Results
|
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 364 | 342 | 0 | 19 | 3 | 94% | 4m 39s |
Testdino Test Results
|
Adds a config variable to control the length of hash UI changes to accomodate Hash pattern policy <img width="720" height="410" alt="image" src="https://github.com/user-attachments/assets/b99e28ec-2b7d-431e-9606-7b855df62f8d" /> <img width="2037" height="1112" alt="image (3)" src="https://github.com/user-attachments/assets/67432a01-c598-4b0a-830b-137a4cd04bf9" /> --------- Co-authored-by: Shrinath Rao <[email protected]>
Adds a config variable to control the length of hash
UI changes to accomodate Hash pattern policy