fix(privacy): include source-omission redactions in final privacy audit snapshot (#586) - #1141

Summary of Changes

This pull request resolves Issue #586 by ensuring that all redactions performed on SourceSurfaceOmission objects are accounted for in the sealed PrivacyAudit snapshot and propagated consistently to downstream reviewer artifacts.

Root Cause

Previously in _sanitize_for_output(), source_omissions were sanitized inline as an argument to _SanitizedSurfaces(...), which occurred after build_privacy_audit() had already sealed the redaction statistics. Consequently, any secret tokens in source omission fields (subject, detail, warning) incremented privacy_stats too late to be included in report.privacy_audit, causing discrepancy with plan.privacy_stats and downstream consumers.

Implementation

In src/agents_shipgate/cli/scan/sanitization.py:

• Moved public_source_omissions list comprehension directly above _sanitize_source_recovery_evidence() and build_privacy_audit().
• Passed the pre-sanitized public_source_omissions into _SanitizedSurfaces.
• Preserved all existing public API contracts, schemas, redaction thresholds, and rule versions (RULES_VERSION = "0.2").

Closes #586

Regression Testing & Verification

• Added Tests in tests/test_privacy.py (494 lines):

  1. test_duplicate_mcp_omission_redactions_included_in_privacy_audit_and_outputs: End-to-end multi-artifact test checking exact occurrence count (11), path counts, reviewer summary synchronization, and absence of raw secrets or SHA-256 digests across 7 generated report files.
  2. test_source_omission_field_redactions_propagate_to_audit_and_reviewer_summary: Parametrized across 5 distinct test vectors:
     • Field isolation: subject alone (openai_api_key)
     • Field isolation: detail alone (openai_api_key)
     • Field isolation: warning alone (stripe_key)
     • Multi-token combination: 4 secrets across subject, detail, and warning (github_token, openai_api_key, stripe_webhook_secret)
     • Legacy baseline compatibility note emission
  3. test_multiple_duplicate_omissions_and_pre_sanitized_warning_do_not_double_count: 3-package duplicate MCP configuration confirming deduplicated warnings and multi-omissions calculate exact occurrences (14) without double-counting.
  4. test_clean_source_omissions_produce_zero_privacy_redactions: Negative control verifying clean omissions yield occurrence_count == 0 and empty redacted_paths.
  5. test_scan_source_omission_redactions_end_to_end: Scan pipeline test with POSIX guard (@pytest.mark.skipif(os.open not in os.supports_dir_fd)).

• Passing Checks:

  • python -m pytest tests/test_privacy.py -v -k "omission": 8 passed, 1 skipped in 2.36s.
  • python -m pytest tests/test_privacy.py -v -k "not test_scan and not test_a_40 and not test_a_finding and not test_verify": 81 passed in 16.09s.
  • python -m pytest tests/test_schema_roundtrip.py tests/test_reviewer_summary.py -k "not test_check_mode" -q: 48 passed in 3.15s.
  • python scripts/verify_dependency_lock.py: All 4 lock closures verified.
  • ruff check . & compileall -q src tests: 0 errors.

Platform Environmental Limitations & Failure Reconciliation

pytest tests/test_surface_exclusions.py tests/test_rule_credential_redaction.py tests/test_permission_rule_url_redaction.py tests/test_workflow_label_redaction.py -q

Reviewer Checklist

• public_source_omissions are sanitized prior to calling build_privacy_audit().
• privacy_stats accurately captures all secret tokens in subject, detail, and warning fields.
• reviewer_summary.privacy_redactions matches privacy_audit.redacted_occurrence_count.
• No raw secrets or unmasked SHA-256 digests appear in rendered reports or paths.
• Clean omissions produce 0 redactions.
• PrivacyAudit schema, redaction thresholds, and public contracts remain unchanged.