Skip to content

fix(privacy): redact AWS path keys and URL-safe Basic credentials - #1106

Merged
pengfei-threemoonslab merged 2 commits into
ThreeMoonsLab:mainfrom
Subodh-17:fix/issue-1082-aws-basic-redaction
Nov 6, 2026
Merged

pengfei-threemoonslab merged 2 commits into
ThreeMoonsLab:mainfrom
Subodh-17:fix/issue-1082-aws-basic-redaction

Conversation

@Subodh-17

Copy link
Copy Markdown
Contributor

Summary

Fix credential redaction gaps involving AWS secret access keys embedded in filesystem paths and URLs, and Basic authentication credentials encoded using URL-safe or unpadded Base64.

Changes

  • Correct AWS secret-key boundary handling for path and URL separators.
  • Support URL-safe Base64 characters in Basic authentication tokens.
  • Normalize valid unpadded Base64 credentials before strict decoding.
  • Preserve existing credential validation and redaction behavior.
  • Add regression coverage for adversarial inputs and lookalike text.

Validation

  • 32 adversarial AWS and Basic Auth cases passed.
  • Privacy regression tests passed as documented in the implementation report.
  • Shard partition suite: 29 passed.
  • Ruff checks passed.
  • Independent security and regression audit completed.

Scope

Two files changed:

  • src/agents_shipgate/core/privacy.py
  • tests/test_privacy.py

Closes #1082

…ed Basic credentials

An unlabeled AWS secret access key inside a path or URL segment
(path/<key>, s3://bucket/<key>/obj, https://h/<key>?a=1) was skipped
because the lookbehind (?<![A-Za-z0-9/+]) and lookahead (?![A-Za-z0-9/+=])
disallowed adjacent '/' characters. The pattern now permits '/'
separators before and after the 40-character AWS base64 alphabet shape,
relying on the existing _is_random_token check to preserve word-like
paths (such as Sources/PaymentKit/Handlers/RefundTools2).

Unlabeled Basic authentication tokens formatted with URL-safe base64
(RFC 4648 §5) or without standard '=' padding were skipped because
BASIC_AUTH_RE only matched [A-Za-z0-9+/] and _is_basic_credential
passed unpadded strings directly to b64decode(validate=True). The
regex now admits URL-safe characters '-' and '_', and
_is_basic_credential normalizes and re-pads unpadded tokens prior to
decoding and validating the 'user:password' UTF-8 structure.

Closes ThreeMoonsLab#1082

@pengfei-threemoonslab pengfei-threemoonslab left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking up #1082. I'm approving this.

@pengfei-threemoonslab
pengfei-threemoonslab merged commit 1e3e372 into ThreeMoonsLab:main Nov 6, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

privacy: redact AWS keys inside path or URL segments and URL-safe or unpadded Basic credentials

2 participants