Skip to content

fix(gmail): detect HTML fragments in LooksLikeHTML to prevent signature escaping#879

Merged
steipete merged 1 commit into
openclaw:mainfrom
kesslerio:fix/gmail-signature-html-escaping
Jun 24, 2026
Merged

fix(gmail): detect HTML fragments in LooksLikeHTML to prevent signature escaping#879
steipete merged 1 commit into
openclaw:mainfrom
kesslerio:fix/gmail-signature-html-escaping

Conversation

@kesslerio

@kesslerio kesslerio commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

What

LooksLikeHTML only detected document-level HTML (<!doctype>, <html>, <head>, <body>, <meta>). When --signature-file pointed at a file starting with <style> or <table> — common in HTML email signatures — it failed detection and fell through to escapeTextToHTML, which HTML-escaped the entire signature. Recipients saw &lt;style&gt; and &lt;table&gt; as visible text instead of rendered HTML.

Why

readComposeSignatureFile in internal/cmd/gmail_send_signature.go:86 calls LooksLikeHTML to decide whether to preserve raw HTML or escape it. A <style>-prefixed signature file is valid HTML but not a full document, so it was misclassified as plain text and escaped.

Changes

  1. internal/gmailcontent/content.go: Replace LooksLikeHTML with a compiled regex (htmlFragmentPattern) that matches real HTML tags and comments using \b word boundary after tag names. This prevents false positives on angle-bracketed plain text like <path>, <PRIVATE_PERSON>, or <[email protected]>.

  2. internal/cmd/backup_export_gmail.go: Delete dead looksLikeHTMLFragment duplicate — LooksLikeHTML is now a superset. Simplify backupEmailMarkdownText to call LooksLikeHTML alone.

  3. internal/cmd/gmail_reply.go: When a text/plain part contains HTML fragments and there is no separate text/html part, carry the fragment into BodyHTML so --quote replies don't lose the original message body.

  4. internal/gmailcontent/lookslikehtml_test.go: Add negative tests for angle-bracketed plain text (<path>, <project>, <[email protected]>, <PRIVATE_PERSON>, <quick brown>).

Tests

go test ./internal/gmailcontent/ -run TestLooksLikeHTML -v
go test ./internal/cmd/ -run 'Gmail|Signature|Reply|Quote|Compose|HTML|Backup' -short
  • TestLooksLikeHTML_DocumentPrefixes — existing document-level detection unchanged
  • TestLooksLikeHTML_FragmentMarkers<style>, <table>, <div>, <p>, <br>, etc. detected as HTML
  • TestLooksLikeHTML_PlainText — plain text, bare URLs, <path>, <PRIVATE_PERSON>, <[email protected]>, a < b not misdetected
  • All existing Gmail reply, compose, and backup tests pass

Real Behavior Proof

Before fix (broken email, Gmail message ID 19ef630fa26b22a2):
Raw RFC822 text/html section shows escaped signature:

<div class="gmail_signature">&lt;style type=&#34;text/css&#34;&gt;<br>
    a[x-apple-data-detectors] {<br>

After fix (verification email, Gmail message ID 19ef6defcf54663e):
Sent gog gmail send --to [email protected] --body-html "$BODY" --signature-file signature-martin-thread-short-no-ps.html. Raw RFC822 text/html section:

text/html; charset="utf-8"
Content-Transfer-Encoding: 7bit

<div style="font-family: Arial, sans-serif;"><p>Verification test - signature HTML fix proof.</p></div>

<div class="gmail_signature"><style type="text/css">
    a[x-apple-data-detectors] {
        color: inherit !important;

Signature HTML is raw (<style type="text/css">), not escaped (&lt;style&gt;). Fix confirmed.

AI Assistance

This PR was authored with AI assistance (Claude Code / OpenCode + GLM 5.2). The bug was discovered when an OpenClaw agent sent a BIPA legal acknowledgement email to counsel at GRSM and the HTML signature was delivered as escaped visible text. Root cause traced through the raw RFC822 message to html.EscapeString output matching exactly.

@clawsweeper

clawsweeper Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 23, 2026, 11:20 PM ET / 03:20 UTC.

Summary
The branch adds a fragment-aware Gmail HTML detector, switches signature-file and backup-export classification to it, and adds focused regression tests plus an unreleased changelog note.

Reproducibility: yes. Source inspection shows current main accepts plain text or HTML signature files, but <style>/

fragments fall through the document-only detector and get escaped; the PR body also includes live raw RFC822 before/after proof.

Review metrics: 2 noteworthy metrics.

  • Patch surface: 7 files, +151/-19. The diff is bounded but touches a shared Gmail content classifier plus send, backup, and tests.
  • Runtime classifier scope: 2 call sites changed. The current head changes signature-file and backup-export behavior while leaving display and reply paths on the legacy detector.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] Existing plain-text signature files that intentionally contain exact HTML tag text such as

    or may now be treated as HTML and sent raw instead of escaped.

  • [P1] The fragment helper is shared with backup export, so classifier false positives or false negatives can change exported Markdown body cleanup as well as outgoing signature rendering.

Maintainer options:

  1. Land The Scoped Helper (recommended)
    Accept the compatibility risk because the current head keeps display/reply behavior unchanged, fixes the signature-file bug, and has focused regression coverage.
  2. Narrow To Signature Files
    If backup export classification should not change in this PR, keep the new fragment detector only on readComposeSignatureFile and leave backup behavior for a separate review.
  3. Require Explicit HTML Selection
    If maintainers want zero ambiguity for literal tag-like plain text, pause this patch and design an explicit HTML signature-file signal instead of relying on heuristics.

Next step before merge

  • No automated repair is needed; the remaining action is maintainer acceptance of the compatibility and delivered-message classifier risk.

Security
Cleared: The diff changes Gmail content classification and tests only; it adds no dependencies, workflows, secret handling, or external code execution paths.

Review details

Best possible solution:

Land the current scoped helper if maintainers accept heuristic fragment detection for explicit signature files and backup cleanup; otherwise narrow the behavior to the signature-file path or require a more explicit HTML signal.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main accepts plain text or HTML signature files, but <style>/

fragments fall through the document-only detector and get escaped; the PR body also includes live raw RFC822 before/after proof.

Is this the best way to solve the issue?

Yes, with maintainer acceptance of the heuristic edge cases. The current scoped helper is narrower than the earlier shared-classifier approach and covers the reported bug plus prior false-positive cases.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 57e3283a520a.

Label changes

Label justifications:

  • P2: This is a normal-priority Gmail formatting bug with a bounded patch and limited blast radius.
  • merge-risk: 🚨 compatibility: The new fragment heuristic can change how existing plain-text signature files and backup text are classified during upgrade.
  • merge-risk: 🚨 message-delivery: Signature classification directly affects delivered Gmail content, including whether markup is sent raw or escaped.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (logs): The PR body includes real Gmail raw RFC822 before/after logs showing the HTML signature fragment preserved after the fix.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes real Gmail raw RFC822 before/after logs showing the HTML signature fragment preserved after the fix.
Evidence reviewed

What I checked:

  • Repository policy checked: AGENTS.md was read fully; its PR review-mode guidance applied, so review used gh pr view/diff and did not switch branches or edit files. (AGENTS.md:31, 57e3283a520a)
  • Documented signature-file contract: Current docs advertise --signature-file as accepting a local signature file in plain text or HTML, so preserving HTML signature fragments fits an existing command contract. (docs/commands/gog-gmail-send.md:52, 57e3283a520a)
  • Current-main bug path: On current main, readComposeSignatureFile preserves raw HTML only when LooksLikeHTML returns true; otherwise the file is converted through escapeTextToHTML. (internal/cmd/gmail_send_signature.go:86, 57e3283a520a)
  • Current-main detector is narrow: LooksLikeHTML only recognizes document-level HTML prefixes or contained <html, so a signature beginning with <style> or
is classified as plain text. (internal/gmailcontent/content.go:127, 57e3283a520a)
  • Current PR scope: The current patch adds LooksLikeHTMLFragment and uses it for signature files and backup export; display and reply paths still call the legacy LooksLikeHTML helper. (internal/gmailcontent/content.go:139, 185cb7755da4)
  • Regression coverage: The PR adds positive fragment tests and negative placeholder/address/path tests, including cases for <PRIVATE_PERSON>, [email protected], dotted tags, and colon-tag text. (internal/gmailcontent/lookslikehtml_test.go:26, 185cb7755da4)
  • Likely related people:

    • steipete: History for the signature-file path, Gmail content decoder, backup export helper, and v0.30.0 snapshot points to this account as the main current-main owner for the affected files. (role: introduced behavior and recent area contributor; confidence: high; commits: 77a16d10efde, 8a58cf7d0b12, 1eaad2556cbe; files: internal/cmd/gmail_send_signature.go, internal/gmailcontent/content.go, internal/cmd/backup_export_gmail.go)
    • malob: Recent merged Gmail quote-formatting work touched the nearby reply behavior that earlier review comments considered when this PR had a broader classifier scope. (role: recent adjacent contributor; confidence: medium; commits: e35aad920a97; files: internal/cmd/gmail_reply.go)
    What the crustacean ranks mean
    • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
    • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
    • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
    • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
    • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
    • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
    • 🌊 off-meta tidepool: rating does not apply to this item.

    Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

    How this review workflow works
    • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
    • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
    • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
    • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
    • Maintainers can also comment @clawsweeper review to request a fresh review only.
    • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
    • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
    • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

    @clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 message-delivery 🚨 Merging this PR could drop, duplicate, misroute, suppress, or wrongly target messages. labels Jun 23, 2026

    @chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

    Copy link
    Copy Markdown

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    💡 Codex Review

    Here are some automated review suggestions for this pull request.

    Reviewed commit: 897723f8dc

    ℹ️ About Codex in GitHub

    Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

    • Open a pull request for review
    • Mark a draft as ready
    • Comment "@codex review".

    If Codex has suggestions, it will comment; otherwise it will react with 👍.

    Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

    Comment thread internal/gmailcontent/content.go Outdated
    }

    for _, marker := range htmlFragmentMarkers {
    if strings.Contains(trimmed, marker) {

    Copy link
    Copy Markdown

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    P2 Badge Constrain fragment markers to real HTML tags

    Because this now treats any occurrence of marker prefixes as HTML, plain text containing angle-bracket placeholders can be misclassified. For example, a --signature-file with Best,\n<PRIVATE_PERSON> matches the "<p" marker here, so readComposeSignatureFile skips escapeTextToHTML and stores the raw value as HTML; mail clients parse <PRIVATE_PERSON> as an unknown tag, so the placeholder disappears instead of being shown literally. The same false positive can affect text/plain message display paths, so the fragment check should require a real tag boundary such as whitespace, >, or / after the tag name.

    Useful? React with 👍 / 👎.

    Comment thread internal/gmailcontent/content.go Outdated
    }

    for _, marker := range htmlFragmentMarkers {
    if strings.Contains(trimmed, marker) {

    Copy link
    Copy Markdown

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    P2 Badge Preserve text/plain HTML fragments when quoting

    When a message has only a text/plain part that contains an HTML fragment such as <div>Hello</div> and no separate text/html part, this new marker match makes LooksLikeHTML return true. In replyInfoFromMessage (internal/cmd/gmail_reply.go:147-150) that suppresses info.Body, while BodyHTML remains empty, so applyQuoteToBodies treats the original body as absent and gog gmail reply --quote drops the quote entirely; before this change, fragments were still included in the quote after escaping. Consider carrying the fragment into BodyHTML or converting it to text instead of using this broader predicate to discard it.

    Useful? React with 👍 / 👎.

    @kesslerio

    Copy link
    Copy Markdown
    Contributor Author

    @clawsweeper re-review

    @clawsweeper

    clawsweeper Bot commented Jun 23, 2026

    Copy link
    Copy Markdown
    Contributor

    🦞🧹
    ClawSweeper re-review requested.

    I asked ClawSweeper to review this item again.
    Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
    Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

    @clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jun 23, 2026
    @kesslerio

    Copy link
    Copy Markdown
    Contributor Author

    @clawsweeper re-review

    @clawsweeper

    clawsweeper Bot commented Jun 24, 2026

    Copy link
    Copy Markdown
    Contributor

    🦞🧹
    ClawSweeper re-review requested.

    I asked ClawSweeper to review this item again.
    Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
    Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

    @clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jun 24, 2026
    @steipete
    steipete force-pushed the fix/gmail-signature-html-escaping branch from 4fbfb44 to 185cb77 Compare June 24, 2026 03:15

    @chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

    Copy link
    Copy Markdown

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    💡 Codex Review

    Here are some automated review suggestions for this pull request.

    Reviewed commit: 185cb7755d

    ℹ️ About Codex in GitHub

    Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

    • Open a pull request for review
    • Mark a draft as ready
    • Comment "@codex review".

    If Codex has suggestions, it will comment; otherwise it will react with 👍.

    Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

    // is required: > (close tag), / (self-closing), or whitespace (start of attributes).
    // Punctuation like @, ., : does not count as a boundary.
    var htmlFragmentPattern = regexp.MustCompile(
    `<!--|</?(?:p|br|div|span|table|tr|td|section|blockquote|a|img|font|style)[\s/>]`,

    Copy link
    Copy Markdown

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    P2 Badge Detect inline-only HTML signatures

    When a local --signature-file is valid HTML made only of inline tags such as <b>Peter</b> or <strong>Peter</strong>, this whitelist does not match, so readComposeSignatureFile falls through to the plaintext branch and sends escaped markup (&lt;b&gt;...) instead of an HTML signature. The option is documented as accepting plain text or HTML, so valid inline-only fragments need to be detected here or handled by a signature-specific parser/path.

    Useful? React with 👍 / 👎.

    @steipete
    steipete merged commit 947b7cf into openclaw:main Jun 24, 2026
    5 checks passed
    @steipete

    Copy link
    Copy Markdown
    Collaborator

    Landed as 947b7cfd.

    Validation:

    • go test ./internal/gmailcontent ./internal/cmd -run 'LooksLikeHTML|Signature|BackupEmailMarkdownBody|Reply|Quote'
    • make ci
    • Final Codex autoreview: clean; no accepted/actionable findings
    • PR CI: Linux test/lint/docs, Windows, macOS/cgo, worker, and Docker all green on the exact landed head
    • Contributor live proof: raw RFC822 before/after showed the signature <style> fragment changed from escaped text to preserved HTML

    Maintainer adjustments: restricted fragment detection to signature-file and backup-export inputs, preserved the existing document-only classifier for message display/reply quoting, added direct signature-file coverage, and added backup regression coverage for literal <code>/<kbd> text. Autoreview identified the broader tag list as a possible backup data-loss regression; the landed detector retains only the pre-existing backup fragment tag set with stricter boundaries.

    Caveat: I did not repeat the live Gmail send; the contributor's raw-message proof remains the live delivery evidence.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Labels

    merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 message-delivery 🚨 Merging this PR could drop, duplicate, misroute, suppress, or wrongly target messages. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

    Projects

    None yet

    Development

    Successfully merging this pull request may close these issues.

    2 participants