Skip to content

fix(msteams): proactive sends fail after conversation migration#104691

Open
jimmypuckett wants to merge 1 commit into
openclaw:mainfrom
jimmypuckett:spinen/msteams-conversation-migration-safety
Open

fix(msteams): proactive sends fail after conversation migration#104691
jimmypuckett wants to merge 1 commit into
openclaw:mainfrom
jimmypuckett:spinen/msteams-conversation-migration-safety

Conversation

@jimmypuckett

Copy link
Copy Markdown
Contributor

Closes #94939.

What Problem This Solves

Fixes an issue where Microsoft Teams users could lose proactive Bot Framework sends after 6.x state migration when a non-empty legacy msteams-conversations.json source produced no conversation references visible in shared plugin state.

When that happens, archiving the legacy source silently removes recoverable Teams conversation data. The bot then cannot send proactively to those conversations until it receives a new message.

Why This Change Was Made

The migration now treats a non-empty legacy Teams conversation source with zero visible migrated rows as unsafe to archive. It leaves the legacy source in place and reports a warning/change result instead of silently discarding recoverable conversation references.

The change is intentionally narrow: empty legacy sources and successfully verified migrations keep the existing archive behavior. This does not attempt to reinterpret every possible legacy migration failure.

User Impact

Teams users and operators keep a recoverable legacy conversation reference source when migration cannot verify that references landed in shared plugin state. That reduces the risk of proactive Teams sends breaking after state migration and gives operators a clear warning instead of silent data loss.

Evidence

Regression/source validation:

  • node scripts/run-vitest.mjs extensions/msteams/doctor-contract-api.test.ts
  • node scripts/run-vitest.mjs extensions/msteams
  • pnpm lint --threads=8
  • pnpm run lint:extensions:bundled
  • pnpm run lint:extensions:channels
  • git diff --check

Redacted local gateway validation:

{
  "installedBundle": {
    "migrationGuardPresentInPatchedBundle": true,
    "migrationGuardPresentInBaselineBundle": false,
    "patchedDoctorContractSha256": "bb75fa0ebca072a1393c359968124e36e8895c0126538d51ad44698e4d4ad629",
    "baselineDoctorContractSha256": "43b77bdaf0da989857df4190b653318e5cdb8d42006a7ef1560f4804c11192ae"
  },
  "runtime": {
    "gatewayProcess": "running",
    "msteamsListener": "listening"
  },
  "stateCounters": {
    "msteamsConversationReferences": 20,
    "msteamsSentMessageRecordsBeforeProactiveSend": 6,
    "msteamsSentMessageRecordsAfterProactiveSend": 7
  },
  "teamsSmoke": {
    "inboundDmReceived": true,
    "botReplyObserved": true,
    "proactiveSendDryRun": false,
    "sentMessageRecordsIncreasedAfterProactiveSend": true,
    "latestPersonalConversationTimestampUpdatedAfterProactiveSend": true,
    "latestPersonalConversationUpdatedAt": "2026-07-02T02:30:21.367Z"
  },
  "privacy": {
    "conversationIds": "redacted",
    "tenantAndUserIds": "redacted",
    "messageBodiesFromLogs": "not included",
    "localHostnamesAndUsernames": "not included"
  }
}

Migration note: This PR replaces #98862 with the same published head commit, moved from spinen/openclaw to jimmypuckett/openclaw so OpenClaw maintainers and repository automation can update the contribution without transferring authorship. Prior review, proof, and CI history remain available on #98862.

@openclaw-barnacle openclaw-barnacle Bot added channel: msteams Channel integration: msteams size: S labels Jul 11, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. labels Jul 11, 2026
@clawsweeper

clawsweeper Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 11, 2026, 6:53 PM ET / 22:53 UTC.

Summary
The PR migrates legacy Microsoft Teams conversation references under embedded canonical conversation IDs, verifies visibility before archival, retains the source after failed or incomplete migration, and adds production SQLite failure/retry coverage.

PR surface: Source +26, Tests +119. Total +145 across 3 files.

Reproducibility: yes. from source and supplied exact-head evidence. The production-store regression and upgrade transcript establish the failure, retained source, retry, canonical lookup, and archival sequence, although this read-only review did not independently execute the PR.

Review metrics: 1 noteworthy metric.

  • Persistent identity: 1 migration key rule changed. Legacy references now use the canonical conversation identity expected by proactive runtime lookup.

Stored data model
Persistent data-model change detected: migration/backfill/repair: extensions/msteams/doctor-contract-api.test.ts, migration/backfill/repair: extensions/msteams/doctor-contract-api.ts, serialized state: extensions/msteams/doctor-contract-api.test.ts, unknown-data-model-change: extensions/msteams/doctor-contract-api.test.ts, unknown-data-model-change: extensions/msteams/doctor-contract-api.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #94939
Summary: The linked issue is the canonical migration regression; this PR is its active candidate fix, while the earlier contribution was replaced without merge.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] The PR is mergeable but behind current main, so the final merged head should preserve newer main changes and repeat the exact-head compatibility and migration gates.

Maintainer options:

  1. Land with current-base gates (recommended)
    Merge after the final current-base head preserves the reviewed migration behavior and all required checks pass.

Next step before merge

  • No automated repair is needed; the remaining action is routine maintainer landing review on the final current-base head.

Security
Cleared: The patch introduces no dependency, workflow, permission, secret, package-resolution, artifact-download, or new code-execution surface.

Review details

Best possible solution:

Land the canonical-key, archive-after-visibility migration and its production-store regressions on the current base, without adding a steady-state legacy fallback.

Do we have a high-confidence way to reproduce the issue?

Yes from source and supplied exact-head evidence. The production-store regression and upgrade transcript establish the failure, retained source, retry, canonical lookup, and archival sequence, although this read-only review did not independently execute the PR.

Is this the best way to solve the issue?

Yes. The patch repairs the migration owner boundary and uses the same normalized identity as proactive runtime lookup, avoiding a new runtime fallback or parallel state source.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 088a9dc33557.

Label changes

Label justifications:

  • P1: The PR repairs a shipped upgrade regression that can stop proactive Microsoft Teams messages for existing users.
  • merge-risk: 🚨 compatibility: It changes persistent legacy-state identity and source archival behavior during an existing-user upgrade migration.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): Exact-head redacted production-SQLite output directly demonstrates failed migration retention, successful retry, canonical-key visibility, and subsequent archival.
  • proof: sufficient: Contributor real behavior proof is sufficient. Exact-head redacted production-SQLite output directly demonstrates failed migration retention, successful retry, canonical-key visibility, and subsequent archival.
Evidence reviewed

PR surface:

Source +26, Tests +119. Total +145 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 2 31 5 +26
Tests 1 119 0 +119
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 3 150 5 +145

What I checked:

  • Canonical migration identity: The patch prefers the normalized embedded reference.conversation.id over the legacy outer JSON key, matching the identity used for proactive lookup. (extensions/msteams/doctor-contract-api.ts:168, ed7b43389155)
  • Runtime lookup contract: The current conversation store normalizes the requested conversation ID and hashes it for plugin-state lookup, confirming the embedded canonical ID is the correct migration destination. (extensions/msteams/src/conversation-store-state.ts:117, 088a9dc33557)
  • Production-store regression: The added test triggers a real plugin-state row-limit failure, confirms that the legacy source remains, retries successfully, and verifies visibility only under the canonical embedded-ID key before archival. (extensions/msteams/doctor-contract-api.test.ts:140, ed7b43389155)
  • Current main remains affected: Current main still derives the destination key from the outer legacy JSON key and archives the source after the import loop without verifying canonical visibility. (extensions/msteams/doctor-contract-api.ts:303, 088a9dc33557)
  • Latest release remains affected: Release v2026.6.11 contains the legacy conversation migration but predates this branch's canonical embedded-ID and archive-verification repair. (extensions/msteams/doctor-contract-api.ts:308, e085fa1a3ffd)
  • Exact-head behavior proof: The contributor supplied redacted exact-head production-SQLite output showing failure retention, successful retry, canonical-key visibility, and archival, with private conversation and account identifiers removed. (ed7b43389155)

Likely related people:

  • steipete: Commit 1b1a41d introduced the current Microsoft Teams shared-state migration and runtime conversation-store implementation that this PR repairs. (role: introduced behavior; confidence: high; commits: 1b1a41d87fe2; files: extensions/msteams/doctor-contract-api.ts, extensions/msteams/src/conversation-store-state.ts, extensions/msteams/src/send-context.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (5 earlier review cycles)
  • reviewed 2026-07-11T20:43:42.589Z sha 7871cfe :: needs maintainer review before merge. :: none
  • reviewed 2026-07-11T21:14:40.546Z sha 9ab0693 :: needs real behavior proof before merge. :: [P1] Restore canonical references before claiming the send fix | [P2] Exercise the production doctor store in the regression
  • reviewed 2026-07-11T21:45:53.661Z sha 9ab0693 :: needs real behavior proof before merge. :: [P1] Exercise a reachable keyed-store failure | [P2] Use the production doctor store in the regression
  • reviewed 2026-07-11T22:07:05.208Z sha ed7b433 :: needs real behavior proof before merge. :: none
  • reviewed 2026-07-11T22:15:35.784Z sha ed7b433 :: needs real behavior proof before merge. :: none

@jimmypuckett
jimmypuckett force-pushed the spinen/msteams-conversation-migration-safety branch 2 times, most recently from 798f85e to 9ab0693 Compare July 11, 2026 21:04
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 11, 2026
@jimmypuckett

Copy link
Copy Markdown
Contributor Author

Exact-head proof update for 9ab0693285ef1bd97c8756444a7adb18f6d591df:

  • GitHub CI is fully green on this exact head: zero pending and zero failing checks. Relevant passing lanes include check-lint, check-additional-extension-bundled, check-additional-extension-channels, Real behavior proof, QA smoke profiles, Node compact shards, type checks, security checks, and boundary checks.
  • Local validation on the same head before push:
    • node scripts/run-vitest.mjs extensions/msteams/doctor-contract-api.test.ts: 7 tests passed
    • node scripts/run-vitest.mjs extensions/msteams: 75 files / 1099 tests passed
    • pnpm run lint:extensions:bundled: passed
    • pnpm run lint:extensions:channels: passed; no extension channel files found
    • pnpm lint --threads=8: passed
    • git diff --check: passed
  • Scope clarification for the proof gap: this PR should be reviewed as a migration-safety/preservation guard, not as a full canonical shared-state repair. It does not claim that the early-return path itself restores runtime-visible conversation references. The intended behavior is narrower: when a non-empty legacy msteams-conversations.json has retained rows attempted but the migration observes imported === 0 and present === 0, the doctor emits a warning and leaves the recoverable source in place instead of silently archiving it.
  • Compatibility/message-delivery risk mitigation: successful verified migrations keep the existing archive behavior; empty legacy sources keep the existing behavior; only the unverifiable non-empty case preserves recoverable legacy data so an operator or later corrected migration still has the conversation references needed for recovery.
  • Redacted runtime proof remains in the PR body: patched bundle installed, migration guard present in patched bundle and absent from baseline bundle, gateway/listener running, msteamsConversationReferences: 20, sent-message records increased 6 -> 7 after a real proactive Teams send, and the latest personal conversation timestamp updated. Conversation IDs, tenant/user IDs, message bodies, hostnames, and local usernames are redacted.

Given the exact-head green CI and the narrowed claim above, please re-review this as the narrower preservation/safety fix rather than a complete canonical proactive-send repair.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot removed the merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. label Jul 11, 2026
@jimmypuckett
jimmypuckett force-pushed the spinen/msteams-conversation-migration-safety branch from 9ab0693 to ed7b433 Compare July 11, 2026 22:00
@jimmypuckett

Copy link
Copy Markdown
Contributor Author

Updated at exact head ed7b433 with the real production-store regression ClawSweeper requested.

Evidence covered by the new regression in extensions/msteams/doctor-contract-api.test.ts:

  • Uses the production doctor migration context and SQLite-backed plugin keyed store, not a fake unsupported store state.
  • Creates a legacy msteams-conversations.json entry keyed by a legacy/noncanonical JSON key while the Bot Framework reference contains the canonical conversation.id used for proactive sends.
  • Pre-fills plugin state, sets the production store test row limit to 1, and verifies the first migration rejects with PLUGIN_STATE_LIMIT_EXCEEDED while leaving msteams-conversations.json in place and not creating .migrated.
  • Removes the row limit, resets the store, retries the same migration, and verifies no warnings, Migrated 1, source archive creation, canonical lookup by reference.conversation.id succeeds, and lookup by the legacy JSON key remains undefined.

Production change: conversation migration now resolves the stored Bot Framework reference conversation.id before falling back to the legacy JSON key, so archived migrations leave rows visible at the same canonical key the Teams proactive send path uses.

Validation on this head:

  • node scripts/run-vitest.mjs extensions/msteams/doctor-contract-api.test.ts
  • node scripts/run-vitest.mjs extensions/msteams
  • pnpm run lint:extensions:bundled
  • pnpm run lint:extensions:channels
  • git diff --check
  • pnpm lint --threads=8
  • pnpm plugin-sdk:check-exports
  • gh pr checks 104691 --repo openclaw/openclaw: zero failed, zero pending; Real behavior proof passed.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jul 11, 2026
@jimmypuckett

Copy link
Copy Markdown
Contributor Author

Exact-head redacted upgrade-path proof for ed7b43389155f9f6c0e3919b8079f532797502d9.

This proof uses a disposable local OpenClaw state home under /var/.../T, with no live gateway, no Dale state, and no real tenant/user/conversation IDs. To avoid stale ignored dist/extensions artifacts in the checkout, the runner imports the exact PR-head source contract at extensions/msteams/doctor-contract-api.ts and exercises it with the same production plugin-state SQLite keyed-store context used by doctor migrations (pluginId: msteams, namespace conversations).

The fixture is an authentic legacy msteams-conversations.json shape: one retained row whose outer JSON key is legacy-user-key, while reference.conversation.id is the canonical Teams conversation ID used for proactive lookup.

Redacted proof transcript:

{
  "head": "ed7b43389155f9f6c0e3919b8079f532797502d9",
  "proofMode": "exact PR-head source msteams doctor contract + production plugin-state SQLite context",
  "disposableStateDir": "<tmp-proof-root>/state-dir",
  "fixture": {
    "legacyJsonKey": "legacy-user-key",
    "embeddedConversationId": "19:[email protected]",
    "retainedConversationEntries": 1
  },
  "afterFailure": {
    "failureCode": "PLUGIN_STATE_LIMIT_EXCEEDED",
    "failureMessage": "Plugin state for msteams exceeds the 1 live row limit.",
    "legacyExists": true,
    "legacySizeBytes": 552,
    "archiveExists": false,
    "sqliteExists": true
  },
  "afterRetry": {
    "changes": [
      "Migrated 1 Microsoft Teams conversation entry -> plugin state",
      "Archived Microsoft Teams conversation legacy source -> <tmp-proof-root>/state-dir/msteams-conversations.json.migrated"
    ],
    "warningCount": 0,
    "legacyExists": false,
    "archiveExists": true,
    "archiveSizeBytes": 552,
    "sqliteExists": true
  },
  "canonicalVisibility": {
    "canonicalKeyPrefix": "36fb5f4161c2",
    "legacyKeyPrefix": "8987278c02e0",
    "pluginStateLookupByEmbeddedConversationIdKey": true,
    "pluginStateLookupByLegacyJsonKey": false,
    "storedConversationId": "19:[email protected]",
    "storedUserId": "29:redacted-user"
  },
  "sqlite": {
    "path": "<tmp-proof-root>/state-dir/state/openclaw.sqlite",
    "msteamsNamespaceCounts": [
      {
        "plugin_id": "msteams",
        "namespace": "conversations",
        "rows": 1
      },
      {
        "plugin_id": "msteams",
        "namespace": "proof-sibling",
        "rows": 1
      }
    ],
    "conversationKeyRows": [
      {
        "namespace": "conversations",
        "keyPrefix": "36fb5f4161c2",
        "matchesCanonicalKey": true,
        "matchesLegacyJsonKey": false
      }
    ]
  }
}

What this demonstrates:

  • Real production SQLite row-limit failure path: PLUGIN_STATE_LIMIT_EXCEEDED.
  • Failed migration leaves the populated legacy source in place and does not create .migrated.
  • Retry with a healthy store imports one Microsoft Teams conversation entry.
  • Retry archives the source only after successful import.
  • Shared plugin-state SQLite contains one msteams/conversations row.
  • The stored row matches the canonical embedded reference.conversation.id key, not the legacy outer JSON key.

Exact-head CI remains green: gh pr checks 104691 --repo openclaw/openclaw exits 0 with zero failed and zero pending checks.

@jimmypuckett

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 11, 2026
@openclaw-barnacle

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jul 26, 2026
@jimmypuckett

Copy link
Copy Markdown
Contributor Author

Active maintenance update: this Microsoft Teams conversation-migration PR is not abandoned. The published PR head still matches its fork branch, and I am tracking the current integration state. The exact PR ref currently has unrelated history from upstream/main in a local merge-tree check, so I am preserving the published branch rather than force-pushing or synthesizing history. I am continuing to investigate the focused recovery path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: msteams Channel integration: msteams merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P1 High-priority user-facing bug, regression, or broken workflow. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S stale Marked as stale due to inactivity status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

1 participant