Bug Description

microsoft-teams-apps (and the underlying entity models in microsoft-teams-api) use a strict Pydantic discriminated union for activity entities[]. Any entity whose type discriminator is not in the SDK's known set causes pydantic.ValidationError, which the HTTP server returns as HTTP 500. The entire activity is rejected, not just the unknown entity — even when the rest of the payload is valid and the unknown entity is irrelevant to the handler.

Microsoft's Bot Framework service sends newer entity types (observed: ClientCapabilities) that the SDK doesn't recognize, so this breaks real-world bots. Impact is severe:

1. Repeated 5xx responses trip the Teams channel's exponential backoff and silently halt delivery to the bot. Recovery takes ~12–24h of consistent 2xx responses (in our case), or an Azure support ticket.
2. Web Chat is affected whenever Microsoft attaches an unknown entity to a message activity (not just conversationUpdate): the bot's @app.on_message handler never runs, so no reply is sent — even though Microsoft sees a 200 ack.
3. Symptoms look like an Azure configuration problem (tenant settings, manifest fields, AppId alignment, channel state), not an SDK bug, leading to long debugging sessions chasing the wrong layers.

Related closed issue: #237 (different specific field, same class of strict-validation rejection).

Steps to Reproduce

⚠️ Not reproducible via the DevTools simulator (microsoft-teams-devtools plugin's localhost:3979/devtools/ UI). DevTools sends synthetic activity payloads that don't include the problematic entity types. The bug only manifests against real Microsoft bot service traffic.

1. Create a minimal bot following examples/echo with microsoft-teams-apps==2.0.11.
2. Deploy to a publicly reachable HTTPS endpoint (Azure Container App, App Service, etc.) reachable by Microsoft's Bot Framework service.
3. Configure an Azure Bot resource pointing at the endpoint.
4. In the Azure portal, open the Azure Bot resource → Test in Web Chat → send any message.
5. Observe in container logs: pydantic.ValidationError and HTTP 500 on activities whose entities[] include ClientCapabilities.

Example activity payload Microsoft is sending:

{
  "type": "message",
  "entities": [
    {
      "type": "ClientCapabilities",
      "supportsListening": true,
      "supportsSplashScreen": true,
      "supportsTts": true
    }
  ]
}

Expected Behavior

The SDK should gracefully ignore unknown entity types instead of rejecting the whole activity. Microsoft's Bot Framework activity schema is documented as forward-compatible: new fields and entity types are added over time without breaking older consumers, and SDKs are expected to ignore what they don't recognize.

For comparison, the older botbuilder-python SDK parses unknown entities loosely and does not reject the activity.

With this behaviour, a message activity containing one unknown entity and several known ones would still be dispatched to @app.on_message with the known entities intact.

Actual Behavior

ActivityTypeAdapter.validate_python (in microsoft_teams/apps/app_process.py:178) fails the entire activity with pydantic.ValidationError. The HTTP server returns 500. The bot's handler never runs, no reply is sent.

Container log excerpt:

[ERROR] microsoft_teams.apps.http.http_server: N validation errors for tagged-union[...]
message.entities.0.ClientInfoEntity.type
  Input should be 'clientInfo' [type=literal_error, input_value='ClientCapabilities', input_type=str]
message.entities.0.MentionEntity.type
  Input should be 'mention' [type=literal_error, input_value='ClientCapabilities', input_type=str]
message.entities.0.MessageEntity.type
  Input should be 'https://schema.org/Message' [type=literal_error, input_value='ClientCapabilities', input_type=str]
... (repeats for AIMessageEntity, StreamInfoEntity, CitationEntity, SensitiveUsageEntity,
     ProductInfoEntity, QuotedReplyEntity, TargetedMessageInfoEntity)
INFO: POST /api/messages HTTP/1.1" 500 Internal Server Error

The SDK's known entity types (derived from microsoft_teams/api/models/entity/*.py):

clientInfo, mention, https://schema.org/Message, streaminfo,
ProductInfo, quotedReply, targetedMessageInfo

Microsoft sends an entity outside this set (ClientCapabilities) → Pydantic walks the discriminated union, finds no match, fails all branches, raises ValidationError → HTTP 500.

Additionally observed (different but related schema-strictness issue): conversationUpdate activities sometimes arrive without a channelData field, which the SDK marks as required, producing the same 500:

conversationUpdate.channelData
  Field required [type=missing, ...]

SDK Version

microsoft-teams-apps: 2.0.11 microsoft-teams-api: 2.0.11

Python Version

3.12

Additional Context

Proposed fix — lenient unknown handling

Change the entity field's type from a strict tagged union to a structure that accepts unknown types, e.g.:

entities: Optional[List[Union[KnownEntity, UnknownEntity]]] = None

class UnknownEntity(BaseModel):
    model_config = ConfigDict(extra="allow")
    type: str   # any string

Where KnownEntity is the existing tagged union of ClientInfoEntity | MentionEntity | …, and UnknownEntity is a fallback that preserves the raw payload without imposing a schema. The discriminator falls back to UnknownEntity when none of the known classes match.

This makes the SDK forward-compatible: future Microsoft additions don't break consumers; they just appear as UnknownEntity instances. Bot authors can opt in to handling them later. Ideally the same philosophy applies to channelData and other optional schema fields the SDK currently marks required.

Workaround for anyone hitting this

FastAPI middleware that strips unknown entities from the request body before they reach ActivityTypeAdapter. Two phases: Phase 1 strips entities so the SDK can parse what's left; Phase 2 coerces residual 500s → 200s so Teams' channel backoff isn't tripped while a fix is in flight.

_SDK_KNOWN_ENTITY_TYPES = {
    "clientInfo",
    "mention",
    "https://schema.org/Message",
    "streaminfo",
    "ProductInfo",
    "quotedReply",
    "targetedMessageInfo",
}

def _normalize_activity_payload(payload: dict) -> dict | None:
    modified = False
    entities = payload.get("entities")
    if isinstance(entities, list):
        kept = [e for e in entities if isinstance(e, dict) and e.get("type") in _SDK_KNOWN_ENTITY_TYPES]
        if len(kept) != len(entities):
            payload["entities"] = kept
            modified = True
    if payload.get("type") == "conversationUpdate" and "channelData" not in payload:
        payload["channelData"] = {}
        modified = True
    return payload if modified else None

@app.server.adapter.app.middleware("http")
async def normalize_and_safeguard_messages(request: Request, call_next):
    if request.url.path != "/api/messages":
        return await call_next(request)
    if request.method == "POST":
        try:
            body = await request.body()
            payload = json.loads(body)
            if isinstance(payload, dict):
                normalized = _normalize_activity_payload(payload)
                if normalized is not None:
                    new_body = json.dumps(normalized).encode()
                    async def receive():
                        return {"type": "http.request", "body": new_body, "more_body": False}
                    request._receive = receive
        except Exception:
            pass
    response = await call_next(request)
    if response.status_code == 500:
        return JSONResponse({"status": "acknowledged"}, status_code=200)
    return response

This is fragile — _SDK_KNOWN_ENTITY_TYPES has to stay in sync with the SDK's models on every upgrade. A proper SDK fix would let everyone delete it.

Observed timeline / pattern

• We've seen Microsoft send ClientCapabilities initially only on conversationUpdate activities, later expanded to message activities too. Suggests Microsoft is rolling out new entity types incrementally; the SDK is consistently behind.
• After the bot started returning 5xx on these activities, Microsoft's Teams channel routing applied exponential backoff. Recovery (resumption of delivery) took roughly 12–24 hours of consistent 2xx responses, with an Azure support case open in parallel (unclear whether support intervened or backoff lifted naturally).

Happy to contribute a PR if there's interest. The two natural shapes are (1) add the missing entity types to the existing tagged union (narrow fix) or (2) make the entity union lenient about unknown types (forward-compatible fix). Let me know which direction fits the SDK's roadmap.