Skip to content

feat(android): add inline Adaptive Card rendering in chat#42304

Open
VikrantSingh01 wants to merge 3 commits intoopenclaw:mainfrom
VikrantSingh01:feat/adaptive-cards-android-rendering
Open

feat(android): add inline Adaptive Card rendering in chat#42304
VikrantSingh01 wants to merge 3 commits intoopenclaw:mainfrom
VikrantSingh01:feat/adaptive-cards-android-rendering

Conversation

@VikrantSingh01
Copy link
Copy Markdown

@VikrantSingh01 VikrantSingh01 commented Mar 10, 2026

Summary

Adds native Adaptive Card rendering to the Android app. When a chat message contains <!--adaptive-card--> markers, the card is parsed and rendered inline in the chat bubble using Jetpack Compose.

New files

  • AdaptiveCardParser.kt — marker extraction + JSON parsing
  • AdaptiveCardComposable.kt — Compose renderer for AC v1.6 elements

Modified files

  • ChatMessageViews.kt — card detection in ChatMessageBody before markdown render

Element support

TextBlock, FactSet, ColumnSet, Container, Image, Action.Submit, Action.OpenUrl

Performance target

<50ms first render, <2MB memory per card.

Ecosystem Context

This PR is part of the Adaptive Cards feature set powered by:

Package Version Role
adaptive-cards-mcp v2.3.0 Shared core — v1.6 JSON Schema validation, 7 host profiles, WCAG a11y scoring (0-100), 21 layout patterns, 924 tests
openclaw-adaptive-cards v4.0.0 OpenClaw plugin — adaptive_card tool, MCP bridge, channel-aware prompt guidance, fallback text generation, action routing

The plugin emits <!--adaptive-card--> markers in tool result text. This PR adds the Android client-side parser and Jetpack Compose renderer that consumes those markers.

Related PRs

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 10, 2026

Greptile Summary

This PR introduces native Adaptive Card rendering in the Android chat UI by adding AdaptiveCardParser.kt for marker-based JSON extraction and AdaptiveCardComposable.kt for the Compose renderer, with ChatMessageViews.kt wired to detect and display cards inline. The approach is well-structured and the remember-based memoization in the message body is a good practice.

However, there are a few issues that should be addressed before merging:

  • Content loss: Text appearing after the <!--/adaptive-card--> closing marker is silently discarded. Only the prefix is captured as fallbackText, so any trailing message text from the AI is never rendered.
  • Non-functional Action.Submit: A tappable button is rendered for Action.Submit actions, but its onClick only calls Log.d(...). No callback is wired through to handle the submission. Users will click it and see no response.
  • Unbounded recursion: RenderContainerRenderElementRenderContainer has no depth limit. A deeply nested or maliciously crafted card can trigger a StackOverflowError and crash the app.
  • Unused import: stripCardMarkers is imported in ChatMessageViews.kt but never used.
  • Dead code in column width logic: Both branches of the else-fallback ternary in RenderColumnSet return 1f, making the condition meaningless.

Confidence Score: 2/5

  • Not safe to merge — the Action.Submit button is silently non-functional and post-card text content is dropped, both of which affect end-user experience directly.
  • Two logic-level bugs cause visible user-facing regressions: content after the closing marker is lost, and Action.Submit renders a dead button. Additionally, the unbounded composable recursion is a latent crash risk against untrusted card payloads. These should be resolved before shipping.
  • AdaptiveCardComposable.kt needs attention for the non-functional submit handler and the recursion depth guard. AdaptiveCardParser.kt needs the post-marker text preservation fix.

Last reviewed commit: 0a0c9af

fun parseAdaptiveCardMarkers(text: String): ParsedAdaptiveCard? {
val match = cardMarkerRegex.find(text) ?: return null
val jsonStr = match.groupValues[1].trim()
val fallback = text.substring(0, match.range.first).trim()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Text after card marker is silently dropped

fallbackText is set to only the substring before the opening marker. Any text appearing after <!--/adaptive-card--> is never captured and is silently discarded.

For example, if the AI message is:

Here is your summary:
<!--adaptive-card-->...<!--/adaptive-card-->
Please review the above card carefully.

The trailing sentence "Please review the above card carefully." would never be rendered.

Consider capturing the suffix text as well:

val prefix = text.substring(0, match.range.first).trim()
val suffix = text.substring(match.range.last + 1).trim()
val fallback = listOf(prefix, suffix).filter { it.isNotBlank() }.joinToString("\n\n")
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/android/app/src/main/java/ai/openclaw/app/chat/AdaptiveCardParser.kt
Line: 22

Comment:
**Text after card marker is silently dropped**

`fallbackText` is set to only the substring *before* the opening marker. Any text appearing after `<!--/adaptive-card-->` is never captured and is silently discarded.

For example, if the AI message is:
```
Here is your summary:
<!--adaptive-card-->...<!--/adaptive-card-->
Please review the above card carefully.
```
The trailing sentence "Please review the above card carefully." would never be rendered.

Consider capturing the suffix text as well:

```kotlin
val prefix = text.substring(0, match.range.first).trim()
val suffix = text.substring(match.range.last + 1).trim()
val fallback = listOf(prefix, suffix).filter { it.isNotBlank() }.joinToString("\n\n")
```

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9fb29e3. Parser now captures both prefix and suffix text around markers, stripping any <!--adaptive-card-data--> blocks from the suffix.

Comment on lines +173 to +179
private fun RenderContainer(element: Map<String, Any>) {
val items = element.typedList("items")
Column(verticalArrangement = Arrangement.spacedBy(4.dp)) {
for (item in items) {
RenderElement(item)
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unbounded recursion can cause StackOverflowError

RenderContainer calls RenderElement for each item, RenderElement can call RenderContainer (or RenderColumnSetRenderElement → …) again, with no depth limit. A deeply nested card — or a maliciously crafted one — can exhaust the call stack and crash the app.

Consider tracking a depth counter and bailing out once it exceeds a reasonable bound (e.g. 10):

@Composable
private fun RenderElement(element: Map<String, Any>, depth: Int = 0) {
    if (depth > 10) {
        Log.w(TAG, "Max nesting depth exceeded, skipping element")
        return
    }
    // pass depth + 1 to RenderContainer / RenderColumnSet
    ...
}
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/android/app/src/main/java/ai/openclaw/app/ui/chat/AdaptiveCardComposable.kt
Line: 173-179

Comment:
**Unbounded recursion can cause `StackOverflowError`**

`RenderContainer` calls `RenderElement` for each item, `RenderElement` can call `RenderContainer` (or `RenderColumnSet``RenderElement` → …) again, with no depth limit. A deeply nested card — or a maliciously crafted one — can exhaust the call stack and crash the app.

Consider tracking a depth counter and bailing out once it exceeds a reasonable bound (e.g. 10):

```kotlin
@Composable
private fun RenderElement(element: Map<String, Any>, depth: Int = 0) {
    if (depth > 10) {
        Log.w(TAG, "Max nesting depth exceeded, skipping element")
        return
    }
    // pass depth + 1 to RenderContainer / RenderColumnSet
    ...
}
```

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9fb29e3. Added depth: Int = 0 parameter to RenderElement. Returns early when depth > 10. All recursive call sites pass depth + 1.

import ai.openclaw.app.chat.ChatMessageContent
import ai.openclaw.app.chat.ChatPendingToolCall
import ai.openclaw.app.chat.parseAdaptiveCardMarkers
import ai.openclaw.app.chat.stripCardMarkers
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused import

stripCardMarkers is imported but never called anywhere in this file. It can be safely removed.

Suggested change
import ai.openclaw.app.chat.stripCardMarkers
import ai.openclaw.app.chat.parseAdaptiveCardMarkers
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/android/app/src/main/java/ai/openclaw/app/ui/chat/ChatMessageViews.kt
Line: 32

Comment:
**Unused import**

`stripCardMarkers` is imported but never called anywhere in this file. It can be safely removed.

```suggestion
import ai.openclaw.app.chat.parseAdaptiveCardMarkers
```

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9fb29e3. Removed unused stripCardMarkers import.

Comment on lines +157 to +158
val weight = widthStr?.removeSuffix("px")?.toFloatOrNull()
?: if (widthStr == "stretch" || widthStr == "auto" || widthStr == null) 1f else 1f
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dead else branch — both arms of the ternary return 1f

The fallback expression always evaluates to 1f regardless of which branch is taken, making the if/else pointless:

?: if (widthStr == "stretch" || widthStr == "auto" || widthStr == null) 1f else 1f
//                                                                             ^^^^ same value

This also means that an Adaptive Card column using an integer weight string (e.g., "width": "2") would never apply a proportional weight — toFloatOrNull() on "2" returns 2f correctly, but any string that fails to parse (e.g. "half") silently falls back to 1f without distinction. Simplify to:

Suggested change
val weight = widthStr?.removeSuffix("px")?.toFloatOrNull()
?: if (widthStr == "stretch" || widthStr == "auto" || widthStr == null) 1f else 1f
val weight = widthStr?.removeSuffix("px")?.toFloatOrNull() ?: 1f
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/android/app/src/main/java/ai/openclaw/app/ui/chat/AdaptiveCardComposable.kt
Line: 157-158

Comment:
**Dead `else` branch — both arms of the ternary return `1f`**

The fallback expression always evaluates to `1f` regardless of which branch is taken, making the `if`/`else` pointless:

```kotlin
?: if (widthStr == "stretch" || widthStr == "auto" || widthStr == null) 1f else 1f
//                                                                             ^^^^ same value
```

This also means that an Adaptive Card column using an integer weight string (e.g., `"width": "2"`) would never apply a proportional weight — `toFloatOrNull()` on `"2"` returns `2f` correctly, but any string that fails to parse (e.g. `"half"`) silently falls back to `1f` without distinction. Simplify to:

```suggestion
      val weight = widthStr?.removeSuffix("px")?.toFloatOrNull() ?: 1f
```

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9fb29e3. Simplified to val weight = widthStr?.removeSuffix("px")?.toFloatOrNull() ?: 1f.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0a0c9af68f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

fun parseAdaptiveCardMarkers(text: String): ParsedAdaptiveCard? {
val match = cardMarkerRegex.find(text) ?: return null
val jsonStr = match.groupValues[1].trim()
val fallback = text.substring(0, match.range.first).trim()
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve text after adaptive-card marker blocks

The parser only stores text.substring(0, match.range.first) as fallback text, so any user-visible text that appears after <!--/adaptive-card--> is dropped once a card is detected. In ChatMessageBody, this fallback is the only non-card text rendered for parsed messages, which means messages containing intro + card + trailing explanation (or multiple card blocks) lose content in the UI.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9fb29e3 — same fix as the Greptile comment above (captures both prefix and suffix text).

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9fb29e38b0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Parse <!--adaptive-card--> markers in message text and render the card
JSON natively using Jetpack Compose. Supports TextBlock, FactSet,
ColumnSet, Container, Image (placeholder), Action.Submit, and
Action.OpenUrl elements. No new Gradle dependencies required.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 33b9164f3c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +157 to +161
val weight = widthStr?.removeSuffix("px")?.toFloatOrNull()
?: if (widthStr == "stretch" || widthStr == "auto" || widthStr == null) 1f else 1f

Column(
modifier = Modifier.weight(weight),
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Validate parsed column widths before applying weight

RenderColumnSet converts column["width"] to a float and feeds it directly into Modifier.weight(weight) without checking that it is finite and greater than zero. If a card contains width values like "0", "-1", or "NaN" (easy to hit with malformed or model-generated payloads), Compose throws when applying the modifier and the message render path crashes instead of failing gracefully.

Useful? React with 👍 / 👎.

Comment on lines +12 to +13
private val cardMarkerRegex =
Regex("<!--adaptive-card-->(.*?)<!--/adaptive-card-->", RegexOption.DOT_MATCHES_ALL)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Parse template-data markers when extracting adaptive cards

The parser only matches <!--adaptive-card-->...<!--/adaptive-card--> and reads templateData only from inside that JSON object, so it ignores the separate <!--adaptive-card-data-->...<!--/adaptive-card-data--> payload format. When messages carry template data in that marker, Android loses the binding context and card expressions render with unresolved placeholders or missing dynamic values.

Useful? React with 👍 / 👎.

Add Table, RichTextBlock, CodeBlock, ActionSet, ImageSet, Rating,
ProgressBar renderers. Fix template data extraction from markers.
Fix fallback text to strip all markers. Add Action.Execute support.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 84419a6817

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

else -> 5
}

val fullStars = value.toInt().coerceIn(0, max)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Guard negative rating max before coerceIn

RenderRating derives max directly from card data and then calls value.toInt().coerceIn(0, max). If a malformed or model-generated card sends a negative max (for example -1), coerceIn throws IllegalArgumentException, which crashes chat rendering for that message instead of degrading gracefully. Clamp max to a non-negative minimum (or skip rendering) before this call.

Useful? React with 👍 / 👎.

Comment on lines +177 to +178
val widthStr = column["width"] as? String
val weight = widthStr?.removeSuffix("px")?.toFloatOrNull()
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle numeric column widths in ColumnSet

RenderColumnSet only reads column["width"] as a String, but valid card payloads often provide weighted widths as JSON numbers (for example "width": 2), which org.json parses as Number. In that case this code falls back to 1f, so intended layouts like 2:1 are rendered as equal-width columns. Parse both numeric and string width values before defaulting.

Useful? React with 👍 / 👎.

* Returns null when no card markers are present.
*/
fun parseAdaptiveCardMarkers(text: String): ParsedAdaptiveCard? {
val match = cardMarkerRegex.find(text) ?: return null
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid dropping additional adaptive-card blocks

parseAdaptiveCardMarkers uses find to capture only the first card block, but fallback text is produced by removing all card markers from the message. When a response contains multiple <!--adaptive-card-->...<!--/adaptive-card--> sections, only the first card is renderable and later cards are silently lost. This should either parse/render all matches or preserve unmatched card blocks in fallback text.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 51f5f23c6d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

if (parsed.fallbackText.isNotBlank()) {
ChatMarkdown(text = parsed.fallbackText, textColor = textColor)
}
AdaptiveCardView(card = parsed.card)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply parsed template data before rendering cards

The parser now extracts templateData from <!--adaptive-card-data-->...<!--/adaptive-card-data-->, but ChatMessageBody drops it and renders only parsed.card. As a result, cards that use templated fields (for example ${userName} in TextBlock.text) render unresolved placeholders on Android even when valid template data is present in the message payload.

Useful? React with 👍 / 👎.

BradGroux

This comment was marked as duplicate.

Copy link
Copy Markdown
Contributor

@BradGroux BradGroux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Android Adaptive Card Renderer (Jetpack Compose)

666 lines adding an Android renderer with AdaptiveCardParser.kt (marker parsing + org.json Map conversion) and AdaptiveCardComposable.kt (Jetpack Compose renderer for AC v1.6 elements). Integration into ChatMessageViews.kt renders fallback markdown alongside the card UI. The Compose implementation is solid and covers a wide range of element types.

Blocker

1. JSONObject.NULL is coerced to empty string, silently breaking null semantics

private fun unwrapJsonValue(value: Any): Any {
  return when (value) {
    ...
    JSONObject.NULL -> ""
    else -> value
  }
}

This converts all JSON null values into empty strings "". Downstream rendering logic that checks for null/absent values (optional fields, conditional visibility, boolean checks) will behave incorrectly because "" is truthy in most comparisons. For example, an optional subtitle field that should hide a TextBlock when null will instead render an empty text block. The fix is to map JSONObject.NULL to Kotlin null and handle nullability properly in the renderer.

Suggestions

2. templateData is parsed but never consumed

data class ParsedAdaptiveCard(
  ...,
  val templateData: Map<String, Any>? = null
)
// templateData is extracted from markers and stored, but no renderer path reads it

The parser extracts template data from <!--adaptive-card-data--> markers and stores it in the parsed result, but the renderer never applies template binding. Either implement the binding step (replacing ${expression} placeholders using the data) or remove the field and extraction to avoid dead code confusion.

3. Action.OpenUrl opens URIs without scheme validation

uriHandler.openUri(it)

There is no allowlist on URL schemes before calling openUri. A card with javascript:, file:, content:, or intent: scheme URLs could trigger unintended behavior on Android. Gate this behind a scheme check (allow https and http only, or at minimum block known dangerous schemes).

4. Action row layout does not handle overflow

Row(...) {
  for (action in actions) {
    RenderAction(action, modifier = Modifier.weight(1f))
  }
}

With many actions, each button gets an increasingly small weight(1f) slice. On narrow devices, buttons will clip or become unreadable. Consider wrapping with FlowRow (which appears to be imported but unused) or capping visible actions with an overflow menu.

Nits

  • There is an unused FlowRow import that may have been intended for the action overflow case above.
  • The renderImage composable uses painterResource(R.drawable.ic_placeholder) as a placeholder, but actual image loading (Coil/Glide) is not wired up. This is likely a known TODO but worth noting.

Summary

Solid Compose implementation with good element coverage. The NULL coercion blocker will cause subtle rendering bugs with any card that uses optional fields (which is most real-world cards). Fix that, validate URL schemes on OpenUrl, and clean up the dead templateData path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants