Repository navigation
fix(android): carry the content description beside the label on the snapshot node - #2912
Conversation
…napshot node An Android node is labelled by its text and falls back to the content description only when it has none, so an accessibility label the app set beside visible text never reached the wire: a labelled text view was named by its content, a filled field by its value, an empty field by its hint, a filled password field by its dots. The parser already read `content-desc`; the hierarchy node now carries it as `contentDescription` whenever it is not already the label, through the raw snapshot node, the unchanged-snapshot comparison, and the selector digest. `label` is unchanged, so nothing that keys on it moves.
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Add regression coverage for changed/equal descriptions and selector-digest propagation.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 2
Open (2)
What changed in this PR
Adds Android contentDescription alongside visible labels and carries it through snapshots and selector digests without changing label behavior.
Changes:
- Preserves distinct Android content descriptions.
- Includes them in snapshot-change detection and selector digests.
- Adds parser coverage for description handling.
| File | Reviewed changes |
|---|---|
src/daemon/response-views.ts |
Includes content descriptions in selector digests. |
src/commands/capture/runtime/snapshot-unchanged.ts |
Compares content descriptions for snapshot freshness. |
src/__tests__/android-ui-hierarchy.test.ts |
Tests content-description parsing. |
packages/platform-android/src/ui-hierarchy.ts |
Derives distinct descriptions. |
packages/platform-android/src/ui-hierarchy-node.ts |
Extends the Android node type. |
packages/platform-android/src/ui-hierarchy-builder.ts |
Publishes descriptions in snapshots. |
packages/kernel/src/snapshot.ts |
Extends the snapshot node contract. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| subrole: true, | ||
| label: true, | ||
| value: true, | ||
| contentDescription: true, |
There was a problem hiding this comment.
Added in #2920 (this PR merged before the rows landed): contentDescription is in the "detects presentation changes" table, and a new case keeps a node whose description did not change as unchanged.
| 'subrole', | ||
| 'label', | ||
| 'value', | ||
| 'contentDescription', |
There was a problem hiding this comment.
Added in #2920 (this PR merged before the rows landed): MATCHED_NODE and COMPACT_NODE now carry contentDescription, so the find/get digest assertions cover its retention.
|
Reviewed at c6dbc8a. The fix looks correct and I found no blocking issues. Not blocking: the new unit test doesn't add a changed-description/equal-description case for I checked the diff, the PR-head files, and the new unit test, which does exercise the real |
thymikee
added
the
ready-for-human
thymikee
left a comment
thymikee
left a comment
There was a problem hiding this comment.
Reviewed at c6dbc8a. The source behavior looks sound: contentDescription belongs in the presentation-equivalence key because it is visible through snapshot JSON and get attrs even while text rendering is deferred, and the parser fallback preserves label without repeating a description that is already the label. I have no additional source-level findings beyond Copilot's test and fixture notes.
Two user-surface updates are missing:
- website/docs/docs/snapshots.md should list contentDescription beside the Android node keys and explain that it is Android-only and appears only when the description is distinct from label.
- CHANGELOG.md needs an Unreleased entry for the new user-visible Android snapshot/get attrs field.
Summary
An Android node is labelled by its text and falls back to the content description only when it has none (
ui-hierarchy.ts,label: text || desc), so an accessibility label set beside visible text never reached the wire: a labelled text view was named by its content, a filled field by its value, an empty field by its hint, a filled password field by its dots. TalkBack reads the content description in each case.The parser already read
content-desc; the hierarchy node now carries it ascontentDescriptionwhenever it is not already the label, through the raw snapshot node, the unchanged-snapshot comparison, and thefind/getdigest.labelis unchanged, so nothing that keys on it moves. 7 files, +37/-1.agent-device snapshot --json --platform android # {"type":"android.widget.EditText","label":"Ada Lovelace","contentDescription":"Name field","identifier":"name-input"}Validation
Tested at c6dbc8a; the test-only follow-up d5ddd39 passed the same gate.
pnpm check:quickandpnpm check:affected --run --base a363a0239: all runnable checks passed (format, lint, typecheck, layering, fallow, build, 852 test files).src/__tests__/android-ui-hierarchy.test.ts: 39 tests, one new for text beside a description, a description equal to the label (not repeated), and a hinted empty field.accessibilityLabelascontentDescription; the status-bar clock carries its spoken time beside its label. Text presentation unchanged.Motivation: e2e-inc/e2e#500.