Skip to content

perf(ios): bound the measurement cache and coalesce pending renders - #1138

Merged
hryhoriiK97 merged 2 commits into
software-mansion:mainfrom
pingdotgg:perf/ios-streaming-memory-bounds
Nov 7, 2026
Merged

perf(ios): bound the measurement cache and coalesce pending renders#1138
hryhoriiK97 merged 2 commits into
software-mansion:mainfrom
pingdotgg:perf/ios-streaming-memory-bounds

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

What/Why?

On iOS, memory grows with every revision while a long document streams in:

  • ENRMAsyncRenderCoordinator dispatches one closure per scheduleRender call. Each closure captures its revision's markdown and style config, and the serial queue holds all of them until it works through the backlog. Every revision is parsed and rendered, even though only the newest result is applied.
  • MeasurementCache keeps up to 512 entries, but each key holds the whole markdown string twice (once in the LRU list, once in the map). Streaming measures a new key for every revision, so a long reply can fill the cache with hundreds of progressively longer copies of itself: tens of MB of key strings.

We ran into this in T3 Code, whose mobile app renders long agent replies as they stream.

This PR:

  • ENRMAsyncRenderCoordinator keeps one render in flight and one pending. A call made before the pending render starts replaces it, and the superseded blocks, with the documents they capture, are released straight away. A burst of updates queues a single worker: the render already in flight finishes, then only the newest revision renders. invalidate also drops the pending render, so a recycled view no longer parses its old document. The apply side is unchanged: a result applies only if no newer call came in.
  • MeasurementCache also caps the bytes its keys hold at 8 MiB, counting both copies of each key's strings, and evicts least recently used entries until a new key fits. A key larger than the whole budget is not cached and evicts nothing. The 512-entry limit stays.

Testing

  • Two new XCTest suites in __tests__/ios:
    • MeasurementCacheTests.mm: going over the byte budget evicts the least recently used document; a key over the budget is not stored and does not evict anything.
    • ENRMAsyncRenderCoordinatorTests.mm: a burst renders only the newest revision after the in-flight one; a superseded pending render is released before it would run; invalidate drops the pending render.
  • All five new tests fail on main and pass with this change.
  • yarn turbo run test:ios --filter=react-native-enriched-markdown-example on an iPhone 16 Pro simulator (iOS 18.6): all 54 tests pass (53 library tests, including the 5 new ones, plus the harness placeholder). The two new suites also passed 100 iterations in a row (-test-iterations 100 -run-tests-until-failure).

PR Checklist

  • Code compiles and runs on iOS
  • Code compiles and runs on Android (iOS-only change)
  • Updated documentation/README if applicable
  • Ran example app to verify changes
  • E2E tests are passing
  • Required regression tests have been added

Model: Claude Opus 5.5. Harness: Claude Code in T3 Code.

@juliusmarminge @claude
perf(ios): bound the measurement cache and coalesce pending renders
baa382f 
While a long document streams in, every revision queued its own render
closure, and each one was parsed and rendered even though only the newest
result applied. The measurement cache was capped at 512 entries, but each
key holds the whole markdown, so streaming filled it with hundreds of
progressively longer copies of the document.

ENRMAsyncRenderCoordinator now keeps one render in flight and one
replaceable pending render, and invalidate drops the pending one.
MeasurementCache also caps the bytes its keys hold at 8 MiB and does not
store a key larger than that.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@juliusmarminge juliusmarminge mentioned this pull request Nov 6, 2026
Merged
@hryhoriiK97 @claude
perf(ios): store each measurement cache key once
d7cea78 
The LRU list and the map each held a copy of every key, so long
markdown was stored twice. The map now refers to the key its list node
owns, and the byte budget counts one copy.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@hryhoriiK97

hryhoriiK97 commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator

Thanks for this. I verified it on a real device and pushed one small follow-up commit to the branch.

What I changed

perf(ios): store each measurement cache key once (d7cea78): the LRU list and the map each held a full copy of every key, so a long markdown string was stored twice. The map now refers to the key its list node owns, and the byte budget counts one copy. Behavior is otherwise unchanged; the same 8 MiB budget now holds twice as many entries, with one string copy per insert instead of two. I updated MeasurementCacheTests for single-copy accounting.

What I verified

All runs: iPhone 15 Pro (iOS 26.6.1), Release builds, the PR base 0b59df11 as the control, streaming an 80 KB document into a github-flavor EnrichedMarkdownText at 60 chars per 16 ms. Memory is the app's physical footprint from Instruments' Activity Monitor.

Scenario Base This PR
streamingAnimation stream App killed: memory climbs about 200 MB/s to 3.36 GB, 33 s in Completes in 24.6 s, about 200 MB while streaming
Plain markdown updates, peak 520 MB 462 MB
Plain stream, memory after unmount about 145 MB 108 MB
CPU during the plain stream about 1.9 cores about 1.9 cores
  • The coalescing coordinator fixes a real crash in streamingAnimation mode. With plain markdown updates, the per-update measurement on the layout thread throttles the update rate, so no render backlog builds up there.
  • The cache bound releases about 37 MB after a plain stream.
  • The build with my commit behaves the same on the device: no crash, about 200 MB while streaming.
  • iOS unit tests: 54 of 54 pass. The five new tests passed 250 of 250 over 50 iterations. All of them are worth keeping: they're deterministic and cheap, and CI picks them up automatically.

Not in this PR (follow-ups on our side)

  • Android has the same pattern: one task per update on a single-thread executor, with the stale check only at apply time. It needs its own fix.

  • Streaming overall: while measuring this, we found the bigger costs are elsewhere:

    • A long view keeps a drawn bitmap of its entire height in memory, roughly 7.5 MB per screen of content. The 80 KB document holds about 790 MB while mounted, even without streaming, because the text view rasterizes its whole height.
    • Each streaming update re-parses, re-renders and redraws the whole document.

    We're going to improve streaming as a separate track: drawing limited to the viewport, incremental apply of the streamed tail, and incremental measurement.

@hryhoriiK97
hryhoriiK97 merged commit f103cbc into software-mansion:main Nov 7, 2026
20 checks passed
juliusmarminge added a commit to pingdotgg/t3code that referenced this pull request Nov 8, 2026
@juliusmarminge @ahalekelly
@Dananz @dansanduleac @Yash-Singh1 @claude
chore(mobile): update Enriched Markdown to t3.18 on upstream 1d00a92a
0c52f67 
Upstream merged the iOS streaming memory bounds
(software-mansion/enriched-markdown#1138), so the fork drops its copy.
Upstream also renders a newly mounted message in its first frame on iOS
(software-mansion/enriched-markdown#1139). The fork adds GitHub's
strikethrough rules for ~, links with hash-route fragments, \( \) and \[ \]
math delimiters, and iOS Copy as Markdown that keeps bold spans whole, each
with an upstream PR.

Co-authored-by: ahalekelly <[email protected]>
Co-authored-by: Tomer Danan <[email protected]>
Co-authored-by: Dan Sanduleac <[email protected]>
Co-authored-by: Yash Singh <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
juliusmarminge added a commit to pingdotgg/t3code that referenced this pull request Nov 9, 2026
@juliusmarminge @ahalekelly
@Dananz @dansanduleac @Yash-Singh1 @claude
chore(mobile): update Enriched Markdown to t3.18 on upstream 1d00a92a
746abf0 
Upstream merged the iOS streaming memory bounds
(software-mansion/enriched-markdown#1138), so the fork drops its copy.
Upstream also renders a newly mounted message in its first frame on iOS
(software-mansion/enriched-markdown#1139). The fork adds GitHub's
strikethrough rules for ~, links with hash-route fragments, \( \) and \[ \]
math delimiters, and iOS Copy as Markdown that keeps bold spans whole, each
with an upstream PR.

Co-authored-by: ahalekelly <[email protected]>
Co-authored-by: Tomer Danan <[email protected]>
Co-authored-by: Dan Sanduleac <[email protected]>
Co-authored-by: Yash Singh <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
shivamhwp pushed a commit to shivamhwp/enriched-markdown that referenced this pull request Nov 9, 2026
@juliusmarminge @claude
Revert "perf(ios): bound markdown cache and pending renders"
34a9f18 
Upstream merged this as software-mansion#1138 (f103cbc)
with a maintainer follow-up, so take its version from the upstream merge.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hryhoriiK97 hryhoriiK97 hryhoriiK97 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@juliusmarminge @hryhoriiK97