Skip to content

RUM-10283: Shallow copy node wireframes before iterating in NodeFlattener#2736

Merged
ambushwork merged 1 commit into
developfrom
yl/sr-concurrent-modification-fix
Jun 25, 2025
Merged

RUM-10283: Shallow copy node wireframes before iterating in NodeFlattener#2736
ambushwork merged 1 commit into
developfrom
yl/sr-concurrent-modification-fix

Conversation

@ambushwork

@ambushwork ambushwork commented Jun 17, 2025

Copy link
Copy Markdown
Member

What does this PR do?

When iterating the wireframes with node.wireframes.map, it can throw ConcurrentModificationException because some wireframes are added as a mutable list, to prevent this, we need to make a shallow copy before iterating it.

Performance consideration: wireframes size usually represents the components count on the screen, so it will be not very large, the range is usually around 0~100, with this size of collection, the cost of .toList() negligible.

Motivation

RUM-10283

Additional Notes

Anything else we should know when reviewing?

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Make sure you discussed the feature or bugfix with the maintaining team in an Issue
  • Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

@ambushwork
ambushwork force-pushed the yl/sr-concurrent-modification-fix branch from ae5f772 to ed951ac Compare June 17, 2025 16:29
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.84%. Comparing base (5556aff) to head (ed951ac).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2736      +/-   ##
===========================================
+ Coverage    69.72%   69.84%   +0.12%     
===========================================
  Files          824      824              
  Lines        30790    30790              
  Branches      5177     5177              
===========================================
+ Hits         21467    21505      +38     
+ Misses        7861     7833      -28     
+ Partials      1462     1452      -10     
Files with missing lines Coverage Δ
.../sessionreplay/internal/processor/NodeFlattener.kt 100.00% <100.00%> (ø)

... and 37 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ambushwork
ambushwork marked this pull request as ready for review June 18, 2025 08:29
@ambushwork
ambushwork requested review from a team as code owners June 18, 2025 08:29
@ambushwork ambushwork changed the title RUM-10283: Make node wireframes immutable before flattening it RUM-10283: Make node wireframes copy before flattening it Jun 18, 2025
@ambushwork ambushwork changed the title RUM-10283: Make node wireframes copy before flattening it RUM-10283: Shallow copy node wireframes before iterating in NodeFlattener Jun 18, 2025
while (stack.isNotEmpty()) {
val node = stack.pop()
node.wireframes
node.wireframes.toList()

@aleksandr-gringauz aleksandr-gringauz Jun 18, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this actually fix the problem?

toList is still doing iteraction over a collection. If wireframes is a MutableList and during this iteration some other thread changes it, then ConcurrentModificationException can be thrown.

@ambushwork ambushwork Jun 18, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it does, toList creates a new list instance of original one, so when you modifying the original list, the new list instance will not be affected anymore during the iteration.

This can be proved by the unit test I added, when removing toList the test fails, when adding it the test passes

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

creates a new list instance of original one

it does it doing an iteraction over an original list. It all comes down to this (copying ArrayList using a constructor)

if during this copying another thread modifies the original list, the copyied list will be in invalid state or copying will crash

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most probably toList doesn't create iterator, but is using System.arrayCopy under the hood, which is native, so behavior may be different in case of concurrent modification.

Indeed, using the test provided outcome is different with and without toList call.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, ConcurrentModificationException is thrown when the code of the collection itself is able to detect that something is wrong (concurrent modification occured).

The fact that it isn't thrown doesn't mean that the code is thread-safe.

System.arrayCopy as it is native most likely can't throw this exception. But it is doing the copy of the array without any locking. So if array is concurrently modified, the result of the copy could be bad.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you create a MutableList, populate it with values and then never modify it again, then at some point pass it to another thread using executorService.execute, it should be fine.

So the only possibility as far as I understand is that the MutableList is modified after it is passed to another thread.

Does it work in the way that we create these wireframes on UI thread and then pass them to the worker thread and they are only read from there? If so, it's strange that we have such problem.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

theoretically session replay wireframes generation works like you described, it should never be modified afterwards. But like I said we currently not able to locate the place where it is modified, only thing we can do is to prevent exception from modification.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the problem lies here:

and is because you are using this callback on the UIThread. So in this moment you can have a WorkerThread iterating your wireframes from one side and this callback iterating the same collection on the UIThread. None of this logic is doing any alteration though on the collection so not 100% sure how the ConcurrentModificationException might be triggered but I am pretty sure is somewhere around this logic.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also @ambushwork even if you're doing a copy there you need to synchronise the code block where you are actually performing the copy at it might happen to have a concurrent modification in the middle of the copy operation.

@ambushwork ambushwork Jun 25, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I explained to Sasha, adding synchronise block can bring performance punishment, and since the crash rate of this ConcurrentModificationException is already very low before this fix, the chance of triggering ConcurrentModificationException when copying a small size of list can be negligible, I think it's a good trade-off

@aleksandr-gringauz

Copy link
Copy Markdown
Contributor

discussed it in person, lgtm

@ambushwork
ambushwork merged commit 41b9e16 into develop Jun 25, 2025
25 checks passed
@ambushwork
ambushwork deleted the yl/sr-concurrent-modification-fix branch June 25, 2025 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants