Skip to content

[CIS-1854] [Internal] Move Model mappings to Background#2237

Merged
polqf merged 58 commits intodevelopfrom
feature/mappings-to-background-new-list
Sep 15, 2022
Merged

[CIS-1854] [Internal] Move Model mappings to Background#2237
polqf merged 58 commits intodevelopfrom
feature/mappings-to-background-new-list

Conversation

@polqf
Copy link
Contributor

@polqf polqf commented Aug 16, 2022

🔗 Issue Links

https://stream-io.atlassian.net/browse/CIS-1854
https://stream-io.atlassian.net/browse/CIS-2059

🎯 Goal

The goal of this PR is to investigate the viability of moving the mapping of DTOs to Model to a background thread, reducing the pressure we put in the view/main thread.

📝 Summary

This PR introduces a new ListDatabaseObserver, matching the same API for simplicity purposes. This new object, named BackgroundListDatabaseObserver does exactly the same as its predecessor, but in a background and asynchronous way, and only notifies about results when those are already mapped.

🛠 Implementation

As of now:

  • The observer listens to a particular fetch request and processes the results of any update that might be coming from the Database
  • It converts the latest snapshot of DTOs into Models in a background thread, thus not leading to pressure on the main thread.
  • It calls its delegate (in this case a block) whenever all the objects have been parsed
  • Because all the mapped objects are assigned to a property on the same (internal) thread, that should result in no data races
  • CoreDataLazy is effectively disabled under the runtime flag _isBackgroundMappingEnabled , unless forceLazy is passed

All this implementation is guarded now behind _isBackgroundMappingEnabled. When the flag is false, the same behaviour as before will be present.

When using DemoApp-StreamDevelopers, it uses Background Mapping by default

Pitfalls:

Because now background mapping tries to map every single updated dto as new updates come in, we end up resolving the whole tree for each object, which results in way too many fetches. This has no direct impact on the main thread per se, but the overhead we add to Core Data can be significant in some cases.

Also, in some cases, there are cyclic relationships, leading to Stack Overflow. This is now prevented from passing forceLazy: true to some models. This preserves the laziness of those properties. This is a temporary solution to those problems, but it is not meant to be the final one

☑️ Contributor Checklist

  • I have signed the Stream CLA (required)
  • Changelog is updated with client-facing changes
  • New code is covered by unit tests
  • This change follows zero ⚠️ policy (required)
  • Comparison screenshots added for visual changes
  • Affected documentation updated (docusaurus, tutorial, CMS)

🎁 Meme

Since the this was actually not causing the messages to jump
Also adds a function to make the removePublicDeclarations.sh more readable
@polqf polqf changed the title [POC] Move Model mappings to Background [CIS-1854] [Internal] Move Model mappings to Background Aug 18, 2022
@polqf polqf marked this pull request as ready for review August 18, 2022 15:17
@polqf polqf requested a review from a team as a code owner August 18, 2022 15:17
Base automatically changed from fix/CIS-2085-Message-list-with-difference-kit to develop August 24, 2022 11:16
@emerge-tools
Copy link

emerge-tools bot commented Aug 24, 2022

📏 Size Analysis

Total install size 9.7 MB | This change: ⬆️ +32.6 kB (+0.336%)

Image of diff

🗂 See size breakdown
Item Install size
➕ StreamChat.framework/StreamChat.BackgroundListDatabaseObserver.listenForRemoveAllDataNotifications ⬆️ 5.1 kB
➕ StreamChat.framework/StreamChat.BackgroundListDatabaseObserver.mapItems(completion) ⬆️ 4.5 kB
➕ StreamChat.framework/StreamChat.BackgroundListDatabaseObserver.init(context,fetchRequest,itemCreator,fetchedResultsControllerType) ⬆️ 2.3 kB
StreamChat.framework/StreamChat.LazyCachedMapCollection.init(source,map) ⬆️ 2.3 kB
StreamChat.framework/StreamChat.CoreDataLazy.projectedValue.setter ⬆️ 2.0 kB
➕ StreamChat.framework/StreamChat.BackgroundListDatabaseObserver.startObserving ⬆️ 1.8 kB
➕ StreamChat.framework/StreamChat.ListDatabaseObserverWrapper.items ⬆️ 1.3 kB
➕ ChatSample/ChatSample.DemoAppConfiguration.showPerformanceTracker ⬆️ 1.0 kB
StreamChat.framework/Code Signature 832 B
➕ StreamChat.framework/StreamChat.BackgroundListDatabaseObserver.processItems 612 B
➕ StreamChat.framework/StreamChat.BackgroundListDatabaseObserver.Swift Metadata 594 B
➕ StreamChat.framework/StreamChat.BackgroundListDatabaseObserver.notifyDidChange(changes) 556 B
➕ StreamChat.framework/StreamChat.ListDatabaseObserverWrapper.startObserving 532 B
➕ StreamChat.framework/StreamChat.BackgroundListDatabaseObserver.notifyWillChange 516 B

🔎 See the full size analysis (741f7c0) merging into develop (f8eb1ca)

Provide a baseSha and pullRequestNumber with your upload to ensure diffs are correct. Read more in the docs

Copy link
Contributor

@martinmitrevski martinmitrevski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code-wise looks really good. I did some testing and there are solid performance improvements in the message list in UIKit. In SwiftUI, there's still a noticeable glitch when loading new messages, but we can iterate on this in the future.
To merge this, we would need some tests, and let's see how we communicate this change. Will there be doc update, where customers can opt in?

@polqf
Copy link
Contributor Author

polqf commented Sep 1, 2022

Will there be doc update, where customers can opt in?

Not as of now, the idea is to use it internally, make sure things get to a better stage, and then allow customers to use it.
At some point down the line, when things are stable enough, enable it for everyone

Copy link
Member

@nuno-vieira nuno-vieira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, the only thing I think we could improve is the Wrapper, and use a protocol instead. Or still, use the Wrapper, but inside, we could just move the ifs statements, to only one if statement, when creating the observer.

Copy link
Member

@nuno-vieira nuno-vieira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! ✅

@polqf polqf merged commit 88f325d into develop Sep 15, 2022
@polqf polqf deleted the feature/mappings-to-background-new-list branch September 15, 2022 08:14
@sonarqubecloud
Copy link

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 5 Code Smells

92.9% 92.9% Coverage
6.4% 6.4% Duplication

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants