Mas i1820 offlinedeserialisation cbo#403
Merged
Merged
Conversation
If Block Index Cache is not full, and we're not yielding
Hypothesis is that the growth in the heap necessary due to continual term_to_binary calls to deserialise blocks is wasting memory - so do this memory-intensive task in a short-lived process.
Testing indicates that the BIC itself is not a primary memory issue - the primary issue is due to a lack of garbage collection and a growing heap. This change enhances the patch to offline serialisation so that: - get_sqn & get_kv are standardised to build the BIC, and hibernate when it is built. - the offline PId is linked to crash this process on failure (as would happen now).
Now deserialise in both parts of the code.
Owner
Author
|
Comparison of memory used in Riak test designed to drive very high key counts relatively quickly. The Control is Riak 3.0.13 (with default beam memory settings e.g. eheap memory sbct of 512KB). |
Owner
Author
|
Throughput comparison on 8-node im4gn.xlarge 24 hour test. Throughput difference was a 2% drop with this patch. However, there was a 10% memory drop with both running eheap memory sbct of 128KB. |
Owner
Author
|
Important that there is a positive change with and without default SBCT setting. Changing the default SBCT setting improves memory usage, but in some cases this improvement can be hard to detect because of the use of MADVISE_FREE not MADVISE_DONTNEED. Ideally it would be helpful to have a fully observable drop in memory. Whereas with sbct=128KB when the single block carriers are shrunk with MADVISE_FREE this is not easily observable. |
Owner
Author
ThomasArts
reviewed
Mar 7, 2023
ThomasArts
left a comment
ThomasArts
left a comment
Contributor
There was a problem hiding this comment.
Timeout discussion necessary.
ThomasArts
reviewed
Mar 7, 2023
ThomasArts
approved these changes
Mar 7, 2023
martinsumner
added a commit
that referenced
this pull request
Mar 10, 2023
* Log report GC Info by manifest level * Hibernate on range query If Block Index Cache is not full, and we're not yielding * Spawn to deserialise blocks offline Hypothesis is that the growth in the heap necessary due to continual term_to_binary calls to deserialise blocks is wasting memory - so do this memory-intensive task in a short-lived process. * Start with hibernate_after option * Always build BIC Testing indicates that the BIC itself is not a primary memory issue - the primary issue is due to a lack of garbage collection and a growing heap. This change enhances the patch to offline serialisation so that: - get_sqn & get_kv are standardised to build the BIC, and hibernate when it is built. - the offline PId is linked to crash this process on failure (as would happen now). * Standardise spawning for check_block/3 Now deserialise in both parts of the code. * Only spawn for check_block if cache not full * Update following review
Merged
martinsumner
added a commit
that referenced
this pull request
Mar 13, 2023
* refactor leveled_sst from gen_fsm to gen_statem * format_status/2 takes State and State Data but this function is deprecated... put in for backward compatibility * refactor leveled_cdb from gen_fsm to gen_statem * disable irrelevant warning ignorer * Remove unnecessary code paths Only support messages, especially info messages, where they are possible. * Mas i1820 offlinedeserialisation cbo (#403) * Log report GC Info by manifest level * Hibernate on range query If Block Index Cache is not full, and we're not yielding * Spawn to deserialise blocks offline Hypothesis is that the growth in the heap necessary due to continual term_to_binary calls to deserialise blocks is wasting memory - so do this memory-intensive task in a short-lived process. * Start with hibernate_after option * Always build BIC Testing indicates that the BIC itself is not a primary memory issue - the primary issue is due to a lack of garbage collection and a growing heap. This change enhances the patch to offline serialisation so that: - get_sqn & get_kv are standardised to build the BIC, and hibernate when it is built. - the offline PId is linked to crash this process on failure (as would happen now). * Standardise spawning for check_block/3 Now deserialise in both parts of the code. * Only spawn for check_block if cache not full * Update following review * Standardise formatting Make test more reliable. Show no new compaction after third compaction. * Update comments --------- Co-authored-by: Thomas Arts <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Use of pro-active hibernate, hibernate_after and offline processing to reduce fluctuation in memory footprints of leveled_sst processes