Low Memory mode#8714
Conversation
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
timvisee
force-pushed
the
low-memory-mode
branch
from
caa6170 to
074bd8b
Compare
| // Low-memory mode `no_populate` suppresses mmap prefault globally. | ||
| // Pages will be faulted in on demand when queries touch them. | ||
| if crate::low_memory::low_memory_mode().skip_populate() { | ||
| return Ok(()); | ||
| } | ||
|
|
There was a problem hiding this comment.
It's my understanding the low memory mode should also suppress population of universal IO disk cache.
@xzfc could you also confirm this from your side?
There was a problem hiding this comment.
I would say yes, because otherwise we can crash because of local disk cash is full
timvisee
left a comment
timvisee
left a comment
There was a problem hiding this comment.
Tested locally, works as expected 👌
In fact, it clearly shows how slow loading into memory is for some of our storage components. In my test loading into memory takes 9 seconds, while starting with no_resident makes it startup in 0.5 seconds. I'm using a local NVMe disk.
* [AI] implement parameter + cover populate + cover quantized vectors * telemetry OpenAPI schema * [AI] hook immutable payload indexes * fmt * do not populate payload index if we fallback to mmap * Reformat * Also suppress universal IO disk cache population --------- Co-authored-by: timvisee <[email protected]>
* [AI] implement parameter + cover populate + cover quantized vectors * telemetry OpenAPI schema * [AI] hook immutable payload indexes * fmt * do not populate payload index if we fallback to mmap * Reformat * Also suppress universal IO disk cache population --------- Co-authored-by: timvisee <[email protected]>
2 participants
Low memory mode
Motivation
It is a frequent situation in production, when customer just keep pushing more data regardless of the machine capacity.
At some point capacity is exhausted and machine cashes. In worst case, machine goes into crash loop, and there are no
nice way to recover it from this situation, as we can't even change config as API are not available.
We need a way to recovery from this situation.
Proposal
Special configuration option
low_memory_modeis added to the config.Should have 3 options:
disabled (default) - no special handling, all collection modules are loaded as usual
no-residentWhen it is set, loading of all collection modules should not force anything to RAM if possible:
always_ram=falseand vectors are on diskon_disk=trueno-populate- same asno-resident, but also no population of RAM from disk should be done. This affects loading of orginal vectors, HNSW index, payload storageImplementation details
Testing scenario: