[Performance] Add --optimistic flag that skips DB checks#687
Merged
yannickwurm merged 1 commit intowurmlab:masterfrom Oct 5, 2023
Merged
[Performance] Add --optimistic flag that skips DB checks#687yannickwurm merged 1 commit intowurmlab:masterfrom
yannickwurm merged 1 commit intowurmlab:masterfrom
Conversation
By default, when running SequenceServer.init i.e. when starting the web server or runninc CLI commands, it will scan the entire database_dir entries to find databases that use older (v4) version or have been created without parse_seqids. If this is the case it will log a warning. If all databases are formatted correctly and the user knows it, running SequenceServer with --optimistic can have a significant impact on reducing startup times. With vast databse directories this can be in tens of seconds. This is because parse_seqids in particular does a lot of IO calls to scan database directories to determine if seqids metadata files are missing for each database.
tadast
force-pushed
the
tt/optimistic-mode
branch
from
8b4f6a9 to
f2cf7d0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
By default, when running SequenceServer.init i.e. when starting the web server or runninc CLI commands, it will scan the entire database_dir entries to find databases that use older (v4) version or have been created without parse_seqids. If this is the case it will log a warning.
If all databases are formatted correctly and the user knows it, running SequenceServer with --optimistic can have a significant impact on reducing startup times. With vast databse directories this can be in tens of seconds.
This is because parse_seqids in particular does a lot of IO calls to scan database directories to determine if seqids metadata files are missing for each database.
I think this also constitutes a 2.2.0 gem release