perf: speed up (>3x) get_uris, display_downloadfile with bash builtin#239
Merged
Lasall merged 1 commit intoilikenwf:masterfrom May 1, 2024
Merged
perf: speed up (>3x) get_uris, display_downloadfile with bash builtin#239Lasall merged 1 commit intoilikenwf:masterfrom
Lasall merged 1 commit intoilikenwf:masterfrom
Conversation
While downloading is relatively faster than the vanilla APT, the overhead of parsing notably slows apt-fast down before it can start downloading. This is because the massive usage of utilities causes bash to spawn processes frequently, which is very expensive. The performance hot spots of apt-fast are get_uris() and display_downloadfile(). The patch replaces some utilities' usage in these functions with bash builtins to eliminate the extra overhead. My brief test (with default /etc/apt-fast.conf) shows the overall parsing performance is >3x faster than before. master: $ time (echo 'n' | sudo apt-fast dist-upgrade | wc -l) 417 real 0m7.785s user 0m0.007s sys 0m0.023s patch applied: $ time (echo 'n' | sudo apt-fast dist-upgrade | wc -l) 417 real 0m2.002s user 0m0.014s sys 0m0.015s Signed-off-by: Rongrong <[email protected]>
Collaborator
|
Very cool, thank you for this PR. |
Lasall
reviewed
Apr 29, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While downloading is relatively faster than the vanilla APT, the overhead of parsing notably slows apt-fast down before it can start downloading. This is because the massive usage of utilities causes bash to spawn processes frequently, which is very expensive.
The performance hot spots of apt-fast are
get_uris()anddisplay_downloadfile(). The patch replaces some utilities' usage in these functions with bash builtins to eliminate the extra overhead. My brief test (with default/etc/apt-fast.conf) shows the overall parsing performance is >3x faster than before.master:
patch applied: