Introduce parallelism for batch conversion: --parallel / -P
#628
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces
--parallel/-Poption to set the number of concurrency of parallel conversion for multiple files.# Convert Markdown files in `./marp-slides` parallel (up to 5 workers: default) marp --parallel 5 ./marp-slidesBy applying the robust async process (async initialization is running only once, and all of successors will wait that initialization) to browser operations, the parallel processing could be achieved in a simple queue and workers.
Marp CLI will use up to 5 workers while converting multiple files in the queue by default. If used Marp CLI for converting multiple Markdowns into PDF/PPTX/image(s), this setting will make drastically faster conversion than previous. CLI user can tweak the number of concurrency by using
--parallelor-Poption.Resolves #509.
Performance improvement
marp-parallelfolder is including 30 empty Markdown files, and Marp CLI tries to convert every Markdown files into PDF.time marp --pdf --no-parallel ./marp-paralleltime marp --pdf --parallel=5 ./marp-paralleltime marp --pdf --parallel=10 ./marp-parallelIncreasing the number of concurrency from 5 to 10, the conversion will become much faster, but the CPU usage may stick to 100% while conversion in my environment. For the balance between the speed and CPU, I decided the default concurrecny as 5.