Skip to content

Use Concurrent::FixedThreadPool to control compressors threads count - #339

Merged
schneems merged 1 commit into
rails:masterfrom
iggant:fixedthreadpool
Sep 6, 2016
Merged

Use Concurrent::FixedThreadPool to control compressors threads count#339
schneems merged 1 commit into
rails:masterfrom
iggant:fixedthreadpool

Conversation

@iggant

@iggant iggant commented Jul 4, 2016

Copy link
Copy Markdown
Contributor

When we have heavy compressors for gzip files assets precompile can end up with 500 or more threads,
with this commit we restrict possible thread counts to processor count.

@rails-bot

Copy link
Copy Markdown

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @schneems (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

Comment thread lib/sprockets/manifest.rb Outdated

filenames = []
concurrent_compressors = []
concurrent_compressors = Concurrent::FixedThreadPool.new( Concurrent.processor_count )

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the space inside the parentheses

@rafaelfranca

Copy link
Copy Markdown
Member

Makes sense. Thanks

@matthewd

matthewd commented Jul 4, 2016

Copy link
Copy Markdown
Member

I daresay you have larger problems if you're outputting 500 compressible assets... but anyway, that's not how that API works.

@schneems

schneems commented Jul 5, 2016

Copy link
Copy Markdown
Member

Can you fix the style issue mentioned by @rafaelfranca. Also if you rebase against master your tests should pass. That should be good enough to get this across the line.

Thanks for the PR!

@schneems

schneems commented Aug 4, 2016

Copy link
Copy Markdown
Member

Can you fix the style issue mentioned by @rafaelfranca. Also if you rebase against master your tests should pass. That should be good enough to get this across the line.

@matthewd

matthewd commented Aug 4, 2016

Copy link
Copy Markdown
Member

A rebase isn't going to fix this; it is Wrong.

@lucasamorimca

Copy link
Copy Markdown
Contributor

@matthewd could you elaborate ?

hansottowirtz added a commit to hansottowirtz/sprockets that referenced this pull request Aug 30, 2016
@hansottowirtz

Copy link
Copy Markdown
Contributor

I also tried this, it hangs. When I try to precompile assets (with Rails), an error shows up: fatal: No live threads left. Deadlock?. Very strange.

hansottowirtz added a commit to hansottowirtz/sprockets that referenced this pull request Sep 2, 2016
@hansottowirtz

Copy link
Copy Markdown
Contributor

I modified the code a little bit, and it's working now. It is in #372 but don't mind copying it.

@lucasamorimca

Copy link
Copy Markdown
Contributor

@hansottowirtz Could you try #376 and let me know if it works for you?

@schneems
schneems merged commit d463a13 into rails:master Sep 6, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants