Add multiprocessing_context= argument to DataLoader #22990
Conversation
pytorchbot
added
module: dataloader
zhangguanheng66
added
the
triaged
|
@pytorchbot rebase this please |
|
@apaszke could you take a look? :) thanks |
apaszke
left a comment
apaszke
left a comment
There was a problem hiding this comment.
Blah, another argument to the DataLoader 😩
LGTM, but might be better to make the attribute always set, with the default being the multiprocessing module. That would make for simpler code.
| assert self.num_workers > 0 | ||
|
|
||
| if loader.multiprocessing_context is None: | ||
| multiprocessing_context = multiprocessing |
There was a problem hiding this comment.
Can't we initialize the attribute with the multiprocessing module by default? You can make an exception for the isinstance() rule if it matches the module by identity.
facebook-github-bot
left a comment
facebook-github-bot
left a comment
There was a problem hiding this comment.
@soumith is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
facebook-github-bot
left a comment
facebook-github-bot
left a comment
There was a problem hiding this comment.
@colesbury has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
|
@colesbury merged this pull request in e982e46. |
Summary: #22990 added a multiprocessing_context argument to DataLoader, but a typo in the test causes the wrong DataLoader class to be used. Pull Request resolved: #43343 Reviewed By: glaringlee Differential Revision: D23299452 Pulled By: malfet fbshipit-source-id: 9489c48b83bce36f46d350cad902f7ad96e1eec4
7 participants
Fixes #22131