adding nvidia models #22

nv-kkudrynski · 2019-06-05T21:40:51Z

No description provided.

ailzhang · 2019-06-05T21:43:37Z

scripts/run_pytorch.sh

-  # FIXME: GAN models checkpoints are on cuda.
-  if [[ $f = $GANs* ]]; then
+  # FIXME: GAN and NVIDIA models checkpoints are on cuda.
+  if [[ $f = $GANs* ]] || [[ $f = $CUDAs* ]]; then


Adding a cuda checkpoint will require user have a cuda pytorch installed to load it. Would it possible to convert them to cpu in hubconf entrypoints and relax it here?

considering this is NVIDIA, i think it's okay

thx. I had to revive the above condition in run_pytorch.sh to pass your CI

netlify · 2019-06-05T21:43:40Z

Deploy preview for pytorch-hub-preview ready!

Built with commit 4bdea5b

https://deploy-preview-22--pytorch-hub-preview.netlify.com

ailzhang · 2019-06-05T21:50:18Z

@nv-kkudrynski Also I'm not sure why this didn't trigger the circleCi check. See #23 as an example, maybe try rebasing?

nvidia_deeplearningexamples_tacotron2.md

nvidia_deeplearningexamples_waveglow.md

gottbrath · 2019-06-06T13:23:25Z

@soumith -- are we going to be able to accept this whole pull request? I believe from side conversation you had some concerns with the NCF. Have those been addressed?

If we can't accept the NCF part would it make sense to split this so that we could get the waveglow and tacotron2 models up first then address the NCF issues?

soumith · 2019-06-06T13:29:59Z

yes, there are a few changes needed.

can we remove NCF from this PR for now -- I dont think the model as it stands is useful for Hub (we are curating), because it isn't really useful for building research on top of it -- or useful to visualize anything about the output
Can you add code that visualizes the outputs of WaveGlow / TacoTron-2. They produce spectrograms, that's great -- it'd be useful to see that. You can use the GAN model that I cleaned up as an example: https://pytorch.org/hub/facebookresearch_pytorch-gan-zoo_dcgan/ . This way, people pulling your model know what to do with it further.

nvidia_deeplearningexamples_tacotron2.md

soumith · 2019-06-06T13:31:25Z

nvidia_deeplearningexamples_tacotron2.md

+hub_model = torch.hub.load(github='nvidia/DeepLearningExamples', model='nvidia_tacotron2')
+hub_model = hub_model.cuda()
+hub_model.eval()
+inp = torch.randint(low=0, high=148, size=(1,140), dtype=torch.long)


describe what this input is supposed to be, for example see https://pytorch.org/hub/facebookresearch_pytorch-gan-zoo_dcgan/

changed strategy of this example to go from plain text to sound using the two torch.hub models in series

soumith · 2019-06-06T13:32:40Z

nvidia_deeplearningexamples_waveglow.md

+hub_model = torch.hub.load(github='nvidia/DeepLearningExamples', model='nvidia_waveglow')
+```
+will load the WaveGlow model pre-trained on [LJ Speech dataset](https://keithito.com/LJ-Speech-Dataset/)
+


same thoughts as above. Add details on what the model takes as input and gives as output.
For example see: https://pytorch.org/hub/facebookresearch_pytorch-gan-zoo_dcgan/

changed strategy of this example to go from plain text to sound using the two torch.hub models in series

soumith · 2019-06-06T13:33:10Z

nvidia_deeplearningexamples_waveglow.md

+print('\nWaveglow model test output:')
+print(out.size())
+```
+


add a visualization of the test output, or some other way to interpret the output

changed strategy of this example to go from plain text to sound using the two torch.hub models in series

soumith · 2019-06-06T13:33:30Z

scripts/run_pytorch.sh

-  # FIXME: GAN models checkpoints are on cuda.
-  if [[ $f = $GANs* ]]; then
+  # FIXME: GAN and NVIDIA models checkpoints are on cuda.
+  if [[ $f = $GANs* ]] || [[ $f = $CUDAs* ]]; then


considering this is NVIDIA, i think it's okay

…ng_nvidia_models # Conflicts: # scripts/install.sh # scripts/run_pytorch.sh

nv-kkudrynski · 2019-06-07T13:05:35Z

can we remove NCF from this PR for now -- I dont think the model as it stands is useful for Hub (we are curating), because it isn't really useful for building research on top of it -- or useful to visualize anything about the output

Indeed, NCF is quite specific. We also had discussions about it. It could however be useful if loaded pretrained=False (I added example in readme and provided reference for details about training). Then pre-trained model could serve as reference. Let me know what you think.

Can you add code that visualizes the outputs of WaveGlow / TacoTron-2. They produce spectrograms, that's great -- it'd be useful to see that. You can use the GAN model that I cleaned up as an example: https://pytorch.org/hub/facebookresearch_pytorch-gan-zoo_dcgan/ . This way, people pulling your model know what to do with it further.

In our flow formatting input/output is not part of the model object, so we need a few internal changes to adapt to your request. We are working on this atm.

soumith · 2019-06-07T13:12:35Z

@nv-kkudrynski i think NCF can serve as a reference with pretrained=False if it's easy to train upon. Is the code snippet that shows -- taking a csv dataset or Pandas Dataframe, training the NCF model with it -- going to be about ~20 lines or less? If so, I think it can be worth it for the reader who comes to the hub page. If the reference model is only useful if the data is preprocessed and run through a high-performance set of instructions that are given in the NCF repo, then it's almost better to just point users to the NCF repo directly without a card.

soumith · 2019-06-07T13:13:51Z

even an "easy-finetuning" starting from the pretrained model possibility has potential, but from what I can tell with the dataset, the pre-trained model doesn't have much to gain wrt fine-tuning -- the features aren't generic to generalize to a different set of users + attributes to do recommendation on

soumith · 2019-06-07T13:17:00Z

In our flow formatting input/output is not part of the model object, so we need a few internal changes to adapt to your request. We are working on this atm.

Great, thanks for the update.
Just FYI, if the input our output is not part of the model object, that is okay. This is why hub entrypoints are callables and not models.
For example, you can have an entry-point for pre-processing, and an entry-point for post-processing.
In the BERT example, they have an entry bertTokenizer which is not a model, it's just a pre-processing tokenizer -- but it's available as an entrypoint.
The example showcases taking an input, tokenizing it, and then passing in the output of bertTokenizer into bertModel. Maybe something like this will help with your case as well, just a thought.

nv-kkudrynski · 2019-06-07T18:43:32Z

Just FYI, if the input our output is not part of the model object, that is okay. This is why hub entrypoints are callables and not models.

I think there was a discussion about it which concluded with exactly the opposite :)
Which is reflected in your guidelines at: https://pytorch.org/docs/stable/hub.html:

Entrypoint function should ALWAYS return a model(nn.module).

But don't worry, update is coming ;)

soumith · 2019-06-07T18:48:38Z

@ailzhang that line needs to be removed I think?

nv-kkudrynski · 2019-06-07T19:51:32Z

I removed NCF from this PR to enable tacotron2/waveglow hopefully before the weekend.
Now the 2 models play very nicely together in the examples - producing sound from input text.

gottbrath · 2019-06-09T18:57:05Z

@soumith - have the changes here addressed all your concerns?

soumith · 2019-06-10T12:40:28Z

thanks so much @nv-kkudrynski , the updated stuff looks good. I might make some additional changes so that it plays well with opening the notebooks in Google Colab.

Additionally, if you are interested as a follow-up (totally up to you and if it makes sense), add end-points for any intermediate features that might make sense for folks to use these models as a feature-extractor / fine-tuner.

nv-kkudrynski · 2019-06-10T16:22:31Z

Thanks to everyone for all the feedback and smooth cooperation!

Summary: update doc as pointed out in pytorch/hub#22 Pull Request resolved: #21568 Differential Revision: D15732927 Pulled By: ailzhang fbshipit-source-id: 78ab026539e5ee59e7c3a8144e2c9fcbbc225733

soumith · 2019-06-11T01:52:40Z

fyi, I have updated the hub card to play well with google colab -- now you can generate the speech and listen to it immediately, all in the colab notebook.

nv-kkudrynski · 2019-06-11T07:59:53Z

Thanks a lot for that!

adding nvidia models

5eddfa4

ailzhang reviewed Jun 5, 2019

View reviewed changes

nvidia_deeplearningexamples_tacotron2.md Outdated Show resolved Hide resolved

nvidia_deeplearningexamples_tacotron2.md Outdated Show resolved Hide resolved

nvidia_deeplearningexamples_tacotron2.md Outdated Show resolved Hide resolved

ailzhang reviewed Jun 5, 2019

View reviewed changes

nvidia_deeplearningexamples_tacotron2.md Outdated Show resolved Hide resolved

nvidia_deeplearningexamples_waveglow.md Outdated Show resolved Hide resolved

fixed titles

9e90d62

soumith suggested changes Jun 6, 2019

View reviewed changes

nv-kkudrynski added 4 commits June 7, 2019 12:41

Merge branch 'master' of https://github.com/pytorch/hub into publishi…

39f91b3

…ng_nvidia_models # Conflicts: # scripts/install.sh # scripts/run_pytorch.sh

adding images, improving ncf, minor PR resolutions

da9e5d6

temporary tagging ncf as nlp (just to pass sanity check)

4b3a1e3

bypassing CI for CUDA checkpoints

0b2c710

nv-kkudrynski added 2 commits June 7, 2019 21:34

making tacotron2 and waveglow more comprehensible

f6c5180

postponing ncf

4bdea5b

ailzhang mentioned this pull request Jun 8, 2019

update hub doc pytorch/pytorch#21568

Closed

soumith approved these changes Jun 10, 2019

View reviewed changes

soumith merged commit aafb3e3 into pytorch:master Jun 10, 2019

nv-kkudrynski deleted the publishing_nvidia_models branch June 10, 2019 20:22

adding nvidia models #22

adding nvidia models #22

Uh oh!

Conversation

nv-kkudrynski commented Jun 5, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

netlify bot commented Jun 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ailzhang commented Jun 5, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gottbrath commented Jun 6, 2019

Uh oh!

soumith commented Jun 6, 2019

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nv-kkudrynski commented Jun 7, 2019

Uh oh!

soumith commented Jun 7, 2019

Uh oh!

soumith commented Jun 7, 2019

Uh oh!

soumith commented Jun 7, 2019

Uh oh!

nv-kkudrynski commented Jun 7, 2019

Uh oh!

soumith commented Jun 7, 2019

Uh oh!

nv-kkudrynski commented Jun 7, 2019

Uh oh!

gottbrath commented Jun 9, 2019

Uh oh!

soumith commented Jun 10, 2019

Uh oh!

nv-kkudrynski commented Jun 10, 2019

Uh oh!

soumith commented Jun 11, 2019

Uh oh!

nv-kkudrynski commented Jun 11, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

netlify bot commented Jun 5, 2019 •

edited

Loading