Core

Core Framework code

• optim + trainer + dataset objects
• Sharing CPU tensors
• Add all operations to autograd
• Free GIL when processing big tensors
• Custom CUDA memory allocator (Sam Gross)
• multi-GPU functions
• nccl integration
• finish legacy.nn
• refactor C API for extensions
• create an example extension with TH/pytorch C API
• Checkpointing and improving torch.save / torch.load to use the same byte order
• implement keyword arguments in cwrap
• go over TH and try to make error messages more descriptive (e.g. if the sizes don't match )
• Sparse Tensors on CPU and GPU (Zeming)
• improve tensor printing
• functional API for autograd variables
• Finish multiple CUDA types (Soumith)
• Add stochastic nodes
• Add all modules to nn
• Improved error messages Error messages to improve #39
• sync legacy.nn with Lua nn
• move Trainer to torch.experimental

Operations

• Integrate CuDNN
• Write nn.LSTM*, nn.GRU* etc. to integrate CuDNN RNNs
• Rewrite LookupTable and SparseLinear to use SparseTensors

FBCode stuff

• Import into FBCode

Open Source stuff

• Binary builds
• Continuous builds for CUDA
• MNIST and ResNet18 Contbuilds
• pip wheels

Backward Compatibility

Lua Bridge

• integrate lutorpy into pytorch (either as optional package or by default)
  • change TH indexing to 0-based and add to cwrap the 1-subtraction and addition

Model Loading

• Build a simple model loader based on https://github.com/bshillingford/python-torchfile
• Apply backward-compatibility patches (they've been removed from legacy nn)

Framework Integration

• Caffe2 Integration
  • Modify TH / THC / THNN / THCUNN to integrate them
  • Have a converter that takes in a (Module and input) or (output) and auto-converts it to caffe model
    • vice versa. take a caffe protobuf and codegen a python class with loading weights
• Keras Integration
  • Have a keras backend. Send in a Pull Request to fchollet/keras
• Converting models between TF and Pytorch
  • Torch2TF: Same as caffe convertor pretty much!
  • TF2Torch: same as caffe, but cover ops like tf.if and tf.while

Website

• Find someone to design and code it
• Getting Started
  • Binary installs
    • anaconda-based which links automatically with MKL
    • Each of them for different CUDA versions. 7.0, 7.5, 8.0
  • Source-based installs
• Showcase Demos / Examples / ModelZoo elegantly
• Tutorials
• Look at gym.openai.com (http://gym.openai.com/)
• Developer docs

Documentation, Demos, Examples, Tutorials, ModelZoo

Demos / Examples / ModelZoo

• Pre-trained models for each demo (in the model zoo)
  • Create a python wrapper that allows to search and download models (like nltk)
• Simple API for retraining / using pre-trained models on custom datasets
• Documentation on how to modify the example for one's own experiments
• Most or all of them should be multi-GPU ready

Demos + Examples

• Basic
  • Autoencoders https://github.com/Kaixhin/Autoencoders
• Vision
  • Supervised
    • fb.resnet.torch / googlenet for image classification (sam)
    • fastrcnn (francisco)
    • Video Classification
    • NeuralTalk2 (paszke)
    • Visual Q&A (paszke)
  • Unsupervised
    • Image super-resolution (wafi2x) (soumith)
    • DCGANs + Improved Training for GANs + InfoGAN
    • Text 2 Image (soumith)
    • Pixel RNNs (soumith)
    • Variational AutoEncoders (joost)
• Games / RL (ludc)
  • Good integration with gym.openai.com (http://gym.openai.com/)
  • RL examples
• NLP / Text
  • OpenNMT - attention based Seq2seq with MT(Lerer)
  • Sentiment Analysis
  • Memory networks for Q & A (memn2n) https://github.com/taey16/MemN2N-python
• Metalearning
  • Neural Turing Machine
  • Learning to Learn by Gradient Descent by Gradient Descent
  • Decoupled Neural Interfaces using Synthetic Gradients https://arxiv.org/abs/1608.05343
• ConvNet-Benchmarks / DeepMark scripts

Tutorials

• See Task to track documentation and tutorials #288

Documentation

• Auto-generate from source / docstrings

Links

• [1] http://elm-lang.org/blog/compiler-errors-for-humans
• [2] http://elm-lang.org/blog/compilers-as-assistants

Postponed for next release

• lazy forward execution engine
• double backprop
• Sharing CUDA tensors
• look into Cython
• a built-in profiler for forward/backward (with automatic hints for speeding up the execution?)

AIViz Integration

• Have an intial attempt, and talk to Allan

• Serveable via some Python REST API

• figure out details for images and videos

• Audio

• wav2letter

• DeepSpeech2 for maybe Switchboard or something (Ask Gabriel)

• Sparse Models (ads?)

Distributed Training

• simple distributed trainer like torch-distlearn / torch-ipc / torch-thrift

• Synchronous, asynchronous and Elastic SGD

• Integrate with Andrew / Yangqing's MPI library when that's ready

• Port image classification and seq2seq to this

• error handling

  • create a dict for translating exceptions and adding some pytorch specific info, sort of like in Elm [1,2]
  • make sure there's a clear error message when multiprocessing runs out of fds