nit: ModuleList -> ModuleDict

Curious would the actual pretty print function prints the content of the ModuleDict as well? For the Python version it seems to do so:

>>> import torch
>>> from torch import nn
>>> a = nn.ModuleDict({
...                         'conv': nn.Conv2d(10, 10, 3),
...                         'pool': nn.MaxPool2d(3)
...                 })
>>> a
ModuleDict(
  (conv): Conv2d(10, 10, kernel_size=(3, 3), stride=(1, 1))
  (pool): MaxPool2d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
)

Should this function be called empty? I think ModuleDict's API should mirror std::map, and std::map has empty instead of is_empty. (We should probably deprecate ModuleList::is_empty and Sequential::is_empty as well and redirect them to the empty function, so that they are more in-line with the API of std::vector.)

Should we add keys and values?

    def keys(self):
        r"""Return an iterable of the ModuleDict keys.
        """
        return self._modules.keys()

    def values(self):
        r"""Return an iterable of the ModuleDict values.
        """
        return self._modules.values()

I think we can combine these two update methods with

  void update(ModuleDictImpl other) {
    children_.update(std::move(other.children_));
  }

which would be optimal in both "copying" and "moving" cases.

Also I believe we should call register_module to register all the newly added modules.

same for this function: we can likely use

  std::shared_ptr<Module>& insert(
      const std::string& key,
      std::shared_ptr<Module> value)

and then do std::move inside the function.

Also I believe we should call register_module to register the newly added module.

It's a bit concerning that we have these two operator[] overloads for ModuleDict but not ModuleList or Sequential (they only have std::shared_ptr<Module> operator[](size_t index) const). I wondered if it would make sense to do a few things:

1. Add ModuleList::operator[] and Sequential::operator[] overloads for both std::shared_ptr<Module>& and const std::shared_ptr<Module>& return types
2. Add test for lvalue assignment for Sequential / ModuleList / ModuleDict's operator[] method, e.g. module_list[0] = SomeModule()

Regarding the data structure for storing the modules, do you think we can use torch::OrderedDict? I think AnyModule is more about supporting calling forward on a module without needing to know its concrete type, and we don't have this requirement for ModuleDict.

Is there a specific reason we need to add this?

It would be awesome to add a ModuleDictImpl constructor that takes a torch::OrderedDict as input, so that we can do something similar to:

torch::OrderedDict<torch::nn::Module> modules = {{"a", ModuleA()}, {"b", ModuleB()}};
auto moduledict = torch::nn::ModuleDict(modules);