Today's API for calling an Autograd function is:

virtual variable_list apply(const variable_list& inputs) override {
  ...
}

This API is unfortunate, because it implies that the caller of apply is obligated to keep inputs (and thus, all of the Tensors it refers to) live for the ENTIRE duration of the apply call. However, particularly large functions (esp. the interpreter we are working on for the JIT) may be able to profitably deallocate inputs before the end of the function.

A very simple API amendment might be to change this to:

virtual variable_list apply(variable_list&&) override {
  ...
}

but we would need to see if we can actually efficiently implement this in the engine.

I'm posting this here because possible API changes will be of interest to other folks working on core.

CC @zdevito @apaszke @colesbury @gchanan