-
Notifications
You must be signed in to change notification settings - Fork 26.3k
Closed
Labels
module: cppRelated to C++ APIRelated to C++ APItriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Description
🐛 Bug
L-BFGS optimizer will not work if there is one or more registered parameters (tensors) with no grad in the model:
terminate called after throwing an instance of 'c10::Error'
what(): There were no tensor arguments to this function (e.g., you passed an empty list of Tensors), but no fallback function is registered for schema aten::view. This usually means that this function requires a non-empty list of Tensors. Available functions are [CUDATensorId, QuantizedCPUTensorId, VariableTensorId, CPUTensorId, MkldnnCPUTensorId] (lookup_ at /pytorch/aten/src/ATen/core/dispatch/DispatchTable.h:245)
To Reproduce
Code to reproduce the error:
/* A custom dataset for test */
namespace torch
{
namespace data
{
namespace datasets
{
struct TEST_TensorDataset : public Dataset<TEST_TensorDataset>
{
private:
torch::Tensor data1, data2;
public:
explicit TEST_TensorDataset(std::vector<torch::Tensor> tensors):
data1(tensors[0]), data2(tensors[1])
{}
torch::data::Example<> get(size_t index) override
{
return {data1[index], data2[index]};
}
optional<size_t> size() const override
{
return data1.sizes()[0];
}
};
}
}
}
/* Test */
int test()
{
struct Net : torch::nn::Module
{
Net():
fc1(128, 256),
fc2(256, 1)
{
register_module("fc1", fc1);
register_module("fc2", fc2);
/* The following line will cause the error */
tmp = register_parameter("tmp", torch::ones({1,1}), false);
}
torch::Tensor forward(torch::Tensor x)
{
x = torch::tanh(fc1->forward(x));
x = fc2->forward(x);
return x;
}
torch::nn::Linear fc1;
torch::nn::Linear fc2;
torch::Tensor tmp;
};
Net model;
int batchsize = 16;
int stop_epoch = 10;
torch::Tensor data = torch::randn({32, 128});
torch::Tensor target = torch::randn({32, 1});
auto dataset = torch::data::datasets::TEST_TensorDataset({data, target}).map(torch::data::transforms::Stack<>());
auto data_loader = torch::data::make_data_loader<torch::data::samplers::RandomSampler>(std::move(dataset), torch::data::DataLoaderOptions().batch_size(batchsize));
torch::optim::LBFGS optimizer_lbfgs = torch::optim::LBFGS(model.parameters(), torch::optim::LBFGSOptions(1E-2));
for (int epoch = 0; epoch <= stop_epoch - 1; epoch++)
{
int batch_idx = 0;
bool output_loss = true;
for (auto & batch : *data_loader)
{
output_loss = true;
auto closure = [&]()
{
model.zero_grad();
torch::Tensor target_net = model.forward(batch.data);
torch::Tensor loss = torch::mse_loss(target_net, batch.target);
loss.backward();
if (output_loss)
{
std::cout << "Epoch: " << epoch << " Batch: " << batch_idx << " loss: " << loss << std::endl;
output_loss = false;
}
return loss;
};
optimizer_lbfgs.step(closure);
batch_idx++;
}
}
return 0;
}
Expected behavior
If line 12 in the function test(): tmp = register_parameter("tmp", torch::ones({1,1}), false); is commented out, then everything goes fine.
Environment
- PyTorch Version (e.g., 1.0): 1.2.0
- OS (e.g., Linux): Ubuntu 16.04.6 LTS
- How you installed PyTorch (
conda,pip, source): pip - Build command you used (if compiling from source):
- Python version: 3.5
- CUDA/cuDNN version: 9.2/7.5
- GPU models and configuration: GeForce GTX TITAN
- Any other relevant information:
GCC version: (Ubuntu 4.9.4-2ubuntu1~16.04) 4.9.4
CMake version: version 3.15.2
cc @yf225
Metadata
Metadata
Assignees
Labels
module: cppRelated to C++ APIRelated to C++ APItriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module