One of the weird things, is that to create a cuda tensor of the same type and size on a different device, you now have to write:

with torch.cuda.device(3):
    x = type(tensor)(tensor.size())

This does not work, because it places x on tensor's device:

with torch.cuda.device(3):
    x = tensor.new(tensor.size())

We should probably allow explicitly choosing the device in the constructor and new functions:

x = torch.cuda.FloatTensor(foo.size(), device=3)
y = x.new(device=3)
y = x.type('torch.cuda.FloatTensor', device=3)