Machine Learning
Pytorch Tutorial 2
Documentation and Common Errors
TA: 石旻翰
2023.02.20
[email protected]
PyTorch Documentation
https://pytorch.org/docs/stable/
● torch.nn -> Neural Network
● torch.optim -> Optimization Algorithms
● torch.utils.data -> Dataset, Dataloader
PyTorch Documentation Example
Function inputs and outputs
Data type and explanation of each input
PyTorch Documentation Example
● Some functions behave
differently with different inputs
● Parameters : You don’t need to
specify the name of the
argument (Positional Arguments)
● Keyword Arguments : You have
to specify the name of the
argument
They are separated by *
PyTorch Documentation Example
● Some functions behave
differently with different inputs
● Arguments with default value :
Some arguments have a default
value (keepdim=False), so
passing a value of this argument
is optional
PyTorch Documentation Example
Three Kinds of torch.max
1. torch.max(input) → Tensor
2. torch.max(input, dim, keepdim=False, *,
out=None) → (Tensor, LongTensor)
3. torch.max(input, other, *, out=None) →
Tensor
input : Tensor, dim : int, keepdim : bool
other : Tensor
PyTorch Documentation Example
1.torch.max(input) → Tensor
Find the maximum value of a tensor, and return that value.
PyTorch Documentation Example
2. torch.max(input, dim, keepdim=False, *,
out=None) → (Tensor, LongTensor)
Find the maximum
value of a tensor
along a dimension,
and return that value,
along with the index
corresponding to that
value.
PyTorch Documentation Example
3.torch.max(input, other) → Tensor
Perform element-wise
comparison between two
tensors of the same size,
and select the maximum of
the two to construct a
tensor with the same size.
Common Errors - torch.max (Colab)
Three Kinds of torch.max Colab code
1. torch.max(input) x = torch.randn(4,5)
→ Tensor y = torch.randn(4,5)
2. torch.max(input, m, idx = torch.max(x,0,False,p)→x
dim, keepdim=False,
*out is a keyword argument
*, out=None) →
m, idx = torch.max(x,True)→x
(Tensor, LongTensor)
3. torch.max(input, *did not specify dim
other,
*, out=None) → Tensor
input : Tensor
dim : int
keepdim : bool
other : Tensor
Common Errors - Tensor on Different Device to Model
model = torch.nn.Linear(5,1).to("cuda:0")
x = torch.randn(5).to("cpu")
y = model(x)
Tensor for * is on CPU, but expected them to be on GPU
=> send the tensor to GPU
x = torch.randn(5).to("cuda:0")
y = model(x)
print(y.shape)
Common Errors - Mismatched Dimensions
x = torch.randn(4,5)
y = torch.randn(5,4)
z = x + y
The size of tensor a (5) must match the size of tensor b (4) at non-singleton
dimension 1
=> the shape of a tensor is incorrect, use transpose, squeeze, unsqueeze to align
the dimensions
y = y.transpose(0,1)
z = x + y
print(z.shape)
Common Errors - Cuda Out of Memory
import torch
import torchvision.models as models
resnet18 = models.resnet18().to( "cuda:0" ) # Neural Networks for Image
Recognition
data = torch.randn( 512,3,244,244) # Create fake data (512
images)
out = resnet18(data.to( "cuda:0" )) # Use Data as Input and Feed to
Model
print(out.shape)
CUDA out of memory. Tried to allocate 350.00 MiB (GPU 0; 14.76 GiB total
capacity; 11.94 GiB already allocated; 123.75 MiB free; 13.71 GiB reserved in
total by PyTorch)
=> The batch size of data is too large to fit in the GPU. Reduce the batch size.
Common Errors - Mismatched Tensor Type
import torch.nn as nn
L = nn.CrossEntropyLoss()
outs = torch.randn(5,5)
labels = torch.Tensor([1,2,3,4,0])
lossval = L(outs, labels) # Calculate CrossEntropyLoss between outs and labels
expected scalar type Long but found Float
=> labels must be long tensors, cast it to type “Long” to fix this issue
labels = labels.long()
lossval = L(outs,labels)
print(lossval)
Any Question?
