Lecture Notes on Applied Aritficial Intelligence (ME-3181)
Introduction to Convolution Neural
Networks (CNN)
Phung Thanh Huy
Ho Chi Minh City Univerisyt of Technology
Vietnam National University Ho Chi Minh City
Ho Chi Minh City, 2023
Review of Artificial Neural Network
Single Perceptron
𝑧 𝑧 = 𝑥𝑖 𝑤𝑖
𝑦
f 𝑖=0
𝑦=𝑓 𝑧
o 𝑥0 = 1, 𝑤0 : bias
o 𝑓 or 𝜎: activation function
o For regression, 𝑓 = 𝑖𝑑 𝑧 or 𝑓 = 𝐿 𝑧
o 𝑓 for binary classification: sigmoid, step, sign, thresholding…
Applied Artificial Intelligence (ME-3181) Pg. 2
Review of Artificial Neural Network
Fully connected Neural Networks
o Repeat the peceptrons
𝑗
• Feedforward: The weights 𝑤𝑖 already exist
Applied Artificial Intelligence (ME-3181) Pg. 3
Review of Artificial Neural Network
Fully connected Neural Networks
o For classification, activation function at the output layer can be sigmoid function or
softmax function
o Other available activation functions: ReLU, Tanh, ELU,….: usually for hidden layers or
regression (at the output layer)
Applied Artificial Intelligence (ME-3181) Pg. 4
Review of Artificial Neural Network
Fully connected Neural Networks
𝑗
o Feedforward: The weights 𝑤𝑖 already exist
𝑗
o Y is calculated from 𝑥 and 𝑤𝑖
Applied Artificial Intelligence (ME-3181) Pg. 5
Review of Artificial Neural Network
Fully connected Neural Networks
𝑗
o Training = finding the weights 𝑤𝑖 from the dataset
o Algorithm:
+ Loss-fuction: shows the errors between the real and
predicted data.
+ Popular loss-functions:
MSE / MAE: regression
Cross entropy functions:
+ Target of training:
Minimize Loss function, or maximize likelihood.
+ Training process:
Back probagation
Gradient Descent
Applied Artificial Intelligence (ME-3181) Pg. 6
Review of Artificial Neural Network
Fully connected Neural Networks
Batch gradient descent:
Calculate the error for all instances and then update
the weights
Stochastic gradient descent:
Calculate the error for each instance and then update
the weights
Minibatch:
Select a pre-defined number of instances in order to
calculate the error and update the weights
Hyperparamenters: the parameters to train a model
Learning rate
Batch
Epoch
Applied Artificial Intelligence (ME-3181) Pg. 7
Review of Artificial Neural Network
Create and train a network
Classify Fashion MNIST Dataset
Applied Artificial Intelligence (ME-3181) Pg. 8
Computer Vision – Convolution and Correlation
Extract the “useful features” from the
images
Applied Artificial Intelligence (ME-3181) Pg. 9
Computer Vision – Convolution and Correlation
Applied Artificial Intelligence (ME-3181) Pg. 10
Computer Vision – Convolution and Correlation
Convolution
Correlation
No-rotation
Applied Artificial Intelligence (ME-3181) Pg. 11
Computer Vision – Convolution and Correlation
2D Convolution
- Convolution: Rotate and multiply
- Correlation: No rotation
Applied Artificial Intelligence (ME-3181) Pg. 12
Computer Vision – Convolution and Correlation
Applied Artificial Intelligence (ME-3181) Pg. 13
Computer Vision – Convolution and Correlation
Applied Artificial Intelligence (ME-3181) Pg. 14
Computer Vision – Convolution and Correlation
Convolution
Xử lý thông tin
Tăng Cường Trích xuất từ Features
Ảnh
Ảnh / Lọc ảnh Features (Nhận dạng…)
Applied Artificial Intelligence (ME-3181) Pg. 15
Structure of Convolution Neural Networks
Step 1 Step 2
Convolutional Neural Network
Applied Artificial Intelligence (ME-3181) Pg. 16
Structure of Convolution Neural Networks
2D Convolution in CNN
• Kernel size:
• Kích thước của ma trận kernel
• Có thể mang giá trị bất kỳ.
• Thường là số lẻ.
• Thường dùng 3, 5, 7.
• Kernel Depth: Chiều sâu của Kernel
• Là số lượng các kernel khác nhau cùng áp dụng vào 1 ảnh.
Applied Artificial Intelligence (ME-3181) Pg. 17
Structure of Convolution Neural Networks
Padding
Applied Artificial Intelligence (ME-3181) Pg. 18
Structure of Convolution Neural Networks
Stride
Applied Artificial Intelligence (ME-3181) Pg. 19
Structure of Convolution Neural Networks
Pooling
• Dùng Pooling để giảm kích thước của ảnh, từ đó giảm số chiều đưa vào
mạng Neuron.
• Giảm overfitting
• Giảm khối lượng tính toán
• Tại sao Pooling hiệu quả: Do các neighbor pixel thường có tương quan.
Average Pool
Applied Artificial Intelligence (ME-3181) Pg. 20
Structure of Convolution Neural Networks
Dropout
• Là một kỹ thuật giảm số chiều và tránh overfitting
Applied Artificial Intelligence (ME-3181) Pg. 21
Structure of Convolution Neural Networks
Flattening
Applied Artificial Intelligence (ME-3181) Pg. 22
Structure of Convolution Neural Networks
CNN structure and dimension
Applied Artificial Intelligence (ME-3181) Pg. 23
Structure of Convolution Neural Networks
CNN structure and dimension
Applied Artificial Intelligence (ME-3181) Pg. 24
Structure of Convolution Neural Networks
CNN structure and dimension
Applied Artificial Intelligence (ME-3181) Pg. 25
Structure of Convolution Neural Networks
CNN structure and dimension
Applied Artificial Intelligence (ME-3181) Pg. 26
Structure of Convolution Neural Networks
CNN structure and dimension
Applied Artificial Intelligence (ME-3181) Pg. 27
Structure of Convolution Neural Networks
CNN structure and dimension
Applied Artificial Intelligence (ME-3181) Pg. 28
