DS5602-Assignment2
satyajitdas
September 2024
Objective
In this assignment, you will implement and compare two variants of the Lenet
Convolutional Neural Network (CNN) architecture for image classification on
the CIFAR-10 dataset. The task requires you to:
1. Train Lenet CNN without Batch Normalization.
2. Modify the Lenet CNN architecture to incorporate Batch Normalization
after each convolutional layer.
3. Analyze and visualize the feature distributions at each convolutional layer
for both architectures.
4. Compare the impact of Batch Normalization on model performance, par-
ticularly focusing on the loss function behavior and accuracy.
Dataset
You will use the CIFAR-10 dataset, which contains 60,000 32x32 color images
in 10 different classes (e.g., airplane, car, bird, cat, etc.). The dataset is already
divided into:
• 50,000 training images
• 10,000 test images
Part 1: Lenet CNN Architecture (Without Batch
Normalization)
Step 1: Implement the classic Lenet CNN architecture as
follows:
1. Input Layer: Input size: 32x32x3 (RGB images from CIFAR-10).
1
2. Convolutional Layer 1: Apply 6 filters of size 5x5. Stride: 1, Padding:
same. Activation: ReLU. Output size: 32x32x6.
3. Max Pooling Layer 1: Pooling window size: 2x2, Stride: 2. Output
size: 16x16x6.
4. Convolutional Layer 2: Apply 16 filters of size 5x5. Stride: 1, Padding:
valid. Activation: ReLU. Output size: 12x12x16.
5. Max Pooling Layer 2: Pooling window size: 2x2, Stride: 2. Output
size: 6x6x16.
6. Fully Connected Layer 1: Flatten the output. Fully connected layer
with 120 neurons. Activation: ReLU.
7. Fully Connected Layer 2: Fully connected layer with 84 neurons. Ac-
tivation: ReLU.
8. Output Layer: Fully connected layer with 10 neurons (one for each
class). Activation: Softmax.
Step 2: Train the model on the CIFAR-10 dataset:
• Optimizer: Adam with a learning rate of 0.001.
• Loss Function: Categorical Crossentropy.
• Batch Size: 64.
• Epochs: 50.
Part 2: Lenet CNN with Batch Normalization
Step 3: Modify the Lenet architecture by adding Batch
Normalization after each convolutional layer:
1. Convolutional Layer 1 + Batch Normalization: Apply 6 filters of
size 5x5. Activation: ReLU. Batch Normalization after the convolution
operation. Output size: 32x32x6.
2. Max Pooling Layer 1: Pooling window size: 2x2, Stride: 2. Output
size: 16x16x6.
3. Convolutional Layer 2 + Batch Normalization: Apply 16 filters of
size 5x5. Activation: ReLU. Batch Normalization after the convolution
operation. Output size: 12x12x16.
4. Max Pooling Layer 2: Pooling window size: 2x2, Stride: 2. Output
size: 6x6x16.
5. The fully connected layers remain the same as in Part 1.
2
Step 4: Train this modified model with the same hyperpa-
rameters as used in Part 1.
Part 3: Visualize Feature Distributions
Step 5: Visualize and compare the feature distributions at
each convolutional layer for both architectures:
• For Lenet without Batch Normalization, visualize the feature distributions
(activations) after Convolutional Layer 1 and Convolutional Layer 2.
• For Lenet with Batch Normalization, visualize the feature distributions
after Convolutional Layer 1 + BN and Convolutional Layer 2 + BN.
You can use histograms or density plots to represent the activations and
compare the differences in distributions. Provide insights on how Batch Nor-
malization impacts the distribution of feature activations across layers.
Part 4: Impact on Loss Function and Accuracy
Step 6: Plot and compare the following metrics for both
models:
1. Training Loss vs. Epochs – Show how the training loss evolves over 50
epochs.
2. Validation Loss vs. Epochs – Compare the validation loss to observe
generalization performance.
3. Training Accuracy vs. Epochs – Compare the model accuracy on the
training dataset.
4. Validation Accuracy vs. Epochs – Compare the model accuracy on the
validation dataset.
Step 7: Analyze and interpret the results:
• How does Batch Normalization impact the convergence of the loss func-
tion?
• Does Batch Normalization lead to better accuracy? If so, explain why.
• Discuss any improvements in the generalization of the model (validation
performance) with Batch Normalization.
3
Submission Guidelines
1. Code Implementation: Submit the Python code (using TensorFlow/Keras
or PyTorch) for both Lenet models (with and without Batch Normaliza-
tion).
2. Visualization: Include visualizations of the feature distributions for each
convolutional layer for both architectures.
3. Loss and Accuracy Plots: Provide the training and validation loss
curves, as well as accuracy plots for both models.
4. Report: Write a report summarizing the differences in feature distribu-
tions, the impact on the loss function, and the performance of the models.
Include your interpretations and conclusions (maximum 2-3 pages).
