CSY3025 Artificial Intelligence Techniques
Deep Learning
Feature extraction with data
augmentation
Feature extraction with data augmentation
• Assemble the models
from keras import models
from keras import layers
model = models.Sequential()
model.add(conv_base)
model.add(layers.Flatten())
model.add(layers.Dense(256, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
>>> model.summary()
Layer (type) Output Shape Param #
vgg16 (Model) (None, 4, 4, 512) 14714688
flatten_1 (Flatten) (None, 8192) 0
dense_1 (Dense) (None, 256) 2097408
dense_2 (Dense) (None, 1) 257
================================================================
Total params: 16,812,353 Trainable params: 16,812,353 Non-trainable params: 0
Feature extraction with data augmentation
• We simply use the trained convolutional base, so we’ll freeze it during training. Freezing a layer or set of
layers means preventing their weights from being updated during training.
• If you don’t do this, then the representations that were previously learned by the convolutional base
will be modified during training.
conv_base.trainable = False
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
vgg16 (Model) (None, 4, 4, 512) 14714688
flatten_2 (Flatten) (None, 8192) 0
dense_3 (Dense) (None, 256) 2097408
dense_4 (Dense) (None, 1) 257
=================================================================
Total params: 16,812,353 Trainable params: 2,097,665 Non-trainable params: 14,714,688
Feature extraction with data augmentation
Additional reading
Visualise covnet learning
Visualise covnet learning
It’s often said that deep-learning models are “black boxes”: learning
representations that are difficult to extract and present in a human-readable form.
There are some techniques to visualise covnet learning:
• Visualizing intermediate convnet outputs (intermediate activations)— Useful for
understanding how successive convnet layers transform their input, and for getting a first idea
of the meaning of individual convnet filters.
• Visualizing convnets filters— Useful for understanding precisely what visual pattern or
concept each filter in a convnet is receptive to.
• Visualizing heatmaps of class activation in an image— Useful for understanding which parts
of an image were identified as belonging to a given class, thus allowing you to localize objects
in images.
Visualizing intermediate convnet outputs
Visualizing intermediate activations consists of displaying the feature
maps that are output by various convolution and pooling layers in a
network, given a certain input.
This gives a view into how an input is decomposed into the different
filters learned by the network. You want to visualize feature maps with
three dimensions: width, height, and depth (channels).
Each channel encodes relatively independent features, so the proper way
to visualize these feature maps is by independently plotting the contents
of every channel as a 2D image.
Visualizing intermediate convnet outputs
>>> from keras.models import load_model
>>> model =
load_model('cats_and_dogs_small_2.h5’)
>>> model.summary()
1st layer, 5th channel 5th layer, 8th channel
As you go higher, the activations become increasingly abstract and less visually interpretable. They begin to encode higher-
level concepts such as “cat ear” and “cat eye.” Higher presentations carry increasingly less information about the visual
contents of the image, and increasingly more information related to the class of the image.
Visualizing intermediate convnet outputs
The features extracted by a layer become increasingly abstract with the depth
of the layer.
The activations of higher layers carry less and less information about the
specific input being seen, and more and more information about the target
(in this case, the class of the image: cat or dog).
A deep neural network effectively acts as an information distillation pipeline,
with raw data going in (in this case, RGB pictures) and being repeatedly
transformed so that irrelevant information is filtered out (for example, the
specific visual appearance of the image), and useful information is magnified
and refined (for example, the class of the image).
Visualizing intermediate convnet outputs
This is analogous to the way humans and animals perceive the world: after observing a scene for a
few seconds, a human can remember which abstract objects were present in it but can’t remember
the specific appearance of these objects.
In fact, if you tried to draw a generic bicycle from memory, chances are you couldn’t get it even
remotely right, even though you’ve seen thousands of bicycles.
Visualizing heatmaps of class activation
This general category of techniques is called class activation map (CAM) visualization,
and it consists of producing heatmaps of class activation over input images.
A class activation heatmap is a 2D grid of scores associated with a specific output
class, computed for every location in any input image, indicating how important each
location is with respect to the class under consideration.
