Week 2 Assignment: Zombie Detection

Welcome to this week's programming assignment! You will use the Object Detection API and
retrain RetinaNet to spot Zombies using just 5 training images. You will setup the model to restore
pretrained weights and fine tune the classification layers.

Important: This colab notebook has read-only access so you won't be able to save your changes. If
you want to save your work periodically, please click File -> Save a Copy in Drive to create a
copy in your account, then work from there.

Exercises
Exercise 1 - Import Object Detection API packages
Exercise 2 - Visualize the training images
Exercise 3 - Define the category index dictionary
Exercise 4 - Download checkpoints
 Exercise 5.1 - Locate and read from the configuration file
Exercise 5.2 - Modify the model configuration
Exercise 5.3 - Modify model_config
Exercise 5.4 - Build the custom model
Exercise 6.1 - Define Checkpoints for the box predictor
Exercise 6.2 - Define the temporary model checkpoint
Exercise 6.3 - Restore the checkpoint
Exercise 7 - Run a dummy image to generate the model variables
Exercise 8 - Set training hyperparameters
Exercise 9 - Select the prediction layer variables
Exercise 10 - Define the training step
Exercise 11 - Preprocess, predict, and post process an image

Installation

You'll start by installing the Tensorflow 2 Object Detection API.

# uncomment the next line if you want to delete an existing models directory
!rm -rf ./models/

# clone the Tensorflow Model Garden

!git clone --depth 1 https://github.com/tensorflow/models/

# Compile the Object Detection API protocol buffers and install the necessary packages
!cd models/research/ && protoc object_detection/protos/*.proto --python_out=. && cp object

Imports
Let's now import the packages you will use in this assignment.

import matplotlib
import matplotlib.pyplot as plt

import os
import random
import zipfile
import io
import scipy.misc
import numpy as np

import glob
import imageio
from six import BytesIO
from PIL import Image, ImageDraw, ImageFont
from IPython.display import display, Javascript
from IPython.display import Image as IPyImage

try:
# %tensorflow_version only exists in Colab.
%tensorflow_version 2.x
except Exception:
pass

import tensorflow as tf
tf.get_logger().setLevel('ERROR')

Exercise 1: Import Object Detection API packages

Import the necessary modules from the object_detection package.

From the utils package:

label_map_util
config_util: You'll use this to read model configurations from a .config file and then
modify that configuration
visualization_utils: please give this the alias viz_utils , as this is what will be used in
some visualization code that is given to you later.
colab_utils

From the builders package:

model_builder: This builds your model according to the model configuration that you'll
specify.

### START CODE HERE (Replace Instances of `None` with your code) ###
# import the label map utility module
from None import None

# import module for reading and updating configuration files.

from None import None

# import module for visualization. use the alias `viz_utils`

from None import None as None

# import module for building the detection model

from None import None
### END CODE HERE ###

# import module for utilities in Colab

from object_detection.utils import colab_utils

Utilities
 You'll define a couple of utility functions for loading images and plotting detections. This code is
provided for you.

def load_image_into_numpy_array(path):
"""Load an image from file into a numpy array.

Puts image into numpy array to feed into tensorflow graph.

Note that by convention we put it into a numpy array with shape
(height, width, channels), where channels=3 for RGB.

Args:
path: a file path.

Returns:
uint8 numpy array with shape (img_height, img_width, 3)
"""

img_data = tf.io.gfile.GFile(path, 'rb').read()

image = Image.open(BytesIO(img_data))
(im_width, im_height) = image.size

return np.array(image.getdata()).reshape(
(im_height, im_width, 3)).astype(np.uint8)

def plot_detections(image_np,
boxes,
classes,
scores,
category_index,
figsize=(12, 16),
image_name=None):
"""Wrapper function to visualize detections.

Args:
image_np: uint8 numpy array with shape (img_height, img_width, 3)
boxes: a numpy array of shape [N, 4]
classes: a numpy array of shape [N]. Note that class indices are 1-based,
and match the keys in the label map.
scores: a numpy array of shape [N] or None. If scores=None, then
this function assumes that the boxes to be plotted are groundtruth
boxes and plot all boxes as black with no classes or scores.
category_index: a dict containing category dictionaries (each holding
category index `id` and category name `name`) keyed by category indices.
figsize: size for the figure.
image_name: a name for the image file.
"""

image_np_with_annotations = image_np.copy()

viz_utils.visualize_boxes_and_labels_on_image_array(
image_np_with_annotations,
boxes,
classes,
scores,
category_index,
use_normalized_coordinates=True,
min_score_thresh=0.8)
 if image_name:
plt.imsave(image_name, image_np_with_annotations)

else:
plt.imshow(image_np_with_annotations)

Download the Zombie data

Now you will get 5 images of zombies that you will use for training.

The zombies are hosted in a Google bucket.

You can download and unzip the images into a local training/ directory by running the cell
below.

# uncomment the next 2 lines if you want to delete an existing zip and training directory
# !rm training-zombie.zip
# !rm -rf ./training

# download the images

!wget --no-check-certificate \
https://storage.googleapis.com/tensorflow-3-public/datasets/training-zombie.zip \
-O ./training-zombie.zip

# unzip to a local directory

local_zip = './training-zombie.zip'
zip_ref = zipfile.ZipFile(local_zip, 'r')
zip_ref.extractall('./training')
zip_ref.close()

Exercise 2: Visualize the training images

Next, you'll want to inspect the images that you just downloaded.

Please replace instances of None below to load and visualize the 5 training images.
You can inspect the training directory (using the Files button on the left side of this Colab)
to see the filenames of the zombie images. The paths for the images will look like this:

./training/training-zombie1.jpg
./training/training-zombie2.jpg
./training/training-zombie3.jpg
./training/training-zombie4.jpg
./training/training-zombie5.jpg

To set file paths, you'll use os.path.join. As an example, if you wanted to create the path
'./parent_folder/file_name1.txt', you could write:

os.path.join('parent_folder', 'file_name' + str(1) + '.txt')

 You should see the 5 training images after running this cell. If not, please inspect your code,
particularly the image_path .

%matplotlib inline

### START CODE HERE (Replace Instances of `None` with your code) ###

# assign the name (string) of the directory containing the training images
train_image_dir = './training'

# declare an empty list

train_images_np = None

# run a for loop for each image

for i in range(1, 6):

# define the path (string) for each image

image_path = os.path.join(None)
print(image_path)

# load images into numpy arrays and append to a list

train_images_np.append(load_image_into_numpy_array(image_path))
### END CODE HERE ###

# configure plot settings via rcParams

plt.rcParams['axes.grid'] = False
plt.rcParams['xtick.labelsize'] = False
plt.rcParams['ytick.labelsize'] = False
plt.rcParams['xtick.top'] = False
plt.rcParams['xtick.bottom'] = False
plt.rcParams['ytick.left'] = False
plt.rcParams['ytick.right'] = False
plt.rcParams['figure.figsize'] = [14, 7]

# plot images
for idx, train_image_np in enumerate(train_images_np):
plt.subplot(1, 5, idx+1)
plt.imshow(train_image_np)

plt.show()

Prepare data for training (Optional)

In this section, you will create your ground truth boxes. You can either draw your own boxes or use
a prepopulated list of coordinates that we have provided below.

# Define the list of ground truth boxes

gt_boxes = []

Option 1: draw your own ground truth boxes

 If you want to draw your own, please run the next cell and the following test code. If not, then skip
these optional cells.

Draw a box around the zombie in each image.

Click the next image button to go to the next image

Click submit when it says "All images completed!!".

Make sure to not make the bounding box too big.

If the box is too big, the model might learn the features of the background (e.g. door,
road, etc) in determining if there is a zombie or not.

Include the entire zombie inside the box.

As an example, scroll to the beginning of this notebook to look at the bounding box around
the zombie.

# Option 1: draw your own ground truth boxes

# annotate the training images

colab_utils.annotate(train_images_np, box_storage_pointer=gt_boxes)

# Option 1: draw your own ground truth boxes

# TEST CODE:
try:
assert(len(gt_boxes) == 5), "Warning: gt_boxes is empty. Did you click `submit`?"

except AssertionError as e:
print(e)

# checks if there are boxes for all 5 images

for gt_box in gt_boxes:
try:
assert(gt_box is not None), "There are less than 5 sets of box coordinates. " \
"Please re-run the cell above to draw the boxes again.\n
"Alternatively, you can run the next cell to load pre-de
"ground truth boxes."

except AssertionError as e:
print(e)
break

ref_gt_boxes = [
np.array([[0.27333333, 0.41500586, 0.74333333, 0.57678781]]),
np.array([[0.29833333, 0.45955451, 0.75666667, 0.61078546]]),
np.array([[0.40833333, 0.18288394, 0.945, 0.34818288]]),
np.array([[0.16166667, 0.61899179, 0.8, 0.91910903]]),
np.array([[0.28833333, 0.12543962, 0.835, 0.35052755]]),
]

for gt_box, ref_gt_box in zip(gt_boxes, ref_gt_boxes):

try:
assert(np.allclose(gt_box, ref_gt_box, atol=0.04)), "One of the boxes is too big or
 "Please re-draw and make the box

except AssertionError as e:
print(e)
break

Option 2: use the given ground truth boxes

You can also use this list if you opt not to draw the boxes yourself.

# Option 2: use given ground truth boxes

# set this to `True` if you want to override the boxes you drew
override = False

# bounding boxes for each of the 5 zombies found in each image.

# you can use these instead of drawing the boxes yourself.
ref_gt_boxes = [
np.array([[0.27333333, 0.41500586, 0.74333333, 0.57678781]]),
np.array([[0.29833333, 0.45955451, 0.75666667, 0.61078546]]),
np.array([[0.40833333, 0.18288394, 0.945, 0.34818288]]),
np.array([[0.16166667, 0.61899179, 0.8, 0.91910903]]),
np.array([[0.28833333, 0.12543962, 0.835, 0.35052755]]),
]

# if gt_boxes is empty, use the reference

if not gt_boxes or override is True:
gt_boxes = ref_gt_boxes

# if gt_boxes does not contain 5 box coordinates, use the reference

for gt_box in gt_boxes:
try:
assert(gt_box is not None)

except:
gt_boxes = ref_gt_boxes

break

View your ground truth box coordinates

Whether you chose to draw your own or use the given boxes, please check your list of ground truth
box coordinates.

# print the coordinates of your ground truth boxes

for gt_box in gt_boxes:
print(gt_box)

Below, we add the class annotations. For simplicity, we assume just a single class, though it
should be straightforward to extend this to handle multiple classes. We will also convert
 everything to the format that the training loop expects (e.g., conversion to tensors, one-hot
representations, etc.).

Exercise 3: Define the category index dictionary

You'll need to tell the model which integer class ID to assign to the 'zombie' category, and what
'name' to associate with that integer id.

zombie_class_id: By convention, class ID integers start numbering from 1,2,3, onward.

If there is ever a 'background' class, it could be assigned the integer 0, but in this case,
you're just predicting the one zombie class.
Since you are just predicting one class (zombie), please assign 1 to the zombie class
ID.

category_index: Please define the category_index dictionary, which will have the same
structure as this:

{human_class_id :
{'id' : human_class_id,
'name': 'human_so_far'}
}

Define category_index similar to the example dictionary above, except for zombies.
This will be used by the succeeding functions to know the class id and name of
zombie images.

num_classes: Since you are predicting one class, please assign 1 to the number of classes
that the model will predict.

This will be used during data preprocessing and again when you configure the model.

### START CODE HERE (Replace instances of `None` with your code ###

# Assign the zombie class ID

zombie_class_id = None

# define a dictionary describing the zombie class

category_index = None

# Specify the number of classes that the model will predict

num_classes = None
### END CODE HERE ###

# TEST CODE:
print(category_index[zombie_class_id])

Expected Output:

{'id': 1, 'name': 'zombie'}

Data preprocessing
You will now do some data preprocessing so it is formatted properly before it is fed to the model:

Convert the class labels to one-hot representations

convert everything (i.e. train images, gt boxes and class labels) to tensors.

This code is provided for you.

# The `label_id_offset` here shifts all classes by a certain number of indices;

# we do this here so that the model receives one-hot labels where non-background
# classes start counting at the zeroth index. This is ordinarily just handled
# automatically in our training binaries, but we need to reproduce it here.

label_id_offset = 1
train_image_tensors = []

# lists containing the one-hot encoded classes and ground truth boxes
gt_classes_one_hot_tensors = []
gt_box_tensors = []

for (train_image_np, gt_box_np) in zip(train_images_np, gt_boxes):

# convert training image to tensor, add batch dimension, and add to list
train_image_tensors.append(tf.expand_dims(tf.convert_to_tensor(
train_image_np, dtype=tf.float32), axis=0))

# convert numpy array to tensor, then add to list

gt_box_tensors.append(tf.convert_to_tensor(gt_box_np, dtype=tf.float32))

# apply offset to to have zero-indexed ground truth classes

zero_indexed_groundtruth_classes = tf.convert_to_tensor(
np.ones(shape=[gt_box_np.shape[0]], dtype=np.int32) - label_id_offset)

# do one-hot encoding to ground truth classes

gt_classes_one_hot_tensors.append(tf.one_hot(
zero_indexed_groundtruth_classes, num_classes))

print('Done prepping data.')

Visualize the zombies with their ground truth bounding boxes

You should see the 5 training images with the bounding boxes after running the cell below. If not,
please re-run the annotation tool again or use the prepopulated gt_boxes array given.
# give boxes a score of 100%
dummy_scores = np.array([1.0], dtype=np.float32)

# define the figure size

plt.figure(figsize=(30, 15))

# use the `plot_detections()` utility function to draw the ground truth boxes
for idx in range(5):
plt.subplot(2, 4, idx+1)
plot_detections(
train_images_np[idx],
gt_boxes[idx],
np.ones(shape=[gt_boxes[idx].shape[0]], dtype=np.int32),
dummy_scores, category_index)

plt.show()

Download the checkpoint containing the pre-trained weights

Next, you will download RetinaNet and copy it inside the object detection directory.

When working with models that are at the frontiers of research, the models and checkpoints may
not yet be organized in a central location like the TensorFlow Garden
(https://github.com/tensorflow/models).

You'll often read a blog post from the researchers, who will usually provide information on:

how to use the model

where to download the models and pre-trained checkpoints.

It's good practice to do some of this "detective work", so that you'll feel more comfortable when
exploring new models yourself! So please try the following steps:

Go to the TensorFlow Blog, where researchers announce new findings.

In the search box at the top of the page, search for "retinanet".
In the search results, click on the blog post titled "TensorFlow 2 meets the Object Detection
API" (it may be the first search result).
Skim through this blog and look for links to either the checkpoints or to Colabs that will show
you how to use the checkpoints.
Try to fill out the following code cell below, which does the following:

Download the compressed SSD Resnet 50 version 1, 640 x 640 checkpoint.

Untar (decompress) the tar file
Move the decompressed checkpoint to
models/research/object_detection/test_data/

If you want some help getting started, please click on the "Initial Hints" cell to get some hints.
 Initial Hints

More Hints

Even More Hints

Exercise 4: Download checkpoints

Download the compressed SSD Resnet 50 version 1, 640 x 640 checkpoint.
Untar (decompress) the tar file
Move the decompressed checkpoint to models/research/object_detection/test_data/

### START CODE HERE ###

# Download the SSD Resnet 50 version 1, 640x640 checkpoint
None

# untar (decompress) the tar file

None

# copy the checkpoint to the test_data folder models/research/object_detection/test_data/

None

### END CODE HERE

Configure the model

Here, you will configure the model for this use case.

Exercise 5.1: Locate and read from the configuration file

pipeline_config

In the Colab, on the left side table of contents, click on the folder icon to display the file
browser for the current workspace.
Navigate to models/research/object_detection/configs/tf2 . The folder has multiple
.config files.
Look for the file corresponding to ssd resnet 50 version 1 640x640.
You can double-click the config file to view its contents. This may help you as you complete
the next few code cells to configure your model.
Set the pipeline_config to a string that contains the full path to the resnet config file, in
other words: models/research/.../... .config
 configs

If you look at the module config_util that you imported, it contains the following function:

def get_configs_from_pipeline_file(pipeline_config_path, config_override=None):

Please use this function to load the configuration from your pipeline_config .

configs will now contain a dictionary.

tf.keras.backend.clear_session()

### START CODE HERE ###

# define the path to the .config file for ssd resnet 50 v1 640x640
pipeline_config = None

# Load the configuration file into a dictionary

configs = None

### END CODE HERE ###

# See what configs looks like
configs

Exercise 5.2: Get the model configuration

model_config

From the configs dictionary, access the object associated with the key 'model'.
model_config now contains an object of type
object_detection.protos.model_pb2.DetectionModel .
If you print model_config , you'll see something like this:

ssd {
num_classes: 90
image_resizer {
fixed_shape_resizer {
height: 640
width: 640
}
}
feature_extractor {
...
...
freeze_batchnorm: false
### START CODE HERE ###
# Read in the object stored at the key 'model' of the configs dictionary
model_config = None

### END CODE HERE

# see what model_config looks like
model_config

Exercise 5.3: Modify model_config

Modify num_classes from the default 90 to the num_classes that you set earlier in this
notebook.

num_classes is nested under ssd. You'll need to use dot notation 'obj.x' and NOT
bracket notation obj['x']` to access num_classes.
Freeze batch normalization

Batch normalization is not frozen in the default configuration.

If you inspect the model_config object, you'll see that freeze_batchnorm is nested
under ssd just like num_classes .
Freeze batch normalization by setting the relevant field to True .

### START CODE HERE ###

# Modify the number of classes from its default of 90
None = None

# Freeze batch normalization

None = None

### END CODE HERE

# See what model_config now looks like after you've customized it!
model_config

Build the model

Recall that you imported model_builder.

You'll use model_builder to build the model according to the configurations that you have
just downloaded and customized.

Exercise 5.4: Build the custom model

model_builder
model_builder has a function build :
 def build(model_config, is_training, add_summaries=True):

model_config: Set this to the model configuration that you just customized.
is_training: Set this to True.
You can keep the default value for the remaining parameter.
Note that it will take some time to build the model.

### START CODE HERE (Replace instances of `None` with your code) ###
detection_model = None
### END CODE HERE ###

print(type(detection_model))

Expected Output:

<class 'object_detection.meta_architectures.ssd_meta_arch.SSDMetaArch'>

Restore weights from your checkpoint

Now, you will selectively restore weights from your checkpoint.

Your end goal is to create a custom model which reuses parts of, but not all of the layers of
RetinaNet (currently stored in the variable detection_model .)

The parts of RetinaNet that you want to reuse are:

Feature extraction layers

Bounding box regression prediction layer
The part of RetinaNet that you will not want to reuse is the classification prediction
layer (since you will define and train your own classification layer specific to zombies).
For the parts of RetinaNet that you want to reuse, you will also restore the weights from
the checkpoint that you selected.

Inspect the detection_model

First, take a look at the type of the detection_model and its Python class.

# Run this to check the type of detection_model

detection_model

Find the source code for detection_model

 You'll see that the type of the model is
object_detection.meta_architectures.ssd_meta_arch.SSDMetaArch . Please practice some
detective work and open up the source code for this class in GitHub repository. Recall that at the
start of this assignment, you cloned from this repository: TensorFlow Models.

Navigate through these subfolders: models -> research -> object_detection.

If you get stuck, go to this link: object_detection

Take a look at this 'object_detection' folder and look for the remaining folders to navigate
based on the class type of detection_model:
object_detection.meta_architectures.ssd_meta_arch.SSDMetaArch

Hopefully you'll find the meta_architectures folder, and within it you'll notice a file
named ssd_meta_arch.py .
Please open and view this ssd_meta_arch.py file.

View the variables in detection_model

Now, check the class variables that are in detection_model .

vars(detection_model)

You'll see that detection_model contains several variables:

Two of these will be relevant to you:

...
_box_predictor': <object_detection.predictors.convolutional_keras_box_predictor.WeightShared
...
_feature_extractor': <object_detection.models.ssd_resnet_v1_fpn_keras_feature_extractor.SSDR

Inspect _feature_extractor

Take a look at the ssd_meta_arch.py code.

# Line 302
feature_extractor: a SSDFeatureExtractor object.

Also

# Line 380
 self._feature_extractor = feature_extractor

So detection_model._feature_extractor is a feature extractor, which you will want to reuse

for your zombie detector model.

Inspect _box_predictor

View the ssd_meta_arch.py file (which is the source code for detection_model)
Notice that in the init constructor for class SSDMetaArch(model.DetectionModel),

...
box_predictor: a box_predictor.BoxPredictor object
...
self._box_predictor = box_predictor

Inspect _box_predictor

Please take a look at the class type of detection_model._box_predictor

# view the type of _box_predictor

detection_model._box_predictor

You'll see that the class type of _box_predictor is

object_detection.predictors.convolutional_keras_box_predictor.WeightSharedConvolutionalBoxPr

You can navigate through the GitHub repository to this path:

objection_detection/predictors
Notice that there is a file named convolutional_keras_box_predictor.py. Please open that file.

View variables in _box_predictor

Also view the variables contained in _box_predictor:

vars(detection_model._box_predictor)

Among the variables listed, a few will be relevant to you:

...
_base_tower_layers_for_heads
 ...
_box_prediction_head
...
_prediction_heads

In the source code for convolutional_keras_box_predictor.py that you just opened, look at the
source code to get a sense for what these three variables represent.

Inspect base_tower_layers_for_heads

If you look at the convolutional_keras_box_predictor.py file, you'll notice this:

# line 302
self._base_tower_layers_for_heads = {
BOX_ENCODINGS: [],
CLASS_PREDICTIONS_WITH_BACKGROUND: [],
}

base_tower_layers_for_heads is a dictionary with two key-value pairs.

BOX_ENCODINGS : points to a list of layers

CLASS_PREDICTIONS_WITH_BACKGROUND : points to a list of layers
If you scan the code, you'll see that for both of these, the lists are filled with all layers
that appear BEFORE the prediction layer.

# Line 377
# Stack the base_tower_layers in the order of conv_layer, batch_norm_la
# and activation_layer
base_tower_layers = []
for i in range(self._num_layers_before_predictor):

So detection_model.box_predictor._base_tower_layers_for_heads contains:

The layers for the prediction before the final bounding box prediction
The layers for the prediction before the final class prediction.

You will want to use these in your model.

Inspect _box_prediction_head

If you again look at convolutional_keras_box_predictor.py file, you'll see this

 # Line 248
box_prediction_head: The head that predicts the boxes.

So detection_model.box_predictor._box_prediction_head points to the bounding box

prediction layer, which you'll want to use for your model.

Inspect _prediction_heads

If you again look at convolutional_keras_box_predictor.py file, you'll see this

# Line 121
self._prediction_heads = {
BOX_ENCODINGS: box_prediction_heads,
CLASS_PREDICTIONS_WITH_BACKGROUND: class_prediction_heads,
}

You'll also see this docstring

# Line 83
class_prediction_heads: A list of heads that predict the classes.

So detection_model.box_predictor._prediction_heads is a dictionary that points to both

prediction layers:

The layer that predicts the bounding boxes

The layer that predicts the class (category).

Which layers will you reuse?

Remember that you are reusing the model for its feature extraction and bounding box detection.

You will create your own classification layer and train it on zombie images.
So you won't need to reuse the class prediction layer of detection_model .

Define checkpoints for desired layers

You will now isolate the layers of detection_model that you wish to reuse so that you can restore
the weights to just those layers.

First, define checkpoints for the box predictor

Next, define checkpoints for the model, which will point to this box predictor checkpoint as
well as the feature extraction layers.
Please use tf.train.Checkpoint.

As a reminder of how to use tf.train.Checkpoint:

tf.train.Checkpoint(
**kwargs
)

Pretend that detection_model contains these variables for which you want to restore weights:

detection_model._ice_cream_sundae
detection_model._pies._apple_pie
detection_model._pies._pecan_pie

Notice that the pies are nested within ._pies .

If you just want the ice cream sundae and apple pie variables (and not the pecan pie) then you can
do the following:

tmp_pies_checkpoint = tf.train.Checkpoint(
_apple_pie = detection_model._pies._apple_pie
)

Next, in order to connect these together in a node graph, do this:

tmp_model_checkpoint = tf.train.Checkpoint(
_pies = tmp_pies_checkpoint,
_ice_cream_sundae = detection_model._ice_cream_sundae
)

Finally, define a checkpoint that uses the key model and takes in the tmp_model_checkpoint.

checkpoint = tf.train.Checkpoint(
model = tmp_model_checkpoint
)

You'll then be ready to restore the weights from the checkpoint that you downloaded.

Try this out step by step!

Exercise 6.1: Define Checkpoints for the box predictor

 Please define box_predictor_checkpoint to be checkpoint for these two layers of the
detection_model 's box predictor:

The base tower layer (the layers the precede both the class prediction and bounding
box prediction layers).
The box prediction head (the prediction layer for bounding boxes).
Note, you won't include the class prediction layer.
Important: Be careful to avoid typos in the key names for the checkpoint. For example, if
there is a layer called _apple_pies and you accidentally added an extra "t" like this:
tmp_pies_checkpoint = tf.train.Checkpoint( _apple_piest =
detection_model._box_predictor._apple_pies ) then, when you restore the
checkpoint, it will update the variable _apple_piest , instead of _apple_pies like you
intended. This will likely make the model train slower in Exercise 10 later.

### START CODE HERE ###

tmp_box_predictor_checkpoint = None

### END CODE HERE

# Check the datatype of this checkpoint

type(tmp_box_predictor_checkpoint)

# Expected output:
# tensorflow.python.training.tracking.util.Checkpoint

# Check the variables of this checkpoint

vars(tmp_box_predictor_checkpoint)

Expected output

You should expect to see a list of variables that include the following:

...
'_base_tower_layers_for_heads': {'box_encodings': ListWrapper([]),
'class_predictions_with_background': ListWrapper([])},
'_box_prediction_head': <object_detection.predictors.heads.keras_box_head.WeightSharedConvo
...

Exercise 6.2: Define the temporary model checkpoint**

 Now define tmp_model_checkpoint so that it points to these two layers:

The feature extractor of the detection model.

The temporary box predictor checkpoint that you just defined.

### START CODE HERE ###

tmp_model_checkpoint = None

### END CODE HERE ###

# Check the datatype of this checkpoint

type(tmp_model_checkpoint)

# Expected output
# tensorflow.python.training.tracking.util.Checkpoint

# Check the vars of this checkpoint

vars(tmp_model_checkpoint)

Expected output

Among the variables of this checkpoint, you should see:

'_box_predictor': <tensorflow.python.training.tracking.util.Checkpoint at 0x7fefac044a20>,

'_feature_extractor': <object_detection.models.ssd_resnet_v1_fpn_keras_feature_extractor.SS

Exercise 6.3: Restore the checkpoint

You can now restore the checkpoint.

First, find and set the checkpoint_path

checkpoint_path:

Using the "files" browser in the left side of Colab, navigate to models -> research ->
object_detection -> test_data .
If you completed the previous code cell that downloads and moves the checkpoint,
you'll see a subfolder named "checkpoint".

The 'checkpoint' folder contains three files:

checkpoint
ckpt-0.data-00000-of-00001
ckpt-0.index
 Please set checkpoint_path to the path to the full path models/.../ckpt-0

Notice that you don't want to include a file extension after ckpt-0 .
IMPORTANT: Please don't set the path to include the .index extension in the
checkpoint file name.

If you do set it to ckpt-0.index , there won't be any immediate error

message, but later during training, you'll notice that your model's loss
doesn't improve, which means that the pre-trained weights were not
restored properly.

Next, define one last checkpoint using tf.train.Checkpoint() .

For the single keyword argument,

Set the key as model=

Set the value to your temporary model checkpoint that you just defined.
IMPORTANT: You'll need to set the keyword argument as model= and not something else
like detection_model= .
If you set this keyword argument to anything else, it won't show an immmediate error, but
when you train your model on the zombie images, your model loss will not decrease (your
model will not learn).

Finally, call this checkpoint's .restore() function, passing in the path to the checkpoint.

### START CODE HERE ###

checkpoint_path = None

# Define a checkpoint that sets `model` to the temporary model checkpoint

checkpoint = None

# Restore the checkpoint to the checkpoint path

None

### END CODE HERE ###

Exercise 7: Run a dummy image to generate the model variables

Run a dummy image through the model so that variables are created. We need to select the
trainable variables later in Exercise 9 and right now, it is still empty. Try running
len(detection_model.trainable_variables) in a code cell and you will get 0 . We will pass in
a dummy image through the forward pass to create these variables.

Recall that detection_model is an object of type

object_detection.meta_architectures.ssd_meta_arch.SSDMetaArch
 Important methods that are available in the detection_model object are:

preprocess():

takes in a tensor representing an image and returns

returns image, shapes
For the dummy image, you can declare a tensor of zeros that has a shape that the
preprocess() method can accept (i.e. [batch, height, width, channels]).
Remember that your images have dimensions 640 x 640 x 3.
You can pass in a batch of 1 when making the dummy image.

predict()

takes in image, shapes which are created by the preprocess() function call.
returns a prediction in a Python dictionary
this will pass the dummy image through the forward pass of the network and create
the model variables

postprocess()

Takes in the prediction_dict and shapes

returns a dictionary of post-processed predictions of detected objects ("detections").

Note: Please use the recommended variable names, which include the prefix tmp_ , since these
variables won't be used later, but you'll define similarly-named variables later for predicting on
actual zombie images.

### START CODE HERE (Replace instances of `None` with your code)###

# use the detection model's `preprocess()` method and pass a dummy image
tmp_image, tmp_shapes = None

# run a prediction with the preprocessed image and shapes

tmp_prediction_dict = None

# postprocess the predictions into final detections

tmp_detections = None

### END CODE HERE ###

print('Weights restored!')

# Test Code:
assert len(detection_model.trainable_variables) > 0, "Please pass in a dummy image to crea

print(detection_model.weights[0].shape)
print(detection_model.weights[231].shape)
print(detection_model.weights[462].shape)

Expected Output:
 (3, 3, 256, 24)
(512,)
(256,)

Eager mode custom training loop

With the data and model now setup, you can now proceed to configure the training.

Exercise 8: Set training hyperparameters

Set an appropriate learning rate and optimizer for the training.

batch_size: you can use 4

You can increase the batch size up to 5, since you have just 5 images for training.
num_batches: You can use 100

You can increase the number of batches but the training will take longer to complete.

learning_rate: You can use 0.01

When you run the training loop later, notice how the initial loss INCREASES` before
decreasing.
You can try a lower learning rate to see if you can avoid this increased loss.
optimizer: you can use tf.keras.optimizers.SGD

Set the learning rate

Set the momentum to 0.9

Training will be fairly quick, so we do encourage you to experiment a bit with these
hyperparameters!

tf.keras.backend.set_learning_phase(True)

### START CODE HERE (Replace instances of `None` with your code)###

# set the batch_size

batch_size = None

# set the number of batches

num_batches = None

# Set the learning rate

learning_rate = None

# set the optimizer and pass in the learning_rate

optimizer = None

### END CODE HERE ###

Choose the layers to fine-tune

To make use of transfer learning and pre-trained weights, you will train just certain parts of the
detection model, namely, the last prediction layers.

Please take a minute to inspect the layers of detection_model .

# Inspect the layers of detection_model

for i,v in enumerate(detection_model.trainable_variables):
print(f"i: {i} \t name: {v.name} \t shape:{v.shape} \t dtype={v.dtype}")

Notice that there are some layers whose names are prefixed with the following:

WeightSharedConvolutionalBoxPredictor/WeightSharedConvolutionalBoxHead
...
WeightSharedConvolutionalBoxPredictor/WeightSharedConvolutionalClassHead
...
WeightSharedConvolutionalBoxPredictor/BoxPredictionTower
...
WeightSharedConvolutionalBoxPredictor/ClassPredictionTower
...

Among these, which do you think are the prediction layers at the "end" of the model?

Recall that when inspecting the source code to restore the checkpoints
(convolutional_keras_box_predictor.py) you noticed that:

_base_tower_layers_for_heads : refers to the layers that are placed right before the
prediction layer
_box_prediction_head refers to the prediction layer for the bounding boxes
_prediction_heads : refers to the set of prediction layers (both for classification and
for bounding boxes)

So you can see that in the source code for this model, "tower" refers to layers that are before the
prediction layer, and "head" refers to the prediction layers.

Exercise 9: Select the prediction layer variables

 Based on inspecting the detection_model.trainable_variables , please select the prediction
layer variables that you will fine tune:

The bounding box head variables (which predict bounding box coordinates)
The class head variables (which predict the class/category)

You have a few options for doing this:

You can access them by their list index:

detection_model.trainable_variables[92]

Alternatively, you can use string matching to select the variables:

tmp_list = []
for v in detection_model.trainable_variables:
if v.name.startswith('ResNet50V1_FPN/bottom_up_block5'):
tmp_list.append(v)

Hint: There are a total of four variables that you want to fine tune.

### START CODE HERE (Replace instances of `None` with your code) ###

# define a list that contains the layers that you wish to fine tune
to_fine_tune = None

### END CODE HERE

# Test Code:

print(to_fine_tune[0].name)
print(to_fine_tune[2].name)

Expected Output:

WeightSharedConvolutionalBoxPredictor/WeightSharedConvolutionalBoxHead/BoxPredictor/kernel:0
WeightSharedConvolutionalBoxPredictor/WeightSharedConvolutionalClassHead/ClassPredictor/kern
 Train your model
You'll define a function that handles training for one batch, which you'll later use in your training
loop.

First, walk through these code cells to learn how you'll perform training using this model.

# Get a batch of your training images

g_images_list = train_image_tensors[0:2]

The detection_model is of class SSDMetaArch, and its source code shows that is has this
function preprocess.

This preprocesses the images so that they can be passed into the model (for training or
prediction):

def preprocess(self, inputs):

"""Feature-extractor specific preprocessing.
...
Args:
inputs: a [batch, height_in, width_in, channels] float tensor representi
a batch of images with values between 0 and 255.0.
Returns:
preprocessed_inputs: a [batch, height_out, width_out, channels] float
tensor representing a batch of images.

true_image_shapes: int32 tensor of shape [batch, 3] where each row is

of the form [height, width, channels] indicating the shapes
of true images in the resized images, as resized images can be padded
with zeros.

# Use .preprocess to preprocess an image

g_preprocessed_image = detection_model.preprocess(g_images_list[0])
print(f"g_preprocessed_image type: {type(g_preprocessed_image)}")
print(f"g_preprocessed_image length: {len(g_preprocessed_image)}")
print(f"index 0 has the preprocessed image of shape {g_preprocessed_image[0].shape}")
print(f"index 1 has information about the image's true shape excluding padding: {g_preproc

You can pre-process each image and save their outputs into two separate lists

One list of the preprocessed images

 One list of the true shape for each preprocessed image

preprocessed_image_list = []
true_shape_list = []

for img in g_images_list:

processed_img, true_shape = detection_model.preprocess(img)
preprocessed_image_list.append(processed_img)
true_shape_list.append(true_shape)

print(f"preprocessed_image_list is of type {type(preprocessed_image_list)}")

print(f"preprocessed_image_list has length {len(preprocessed_image_list)}")
print()
print(f"true_shape_list is of type {type(true_shape_list)}")
print(f"true_shape_list has length {len(true_shape_list)}")

Make a prediction
The detection_model also has a .predict function. According to the source code for predict

def predict(self, preprocessed_inputs, true_image_shapes):

"""Predicts unpostprocessed tensors from input tensor.
This function takes an input batch of images and runs it through the forward
pass of the network to yield unpostprocessesed predictions.
...
Args:
preprocessed_inputs: a [batch, height, width, channels] image tensor.

true_image_shapes: int32 tensor of shape [batch, 3] where each row is

of the form [height, width, channels] indicating the shapes
of true images in the resized images, as resized images can be padded
with zeros.

Returns:
prediction_dict: a dictionary holding "raw" prediction tensors:
1) preprocessed_inputs: the [batch, height, width, channels] image
tensor.
2) box_encodings: 4-D float tensor of shape [batch_size, num_anchors,
box_code_dimension] containing predicted boxes.
3) class_predictions_with_background: 3-D float tensor of shape
[batch_size, num_anchors, num_classes+1] containing class predictions
(logits) for each of the anchors. Note that this tensor *includes*
background class predictions (at class index 0).
4) feature_maps: a list of tensors where the ith tensor has shape
[batch, height_i, width_i, depth_i].
5) anchors: 2-D float tensor of shape [num_anchors, 4] containing
 the generated anchors in normalized coordinates.
6) final_anchors: 3-D float tensor of shape [batch_size, num_anchors, 4]
containing the generated anchors in normalized coordinates.
If self._return_raw_detections_during_predict is True, the dictionary
will also contain:
7) raw_detection_boxes: a 4-D float32 tensor with shape
[batch_size, self.max_num_proposals, 4] in normalized coordinates.
8) raw_detection_feature_map_indices: a 3-D int32 tensor with shape
[batch_size, self.max_num_proposals].
"""

Notice that .predict takes its inputs as tensors. If you tried to pass in the preprocessed images
and true shapes, you'll get an error.

# Try to call `predict` and pass in lists; look at the error message
try:
detection_model.predict(preprocessed_image_list, true_shape_list)
except AttributeError as e:
print("Error message:", e)

But don't worry! You can check how to properly use predict :

Notice that the source code documentation says that preprocessed_inputs and
true_image_shapes are expected to be tensors and not lists of tensors.
One way to turn a list of tensors into a tensor is to use tf.concat

tf.concat(
values, axis, name='concat'
)

# Turn a list of tensors into a tensor

preprocessed_image_tensor = tf.concat(preprocessed_image_list, axis=0)
true_shape_tensor = tf.concat(true_shape_list, axis=0)

print(f"preprocessed_image_tensor shape: {preprocessed_image_tensor.shape}")

print(f"true_shape_tensor shape: {true_shape_tensor.shape}")

Now you can make predictions for the images. According to the source code, predict returns a
dictionary containing the prediction information, including:

The bounding box predictions

The class predictions
# Make predictions on the images
prediction_dict = detection_model.predict(preprocessed_image_tensor, true_shape_tensor)

print("keys in prediction_dict:")
for key in prediction_dict.keys():
print(key)

Calculate loss
Now that your model has made its prediction, you want to compare it to the ground truth in order
to calculate a loss.

The detection_model has a loss function.

def loss(self, prediction_dict, true_image_shapes, scope=None):

"""Compute scalar loss tensors with respect to provided groundtruth.
Calling this function requires that groundtruth tensors have been
provided via the provide_groundtruth function.
Args:
prediction_dict: a dictionary holding prediction tensors with
1) box_encodings: 3-D float tensor of shape [batch_size, num_anchors,
box_code_dimension] containing predicted boxes.
2) class_predictions_with_background: 3-D float tensor of shape
[batch_size, num_anchors, num_classes+1] containing class predictions
(logits) for each of the anchors. Note that this tensor *includes*
background class predictions.
true_image_shapes: int32 tensor of shape [batch, 3] where each row is
of the form [height, width, channels] indicating the shapes
of true images in the resized images, as resized images can be padded
with zeros.
scope: Optional scope name.
Returns:
a dictionary mapping loss keys (`localization_loss` and
`classification_loss`) to scalar tensors representing corresponding loss
values.
"""

It takes in:

The prediction dictionary that comes from your call to .predict() .

the true images shape that comes from your call to .preprocess() followed by the
conversion from a list to a tensor.

Try calling .loss . You'll see an error message that you'll addres in order to run the .loss
function.
try:
losses_dict = detection_model.loss(prediction_dict, true_shape_tensor)
except RuntimeError as e:
print(e)

This is giving an error about groundtruth_classes_list:

The graph tensor has name: groundtruth_classes_list:0

Notice in the docstring for loss (shown above), it says:

Calling this function requires that groundtruth tensors have been

provided via the provide_groundtruth function.

So you'll first want to set the ground truth (true labels and true bounding boxes) before you
calculate the loss.

This makes sense, since the loss is comparing the prediction to the ground truth, and so the
loss function needs to know the ground truth.

Provide the ground truth

The source code for providing the ground truth is located in the parent class of
SSDMetaArch , model.DetectionModel .
Here is the link to the code for provide_ground_truth

def provide_groundtruth(
self,
groundtruth_boxes_list,
groundtruth_classes_list,
... # more parameters not show here
"""
Args:
groundtruth_boxes_list: a list of 2-D tf.float32 tensors of shape
[num_boxes, 4] containing coordinates of the groundtruth boxes.
Groundtruth boxes are provided in [y_min, x_min, y_max, x_max]
format and assumed to be normalized and clipped
relative to the image window with y_min <= y_max and x_min <= x_max.
groundtruth_classes_list: a list of 2-D tf.float32 one-hot (or k-hot)
tensors of shape [num_boxes, num_classes] containing the class targets
with the 0th index assumed to map to the first non-background class.
"""
 You'll set two parameters in provide_ground_truth :

The true bounding boxes

The true classes

# Get the ground truth bounding boxes

gt_boxes_list = gt_box_tensors[0:2]

# Get the ground truth class labels

gt_classes_list = gt_classes_one_hot_tensors[0:2]

# Provide the ground truth to the model

detection_model.provide_groundtruth(
groundtruth_boxes_list=gt_boxes_list,
groundtruth_classes_list=gt_classes_list)

Now you can calculate the loss

# Calculate the loss after you've provided the ground truth

losses_dict = detection_model.loss(prediction_dict, true_shape_tensor)

# View the loss dictionary

losses_dict = detection_model.loss(prediction_dict, true_shape_tensor)
print(f"loss dictionary keys: {losses_dict.keys()}")
print(f"localization loss {losses_dict['Loss/localization_loss']:.8f}")
print(f"classification loss {losses_dict['Loss/classification_loss']:.8f}")

You can now calculate the gradient and optimize the variables that you selected to fine tune.

Use tf.GradientTape

with tf.GradientTape() as tape:

# Make the prediction

# calculate the loss

# calculate the gradient of each model variable with respect to each loss
gradients = tape.gradient([some loss], variables to fine tune)

# apply the gradients to update these model variables

optimizer.apply_gradients(zip(gradients, variables to fine tune))

# Let's just reset the model so that you can practice setting it up yourself!
detection_model.provide_groundtruth(groundtruth_boxes_list=[], groundtruth_classes_list=[]

Exercise 10: Define the training step

 Please complete the function below to set up one training step.

Preprocess the images

Make a prediction
Calculate the loss (and make sure the loss function has the ground truth to compare with the
prediction)
Calculate the total loss:

total_loss = localization_loss + classification_loss

Note: this is different than the example code that you saw above

Calculate gradients with respect to the variables you selected to train.

Optimize the model's variables

# decorate with @tf.function for faster training (remember, graph mode!)

@tf.function
def train_step_fn(image_list,
groundtruth_boxes_list,
groundtruth_classes_list,
model,
optimizer,
vars_to_fine_tune):
"""A single training iteration.

Args:
image_list: A list of [1, height, width, 3] Tensor of type tf.float32.
Note that the height and width can vary across images, as they are
reshaped within this function to be 640x640.
groundtruth_boxes_list: A list of Tensors of shape [N_i, 4] with type
tf.float32 representing groundtruth boxes for each image in the batch.
groundtruth_classes_list: A list of Tensors of shape [N_i, num_classes]
with type tf.float32 representing groundtruth boxes for each image in
the batch.

Returns:
A scalar tensor representing the total loss for the input batch.
"""

with tf.GradientTape() as tape:

### START CODE HERE (Replace instances of `None` with your code) ###

# Preprocess the images

preprocessed_image_tensor = None
true_shape_tensor = None

# Make a prediction
prediction_dict = None
 # Calculate the total loss (sum of both losses)

total_loss = None

# Calculate the gradients

gradients = None

# Optimize the model's selected variables

### END CODE HERE ###

return total_loss

Run the training loop

Run the training loop using the training step function that you just defined.

print('Start fine-tuning!', flush=True)

for idx in range(num_batches):

# Grab keys for a random subset of examples
all_keys = list(range(len(train_images_np)))
random.shuffle(all_keys)
example_keys = all_keys[:batch_size]

# Get the ground truth

gt_boxes_list = [gt_box_tensors[key] for key in example_keys]
gt_classes_list = [gt_classes_one_hot_tensors[key] for key in example_keys]

# get the images

image_tensors = [train_image_tensors[key] for key in example_keys]

# Training step (forward pass + backwards pass)

total_loss = train_step_fn(image_tensors,
gt_boxes_list,
gt_classes_list,
detection_model,
optimizer,
to_fine_tune
)

if idx % 10 == 0:
print('batch ' + str(idx) + ' of ' + str(num_batches)
+ ', loss=' + str(total_loss.numpy()), flush=True)

print('Done fine-tuning!')

Expected Output:
 Total loss should be decreasing and should be less than 1 after fine tuning. For example:

Start fine-tuning!
batch 0 of 100, loss=1.2559178
batch 10 of 100, loss=16.067217
batch 20 of 100, loss=8.094654
batch 30 of 100, loss=0.34514275
batch 40 of 100, loss=0.033170983
batch 50 of 100, loss=0.0024622646
batch 60 of 100, loss=0.00074224477
batch 70 of 100, loss=0.0006149876
batch 80 of 100, loss=0.00046916265
batch 90 of 100, loss=0.0004159231
Done fine-tuning!

Load test images and run inference with new model!

You can now test your model on a new set of images. The cell below downloads 237 images of a
walking zombie and stores them in a results/ directory.

# uncomment if you want to delete existing files

!rm zombie-walk-frames.zip
!rm -rf ./zombie-walk
!rm -rf ./results

# download test images

!wget --no-check-certificate \
https://storage.googleapis.com/tensorflow-3-public/datasets/zombie-walk-frames.zip \
-O zombie-walk-frames.zip

# unzip test images

local_zip = './zombie-walk-frames.zip'
zip_ref = zipfile.ZipFile(local_zip, 'r')
zip_ref.extractall('./results')
zip_ref.close()

You will load these images into numpy arrays to prepare it for inference.

test_image_dir = './results/'
test_images_np = []

# load images into a numpy array. this will take a few minutes to complete.
for i in range(0, 237):
image_path = os.path.join(test_image_dir, 'zombie-walk' + "{0:04}".format(i) + '.jpg')
print(image_path)
test_images_np.append(np.expand_dims(
load_image_into_numpy_array(image_path), axis=0))
 Exercise 11: Preprocess, predict, and post process an image
Define a function that returns the detection boxes, classes, and scores.

# Again, uncomment this decorator if you want to run inference eagerly

@tf.function
def detect(input_tensor):
"""Run detection on an input image.

Args:
input_tensor: A [1, height, width, 3] Tensor of type tf.float32.
Note that height and width can be anything since the image will be
immediately resized according to the needs of the model within this
function.

Returns:
A dict containing 3 Tensors (`detection_boxes`, `detection_classes`,
and `detection_scores`).
"""
preprocessed_image, shapes = detection_model.preprocess(input_tensor)
prediction_dict = detection_model.predict(preprocessed_image, shapes)

### START CODE HERE (Replace instances of `None` with your code) ###
# use the detection model's postprocess() method to get the the final detections
detections = None
### END CODE HERE ###

return detections

You can now loop through the test images and get the detection scores and bounding boxes to
overlay in the original image. We will save each result in a results dictionary and the autograder
will use this to evaluate your results.

# Note that the first frame will trigger tracing of the tf.function, which will
# take some time, after which inference should be fast.

label_id_offset = 1
results = {'boxes': [], 'scores': []}

for i in range(len(test_images_np)):
input_tensor = tf.convert_to_tensor(test_images_np[i], dtype=tf.float32)
detections = detect(input_tensor)
plot_detections(
test_images_np[i][0],
detections['detection_boxes'][0].numpy(),
detections['detection_classes'][0].numpy().astype(np.uint32)
+ label_id_offset,
detections['detection_scores'][0].numpy(),
category_index, figsize=(15, 20), image_name="./results/gif_frame_" + ('%03d' % i) +
results['boxes'].append(detections['detection_boxes'][0][0].numpy())
results['scores'].append(detections['detection_scores'][0][0].numpy())
# TEST CODE

print(len(results['boxes']))
print(results['boxes'][0].shape)
print()

# compare with expected bounding boxes

print(np.allclose(results['boxes'][0], [0.28838485, 0.06830047, 0.7213766 , 0.19833465], r
print(np.allclose(results['boxes'][5], [0.29168868, 0.07529271, 0.72504973, 0.20099735], r
print(np.allclose(results['boxes'][10], [0.29548776, 0.07994056, 0.7238164 , 0.20778716],

Expected Output: Ideally the three boolean values at the bottom should be True . But if you only
get two, you can still try submitting. This compares your resulting bounding boxes for each zombie
image to some preloaded coordinates (i.e. the hardcoded values in the test cell above). Depending
on how you annotated the training images,it's possible that some of your results differ for these
three frames but still get good results overall when all images are examined by the grader. If two or
all are False, please try annotating the images again with a tighter bounding box or use the
predefined gt_boxes list.

237
(4,)

True
True
True

You can also check if the model detects a zombie class in the images by examining the scores
key of the results dictionary. You should get higher than 88.0 here.

x = np.array(results['scores'])

# percent of frames where a zombie is detected

zombie_detected = (np.where(x > 0.9, 1, 0).sum())/237*100
print(zombie_detected)

You can also display some still frames and inspect visually. If you don't see a bounding box around
the zombie, please consider re-annotating the ground truth or use the predefined gt_boxes here

print('Frame 0')
display(IPyImage('./results/gif_frame_000.jpg'))
print()
print('Frame 5')
display(IPyImage('./results/gif_frame_005.jpg'))
print()
print('Frame 10')
display(IPyImage('./results/gif_frame_010.jpg'))

Create a zip of the zombie-walk images.

You can download this if you like to create your own animations

zipf = zipfile.ZipFile('./zombie.zip', 'w', zipfile.ZIP_DEFLATED)

filenames = glob.glob('./results/gif_frame_*.jpg')
filenames = sorted(filenames)

for filename in filenames:

zipf.write(filename)

zipf.close()

Create Zombie animation

imageio.plugins.freeimage.download()

!rm -rf ./results/zombie-anim.gif

anim_file = './zombie-anim.gif'

filenames = glob.glob('./results/gif_frame_*.jpg')
filenames = sorted(filenames)
last = -1
images = []

for filename in filenames:

image = imageio.imread(filename)
images.append(image)

imageio.mimsave(anim_file, images, 'GIF-FI', fps=10)

Unfortunately, using IPyImage in the notebook (as you've done in the rubber ducky detection
tutorial) for the large gif generated will disconnect the runtime. To view the animation, you can
instead use the Files pane on the left and double-click on zombie-anim.gif . That will open a
preview page on the right. It will take 2 to 3 minutes to load and see the walking zombie.

Save results file for grading

Run the cell below to save your results. Download the results.data file and upload it to the
grader in the classroom.