Multi-pretrained weight support - FasterRCNN ResNet50 by datumbox · Pull Request #4613 · pytorch/vision

datumbox · 2021-10-14T12:00:47Z

Resolves #4671

Example usage:

from torchvision.io.image import read_image
from torchvision.utils import draw_bounding_boxes
from torchvision.transforms.functional import to_pil_image
from torchvision import prototype as P


img = read_image("test/assets/encode_jpeg/grace_hopper_517x606.jpg")

# Initialize model
weights = P.models.detection.FasterRCNNResNet50FPNWeights.Coco_RefV1
model = P.models.detection.fasterrcnn_resnet50_fpn(weights=weights)
model.eval()

# Initialize inference transforms
preprocess = weights.transforms()

# Apply inference preprocessing transforms
batch = [preprocess(img)[0]]
prediction = model(batch)[0]

# Use meta to get labels
labels = [weights.meta["categories"][i] for i in prediction["labels"]]
box = draw_bounding_boxes(img, boxes=prediction["boxes"],
                          labels=labels,
                          colors="red",
                          width=4, font_size=30)
im = to_pil_image(box.detach())
im.show()

cc @datumbox @pmeier @bjuncek

datumbox

Some clarification comments below:

datumbox · 2021-10-14T14:44:52Z

torchvision/prototype/models/detection/backbone_utils.py

+    returned_layers=None,
+    extra_blocks=None,
+):
+    backbone = resnet.__dict__[backbone_name](weights=weights, norm_layer=norm_layer)


Unfortunately I'm forced to copy the whole function just to change the pretrained to weights param. I refactored to minimize copy-pasted code.

datumbox · 2021-10-14T14:45:31Z

torchvision/prototype/transforms/presets.py



-# Allows handling of both PIL and Tensor images
-class ConvertImageDtype(nn.Module):


Removed the standalone transform to avoid introducing a new class here.

datumbox · 2021-10-14T14:46:42Z

torchvision/prototype/models/detection/faster_rcnn.py

+import warnings
+from typing import Any, Optional
+
+from ....models.detection.faster_rcnn import FasterRCNN, overwrite_eps, _validate_trainable_layers


Inherit as much as possible. The changes below will be moved on the existing files once we move to torchvision.

datumbox · 2021-10-14T15:12:13Z

torchvision/prototype/models/detection/faster_rcnn.py

+
+def fasterrcnn_resnet50_fpn(
+    weights: Optional[FasterRCNNResNet50FPNWeights] = None,
+    weights_backbone: Optional[ResNet50Weights] = ResNet50Weights.ImageNet1K_RefV1,  # TODO: Should we default to None?


This default value is to align with the old logic, NEVERTHELESS it's not necessary for BC (due to the way I handled the pretrained param below).

Personally I don't like that we have hardcoded an OLD set of weights here. If we set the default value to None and force the user to choose, we will eliminate future BC considerations if these weights become too old and not-optimal.

This can be addressed on a follow up PR.

cc @fmassa

I'm going to set this to None, happy to review if someone has a strong opinion.

prabhat00155

Thanks @datumbox! Few minor comments.

torchvision/models/detection/backbone_utils.py

torchvision/prototype/models/__init__.py

torchvision/prototype/models/_meta.py

fmassa

LGTM, thanks!

torchvision/prototype/models/detection/faster_rcnn.py

fmassa · 2021-10-15T09:46:47Z

torchvision/prototype/models/detection/faster_rcnn.py

+
+    if weights is not None:
+        weights_backbone = None
+        num_classes = len(weights.meta["categories"])


We should probably raise an error / warning if the user modifies the num_classes and passes a weights argument. Otherwise they might silently think that we are doing magic inside

Sounds good. I'll add this check to resnet as well.

I thought about this and it's a bit problematic. The num_classes parameter has a default value in all of our model builders. So to see i it was modified, we need to see if the default value was changed which can lead to messy code. An alternative approach could be to throw a warning if the num_classes != len(weights.meta["categories"]) but still overwrite it to make the life of users easier.

Because it's not clear how this should be handled, I'm going to merge the PR to unblock the work but I'm happy to discuss the policy here and update everywhere in a follow up PR.

github-actions · 2021-10-15T12:49:31Z

Hey @datumbox!

You merged this PR, but no labels were added. The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py

* Adding FasterRCNN ResNet50. * Refactoring to remove duplicate code. * Adding typing info. * Setting weights_backbone=None as default value. * Overwrite eps only for specific weights.

Summary: * Adding FasterRCNN ResNet50. * Refactoring to remove duplicate code. * Adding typing info. * Setting weights_backbone=None as default value. * Overwrite eps only for specific weights. Reviewed By: NicolasHug Differential Revision: D31758312 fbshipit-source-id: 714a714d897bb4b4d9da1298ad5e2606998898b9

* Adding FasterRCNN ResNet50. * Refactoring to remove duplicate code. * Adding typing info. * Setting weights_backbone=None as default value. * Overwrite eps only for specific weights.

Adding FasterRCNN ResNet50.

ccb7899

datumbox added module: models prototype labels Oct 14, 2021

pytorch-probot bot added the ciflow/default label Oct 14, 2021

datumbox requested a review from fmassa October 14, 2021 12:01

datumbox mentioned this pull request Oct 14, 2021

Extend TorchVision's existing model builders to support multiple weights #4611

Closed

facebook-github-bot added the cla signed label Oct 14, 2021

datumbox commented Oct 14, 2021

View reviewed changes

Refactoring to remove duplicate code.

7c256a2

datumbox commented Oct 14, 2021

View reviewed changes

prabhat00155 approved these changes Oct 14, 2021

View reviewed changes

torchvision/models/detection/backbone_utils.py Outdated Show resolved Hide resolved

torchvision/prototype/models/__init__.py Show resolved Hide resolved

torchvision/prototype/models/_meta.py Show resolved Hide resolved

datumbox added 2 commits October 14, 2021 18:12

Adding typing info.

802d45a

Setting weights_backbone=None as default value.

0146a23

fmassa approved these changes Oct 15, 2021

View reviewed changes

datumbox and others added 2 commits October 15, 2021 11:18

Merge branch 'main' into multiweight/fasterrcnn

2d1c94e

Overwrite eps only for specific weights.

32f2a28

datumbox merged commit ec2456a into pytorch:main Oct 15, 2021

datumbox deleted the multiweight/fasterrcnn branch October 15, 2021 12:49

datumbox added the enhancement label Oct 15, 2021

This was referenced Oct 15, 2021

Multi-pretrained weight support - Quantized ResNet50 #4627

Merged

Refactor Segmentation models #4646

Merged

sadra-barikbin mentioned this pull request Sep 20, 2024

COCO mAP metric pytorch/ignite#2901

Merged

3 tasks



		# Allows handling of both PIL and Tensor images
		class ConvertImageDtype(nn.Module):

Conversation

datumbox commented Oct 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datumbox left a comment

Choose a reason for hiding this comment

Uh oh!

datumbox Oct 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

datumbox Oct 14, 2021

Choose a reason for hiding this comment

Uh oh!

datumbox Oct 14, 2021

Choose a reason for hiding this comment

Uh oh!

datumbox Oct 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

datumbox Oct 14, 2021

Choose a reason for hiding this comment

Uh oh!

prabhat00155 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fmassa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fmassa Oct 15, 2021

Choose a reason for hiding this comment

Uh oh!

datumbox Oct 15, 2021

Choose a reason for hiding this comment

Uh oh!

datumbox Oct 15, 2021

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

datumbox commented Oct 14, 2021 •

edited

Loading

datumbox Oct 14, 2021 •

edited

Loading

datumbox Oct 14, 2021 •

edited

Loading