Code Release: CARAFE: Content-Aware ReAssembly of FEatures (ICCV 2019) #1583

myownskyW7 · 2019-10-24T13:56:52Z

No description provided.

ShihuaiXu · 2019-10-31T06:00:12Z

Error limit reached.
100 errors detected in the compilation of "/tmp/tmpxft_00006868_00000000-6_carafe_cuda_kernel_benchmark.cpp1.ii".
Compilation terminated.
error: command '/usr/local/cuda/bin/nvcc' failed with exit status 1
occur this error, please help me

hellock · 2019-12-29T11:16:32Z

mmdet/models/mask_heads/fcn_mask_head.py

                m.weight, mode='fan_out', nonlinearity='relu')
            nn.init.constant_(m.bias, 0)
+        if self.with_carafe:
+            self.upsample.init_weights()


for m in [self.upsample, self.conv_logits]: if m is None: continue elif isinstance(m, CARAFEPack): m.init_weights() else: nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu') nn.init.constant_(m.bias, 0)

hellock · 2019-12-29T11:33:50Z

mmdet/models/mask_heads/fcn_mask_head.py

                 upsample_ratio=2,
                 num_classes=81,
                 class_agnostic=False,
+                 carafe_cfg=None,


Maybe renamed to upsample_cfg for future usage.

hellock · 2019-12-29T11:36:13Z

mmdet/models/mask_heads/fcn_mask_head.py

                self.upsample_ratio,
                stride=self.upsample_ratio)
+        elif self.upsample_method == 'carafe':
+            self.upsample = CARAFEPack(upsample_in_channels, upsample_ratio,


To be consistent, use self.upsample_ratio instead.

hellock · 2019-12-29T11:40:46Z

mmdet/models/necks/fpn_carafe.py

+                        out_channels,
+                        self.upsample_kernel,
+                        stride=2,
+                        padding=int((self.upsample_kernel - 1) / 2),


(self.upsample_kernel - 1) // 2

hellock · 2019-12-29T11:41:52Z

mmdet/models/necks/fpn_carafe.py

+                    upsample_module = nn.ConvTranspose2d(
+                        out_channels,
+                        out_channels,
+                        self.upsample_kernel,


self.upsample_kernel is undefined.

hellock · 2019-12-29T11:58:11Z

mmdet/models/necks/fpn_carafe.py

+        for i in range(len(laterals) - 1, 0, -1):
+            if self.upsample is not None:
+                if (self.upsample == 'nearest' or self.upsample == 'bilinear'):
+                    align_corners = (None


Add some comments here like suppress warnings.

hellock · 2019-12-29T11:59:01Z

mmdet/ops/carafe/carafe.py

+carafe_naive = CARAFENAIVEFunction.apply
+
+
+class CARAFENAIVE(Module):


Better to name it CARAFENaive.

hellock · 2019-12-29T11:59:44Z

mmdet/ops/carafe/setup.py

+
+from torch.utils.cpp_extension import BuildExtension, CUDAExtension
+
+cxx_args = ['-std=c++11']


unused variables

hellock · 2019-12-29T12:00:27Z

mmdet/ops/carafe/src/carafe_cuda_kernel.cu

+  const int n = index / width / height;
+
+  // const int down_pw = pw / scale_factor;
+  // const int down_ph = ph / scale_factor;


useless comments

hellock · 2019-12-29T12:01:53Z

mmdet/ops/carafe/src/carafe_naive_cuda_kernel.cu

+  const int output_size = batch_size * channels * height * width;
+
+  // TODO: use AT_DISPATCH_FLOATING_TYPES_AND_HALF when atomicAdd is resolved
+  AT_DISPATCH_FLOATING_TYPES(


AT_DISPATCH_FLOATING_TYPES_AND_HALF is available here.

mmdet/models/necks/fpn_carafe.py

mmdet/ops/carafe/carafe.py

mmdet/models/necks/fpn_carafe.py

mmdet/ops/carafe/src/carafe_cuda.cpp

configs/carafe/README.md

mmdet/models/mask_heads/fcn_mask_head.py

hellock · 2020-02-19T16:30:13Z

mmdet/models/utils/upsampler.py

+
+
+upsampler_cfg = {
+    # format: layer_type: (abbreviation, module)


layer_type and abbreviation are the same. There is no need to use a tuple for dict values.

hellock · 2020-02-19T16:30:34Z

mmdet/models/utils/upsampler.py

+}
+
+
+def build_upsampler_layer(cfg, postfix=''):


I suggest using a consistent term "upsample" instead of "upsample" and "upsampler".

postfix is unnecessary.

hellock · 2020-02-19T16:43:00Z

mmdet/models/utils/upsampler.py

+from mmdet.ops.carafe import CARAFEPack
+
+
+class PixelShufflePack(nn.Module):


Add docstring here.

hellock · 2020-02-20T15:31:48Z

configs/carafe/faster_rcnn_r50_fpn_carafe_1x.py

+total_epochs = 12
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+work_dir = './work_dirs/faster_rcnn_r50_fpn_1x'


Use the same name as the config file.

mmdet/models/mask_heads/fcn_mask_head.py

hellock · 2020-02-20T15:43:30Z

mmdet/models/mask_heads/fcn_mask_head.py

-        self.upsample_method = upsample_method
-        self.upsample_ratio = upsample_ratio
+        self.upsample_method = self.upsample_cfg.pop('type')
+        self.upsample_ratio = self.upsample_cfg.pop('upsample_ratio')


We may consider renaming upsample_ratio to scale_factor since all upsample operators below have the argument scale_factor.

It will be simpler to manipulate the args.

self.upsample_method = upsample_cfg_.get('type') if self.upsample_method is None: self.upsample = None elif self.upsample_method == 'deconv': upsample_cfg_.update( in_channels=upsample_in_channels, out_channels=self.conv_out_channels, kernel_size=self.upsample_ratio, stride=self.upsample_ratio) elif self.upsample_method == 'carafe': upsampler_cfg_.update( channels=upsample_in_channels, scale_factor=self.upsample_ratio) else: xxxxx

hellock · 2020-02-20T15:45:16Z

mmdet/models/utils/upsample.py

+
+
+class PixelShufflePack(nn.Module):
+    """ pixel shuffle upsample layer


"""Pixel Shuffle upsample layer

hellock · 2020-02-20T15:46:16Z

mmdet/models/utils/upsample.py

+    """ pixel shuffle upsample layer
+
+    Args:
+        in_channels (int): number of input channels


The description is a sentence and use upper case for the first character.

mmdet/models/utils/upsample.py

mmdet/ops/carafe/carafe.py

hellock · 2020-02-20T15:51:21Z

mmdet/ops/carafe/carafe.py

+
+        self.kernel_size = int(kernel_size)
+        self.group_size = int(group_size)
+        self.scale_factor = int(scale_factor)


Add a type check instead of simply casting to int.

* jon/dev/fix_fpn2: (33 commits) Fix FPN upscale fix analyze log (open-mmlab#2150) Fix a documentation error in GETTING_STARTED.md (open-mmlab#2149) add optimizer registry (open-mmlab#2139) Update version to 1.1 (open-mmlab#2144) Fix IOU assigners when ignore_of_thr > 0 and no pred boxes (open-mmlab#2135) reset worker_seed (open-mmlab#2111) Fix issue with list of metrics in CustomDataset.evaluate (open-mmlab#2128) Code Release: CARAFE: Content-Aware ReAssembly of FEatures (ICCV 2019) (open-mmlab#1583) fixed test package (open-mmlab#2127) add an argument format-only to handle the json formating (open-mmlab#2114) fix (get_cls_results): use np.empty for empty bbox rather than np.arrary (open-mmlab#2116) fix (dpool): directly return empty if rois's length is 0 (open-mmlab#2099) fix workflow problem (open-mmlab#2103) Uint8 fix (open-mmlab#2105) Fix device bug (open-mmlab#2098) fix test ddp initialize (open-mmlab#2100) set FORCE_CUDA (open-mmlab#2097) Speed up sampler (open-mmlab#2094) Use official DDP to implement MMDDP (open-mmlab#2090) ...

* tag 'v1.1.0': (29 commits) Update version to 1.1 (open-mmlab#2144) Fix IOU assigners when ignore_of_thr > 0 and no pred boxes (open-mmlab#2135) reset worker_seed (open-mmlab#2111) Fix issue with list of metrics in CustomDataset.evaluate (open-mmlab#2128) Code Release: CARAFE: Content-Aware ReAssembly of FEatures (ICCV 2019) (open-mmlab#1583) fixed test package (open-mmlab#2127) add an argument format-only to handle the json formating (open-mmlab#2114) fix (get_cls_results): use np.empty for empty bbox rather than np.arrary (open-mmlab#2116) fix (dpool): directly return empty if rois's length is 0 (open-mmlab#2099) fix workflow problem (open-mmlab#2103) Uint8 fix (open-mmlab#2105) Fix device bug (open-mmlab#2098) fix test ddp initialize (open-mmlab#2100) set FORCE_CUDA (open-mmlab#2097) Speed up sampler (open-mmlab#2094) Use official DDP to implement MMDDP (open-mmlab#2090) log meta (open-mmlab#2086) fix pad_val not used in class Pad when pad to a fixed size (open-mmlab#2093) remove cython docs (open-mmlab#2091) remove new_tensor (open-mmlab#2092) ...

open-mmlab#1583) * add carafe ops * rename carafe benchmark * grad check fix * update grad check * update grad check output * add fpn carafe & mask head carafe * add ReadMe * update readme * add carafe setup * update naive carafe * update readme and setup * readme typo fix * fix flake8 error * fix flake 8 error * fix flake 8 * fix flake 8 more * flake 8 fix plus * flake 8 fix * fix flake 8 * reformat ops files * update fpn files and cfgs * update readme * update fcn_mask_head * update fpn_carafe * update kernel * update * update * add docstring in FPN_CARAFE * reformat with yapf * update * update * add build upsampler * fix mask head build error * reformat build upsample layer * add doc string for CARAFE and PixelShuffle * update * update upsample_cfg_ * update * update doc string * rm abbr in build upsample layer * update readme * update model_zoo * add link to other features in ReadMe

myownskyW7 added 11 commits October 23, 2019 08:13

add carafe ops

107561b

rename carafe benchmark

7d3627f

grad check fix

9a56113

update grad check

57e2ef7

update grad check output

6b30375

add fpn carafe & mask head carafe

420f1b9

add ReadMe

2879f3d

update readme

0a0658e

add carafe setup

35fc80c

update naive carafe

3561d69

update readme and setup

ff59705

myownskyW7 requested a review from hellock October 24, 2019 13:57

myownskyW7 added 8 commits October 24, 2019 07:00

readme typo fix

a684d68

fix flake8 error

e44f151

fix flake 8 error

e0f3208

fix flake 8

f28bc17

fix flake 8 more

9d60836

flake 8 fix plus

16a4446

flake 8 fix

837d2a2

fix flake 8

3748e2b

myownskyW7 mentioned this pull request Oct 25, 2019

Where's the implementation of CARAFE #1535

Closed

hellock mentioned this pull request Dec 27, 2019

when will the CARAFE source codes can be release? #1397

Closed

myownskyW7 added 4 commits December 28, 2019 23:59

merge master

b687e3f

reformat ops files

ff575f4

update fpn files and cfgs

6ac0031

update readme

bda3bdb

hellock reviewed Dec 29, 2019

View reviewed changes

myownskyW7 added 2 commits December 29, 2019 05:18

update fcn_mask_head

f89b4c5

update fpn_carafe

b0c6761

myownskyW7 added 5 commits February 18, 2020 21:37

Merge branch 'master' into carafe

303a630

update

c30fb88

add docstring in FPN_CARAFE

be05c8d

reformat with yapf

318bd6e

update

c45fdba

hellock reviewed Feb 18, 2020

View reviewed changes

configs/carafe/README.md Outdated Show resolved Hide resolved

myownskyW7 added 3 commits February 19, 2020 20:30

update

4312ab1

add build upsampler

48a6d29

fix mask head build error

160d401

hellock reviewed Feb 19, 2020

View reviewed changes

myownskyW7 added 2 commits February 20, 2020 22:39

reformat build upsample layer

0da8e2d

add doc string for CARAFE and PixelShuffle

0ad92ca

hellock reviewed Feb 20, 2020

View reviewed changes

myownskyW7 added 9 commits February 21, 2020 15:29

update

34c4a16

update upsample_cfg_

33aac9e

update

a695807

Merge branch 'master' into carafe

35d6fd1

update doc string

3095a6d

rm abbr in build upsample layer

f79e75f

update readme

48d85fa

update model_zoo

74ecb69

add link to other features in ReadMe

ac9b8b4

hellock approved these changes Feb 21, 2020

View reviewed changes

hellock merged commit b543109 into open-mmlab:master Feb 21, 2020

		carafe_naive = CARAFENAIVEFunction.apply


		class CARAFENAIVE(Module):


		from torch.utils.cpp_extension import BuildExtension, CUDAExtension

		cxx_args = ['-std=c++11']



		upsampler_cfg = {
		# format: layer_type: (abbreviation, module)

		from mmdet.ops.carafe import CARAFEPack


		class PixelShufflePack(nn.Module):



		class PixelShufflePack(nn.Module):
		""" pixel shuffle upsample layer

Code Release: CARAFE: Content-Aware ReAssembly of FEatures (ICCV 2019) #1583

Code Release: CARAFE: Content-Aware ReAssembly of FEatures (ICCV 2019) #1583

Uh oh!

Conversation

myownskyW7 commented Oct 24, 2019

Uh oh!

ShihuaiXu commented Oct 31, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ShihuaiXu commented Oct 31, 2019 •

edited

Loading