Skip to content

[IE VPU] Enable variable number of inputs for ExpPriorGridGenerator#855

Merged
ggladilov merged 2 commits intoopenvinotoolkit:masterfrom
monicael:vpu/monicael/priorgridgen-rewrite-input-checks
Jun 15, 2020
Merged

[IE VPU] Enable variable number of inputs for ExpPriorGridGenerator#855
ggladilov merged 2 commits intoopenvinotoolkit:masterfrom
monicael:vpu/monicael/priorgridgen-rewrite-input-checks

Conversation

@monicael
Copy link
Copy Markdown
Contributor

@monicael monicael commented Jun 10, 2020

Description

ExperimentalPriorGridGenerator layer has at least one input tensor and at most three. The second and third tensors are required only if the parameters w, h and stride_x, stride_y, respectively, are zero. In such cases, the width/height dimensions of the last two layers are used to compute replacements for the parameters.

In the current implementation, graph transformer does not allow the ExperimentalPriorGridGenerator layer to have less then three inputs.

Here is an example of how an ExpPriorGridGenerator layer with two inputs is described in xml:

<layer id=*id* name=*layer_name* type="ExperimentalDetectronPriorGridGenerator" version="experimental">
    <data flatten="1" h="0" stride_x=*stride_x* stride_y=*stride_y* w="0"/>
    <input>
            <port id="0">
                    <dim> *anchors_num* </dim>
                    <dim>4</dim>
            </port>
            <port id="1">
                    <dim>1</dim>
                    <dim>*feature map channels num*</dim>
                    <dim>*layer_height*</dim>
                    <dim>*layer_width*</dim>
            </port>
            <port id="2">
                    <dim>1</dim>
                    <dim>1</dim>
                    <dim>1</dim>
                    <dim>1</dim>
            </port>
    </input>
    <output>
            <port id="3" precision="FP32">
                    <dim>*num_boxes*</dim>
                    <dim>4</dim>
            </port>
    </output>
</layer>

@monicael monicael requested review from a team and andrejsokolov June 10, 2020 11:20
@monicael monicael self-assigned this Jun 10, 2020
@andrejsokolov
Copy link
Copy Markdown
Contributor

@monicael, does mdk device part support this feature?

@andrejsokolov andrejsokolov requested a review from avershov June 10, 2020 13:06
@ggladilov ggladilov added this to the 2021.1 milestone Jun 10, 2020
@monicael
Copy link
Copy Markdown
Contributor Author

@monicael, does mdk device part support this feature?

Yes. MDK side will always consider the layer params first and will check the shape of the second and third tensors only if their respective set of params are 0 (w/h for the second input, stride_x/stride_y for the third). The actual contents of the second and third tensors are never used.

@ggladilov
Copy link
Copy Markdown
Contributor

@monicael should test be updated as well (add new test cases where another amount of inputs?)

@monicael
Copy link
Copy Markdown
Contributor Author

@monicael should test be updated as well (add new test cases where another amount of inputs?)

I don't think that's necessary. In the example in the description, the xml still has three inputs, but the last one (which should have been the input image of the net) has a 1x1x1x1 shape and is not connected to the input of the net. I'm not sure exactly why it appears in this form, instead of missing altogether. But in the current form, the shape of one of the optional inputs doesn't matter if the associated params are not 0. So, for example, in a test where stride_x and stride_y are non-zero, it doesn't matter if the test dimensions for the third input are 1x1x1x1 or 1x3x225x225, because they will never be taken into consideration and the layer will run the same way in both cases.

@ggladilov
Copy link
Copy Markdown
Contributor

@monicael should test be updated as well (add new test cases where another amount of inputs?)

I don't think that's necessary. In the example in the description, the xml still has three inputs, but the last one (which should have been the input image of the net) has a 1x1x1x1 shape and is not connected to the input of the net. I'm not sure exactly why it appears in this form, instead of missing altogether. But in the current form, the shape of one of the optional inputs doesn't matter if the associated params are not 0. So, for example, in a test where stride_x and stride_y are non-zero, it doesn't matter if the test dimensions for the third input are 1x1x1x1 or 1x3x225x225, because they will never be taken into consideration and the layer will run the same way in both cases.

Still you add new feature (optional inputs) and there is no tests for that. If there is a concern that such a case should not happen and operation always should have all inputs, so it's better to investigate the root cause rather adding this feature. I understand that in some cases values of optional inputs are ignored, but we need to check that (by writing tests). In addition, through tests it'd be clear under which circumstances some inputs may be omitted.

@monicael
Copy link
Copy Markdown
Contributor Author

@monicael should test be updated as well (add new test cases where another amount of inputs?)

I don't think that's necessary. In the example in the description, the xml still has three inputs, but the last one (which should have been the input image of the net) has a 1x1x1x1 shape and is not connected to the input of the net. I'm not sure exactly why it appears in this form, instead of missing altogether. But in the current form, the shape of one of the optional inputs doesn't matter if the associated params are not 0. So, for example, in a test where stride_x and stride_y are non-zero, it doesn't matter if the test dimensions for the third input are 1x1x1x1 or 1x3x225x225, because they will never be taken into consideration and the layer will run the same way in both cases.

Still you add new feature (optional inputs) and there is no tests for that. If there is a concern that such a case should not happen and operation always should have all inputs, so it's better to investigate the root cause rather adding this feature. I understand that in some cases values of optional inputs are ignored, but we need to check that (by writing tests). In addition, through tests it'd be clear under which circumstances some inputs may be omitted.

Alright, that makes sense. I'll write tests for the optional input cases and will let you know when they're done :)

@monicael monicael requested a review from a team June 12, 2020 18:30
@ggladilov ggladilov merged commit 1e180dd into openvinotoolkit:master Jun 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants