Just one remark but it looks good !

Hi @vfdev-5 @sdesrozis ,

Thanks for the quick PR, looks very good to me.
We also added support for list of tensors to MONAI last week: https://github.com/Project-MONAI/MONAI/blob/dev/monai/metrics/metric.py#L61
I have a quick question about the metrics support for dict:
Is it possible to make the global variable required_output_keys of ignite Metric to be an arg of Metric.__init__()?
We already received much feedback that our workflow programs have too many lambda functions and hard to understand. I am trying to reduce the output_transform= lambda x: ... in the code. If we can explicitly specify the keys of y_pred and y, maybe it's easier to use and understand?
CC @wyli

Thanks.

Hi @Nic-Ma ,

Is it possible to make the global variable required_output_keys of ignite Metric to be an arg of Metric.init()?

thanks for the feedback, required_output_keys is a class attribute and I think it would be better to set required_output_keys globally instead of for each metric:

from ignite.engine import Engine
from ignite.metrics import Accuracy, Metric

Metric.required_output_keys = ("my_y_pred", "my_y_true")

acc = Accuracy()
print(acc.required_output_keys)

engine = Engine(lambda e, b: None)
engine.state.output = {"my_y_pred": torch.rand(4, 10), "my_y_true": torch.randint(0, 10, size=(4, ))}

acc.iteration_completed(engine)
acc.compute()
> ('my_y_pred', 'my_y_true')
> 0.125

what do you think ?

Hi @vfdev-5 ,

Thanks for your proposal and sample code!
It can solve our problem, but maybe a little bit "hacking", especially when setting ignite args in the JSON config of NVIDIA Clara-Train SDK.
Our use case is something like:

"trainer": {
    "name": "SupervisedTrainer",
    "args": {
        ... ...
    }
},
post_transforms: [
    {
        "name": "CopyItemsD":
        "args": {"keys": "pred", "times": 1, "names": ["pred_inverted"]}
    },
    {
        "name": "InvertD",
        "args": {"keys": "pred_inverted"}
    },
    {
        "name": "SplitChannelD",
        "args": {"keys": "pred_inverted", "output_postfixes": ["background", "class1", "class2"]}
    }
]
"metrics": [
    {
        "name": "Accuracy",  # ignite metric
        "args": {"output_keys": ["pred", "label"]}
    },
    {
        "name": "MeanDice",  # subclass of ignite metric
        "args": {"output_keys": ["pred_inverted", "label_inverted"]}
    },
    {
        "name": "MeanDice",  # metric on class 1
        "args": {"output_keys": ["pred_inverted_class1", "label_inverted_class1"]}
    },
    {
        "name": "MeanDice",  # metric on class 2
        "args": {"output_keys": ["pred_inverted_class2", "label_inverted_class2"]}
    }
]

What do you think is the best way to support it?

Thanks.

Thanks for the details, Nic ! That's definitely requires output_transform to handle each case. Let me think about that.

We already received much feedback that our workflow programs have too many lambda functions and hard to understand. I am trying to reduce the output_transform= lambda x: ... in the code.

In this case, maybe, it is possible to introduce one function to deal with those required keys for each metric like:

def from_dict(*keys):
    def wrapper(output: dict):
        return tuple(output[k] for k in keys)
    return wrapper

acc = Accuracy(output_transform=from_dict("pred", "label"))
mean_dice = MeanDice(output_transform=from_dict("pred_inverted", "label_inverted"))

what do you think ?

OK, thanks for the solution.
Let me try to add this API in MONAI utilities.

Thanks.

Hi @vfdev-5 ,

BTW, may I know when do you plan to release ignite v0.4.5?

Thanks.

Hi Nic,
It should be done by this friday.

Sounds great!
Thanks for letting me know.