Classification can't work in distributed data parallel if has rich meta data #1570
Description
Describe the bug
When evaluate on nifti data, the meta data is very rich, and we only all_gather the file names.
It will cause the length of other meta data can't match model output:
2021-02-09 02:12:24,253 - ignite.engine.engine.SupervisedEvaluator - INFO - Engine run resuming from iteration 0, epoch 0 until 1 epochs
2021-02-09 02:12:24,436 - ignite.engine.engine.SupervisedEvaluator - INFO - Restored all variables from /home/madil/dlmed/problems/pt_mmars/classification_chestxray_ea2/commands/../models/model.pt
2021-02-09 02:12:29,927 - ignite.engine.engine.SupervisedEvaluator - ERROR - Current run is terminating due to exception: list index out of range.
2021-02-09 02:12:29,928 - ignite.engine.engine.SupervisedEvaluator - ERROR - Exception: list index out of range
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/ignite/engine/engine.py", line 812, in _run_once_on_dataset
self._fire_event(Events.ITERATION_COMPLETED)
File "/opt/conda/lib/python3.6/site-packages/ignite/engine/engine.py", line 423, in _fire_event
func(*first, *(event_args + others), **kwargs)
File "/opt/monai/monai/handlers/classification_saver.py", line 97, in __call__
self.saver.save_batch(_engine_output, _meta_data)
File "/opt/monai/monai/data/csv_saver.py", line 95, in save_batch
self.save(data, {k: meta_data[k][i] for k in meta_data} if meta_data else None)
File "/opt/monai/monai/data/csv_saver.py", line 95, in <dictcomp>
self.save(data, {k: meta_data[k][i] for k in meta_data} if meta_data else None)
IndexError: list index out of range
Expected behavior
Only save file names in the meta data and call CSV saver.