I reckon it'd be fantastic to get rid of these complex nested structures and break up large functions by spreading their functionality across smaller functions.

Hi @AlexanderChaptykov , thanks for your comment. Yes that would be nice, but I need a short time to add a few commits beforehand.

@vfdev-5 , finally It's ready to get reviewed.

Hi @vfdev-5 , finally this seems to be ready for review.

failure reasons:

• MPS tests: Cummax has yet to be implemented for Pytorch MPS backend. User could set fallback MPS to CPU environment variable. Is it OK?
• Unit tests / MacOS: It raises NotEnoughMemoryError while its MPS memory is empty. I could not figure out its reason. Any idea?

@sadra-barikbin sounds great!

MPS tests: Cummax has yet to be implemented for Pytorch MPS backend

Let's do the following:

# precision_integrand = precision.flip(-1).cummax(dim=-1).values.flip(-1)


if precision.device.type == "mps":
    # Manual fallback to CPU if precision is on MPS due to the error:
    # NotImplementedError: The operator 'aten::_cummax_helper' is not currently implemented for the MPS device
    device = precision.device
    precision_integrand = precision.flip(-1).cpu()
    precision_integrand = precision_integrand.cummax(dim=-1).values
    precision_integrand = precision_integrand.to(device=device).flip(-1)
else:
    precision_integrand = precision.flip(-1).cummax(dim=-1).values.flip(-1)

Unit tests / MacOS: It raises NotEnoughMemoryError while its MPS memory is empty

I haven't seen that previously. If we have large tensors in the test, we can skip those tests on MPS device.

Concerning failing MPS tests:

2024-09-05T21:11:39.5671470Z FAILED tests/ignite/metrics/vision/test_object_detection_map.py::test_compute[sample0] - RuntimeError: MPS backend out of memory (MPS allocated: 8.00 MB, other allocations: 1.02 MB, max allowed: 7.93 GB). Tried to allocate 256 bytes on shared pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).

Let's skip the tests on MPS

The link https://docs.wandb.ai/library/init apparently isn't valid anymore.

We can replace it with : https://docs.wandb.ai/ref/python/init

@vfdev-5 , any blocker on this?

@sadra-barikbin can you please fix meanwhile the issue with idist.all_gather_tensors_with_shapes on Horovod?
I'll make another quick review pass

Any blocker on this? If something is missing I can volunteer to finish / fix it.

I am using this and would love to see it merged ❤️

Any blocker on this? If something is missing I can volunteer to finish / fix it.

I am using this and would love to see it merged ❤️

@tarbaig Thanks for the feedback. I think we can merge this PR. It remains to fix the problems with idist.all_gather_tensors_with_shapes on Horovod. I have a PR improving the tests: #3302 but Horovod is still failing, so need to make a pass on that to understand why.

@sadra-barikbin can you please fix this issue with older pytorch versions (v1.10-v1.13.1):

2025-02-27T00:30:03.9316464Z         precision_integrand = precision_integrand.take_along_dim(
2025-02-27T00:30:03.9316653Z >           rec_thresh_indices.where(rec_thresh_indices != recall.size(-1), 0), dim=-1
2025-02-27T00:30:03.9316767Z         ).where(rec_thresh_indices != recall.size(-1), 0)
2025-02-27T00:30:03.9316988Z E       TypeError: where(): argument 'other' (position 2) must be Tensor, not int

• https://github.com/pytorch/ignite/actions/runs/13556010764