Skip to content

fillNeighborsMPI() mixes subcommunicator receives with default-communicator sends #5060

@WeiqunZhang

Description

@WeiqunZhang
  • Type: Correctness / MPI communicator mismatch
  • Severity: High
  • Component: Neighbor particle CPU MPI path
  • Location:
    • Src/Particle/AMReX_NeighborParticlesCPUImpl.H:553
    • Src/Particle/AMReX_NeighborParticlesCPUImpl.H:566

Problem

fillNeighborsMPI() posts receives on ParallelContext::CommunicatorSub():

ParallelDescriptor::Arecv(..., ParallelContext::CommunicatorSub())

but sends payload with default communicator:

ParallelDescriptor::Send(kv.second.data(), Cnt, Who, SeqNum);

This is inconsistent and looks like copy-paste drift (nearby send sites use CommunicatorSub()).

Impact

  • In subcommunicator/team execution, sends and receives can target different communicator spaces.
  • Can lead to deadlock or unmatched message behavior.

Suggested patch

Use the same subcommunicator for the send call.

--- a/Src/Particle/AMReX_NeighborParticlesCPUImpl.H
+++ b/Src/Particle/AMReX_NeighborParticlesCPUImpl.H
@@
-        ParallelDescriptor::Send(kv.second.data(), Cnt, Who, SeqNum);
+        ParallelDescriptor::Send(kv.second.data(), Cnt, Who, SeqNum,
+                                 ParallelContext::CommunicatorSub());

Prepared by Codex

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions