Hi,
Has anyone tried a configuration with 3 DGX Sparks, taking advantage of each having 2 ConnectX-7 ports?
As we do not see it anyware, we are assuming there are problems with this approach, probably with the routing (as there would be 3 different nets to talk throgh), but we are considering using multicast as a workaround.
Any idea?
Thanks a lot for your inputs :-)
I have not tried that, but would love to see the results. I think it will work, but I don’t have 3 sparks to test it on.
Routing shouldn’t be an issue if you just set static routes on each host for each other host. Doing that will tell the os exactly the ip and source interface to send traffic from when it needs to send traffic to a specific system.
“ip route get” will be your friend when it comes to debugging. Just watch for asymmetric routes, make sure each one can ping each other both ways, tcpdump is pretty useful for these things with a good bpf (filter).
There very well may be other issues preventing this from working, but I don’t see routing being the thing that makes it impossible.