[smlacy]

Is there a linux equivalent of this setup? I see some mention of RDNA support for linux distros, but it's unclear to me if this is hardware-specific (requires ConnectX or in this case Apple Thunderbolt) or is there something interesting that can be done with "vanilla 10G NIC" hardware?

[Maxious]

To get the production level performance, you do need the RDNA compatible hardware.

However, vLLM supports multi node clusters over normal ethernet too https://docs.vllm.ai/en/stable/serving/parallelism_scaling/#...