[ioedward]

Normally people split up the model across multiple GPUs, i.e. model/tensor parallelism.