[l3jin]

Universal deployment is indeed attractive. I have tested the Llama2-70B on 7900 XTX. Love the performance!

Also saw a report earlier today on MLC’s discord about AMD MI-100:

GPU Count | Model Size | Prefill Speed | Decode Speed

1 | 33b | 102.2 | 22.3

2 | 33b | 112.3 | 33.0

4 | 33b | 144.8 | 41.2