top | item 46390739

(no title)

runeblaze | 2 months ago

> And if for some ungodly reason you had to do it in Python

I literally invoke sglang and vllm in Python. You are supposed to (if not using them over-the-network) use the two fastest inference engines there is via Python.

discuss

order

No comments yet.