top | item 45510887 (no title) jochalek | 4 months ago Sounds like something that could be implemented with llm-d, though I've not experimented with it.https://llm-d.ai/blog/intelligent-inference-scheduling-with-... discuss order hn newest rgthelen|4 months ago Yeah, I don't see why we could not integrate that. I think that is the next step as we move our workloads to production. mhamann|4 months ago `lf deploy` here we come!
rgthelen|4 months ago Yeah, I don't see why we could not integrate that. I think that is the next step as we move our workloads to production. mhamann|4 months ago `lf deploy` here we come!
rgthelen|4 months ago
mhamann|4 months ago