Show HN: VLLM with JSON Guided Generation

[btwillard]

Our project, Outlines, now offers guided/constrained generation (e.g. according to a JSON schema) via the VLLM library.

My colleague, Rémi, created some patches that allow one to pass vLLM a JSON schema along with the prompt, which dramatically simplifies deployment of JSON-guided generation. He also added a new `serve` interface that puts it all together and makes serving such models a 2-3 line process.

Check it out and tell us what you think!

[jilijeanlouis]

this is really cool probably the most wanted feature in LLM open source. It would be nice to have regex support too?

do you plan to support regex?

[remilouf]

We already support regex-guided generation in the library, and could easily make an API to serve this as well if that's a feature people want!