[chiragjn]

Disclaimer: I work at Truefoundry

You can give us a shot at https://truefoundry.com We are a general purpose ML Deployments platform which works on top of your existing Kubernetes clusters (AWS EKS, GCP GKE or Azure AKS) abstracting away the complexity of dealing with cloud providers and Kubernetes. We support Services for ML web apps, APIs, Jobs for ML training jobs, Model Registry for storing models, Model Servers for no code model deployments. (Our platform can be partially or completely self hosted for privacy and compliance)

Adding one or more GPUs (V100, T4, A10, A100, etc) is simply one extra line https://docs.truefoundry.com/docs/gpus#adding-gpu-to-service...

Examples:

- Stable Diffusion with Gradio: https://github.com/truefoundry/truefoundry-examples/tree/mai...

- GPT-J 6B fp16 with FastAPI: https://github.com/truefoundry/truefoundry-examples/tree/mai...

[Monika1012]

Love TrueFoundry! We use it for Infra provisioning on our own cloud and deploying the ML Models behind a choice of a specific model server. Pricing model is also good for early start-ups :)

[agutgutia1991]

PS: (I am one of the founders) - you can write to us at [e-mail] We can help understand your use case and try to suggest whatever is best from whatever is available in the ML Serving ecosystem.