[saternius]

What we do at https://quillbot.com

Training:

Currently we just use a bunch of beefy desktop workstations for training (using Pytorch).

Deployment:

This is the vast majority of our cost, each time a paraphrase comes in we add it to a queue through google cloud Pubsub. We have a cluster of GPU (T4) servers pulling from the queue, generating paraphrases and then sending the responses back through Redis pub/sub. I think ideally we would have a system that makes it easier to batch sentences of similar length together, but this seems to be the most cost effective way for models that are too computationally expensive for the CPU that is relatively simple to put together.