[ted537]

Yeah its an awkward position, as self-hosting is going to be insanely expensive unless you have a substantial userbase to amortize the costs over. At least for a model comparable to GPT-4o or deepseek.

But at least if you use an API in the same region as your customers, court order shenanigans won't get you caught between different jurisdictions.

[999900000999]

Ideally smaller models will get better.

For most tasks I don't need the best model in existence, I just need good enough. A small law firm using LLMs for summaries can probably do it on prem and hire a smart college student to setup a PC to do it.

The problem is that's still more difficult ( let's say our hypothetical junior IT only makes 60k a year) than just sending all your private business information to some 3rd party API. You can then act shocked and concerned when your 3rd party leaks the data.