[summarity]

> The GPU version is commercial software. Please contact...

Shame.

[versteegen]

The GPU version of libNC is available as a free binary, and you can find MIT-licensed source code implementing (training and inference of) transformers using libNC at https://bellard.org/nncp. (nncp was meant to be a submission to the Hutter Prize, which would have required open-sourcing libNC too, but it didn't qualify due to using AVX2 and too much RAM. At least the CPU version binary is MIT licensed since ts_server is.) I think it wouldn't be that big a project to support LLaMa starting from that code, although it is dense code.

Edit: licensing

[sroecker]

That version of libnc_cuda.so doesn't seem to be compatible with ts_server though.

[etaioinshrdlu]

I think it's fine if a software legend (or anyone) wants to make some income.

[rvz]

"Shame. I cannot take the code / binary and run it as a SaaS efficiently." - HNers

Good for him to commercialize it, and at least he is not pretending to be a non-profit accepting VC money.

[alexvoda]

Countering SaaS-ifying by others can also be achieved through the AGPL or through the BSL (initially not open-source, reverts to open source after set period).

I do believe one of the failures of GPL was not being AGPL from the start.

[copperx]

Shame as in "it's a shame" or as in "shame on them"?

[minxomat]

The former. Makes sense for their business model.

[dark-star]

I'm pretty sure a dev skilled in ML and GPUs has no trouble modifying the MIT code (which runs on the CPU) to something that runs on GPUs...

[EMIRELADERO]

The CPU version is provided under binary format only.