top | item 46366671

(no title)

lewisjoe | 2 months ago

It'd be great if it supports a wasm/web backend as well.

I bet a lot of trivial text capabilities (grammar checking, autocomplete, etc) will benefit from this rather than sending everything to a hosted model.

It's possible right now with onnx / transformers.js / tensorflow.js - but none of them are quite there yet in terms of efficiency. Given the target for microcontrollers, it'd be great to bring that efficiency to browsers as well.

discuss

klaussilveira|2 months ago

If you need WASM, I think Candle is your current best bet: https://github.com/huggingface/candle

miguel_martin|2 months ago

You can compile to wasm, I have done so via the XNNPACK backend - you might have to tweak the compilation settings and upgrade the XNNPACK submodule/patch some code. But this only supports CPU, not a WebGPU or WebGL backend.