top | item 44045130

(no title)

IceWreck | 9 months ago

According to the readme here - https://huggingface.co/google/gemma-3n-E4B-it-litert-preview

E4B has a score of 44.4 in the Aider polyglot dashboard. Which means its on-par with gemini-2.5-flash (not the latest preview but the version used for the bench on aider's website), gpt4o and gpt4.5.

Thats sounds very good - imagine what a coding focused version of this could do if this is a "generic" embedded only model.

On the other hand - this does have a much lower score for livecodebench.

discuss

order

nolist_policy|9 months ago

Hmm, the Aider polyglot benchmark has been removed from the huggingface readme.

Also:

> These models were evaluated at full precision (float32)

For 4B effective parameters that's 16 GB ram.