top | item 46804014

(no title)

rao-v | 1 month ago

I'm really charmed by this project (I know there are a few like it).

In particular it's got a single ~600 line file (https://github.com/robitec97/gemma3.c/blob/main/gemma3_kerne...) with a clear straightforward implementation of every major function used in inferencing (google's models) from gelu to rope.

I'm curious how many more functions you'd need to add to have full coverage of every publically available LLM innovation (e.g. QK-Norm from Qwen3, SwiGLU etc.).

Obviously llama.cpp has a much bigger library but it's lovely to see everything in one clean file.

discuss

order

No comments yet.