[xs83]

Now this looks much more interesting! Is the top one input tokens and the second one output tokens?

So 38.54 t/s on 120B? Have you tested filling the context too?

[ggerganov]

Yes, I provided detailed numbers here: https://github.com/ggml-org/llama.cpp/discussions/16578