top | item 42124602

(no title)

@simonw what is the token/s like on your 64gb m2 mbp?

discuss

simonw|1 year ago

With MLX:

    Prompt: 49 tokens, 95.691 tokens-per-sec
    Generation: 723 tokens, 10.016 tokens-per-sec
    Peak memory: 32.685 GB

joseferben|1 year ago

so quite usable, thanks!