top | item 44844867

(no title)

lhl | 6 months ago

You're right on ratios, but actually the ratio is much worse than 6:1 since they are MoEs. The 20B has 3.6B active, and the 120B has only 5.1B active, only about 40% more!

discuss

order

No comments yet.