[nivvis]

for posterity, since shown that is it actually MoE

> 21B parameters with 3.6B active parameters