top | item 46675678

(no title)

tuned | 1 month ago

it most-likely will in terms of performance as it uses 50% less memory (for sure it will at inference time that is the most used operation on web services), because it can leverage longer T and D if the design is confirmed and the quality of generation is comparable to other models. If this very basic assumption is correct, it means a lot of savings in electricity as the same GPUs can resolve more requests.

discuss

order

throwaway314155|1 month ago

By performance, I meant the accuracy of the model, not the runtime/memory characteristics.