top | item 45050306

(no title)

Freedom5093 | 6 months ago

I don't understand:

> all of the prompt tokens are available in advance and do not require decoding

> The other technique is called batching: this technique aggregates multiple prompts into a single decode operation.

So do prompts get decoded or not? Are there 2 decode steps? Unclear

discuss

order

No comments yet.