[IceHegel]

By batch size, do you mean the number of tokens in the context window that were generated by the model vs. external tokens?

Because my understandings is that, however you get to 100K, the 100,001st token is generated the same way as far as the model is concerned.