(no title)
Mehvix | 1 year ago
This is just saying throughput is increased, yes? The time to train, and thus iterate (i.e. dialing in hyperpaprams) will decrease.
Mehvix | 1 year ago
This is just saying throughput is increased, yes? The time to train, and thus iterate (i.e. dialing in hyperpaprams) will decrease.
kelseyfrog|1 year ago
Ie: more efficient steam engines lead to both an increase of steam engine throughput as well as coal consumption, an increase in AI efficiency can lead to an increase in training throughput and energy consumption.
The paradox is a result of prevalence scaling faster than efficiency and efficiency driving prevalence.