[danrocks]

> I have no idea how it will slow down. Someone just figured out how to reduce the cost of building a multi million dollar model to around $600. That was supposed to take another decade.

I don't think this is accurate. The Stanford team used LLaMA as base model and added a smaller model on top of it - training the joint model using data (generated from ChatGPT) is what cost $600. Nobody trained a GPT-like model from scratch for $600 - this experiment took advantage of the millions of USD used to train the larger models.