I think your conclusion is too strong. Yes, we know bigger models and data generally lead to better performance (e.g. BigGAN results last year) but progress in architecture can still speed up progress in ML tasks. If we were still stuck using RNNs and LSTMs for language modeling we wouldn't be talking about this news today.
No comments yet.