[tristanz]

Collaborative incremental improvement of models would be extremely disruptive. While this happens via research, it's massively inefficient, particularly as pretrained models get larger and span multiple modalities.

[fennecfoxen]

I assume you mean "disruptive" in the sense that it would enable an uninteresting status quo to be replaced with something more dynamic, rather than actually disrupting peoples' work with models.

[robbedpeter]

Interesting - the modular idea is one of the most interesting to me. The recent hierarchical transformers papers hint that models can be made smaller and might open the door to modular approaches, which could mean highly nuanced customization of your domains of interest, and fitting the model size to the capacity of consumer hardware like phones.

Thanks for the effort you're putting into this!