[metanonsense]

I am out of the game for a year or so (and was never completely in the game), but back then BERT was the basis for lots of interesting applications. The original Vision Transformer (ViT) was based (or at least inspired by) BERT, it was used for graph transformers, visual language understanding, etc.