This is not true. She does discuss how some countries and regions of the world use the Internet less, or not at all, and thus Transformer/BERT are training on the linguistic habits of the most affluent of the world.
It was still an extremely scattergun approach. I'm moderately familiar with NLP but nothing in that paper really had traction overall, it was more of a list of mildly bad things than anything huge.
urthor|5 years ago