[opprobium]

Not just efficiently, can't solve.

[logicchains]

They can solve it if you keep adding layers to the transformer, it's just not efficient; you'd need exponentially more layers than a similarly sized RNN.