"Attention is all you need" paper digested (2018)

[wanderingstan]

For a higher level shorter overview, I found this video informative: https://youtu.be/SZorAJ4I-sA?si=pnfzZ17PYQfV4aqq

[personjerry]

IMO understanding this kind of content depends on where you're starting out, or you're getting too much or too little explanation of the supporting components.

Personally, I used ChatGPT to help me understand this paper from the stance of a senior engineer with some ML experience, and followed up to clarify where I had questions: https://chat.openai.com/share/c17dbcfc-7e3a-44fd-aa88-28c117...

[alexfromapex]

Attention is super helpful when you have enough data but it didn’t really help me on a problem with a tiny dataset

[machinelearning]

https://www.askyoutube.ai/share/6521aa6077733b7c0ad24b55

[piyh]

I need this breakdown, I tried to read the paper but it's not accessible to a more than average interested layman

[dzign]

Google regrets publishing this paper!

[random3]

Maybe, but it would have surfaced regardless, either directly or through related things. While the transformer may evolve into the next thing, it's equaly likely the next evolution will be unrelated to transformer.

Moreover while the transformers and current LLMs are a leap, the monoculture around them is not necessarily a good thing, defocusing many good researchers from otherwise promising tech.

Finally, cross-polination of ideas is where the magic happens.

[mensetmanusman]

It’s probably Nobel worth based on its world impact.

[Alifatisk]

Here’s the thing, if google didn’t publish this paper, it would probably just collect dust somewhere at Google.

OpenAi (ClosedAi) saw the potential and demonstrated its impressive and unbelievable capabilities!

If Google never shared the transformer model with the world, we would probably not have what we have today.

[abhishekjha]

Not sure if sarcasm or not.

[artninja1988]

Love Yannic

[behnamoh]

Actually, all you need is all you need—anything that claims otherwise is just wrong. In other words, you need all the things that you need.