This is what other people have found over on reddit: https://reddit.com/r/StableDiffusion/comments/xzipjx/automat...
According to that comment, this (https://user-images.githubusercontent.com/23345188/194727572...) is allegedly the only reasonable match with the leaked code from NovelAI, but note that I have not verified that myself, and since NovelAI/Stability never said which part they take issue with it is hard to tell.
This is the file in that other repo that the code actually seems to originate from https://github.com/lucidrains/perceiver-pytorch/blame/main/p...As you can see, that repo from 2 years ago even originates the "# attention, what we cannot get enough of" comment and is an exact 1:1 match to Automatics commit, while the one from NovelAI even has a small change in the if clause that Automatic doesn't have.
nobo122|3 years ago