[tmjdev]

This looks like the video equivalent of Dall-E 1. Hard to believe how far we've come so quickly.

The paper talks about "pseudo 3D attention layers" that are used in place of temporal attention layers for each dimension due to memory consumption. It seems like AI research is vastly outpacing GPU development.

[londons_explore]

Indeed - it's not hard from a research point of view - it's hard from a compute perspective because adding one more dimension requires hundreds of times more compute.

Even then, these videos are only like 50 frames long - and a real movie you would want to be hundreds of thousands of frames long.

[Filligree]

So you need to optimise on compressed version, not the whole thing. What they’re doing right now is akin to a human trying to hold an entire picture - or entire movie - in their head all at once.

We can’t do it. AIs can sort of do it.

Latent diffusion models already demonstrated that operating on a compressed representation gives far better results, faster, but I don’t think we’re anywhere near the limit for what’s possible there. It’s no coincidence that this is how humans work.

[elephanlemon]

>and a real movie you would want to be hundreds of thousands of frames long.

Yes, but consider that most films are made up of many different shots, each of which are often just seconds long.

[tiborsaas]

Hardware was probably always lagging behind cutting edge research, just consider video games, they pushed hardware limitations very hard since Pong.

It's a good thing to be fair, forcing research teams to optimize their projects is beneficial and creates a competition for limited resources. This gets a bit skewed when we consider a university research team vs. a MANGA type company, but the team behind Stable diffusion proved that innovation can come from unexpected places.

[elil17]

Looks a bit better than DALLE1 IMHO. They've demonstrated greater range.

[htrp]

i wonder how much vram these models cost ?