[pkroll]

You're not the only one thinking that: https://www.nvidia.com/en-us/project-digits/

128G of unified memory. $3K. Throw ollama and ComfyUI on that sucker and things could get interesting. The question is how much slower than a 5090, is this gonna be? The memory bandwidth isn't going to match a 512 bit bus.

[KeplerBoy]

It's going to be waaay slower than a 5090. We're looking at something like 60W TDP for the entire system vs 600W for a 5090 GPU.

It's going to be very energy efficient, it will get plenty of flops, but they won't be able to cheat physics.

[lostmsu]

AFAIK this uses even slower memory.

[sroussey]

And a fraction of the 5090 cores.

[Keyframe]

I think digits is STARTS AT $3k. We'll see.

[manojlds]

It's LPDDR5.

[ein0p]

That's actually a good thing. That's how you get a ton of DRAM without it costing a fortune. M2 Ultra is able to get GPU-like 800GB/sec with DDR4. From that it follows that if you can design a specialized chip, you can get a respectable 1 TB/sec quite easily with LPDDR5, provided that you're willing to design a chip to support a ton of memory channels (and potentially also a wider memory bus). In fact, I'm baffled that such devices don't already exist outside Apple's product line. Seems like a rather obvious thing to do, and Apple has a "proof of concept" already. I can think of at least four companies off the top of my head that could do it quite easily, besides Apple.