[tarruda]

Would love to see a Qwen 3.5 release in the range of 80-110B which would be perfect for 128GB devices. While Qwen3-Next is 80b, it unfortunately doesn't have a vision encoder.

[Tepix]

Have you thought about getting a second 128GB device? Open weights models are rapidly increasing in size, unfortunately.

[tarruda]

Considered getting a 512G mac studio, but I don't like Apple devices due to the closed software stack. I would never have gotten this Mac Studio if Strix Halo existed mid 2024.

For now I will just wait for AMD or Intel to release a x86 platform with 256G of unified memory, which would allow me to run larger models and stick to Linux as the inference platform.

[PlatoIsADisease]

Why 128GB?

At 80B, you could do 2 A6000s.

What device is 128gb?

[the_pwner224]

AMD Strix Halo / Ryzen AI Max+ (in the Asus Flow Z13 13 inch "gaming" tablet as well as the Framework Desktop) has 128 GB of shared APU memory.

[tgtweak]

Spark DGX and any A10 devices, strix halo with max memory config, several mac mini/mac studio configs, HP ZBook Ultra G1a, most servers

If you're targeting end user devices then a more reasonable target is 20GB VRAM since there are quite a lot of gpu/ram/APU combinations in that range. (orders of magnitude more than 128GB).

[kristianp]

By A6000, do you mean the older Ampere generation model? 48 GB ddr6, released 2020 [1]. Can you even buy those new still?

[1] https://www.techpowerup.com/gpu-specs/rtx-a6000.c3686

[lm28469]

That's the maximum you can get for $3k-$4k with ryzen max+ 395 and apple studio Ms. They're cheaper than dedicated GPUs by far.

[tarruda]

Mac Studios or Strix Halo. GPT-OSS 120b, Qwen3-Next, Step 3.5-Flash all work great on a M1 Ultra.

[sowbug]

All the GB10-based devices -- DGX Spark, Dell Pro Max, etc.

[vladovskiy]

Guess, it is mac m series

[bytesandbits]

maybe a deepseek v4 distill. give it a few days