top | item 46682881 (no title) arbuge | 1 month ago Perhaps somebody more familiar with HF can explain this to me... I'm not too sure what's going on here:https://huggingface.co/inference/models?model=zai-org%2FGLM-... discuss order hn newest Mattwmaster58|1 month ago I assume you're talking about 50t/s? My guess is that providers are poorly managing resources.Slow inference is also present on z.ai, eyeballing it the 4.7 flash model was twice as slow as regular 4.7 right now. arbuge|1 month ago None of it makes much sense. The model labelled as fastest has much higher latency. The one labelled as cheapest costs something, whereas the other one appears to be free (price is blank). Context on that one is blank and also unclear.
Mattwmaster58|1 month ago I assume you're talking about 50t/s? My guess is that providers are poorly managing resources.Slow inference is also present on z.ai, eyeballing it the 4.7 flash model was twice as slow as regular 4.7 right now. arbuge|1 month ago None of it makes much sense. The model labelled as fastest has much higher latency. The one labelled as cheapest costs something, whereas the other one appears to be free (price is blank). Context on that one is blank and also unclear.
arbuge|1 month ago None of it makes much sense. The model labelled as fastest has much higher latency. The one labelled as cheapest costs something, whereas the other one appears to be free (price is blank). Context on that one is blank and also unclear.
Mattwmaster58|1 month ago
Slow inference is also present on z.ai, eyeballing it the 4.7 flash model was twice as slow as regular 4.7 right now.
arbuge|1 month ago