top | item 43965017

Ask HN: Can On-device AI solve projected energy crisis?

1 points| vkkhare | 9 months ago

3 comments

bigyabai|9 months ago

No, on-device AI is not a replacement product for the high-consumption AI that is casuing the problem.

vkkhare|9 months ago

Isn't the inference cost of running these models at scale challenging? Currently it feels like small LLMs (1B-4B) are able to perform well for simpler agentic workfows. There are definitely some constraints but surely much easier than to pay for big clusters on cloud running for these tasks. I believe it distributes the cost more uniformly