Any external device would need to be cheaper and/or perform better than a current gen iPhone/android device onboard. Seems like a limited market. What would the benefits be over a spec'd out mobile pc doing the same thing?
Not necessarily. Separating the battery, not having to deal with the OS killing inference because of memory pressure (which will be a real problem), being able to schedule tasks (a la Grok), and having more freedom to let the LLM actually manage files that you can't get on iOS, and active cooling are all very good reasons to have a dedicated piece of hardware.
msgodel|7 months ago