(no title)
gala8y
|
2 months ago
Can you list some useful things you can do with such models which are beyond 'fancy' use, like image generation, or standard chat (which is subpar compared to frontier)?
I use my RTX4070 (12VRAM/64RAM) mostly for STT, though I am having real trouble to set up working environment for any Whisper derivatives after migrating to Fedora.
mindcrash|2 months ago
One of the things I am currently experimenting with is building out my own agentic/assisted computing environment which instead of extending into Google/Microsoft/Apple owned cloud based services, extend into services which run on my homelab environment instead.
As a simple example: A local model which can hook into a MCP service making it understand calendars and appointments which hooks into my own locally hosted Radicale CalDAV service, enabling me to quickly make a appointment through text (or possibly even STT later). I'm curious how much I can get something like Thunderbird to disappear.
A somewhat advanced example: Another thing which recently popped up as a idea, I'm quite excited about and I hope will work out is that I can teach a model the concept of a "package repository", a "package manager" and "systems", which (hopefully) means I can install, uninstall, update and track the status of software packages on my Linux systems without using the terminal or shelling into a system myself.
Summarized: I think some things Big Tech wants are pretty neat, but I would like something without heavy involvement of Big Tech (and/or subscription based computing) instead.
gala8y|2 months ago
What I can see myself trying to do is some new ways of working with body of text notes. Local RAG for chatting with documents is also interesting.
And yes, with 'subscription based computing' shreds of privacy we had are gone.