[edkennedy]

I'm really hoping for the ability to load in different sets of trained material as embeddings/textual inversions like in Stable Diffusion. Imagine scanning in some of your favorite philosophy and design books and throwing them with small weighting as a little flavor to your answer. The crossovers between LLM and Stable Diffusion type models (like Loras) is such a fascinating space to explore.

[alex_sf]

This already exists for LLaMA, I'm sure it'll be implemented for StableLM as well.

https://github.com/lxe/simple-llm-finetuner

[pksebben]

good looks on the link! I was experimenting with script writing the other day and thought "gee, I really wish I could finetune on Beckett plays specifically".

I don't know if anyone else has experienced this same tipping point, but when I used to have ideas, I would look them up and discover that implementing them was probably out of scope. These days, I think "wouldn't it be cool..." and immediately stumble on a way to make it happen, by accident.

[cheald]

This has been around for GPT models for a while in the form of "soft prompts", which are rather approximate to textual inversion in the Stable Diffusion space.