[Dorialexander]

Already in the work. Just had a meeting today with two latinists about it.

[LightMachine]

Why don't you restrict the training just to old text? That would be awesome

[Dorialexander]

Not feasible to go with pretraining only.

What is possible is to use a larger learning rate but this will be a hard trade-off with conversational capacities. Fine tuning is currently based on original texts with a synthetic prompt. The issues that people have noticed (repetitions, not remembering what was in the prompt) will be more significant if the learning rate is higher.

Maybe a solution will be to provide two different variant of the same model, one less immersive and more workable, and the other more immersive and buggy.

[isoprophlex]

I'm not sure you'd be able to scrape together enough data for that to be viable...

[ekianjo]

Awesome!!