[apike]

> Also "generate some better examples" sounds like fudging data to fit the expected outcome.

LLMs are tools. As a tool author, you have certain desired outcomes for certain use cases. If the current data you’re training on isn’t giving you those outcomes, it is absolutely reasonable to "fudge" the data. This might mean reducing bias, or adding bias, or any number of nudges. Training an LLM is not a scientific study, it’s a product development effort.

[surfingdino]

Agreed. However, you are then giving your tools to people who have none of that experience and understanding and apply it to the problems they are trying to solve without then taking a pause and checking the results against facts. There is a lot of trust in the outputs and little vigilance. A common reply to such concerns is "well, you should be able to spot incorrect information in the outputs" which is tricky if we are talking about education where by definition students are yet to learn correct answers or lower levels of career development, very much similar to education when they are learning on the job. The lack of ability to quote and trace sources of information used to construct output by an LLM is a major red flag for me, sensitive information leakage is another. They way LLMs are sold is irresponsible, they are sold as tools to solve problems, not as a thousand monkeys trying to type up the whole works of Shakespeare, which isv where we are at the moment.