[gbickford]

Have you tried generating two sets of qapairs, one with bad answers, and using DPO?

[lewq]

Not yet, sounds promising!