[ejddhbrbrrnrn]

Is this a general issue rather than unsloth specific. How wide is this problem? Sounds wild if it has been affecting everyones training.

[danielhanchen]

Unfortunately it's not an Unsloth issue but a general issue affecting nearly all trainers which use grad accum. We worked with Huggingface so their trainers should be fixed now though in the main branch