Honestly it's a large enough library with enough weirdness and untested areas, footguns, and bugs that I'd deem it just as valid as React for example.
Why did tensor_parallel have output += mod instead of output = output + mod? (The += breaks backprop). Nobody tested it! A user had to notice it was broken and make a PR!
For an uni course I tried to fine tune Gemma in a few days, it wasn't easy because tutorials often were written with old version of hf libraries that now work differently, there's a lot of areas to improve, everything still seems kinda fresh and so it's a pain in the ass to deviate from simple walkthroughs to something tailored to your needs.
buildbot|6 months ago
Why did tensor_parallel have output += mod instead of output = output + mod? (The += breaks backprop). Nobody tested it! A user had to notice it was broken and make a PR!
amarcheschi|6 months ago