[foxes]

I feel its not that bad an abuse of notation as kinda consistent with other areas of mathematics -

A coset, quotients r + I, affine subspaces v + W, etc. Not literal sets but comparing some representative with a class label, and the `=, +` is defined not just on the actual objects but on some other structure used to make some comparison too.

[mathgradthrow]

No. Equality is defined on sets, with exactly one exception, this stupid fucking notation.

[foxes]

Notation is invented to be useful. A lot of mathematicians and physicists write stuff like

f = g + O(h)

and then proceed to do a bunch of manipulations without getting bogged down in set theory syntax.

Also equality isnt just defined on sets, usually people start by defining it as like a formal thing satisfying a bunch of rules as a part of their logic, and going from there.