[freefaler]

Well try to listen to a group theory lecture (for example on cohomology of groups) while doing chores :) But the lecture was indeed useful if you stop and rewind and see how the lecturer was explaining (there were some interesting graphs).

Your brain can't hold the context long enough to go to the required level of abstraction, while you're multitasking (may be walking or something deeply automated doesn't count.

[griffzhowl]

I would say the main reason it's hard to follow abstract maths lectures by listening is not the difficulty of the concepts per se, but simply because it's so visual: it relies on notation and diagrams

But mathematicians can talk to each other about arbitrarily abstract concepts, as long as they have enough shared background, and they don't (always) need a blackboard to do it.

Conversely, you can have conceptually very simple things that are basically impossible to follow just by listening, like multiplying two nine-digit numbers or following one of Euclid's proofs in plane geometry. The difficulty isn't about abstraction, but how many things you have to hold in working memory