[nirvana]

The answer is that lightening is a general purpose next generation serial format.

This allows them to support different formats in the future, for instance, say when HDMI isn't popular.

The 30 pin connector didn't have HDMI on it at the time it was created and they have repurposed pins over the years regularly.

Here they can have a much less complicated solution-- a standardised serial bus over which they can stream any future protocol they want to support.

It's a lot more future proof.

[DannyBee]

This isn't an answer, and in fact, supports the opposite of your point :)

It would have been just as easy to use micro-usb 3.0, which had plenty of bandwidth, and do the exact same pin or format conversions.

Standardized serial buses that are less complicated already existed. What exactly do you think USB 3.0 is?

If the only reason for lighting was "a general purpose next generation serial format", then it is definitely a horrible idea.

Of course, none of this (except DRM concerns) answers why they'd not bitstream the format and convert it to HDMI signaling instead of doing the weird crap they did here.