[iamnafets]

You can test this. Stratify users into groups with varying ad load and watch retention metrics. There's a bunch of 2nd-order effects that are hard to control, but you don't get to be a business doing billions in revenue without fine-grained understanding here.

[freyfogle]

Oh of course. No doubt the techniques have advanced, but all that was possible even 20 years ago.

The question is what do you do when you don't get the answer you want? - when it shows that you are losing audience, though perhaps slowly. It is very difficult/impossible for a manager to say "let's miss this month's revenue goal for the long term benefit of slowing audience loss". And so instead you come with other reasons to explain the audience loss and/or justify keeping the ads. And so the audience loss continues. You are the boiling frog.

[bombcar]

Or you don’t do this on the downward side because the execs won’t like the answer it gives.

Certain parts of a datalake are not for fishing in.