[addy_osmani]

I lead engineering for some of Google's developer tooling like Lighthouse. You are correct that we currently have a gap in our ability to measure subframes in the lab locally on your machine (doing so cross-origin reliably is complex), but we're looking at how we can improve this. The important bit I want to call out here is that "framehole" is a measurement gap in LH: you can currently game the score but you're only fooling yourself in the process — there is no benefit to the user experience.

Speaking of real world user experience, accounting for iframes has a number of complicated privacy and security challenges at the resolution of an individual page load. However, the good news is that the Chrome User Experience Report* aggregates anonymized data from many opted-in users and, as a result, is able to account for iframes in its measurement. Which is to say, the field data we surface in our tools (powered by CrUX) does not have this problem. We’re working to get our lab tools to align with this behavior as well.

* https://developers.google.com/web/tools/chrome-user-experien...

[nathanyz]

I appreciate the reasoned and thought out reply. At the engineering level, I get why capturing the results from the iframe is challenging and that its just a gap in the measurement by the tools.

Here is the problem though. When these new measurements roll out to Pagespeed Insights, that score will be treated like the holy bible of performance. Most who use and make decisions off of that score will not read nor understand the engineering level nuance that it doesn't take into account Iframe content.

That means that all over the world, people will be optimizing for the score even if the score is incorrect. Decisions will be made on whether the score goes up or down when using various tools, including our service. As it stands, the score ignores iframed content. So it will ALWAYS be the case that using a 3rd party embed of some content will lead to a better score than natively including the content on the page. Anything embedded from a third party simply disappears in the scoring.

While I don't believe that this choice is being made on purpose to favor Google properties, at the end of the day that is exactly what happens. YouTube embeds improve page speed, slow loading advert iframes, don't impact the score.

So we are trying to shine some light on exactly how important this now is, and how at the extreme excluding iframes can be used to show a score of 100 even when nothing else has been done to improve the actual speed. If there are engineering challenges that prevent counting iframes as of yet, then I would submit that the score is not ready for primetime until that can be worked through. It may not have seemed like a big enough issue, but hopefully I can convince you that it is.