[emurlin]

  > Many valuable, high-quality pieces of content that people would find useful never make it into Google's index.

I see your point in the light of the article (not indexed = not visible), but it feels like the things that _do_ make it need to follow very particular content and style patterns to rank high.

Anecdotally, this observation comes from searching for any term and seeing the results: they are usually similar-looking plausible-looking-but-actually-low-quality results that seem to follow the same or similar structure and have the same content. This does indeed limit the diversity and depth of information, but I'm not so sure it reduces spam, as these low-quality sites seem to be as prevalent as ever before, if not more.

From experience writing articles to a small tech blog, this means that it's quite difficult to get well-researched articles to rank well, even if they're indexed.

For example, I've written an article on how to block hotlinking (I've just checked, and Google says it's indexed). If you search for this, my article on a not-so-well-known blog is nowhere to be found(*)(**), and this is somewhat expected, for a myriad of reasons. The problem isn't that my post doesn't rank, but rather that none of the top-ranking (or even not-so-top-ranking) results are wrong. They are either about how to do this on cPanel or whatever, which is ineffective (but granted, could be what people are looking for), or instructions using the `Referer` header, which is ineffective.

These days, browsers offer headers like `Cross-Origin-Resource-Policy` which can completely solve the particular issue of hotlinking, unlike `Referer` which is easily bypassed using `referrerpolicy="no-referrer"`. However, because most 'authorities' seem to be wrong on this issue, the correct result isn't displayed, because it's a hard problem to solve algorithmically (or even manually).

(*) This doesn't affect just Google, though.

(**) Because it's indexed, adding the right keywords (which you wouldn't do in this case unless you already knew the answer) does bring it up, although from federated high-authority sites instead of the original canonical source.