top | item 36327210

The Reddit blackout has left Google barren and full of holes

476 points| pg_1234 | 2 years ago |techradar.com

424 comments

order
[+] hourago|2 years ago|reply
The problem is that the open internet is dying. What used to be blogs and forums now is Reddit. Tens of thousands (hundreds of thousands?) of communities that had their own wiki, blog, forum, ... are now reduced to be a Subreddit.

The Internet has become fragile. One service goes down and everybody suffers. If the top 10 services went down most people would think that there is no Internet at all.

E-Mail is the last standing service that is way more open that the rest. But the raise of Whatsapp and equivalents are challenging that. One day all our communication will depend on a monopoly. We are starting to know what would have happened if AT&T have never been split.

[+] supernikio2|2 years ago|reply
It's getting worse by the minute, too. What used to be a subreddit might now be a Discord server, which is in my opinion even worse, keeping solutions/questions to certain issues within Discord's own walls.

You now have to enter a server, search for your specific problem, and, if not found, ask someone and wait for them to respond. If results are on the first page of google it's much more friction-less and convenient imho.

[+] lisper|2 years ago|reply
Email is dying too. I have run my own email server for 20 years. A few years ago my original ISP was acquired, forcing me to switch providers and hence get a new IP address. Ever since then my server has been blacklisted by one major player or another as a source of spam. I even tried changing to a new IP address, but the result of that was just a different set of providers blacklisting me. I'm about ready to throw in the towel and switch to Fastmail. :-(
[+] TrackerFF|2 years ago|reply
I still use a couple "old school" forums, but I have to say - the reddit tree format is just so, so much easier. It is impossible to de-rail a thread, and there's a certain level of anti-spam and self-moderation mechanism to the reddit format. The older forums still need some hardcore moderation, or else many threads become unreadable. Nothing worse than when two users start arguing, and the thread turns into shit.

But that's just the how things have evolved. It's so much easier to just start a subreddit, than to host your own forum.

[+] willio58|2 years ago|reply
I think what's happening with Twitter and Reddit is actually a good thing in that it's exposing the failures of centralized networks. The thing is, these failures aren't big enough yet for people to jump ship and I think the main problem is none of the decentralized social networks have provided a solution that's better.

Mastodon is theoretically a better version of Twitter, but normal people get lost the moment someone starts talking about "instances". It's this gap we need to cross somehow. How do networks provide all or most of the pros of being centralized without being centralized?

[+] JohnBooty|2 years ago|reply

    the open internet is dying. What used to be blogs and forums now is Reddit.
How "open" were those blogs and forums, though in a functional sense?

Instead of an untrustworthy/unreliable corporation like Twitter or Reddit owning a data and community.... we used to have a zillion untrustworthy/unreliable individual "webmasters" owning those blogs and forums. "Your" "community" might be gone in the blink of an eye because the guy running it threw a hissy fit, or forgot to pay his hosting bill.

All other things being equal, the current near-monoculture does suck worse. But the old days sort of sucked too.

[+] mrtksn|2 years ago|reply
> are now reduced to be a Subreddit.

I don't know if anything else was possible, the IT need for communities is about the same and it makes total sense to have those on one place. It solves the infrastructure problem, it solves discoverability problem, it solves the user account management problem and it is all "free".

It comes at the cost of fragility, true, but as always convenience is the king.

[+] AndrewKemendo|2 years ago|reply
Honestly I’d LOVE to revert to the old phpBB or vBulletin forum days

There’s literally no reason why not and it’s never been cheaper to run

[+] fennecfoxy|2 years ago|reply
Ah yes, signing up to a dozen different forums. Don't want to go back to that, either.

Tbh we've kind of failed to unify software properly.

Hardware is semi unified, Apple's efforts besides, but things like Wifi/Ethernet TCP/IP, HDMI, USB C etc all tie things together decently well now (but it could still be better).

But when it comes to software...why don't we have a universal identity/login system that everyone & everything supports?

Why don't governments enforce/mandate hardware/software compatibility for (larger?) companies. It should be like "if you want to support your own custom login/identity you can but you must also support minimal x"

Wifi for example, as a standard isn't legally enforced, it's purely supported for business reasons. If Apple could get away with rolling their own that only works with iDevices I'm sure they would. But if the usage purely for economical reasons disappears, we need law to force companies to use a minimal standard for interoperability.

[+] Quarrel|2 years ago|reply
A big part is that a lot of the internet isn't available to google anymore.

Lots of good forums just have a robots.txt with "Disallow: /". Google can be a pain. People use DMCA in flagrantly illegal ways, etc, so lots of places just opt out of being easily found.

Then, as others have said, there are the discord etc, or irc style, that have never been very searchable, but have lots of good content.

It is fracturing back to more like it used to be?

[+] musha68k|2 years ago|reply
It’s also a clash between the profit motive of these platform runners / investors and the increasing value of data itself (in the “age of AI”).

Enthusiasts need our help as Usenet as is, phpBB, etc just don’t cut it anymore in terms of ease of use for admins and exploration and fun for users.

Todos then:

1. Centralized solution it ain’t

2. Remove (boundless, blunt) profit motive out of equation (don’t depend on this at least)

3. Actually care and be professional about UX. UX quality for centralized services has been decreasing over the last ~10 years (I blame Apple for continuously dropping their own ball there; role models are important) so it’s actually not as high a barrier as it used to be (relatively speaking).

[+] paulpauper|2 years ago|reply
The problem is that the open internet is dying. What used to be blogs and forums now is Reddit. Tens of thousands (hundreds of thousands?) of communities that had their own wiki, blog, forum, ... are now reduced to be a Subreddit.

Some people argue that blogs are the answer to a more centralized or impersonal web. I disagree. Reddit answers and comments are better than blogs, which are full of ads, SEO-spam, signup forms, and generic and vague copywritten filler . Reddit comments are written by actual humans who are either looking for answers, or are supplying incisive answers or explanations, not filler. And most importantly, based on unbiased, personal, first-hand experiences. Many bloggers are paid to promote products and are not fully honest, or do not use the products they endorse. If you want to know if interment fasting works, instead of reading a vague blog about the benefits of fasting or having to sign up to a newsletter, why not read actual people's experiences with it.

[+] notacoward|2 years ago|reply
> What used to be blogs and forums now is Reddit.

One might quibble about blogs being Reddit when Tumblr and Medium and Substack exist, or forums being Reddit when Facebook groups exist, but that would be missing the point. Whether the number is one or a dozen, or even several dozen, it's not like the thousands upon thousands of independent sites that used to exist. While openness and distributed ownership/control are not the same thing, it seems like it's hard for the former to survive without the latter.

That's why so many people are looking to the fediverse and its imitators to replace the falling monoliths. There are problems there as well, both technical and social, but at least there's hope that they can grow into solutions without being assassinated by a single rich psycho. All it takes is for people to stop thought-experimenting on yet another rich-guy-owned monolith and start actually hacking toward a permanent and public solution.

[+] judge2020|2 years ago|reply
Centralization was always inevitable with scale.

New users aren't going to spend the same amount of time and brainpower managing their internet presence as OGs do/have. So they will choose the services that enable them to reduce the overhead of using the internet. For many, this doesn't just mean using Reddit for their interest-based communities, but using Gmail for spam prevention, Apple products for their curated store, Steam for ease of access, etc.

This would only be solvable if you had government-forced interop so that e.g. Twitter posts were accessible on Mastodon timelines with no cost to the site operators, but even then any small barrier is something for a platform to optimize and sell in the form of a subscription/using their user base for ads.

[+] kafkaesque|2 years ago|reply
Here is an idea: forums are easy to build. What if people got back into them and just hosted them on their domain and then there was a webring attached to all forums like sites were back in the 90s, except there would now be a main site that would serve as a directory of forums divided by interest. so everything would be owned separately: the directory and each forum, but all connected through a webring like it was before. It would "decentralize" the Web in some ways (except for there being a directory).
[+] EscapeFromNY|2 years ago|reply
My theory is that Google lost the spam war. All the leetcode in the world won't teach someone to build an ML model that can tell whether a website is spam. So Google outsourced the spam detection problem -- they heavily bias their results towards only the most popular sites, who either have human moderators or paid contributors, and those sites do the spam management that Google's automated approach is incapable of.
[+] falcolas|2 years ago|reply
Google's recency bias in search results hurts them too. There are many older resources which are still valid out there, but you won't find them (easily) via Google Search. Instead, you get the SEO spam which doesn't match the search as well, but is newer.
[+] diegocg|2 years ago|reply
It is more than the spam war...a lot of content is just no longer produced and exposed to the open web (think about how much content goes into tiktok, discord, etc and you will never get that into your search results). Google has less useful content to index, algorithms can't fix that. There is more spam only because that's the only open content that gets added massively.

The winners of this battle will be the places where content is generated (or curated) - and reddit is perhaps the most important content hub generator (it's not just an aggregator anymore - comments about some news can be often more interesting than the news itself). Indexers (and language models) are useless without content to scrap.

[+] Guest9081239812|2 years ago|reply
I think this causes further problems as well. These big companies know they can easily be listed at the top of Google, and therefore they pump out low quality articles for every popular key phrase. They can write a "Top 10 X of 2023" list and bring in lots of traffic and referral sales, while a smaller site has no chance of doing the same. Then, the big site takes all that income, and pumps out more low quality articles. It's a rich get richer scenario.

If I search, "The top 10 movies on Netflix", I get sent to this page...

https://decider.com/list/top-10-movies-on-netflix-most-popul...

The top 5 movies above are...

1. The Boss Baby (2017)

2. The Angry Birds Movie (2016)

3. We're the Millers (2013)

4. Zookeeper (2011)

5. Mean Girls (2004)

Is this really the best the world has to offer for my search query? I could spend a week putting together a far better page for that query, but what's the point? It's not going to rank anywhere notable in the Google results. There's no incentive for me to produce that quality content.

[+] deeviant|2 years ago|reply
The real problem isn't that Google can't detect what spam is, it's that this spam drives so much of the traffic, and thus their revenue that they cannot remove it.

Google at this point is literally teaming up with spammers. Just go on a android phone and swipe left from homepage for "Google", count how many clicks it takes to get to an 100% AI generated spam page.

[+] s1artibartfast|2 years ago|reply
I wonder if Google wants to prioritize video to try to address this spam problem.

Sometimes when I search a topic all I get is video in the natural results, not a video search, and I suspect I'm part of an A versus B test.

I personally don't find it to be a compelling solution and click away because I'm not willing to sift through 10 10 minute videos to see if there is relevant content.

[+] choppaface|2 years ago|reply
Google lost the spam war when Matt Cutts left. Google effectively lost its community outreach, and simultaneously pivoted to aggressive moat protection through AMP, Chrome, Jedi Blue, etc. It’s less that the spammers won and more that Google lost their aim at high-quality content.
[+] bamfly|2 years ago|reply
For sure. They threw in the towel around '08 or '09. The prior years of yo-yoing between front page results containing ~60% webspam and 0% webspam settled into a permanent 60%, and they evidently just heavily downranked and/or started ignoring low-traffic sites to keep it from growing to 100%.

Their shifting from text-only "ethical" ads to being a more-ordinary web ad service, and putting ads inline with results, roughly corresponds with that, IIRC, which probably isn't a coincidence. Your ad-"results" probably get more clicks if much of the rest of first result page is crap. Many of the webspammers probably funnel money to Google one way or another, now. It screwed up their incentives to keep fighting that battle, I'd guess, which is likely part of why they stopped putting so much effort into it.

[+] eastof|2 years ago|reply
Maybe the ML model is predicting that popular curated sites generally have less spam?
[+] xnx|2 years ago|reply
Expecting Google's code to flawlessly understand the truth and trustworthiness of all possible sources for every conceivable question is a fair/unbiased way for all topics past present and future is a very high bar. This is a really hard problem, and unlike many other hard problems (rocket science, neurosurgery, particle physics, etc.), Google has millions of financially motivated adversaries actively trying to trick them.
[+] s1artibartfast|2 years ago|reply
Google won the spam War, they were just fighting for the opposite side. I know it's trite but it bears repeating: Google only cares about user satisfaction if it positively impacts their bottom line.

As long as they can monetize remaining users more than what they lose from users abandoning search, the trend will continue

[+] durumu|2 years ago|reply
I mean I think you’re mostly right but to be fair to Google it’s an incredibly adversarial. Thousands of people have made it their full time job to try to trick Google into thinking their blogspam is good content.
[+] pnathan|2 years ago|reply
I do agree that Google lost the spam war.

This is particularly notable when looking for recipes.

[+] pradn|2 years ago|reply
Google can't index an internet that doesn't exist. The open web is truly dying.

I believe in the open web, but it seems like an ideological position, not something most people want to participate in. (Just like decentralization!) There's a few reasons:

1. Putting things on the web means your words are public, so they can be archived forever, and anyone can look at them. Do I want employers looking at my high school blog? (Hell, mine is still up, go look it up and you'll find things I no longer believe are appropriate!) Recall that keeping posts private was an important innovation w/ Facebook, over MySpace, which was largely all-public.

2. Most people don't want to bother with hosting their own website. It costs money and requires expertise to run. And it can easily go awry (DDOS attacks! getting hacked! losing your domain!).

3. If you don't want to bother with hosting your own site, you are at the mercy of platforms that always (always!) devolve into bloat, over-commercialization, and perhaps censorship. Medium, Reddit, Wordpress.com, Tumblr - you name it!

4. Most people don't want to put the effort into writing/posting anything beyond basic food/vacation photos or whatnot. Even the people hyper-engaged in politics online are a minority. How often did people write a diary, even, before the internet. Putting in this sort of effort is not something most people want to do. I don't blame them - life's busy and work/school takes the lion's share of everyone's energy. Look at me - all I do is post comments on HN! I haven't updated my two-page static website in years!

5. It's lonely running a website! There's no guaranteed engagement, as there is with posting an Insta story or a Facebook profile picture. It really is like shouting into the void. Yes, one can join a web-ring, but you could also make your own butter...

[+] SpacePortKnight|2 years ago|reply
I noticed this too when I was trying to search for local opinions like affordable clothing in netherlands and for general recommendations like cloud storage with execellent file management.

Without reddit, outdated generic blogs with affliate links show up. They are just advertisement in disguise and not the genuine opinions which I'm looking for.

In a way this shows the value of Reddit. Even with their awful UI and app, their users seems to provide the most value to search queries.

[+] sleepytimetea|2 years ago|reply
I search for reviews of DIY tools and all Google shows me are review sites that are just thinly veiled affiliate link farms. I even read a post somewhere about a guy boasting he has SEO optimized his review sites to show up at the top of google searches and he makes a bundle on all the Amazon affiliate links people follow.

For me, the solution is not Reddit but Youtube - luckily, Google can now link you directly to the point in the video that matters otherwise videos are not random searchable, and I haven't seen all videos have text transcripts you can search through.

[+] moi2388|2 years ago|reply
I think affiliate deals should be banned. I hate it that much. Whilst you’re at it, van advertisement without consent. You can use the kids argument if necessary.
[+] bufio|2 years ago|reply
The mainstream web seems to be crumpling under the weight of its enshittification.
[+] bioemerl|2 years ago|reply
I hope this is finally a wakeup call to Google to try to start to prioritize non reddit user content on the internet again. Web forums have been suppressed by their results for ages now and Google's choice to do that has been a big part of what killed the internet.
[+] aurareturn|2 years ago|reply
We're going to get more Quora results, huh? Please come back Reddit and save us.

Quora results are the worst. I can't believe how much management ruined Quora.

[+] Guest9081239812|2 years ago|reply
Reddit should have leveraged their content to compete with Google.

1. Improve old.reddit.com instead of creating this terrible Reddit v2 that encourages replacing real content with memes.

2. Build a dedicated Reddit search engine on a short trendy domain.

3. Remove all Reddit content from Google.

4. Carefully curate other quality sites into the search engine.

5. Leave Google to rot with SEO and Amazon referral spam.

[+] vbezhenar|2 years ago|reply
Google should be more pro-active in checking its results health and present cached versions if underlying sources are gone. It's not only about reddit, I also have plenty of websites which don't open in my computer (either because my country banned them or because they banned my country) and Google still presents those pages in my search results, wasting my time.
[+] OptionX|2 years ago|reply
Protip: If you really need to see the Reddit page on a private'd sub remember Google has a cached page feature.
[+] mintplant|2 years ago|reply
I've been bumping into this a bunch trying to research different neighborhoods of Washington, D.C., for an upcoming three-month stay. A lot of useful info is locked up in r/washingtondc right now, with much of it missing from Google's cache and the Wayback Machine. I still support the blackouts—I used to mod a default subreddit—but I wasn't expecting this particular effect!

(Tangentially, anyone know how the Mount Vernon Triangle area is these days?)

[+] f0e4c2f7|2 years ago|reply
Have been running into this the last few days and have realized just how much I use reddit in search results even though I don't visit the frontpage anymore.

Lot of pasting links into archive.org.

I've also been using Bing Chat to ask it what reddit thinks since it has most of reddit indexed.

Concerning though because it makes me feel like quite a lot of important information is rather fragile.

You can imagine various ways this data might be lost forever, I wish it felt more durable.

[+] hoofhearted|2 years ago|reply
Currently struggling with searching for the usual developer queries, but not getting the answers I need :(

“Something something random package name is suddenly broken Reddit”

The backup is using the same query with “stack overflow”, but that knowledge has become dated.

Around the rise of Slack, and then Discord and Teams, knowledge started becoming transferred between open source devs within their corresponding chat. People would get their answer on a VueJs issue and move on… The end result was that Google never picked that up, and the next dev popped in the chat the next day with the exact same question.

In the last couple years, Reddit has become the place for devs to do q&a as well.

It’s also making it difficult for me to communicate with prospective users online for simple market research and feedback.

[+] MAGZine|2 years ago|reply
it's not just google. it's the web.

all of the web's useful information--decades of discussion--is stored in reddit's garage. and they've decided to close the door.

[+] shswkna|2 years ago|reply
Humans built this wonderful common structure, where we can all communicate boundlessly.

The huge benefits only last briefly for humanity, until our inherent nature causes it to degenerate into a mess. Everybody goes back to doing their own thing.

Sounds like the Tower of Babylon from the bible.

[+] mkl95|2 years ago|reply
Google is mostly a data mining operation. Reddit is an organic content mine. Google is a behemoth, while Reddit are struggling to go public. It speaks volumes about how the internet has evolved in the last 25 years or so.
[+] mentos|2 years ago|reply
Doesn’t this illuminate an opportunity for Reddit?

Close the doors and invest in Reddit search monetized with ad placement like google has?

[+] ipkpjersi|2 years ago|reply
I feel like the blackout isn't gonna be effective only because Reddit is just gonna forcibly unprivate the large communities and assign new mods. Pretty disappointing but not surprising. I know I don't have any interest going back there.
[+] BashiBazouk|2 years ago|reply
If it's this important to google, couldn't they just buy Reddit with what amounts to pocket change? That would be an interesting plot twist...
[+] bravoetch|2 years ago|reply
It would be too funny if Google now announced they will no longer index reddit because the API is too expensive. Maybe that would spur reddit to improve search, or their own app and web offerings.