Ask HN: How do I search the web in the age of enshittified search engines?

[ManlyBread]

Like most internet users I used to use Google to search for information but now the quality of the results is abysmal and I no longer find it useful most of the time.

I've tried other search engines (Bing, Yandex) and the results are a bit better but still nowhere near the old Google. Kagi is a thing but I'm not sure if the quality justifies $120/year, especially since I don't earn my salary in USD.

Putting "site:reddit.com" helps a bit but there's plenty of stuff that simply isn't there, so it's not a solution either.

LLMs are a bit better with the quality of the searches but hallucinations are a thing so I need to verify the information myself so I am back to square one.

Is there anything else I can do to find information easier or to improve the quality of the searches?

[not_your_vase]

If you know the below mentioned, then just ignore. Otherwise give it a try.

There is one thing, that many people either forgot, or never knew: how to create a search query. Nowadays most people put in a human question: "how to bake bread", "how to use a red toilet seat", "what happens to today around the corner".

However even to this day, search engines gives better results if instead this, you try to imagine the results, and search for text that you think appears on the correct result: "bread recipe", "toilet seat user manual", "concert Tuvalu 2024 november"

[skydhash]

This. Search engines are an index, not an oracle. Even though google and the others have tried to mystified it by adding those answer boxes and doing a bunch of natural language processing. Another trick is to use the `insite:` keyword. Works wonder for searching docs.

[ManlyBread]

I no longer find this useful because I tend to get the same results no matter how I format my query. And these aren't even good results. Most often I get results for the stuff that is similar but at the same time completely unrelated to what I am actually searching for.

An example from yesterday: I wanted to search for curtains that would be resistant to cat scratches. No matter what I'd search for I would get the same set of links including the following:

* shops that sell textiles that are cat resistant but aren't drapes

* various surface-level guides that talk about this subject but no mention of any specific products (SEO spam)

* ways that are """helpful""" but ultimately harmful, such as spraying smells that cats doesn't like

Not a single mention of anything actually useful or related to what I actually searched for. I am either the first person who asked about this topic ever or there's something wrong with the search.

[MerelyMortal]

That is how I always did Google searches, even from 20 years ago, and I still get garbage results.

[djohnston]

The HYDE approach as it’s called in RAG vernacular.

[pino82]

I'd say "-site:reddit.com" helps a lot. And also "-site:youtube.com" and some others :)

Admittedly, I usually do the exclusion in my brain and not actually via the search query.

I don't know where the hype comes from, which says that reddit posts are particularly smart and useful. Maybe just from their own marketing, which is repeated over and over in social media by these 'smart' guys?! It's completely not reflected by my practical experiences at all.

When I accidentally end on a reddit thread and read some posts, I'm always like "Well, and that's it? What a waste of time..."

[KomoD]

> I don't know where the hype comes from, which says that reddit posts are particularly smart and useful

They're useful because there's comments from actual people.

I do site:reddit.com queries when I'm looking for product (and software) recommendations because otherwise you just find affiliate spammers (like cnet, wired, nyt and forbes).

[throwerofstone]

While I still mainly use Google to search for terms online, I am increasingly using the free version of Perplexity for more advanced topics or general questions. Perplexity is an LLM like Claude and ChatGPT, but instead of relying on the data it's been trained on, Perplexity gathers a whole bunch of sources (websites, youtube video transcripts, etc.) related to your query, and then uses the contents of those sources to generate an answer. So while it may not be as smart as Claude or ChatGPT on certain topics, it does seem to hallucinate a whole lot less. And at times when I'm not given the answer I'm looking for, or when I want to make sure it's not making things up, I simply browse the list of sources it used to generate its answer.

[atmanactive]

https://kagi.com/

[butterlettuce]

I’m personally not liking Kagi anymore. There are a few bugs that I’ve been dealing with for a year now that haven’t been fixed.

I’m using Safari on iOS. After you make a search and you want to modify your search text by pressing the search bar, the cursor doesn’t show up at all. So when you type, it either doesn’t type in anything or it types in the middle of the search words, if that makes sense. You don’t know where the cursor is until you type.

Just those little things makes me not want to use it.

And the search results aren’t all that great either and I have to depend on Google sometimes.

It’s hit or miss but when I expect Github results for a repo, it doesn’t show up unless I add “gh” or “github” to make it show up. It’s not like I ranked github lower, it’s actually in the normal rank setting.

[ManlyBread]

I've already mentioned Kagi.

[cpach]

I’m afraid there’s no silver bullet.

Marginalia can be useful, even if the index is quite small compared to other search engines: https://search.marginalia.nu/

[paxys]

What are you trying to find? It's not the quality of search that has declined but the quality of the web itself. What used to be interesting blogs are now ad-filled SEO garbage. Good content is now more and more behind logins and paywalls. So much of the rest is bot generated.

Search as a concept is useless unless you know exactly what you are looking for and know that it will be in the top 3 results (for example a company's official website or a wikipedia article).

[Yawrehto]

If you're getting a ton of irrelevant results, it's helpful to find a word common to them and exclude it from results; I don't have an easy way of measuring how many relevant results are excluded but if you choose it well it'll help. Similarly, find some list of all the operators for a given search engine and try them out.

I've found searching multiple indexes can be helpful, especially ones that seek different categories of thing out. For instance if I can't find something using Google, Marginalia can be surprisingly helpful. A list of sites to try is available here: https://seirdy.one/posts/2021/03/10/search-engines-with-own-....

Also, often finding some discussion of something that seems close to what you want and might discuss it briefly and seeing what it links to will help. But that's from mostly looking at academic stuff, which has copious citations that often won't show up in a search query for a term, but can be found in the likes of JSTOR or LibGen/SciHub.

Also from academic stuff: if you can find directories of things related to what you want (eg 'old books about the Standard Oil Company of New Jersey,' to take a real question I had), often a more general term ('old books') and then searching those websites (HathiTrust, Internet Archive, etc) directly will find things.

But that's for someone who primarily runs into trouble because I'm researching obscure subjects and probably are less applicable in other areas.

[FrankWilhoit]

StartPage scrapes Google but manages to get rid of a lot of the noise.

[KomoD]

Pretty sure they're paying Google for the data and not scraping it.

[bottedconvos]

Oh man, I feel you. Searching for good info these days feels like panning for gold in a kiddie pool—every once in a while, you strike it rich, but mostly, it’s just wet disappointment.

I’ve been down the same rabbit hole (tried Kagi, got sticker shock, and yep, LLMs are great... until they start making stuff up). One thing that’s helped me cut through the noise is sharpening my critical thinking skills—it’s like giving your brain a search engine upgrade. I wrote a newsletter - just started - post about this (with some hopefully helpful tips), so if you’re up for a quick read, check it out: https://bottedconversations.substack.com/p/critical-thinking....

Let me know what you think, please! Always looking for better ways to avoid the search engine struggle bus!

[danjl]

Pay. Simple. The enshitification is a direct result of the free business model, where the users of the service are actually the product, and the advertisers are the real customers. Even if kagi feels too expensive, you are still voting with your wallet by choosing a service that you pay for and showing companies that they can focus on building good products for the people who use them and are willing to pay for them. FWIW, the same is true of social networks, but they suffer from the additional complexity of needing enough people to use a service to make it worthwhile.

[Barrin92]

search operators (https://ahrefs.com/blog/google-advanced-search-operators/)

in particular: site; intitle; - (to exlude search terms); * (wildcard matches); before/after:

In addition on Youtube using any search operator also seems to get rid of most of the "for you / you already watched.." crap that they for some reason decide to shove into the middle of the search results.

[helph67]

I regularly use Ecosia, Dogpile, Qwant. Saving D.D.G for simple tasks. Wikipedia displays a Timeline List with active engines highlighted https://en.wikipedia.org/wiki/Search_engine

[tionichm]

Is there a specific reason why I'm seeing so little mention of DuckDuckGo?

It's not perfect, but it gets the job done 99% of the time, and I feel much more trust towards them than the alternatives.

I still need to figure out how to optimally perform the classic convenience search "X near me" without relying bigger names.

[achempion]

Could you provide example search queries that return bad results? My solution is to ask generic stuff from LLMs (and rarely youtube) and read specific on vendor's website. HN search works good as well.

[Elfener]

Google still works for me. I don't know if it's because I am in the EU or because uBlock Origin gets rid of the promoted results, probably both.

Also, if you are searching for something time-specific then "after:" and "before:" work great.

[41599799]

I don't have a good answer, but we sure could use (a new) +Fravia in this day and age.

[beretguy]

Try brave search.

[ericb]

Bing is now better than google.

You're right, though, it isn't quite as good as old google.