Hi, I'm the CEO & Founder of DuckDuckGo. To be clear (since I see a ton of confusion going around about this story), when you load our search results, you are anonymous, including ads. That is, this news cycle is not about our search engine, it's about our browsers -- and, contrary to the headlines, we actually do restrict Microsoft trackers there today a great deal, and more than the major browsers do.
When most other browsers on the market talk about tracking protection they are referring to 3rd-party cookie protection e.g., blocking them) and fingerprinting protection (i.e., restricting APIS scripts can use), and our browsers impose these restrictions on all third-party tracking scripts, including those from Microsoft. We also have a lot of other web protections that also apply to Microsoft scripts (and everyone else) that most browsers don't do, including Global Privacy Control, 1st-party cookie expiration, referrer header trimming, new cookie consent handling (in our Mac beta), fire button (one-click) data clearing, and more.
What this article is talking about specifically is one web protection that the major browsers don't even attempt to do — stopping third-party tracking scripts from even loading on third-party websites. See for yourself the bottom 'tracker content blocking' section of this audit site: https://privacytests.org/ios.html (scroll to the very bottom).
This web protection is a particular challenge to get right because websites can easily break when scripts that they depend on don't load, but because it makes for better privacy and faster page loads, we've taken it on while still trying to not break sites. As a result, it is far from perfect, as we can't block all scripts do to persistent breakage issues (which we try to workaround), and domains change all the time (and we continuously crawl sites looking for new tracking domains).
Though because we're doing this protection where we can, and also offer many other unique protections (e.g., Google AMP protection, smarter HTTPS upgrading, tracking protection for other apps in Android, email protection to block trackers for emails sent to your regular inbox, etc.), users get way more privacy protection with our app than they would using other browsers. Our goal has always been and remains to provide the most privacy we can in one download. And we have a lot more planned.
The issue at hand is all of our web protections apply to Microsoft scripts on 3rd-party sites (again, this is off of DuckDuckGo.com, i.e., not related to search) except this one around full content blocking, as we are currently contractually restricted by Microsoft there. However, this limited contractual restriction (about this one web protection) is the only one we have, and we have been and are working with them as we speak to reduce or remove it.
I understand this is all rather confusing because it is a search syndication contract that is preventing us from doing a non-search thing. That's because our product is a bundle of multiple privacy protections, and this is a distribution requirement imposed on us as part of the search syndication agreement that helps us privately use Bing results to provide you with better private search results overall. While a lot of what you see on our results page privately incorporates content from other sources (including our own indexes), e.g., Wikipedia, local listings, sports, stocks, lyrics, weather, quick answers, etc., etc., we source most of our traditional links and images privately from Bing (though because of other search technology our link and image results still may look different). Really only two companies (Google and Microsoft) have a high-quality global web link index (because I believe it costs upwards of a billion dollars a year to do), and so literally every other global search engine that wants to offer a search product competitive with Google for mainstream users, needs to bootstrap with one or both of them. The same is true for maps btw -- only the biggest companies can similarly afford to accurately map every neighborhood. And we are still a very small company relative to these companies -- literally on the order of 1,000 times smaller.
Anyway, I hope this provides some helpful context. I understand why people are upset with us on this, and we will do better. We're working on updates to our app store descriptions and other materials.
(Also FYI -- this was discussed extensively at https://news.ycombinator.com/item?id=31490515). This is also a somewhat misleading title since it's not about our search engine, which people would assume, and also as noted above Microsoft scripts are actually restricted in our browser a lot, so I would suggest changing it.)
So many words to minimize what happened, makes the conclusion of "I understand why people are upset with us on this" seem a little insincere.
To me, this is really simple. You entered a contract that allowed a company to circumvent tracking protections. the extent and which product this happened on are only semantic details. It fundamentally undermines your trustworthiness as a supposedly privacy-oriented company.
> This is also a somewhat misleading title since it's not about our search engine, which people would assume
I understand the controversy (or at least I think I do), and I also respect what you're trying to explain.
But if DuckDuckGo has to have a browser (instead of simply recommending to its users to change their default search engine on their current browser), then that browser needs to live up to its promises, and also to very high standards.
On the current page for the DDG browser [1] it says at the top:
> Seamlessly take control of your personal information, no matter where the Internet takes you.
If this means anything, it means the DDG browser blocks all trackers, always, in all places, no?
“[W]hen you search, you expect unbiased results, but that’s not what you get on Google,” @matthewde_silva quotes @yegg"
Seems you're backtracked on being unbiased with the recent announcement of DuckDuckGo censoring sources on the back of the Ukrainian War. Any comment on that - seems like the bigger DDG gets, the further you stray from your initial values
> Really only two companies (Google and Microsoft) have a high-quality global web link index ...
Yandex and Baidu also have pretty good index, and perhaps would be even cheaper than Bing. I'd say they only lack in terms of the data (search queries, search result links clicked etc). It's just US protectionist policy (on behalf of the BigTech) at play here to avoid these services. Both Yandex and Baidu outperform Google when it comes to Russian and Chinese search queries, clearly indicating they are at par (or even better) than Google in terms of search algorithms. The only thing they lack is better data, and that's why they struggle with foreign search queries. (Though I find Yandex has improved a lot here, enough to make it my regular search engine).
"literally every other global search engine that wants to offer a search product competitive with Google for mainstream users, needs to bootstrap with one or both of them."
I don't want to read too far into the specific wording, but you mentioned "bootstrap" and this feels like a good opportunity to ask the question. Is DDG planning or thinking about how it might move towards it's own index for general results? I feel like most of the criticism DDG receives these days is because it's backed by another, less privacy concerned, mega-corp. While it may not be something that's feasible today, and not in it's entirety, are there moves being made towards building an independent global web link index?
It should be said that, if I understand it correctly, these worries are about the DDG browser, not their search, which is what most people (myself included) associate with them and care about.
Yes, I (CEO/Founder, DuckDuckGo) left a top-level comment responding to the article here: https://news.ycombinator.com/item?id=31557587. Would suggest reading it for a fuller explanation.
In short though, you're right it isn't about our search engine. It also isn't about 3rd-party cookies (another point of confusion I've seen), which our browsers block (including for 3rd-party Microsoft scripts). 3rd-party Microsoft scripts are also further restricted in our browsers with additional web protections, e.g., fingerprinting protection, Global Privacy Control, referrer header trimming, etc.
What this article is talking about specifically is one web protection that the major browsers don't even attempt to do — stopping third-party tracking scripts from even loading on third-party websites. You can see that for yourself at the bottom 'tracker content blocking' section of this audit site: https://privacytests.org/ios.html (scroll to the very bottom).
I think that's letting them off easy. Even if you don't care about their browser, their conduct in regards to the browser is a canary for their potential conduct regarding search and other products in the future. I think people are wrong to brush this off because it's "only about the browser"
I switched to Kagi[0] when DDG announced their implementation of selective censorship[1]. The results are much better, and I use the !g bang way less often (for DDG I was basically always using it, making the bangs the only actually useful feature of DDG for me).
[1] https://twitter.com/yegg/status/1501716484761997318 — I think it's sad that I have to clarify that I am personally also sickened by Russia's invasion of Ukraine, but that I don't want DDG to be an arbiter of truth despite that
Calling what DDG did "selective censorship" is a very unfair description. Essentially a search engine is about sorting search results and any search engine must do that based on some idea about what information is good and what is bad. DDG has been doing that since day one. In fact, the primary reason DDG was created was because its creator thought they could improve search results by getting rid of things that showed up in Google that they thought where bad (the privacy selling point came later)[1]. You may disagree with some of these choices (which is fair) but calling them "censorship" is just using bad sounding words that don't really apply.
Hi, CEO & Founder of DuckDuckGo here. I see a lot of confusion here too, we are not ranking based on my politics (or anyone's politics for that matter).
We actually do not intentionally censor any news results, meaning media outlets are not being removed or their stories displayed so far down in the results they are effectively removed. That is, unless legally prohibited, you should find all media outlets in our results, and they should generally show on top if you search for them by name or domain name. If you are seeing otherwise, please let me know and we will investigate.
A search engine's primary job is to rank results, trying to put results that most quickly and accurately answer the query on top. We do this ranking in a strictly non-partisan manner. Ranking for news-related searches is particularly difficult because for most news stories there are often hundreds of media outlets covering the same story, many with similar relevancy in terms of keyword matching and popularity. As such, we look to another ranking factor to ensure just the top of the results aren't taken by obviously very low-quality news results so that users have more sources of relevant, high-quality news results to compare and choose between.
The non-partisan factor we've found to help accomplish this is a rare, but well-documented history of a site's complete lack of news reporting standards, such as routinely using spam or clickbait to artificially inflate traffic, consistently publishing stories without citing sources, censoring stories due to operating with very limited press freedom, or misleading readers about who owns, funds, and authors stories for the site. And since we do not censor sites, even state-sponsored media in countries with very limited press freedom, these sites will still show up in results, and even on top like when you search for them directly.
Even though Kagi looks and sound nice, I can't cope with signed-in-required search.
I understand that their business model is "you pay, you are our user, we won't sell your data"[1] But there is a simple privacy-friendly business model to be made with free anonymous search engine as well, when you look for "horses" we show you ads of horses. This is what Duckduckgo Search and Google Search both do. (Well google then runs ads on many website which are then linked to your search query, that's the real issue)
Now with Kagi, I have a company which knows all my darkest secrets whether I use a different computer behind a VPN or not.
That's exactly the kind of "censorship" you want. I understand the sentiment, but you couldn't have chosen a worst example, I hope it's not this specific instance of censorship that really troubles you. Removing what is objectively, yes, spam is ok.
People complain about search engines being full of garbage and clearly false info, then complain that DDG took action against sources that publish false info. Can’t win.
under the hood they track the shit out of you, even more than DDG with their machine learning tools for user based scoring system for links
remember everyone, duckduckgo refused every independent audits, proprietary stuff like kagi is no exception, history repeats itself
also i find it weird that product placement comments like this one are always ranked super high on this website
they are done with DDG, they shifted focus to kagi instead? suspicious, it's not even out yet, only work on macOS, and is a team of "benevolent living in San Fransisco", living on donations in the most expensive state of the US, sure :thinking_face:
Every time I tried to switch to DDG I ended up reverting to using the !g bang on every other query. While Kagi supports the same functionality, I haven’t felt the need to use it once in the past two months that I’ve been trialing the private beta. It’s genuinely better than Google for me, which I never thought would happen.
it's not perse about truth but it is about facts, I personally don't have a problem with downgrading lies or denies of facts. In the end of the day a algo is ranking pages, algo's can be manipulated or gamed and they are not only by lot's of marketeers but also by government agents.
While I don't believe a company should decide if an opinion about gun controle should be rankt or not. but if someone is claiming that these are just crisis actors and nothing happend I don't see a problem to block it. After all we are used to blocking spam these things are spam. I don't see a reason to treat this as any different just because they are selling political lies instate of product lies (and a lot of times these people are selling products in the end)
these kind of lies are eroding democracy and should be treaded the same way as some one yelling fire in a cinema
If I search for something on a search engine, I am trusting the operator of that to filter and down rank based on their own criteria, whether that's baidu, yandex, Google or DDG. It's completely reasonable for a site who's job it is to filter and aggregate results to penalise the spread of misinformation. If they don't, why am I using it at all?
It's really strange how people seem to be going out of their way to defend DDG.
They actively censor content based on political views and they now selectively enable Microsoft tracking. All in the span of like a month. And in return for all of this you still get a half-baked browsing experience.
Someone remind me what value DDG actually provides at this point? It feels like we're witnessing a self-implosion.
DDG followed by a !s (Startpage) if needed always gives me more and higher quality results than Google or any other combination of search engines I've tried. YMMV. Not sure there is any other privacy related search engine worth using. Brave feels pretty sketchy to me skimming ad dollars.
Been on brave search for a week roundabout. All of a sudden I got all these "fake-github" results I heard people speaking of, and which I never saw myself on DDG.
Any search engine which is on the drip of <insert big one> will ultimately fail. Search engines are not philanthropic. Bing used to behave in good manner as they couldn't attract enough paying customers so their strategy was to impair Google by attracting traffic to Bing wherever and by whatever means, even if it meant a loss for Microsoft.
Now as Google search is the entry page to Amazon, Bing sees some increase in serious traffic and MS is getting more serious about their business strategy.
We don't need a Web3, we need a decentralized spider.
I have signed up a trial of Kagi few months ago and once I got the invite I was sceptical but I thought I’ll give it a go and I never switched back since. It’s bloody awesome, results are way better than Google, never-mind DDG. Customisation is awesome too. I’m very happy with it and as soon as it comes out of beta, I’ll happily support it.
DDG needs to understand that they are in the business of "trust", which fuels their brand, and trust is gained each and every day over a long period of time, and it can be lost in an instance.
And when we deal with "trust" perception is just as relevant as reality.
This is a fork in the road for DDG, they can come out of this stronger than ever, or be tarnished and doomed.
[+] [-] akaBruce|3 years ago|reply
https://news.ycombinator.com/item?id=31490515
[+] [-] yegg|3 years ago|reply
When most other browsers on the market talk about tracking protection they are referring to 3rd-party cookie protection e.g., blocking them) and fingerprinting protection (i.e., restricting APIS scripts can use), and our browsers impose these restrictions on all third-party tracking scripts, including those from Microsoft. We also have a lot of other web protections that also apply to Microsoft scripts (and everyone else) that most browsers don't do, including Global Privacy Control, 1st-party cookie expiration, referrer header trimming, new cookie consent handling (in our Mac beta), fire button (one-click) data clearing, and more.
What this article is talking about specifically is one web protection that the major browsers don't even attempt to do — stopping third-party tracking scripts from even loading on third-party websites. See for yourself the bottom 'tracker content blocking' section of this audit site: https://privacytests.org/ios.html (scroll to the very bottom).
This web protection is a particular challenge to get right because websites can easily break when scripts that they depend on don't load, but because it makes for better privacy and faster page loads, we've taken it on while still trying to not break sites. As a result, it is far from perfect, as we can't block all scripts do to persistent breakage issues (which we try to workaround), and domains change all the time (and we continuously crawl sites looking for new tracking domains).
Though because we're doing this protection where we can, and also offer many other unique protections (e.g., Google AMP protection, smarter HTTPS upgrading, tracking protection for other apps in Android, email protection to block trackers for emails sent to your regular inbox, etc.), users get way more privacy protection with our app than they would using other browsers. Our goal has always been and remains to provide the most privacy we can in one download. And we have a lot more planned.
The issue at hand is all of our web protections apply to Microsoft scripts on 3rd-party sites (again, this is off of DuckDuckGo.com, i.e., not related to search) except this one around full content blocking, as we are currently contractually restricted by Microsoft there. However, this limited contractual restriction (about this one web protection) is the only one we have, and we have been and are working with them as we speak to reduce or remove it.
I understand this is all rather confusing because it is a search syndication contract that is preventing us from doing a non-search thing. That's because our product is a bundle of multiple privacy protections, and this is a distribution requirement imposed on us as part of the search syndication agreement that helps us privately use Bing results to provide you with better private search results overall. While a lot of what you see on our results page privately incorporates content from other sources (including our own indexes), e.g., Wikipedia, local listings, sports, stocks, lyrics, weather, quick answers, etc., etc., we source most of our traditional links and images privately from Bing (though because of other search technology our link and image results still may look different). Really only two companies (Google and Microsoft) have a high-quality global web link index (because I believe it costs upwards of a billion dollars a year to do), and so literally every other global search engine that wants to offer a search product competitive with Google for mainstream users, needs to bootstrap with one or both of them. The same is true for maps btw -- only the biggest companies can similarly afford to accurately map every neighborhood. And we are still a very small company relative to these companies -- literally on the order of 1,000 times smaller.
Anyway, I hope this provides some helpful context. I understand why people are upset with us on this, and we will do better. We're working on updates to our app store descriptions and other materials.
(Also FYI -- this was discussed extensively at https://news.ycombinator.com/item?id=31490515). This is also a somewhat misleading title since it's not about our search engine, which people would assume, and also as noted above Microsoft scripts are actually restricted in our browser a lot, so I would suggest changing it.)
[+] [-] asddubs|3 years ago|reply
To me, this is really simple. You entered a contract that allowed a company to circumvent tracking protections. the extent and which product this happened on are only semantic details. It fundamentally undermines your trustworthiness as a supposedly privacy-oriented company.
[+] [-] bambax|3 years ago|reply
I understand the controversy (or at least I think I do), and I also respect what you're trying to explain.
But if DuckDuckGo has to have a browser (instead of simply recommending to its users to change their default search engine on their current browser), then that browser needs to live up to its promises, and also to very high standards.
On the current page for the DDG browser [1] it says at the top:
> Seamlessly take control of your personal information, no matter where the Internet takes you.
If this means anything, it means the DDG browser blocks all trackers, always, in all places, no?
[1] https://duckduckgo.com/app
[+] [-] muchtest|3 years ago|reply
“[W]hen you search, you expect unbiased results, but that’s not what you get on Google,” @matthewde_silva quotes @yegg"
Seems you're backtracked on being unbiased with the recent announcement of DuckDuckGo censoring sources on the back of the Ukrainian War. Any comment on that - seems like the bigger DDG gets, the further you stray from your initial values
https://twitter.com/DuckDuckGo/status/1114524914227253249
[+] [-] webmobdev|3 years ago|reply
Yandex and Baidu also have pretty good index, and perhaps would be even cheaper than Bing. I'd say they only lack in terms of the data (search queries, search result links clicked etc). It's just US protectionist policy (on behalf of the BigTech) at play here to avoid these services. Both Yandex and Baidu outperform Google when it comes to Russian and Chinese search queries, clearly indicating they are at par (or even better) than Google in terms of search algorithms. The only thing they lack is better data, and that's why they struggle with foreign search queries. (Though I find Yandex has improved a lot here, enough to make it my regular search engine).
[+] [-] bluehatbrit|3 years ago|reply
I don't want to read too far into the specific wording, but you mentioned "bootstrap" and this feels like a good opportunity to ask the question. Is DDG planning or thinking about how it might move towards it's own index for general results? I feel like most of the criticism DDG receives these days is because it's backed by another, less privacy concerned, mega-corp. While it may not be something that's feasible today, and not in it's entirety, are there moves being made towards building an independent global web link index?
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] Schiphol|3 years ago|reply
[+] [-] criddell|3 years ago|reply
If RMS took a job with Adobe, do you think people would question his commitment to open source?
[+] [-] yegg|3 years ago|reply
In short though, you're right it isn't about our search engine. It also isn't about 3rd-party cookies (another point of confusion I've seen), which our browsers block (including for 3rd-party Microsoft scripts). 3rd-party Microsoft scripts are also further restricted in our browsers with additional web protections, e.g., fingerprinting protection, Global Privacy Control, referrer header trimming, etc.
What this article is talking about specifically is one web protection that the major browsers don't even attempt to do — stopping third-party tracking scripts from even loading on third-party websites. You can see that for yourself at the bottom 'tracker content blocking' section of this audit site: https://privacytests.org/ios.html (scroll to the very bottom).
[+] [-] asddubs|3 years ago|reply
[+] [-] bilekas|3 years ago|reply
I don't know many who use their browser, personally I moved over to Brave recently and so far so good.
[+] [-] valvar|3 years ago|reply
[0] https://kagi.com/
[1] https://twitter.com/yegg/status/1501716484761997318 — I think it's sad that I have to clarify that I am personally also sickened by Russia's invasion of Ukraine, but that I don't want DDG to be an arbiter of truth despite that
[+] [-] paldepind2|3 years ago|reply
1: Source, Gabriel Weinberg's book Traction.
[+] [-] yegg|3 years ago|reply
We actually do not intentionally censor any news results, meaning media outlets are not being removed or their stories displayed so far down in the results they are effectively removed. That is, unless legally prohibited, you should find all media outlets in our results, and they should generally show on top if you search for them by name or domain name. If you are seeing otherwise, please let me know and we will investigate.
A search engine's primary job is to rank results, trying to put results that most quickly and accurately answer the query on top. We do this ranking in a strictly non-partisan manner. Ranking for news-related searches is particularly difficult because for most news stories there are often hundreds of media outlets covering the same story, many with similar relevancy in terms of keyword matching and popularity. As such, we look to another ranking factor to ensure just the top of the results aren't taken by obviously very low-quality news results so that users have more sources of relevant, high-quality news results to compare and choose between.
The non-partisan factor we've found to help accomplish this is a rare, but well-documented history of a site's complete lack of news reporting standards, such as routinely using spam or clickbait to artificially inflate traffic, consistently publishing stories without citing sources, censoring stories due to operating with very limited press freedom, or misleading readers about who owns, funds, and authors stories for the site. And since we do not censor sites, even state-sponsored media in countries with very limited press freedom, these sites will still show up in results, and even on top like when you search for them directly.
[+] [-] acatton|3 years ago|reply
I understand that their business model is "you pay, you are our user, we won't sell your data"[1] But there is a simple privacy-friendly business model to be made with free anonymous search engine as well, when you look for "horses" we show you ads of horses. This is what Duckduckgo Search and Google Search both do. (Well google then runs ads on many website which are then linked to your search query, that's the real issue)
Now with Kagi, I have a company which knows all my darkest secrets whether I use a different computer behind a VPN or not.
[1] https://kagi.com/faq#privacy
[+] [-] Copenjin|3 years ago|reply
[+] [-] grapeskin|3 years ago|reply
[+] [-] endemic|3 years ago|reply
[+] [-] Shadonototra|3 years ago|reply
under the hood they track the shit out of you, even more than DDG with their machine learning tools for user based scoring system for links
remember everyone, duckduckgo refused every independent audits, proprietary stuff like kagi is no exception, history repeats itself
also i find it weird that product placement comments like this one are always ranked super high on this website
they are done with DDG, they shifted focus to kagi instead? suspicious, it's not even out yet, only work on macOS, and is a team of "benevolent living in San Fransisco", living on donations in the most expensive state of the US, sure :thinking_face:
[+] [-] dkobia|3 years ago|reply
[+] [-] leadingthenet|3 years ago|reply
Every time I tried to switch to DDG I ended up reverting to using the !g bang on every other query. While Kagi supports the same functionality, I haven’t felt the need to use it once in the past two months that I’ve been trialing the private beta. It’s genuinely better than Google for me, which I never thought would happen.
[+] [-] stefanve|3 years ago|reply
While I don't believe a company should decide if an opinion about gun controle should be rankt or not. but if someone is claiming that these are just crisis actors and nothing happend I don't see a problem to block it. After all we are used to blocking spam these things are spam. I don't see a reason to treat this as any different just because they are selling political lies instate of product lies (and a lot of times these people are selling products in the end)
these kind of lies are eroding democracy and should be treaded the same way as some one yelling fire in a cinema
[+] [-] 10729287|3 years ago|reply
[+] [-] sofixa|3 years ago|reply
[deleted]
[+] [-] JimWestergren|3 years ago|reply
It took DuckDuckGo years to built up their positive privacy reputation and it can all fall apart in a few days.
[+] [-] muchtest|3 years ago|reply
[+] [-] dathos|3 years ago|reply
[+] [-] maccard|3 years ago|reply
[+] [-] someotherperson|3 years ago|reply
They actively censor content based on political views and they now selectively enable Microsoft tracking. All in the span of like a month. And in return for all of this you still get a half-baked browsing experience.
Someone remind me what value DDG actually provides at this point? It feels like we're witnessing a self-implosion.
[+] [-] car_analogy|3 years ago|reply
This is news to me - could you elaborate? Does it go beyond just passively duplicating Bing's censorship due to using their search results?
[+] [-] Schopenhauer01|3 years ago|reply
[+] [-] infamia|3 years ago|reply
[+] [-] riidom|3 years ago|reply
[+] [-] ulimn|3 years ago|reply
[+] [-] rglullis|3 years ago|reply
[+] [-] jhoechtl|3 years ago|reply
Now as Google search is the entry page to Amazon, Bing sees some increase in serious traffic and MS is getting more serious about their business strategy.
We don't need a Web3, we need a decentralized spider.
[+] [-] gnuj3|3 years ago|reply
[+] [-] SMAAART|3 years ago|reply
And when we deal with "trust" perception is just as relevant as reality.
This is a fork in the road for DDG, they can come out of this stronger than ever, or be tarnished and doomed.
[+] [-] urlwolf|3 years ago|reply
[+] [-] Wright123|3 years ago|reply
[deleted]
[+] [-] atc|3 years ago|reply
[deleted]
[+] [-] godmode2019|3 years ago|reply
Even bing is better than duckduckgo's results
[+] [-] Kiro|3 years ago|reply