Now, the partnership is in jeopardy. Last Tuesday, the Justice Department filed a landmark lawsuit against Google — the U.S. government’s biggest antitrust case in two decades — and homed in on the alliance as a prime example of what prosecutors say are the company’s illegal tactics to protect its monopoly and choke off competition in web search.
The scrutiny of the pact, which was first inked 15 years ago and has rarely been discussed by either company, has highlighted the special relationship between Silicon Valley’s two most valuable companies — an unlikely union of rivals that regulators say is unfairly preventing smaller companies from flourishing.
The search market is entirely locked down. I’m a DuckDuckGo user, but DDG is just a frontend to Bing, warts and all. I’ve been having very negative experiences with DDG lately, but the only other real option is Google – I’ve got nowhere else to go. So either I accept Google’s filter bubble, or I accept DDG having terrible results filled with crazy conspiracy pseudoscience.
What choice do we really have?
The search engine is a tool, and like with any tool one needs to build the skills to use it properly. I still cannot fathom why people seem to think it’s okey to turn off their brains just because the tool happens to involve information technology.
There are four crawlers: Google, Bing, Yandex, Baidu. Doesn’t seem as though it should be difficult to build another, except that the internet is quite big now, so there’s a bandwidth price of entry.
You can probably get away without global search for most things though. Subscribe to a few actual news services for news feeds and use an RSS aggregator for the rest. Search for “known stuff” in Wikipedia. Search for products in Amazon or whatever your preferred local e-taler is. Maintain a favourites list and let your browser search that.
Google still mostly works for me, but it’s been getting steadily worse for tracking down specific technical problems or finding discussion of error messages, IMO. Replaced with lots of stupid beginner-level click-bait, which is probably more the fault of SEOs and monetization than google as such. Not bad enough for me to go looking for alternatives though.
There was also DeuSu.org , but the person doing it finally throw in the towel (it was a one-man “volunteer-effort”, payed by himself/herself and some donations).
I have always been marveled at how usable it was despite the paltry hardware it was running on.
BTW, the code is at https://github.com/mdbs99/DeuSu
SearX was also interesting but its decentralised nature made it somewhat “unreliable”.
Antartica_,
Yeah, it’s almost impossible to compete. Even former google employees notably failed in their attempt to create a new search engine competitor cuil.
it makes me wonder how much it would cost to create a dent in the market and become a successful competitor these days. The costs of building & running crawlers are enormous, and that doesn’t even get your foot in the market. Google bundle’s it’s search engine and analytics on billions or devices and websites. It’s paying hundreds of millions a year to set google as the default search engine on firefox and elsewhere.
Most of the competitors today are value added “front ends” for other search providers like yahoo and duck duck go. For better or worse this seems to be most viable path for alternatives. Part of me would like to see a truly open index…something that anyone can build a service around. Building/maintaining the index is the most expensive part and it ends up being an impediment for those who’d like to offer more innovation. It could be similar to what open-street-maps offers, which powers a lot of alternative mapping services.
I find that Google’s pseudoscience as well as the SJW garbage that the company (and to be fair all of silicon valley) props up to be a far worse negative to society than any problems I’ve found with DDG’s search results.
With that said, I’m curious what Thom regards to be crazy conspiracy pseudoscience.
He’s apparently upset that when searching for 5G in DDG it returns some of the classic “5G causes death” kind of hits. Not sure why that upsets him so much, if you want to escape the Google safe space, there is a price to pay in that you’ll need to wade through that yourself. But getting around Google protecting you from your own search terms is definitely worth it.
And that’s not even getting into the privacy respecting benefits of DDG.
Yes, I’ve had this problem with Google as well. It’s gotten worse in the past year or so. Attempting to research anything “problematic” results in Google trying to tell you what to think. This is particularly noticeable with anything medical right now.
I decided to shun Google as much as possible and use Qwant instead. Works for me quite nicely (unlike DDG). YMMV.
With all due respect mr. Holwerda, with regards to your “very negative experiences” with DDG.
I’m under the (obviously minority) opinion that firstly, a search engine should be a tool, and secondly, a search engine should be dumb (think Google’s initial page-rank algorithm based on term-appearance and number of user clicks). The only pages that should possibly be excluded from search are bot-generated pages with no content and artificially inflated rank. Based on dumb-algorithms alone, a search engine cannot present truth. What it can do, is present the wide gamut of opinion of what the internet thinks. And, by interacting with the internet, a user can shift rankings of such opinions or create new ones. The reason “wild west internet” was seen as a threat was that some “non-sanctioned” user could present a more truthful angle and gain a wider audience than the sanctioned ones. Sure, most of the will be fringe, but then you have people like Julian Assange or project Veritas .
Now, you might be thinking: “how harmful could it be if we let a small group of people working with the search engine company curate the results and exclude bad ones?” As someone from a former communist country, I can tell you the answer to that would be “very”. You’re offended because search results are heterogeneous (as opinions of internet-users are bound to be) and you have to do some thinking to exclude “bad” ones. Well, I’m offended that Google tells me what to think, does my thinking for me, and I can no longer find more than one opinion, regardless of what I use it to search. Either a search engine can present contradictory results that might require you to think, or else it does your thinking for you. I prefer the former. If I want “sanitized” homogeneous opinions from official sources, I open the television. Or use Google, that’s basically the same now. I can’t tell what the future of the internet will be, but I dearly hope it doesn’t become another passive medium like TV, with the forming of opinions left solely in the hands of unaccountable corporations.
Zero_01,
I mostly agree, but a page-rank algorithm has some major flaws IMHO. An obvious one is that’s it’s notoriously gameable by advertisers and propagandists and encourages comment spam. However let’s overlook that and assume you’d got a way to filter it out. The next issue is that page rank is effectively a popularity contest. The winners are those with the most popularity rather than those with the best or most relevant results. It effectively ranks sites higher because they are popular rather than because they are good, which is at odds with what I think the internet should be. It’s very hard to do anything about this algorithmicly because it’s a pervasive problem that exists throughout our culture. So much so that news sources will write more articles citing celebrity’s opinions over the experts. They do this all of the time going out of their way to cover someone popular rather than someone qualified. Celebrity is more important than substance 🙁
This is extremely apparent on youtube where they have tons of content, yet their algorithm is always promoting popular/celebrity channels leading to tens of millions of views for their channels and embarrassingly few views for content from unknowns. Youtube’s got some great content, yet youtube’s home page is so stale that it’s frustrating. I acknowledge this is in part because I don’t have an account to track my preferences, however even so it speaks to the ridiculous amount of over-promotion a tiny set channels are getting from youtube’s popularity based algorithm. Many of those videos with 25M views are completely unoriginal crap that would not have gotten those massive views organically without youtube’s tendency to keep pushing popular videos ever higher. It is what it is, but it highlights the flaw with page-rank type algorithms. It’s certainly not a way to promote quality. To be clear some popular channels do have good content, but it’s mostly coincidental and some quality content never manages to become popular because they lack popularity.
I don’t have a problem with corporations curating content for their properties in principal, however it is concerning how much control a few corporations have amassed over our technology. A few companies are controlling ever more of what we see and what we can install on the devices they own. This is dangerous amount of control and I’d like to see technology become less centralized to undo these trends. But realistically these things happened in the first place because corporations wanted this, and they may not be willing to voluntarily give up their positions of power.
Thom, you can use startpage.com to search Google without their filter bubble and tracking. There’s always an alternative to something you don’t like. 🙂
I can’t say I’ve had DDG give me any “crazy conspiracy pseudo-science” (a term that seems to mean anything I disagree with these days), however I have had a deal of trouble getting it to return results which are relevant to the topic for which I’m actually searching. The more detailed and technical the topic, the less relevant the results DDG returns. In my case, “crazy conspiracy pseudo-science” would actually be an improvement compared to what I often get from DDG.
darknexus,
Can we look at an example? I don’t have a problem with it, at least it’s had good results for me.
The choice you have is to not use search engines, like people did before the internet was a thing.
Bottom line is Google’s search is a service, as is Bing and DDG. Those entities have devoted a lot of time and resources to build them, and if you want to use them, you have to do it in their terms..
There is no entitlement to use of the service on your own terms, only those made available by the service provider.
mkone,
…except the world has changed and we need the internet for many aspects of modern life. It’s not like the past even if we wanted it to be.
You can say the same thing about any product or service, but as markets become increasingly concentrated it becomes more imperative to do something to rebalance markets for the overall good, lest we end up further in monopoly & oligopoly territory. Given the current trends and our overall laissez-faire policies, we will witness a resurgence of a new robber barron era with most markets being dominated by a few giants and competition being futile.
In an ideal world we’d learn to get better at steady-state regulation, but in the real world often things have to hit rock bottom before they can get better. So many people won’t do anything to stand up against imbalances, which leaves many opportunities for those in power to take advantage of them 🙁
Bottom line, it takes a buttload of money and geniuses to come up with a somewhat competitor to the Goog. Who will be Open Index’s Daddy Warbucks(es)?
you could try swisscows.ch
Regards,
Jan
Thom,
Thanks for the discussion. I will try to chime in — and only on a tangential question on your Twitter thread ( since it would be conflict of interest otherwise).
The context is receiving bad results on some conspiracy theories. However it will always be a cat and mouse game between spammers and search engine developers.
That being said, the deeper issue is the “QA” you mention. Down-ranking / filtering results are always contentious, and goes against having a fully neutral search engine. We cannot have a web search engine which produces results that are agreed upon by everyone, in some cases even a majority.