But Google results are a zero-sum game. If the search engine sends traffic to one site, it has to take it from another, and the effects on the losers in this Reddit equation are just as dramatic. “Google’s just committing war on publisher websites,” Ray says. “It’s almost as if Google designed an algorithm update to specifically go after small bloggers. I’ve talked to so many people who’ve just had everything wiped out,” she says.
A number of website owners and search experts who spoke to the BBC said there’s been a general shift in Google results towards websites with big established brands, and away from small and independent sites, that seems totally disconnected from the quality of the content.
↫ Thomas Germain at the BBC
These stories are coming out left, right, and centre now – and the stories are heartbreaking. Websites that publish truly quality content with honest, valuable, real reviews are now not only having to combat the monster of Google’s own creation – SEO spam websites – but also Google itself, who has started downranking them in favour of fucksmith on Reddit. Add to that the various “AI” boxes and answers Google is adding to its site, and the assault on quality content is coming from all angles.
I don’t look at our numbers or traffic sources, since I don’t want to be influenced by any of that stuff. I don’t think OSNews really lives or dies by a constant flow of Google results, but if we do, there’s really not much I can do about it anyway. Google Search once gaveth, and ever since that fateful day it’s mostly been Google Search taketh. I can’t control it, so I’m not going to worry about it. All I can do is keep the site updated, point out we really do need your support on Patreon and Ko-Fi – to keep OSNews running, and perhaps maybe ever going ad-free entirely – and hope for the best.
I do feel for the people who still make quality content on the web, though – especially people like the ones mentioned in the linked BBC article, who set up an entire business around honest, quality reviews of something as mundane as air purifiers. It must be devastating to see all you’ve worked for destroyed by SEO spam, fucksmith on Reddit, and answers from an “AI” high on crack.
What is AI, Thom?
In this particular case, it means an LLM. Generally, it should be clear by now that ChatGPT, CoPilot, Google AI Overviews, and Google Gemini/Bard (and anything that relies on either of those) is the LLM kind of AI.
You have to be completely retarded or completely ignorant about technology to not be able to figure this out by yourself, and yet you keep asking, again and again:
https://www.osnews.com/story/139601/stack-overflow-signs-deal-with-openai-bans-users-trying-to-alter-answers/
Why?
Sorry guys, but in my opinion, anyone who builds a business on the hope that an unrelated, untrusted and unchecked third party speaks always favorable of it, is just busted from start.
Google is not the good guy, but it is also not to blame here. There are no sides, only players.
Also, the unfortunate truth is: if you have the best product or content in the world but are not able to sell it and to sustain you business, then you have no product at all. Business is about finding this sweet spot where you can get behind your product with all your heart and passion and still will be able to sell it.
Countless times people have asked me, to make my commercial product actually worse and I always obliged because we are all private dancers one way or the other. That’s the reason why I publish open source: nobody can tell me what to do in that niche. True freedom.
Whilst I agree with the sentiment that one shouldn’t base one’s income exclusively on the goodwill of a single external party, I vehemently disagree with the claim that google isn’t a guilty party. They are reducing the quality of their results, which should be a red flag even if one doesn’t care about the income of the websites in question.
You make some fine points.
But wouldn’t it be helpful if there were other search engines or search engine modes that could discover lower SEO content?
Wouldn’t that help offer a boost to some smaller, lesser recognized up and coming or underground websites that sometimes might actually exhibit some promise or ingenuity?
It’s time to try metasearch and yacy…
https://github.com/searxng/searxng
https://yacy.net
That is if you see everything as a product that needs commercial viability to warrant an existence. The world is more than that.
Is your local jazz band to be killed off because they don’t reach an audience we deem big enough? Is Wiki supposed to charge you a subscription so they can stay alive?
M
If possible money should be replaced or wildly reformed. Or else small businesses should be given starter money from the government (like in China) and patents removed so anybody can make anything.
And bigger companies should be taxed more heavily.
Alternate forms of income should be supported.
Ie. Cryptocurrency should be made widely available and easy to the common man. Legal in the EU if you only have X amount of rigs or less, not legal for businesses to mine en masse (pollutants).
But I’m just a dreamer.
@Thom: kudos for not following Google/Bing statistics blindly. It will take longer to pan out but will be more sustainable.
On the business side, I see commercial potential for the OSNews niche. But in my opinion you will need a better technical platform and also slight less bias (your piece on the AI Voice line of the actor was a disaster). Business and personal vendettas won’t mix well. Have been there, have done that…
The problem is that there is not much money in news, but much more in opinions. People flock to controversy and debate and because Thom is now more dependent on making money from OSnews, expect more “controversial” topics.
Just look at the most commented posts on OSnews, it’s almost always topics that can have a wide range of reactions that are high up in the ranking.
It’s in Thom’s financial interest to stirr the discussion. Nobody can escape the blackhole that is called advertising.
You can also see it at sites like The Verge or Ars Technica, less news, more opinion (and of course product placement).
This isn’t a jab at Thom, just stating a fact.
Wondercool2,
Yep, that’s an insightful point. Nothing like a bit of controversy to make people determined to speak their minds. At the same time though look at what it’s doing to society…
Like everyone else on this planet, I’ve been fed up with google for a while now.
I’m thinking of moving to Kagi (and yes, paying for a search engine). What do you people think about it? I got a trial account, but don’t have the time to make a quantitative or even qualitative comparison. It feels better than the junk google offers though.
DDG doesn’t work here in Turkey. I mean, it nominally works, but bing’s coverage of Turkish sites is so poor that it’s simply unusable.
I use Kagi for some searches (the ones that are heavily SEO-spammed), really liking personalization. It is not general Google replacement for most purposes.
If you to deshitify your Google searches start here
https://udm14.com/
It strips out all the crap they just added
Cool trick, just found out about the udm=14 URL parameter. Thanks for that!
Just found that Ars has a good article about this, including how to add a custom Google search engine with udm=14 to Chrome and Firefox, Admittedly the easier way is to add the udm14.com search engine you linked to, however I prefer the homemade way to avoid sending my data to yet another third party.
https://arstechnica.com/gadgets/2024/05/google-searchs-udm14-trick-lets-you-kill-ai-search-for-good/
I tried it but I don’t like the results. No matter what you type in, 90 percent of the top results are (big) companies or sites like Wikipedia/Reddit. Very hard to find any link to any independent website.
Wondercool2,
Capitalism ultimately consolidates everything into a few dominant players. Obviously google wield a great deal of power I strongly dislike google sending more and more clicks to other giants. But at the same time, honestly, I don’t think their algorithm goes against the flow, it’s probably mirroring market trends that are already happening. Unfortunately small players are already very disadvantaged and unfortunately when the majority of the public vote with their wallets, they flock to the big brands. There’s very few locally owned stores anymore and it’s sad but they can’t really compete.
You call this high quality, when you even cannot swipe? Oh yes
https://ibb.co/XJwjfMb
Oh yes, it’s your page. And it’s useless.
Having un-googled my life a few years back, articles like this are kind of odd to read. I know that for 75% or more of the world that it’s true that if Google changes something about their search algorithms it affects their lives directly. It’s a weird feeling to read things like this and just have no thought about it at all.
It’s a feeling like walking along a street and looking in the windows of restaurants at all the people eating, but not being hungry. It’s like that feeling I get sometimes when I ride my bicycle past a long line of cars that are stuck in a traffic jam, looking at all the angry people through their car windshields, but feeling disconnected from their experience as I leisurely ride along and enjoy the wind blowing past me. It’s like the Heinlein novel – A Stranger in a Strange Land – which was Heinlein’s take on Abraham’s experience in the land of the Canaanites. Like I’m no longer a citizen of this place, I’m just a sojourner, a passing observer.
andyprough,
I’ve done my best to avoid google accounts for myself. But unfortunately it can be a lot harder to avoid them when your work and/or school require it. They have quite a large monopoly with most of my clients. This year, as part of a project, one client was having issues with google’s local market API. After 8 or so interactions with google tech support, google had zero clue how the API worked and they couldn’t even direct us to anyone within google who could fix the problem. Maybe after their massive layoffs, they legitimately had nobody left who was able to support the API? I don’t know but as anyone who has ever dealt with google knows, a grade of “F” is par for the course for google. I am the one who didn’t get completely paid for that project because google’s API didn’t work and it couldn’t be completed. The client was attached to all the correspondence and they completely understood the situation was google’s fault. Yet at the end of the day google has the eyeballs so…google keeps getting paid no matter how bad their service is.
>”I’ve done my best to avoid google accounts for myself. But unfortunately it can be a lot harder to avoid them when your work and/or school require it.”
I understand. I wanted to get Google completely out of my life the moment the Snowden revelations were hitting the press in 2013, but it took me a few more years due to the work/school requirements.
>”I am the one who didn’t get completely paid for that project because google’s API didn’t work and it couldn’t be completed.”
Can you build that into future contracts – some sort of financial protection for yourself if Google can’t get their own API to work?
andyprough,
IDK, maybe. Every job is so different it’s hard to anticipate the roadblocks. Most jobs go fine from a technical point of view but my biggest gripe these days is offshoring. Doing the work myself is fighting the grain. The project managers seem to have a better work/life balance than coders these days, but maybe that’s just a case of “the grass is greener…” I’m just tired of the grind.
Maybe this is going to be the saving grace of AI? Stopping the grind and make it possible for humanity to spend time on art and self expression? I mean, if machines are going to do all the work, surely the capitalistic system of acquiring more money won’t make sense?
Wondercool2,
I’ve thought of that too. I think so much of what humans do is busywork that nobody likes to do but they do it because they need to make a living. Automation/AI really is one of the best opportunities to eliminate that and return our time….but the huge caveat is that under capitalism people still need to make a living or otherwise become impoverished. I agree the more that gets automated the less capitalism makes sense, however capitalism has been so ingrained into our social institutions, corporations, stock markets, government bodies including the IRS etc that it almost seems unfathomable that we would be able to change. Technology has made huge advancements in productivity in the past half century, and yet 100% of the benefit has gone to the owning class. Meanwhile working class families have made no progress relative to inflation despite productivity gains.
AI will once again bring huge new productivity boosts, but rather than using that as a genuine opportunity to make life better for everyone. I fear it will (once again) be squandered to prop up old institutions. Morally we’ve got it all backwards: rather than our economic institutions adopting to humanity, we end up adopting to our economic institutions, prolonging the suffering, the grind, the poverty that comes with them.
Regardless of feelings, Google is not the bad guy here.
As the article mentions, they have to choose the top results, somehow. And if you are not on the first page, people do not usually visit you.
(Again, we can blame other practices, like putting three rows of ads before organic results, but that is another story).
If Google has realized people, in general, prefer certain set of results over some others, they will of course tweak the algorithm to boost more popular ones, over niche results. That does not mean other things like result diversity is not taken into account (diversity as in “apple music”, “apple fruit”, “apple computer company”, and so on).
The only case could be, do that purposefully prefer increased ad revenue over decreased user satisfaction. But that is harder to measure. (Remember, those of us visiting OSNews and similar sites regularly are *not* average consumers).
sukru,
These two points do seem contradictory. You might argue that the popularity contest would take place regardless of google’s involvement, but the fact remains that google are taking place in the popularity contest and their involvement is actively harming alternatives. I see this happening all the time on their social media platforms like youtube. There is so much good interesting content on there, but google consistently promote the big channels first. I’m not denying there is a human tendency to do this anyway, but to suggest that google aren’t guilty of doing it too is just not true.
That is true, but I’m pretty sure the exact same popularity concentration that we see in our domain also takes place regardless of interests: sewing or cooking or sports or movies or whatever. In other words, we’re not exceptional.
Alfman,
Popularity is *but one of the* features used in ranking. Without going into much detail there are very large number of parameters that can be grouped into: popularity, age/recency, content quality, query topicality, and topic diversity as high level ranking features, among others.
Taking *any* of them them and using only that causes a bad experience with not useful results. This is true even for quality. (I.e.: do you want to see all your results from Wikipedia only, even when you are asking for “where is the nearest Walmart”?, assuming Wikipedia is a source with higher quality than others).
Hence, I do not expect Google to use only popularity, or even prefer popularity in ranking, however high level tune-ups might have changed the formula for certain subset of queries.
This is a common issue when optimizing for multiple and conflicting goals. For some people, returning results from only gossip magazine sites is very desirable, while others might prefer arXiv, wikipedia and pubmed. The *art* is identifying each individual users’ taste, and also the query (maybe you are a professional who wants to relax after work), and return results appropriately.
The problem is, if you entirely automate this, it is very easy to “curb stump” certain sub-populations. For example (making this up), if you return no results from Brazil your Portuguese speaking users in general might get better overall. So, maybe 10% population gets 0 score, while 90% of population gets a 11 percent score boost (again completely made up).
Fortunately there were processes at Google that prevented this scenario. I am assuming (hoping) they continue to monitor each change for their effect on all possible sub-populations.
Anyway, again, there are other things to criticize about Search, however changing the algorithm that *they think* will benefit more users at the cost of some publishers is a not a bad thing in general.
sukru,
I am only witness to what I see, but as someone who blocks trackers and avoids logging into accounts, I do believe I am seeing google’s default rankings prior to applying user specific demographics.
Granted you could say that google isn’t able to tailor content for me, and I accept that, but at the same time I hate the way google places so much emphasis on popularity by default. It’s virtually the same channels promoted over and over and over again. It’s no wonder the popular channels get more popular, when new users who come to the platform get exposed to them almost exclusively from the start.
I am not accusing google of intentionally or deliberately reinforcing the popularity feedback loop, but I do believe this is a consequence of algorithms that maximize profits. Pushing the bigger channels probably has a correlation with higher profits for google (as well as microsoft, etc).
I’ll admit it’s gotten better recently, but just a couple of years ago the amount of misinformation google were promoting on youtube’s home page was genuinely alarming. I believe youtube was successfully manipulated in exactly the way Wondercool2 described in this earlier comment and google’s algorithms were more than “happy” to oblige those content farms with millions and millions of youtube exposures.
https://www.osnews.com/story/139795/google-just-updated-its-algorithm-and-the-internet-will-never-be-the-same/#comment-10440005
Especially around the election there was so much trash content, some even obscene. I’m glad google has finally decided to put some restrictions on their own algorithm spreading misinformation. Although in the Louis Rossman link I shared earlier, he describes his channel’s experience with content filtering AI flagging some of his videos for misinformation and it wasn’t here or there, the AI flag was wrong. Turning AI into the content police is a bit concerning. Anyway that is quite a different discussion, haha.
In general I expect companies to do what’s best for themselves, and google aren’t an exception. The “do no evil” line was commendable, but it didn’t last. Money has the power to turn corporate executives into hypocrites, it seems almost inevitable. 🙁
So, Google essentially conceded defeat to spamsites with zero original content and opted to show results from the 10 biggest websites on the internet as a way to “fix” their search results,
The signs they were losing the battle were already there, the fact people started adding “reddit” to the end of their queries is a major tell, and with this latest change, they finally stopped trying.
So, what happens now? Back to web catalogs? Does this mean websites will need to have functional navigation again? Or do we just dispense with the idea of the open web altogether?
kurkosdr,
I would argue that this is not a problem specific to Google, but to web in general. And not only web search engines, but the overall web.
There is a decline in the quality of output. Even once reputable sites are now trying to fill in quotas with AI generated random text that are optimized to bring in ad revenue. Large news publishers add so many fake tabloid stories, it is difficult to distinguish what is real and what is made up.
Worse?
People do like to consume fake information. Hence the once very valuable data point “click-through-rate” (CTR), which measured what results users actually clicked for a query is no longer reliable.
As you said, it is difficult to find a solution. Yahoo Directory is long gone, and there are no viable alternatives: https://en.wikipedia.org/wiki/List_of_web_directories. Google and other search engines try to correct, but usually “overcorrect”. (I am asking for a specific result with a typo, please do not correct it!), and no I am not ready to give up on open web yet, but it will require some fixing.
Replied below (sorry, typing on phone)
kurkosdr,
I do this too for the odd times I’m not around a computer and oh boy…quoting and posting comments (with html) is quite the chore! Even basic copy and paste operations turn into a stressful trial and error operations. Fat finger syndrome can yield accidental clicks that erase your entire post. As popular as mobile devices are, it’s disappointing how bad they can be at such basic internet tasks.
/gripe
But why is there a decline in the quality of output, even on sites that aren’t of the timewaster variety? It’s because SEO begets SEO. Once someone does a trick to boost their search rankings, everyone else has to do it or risk falling behind (and I mean behind competitors and spamsites). SEO is not just about structuring your website in a certain way and having the right metadata anymore, it now involves tricks with the actual content and tricks with content “freshness”. There was a story some days ago about cnet deleting old articles to boost its “freshness” stats, an outright destructive move which however cnet deems necessary to increase its “freshness” as perceived by Google. Again, SEO begets SEO. Also, dark design patterns can trick humans to increase click-through rate.
The only viable solution I see so that the web doesn’t become “a group of five websites, each consisting of screenshots of text from the other four”[1] is a form of curated catalog, possibly with Google Search enabled within that curated catalog. But Google Search on the open web seems to have run its course as a technology, since no amount of “intelligence” can tell the spamsites from good sites anymore.
Something similar happened with email, where after years of declining effectiveness of spam filtering using “intelligent” filters, spam filters simply started allowing email only from a whitelist of known-good bulk emailers. It’s the reason bulk email services like MailChimp are a thing despite the fact it theoretically costs nothing to run an email server from your existing web server: good lack achieving whitelist status on every spam filter out there for your email server without pouring days into achieving whitelist status. But still, it’s better than only allowing emails from the top 5 email providers to get through. Something similar will eventually happen to the web.
[1] https://x.com/tveastman/status/1069674780826071040 (I want to believe it’s more than 5, but I agree with the general sentiment of this post)
kurkosdr,
That is a valid point. But let’s not ignore the rule of user behavior in this loop.
People do actually visit these sites more often. And it usually results in higher ad revenue for these publishers as well, and of course Google and other ad networks too.
Not sure we can expect to have a meaningful change, without convincing people to seek higher quality content.
Reviews. I don’t know whats good or not or what reviewer to trust. housefresh.com sounds scammy is a small mom and pop reviewer site. I can’t verify my trust of them without other people reviewing the reviews. That’s what can make Reddit so good. There are likely subredits that talk not just about air purifiers but review sites as well. Should note there are also large companies doing reviews that I *know* I can not trust. Its hard choosing things… The good news is the air purifiers I chose several years ago are also highly rated by them, so I didn’t miss much by not finding them. Reviewers can also get burn out reviewing hundreds of the same device for short periods of time, hyper focus on rarely used features, and miss what its like having them a couple months/years in. I think google got it right no that adjustment.
There is a rumored leak on Google search:
https://searchengineland.com/google-search-document-leak-ranking-442617
(Will not add further comments, except if these are true, they would potentially be used for spamming by SEO algorithms)
sukru,
Oh no. It’s little tidbits like this that become the mana of the SEO industry. Clients desperate for traffic pay SEO paladins to bark pointless websites changes to web developers. Instead of focusing our limited time and resources on genuien feature improvements we refocus our efforts on sacrificial lambs for the consideration of the search engine gods in bestowing upon us more clicks. We all do this rain dance not because it benefits end users in any significant way nor because they’ll even notice, but because everyone is so desperate for existential traffic and they recognize the fact that the google monopoly has the power to make or break most businesses on the internet.
sukru,
A few lines from your link did catch my interest….
This seems like something that could be deliberately exploited, depending on how quickly google’s crawler visits the pages.
We see this popularity positive feedback loop everywhere. The tragedy for alternatives isn’t that there’s a hidden conspiracy to keep them repressed, but that the whole world (including google) are optimized to promote celebrities and big brands because that’s what most of the world cares about. This promotes the rise in brands where celebrity names carry more weight among consumers than business experience or quality. I don’t like this but it’s futile and the mechanisms responsible are bigger than google.
I don’t really like the idea of chrome sending user browsing data to google, but here I’d like to mention that websites with a higher ratio of alternative browser users may be inadvertently penalized for it.
I don’t know if deleting content actually helps, but at face value this would seem to back the SEO companies that suggest deleting old content is beneficial for freshness rankings. The mere perception that this could be true does a massive disservice to the world as it creates motivation to delete archival content, which as a user is one of the worst outcomes to come out of SEO practices.
It’s time to try metasearch and yacy…
https://github.com/searxng/searxng
https://yacy.net