Linked by Thom Holwerda on Tue 16th Jan 2018 22:10 UTC
Google

Tim Bray, former Google employee, currently working at Amazon, writes:

I think Google has stopped indexing the older parts of the Web. I think I can prove it. Google’s competition is doing better.

It's an interesting theory for sure, but it seems hard to back this up with any tangible evidence. How would you even test this? You can pick specific web sites to test this with, but that will always be an incredibly small - infinitesimally, unbelievably small - subset of web sites, and there's no way to extrapolate any of that to the web as a whole. To make matters worse, Google tailors search results to the information they have on you, making this even harder.

Order by: Score:
Google Has Been On A Downward Slope...
by walkerdude on Tue 16th Jan 2018 22:28 UTC
walkerdude
Member since:
2017-07-31

...since they got rid of easy boolean searches, and began putting their moronic "suggestions" at the top of the page. All I want are two search engines: one for exact text searches; one for giving me actual website addresses for companies, and not bestlistofcompanies2018lololol.menu and their delightful, useless top ten of anything.

Reply Score: 2

Savior Member since:
2006-09-02

I completely agree. I cannot quite put my finger on what got worse, though it feels as if Google had given up proper (text-based) search and just used clickthrough data. Which might be a useful resource, but it is not a panacea. In my case, the results have gotten much worse, compared to a few (?) years ago.

Reply Score: 0

SEO
by Alfman on Tue 16th Jan 2018 22:55 UTC
Alfman
Member since:
2011-01-28

Thom Holwerda,

It's an interesting theory for sure, but it seems hard to back this up with any tangible evidence. How would you even test this? You can pick specific web sites to test this with, but that will always be an incredibly small - infinitesimally, unbelievably small - subset of web sites, and there's no way to extrapolate any of that to the web as a whole. To make matters worse, Google tailors search results to the information they have on you, making this even harder.


Shouldn't be too difficult to find an SEO guys who knows, they're testing these things all the time.
I don't know if the author has evidence that those pages were crawled originally?

Through my clients though, I've experienced how difficult it can be to get pages to show up in google. It's frustrating that google not show some of our pages with literal text matches in favor of more popular sites with fuzzy matches. This is true even of pages that have been crawled. They only seem to show up after you type the domain name in addition to the query.

Alas, it's the epitome of the network effect, sites get good google rankings because they're popular, but conversely they're popular because google ranks them highly. Sites that don't get listed have to find a way to become popular beyond just putting up content. Google won't even crawl all the pages if you aren't popular enough.

While it's officially looked down upon (aka the notorious "penguin" updates), and despite google's stated goal of fighting web spam, increasing inbound links is still the main strategy that I've seen work. If you get a popular site to link to you, suddenly all of your links start getting listed (which is exactly what happened here). Google would have to concede that nothing whatsoever changed with regards to the quality of his content quality, the only thing that changed was that popular sites began linking to him.

Heck, osnews.com, with a google page rank of 6, could save many sites from the slums of the internet just by writing an article for them. Not saying you should, but that's the way it works.


Edit:
As far as literal text matches go, one reason google may ignore it is that if google always showed literal text matches, this would be easy to exploit. But it just sucks for people who are looking for a specific unpopular site with a literal text match and not being able to find it. One of my huge gripes with google is when I type something in quotes, they still return popular sites/pages that don't match the criteria?! This is NEVER useful to me, I used the quotes for a reason, so google gets a failing grade with quotes! ;)

Edited 2018-01-16 23:09 UTC

Reply Score: 3

v Comment Title
by Dr.Cyber on Tue 16th Jan 2018 23:47 UTC
RE: Comment Title
by The123king on Wed 17th Jan 2018 12:56 UTC in reply to "Comment Title"
The123king Member since:
2009-05-28

...I don't even want to click that link

Reply Score: 1

RE[2]: Comment Title
by Dr.Cyber on Wed 17th Jan 2018 13:14 UTC in reply to "RE: Comment Title"
Dr.Cyber Member since:
2017-06-17

I guess you are not a curious kind of person. But I will spoil it for you anyway: It is DuckDuckGo on tor.

Reply Score: 0

RE[3]: Comment Title
by Savior on Wed 17th Jan 2018 16:19 UTC in reply to "RE[2]: Comment Title"
Savior Member since:
2006-09-02

It has nothing to do with curiosity, but much more with common sense. Clicking on obfuscated links in the days of Meltdown and Spectre feels as safe as surfing the .ru domain...

Reply Score: 2

v RE[4]: Comment Title
by Dr.Cyber on Wed 17th Jan 2018 16:33 UTC in reply to "RE[3]: Comment Title"
RE[4]: Comment Title
by zima on Fri 19th Jan 2018 16:29 UTC in reply to "RE[3]: Comment Title"
zima Member since:
2005-07-06

feels as safe as surfing the .ru domain...

.ru domains are most likely fairly safe, it's not like Russian cybercriminals can't get .com, .net and so on domains.
Plus supposedly they have an unwritten agreement with authorities not to target local population...

Reply Score: 3

Google is shit now
by The123king on Wed 17th Jan 2018 12:55 UTC
The123king
Member since:
2009-05-28

It's just terribly awful. I find most of the time it ends up searching for stuff completely irrelevant to the search terms i put into it. It's so determined to search for what it thinks i want, instead of what i ask it to, i end up switching to competitors to actually get the results i want. It drives me fucking ballistic!

Reply Score: 0

rener
Member since:
2006-02-27

for my work on our #t2sde open source, build from source system development environment (https://t2sde.org) I constantly see obscure compile errors, due new or old gcc, glibc, musl, cross compilation, mips, powerpc, arm, assembly you name it. A decade ago Google always found other projects and bug reports for such errors. Nowadays I often get zero or not many hits. For months I already had the feeling Google's search results for hard core tech became worse, ...

Edited 2018-01-17 16:01 UTC

Reply Score: 1

rener Member since:
2006-02-27

here is what I was working on over christmas and new year and needed google for - Sgi Octane MIPS64/Linux: https://www.youtube.com/watch?v=AU_RV8uoTIo

Reply Score: 1

Google has always played with results
by CaptainN- on Wed 17th Jan 2018 18:01 UTC
CaptainN-
Member since:
2005-07-07

A little anecdote:

My tiny almost irrelevant blog (on web dev stuff) back in the day, I experimented with ads. Not because I was trying to make money, but because I wanted to see what was involved with adding them on a WordPress site.

What I found was after adding Google's AdSense to my site, the reported visits to my site jumped up a bit. When I removed them, they went right back down.

Now, I don't know how much of this is coincidence, but Google's revenue model is dependant on selling ads. They don't make any money if they send you to a relevant site, that they don't have any ads on. I don't take the requirements of a business's revenue model lightly - I'm certain that it has a very large impact on the decisions, both macro and micro made through a company (including Google and Facebook - both ad driven).

Reply Score: 3

what?
by nicubunu on Thu 18th Jan 2018 11:37 UTC
nicubunu
Member since:
2014-01-08

I am not sure what is his claim:
A. His old articles were never indexed by Google (so they didn't stop 'now')
B. The old articles used to be in Google's index database but got removed at some point (again, something else than stopping now)

Reply Score: 1

Some counterbalance...
by zima on Fri 19th Jan 2018 16:35 UTC
zima
Member since:
2005-07-06

In contrast to few posters above, I think Google is great. ;) Almost always it gives me what I want in first search, sometimes after one refinement of terms ...heck, it usually gives too many relevant links / I kinda waste time reading interesting info.
Maybe I ~subconciously learned to ask it the "proper" way, but I like to think I represent the silent majority that's satisfied with the results. ;)

Reply Score: 3