The stories of “AI” bots and crawlers absolutely ravaging websites and services keep on coming, and the amount of work people have to do just to survive these “AI” bot and crawler assaults is insane.
I run Weird Gloop, which hosts some of the biggest video game wikis ever, like Minecraft, OSRS and League. Over the last 3 years, we’ve had to spend more and more of our time fighting with this bot traffic that is spiky, disproportionately expensive, and getting harder to distinguish from humans. If we weren’t constantly mitigating the bots, they would use ~10x more of our compute resources than everything else put together – even though that “everything else” includes tens of millions of (human) pageviews and tens of thousands of edits a day.
Everyone who runs wikis is dealing with the exact same problem. The Wikimedia Foundation has a post about it impacting operations, every major wiki farm has had varying degrees of service outages, and some smaller independent wikis have been knocked completely offline. Overall, I’d guess that about 95% of all server issues in the wiki ecosystem this year have been caused by bad scrapers.
↫ cookmeplox at the Weird Gloop blog
“AI” tools are a quintessential example of “shitting where you eat”. All of these tools just suck up huge amounts of content created by actual humans, only to regurgitate bits and pieces of that content upon request according statistical models. If in that process of sucking up everybody’s content, these tools are placing such amounts of undue stress and cost on the people making and hosting that content that said people stop making and hosting such content, where are these “AI” tools going to get their content from next?
With every person that throws up their hands in the air in utter frustration as they see they’re hosting bills skyrocket and their sites become unusable, “AI” tools are agents of their own destruction, since ingesting the slop they themselves create only makes these “AI” tools worse.
Nobody has ever used “AI” to produce anything of value, after all.
No Responses