Who doesn’t love a bug bounty program? Fix some bugs, get some money – you scratch my back, I pay you for it. The CycloneDX Rust (Cargo) Plugin decided to run one, funded by the Bug Resilience Program run by the Sovereign Tech Fund. That is, until “AI” killed it.
We received almost entirely AI slop reports that are irrelevant to our tool. It’s a library and most reporters didn’t even bother to read the rules or even look at what the intended purpose of the tool is/was.
This caused a lot of extra work which is why we decided to abandon the program. Thanks AI.
↫ Lars Francke
On a slightly related note, I had to do search the web today because I’m having some issues getting OpenIndiana to boot properly on my mini PC. For whatever reason, starting LightDM fails when booting the live USB, and LightDM’s log is giving some helpful error messages. So, I searched for "failed to get list of logind seats" openindiana, and Google’s automatic “AI Overview” ‘feature’, which takes up everything above the fold so is impossible to miss, confidently told me to check the status of the logind service… With systemctl.
We’ve automated stupidity.
I’m surprised you still have that nonsense turned on in your browsers.
I’ve NEVER seen it… probably because they rolled it out in Canada later than the U.S. and I jumped the gun with things like adding &udm=14 to my search templates.
On the upside, the degradation of search has ensured that I have been changing my search habits. If, for example, I want to know how to set the offline mode in the alpine mail client, I search for said client’s home page, and navigate the documentation.
(Bonus points for those who catch the joke)
I have an idea! When you wrote an article, ask an AI to rewrite using less toxic language. Maybe I’ll write a browser plugin for it. 😀
Or perhaps you can use AI search to find articles that live up to your expectations of AI hype?
Sure, the problem isn’t AI’s negative value, it’s people apparently being mean about it. I, for one, am glad we have you to respond to every example of AI unambiguously making the world worse, churning out one empty reply that ignores the article itself after another.
Hey, I’m just doing my part. When someone puts quotes around “AI” while knowing basically nothing about the technology and refuses to explain what AI with no quotes would I feel it’s my duty to bring balance to the force.
As for the shitty uses of AI, yeah, corporations suck. Always did. That’s not news. AI development itself is news.
Alright I’ll bite. Firstly, what is being referred to by the “AI” buzzword is ML (Machine Learning). Referring to LLMs as “AI”, would be as silly as referring to apples as oranges, just because both are technically fruits.
Now the “AI” marketing bandwagon has shifted to claiming that they are on the road to reaching AGI (which is what the term AI originally meant). They are on the cusp of achieving it any day now, we are on precipice of AI replacing all human performed tasks and human robots are just around the corner.
This reeks of the same bs from the DotCom era. Crypto (Web 3) did not take off, so the tech bros are now riding this new hype horse. We’ve grown accustomed to the lack of accountability in the last 2 decades, so why not let “AI” do it for us. Let’s create more misinformation through ML hallucinations (i.e. Artificially Stupid Syndrome – ASS) and let our thinking caps atrophy. But don’t you worry, regardless what actual experts say in the field (I’m not referring to dishonest, overpaid execs), we can disinvest in our crucial basic skills ’cause “AI” is gonna do it all for us.
Ok, so first of all, AGI is a meaningless term that people define in vastly different ways, so I’ll ignore it.
Next, OK fine, AI is “just” ML. However, machine learning means that machine are… learning. There is plenty of papers showing that LLM are able to generalize and perform computation.
The success of the technology is self-evident to anyone bothering to actually use it.
Is it human intelligence? No. But, is it intelligence? Of course! Try asking it a question that you are sure no one has asked and it will respond fine.
As for the hallucinations, take a piece of paper and write a fully correct program utilizing multiple libraries. Don’t make any mistakes, lest you hallucinate.
I am not saying, we have HAL9000, but I’m saying dismissing the technology because tech bros want to make money is myopic.
And I do claim, it is AI! We don’t have a better term for what we are seeing.
Here is the literal definition from a dictionary:
artificial intelligence
: the capability of computer systems or algorithms to imitate intelligent human behavior
Source: https://www.merriam-webster.com/dictionary/artificial%20intelligence
So no, “AI” is not ML. ML is just a subset of AI at best. It is closer to statistical methods.
Writing a fully correct program is doable if you’re a senior dev. In case you do not know it yet. you still need an experienced dev to review/validate AI generated slop for accuracy, performance and maintainability. Yeah, “vibe coding” is not an actual thing that will replace devs.
HAL9000? You do know that HAL almost killed the entire crew right? I hope you realize that there is a lesson to be learnt here.
Well we certainly don’t want to hurt AI’s feelings!
@drstorm: Sure AI is very useful in medicine, security and other fields. The current focus though is organized theft of copyrighted material so excuse me for a while if I’m skeptic about this technology that is _designed_ to make up stuff when it lacks data for a suitable answer.
OlaTheGhost,
For better or worse, creating new expressions of ideas published in other works is not considered copyright infringement. Humans are allowed to do this. As far as I can tell there’s no concept of “only humans are allowed to do this” written into the law….so it’s a muddy topic!
I take issue with this claim that it “is _designed_ to make up stuff”. It’s a consequence of the predictive modeling. Yes it’s unfortunate that predictions are wrong, but saying it’s by design implies a goal that isn’t true. When weatherman make bad predictions, we’re annoyed but it would not be right to accuse their models of making up stuff “by design”. The more (good) data a model has, the more accurate predictions become. Having too few data points make models less reliable, including LLMs.
That said, I agree it’s a problem. Maybe a solution would be to have LLMs output a degree of confidence alongside the output. Doing this could be genuinely useful for tackling the misinformation problem (*). I think it’s a great idea that could give users more information when a predictive model isn’t confident.
To be clear, “confidence” in this context means that there is substantial confirmation within the NN with few contradictions…but it still would not make the LLM impervious to the “garbage in garbage out” problem.
I recognize this. There were also cases where colleagues suggested things that did not make sense or contradicted the documentation. When I asked where it came from, the answer was “Google”. I’m making a point now of telling people to read the documentation, showing how to get the documentation, the man command, etc. Also people are switching search engines.
People will have to relearn to use the actual sources of information. Integrating large-language models, which were never intended to be accurate just realistic human-like, into search engines doesn’t make any sense. Probably it was done in panic.
> People will have to relearn to use the actual sources of information.
Hear hear! Like documentation was never wrong or outdated. Have you ever tried to use a rather old software which has been salvaged/ported since (several times) and was a bit more complex than the normal core-utils commands? Something like a Parser Generator or an Accounting Rules Engine or a UI Toolkit?
Welcome in documentation hell! And to be clear: I plead guilty as in “death sentence” for committing the same crime continuously.
(Side note: Actual source is the only true documentation.)
I myself find ChatGPT pretty much the best source of “software documentation” because it provides me super fast with practical examples (matching my spec) that **could** work and allow me to test it immediately without reading loads of abstract grammar. Sure, it also fails sometimes but then it still leaves enough bread crumbs to find the solution.
And what will happen when the man page makers and documentation writers start using LLMs to create manuals?
I also wonder how good those AI driven “search” engines will be in the future when more and more content on the web is going to be generated by machines and original material writers desperately trying to block crawlers from picking up their musings and/or are desperately trying to extract money from AI companies (Reddit for instance).
The shit show will get worse.
If it’s intelligent you can correct it when it’s wrong, that’s what we used to do with kids.
It also believes that France won the 7 years war.
Well…. ChatGPT told me that OS/2 Warp was open source. I correct it and said “I’m sorry”.
At least it’s politely wrong.
Did you ask it for the repo? Maybe it knows something we don’t!
Wow, who would have thought that AI wouldn’t be useful for such a mainstream use-case as troubleshooting OpenIndiana display manager… Really doesn’t make sense that it would only work well for niches such as Linux, macOS and Windows.
I had a bit the opposite experience the other day.
I was trying to get BrowserStack working with Selenium and it just was not working. All the Google searches and YouTube videos were telling me to do something that did not work (no longer worked it turns out). The documentation the BrowserStack website was telling me to do the same stuff (that did not work).
I wasted well over an hour going through all of this. I mean, shouldn’t the company website be authoritative for its own API?
Finally, I asked an LLM to write the code. For its first attempt, it did it exactly the way that everything above told me to do. Again, it did not work. But this time, I fed the error messages back into the chat and it had an epiphany about thread safety and suggested slightly different code. It still failed to create a remote web driver so I fed the new errors back to the LLM. Suddenly it realized that it “was using the old API” and suggested new code. This was it!! There was one more round of smaller errors but I got working code from the LLM. More importantly, I discovered the “new” API that was not what even BrowserStack themselves was offering me. I was then able to complete the task myself.
I guess I was “vibe coding” though all I really wanted was to search.
I have not been a particularly avid user of AI to this point (hence all my manual searching and doc reading). In the future, however, I will be faster to reach for the AI for its vast reach and ability to quickly source resources that I may not be finding. I do not want to rely on AI but it is another arrow in the quiver. It is not like every Google yields gold either. The two tools can be complimentary.
LeFantome,
Despite limitations, I am impressed with what LLMs are cable of. They contain so much knowledge, sometimes more than subject matter experts themselves. But their inability to learn and observe impedes their effectiveness. Right now a human (you in your example) needs to manually test and observe the LLM’s output. This has to be “outsourced” to humans because LLMs don’t have facilities to test/execute instructions. A major shortcoming for current generation LLM is that they rely on static “facts” that make up their training. but I do think it’s one that can be solved in domains like programming by having LLMs that can go out and iteratively test their output on actual development environments. In the future I expect LLMs to have more tooling conducive to reinforcement learning. In much the same way you were able to collective more information for the LLM to improve it’s answer, it’s a process that will become automated.
Here’s an example to think about: Today LLM’s are comedic at playing chess, largely because they don’t have a chess engine. The solution is not to use LLMs to play chess directly, but rather to have iterative LLMs that are proficient at reinforcement learning write formidable chess software. Rather than trying to guess the next best move, the LLM will be able to query the software generated through reinforcement learning.
This is similar to humans writing AI bots that play even better than the developers who built them. IMHO this will be the next generational leap for LLM & AI.
“I can’t play chess very well, but what I can do is write a bot that does”
– Future LLM