We’re all being told that “AI” is revolutionizing programming. Whether the marketing is coming from Cursor, Copilot, Claude, Google, or the countless other players in this area, it’s all emphasizing the massive productivity and speed gains programmers who use “AI” tools will achieve. The relentless marketing is clearly influencing both managers and programmers alike, with the former forcing “AI” down their subordinates’ throats, and the latter claiming to see absolutely bizarre productivity gains.
The impact of the marketing is real – people are being fired, programmers are expected to be ridiculously more productive without commensurate pay raises, and anyone questioning this new corporate gospel will probably end up on the chopping block next. It’s like the industry has become a nunnery, and all the nuns are meowing like cats.1
The reality seems to be, though, that none of these “AI” programming tools are making anyone more productive. Up until recently, Mike Judge truly believed “AI” was making him a much more productive programmer – until he ran the numbers of his own work, and realised that he was not one bit more productive at all, and his point is that if the marketing is true, and programmers are indeed becoming vastly more productive, where’s the evidence?
And yet, despite the most widespread adoption one could imagine, these tools don’t work.
My argument: If so many developers are so extraordinarily productive using these tools, where is the flood of shovelware? We should be seeing apps of all shapes and sizes, video games, new websites, mobile apps, software-as-a-service apps — we should be drowning in choice. We should be in the middle of an indie software revolution. We should be seeing 10,000 Tetris clones on Steam.
↫ Mike Judge
He proceeded to collect tons of data about new software releases on the iOS App Store, the Play Store, Steam, GitHub, and so on, as well as the number of domain registrations, and the numbers paint a very different picture from the exuberant marketing. Every single metric is flat. There’s no spike in new games, new applications, new repositories, new domain registrations. It’s all proceeding as if “AI” had had zero effect on productivity.
This whole thing is bullshit.
So if you’re a developer feeling pressured to adopt these tools — by your manager, your peers, or the general industry hysteria — trust your gut. If these tools feel clunky, if they’re slowing you down, if you’re confused how other people can be so productive, you’re not broken. The data backs up what you’re experiencing. You’re not falling behind by sticking with what you know works. If you’re feeling brave, show your manager these charts and ask them what they think about it.
If you take away anything from this it should be that (A) developers aren’t shipping anything more than they were before (that’s the only metric that matters), and (B) if someone — whether it’s your CEO, your tech lead, or some Reddit dork — claims they’re now a 10xer because of AI, that’s almost assuredly untrue, demand they show receipts or shut the fuck up.
↫ Mike Judge
Extraordinary claims require extraordinary evidence, and the evidence just isn’t there. The corporate world has an endless list of productivity metrics – some more reliable than others – and I have the sneaking suspicion we’re only fed marketing instead of facts because none of those metrics are showing any impact of “AI” whatsoever, because if they did, we know the “AI” pushers wouldn’t shut the fuck up about it.
Show me more than meowing nuns, and I’ll believe the hype is real.
- The story goes that in a French convent, at some point, one nun started meowing like a cat. Other nuns soon followed, until all the nuns were meowing like cats at set times during the day. The story is often used as an example of mass psychogenic illness, but the veracity of the meowing nuns is disputed. Still a great story. ↩︎
As someone who uses AI professionally…
I would say your metaphor is close, but not entirely correct.
It is more like herding cats
The AI acts like a teenager who has just started coding, but memorized everything on stackoverflow and leetcode. It can do mechanical parts very well, sometimes significantly faster than I can do.
However it does not actually understand intent.
So… herding cats, being able to use it is actually a skill (just like being proficient in Excel, or any other software tool).
Its amazing how many devs don’t grasp this. It generates code token by token, probabilistically, based on a vector database. This has significant limitations – the Zed editor makers had a recent blog post about this. It’s amazing to me that people use phrases like “it understand the code” when that’s as far from the truth as is physically possible – an LLM literally never “understands” anything.
That said, once you know it’s just a tool, and know how the tool works, you start working with it appropriately. For example – iterating, conversation style is almost always a bad idea. If it doesn’t generate the thing you need immediately, throw it away, modify your prompt, and try again – from scratch. This can get pretty good results – very often, far better than what you get by iteration.
LLMs absolutely do not think. They do not reason. They do not “remember” and do not train on your inputs (at least not synchronously.) They don’t learn. You are just modifying the inputs on a token generator, then it generates tokens. It’s a simple tool.
Zed blog post: https://zed.dev/blog/why-llms-cant-build-software
From somebody else that also uses AI quite alot every day (Lovable, Cursor, Warp, chatgpt etc)
In my experince – those that is complaining about the vibe coding… is usually those who is afraid they will loose their job – instead of just understanding that its just another tool.
To quote myself – Lovable can not develop anything i can’t do myself.. i just does it a hell lot faster and with more bugs” – BUT – i am more than skilled enough to actually fix those things and make it home for dinner.
Lovable is great for prototyping a new interface for a customer, its great for building initial API integration and a excellent sparing partner when discussing ideas.
Lovable is one step on the right direction – a direction that have alot of steps that needs to be taken.
Valkin,
Yes, it will be a world where some will be able to effectively use the new tools, while others will complain it is not working for them.
Many similar examples in the past, including the very computers themselves.
sukru,
I worry about people who refuse to adapt out of principal. It’s not that I disagree with them necessarily, but it’s dangerous to stay positioned in the path of AI competition. Even if you can beat AI today, ten years out the situation is likely to become worse for human workers. Maybe those who are already high enough in the ranks can keep cushy jobs until retirement, but I don’t think new generation of workers have the same luxury – they’d better find something that can’t be threatened by AI as a career.
I think ATMs are a good example.
Literally replacing an entire job category with machines.
What happened?
Bankers happened. Instead of spending time counting money and updating ledgers, bankers can do higher value work like selling mortgage, or personal banking.
(Contrast that to automobile that killed horse and buggy industry).
The same could happen in software development, as coders would move up into highly valued senior developer roles with a greater toolset.
I will bother trying AI when it actually works as advertised
I’m sysadmin by trade and quite a sucky programmer. Since AI coding showed up, I was able to fill many gaps I’ve had in my automation and create tools that simply never existed before and do exactly what I need. Great for my needs and I couldn’t care less on how, why and if others are using or not using it and what are their intentions.
Could one aspect of the promise of better “coder productivity” through AI tooling be the inconvenient truth that the “industry” has standardised on software paradigms (languages, tools, philosophies, etc.) that do not allow us humans to form an uber-{intuitive,clear} mental-model of the software-problem we are meant to solve.
One problem is that “time means money” in the corporate world and the “safe” approach is to follow “standards”; i,e. workflow patterns that appear to be safe/productive.. The problem is that some patterns may need to play-out for years/decades before it is realised that these patterns are deleterious at a fundamental level. Much time would have passed and the legacy-connection with the deleterious patterns becomes cost-prohibitive for the corporate entity to replace those bad patterns with patterns that promote a better mental-model of the respective domain-space in addition to also being productive. A simple example is the continued use, due to legacy/cost reasons, of a high-cognitive-load less-structured programming language and not being replaced with a low-cognitive-load well-structured programming language.
When our coder-related tooling has a lower-cognitive load, then our human-based cognition has more opportunity to develop; or at least be less-distracted from the respective problem-space.
“Patience is bitter but it’s fruit is sweet”.
Iterating over a problem-space using different designs is a good form of human-based learning; very useful for solving engineering type problems. You cannot expect to make the “best” decision, the first time, all the time; at least early on when having little experience in the respective discipline. There’s balance, trade-offs, risk-reward-ratio. etc. Of course, it gets much easier as you get more familiar with the constraints/patterns/etc. of the respective problem-space.
For decades I have treated coding like an “art form”; i..e. coding being a medium for creative/imaginative (i,e, “demiurgic”) self-expression and has been a very fun activity..
As an applied-scientist-based software engineer, coding with study of relevant theory, greatly helps in developing our cognition to the point where we become practical experts in the respective problem-space.
However, this takes time and the time must be done.
Now, would I want machine-centric “intelligence” to come between me and my “art form” at some core-level, affecting the development of my organic-based learning/neuron/synapses/etc. subsystems …..
“HELL NO” !!!!!!!!!!!!!!!!!!!!!!!
Many people actually fail to understand what programming is about. I recommend everyone read Peter Naur’s paper called Programming as Theory Building (1985). The part of the process that creates value is not the coding itself. It is the theory and understanding of the problem expressed in the form of code. To understand code you need to understand not only the problem it is solving, but the way the programmer understood the problem (created the theory) and how he then expressed it in the form of code.
So basically there is a secret sauce or magical something or ghost in the machine, which is the theory about the problem and the process of expressing its solution in code. The code alone is purely an appearance. It is to the theory what a cake is to its recipe. The catch with AI is that it only deals in appearances. It’s like a machine that has been fed thousands of cakes, with names and descriptions attached. Now you can ask it for a cake by giving a matching description. Or if your description falls between two cakes, the machine will mix the cakes for you.
Not the recipes. Not the ingredients. The cakes. At no point does the machine know what the appearances mean or how they are made. Its knowledge of them is based on the associated tags and the output depends solely on how the request matches the tags. You can ask an AI to program you a Microsoft Windows 10.5, and assuming that it has first been given the source code of both Windows 10 and 11, it will return you an unholy mess that is a mixture of these two that will never compile or run unless rewritten manually.
So when people predict that AI will become wonderful any time now, they often work from a misguided assumption that AI is some kind of power or energy. You only need to add more power or put in more energy, and the AI will transform that into output that has more and more the appearance of intelligibility.
This is like thinking that if you detonate just the right kind of bomb in a junkyard, you will get a Porsche. Or that you put ten thousand mediocre programmers into one giant room. Their mediocrity will somehow magically begin condensing and gaining in density, until the room appears to contain the next John Carmack. By all this I am referring to the mysterious concept that everyone assumes will save AI or propel its next advance, namely “emergent properties”.
It took me quite long to realize, because there was so much hype and so little explanations, that emergent properties are just a name given to the most impressive appearances some recent AI has been able to generate. From this people assume that there is “something” inside it, though nobody knows what, that can “take over” any day. This is like saying that while working with Excel or Matlab, some cells or some values “took over” the spreadsheet or the data structure.
It is plainly obvious that inside an AI, some tokens equal words, which naturally equal concepts. But having words or concepts (or tokens) next to one another does not constitute thinking or reasoning. Though sometimes it may give the appearance of such. These letters you are reading are an “emergent property” of the pixels on your screen. These pixels appear like letters, but at no point do the pixels itself turn into anything else.
In the end this is a bet on metaphysical materialism. You may assume that since humans can intend and reason, it must be possible to make pure matter intend and reason too. But if you start with the properties you’re trying to achieve, you’re basically left with the conviction that it should be possible, but no possible way to square the circle, i.e. translate the known properties of intention or reasoning into known properties of matter. It is obvious that something that can intend and reason can use matter to store information about intentions and reasonings, but it is neither obvious how to translate that information back into its active form nor how to process it in its stored form.
The bet is in effect that our best hope is to create a conceptual black box, such as “emergent properties”, or a physical black box (a bigger AI model than ever before), throw everything into it and hope for the best. Or that intention and reasoning do not really exist, but are, in fact, always only appearances. But then, one must ask, appearances of what, and you’re back at square one.
Eudocimus,
I appreciate your comments. However I don’t agree completely. In many ways programming big things is a matter of breaking down the problem. Small pieces are typically easy to solve. From there you abstract and build up. I think this applies to both machines and humans. It’s probably fair to say that today’s LLMs are better trained at lower level pieces. When you study so many thousands of instances of a pattern you become good at replicating it reliably. That’s what an LLM is good at and this is likely the most effective way to use them today. The programmer can break up a large problem using the divide and conquer method into many pieces that the LLM can solve. Is it doing something the human programmer can’t do? No, but can it do it much faster and less costly than humans? Yes it can.
I think we call them “emergent properties” because they surprise us. Very intelligent behavior can arise from things as dumb as neurons. Traditionally programmers put in lots of effort and human intelligence to solve specific problems. But machine learning models have already proven that emergent expert intelligence is possible using relatively simple rules. However this is very dependent on using the right tool for the job. We can’t just throw an LLM at every task. For example LLMs are good at languages but terrible at chess. Meanwhile reinforcement and adversarial learning techniques can beat humans at technical skill challenges like chess and video games, but aren’t that good at languages.
So while critics have been keen to highlight LLMs weaknesses, especially when it comes to doing things outside of it’s training set, which is fair enough. I don’t expect LLMs to beat us at programming/video games/etc. But this doesn’t mean it’s the end of the line for AI, far from it! The industry is going to evolve with hybrid AI techniques being combined. Look at it this way: I can’t beat the best human chess players, but given enough compute I can realistically train an AI that does so. LLM are moving in the same direction. I think we’ll get more mileage with an LLM if, rather than trying to feed them infinitely more data, we give them the ability to create their own reinforcement/adversarial learning models. That’s going to be the event that really kicks AI intelligence up a notch.
You’re talking about AGI here. I for one don’t understand consciousnesses in humans, and won’t pretend to understand how a machine can being conscious either. However I do know that consciousness is not a prerequisite to beat humans at programming any more than consciousness is prerequisite to beat humans at chess or go. Consciousness is an interesting topic for sure, but the reality is it doesn’t have to be solved to displace most human workers.
You’re asking all of these questions in the context of AI, but they arise in the context of ourselves too. Our lack of philosophical answers brings some people to religion, although I for one have never been satisfied that the universe’s unknowns are answered through blind faith. Perhaps more importantly, I see with my own eyes how many people with blind faith become exploited to immoral ends.
These tools are extremely good at writing code for solved problems – if most of the work you do is low effort, solved problems – then these tools are very useful. If you are writing something novel however, they are utterly useless, and even destructive. It really just depends on what you are writing. Most people are just making brochureware websites – the AI tools are great at that.
Also – I agree that “AI is useless” if you are just talking about VS Code’s co-pilot – or maybe even Cursor (haven’t tried it in a while.) But if you are still on the AI sucks bandwagon, you probably haven’t use better tools, like Cline or Kilo. Yes, they are expensive – but unlike the afore mentioned tools, they actually work.