Ubuntu has been replacing core utilities with Rust rewrites, and it’s now completed the process.
cp,mvandrmwere held back on their GNU versions in Ubuntu 26.04 LTS due to a crop of TOCTOU (time-of-check to time-of-use) issues that needed to be fixed in theuutilsversions.With those issues resolved upstream, Ubuntu 26.10 finishes the job. The ‘Stonking Stingray’ ships a full set of Rust core utilities, which encompasses common command-line tools like
↫ Joey Sneddon at OMG! Ubuntuls,cat,chmodanddu.
I’m definitely not qualified enough to make any useful remarks about this, but the idea of replacing such foundational, battle-tested utilities with brand new ones, even when written in a memory-safe language, does make feel a little hesitant. Still, at least this way Ubuntu users can work out any issues so that if and when other distributions – like the one I use, Fedora – follows suit.
@Thom,
In my opinion, in this case you worry too much: those are rather simple, well encapsulated programs which you can translate line by line from one language into another. At the end you will run Fable 5.1 over it and and you are done.
At the same time, while I am in favor of the idea writing things in RUST (instead of C), in this case it likely won’t yield much either: those are rather simple, well encapsulated programs where even C can’t get much wrong. So I don’t think, there is much risk or reward here on the technical side, but maybe its a good promotion for wider RUST adoption.
Andreas Reichel,
I think that’s a fair assessment. Although human error is ever present, it’s far less likely to happen in simple code because the code is straitforward. Simple tools happen to be the lowest hanging fruit. This is why they are ported first even though it’s actually more complex tools that have more to gain.
It also provides a great chance to test (and tune) the language features in practice. They should have done that right at begin when they developed RUST. Maybe the syntax would have looked better.
In theory there is never any difference between theory and practice. In practice, there always is.
Why even mention “line by line” translation and AI models? The uutils project didn’t start this way. It’s also likely that any kind of 1-1 translation (with or without AI) wouldn’t be as useful as you might think since the project’s aims are for making improvements over the tools.
I used a simplification for illustrating, why we should not overly concerned about robustness or maturity.
I am well aware, that the project achieved some nice performance gains and improvements.
My main point was: rewriting an atomic command in a safe language is not really a suicide mission, but a well controlled process with an almost certain outcome.
I agree about the relatively simplicity of the programs. I’d also add that by this point their functionality is so well understood that replicating them should be easier than for other tools.
The fascinating thing is that despite this Zellic’s security assessment in January still exposed numerous functionality and security issues: 73 in total including seven of critical impact.
https://github.com/Zellic/publications/blob/master/uutils%20coreutils%20-%20Zellic%20Audit%20Report.pdf
The report is well worth a read for highlighting the subtlety of issues involved, many of them logic errors that affect behaviour especially when used with other tools. Rust’s borrow checker was never going to help with these, but I agree with you that similarity of language constructs between C and Rust ought to make these sorts of errors less likely.
flypig,
Thanks for the informative link!
It’s seems pretty obvious to me that the rust developer(s) implemented their versions without looking at the original C source code as a spec. They probably implemented the features based on the manpages, but the manpages don’t cover all the cases in the GNU source code and so leaves room for implementation ambiguity.
I did scan through some of the GNU code and based on that I would say a rewrite for some poor/confusing code is warranted. But given that the scope of this project was to specifically replicate the GNU tools, the only way to do that is to take a magnifying glass to GNU’s code to replicate all its quirks. Even reasonable assumptions can be wrong.
@Alfman
They could not have taken a magnifying glass to the GNU code as it is GPL. If they did that, uutils could not be MIT licensed. This is why they need to rely on the GNU test suite and why, even though they use that, differences have still been found. But the pace of those discoveries has diminished. At this point, the uutils shipped in Ubuntu are widely enough used that we can assume few significant unknowns remain.
LeFantome,
Oh, I didn’t know they changed the license.
I bet there are a lot of small weird differences because people don’t encounter the quirky cases. Still though, given the goal to copy GNU’s behavior exactly…the source is the root of truth and avoiding it is impractical to that goal.
I randomly looked at the code for “wc” and the GNU code is significantly more complicated than the nature of the task would suggest. There is complexity to the point where it’s difficult to tell what’s going on. I actually wouldn’t be in favor of copying this code as is to rust. I feel there is merit in cleaning things up to make the code legible and maintainable.
@Alfman
> the GNU code is significantly more complicated than the nature of the task would suggest
That is generally a criticism I have seen of a lot of GNU code. Part of it may be the longevity but a lot of it also seems like over-engineering as a culture. The MUSL team seems to think that glibc is overly complex. The Chimera Linux lead has a lot of negative things to say technically about gnulib. RMS made the GCC code intentionally less modular and more difficult to work with in order to discourage extension by non-free users.
I think compatibility via external tests is fine. That is the same mechanism that ensures that new GNU releases are compatible with older ones.
Compatibility is the goal but the value of that compatibility goes down as incompatibilities get more obscure and less used. Edge cases can be reported an eventually fixed. But if 99% of us are not running into problems, it does not matter much. It is not like older and newer versions of the GNU coreutils themselves are always 100% compatible with each other. The last release of the GNU tools fixed a bug that had been there for decades and that uutils already did not have. But it was not impacting almost anybody and the fix is not going to break things that matter either. That kind of thing is to be expected.
LeFantome,
I’ve experienced many projects where bloat makes it’s way in over time, but overengineering looks like it could apply here.
It doesn’t change the logic, but the GNU uses 0=0 which isn’t the way I “think”.
I never heard about this, but if that’s true, it’s fascinating. Do you have any sources for this?
Agree, I’m not complaining, only pointing out the black box treatment will likely miss some edge cases. If nobody complains, it probably wasn’t that important, haha.
Edit: wordpress botched the formatting…ugh…
These are the same but I find it more natural to put the variable/function being tested on the left.
I’m not comparing 0 to something, but instead comparing something to 0.
@Alfman
I do not have time to find a proper paper trail showing that RMS kept GCC difficult to work with on purpose but here is something that came up on my first Google.
“This is an FSF policy. In order to protect the value of GCC, and in
order to prevent people from using devious methods to circumvent the
GPL, we are not allowed to let outside projects use gcc internals.
Of course, the code is GPL, so you can write your own interfaces if you
want, but we will not be able to accept the patches. GCC changes at
such a rapid pace that it is very expensive to maintain your own
patches, and hence this discourages most people from trying. If someone
is able to do this using existing gcc features, then we may obfuscate
the feature to prevent this use.
This policy does unfortunately complicate gcc development, and there are
a number of people that would like to see it changed, but for now we
must follow it.”
https://inbox.sourceware.org/gcc/[email protected]/
It was an issue for years (relaxed under competition from Clang). GCC was kept monolithic and tightly coupled with intentionally poorly documented internals (outside the code). GCC also limited its debugging symbols, again to make it difficult to extend and modularize.
LeFantome,
Thanks for the link. I don’t know how that comes up for you, google did not return that page at all when I searched for that literal text I stopped looking on page 4. Meh, google keeps getting worse as a search engine. Anyway…
That’s a very interesting thread…I actually had the exact same needs as the OP in the past…
To which Jim (who I don’t know) says…
I follow the point about there being no guarantee, which makes perfect sense. But I’d have to push back strongly on Jim’s view that source obfuscation is desirable. That’s the kind of crap that proprietary companies do to make software less accessible. Sure one needs to enforce their copyrights/licenses but making open source code objectively less accessible is hypocrisy that flies in the face of what open source is about. It’s an anti-feature and any corporation caught pushing out intentionally obfuscated FOSS code would be justifiably criticized for it. I would like to see evidence that it was a real FSF policy though rather than Jim’s take from 2003.
I accept your point that GCC is monolithic, not modular, and poorly documented, This has enabled LLVM to take GCC’s lunch when it comes to compiler middleware. LLVM’s intermediate representation has opened up a lot of novel use cases that just aren’t achievable with GCC.
I’m not so sure whether deliberate obfuscation explains complexity in GNU core utils. I’ll grant you it’s an interesting theory. I’m hoping it’s not true though because I don’t want to deal with the cognitive dissonance this stirs up, haha.
Ubuntu already had to roll them back once due to insufficient testing. Such well-established package has likely a multitude of tools relying on its corner-case behaviours that are hard to catch.
And they will surface at the worst possible moment.
> Ubuntu already had to roll them back once due to insufficient testing
I do not believe that actually happened. They held back some utilities due to what they found during testing. And during testing, there were some bugs, even bugs that made it impossible to build the Ubuntu iso. But none of that made it into the shipping versions.
I do not think it makes sense to both say they did not test it enough and also to complain that they found stuff during testing.
@Andreas
I agree that people worry too much in this case. I do not agree that C cannot go wrong in these cases though. These utilities largely operate on untrusted user input. There is A LOT that could go wrong. The GNU utils are well tested and widely used of course so they work well but even these mature utilities do evolve and accumulate new features. C is still easier to introduce a problem into.
On of the interesting things that were produced by this reimplementation is the test harness : from their own to the ones pointed to by Zellic, the uutils team added more test coverage they contributed to GNU core utils. That’s a good result too that will help all the core utils implementations, not only the GNU originals but things like busybox or even BSD sets too.
As for the license, it’s not mandatory to follow GPL everywhere anyway 🙂
I think everyone is underestimating how much bug fixes are enclosed in what they call “legacy” code.
Here’s an interesting quote from Joel Spolsky’s famous articles “things you should never do” (ie rewriting code that was working super well)
https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/
“The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they’ve been fixed. There’s nothing wrong with it. It doesn’t acquire bugs just by sitting around on your hard drive. Au contraire, baby! Is software supposed to be like an old Dodge Dart, that rusts just sitting in the garage? Is software like a teddy bear that’s kind of gross if it’s not made out of all new material?
Back to that two page function. Yes, I know, it’s just a simple function to display a window, but it has grown little hairs and stuff on it and nobody knows why. Well, I’ll tell you why: those are bug fixes. One of them fixes that bug that Nancy had when she tried to install the thing on a computer that didn’t have Internet Explorer. Another one fixes that bug that occurs in low memory conditions. Another one fixes that bug that occurred when the file is on a floppy disk and the user yanks out the disk in the middle. That LoadLibrary call is ugly but it makes the code work on old versions of Windows 95.
Each of these bugs took weeks of real-world usage before they were found. The programmer might have spent a couple of days reproducing the bug in the lab and fixing it. If it’s like a lot of bugs, the fix might be one line of code, or it might even be a couple of characters, but a lot of work and time went into those two characters.”
Frankly I don’t understand the motivation, if it was just to avoid the GPL, juste use the BSD implementation and make these commands option compatible with the gnu coreutils ones.
djame,
People love to reference Joel because he speaks authoritatively with absolute rules. But the problem is that absolute rules ignore the fact that the real world is not black and white. Software developers in general (Joel included) often overlook the long term costs of bad code foundations. The fact is there are times when staying with existing code is more costly and leads to less reliable software in the long term…nobody should blindly insist on keeping bad code alive on the basis of one guy’s opinion from 26 years ago. Genuinely horrible code should be seen for what it is.
To be clear, it’s not that there’s never merit in keeping old code, only that it needs to be evaluated on a case by case basis. Although I understand why people might like to cite some absolute rules to justify their opinions, it doesn’t actually fit every case.
Yes, this is often what happens with old code. If we don’t clean it out we tend to accumulate more cruft over decades.. This might still be useful, but we shouldn’t automatically assume that it is. If ancient code increases code complexity and only exists for obsolete use cases, a case might be made that it’s served it’s purpose but isn’t needed any longer.
This past year I had a project wherein I had to update code that was so spaghetti-like that it was impossible to isolate bugs and scale out. The code doing IO was literally mixed in with the GUI/menus and even across DLL boundaries…it all technically worked, but the lack of abstraction made it horrendous to work on and I was tasked to update the IO’s encryption. It quickly became clear that doing this on top of the existing code would have made a bad situation worse. Naturally the client wanted to keep the old code, but several developers from the Joel school of thinking tried and failed to maintain the project…it just got worse and worse over time because of the lousy foundational design. Finally they brought me in to fix the mess because they finally understood that the old code wasn’t viable. I ripped out all of that code and designed a robust IO subsystem for the project. It wasn’t easy, but the new code is less complex, far more manageable, easier to debug and extend.
Again I want to be clear, the decision to rewrite code can’t be dictated by absolutist rules. If the original authors had created good abstractions in the first place, it could have leaned in favor of reusing that code. But unfortunately they did not, the code was spayed everywhere, fragile and bug prone…and so these critical factors favored a re-write.
That’s a fair point. Many people feel the existing code is already good enough although there’s also the issue of unsafe programming languages. C is responsible for countless software faults. The majority of faults that we get over and over again are memory faults that are mostly exclusive to C code these days. IMHO it’s continued use seems somewhat irresponsible since it implicitly condemns the future to faulty software. I’d argue the onus is on our industry to eventually retire C so that future generations don’t have to continue repeating our mistakes forever.
I understand that not everyone agrees with this, developers who grew up on C are stubborn to change, and I include myself in this generalization 🙂
“I’m definitely not qualified enough to make any useful remarks about this, but the idea of replacing such foundational, battle-tested utilities with brand new ones, even when written in a memory-safe language, does make feel a little hesitant”
History rather adequately demonstrates that Ubuntu’s definition of “ready for release” is somewhat less stringent than I would have expected of such a major and respected distribution.
Stil, this is a .10 release so maybe Canonical will surprise me and have everything up to snuff in time for the next lts in ’28.
@FlyingBoats
I am not using Ubuntu for exactly the same reason. Although I feel like in this context this is a bit of unfair. This RUST rewrite got a lot of attention and scrutiny and has excellent test coverage. It is probably the best counter checked software package in Ubuntu right now.
You spend decades excoriating Windows for not getting rid of cruft, and now are worried about throwing away cruft while maintaining backwards compatibility? Maybe you just didn’t like Windows?
Thom wrote nothing about `backwards compatibility` or `removing cruft` in his post. Its a 1:1 replacement, just in a memory safe language. If you want to be polemic, please do much better. You make us look bad this way. Quality matters when trolling.
Does this mean Ubuntu is no longer GNU/Linux?
Lord, no. GCC and GlibC (which are GNU projects that every Linux distro depends on) are a Very Big Deal. Like, GCC is about as large and complex a piece of software as the Linux kernel and almost certainly more historically significant.
You can have GCC and open-source/Free software without Linux, but you can’t have Linux — or almost anything in post-1995 computing history — without GCC.
The Linux kernel has been compilable with Clang since ~2022. musl exists. Sure, most distros use GCC and GlibC, but neither are essential to Linux.
> GCC and GlibC (which are GNU projects that every Linux distro depends on
Not “every” distro. A few distros use LLVM/Clang instead of GCC (like Mandrake). A few use MUSL instead of Glibc (like Alpine). And some do both, like Chimera Linux.
The last part has not been true for like 6+ years now.. One of the few other major pieces, glibc, was also fixed relatively recently (aarch64/amd64 so far) as well.
Uutils has been the default on Ubuntu for months now. And, outside of very specific politically charged circles, it has been a big nothing burger in terms of end-user problems. And the edge cases that have been found are being fixed.
Think of all the workloads that have been run on Ubuntu over that time. After all, it is still one of the most popular distros both desktop and server. Uutils passes more GNU tests than there were GNU tests a year ago. The test suite has grown as differences between Uutils and GNU have surfaced over time. But that has hit a plateau and will likely stop growing at this point. Maybe something will be found in cp now that more people are going to use it.
https://raw.githubusercontent.com/uutils/coreutils-tracking/refs/heads/main/gnu-results.svg
Of course, not surprising, ignorant people act like they’ve just slapped the code together and haven’t been running thousands of unit tests on the code as they go along to make sure it is bug free and feature / behaviorally compliant.