The current solutions involve packaging entire alternate runtimes in containerized environments. Flatpak, Snap, AppImage, Docker, and Steam: these all provide an app packaging mechanism that replaces most or all of the system’s runtime libraries, and they now all use containerization to accomplish this.
Flatpak calls itself “the future of application distribution”. I am not a fan. I’m going to outline here some of the technical, security and usability problems with Flatpak and others. I’ll try to avoid addressing “fixable” problems (like theming) and instead focus on fundamental problems inherent in their design. I aim to convince you that these are not the future of desktop Linux apps.
I fully agree. If you’re a Linux application developer, packaging your application up as an RPM and DEB is really all you need to do; you’ll cover by far the most desktop Linux users, and your code will most likely be packaged up by package maintainers of smaller package management systems as well. All these “solutions” just add additional layers of confusion, bloat, issues, and bugs that can be easily avoided by sticking to your distribution’s own package manager.
I simply avoid any application packaged up in any of these formats – with the exception of Steam – and move on to something from a developer who does understand and care about desktop Linux.
DEB and RPM would not help those running Arch, which seems to be growing in popularity. I suppose that as long as the binaries are available, folks inevitably will take care of packaging and maintaining it through the AUR (though that feels like a crutch).
The challenge isn’t just the packaging format; it’s also about version control. Distro maintainers are gatekeepers between your app and potential users, assuming the maintainers accept and add your app into official repos. And even then, as you continue to update and improve your app, only users running rolling distros like Arch will get the latest and greatest updates. Yes, there are ways to get newer apps on more “stable,” non-rolling distros, but these are not ideal and somewhat undermine the benefits of package managers on Linux.
It’s a simple matter to write an Arch package which extracts a deb or rpm using standard tools and converts it to an Arch package. There are many examples in the AUR. It’s not ideal, but works well enough.
Flatpak has it’s place, I don’t think it should be used for all applications. Just like your Steam example. Anything that’s already relatively large where a minor penalty for disk usage or memory usage is mostly irrelevant and outweighed by the portability advantages.
So I absolutely think it’s the future for SOME application delivery, but not all. I use a handful via flatpak because I’ve had much better success. Discord and OnlyOffice come to mind. I actually run Steam using the native distro packages.
Thom Holwerda,
On the one hand, I concur that distro packages are the easiest for users, however this centralization can be problematic for various reasons. Very frequently the distro’s packages are not up to date with the source. There are both pros and cons here, but I’ve encountered numerous occasions when I was forced to sidestep the repos. Alas that’s when the problems begin, on linux installing software is far more difficult than it should be or needs to be, Messy installs & dependency hell are all too common. This is exactly what SNAP/Flatpak are designed to help with.
If you’re a commercial software vendor, for better or worse the linux repos aren’t particularly suitable. People are often of two minds on this subject: “I don’t care about commercial software, the less of it linux has, the better” and “I’d like to use linux except the software I need is on windows”. My own opinion is that the FOSS philosophy shouldn’t be used to justify keeping installs difficult; it should be much easier to install independent software on linux. Even when it comes to open source, centralized repos aren’t everyone’s cup. I think we should concede software has been traditionally quite difficult to install on linux outside of the repos and with better tools/standards it doesn’t have to be so difficult.
The thing is that a lot of these issues can simply be worked on. Whether or not they are is another thing. Yet, there is hope that these improve.
Excessive runtimes are a thing. Older/unused runtimes can be pruned from a system.
Applications could specify if they work with multiple runtimes avoiding a download in recognition of backwards compatibility. Or perhaps something like how Steam manages proton versions where there is an easy way to change runtimes for an application locally.
Permissions and security can be improved…
All in all, there is nothing fundamentally broken about Snap/flatpak that can’t be changed in the future as these are worked on.
I think it’s great to point out their flaws, but it’s a bit problematic to say they’re not the future. What you have are a bunch of complaints that can be worked on and there’s nothing fundamental about Snap/Flatpak that prevents these issues from being solved.
The main problem is the lack of a well-defined platform API for Linux. Instead you have a huge collection of libraries, written and maintained by different groups with not much co-ordination, which may or may not be present on a given Linux distro, in differing versions. The only real stable API is the Linux kernel API which is pretty limiting. I’m glad to hear that GNU libc have committed to no breaking changes, that was a big problem in the past for binary distribution (we don’t really need any new innovations in a basic library like this). Creating a stable platform is boring and unglamorous work, the constant churn of chasing the next new shiny thing is more attractive to developers, but is bad for the long-term. Standardisation of a base system (set of libraries, minimum versions) would be a good thing. This could also work well in conjunction to Flatpak and Snap for sharing base images.
@Luke McCarthy
Maybe a shift in perspective to a more industrial view than a self-organising hacker view might help at the architectural level. It’s really a question of priorities and where you put your energy.
It’ll take more than a shift in perspective. The work of maintaining an SDK takes over a thousand programmers, and is dull work that developers don’t like spending free time on. Hence the kernel is reliable, but the desktop experience is anything but. It’ll take both unification and some serious money being dropped on hiring lots of full time developers to get a proper SDK/API for desktop.
Flatpack, snap, etc. are all a half assed alternative to having a real SDK. Until the Linux desktop devs realize this and unite behind making a reliable SDK for 3rd parties to build their apps with, desktop Linux will just have tons of unnecessary problems that don’t happen in other OSes.
The linkage-related aspects of Flatpak serve as an equivalent to WinSxS.
Even Microsoft needed something like this and they control the platform and development tools far more than Linux developers ever could. The only reason later versions of MSVC don’t use it is because Microsoft can exert that level of control to enforce proper policy for bumping in-filename versions when the ABI changes.
…not to mention that having a better SDK wouldn’t add the sandboxing part, even if you can theoretically get distros to come together on cross-distro packaging and updates aspects.
I have read the article and what it basically suggest is AppImage is the future. That is you don’t really have to include the whole runtime in the AppImage. If you can or better could rely on underlying libraries on all different Linux distributions to be “compatible”. But if that would be the case the future would likely already be here. As for RPM and DEB packages. They work great regarding packaging software for some specific Linux distribution. More specifically version of some Linux distribution. Beyond that the packages are “incompatible”. That is for majority of software you can’t simply create a RPM and DEB package to be used and shared by Linux distributions out there and to expect that is it. In a year or two it won’t work anymore on majority of Linux distributions. Hence Flatpak might not be the future. Said that the same can be said for RPM and DEB packages in their current form. In my opinion it should happen. That is acceptable packaging format for Linux for the future. That should happen and in my opinion we are not there yet.
I feel like it funny that Linus actually thinks Appimages are a better way to go about app packaging. But the purists aren’t helping in any way to make app distribution any better. So what’s te solution? Keep doing things the same as always? Because it’s “the Linux way?”
This is why Linux desktop will never take off.
I fully agree with the article author, I have been skeptical of Flatpak and AppImage ever since I first heard about them, and Snaps are completely alien to the idea of a Free operating system. Maybe I’m old fashioned, but even the dependency hell one can get into when mixing Debian repos is nothing compared to the nightmare of “sandboxed” apps that aren’t actually sandboxed, and the thought of Canonical’s proprietary, secretive servers being in charge of the apps on one’s box is monumentally creepy. I prefer security over convenience.
In other words, if you want to distribute apps for Desktop Linux, you need to use the repositories (aka the Play Store From Hell), where you hand over your code to a third party and you have no control whatsoever after that. At least developers pushing apps to the Play Store have control over the final binary and can opt for a proprietary license if they want.
BTW here is a hilarious guide to getting libdvdcss working for VLC on Ubuntu:
https://www.omgubuntu.co.uk/2012/10/how-to-enable-dvd-playback-in-ubuntu
Meanwhile, on Windows, VideoLan has full control of the VLC binary and libdvdcss simply comes backed-in. And you can install the latest VLC even on Windows 7, you don’t have to wait for anyone to “package” anything.
There was an article somewhere about how, from the perspective of the user who doesn’t care about the source code, Windows is more open because you can just install binaries without having to go through a central repository. Or have to wait for anyone to “package” the app for your distro and version combination.
So… is Flatpack a solution for a problem that shouldn’t exist in the first place? Yes. Do I still want the solution considering the problem DOES exist? Also yes.
I know it’s just an example, but it’s over 9 years old and doesn’t really help your case. Here’s the current steps needed (for Debian-derived systems):
sudo apt install libdvd-pkg
sudo dpkg-reconfigure libdvd-pkg
That’s it. The rest is handled automatically for you. And it’s even listed right on the VLC website:
https://www.videolan.org/developers/libdvdcss.html
Still, the fact I have to type these scribbles when VLC for Windows has libdvdcss backed-in is ridiculous.
I’m sorry but I’m going to have to disagree strongly with you, for the same reason that “sysvinit/OpenRC/etc. is all you need” anti-systemd people are wrong. RPM and DEB are only good enough if your use-cases are circumscribed by the needs they were originally developed for.
To some extent, the containerization side of Flatpak is EXACTLY like systemd, in that it’s a “we’ve given you twenty years to fix this. We’re tired of waiting” response to people who preach about “the right way to do it” and then fail to get sufficient traction out of initiatives like the LSB.
For example:
1. I want to run an LTS distro, but I also need one or two newer apps. My choices are a PPA, which is the wild west as far as trust goes, or Flatpak, which provides a centralized system of sandboxing, allows me to easily customize the sandboxing manifest, and, if it’s Flathub, approval isn’t *fully* automatic like for PPAs.
2. Ubuntu has decided to stop packaging Firefox with APT in favour of official Mozilla snaps. Mozilla also officially maintains their Flatpak and binary tarballs with automatic updates… which one has sandboxing?
3. I want to sandbox as many desktop applications as possible for improved security. Which is less likely to cause me strife: Retrofitting other people’s packages using Firejail or just tweaking the Flatpak installs using Flatseal?
Beyond that, I’m reminded of articles like Drew DeVault’s “Developers: Let distros do their job” which say “Ship your software as a simple tarball. Don’t ship pre-built binaries” and “One thing you shouldn’t do is go around asking distros to add your program to their repos. Once you ship your tarballs, your job is done. It’s the users who will go to their distro and ask for a new package.” and brush under the rug the effect it has: “We developers should be elite gatekeepers over everyone who doesn’t know how to compile software from source”… imagine if audiophiles got to have veto power over every product targeted at the market segment of people who don’t know how to hook up a multi-component hi-fi system!
No, this is “Red Hat/Debian/whoever didn’t make an RPM/Deb/etc. for the app I wanted” all over again… literally. Flatpak is just less willing to foist “Run
alienand pick up the pieces” off on the end users in the 21st century.No argument. Snap’s architecture is garbage, even by containerized application standards.
Do I really need to start enumerating things GNOME gets wrong in other spheres?
The Flatpak CLI is the first-party UI and it gets announcing required permissions right. The KDE Discover plugin doesn’t make a half effort like that and just presents Flatpak packages with the same UI as native packages. Don’t be that infamous anti-Flatpak FUD site I won’t give free publicity to.
In short, this is another case of “WORKSFORME/WONTFIX: Your needs are invalid”, just like the last two decades that inspired Flatpak and snaps and AppImage and the like.
The passage you quoted says “meaning that applications don’t need to do any additional work”. “Don’t need”, not “shouldn’t”. Emphasis mine.
It also omits the next sentence: “Applications that aren’t using a toolkit with support for portals can refer to the xdg-desktop-portal API documentation for information on how to use them.”
That sounds like a pretty standard “Don’t reinvent platform APIs if you can work within them” that would also be appropriate for something like ensuring reliability on the Linux Standard Base or SDL or any other means of ensuring forward compatibility with the proposed paradigm.
This argument sounds like a disingenuous case of “Any solution but mine must run up against a heavy incumbent advantage that favours my solution”. Retrofitting the platform APIs is a very elegant way to get most of the way there with little to no demand on each individual upstream developer’s time and the main reason it works as poorly as it does in my experience is that GtkFileChooserNative is a young API.
How is this any different from the mess with Debian vs. Fedora RPM vs. OpenSuSE RPM vs. etc. when it comes to package naming? A good tool can’t magically fix a bad maintainer.
Don’t play the nirvana fallacy card when APT/RPM, Linux itself, and UNIX before it, are perfect examples of “good enough” solutions running rings around attempts to attain perfection.
The post even mentions “Worse is Better” in the next section, so I get a strong sense that this is a disingenuous article, not arguing in good faith. As I said before, people who think like you have had 20 years to come up with a solution that meets the needs Flatpak has set its sights on.
More an acknowledgement that Linux can’t afford to ignore the “Windows won because it was backwards compatible with DOS. Later Windows won because all 32-bit versions of Windows are still compatible with 16-bit Windows apps. etc. etc. etc.” network effects.
Flatpak is an acknowledgement that you only argue for a clean-slate approach to sandboxing if you *want* it to fail.
Not pictured: Steam spyware-ing the user’s DNS cache and doing other shady things in the name of “anti-cheat”.
This is willfully blind at best.
Canonical already tried this using traditional APT-and-Deb packaging, no snaps needed. Containerized packaging is orthogonal to attempts to monetize.
All you need is a repository-based package manager (APT) and suitable metadata (AppStream)… both of which are requirements completely independent of these new packaging formats.
Oh, like my Humble Bundle games where, to this day, I have to root around through old Ubuntu/Debian package archives for libraries with the right `.so` version or google up answers for why they crash mysteriously on startup?
…and every time I upgrade my Ubuntu, I’ll have at least one or two games where I have to go in and delete one of the bundled libraries to resolve a “crashes on startup” issue.
Not to mention, vintage games don’t have the “security updates are being released that need to make it to the end users promptly” issue.
As a retro-computing enthusiast, this reminds me of the Good Old Days™ of Windows 9x when malware was in its infancy and gamers hadn’t yet started to trickle away to the convenience of consoles before Steam made things more convenient.
An “Easier said than done.” argument. Where do you draw the line on which libraries are safe to assume will be present in perpetuity? How do you get distros to agree on it? How do you ensure they keep offering access to the same ancient versions forever? What if upstream don’t properly follow SONAME versioning for ABI compatibility? (Automatic analysis has shown humans suck at that)
How do propose that we *now* get all the distros agreeing on dependency metadata for packages they consider beneath their notice when they weren’t before.
Again, the phrase that defines Flatpak’s raison d’etre… “things the open-source ecosystem is failing to deliver on”… just like why systemd replaced sysvinit.
I have no argument with this. As a KDE user, I agree that the “and pre-installed on the vast majority of Linux desktop” makes his argument accurate.
…I won’t use GTK 3.x now that it’s grown too many GNOME-isms and still lacks stuff Qt provides by default like minimal toolbar customization, but I agree with the idea to arm-twist GNOME into being better stewards of it.
Let me know when you manage to get sufficient buy-in on that “declare library dependencies that don’t care about Debian vs. Fedora vs. OpenSuSE vs. Arch vs. … naming conventions”, “Convince all app developers to retrofit their apps for a new API”, and “Deprecate install-time permissions so aggressively as to kill off any momentum already gained”.
Final Verdict: Talk is cheap. Prove you can do differently than the last 20 years of APT and RPM that inspired these packaging systems and I’ll listen.
Thank you for such a high quality comment, rebutting the common wisdom. I don’t have any fast and easy responses will need to ponder.
ssokolow,
I object to this stereotype of systemd critics. I strongly dislike sysvinit, but it doesn’t mean I think systemd got everything right. Anyways I think systemd is a bit off topic here, so I won’t go into it more than this.
I agree on the need for sandboxing, I’ve been calling on linux application sandboxing for ages. I’m not so sure flatpaks are the best way to do it though because security boundaries don’t always correspond with application boundaries. Sometimes it does make sense, like when applications are completely unrelated to one another. But at other times the sandboxing doesn’t really fit the desired outcome and you kind of have to fight with it. Here’s an example:
https://discussion.fedoraproject.org/t/how-to-run-firefox-and-keepassxc-in-a-flatpak-and-get-the-keepassxc-browser-add-on-to-work/19452
I’ve had my fair share of that. Obviously the package managers take care of a lot of these problems so that users don’t see them, but it seems that you would agree that it’s a major problem that this doesn’t work for independent software. One hurdle that keeps us from automating this is that C libraries don’t have the metadata needed to determine API compatibility. Sometimes it is assumed functions with the same name are compatible, but it’s not always true. Better runtime tooling, better meta data, and better practices could go a long way in making this better for all applications. Alas, history shows it’s not realistic to expect developers to change their programming languages or their practices. For this reason solutions like flatpak & snaps that don’t attempt to solve the root causes of this mess but instead just package things as is in isolated bundles with all dependencies pre-loaded may be the best that we can realistically expect to achieve.
Dependencies are hard, standardization is hard. There is no universal standard body that’s going to accomplish this. We’ve got individual distros that do a fairly good job within a distro, but they’re acting individually and not solving problems between distros. There are some problems with flatpaks, but it’s a bunch of tradeoffs and I don’t think we’re going to find a single best answer for everyone.
I generally agree with everything you’ve said.
The problem is that Flatpak is essentially an executable form of the phrase “I’m tired of waiting for you to fix this”, just like systemd was. Just like PulseAudio was.
It may ruffle people’s feathers and it may not be the ideal solution, but it’s getting adoption because people see using it as better than not using it for various reasons. (eg. Effortless installation of newer-than-repo versions of apps without risk of breaking the base system in Flatpak’s case, Bluetooth headset support in PulseAudio’s case, etc.)
The post makes all sorts of good points, but they’re all things I’ve been seeing since the days of Loki Software. Unless it can give something substantially new, it’s effectively no different from its author stamping their foot and shouting “I told you I’m working on it!” for the fiftieth time.
As for Flatpak not solving the root causes, bad actors aren’t waiting for good actors to get their act together and the people behind Flatpak don’t have the manpower to fix everything properly and we’re out of time.
In a very real sense, Flatpak, systemd, and PulseAudio are an example of when you can “fix” human problems with technical solutions, simply by powering through with something desirable enough that the people maintaining the deadlock get forced aside.
Oh, regarding the “this stereotype of systemd critics” part, I’m also a critic of systemd. I think it puts far too much into PID 1.
…but when you look at what it brings, it’s also very clear why distros and sysadmins felt it was the lesser of two evils. I wasn’t stereotyping systemd critics, but referring to a specific subset of users, who I’ve observed a disproportionate number of in the Phoronix forums.
Another thought that came to mind to add to this bit I said:
It’s also something that the big “admirable for their compatibility” platforms rely on.
WinVDM and WoW64 are Wine-like retrofits to allow 16-bit and 32-bit apps, respectively, to run seamlessly on newer Windows versions. I remember how unpleasant it was as a kid when I ran into something that required Reboot into MS-DOS Mode, and, more recently, I’ve enjoyed Raymond Chen’s tales of how much they bent over backwards to retain compatibility.
DLL Hell is something we can laugh at these days because of WinSxS, a more fragile version of what Flatpak runtimes do. Windows also bundled 32-bit replacements for 16-bit InstallShield stubs to be transparently substituted in on 64-bit Windows.
Windows did a retrofit akin to getting Flatpak’s filesystem overlays and support for portals into GTK and Qt so Vista could implement UAC without having all older apps continuing to demand an unconstrained administrator account. (Remember Registry redirection?)
Likewise, Apple implemented compatibility layers so applications that expected single-tasking, un-constrained Classic MacOS apps could continue to run on MacOS X (to use the capitalization of the time period).
Fundamentally, nobody has yet demonstrated a viable way to reconcile the desired small on-disk footprint and high application reliability. If you want Windows-style self-contained installers and/or forward compatibility, then you need Flatpak/WinSxS-style disk consumption unless you’re willing to return to the bad old days of Windows 3.1x and 9x DLL Hell.
(And I think resolving that is going to require whole new programming languages, for the same reason that not even Microsoft, Google, Apple, etc. could move the needle on “roughly 70% of CVEs are memory safety issues” when working with C or C++. If the tooling that keeps you from breaking ABI is optional or not integrated deeply enough into the production of the ABIs, then upstream is going to make mistakes. That’s just human nature.)
*facepalm* And I just realized I wrote “WinVDM” rather than “NTVDM” right as the edit counter ticked to 2 seconds.
I also forgot to mention MultiFinder as what retrofitted Classic MacOS to support cooperative multitasking.
Excellent summarization, I personally have felt like Flatpak was The Right Way for a while now, but your in-depth experience and knowledge have definitively driven the point home for me.
Maybe one good thing from all current attempts will be the introduction of a “common runtime”. That is lets say you install a Linux distribution of your choice. Can still use “traditional” packages on it. But you can opt-in or to be provided by default to a runtime that is not too excessive and “universal” apps can just target that. This runtime should be developed and maintained by some entity others would trust. A couple of years of support should be granted for each version. In this case the apps, like calculator, could be small in size and issues like the load times would not exist. AppImage, Snap, Flatpak, Steam … could all target it. Likely some approach like that is the future. Compared to current approach where i guess the fragmentation of runtime environments is the biggest no-go working against them.
Uhhh did you not see when Linus Torvalds himself explained why some of the projects he works on does not have a Linux version but DOES have Windows and Mac? Because the Linux ecosystem is a nightmare when it comes to ABI and without a standardized ABI shit breaks constantly. He says they can release a single Windows and Mac version and it will last for many years but thanks to the constantly shifting sands that is the Linux ABI anything they put out would be completely broken in a year if not less so they simply do not bother.
Just having an RPM or DEB will NOT work as things shift too quickly and soon those if not maintained will be completely trashed. I can take a fricking Vista driver from 2006 and use it in Windows 10 in 2021 and tada! It works. I have a customer on Mac and he has had zero issues moving from one version to another over the past decade, even with his brand new ARM M1 version Rosetta just kicked in when he installed his Intel based apps and it “just worked”.
Now try that with Linux, trust me I know as I have an AMD APU based laptop I’d love to run Linux on as even with an SSD it takes Win 10 ages to load…can’t because while I can just load the Win 7 X64 driver and have full hardware accelerated GPU support the Linux driver hasn’t been supported since 2014, won’t load on any distro I’ve tried, and I’m not a comp sci major so I can’t rebuild an APU driver from 2014 to work on 2021 Linux so nope, not gonna happen. I’ve seen the same with older Linux apps that are no longer supported, unless you can edit the source code and fix it yourself? GLWT, meanwhile my copy of Office 2K and even the majority of my Win 9x games “just work”.
For curiosity, has anyone measured how static libraries would compare?
Given that Flatpak applications are unpacked on disk, in a maze of folders ensuring no naming collisions between different minor revisions of the same library, and any performance loss is the same as running a traditionally packaged application under Firejail (i.e. just the effects of the syscall filter), I’d say that the performance line from this should apply:
Dynamic linking – drewdevault.com
TL;DR: Statically linked applications start significantly faster.
There are two problems with that:
1. Static linking kills the ability to apply fixes to open-source components used in closed-source applications. This was a big enough problem for games that SDL 2 reinvented dynamic linking internally. (Even if you statically link SDL 2, the
*_Init()functions will look for aSDL_DYNAMIC_APIenvironment variable pointing at a newer version of SDL and fix up the library’s internal dispatch table to use it.)2. It destroys what dependency deduplication Flatpak gets from the underlying OSTree system
Brief rebuttal to the arguments in TFA:
https://blogs.gnome.org/wjjt/2021/11/24/on-flatpak-disk-usage-and-deduplication/
That missed why Windows 11 takes 27Gb clean install. Large amount of that 27GB is SXS stuff as in windows version of flatpak with many different versions of runtimes so applications work.
Application compatibility of windows and mac OS has a huge amount of OS included scaffolding so it works.
Distro packages have a HUGE advantage over Flatpacks etc produced directly by the developer. Another set of eyes of the distro package maintainer.
If Linux went totally to flatpaks, etc that you get directly from the software developer directly, there is a danger it could become windows-like, in that we might find that some software would start to be packaged with spyware, extra trial software, advertising, etc. And be back to what it is like now on windows, where you have to go to a site to download an exe, and you really have no idea what it is going to install.