The POSIX standard for APIs was developed over 25 years ago. We explored how applications in Android, OS X, and Ubuntu Linux use these interfaces today and found that weaknesses or deficiencies in POSIX have led to divergence in how modern applications use the POSIX APIs. In this article, we present our analysis of over a million applications and show how developers have created workarounds to shortcut POSIX and implement functionality missing from POSIX.
POSIX can never be *outdated* as a Chair can’t either.
POSIX can be *deprecated* at most, or, if there were a versioning system, it could get *new releases*.
Having cleared this, someone must be truly stupid to think the system was designed with *hassle-free, perfection, simplicity, etc.* in mind as such think will never exist (we can debate if it will be, but believe me, it won’t as each program has different requierements from others).
So, we have something that can’t be deprecated, something that (when designed) it was obvious that it had flaws.
Now, you’re jumping because POSIX didn’t take care/in mind during designing some hardware that weren’t even dreamt about (GPUs for heavy 3D, VR, touchscreens, etc.) Hell, point me to one that can design something with *future* optimization that I will hire him/her right now!
So, POSIX can’t be deprecated, it had known flaws when designed, and you’re screaming it doesn’t work perfectly with hardware to be designed in the future…
Resuming: nothing to see, move along.
POSIX has always existed as a few steps behind what had already been implemented, so really, it is, by design, outdated.
With your logic, then, all GNU/Linux ecosystem is outdated.
Even Windows drivers are outdated as when just released, they’re working in new hardware not covered by it. I even remember when they release a new Windows version and hardware has no drivers for it.
That’s not logic: that’s fact. POSIX standardization comes long after something has been in use for some time and often don’t capture the state of the art.
Linux is not based on a standard, it is it’s own standard in its ecosystem. It can’t be outdated compared to itself.
Why don’t you try to understand the actual point being made next time?
Did you read it all the way through?
Yes, they point out that graphics are one of the biggest non-POSIX things to do, but then they continue on to explore how the two runners-up are examples of things POSIX does provide which are being reinvented because the POSIX offerings are insufficient for modern uses. That sounds like the definition of “outdated” to me.
Those cases are:
* IPC: Using platform-specific in-kernel solutions like Binder and Mach IPC in place of pipes and UNIX domain sockets because the POSIX-specified ones scale poorly compared to modern zero-copy/handle-passing solutions.
* Async I/O: Replacing POSIX interfaces like select with non-standard but better-performing interfaces like epoll, kqueue, etc. and building event+threadpool abstractions which all work roughly the same way but have incompatible APIs.
Edited 2017-02-14 01:41 UTC
Yes, and was quite boring and “nothing to see, move along” for me.
The examples you’ve just provided (IPC and Async IO) just prove that parts of POSIX were “replaced” not “outdated”.
No, their aren’t any official POSIX replacements for those. The POSIX IPC apis are unsuitable for many applications. They were good, but are not any more.
There is a versioning system. The current version is POSIX.1-2008.
I stopped at
“IPC in Linux: The D-Bus protocol”
People who wrote the article have obviously no idea what the Linux IPC is.
DEF-,
Obviously the linux kernel (and other operating systems) also support the standard IPC functionality defined by POSIX, but the paper was looking at ways that applications are increasingly sidestepping POSIX.
Edited 2017-02-14 00:58 UTC
They also taked about QtCore based on the GNOME desktop environment..
I think they do have an idea about what they are talking about, they just know next to nothing about Linux.
It was not so long ago many of us were complaining about posix, I guess it’s about time we get an article about it
I agree with many of their assertions. We do see a lot of functional divergence from posix in several areas (ipc, aio, gpu, threading). Kernels, frameworks, languages have evolved but posix mostly has not leaving us to work around it with platform specific code.
Posix isn’t going anywhere though, nobody wants to pay to rewrite all that software!
Hi,
We’ll just end up with the equivalent of a “POSIX compatibility layer” within Linux (and other “*nix-like” OSs); where modern software bypasses most of POSIX, but it still exists (within a “POSIX standard shared library” that’s implemented on top of the kernel’s “not POSIX at all” kernel API) for old software.
– Brendan
Brendan,
Theoretically, that makes sense, and I’d like to see it happen so we could clean up past mistakes. But it’s like moving mountains. Posix is closely tethered to just about everything. Consider libc, which is the building block for virtually everything including many languages. Does libc continue to be posix compliant? Does it embrace a new API? Do we try and support both simultaneously? Will there be incompatibilities using libraries from both APIs? Some POSIX specifications might have to remain in the kernel for technical reasons. Even if we could find some people who think it’s a good idea to give linux new APIs, there will be many others who feel we were ripping out it’s soul.
I’m still not sure anyone would want to pay to port software to the new APIs, my concern is an outcome where we have “yet another API” and “yet another layer” while the world remains stuck on the old API. Ultimately the risk is that we create new inefficiencies, incompatibilities and complexity even though our intention is to reduce all of those.
That said, I think there’s a lot of good things we could do if we could get past the hurdles.
It’s a strange implication that POSIX should include graphics. And the idea that using ioctls is a bad sign for a driver is strange, and to me implies the author has no idea what a GPU driver looks like inside. Or any driver, really. If you are not intimately familiar with both the OS and the HW, any driver will seem impossible to understand. Using ioctls is totally irrelevant.
FlyingJester,
Obviously I can’t speak for the author, but my issue isn’t necessarily putting more into the standard, but just cleaning it up in general. You mention drivers, they latch in all over the place (proc, dev, sys, ioctl, syscall, netlink, pty, etc). There’s no rhyme or reason for it other than “this is how it’s always been” or “we just needed a place to put it”. Some operations are even duplicated in multiple places by some drivers, others are not. Yes it works, but the whole thing needs a good housecleaning for the sake of consistency.
Projects like plan9 tried this before, but despite their technical progress they never gain traction. I’d like to see linux use fewer, yet more consistent mechanisms. I really like that sysfs is self-documenting, so perhaps something could be done to add reflection & metadata to all the objects exposed by the kernel.
Maybe now I can re-discuss this without Rust vs C++ getting in your way?
The reason why projects like Plan9 don’t gain traction is because it takes too long to become “perfect”, and inevitably there’s always a real world constraint that makes “consistency” take a back seat to what we want now.
And we can see what happens when anyone tries to force Linux to become “simpler”: systemd.
kwan_e,
I don’t know many who would say systemd was simpler. IMHO it got attacked largely because it violated the longstanding unix “do one thing and do it well” principal.
Which is kind of silly, since Linux itself long abandoned the “do one thing and do it well” principle – after all, that’s what got Andrew Tanenbaum in a tizzy. It’s arguable how much the UNIX ecosystem itself stuck to the same philosophy. X11 is a glaring counter-example.
There’s another UNIX principle: worse is better, and time and again is what wins out.
kwan_e,
I searched for this but it didn’t turn up anything. I don’t think it’s anyone’s intentional principal. It’s just one of those things that happens when something reaches critical mass, whether it deserved to or not, it’s inherently hard to replace once it’s popular. This is why, even though I criticize it, I concede that Posix isn’t going anywhere.
IOCTL is one of several obvious examples that Unix doesn’t really support the abstractions that are claimed as fundamental. Everything isn’t a file, hacking something into the same name space as files and then use unstructured communication? It’s a hack and Unix (and therefore POSIX) is full of them.
And the poster that implied that KISS is a guiding principle in POSIX systems? A fundamental misunderstanding in what simplicity is – Unix based systems aren’t simple in any way and form.
I like the ideas on asynchronous file write completions. That would be far more useful than waiting on fsync().
And it would be great if someone in POSIX made it a requirement that rename() be atomic always, on running systems and after crash recovery.
Absolutely. Just because the headline is a bit sensational, don’t dismiss some of the good ideas inside.
I was hoping for a bit more profound discussion about the shortcomings of posix when dealing with internationalization and IPC.
To expect that the complexity of modern graphics and input interactions to be covered by posix is, frankly, a bit naive. It is something that changed a lot on every OS and on Linux we are in the middle of a transition to Wayland. It is something better left to other standard bodies.
As noted by others, the article is not up-to-date when talking about the Linux IPC, perhaps because it was published on 2016 fall and many things ended taking a different path.
Last, someone should correct the funny assertion they left close to the end of the article: :p
Edited 2017-02-14 01:27 UTC
Next you are going to tell me that flat is beautiful and KISS is no longer valid as a design principle?
judgen,
Yep.
Edited 2017-02-14 05:03 UTC
Another aspect, not considered in the original, is safety. In the modern world, I am not sure POSIX compliance is even desirable, as the APIs are not designed with issues such as memory safety in mind, and the world has simply changed such that this should be a consideration of any systems.
There are some things I find rather interesting about this:
* The fact that they use OpenGL as an explanation for POSIX not having graphics support. POSIX doesn’t have graphics because graphics was and still is an entirely use case dependent aspect of a system. Disk I/O, memory management, IPC, and most of the other stuff in POSIX is pretty consistently used (even if it’s not a POSIX implementation) by just about everything. Graphics isn’t, and that’s why it’s not within the scope of POSIX. This is the same reason that there is no POSIX standard for audio.
* They list only pipes and UDS for IPC comparisons, completely ignoring shared memory segments, message queues, signals, and pseudo-terminals. Shared memory and signals are used all the time even by otherwise ‘non-POSIX using’ applications, and while MQ and PTS users are not as common, they still do exist. I find the lack of shared memory especially interesting, because at a low level, that’s essentially what Binder and the OS X implementation of Mach IPC are (and I think it may be how Darwin implements pipes and sockets too).
* They talk about AIO being largely replaced by custom options, but don’t seem to mention that almost nobody used it in the first place since it’s inflexible and a serious pain to work with. Thread pools have been more popular since at least the early 90’s, partly because they’re way more flexible, and a lot more portable.
ahferroin7,
That’s somewhat of an unfair tautology though, AIO on unix is difficult and frustrating exactly because of the poor job POSIX did. AIO patterns are absolutely crucial in solving the daemon scalability problem. Windows was well ahead of unix in this regard. This problem is why every single platform was forced to implement their own AIO primitives to solve real problems with POSIX.
http://www.kegel.com/c10k.html
And by the time POSIX decided it was important to standardize something, the field was already fragmented with platform specific solutions. Beyond that the POSIX solution was fundamentally incompatible with many existing platforms, the result was that Posix compliant AIO had to be emulated using threads, which defeats the entire purpose of AIO. Posix AIO combines all the negatives of threaded solutions with all the negatives of AIO for the benefits of neither.
There is a why high performance servers like nginx, lighttp, etc use non-standards compliant mechanisms.
I know it sucks that the code isn’t portable without platform specific kludges, but if POSIX had done a better job standardizing better solutions, we wouldn’t be in this mess. They are really not without blame here.