Linked by Thom Holwerda on Mon 13th Feb 2017 22:57 UTC
General Development

The POSIX standard for APIs was developed over 25 years ago. We explored how applications in Android, OS X, and Ubuntu Linux use these interfaces today and found that weaknesses or deficiencies in POSIX have led to divergence in how modern applications use the POSIX APIs. In this article, we present our analysis of over a million applications and show how developers have created workarounds to shortcut POSIX and implement functionality missing from POSIX.

Order by: Score:
TL;DR
by franzrogar on Tue 14th Feb 2017 00:10 UTC
franzrogar
Member since:
2012-05-17

POSIX can never be *outdated* as a Chair can't either.

POSIX can be *deprecated* at most, or, if there were a versioning system, it could get *new releases*.

Having cleared this, someone must be truly stupid to think the system was designed with *hassle-free, perfection, simplicity, etc.* in mind as such think will never exist (we can debate if it will be, but believe me, it won't as each program has different requierements from others).

So, we have something that can't be deprecated, something that (when designed) it was obvious that it had flaws.

Now, you're jumping because POSIX didn't take care/in mind during designing some hardware that weren't even dreamt about (GPUs for heavy 3D, VR, touchscreens, etc.) Hell, point me to one that can design something with *future* optimization that I will hire him/her right now!

So, POSIX can't be deprecated, it had known flaws when designed, and you're screaming it doesn't work perfectly with hardware to be designed in the future...

Resuming: nothing to see, move along.

Reply Score: 7

RE: TL;DR
by kwan_e on Tue 14th Feb 2017 00:37 UTC in reply to "TL;DR"
kwan_e Member since:
2007-02-18

POSIX has always existed as a few steps behind what had already been implemented, so really, it is, by design, outdated.

Reply Score: 2

RE[2]: TL;DR
by franzrogar on Tue 14th Feb 2017 08:42 UTC in reply to "RE: TL;DR"
franzrogar Member since:
2012-05-17

With your logic, then, all GNU/Linux ecosystem is outdated.

Even Windows drivers are outdated as when just released, they're working in new hardware not covered by it. I even remember when they release a new Windows version and hardware has no drivers for it.

Reply Score: 0

RE[3]: TL;DR
by kwan_e on Tue 14th Feb 2017 11:08 UTC in reply to "RE[2]: TL;DR"
kwan_e Member since:
2007-02-18

With your logic, then, all GNU/Linux ecosystem is outdated.


That's not logic: that's fact. POSIX standardization comes long after something has been in use for some time and often don't capture the state of the art.

Linux is not based on a standard, it is it's own standard in its ecosystem. It can't be outdated compared to itself.

Why don't you try to understand the actual point being made next time?

Reply Score: 2

RE: TL;DR
by ssokolow on Tue 14th Feb 2017 01:39 UTC in reply to "TL;DR"
ssokolow Member since:
2010-01-21

POSIX can never be *outdated* as a Chair can't either.


Did you read it all the way through?

Yes, they point out that graphics are one of the biggest non-POSIX things to do, but then they continue on to explore how the two runners-up are examples of things POSIX does provide which are being reinvented because the POSIX offerings are insufficient for modern uses. That sounds like the definition of "outdated" to me.

Those cases are:

* IPC: Using platform-specific in-kernel solutions like Binder and Mach IPC in place of pipes and UNIX domain sockets because the POSIX-specified ones scale poorly compared to modern zero-copy/handle-passing solutions.

* Async I/O: Replacing POSIX interfaces like select with non-standard but better-performing interfaces like epoll, kqueue, etc. and building event+threadpool abstractions which all work roughly the same way but have incompatible APIs.

Edited 2017-02-14 01:41 UTC

Reply Score: 4

RE[2]: TL;DR
by franzrogar on Tue 14th Feb 2017 08:43 UTC in reply to "RE: TL;DR"
franzrogar Member since:
2012-05-17

Did you read it all the way through?


Yes, and was quite boring and "nothing to see, move along" for me.

The examples you've just provided (IPC and Async IO) just prove that parts of POSIX were "replaced" not "outdated".

Reply Score: 1

RE[3]: TL;DR
by Bill Shooter of Bul on Wed 15th Feb 2017 14:39 UTC in reply to "RE[2]: TL;DR"
Bill Shooter of Bul Member since:
2006-07-14

No, their aren't any official POSIX replacements for those. The POSIX IPC apis are unsuitable for many applications. They were good, but are not any more.

Reply Score: 2

RE: TL;DR
by mack on Tue 14th Feb 2017 10:19 UTC in reply to "TL;DR"
mack Member since:
2015-02-18

POSIX can be *deprecated* at most, or, if there were a versioning system, it could get *new releases*.

There is a versioning system. The current version is POSIX.1-2008.

Reply Score: 1

I can't read all the paper.
by DEF- on Tue 14th Feb 2017 00:23 UTC
DEF-
Member since:
2009-01-20

I stopped at
"IPC in Linux: The D-Bus protocol"

People who wrote the article have obviously no idea what the Linux IPC is.

Reply Score: 2

RE: I can't read all the paper.
by Alfman on Tue 14th Feb 2017 00:39 UTC in reply to "I can't read all the paper."
Alfman Member since:
2011-01-28

DEF-,

I stopped at
"IPC in Linux: The D-Bus protocol"

People who wrote the article have obviously no idea what the Linux IPC is.


I believe they were talking about kdbus, which was supposed to be added to the linux kernel in order to support systemd.


Edit: I guess kdbus never got merged, or rather it did and then got pulled due to controversies.

http://www.linuxtoday.com/infrastructure/kdbus-is-merged-into-linux...

kdbus is a somewhat contentious kernel patch that is intended to provide the dbus api in kernel space. It is a kernel implementation for dbus (user space), with the initial beneficiary of the merge the systemd software that is present on most recent distributions. With linux 4.3rc1 released (which does not include kdbus), linux-next has been made available, and it does indeed include kdbus.


https://en.wikipedia.org/wiki/D-Bus
kdbus was a project that aimed to reimplement D-Bus as a kernel-mediated peer-to-peer inter-process communication mechanism. Beside performance improvements, kdbus would have advantages arising from other Linux kernel features such as namespaces and auditing,[30][34] security from the kernel mediating, closing race conditions, and allowing D-Bus to be used during boot and shutdown (as needed by systemd).[35] kdbus inclusion in the Linux kernel was proved controversial,[36] and was dropped in favor of BUS1, as a more generic inter-process communication.[37]


http://www.bus1.org/
Bus1 is a subsystem to provide object-oriented Inter-Process Communication on Linux. It is a lightweight and scalable way for services and operating system tasks to share signals, data and resources; while at the same time allowing modularization, privilege separation, information hiding and isolation.



Obviously the linux kernel (and other operating systems) also support the standard IPC functionality defined by POSIX, but the paper was looking at ways that applications are increasingly sidestepping POSIX.

Edited 2017-02-14 00:58 UTC

Reply Score: 3

RE: I can't read all the paper.
by Carewolf on Tue 14th Feb 2017 19:41 UTC in reply to "I can't read all the paper."
Carewolf Member since:
2005-09-08

I stopped at
"IPC in Linux: The D-Bus protocol"

People who wrote the article have obviously no idea what the Linux IPC is.

They also taked about QtCore based on the GNOME desktop environment..

I think they do have an idea about what they are talking about, they just know next to nothing about Linux.

Reply Score: 3

Posix
by Alfman on Tue 14th Feb 2017 00:30 UTC
Alfman
Member since:
2011-01-28

It was not so long ago many of us were complaining about posix, I guess it's about time we get an article about it ;)


I agree with many of their assertions. We do see a lot of functional divergence from posix in several areas (ipc, aio, gpu, threading). Kernels, frameworks, languages have evolved but posix mostly has not leaving us to work around it with platform specific code.

Posix isn't going anywhere though, nobody wants to pay to rewrite all that software!

Reply Score: 3

RE: Posix
by Brendan on Tue 14th Feb 2017 04:05 UTC in reply to "Posix"
Brendan Member since:
2005-11-16

Hi,

Posix isn't going anywhere though, nobody wants to pay to rewrite all that software!


We'll just end up with the equivalent of a "POSIX compatibility layer" within Linux (and other "*nix-like" OSs); where modern software bypasses most of POSIX, but it still exists (within a "POSIX standard shared library" that's implemented on top of the kernel's "not POSIX at all" kernel API) for old software.

- Brendan

Reply Score: 3

RE[2]: Posix
by Alfman on Tue 14th Feb 2017 05:50 UTC in reply to "RE: Posix"
Alfman Member since:
2011-01-28

Brendan,

We'll just end up with the equivalent of a "POSIX compatibility layer" within Linux (and other "*nix-like" OSs); where modern software bypasses most of POSIX, but it still exists (within a "POSIX standard shared library" that's implemented on top of the kernel's "not POSIX at all" kernel API) for old software.


Theoretically, that makes sense, and I'd like to see it happen so we could clean up past mistakes. But it's like moving mountains. Posix is closely tethered to just about everything. Consider libc, which is the building block for virtually everything including many languages. Does libc continue to be posix compliant? Does it embrace a new API? Do we try and support both simultaneously? Will there be incompatibilities using libraries from both APIs? Some POSIX specifications might have to remain in the kernel for technical reasons. Even if we could find some people who think it's a good idea to give linux new APIs, there will be many others who feel we were ripping out it's soul.

I'm still not sure anyone would want to pay to port software to the new APIs, my concern is an outcome where we have "yet another API" and "yet another layer" while the world remains stuck on the old API. Ultimately the risk is that we create new inefficiencies, incompatibilities and complexity even though our intention is to reduce all of those.

That said, I think there's a lot of good things we could do if we could get past the hurdles.

Reply Score: 2

RE: Posix
by FlyingJester on Tue 14th Feb 2017 05:44 UTC in reply to "Posix"
FlyingJester Member since:
2016-05-11

It's a strange implication that POSIX should include graphics. And the idea that using ioctls is a bad sign for a driver is strange, and to me implies the author has no idea what a GPU driver looks like inside. Or any driver, really. If you are not intimately familiar with both the OS and the HW, any driver will seem impossible to understand. Using ioctls is totally irrelevant.

Reply Score: 2

RE[2]: Posix
by Alfman on Tue 14th Feb 2017 06:35 UTC in reply to "RE: Posix"
Alfman Member since:
2011-01-28

FlyingJester,

It's a strange implication that POSIX should include graphics. And the idea that using ioctls is a bad sign for a driver is strange, and to me implies the author has no idea what a GPU driver looks like inside. Or any driver, really. If you are not intimately familiar with both the OS and the HW, any driver will seem impossible to understand. Using ioctls is totally irrelevant.


Obviously I can't speak for the author, but my issue isn't necessarily putting more into the standard, but just cleaning it up in general. You mention drivers, they latch in all over the place (proc, dev, sys, ioctl, syscall, netlink, pty, etc). There's no rhyme or reason for it other than "this is how it's always been" or "we just needed a place to put it". Some operations are even duplicated in multiple places by some drivers, others are not. Yes it works, but the whole thing needs a good housecleaning for the sake of consistency.

Projects like plan9 tried this before, but despite their technical progress they never gain traction. I'd like to see linux use fewer, yet more consistent mechanisms. I really like that sysfs is self-documenting, so perhaps something could be done to add reflection & metadata to all the objects exposed by the kernel.

Reply Score: 2

RE[3]: Posix
by kwan_e on Tue 14th Feb 2017 11:21 UTC in reply to "RE[2]: Posix"
kwan_e Member since:
2007-02-18

Projects like plan9 tried this before, but despite their technical progress they never gain traction. I'd like to see linux use fewer, yet more consistent mechanisms.


Maybe now I can re-discuss this without Rust vs C++ getting in your way? ;)

The reason why projects like Plan9 don't gain traction is because it takes too long to become "perfect", and inevitably there's always a real world constraint that makes "consistency" take a back seat to what we want now.

And we can see what happens when anyone tries to force Linux to become "simpler": systemd.

Reply Score: 3

RE[4]: Posix
by Alfman on Tue 14th Feb 2017 12:51 UTC in reply to "RE[3]: Posix"
Alfman Member since:
2011-01-28

kwan_e,

Maybe now I can re-discuss this without Rust vs C++ getting in your way? ;)

The reason why projects like Plan9 don't gain traction is because it takes too long to become "perfect", and inevitably there's always a real world constraint that makes "consistency" take a back seat to what we want now.


Actually, I think it's simpler than that. Plan9 failed to get critical mass for the same reason that all alternative operating systems do: compatibility with something more popular. Linux was a unix clone by design. On the one hand it didn't innovate or bring anything particularly new to the scene, but on the other hand this is exactly what corporations looking to switch from expensive proprietary unix systems wanted: a cheap/free unix clone that ran on cheap commodity hardware.


And we can see what happens when anyone tries to force Linux to become "simpler": systemd.


I don't know many who would say systemd was simpler. IMHO it got attacked largely because it violated the longstanding unix "do one thing and do it well" principal.

Reply Score: 2

RE[5]: Posix
by kwan_e on Tue 14th Feb 2017 13:58 UTC in reply to "RE[4]: Posix"
kwan_e Member since:
2007-02-18

Actually, I think it's simpler than that. Plan9 failed to get critical mass for the same reason that all alternative operating systems do: compatibility with something more popular. Linux was a unix clone by design. On the one hand it didn't innovate or bring anything particularly new to the scene, but on the other hand this is exactly what corporations looking to switch from expensive proprietary unix systems wanted: a cheap/free unix clone that ran on cheap commodity hardware.


But BSD was on the UNIX "clone" scene before Linux. So was Minix. They too suffer from a similar problem. I don't think anyone would contest that BSD is more UNIX than Linux, but people flocked to Linux. I would also argue that something can't get critical mass if it takes too long to become "perfect".

Tech history is just rife with examples of the inferior solution winning out. VHS vs Betamax, CD vs minidisc, WWW vs whatever bidirectionally indexed application layer protocol.

I don't know many who would say systemd was simpler.


I don't know much about systemd, but similar other projects were also trying to get around the mess of init scripts. And from my reading of the flamewars, it seems a lot of haters conflate distributions bundling all of systemd into the system as the same as systemd being designed that way.

Now, I haven't actually bothered looking at the source, but my understanding is that, built from scratch, it doesn't depend on a lot. It's just convenient if it was built with everything together.

IMHO it got attacked largely because it violated the longstanding unix "do one thing and do it well" principal.


Which is kind of silly, since Linux itself long abandoned the "do one thing and do it well" principle - after all, that's what got Andrew Tanenbaum in a tizzy. It's arguable how much the UNIX ecosystem itself stuck to the same philosophy. X11 is a glaring counter-example.

There's another UNIX principle: worse is better, and time and again is what wins out.

Reply Score: 2

RE[6]: Posix
by Alfman on Tue 14th Feb 2017 15:39 UTC in reply to "RE[5]: Posix"
Alfman Member since:
2011-01-28

kwan_e,

But BSD was on the UNIX "clone" scene before Linux. So was Minix. They too suffer from a similar problem. I don't think anyone would contest that BSD is more UNIX than Linux, but people flocked to Linux. I would also argue that something can't get critical mass if it takes too long to become "perfect".

Tech history is just rife with examples of the inferior solution winning out. VHS vs Betamax, CD vs minidisc, WWW vs whatever bidirectionally indexed application layer protocol.


Well, we could discuss the merits of BSD versus linux, I think BSD was better engineered and all things being equal would have been the better choice. But as I understand it in the beginning BSD was set back by legal challenges over ownership, which hurt it's reputation at the time. Although this was before I'd ever heard of it so I can't say too much about that.


I don't know much about systemd, but similar other projects were also trying to get around the mess of init scripts. And from my reading of the flamewars, it seems a lot of haters conflate distributions bundling all of systemd into the system as the same as systemd being designed that way.

Now, I haven't actually bothered looking at the source, but my understanding is that, built from scratch, it doesn't depend on a lot. It's just convenient if it was built with everything together.



Sysv init scripts really needed to be replaced, but instead of making a simple init replacement, Lennart went overboard with a kitchen sink approach, which turns me off a bit, but it is what it is.

The home grown init system I built for my linux distro is much simpler than both systemd and sysv init scripts by far because it sticks to the limited scope of launching system processes. It doesn't attempt to setup the network or anything like that because IMHO those things are out of scope and better left to other processes. It's similar to "inittab" in that it simply launches, monitors and relaunches children. But unlike inittab (which is limited to editing a text file and raising signals to control it), my init system has a control socket and a client to launch adhoc jobs and is completely scriptable.

This last feature has been really awesome for me because it means that with one-liners I can configure & launch jobs even on a read-only file system, which seems like an obvious thing but it's a stumbling block for many init systems that force everything to be mapped to a file system hierarchy.


There's another UNIX principle: worse is better, and time and again is what wins out.


I searched for this but it didn't turn up anything. I don't think it's anyone's intentional principal. It's just one of those things that happens when something reaches critical mass, whether it deserved to or not, it's inherently hard to replace once it's popular. This is why, even though I criticize it, I concede that Posix isn't going anywhere.

Reply Score: 2

RE[2]: Posix
by Megol on Thu 16th Feb 2017 15:31 UTC in reply to "RE: Posix"
Megol Member since:
2011-04-11

It's a strange implication that POSIX should include graphics. And the idea that using ioctls is a bad sign for a driver is strange, and to me implies the author has no idea what a GPU driver looks like inside. Or any driver, really. If you are not intimately familiar with both the OS and the HW, any driver will seem impossible to understand. Using ioctls is totally irrelevant.


IOCTL is one of several obvious examples that Unix doesn't really support the abstractions that are claimed as fundamental. Everything isn't a file, hacking something into the same name space as files and then use unstructured communication? It's a hack and Unix (and therefore POSIX) is full of them.

And the poster that implied that KISS is a guiding principle in POSIX systems? A fundamental misunderstanding in what simplicity is - Unix based systems aren't simple in any way and form.

Reply Score: 2

Some nice ideas there
by zlynx on Tue 14th Feb 2017 01:19 UTC
zlynx
Member since:
2005-07-20

I like the ideas on asynchronous file write completions. That would be far more useful than waiting on fsync().

And it would be great if someone in POSIX made it a requirement that rename() be atomic always, on running systems and after crash recovery.

Reply Score: 2

RE: Some nice ideas there
by Bill Shooter of Bul on Wed 15th Feb 2017 14:45 UTC in reply to "Some nice ideas there"
Bill Shooter of Bul Member since:
2006-07-14

Absolutely. Just because the headline is a bit sensational, don't dismiss some of the good ideas inside.

Reply Score: 2

Hum ..
by acobar on Tue 14th Feb 2017 01:24 UTC
acobar
Member since:
2005-11-15

I was hoping for a bit more profound discussion about the shortcomings of posix when dealing with internationalization and IPC.

To expect that the complexity of modern graphics and input interactions to be covered by posix is, frankly, a bit naive. It is something that changed a lot on every OS and on Linux we are in the middle of a transition to Wayland. It is something better left to other standard bodies.

As noted by others, the article is not up-to-date when talking about the Linux IPC, perhaps because it was published on 2016 fall and many things ended taking a different path.

Last, someone should correct the funny assertion they left close to the end of the article: :p

For example, the libglib, libQtCore, and libnspr all provide high-level thread abstractions based on the GNOME desktop environment.


Edited 2017-02-14 01:27 UTC

Reply Score: 3

KISS
by judgen on Tue 14th Feb 2017 01:59 UTC
judgen
Member since:
2006-07-12

Next you are going to tell me that flat is beautiful and KISS is no longer valid as a design principle?

Reply Score: 4

RE: KISS
by Alfman on Tue 14th Feb 2017 04:59 UTC in reply to "KISS"
Alfman Member since:
2011-01-28

judgen,

Next you are going to tell me that flat is beautiful and KISS is no longer valid as a design principle?


"keep it simple, stupid" is a valid principal, the problem is the stupidity of POSIX sometimes. It introduces unnecessary complexity/hair pulling/quirks. There are tons of things you don't think about right away, but are kind of broken when you do.

TTY/job control/session management is a hell of a lot more complicated than it needs to be. Basically these parts of unix were designed for specific use cases at that time, which became standardized. Rather than putting most of this code in the shell, where it logically belongs, it now pollutes the POSIX API with complicated and quirky behavior. Granted, most people don't actually care because they have never written their own shell or init system, but I would personally prefer for most of this stuff to just left out of the kernel entirely.

Here's one that affects most developers, whether they realize it or not: there's no way to pass specific file handles to a child process. Sounds obvious that there should be, right? But posix takes the exact opposite approach, requiring the caller of fork/posix_spawn to explicitly close everything is doesn't want to pass to the child process. The problem here is that the file handles to be closed aren't even knowable to the multithreaded or library code that would need to close the descriptors. The code spawning the process may be in library A and meanwhile a completely unrelated library B may be holding an open socket. It 100% unreasonable that these two libraries should have to know anything about each other, and yet when A spawns a child, it can leak B's socket to the child. This is unwanted behavior at best, at worst it can open up security vulnerabilities and prevents sockets from closing correctly. Well that's no good.

The POSIX compliant workaround literally requires thousands of syscalls...
http://stackoverflow.com/questions/899038/getting-the-highest-alloc...
int maxfd=sysconf(_SC_OPEN_MAX);
for(int fd=3; fd<maxfd; fd++) close(fd);



This nuance cascades into all sorts of languages and places you wouldn't expect it to, many developers don't even realize this is happening at all unless they're looking very closely!

https://groups.google.com/forum/#!topic/comp.lang.python/x-C31fCSZso
http://www.perlmonks.org/?node_id=476086
http://www.perlmonks.org/?node_id=378033

In my own code I be careful to set CLOEXEC on code that I control, but I wish linux would allow it to be set by default.


Things get even worse with unexpected posix semantics leading to possible corruption. This link below shows code for two file handles to the same file, the first holds a lock, but when the second handle is closed, the lock on the first mysteriously disappears.

https://www.samba.org/samba/news/articles/low_point/tale_two_stds_os...
Let me be clear to everyone: this behavior is never what you would want. Even experienced programmers are surprised by this behavior, because it makes no sense. Even after I've described this to Linux kernel hackers their response has been one of stunned silence, followed by a "but why would it do that"?


He goes on to explain why POSIX standardized such a critical flaw:
The reason is historical and reflects a flaw in the POSIX standards process, in my opinion, one that hopefully won't be repeated in the future. I finally tracked down why this insane behavior was standardized by the POSIX committee by talking to long-time BSD hacker and POSIX standards committee member Kirk McKusick (he of the BSD daemon artwork). As he recalls, AT&T brought the current behavior to the standards committee as a proposal for byte-range locking, as this was how their current code implementation worked. The committee asked other ISVs if this was how locking should be done. The ISVs who cared about byte range locking were the large database vendors such as Oracle, Sybase and Informix (at the time). All of these companies did their own byte range locking within their own applications, none of them depended on or needed the underlying operating system to provide locking services for them. So their unanimous answer was "we don't care". In the absence of any strong negative feedback on a proposal, the committee added it "as-is", and took as the desired behavior the specifics of the first implementation, the brain-dead one from AT&T.


Yep.

Edited 2017-02-14 05:03 UTC

Reply Score: 4

Safety
by RobG on Tue 14th Feb 2017 11:13 UTC
RobG
Member since:
2012-10-17

Another aspect, not considered in the original, is safety. In the modern world, I am not sure POSIX compliance is even desirable, as the APIs are not designed with issues such as memory safety in mind, and the world has simply changed such that this should be a consideration of any systems.

Reply Score: 2

My thoughts on this
by ahferroin7 on Tue 14th Feb 2017 13:33 UTC
ahferroin7
Member since:
2015-10-30

There are some things I find rather interesting about this:
* The fact that they use OpenGL as an explanation for POSIX not having graphics support. POSIX doesn't have graphics because graphics was and still is an entirely use case dependent aspect of a system. Disk I/O, memory management, IPC, and most of the other stuff in POSIX is pretty consistently used (even if it's not a POSIX implementation) by just about everything. Graphics isn't, and that's why it's not within the scope of POSIX. This is the same reason that there is no POSIX standard for audio.
* They list only pipes and UDS for IPC comparisons, completely ignoring shared memory segments, message queues, signals, and pseudo-terminals. Shared memory and signals are used all the time even by otherwise 'non-POSIX using' applications, and while MQ and PTS users are not as common, they still do exist. I find the lack of shared memory especially interesting, because at a low level, that's essentially what Binder and the OS X implementation of Mach IPC are (and I think it may be how Darwin implements pipes and sockets too).
* They talk about AIO being largely replaced by custom options, but don't seem to mention that almost nobody used it in the first place since it's inflexible and a serious pain to work with. Thread pools have been more popular since at least the early 90's, partly because they're way more flexible, and a lot more portable.

Reply Score: 3

RE: My thoughts on this
by Alfman on Tue 14th Feb 2017 14:59 UTC in reply to "My thoughts on this"
Alfman Member since:
2011-01-28

ahferroin7,

* They talk about AIO being largely replaced by custom options, but don't seem to mention that almost nobody used it in the first place since it's inflexible and a serious pain to work with. Thread pools have been more popular since at least the early 90's, partly because they're way more flexible, and a lot more portable.



That's somewhat of an unfair tautology though, AIO on unix is difficult and frustrating exactly because of the poor job POSIX did. AIO patterns are absolutely crucial in solving the daemon scalability problem. Windows was well ahead of unix in this regard. This problem is why every single platform was forced to implement their own AIO primitives to solve real problems with POSIX.

http://www.kegel.com/c10k.html

And by the time POSIX decided it was important to standardize something, the field was already fragmented with platform specific solutions. Beyond that the POSIX solution was fundamentally incompatible with many existing platforms, the result was that Posix compliant AIO had to be emulated using threads, which defeats the entire purpose of AIO. Posix AIO combines all the negatives of threaded solutions with all the negatives of AIO for the benefits of neither.

There is a why high performance servers like nginx, lighttp, etc use non-standards compliant mechanisms.

I know it sucks that the code isn't portable without platform specific kludges, but if POSIX had done a better job standardizing better solutions, we wouldn't be in this mess. They are really not without blame here.

Reply Score: 2