“Recent advances in Linux’s threading implementation are expected to continue to ease migration from other Unix-like operating systems. These advancements have arrived with intense activity on two fronts. First, thread-handling improvements have greatly enhanced the kernel’s scalability even to thousands of threads. Second, there are now two fresh, competing implementations of the POSIX pthreads standard (NGPT and NPTL) set to replace the aging LinuxThreads library.” Read the article at OnLamp.
Linux Multithreading Advances
About The Author
Eugenia Loli
Ex-programmer, ex-editor in chief at OSNews.com, now a visual artist/filmmaker.
Follow me on Twitter @EugeniaLoli
competition? isn’t this the wrong place to have competition?
They are both POSIX compliant, so I don’t see the problem. If you code for POSIX compliant systems, it should work no matter which library is installed on the target machine.
ummm no. competition is great. the best implimentation will win and then the winner will get more help to make his/her implimentation better.
if you only have one choice as was done with linuxthread, you end up with a crappy implimentation…like linuxthread.
even in corprate development there is competition for implimentation.
my question is, how will thiss effect the desktop? will it allow better desktop performance? or will it realy only be useful on server operations?
but as we all know, implimentations can suck even though they work.
This probably won’t effect the desktop much at all. *NIX developers are still deeply afraid of multithreading, and as such, won’t use it no matter how well it performs. The current linuxthreads implementation (aside from the POSIX complience aspect) works just fine for the small number of threads need in desktop apps (total number in the hundreds rather than the thousands). But developers don’t use it, and until they start using it, the responsiveness of desktop apps won’t improve.
since you raised the concern…I mean woudl it not be beificial or does it make it harder to port or what?
If you’re using C, then managing threads makes things a fair bit more complicated – I assume thats what Rayiner is referring to.
I also wonder how much (if at all) the NPTL was influenced by Sun’s move in the direction of 1:1 in Solaris 9. Unless NGPT can show some super-significant gains by using some of the new kernel tricks, I’d bet that NPTL will probably be the one to go with.
The generally fear multithreading because it requires locking and can lead to deadlocks. Examples:
Locking: Two threads, A and B, and global variable gVar. They have to lock it to make sure only one thread uses it at a time. For example, if both A and B each want to increment gVar
gVar = 6
A reads gVar, and stores 6
<kernel switches to B>
B reads gVar, and stores 6
B adds one to 6, and stores 7 in gVar
<kernel switches to A>
A adds one to 6, and stores 7 in gVar
Now, gVar is supposed to be 8 (because A and B both incremented it) but it is 7 instead because they tried to do it at the same time.
Deadlock:
Now, using locks have their own problems. Say there are two variables, var1 and var2. Each have their own locks. Now, both A and B need to update var1 and var2 at the same time, so they need to lock both before changing either.
The code for A:
lock(var1);
lock(var2);
<do stuff to var1 and var2>
unlock(var1);
unlock(var2);
The code for B:
lock(var2);
lock(var1);
<do stuff to var1 and var 2>
unlock(var1)
unlock(var2)
That leads to something that could run like this:
A locks var1
<kernel switches to B>
B locks var2, then tries to lock var1. A already has the lock for var1, so B waits for A to release the lock.
<kernel switches to A (because B’s waiting)>
A tries to lock var2, but B already has the lock, so it waits for B to release the lock.
Now, you’ve got A and B waiting on locks that the other holds. They’re effectively frozen at this point.
then that would be why your computer science teacher told you NOT to use Global variables 🙂
so, Multi-treading it great if you follow good programming procedures 😉
Now, you’ve got A and B waiting on locks that the other holds. They’re effectively frozen at this point
unless you use a good mutex implementation which either a) locks in order, b) releases if it cannot acquire
UNIX programmers can’t be afraid of deadlocks too much, plenty of them exist already in UNIX, particularly in NFS (in the protocol!), file-system and kernels. no UNIX kernel, to my knowledge, implements a 100% deadlock-free kernel, even Solaris and AIX. they all rely on good programming practices in the presence of limited resources. a great deal of effort is going into near-out-of-memory situations in linux, to improve its enterprise-ability.
Yeah, now if we could only all just program in assembly directly by using “good programming procdures” everything would be so much more optimal. However, we have limited intellectual resources – and threading imposes a cost on that when programming.
deb-man:
“Sequential programs often employ shared variables as a matter of convenience – for global data structures, for example – but they can be written without them. In fact, many would argue that they should be written without them.
Concurrent programs, on the other hand, absolutely depend on the use of shared components.”
– Gregory R Andrews, Foundations of Multithreaded, Parallel, and Distributed Programming
In the real world, there are things that are effectively global. The graphics card, for example, is a global resource. More than one thread at a time cannot send commands to the graphics card. If you’ve got multiple rendering threads, you’ll have to do locking, and if you have to do locking, the possibility of deadlocks creeps in. Good programming practices alleviate this problem, but don’t eliminate it entirely. In the example I gave you, the code would be deadlock free if A and B always acquired the locks in the same order (var1 first, then var2). Is grabbing var1 first then var2 vs grabbing var2 first than var1 a matter of good programming practice? Not really. It’s a matter of extreme attention to detail, which is where the dangers of multithreading creep in.
just have to say…..
the more complicated the programming, the more attention to deatil you must give.
is that it’s a lot harder (often almost impossible) to reliably reproduce locking bugs, often the bugs show up only rarely, so it’s harder to do sufficiently thorough testing and finally, it’s more difficult to debug them.
All having to do with the fact that the behaviour of your program may depend on the exact points in time at which the system switches from one thread to another.
No trolls in this one? I was just waiting for that “Yeah but Linux will die anyway…” comment.
Both of them create threads very fast. Now the interesting thing is to see who performs better runing a large amount of threads (for example Apache2)
The libraries should have binary comapatibility (to change only the kernel an the library without needing to recompile the aplication)
Deadlocks as in Rayiners example are quite rare. Simply because it is easy to avoid; you either ensure you always lock in the same order, or you can bundle the two resources in a struct or union and then use a single lock for both of them. With POSIX threads, you can use pthread_mutex_trylock(), which will return instead of blocking if it cannot lock the mutex. If your pthreads implemention has realtime support, you can use Priority Ceilings to avoid deadlocks and priority inversion problems. If anyone is interested in all this, you can read the Open Groups POSIX Threads reference at http://www.opengroup.org/onlinepubs/007908799/xsh/threads.html, which is very useful, and has some examples.
For really quick ‘n dirty coding, you can use one, big, global lock that every part of your code aquires before every critical region. Its not exactly recomended, but until recently a both FreeBSD and Linux used this “One Big Lock” approach to control concurency in the kernel.
Now having said that, yes, a lot of Linux and Unix coders seem to fear threads. Qt has the QThread class, but then doesn’t help itself with the fact that many of its own classes are not thread safe. Both Qt and GTK+ use the One Big Lock approach to threads. Basically, most desktop applications for Linux either can’t use threads, or do not see much of an improvement when using threads. No wonder that threads arn’t catching on with Linux!
I have recently added POSIX threads to one of my programs, in order to handle an asynchronous I/O situation.
This was done with GNU/Linux (Mandrake 9.0 distribution).
I found all the tools and documentation were readily available.
I think adoption of threads will come gradually but will definitely increase over time.
> it should work no matter which library is installed on the target machine.
No. They are only source compatible, not binary compatible.
Don’t expect anyone to be able(or want) to compile all their programs. Making binaries ready to install is a good thing for useres, bothering the user with library hell is not.
>This probably won’t effect the desktop much at all. *NIX >developers are still deeply afraid of multithreading
So the fact that many Gnome apps are multithreaded? KDE? Mozilla ? MySQL ? OpenOffice ? Apache2 ? Python ? means they were probably created by non unix programmers ?
We BeOS programmers have been using threads since the beginning of BeOS. Threads are great, in that they get your program to do more than one thing, allow your UI to be responsive, and keep your users happy. In BeOS there is no such thing as the Watch cursor, since the UI is always responsive and apps are designed to use 2 Threads (1 for the app, 1 for each Window).
The problem with Linux and threads, apart from the poor kernel support for them, is that there is no good model for using them in application programming at the level of an application framework, like the BeOS API. All in threading in Linux needs to be done using very low level POSIX C interfaces to the kernel that are not UI friendly. Take the BeOS API for example. It abstracts inter-thread communication using a message passing metaphor that allows programmers to communicate between threads and UI components very easily. Some locking is required, when doing writes to UI components for example, but most of this is abstracted from the programmer.
If such an elegant model were available on Linux I could see an increased use of threads. But really, who wants to do that anyway, just program in BeOS, since it supports threads in such a better way than Linux.
First of all, the kernel support for threads isn’t bad at all. The new libraries are a matter of complience with POSIX and support for highly multithreaded server applications. The Linux kernel handles threads better than the BeOS kernel ever did. On my P2-300 BeOS would choke when you got up to ~400 threads. Windows (most Windows apps are highly multithreaded) with a moderate number of IE windows open and some other apps has easily over 300 threads, and you don’t notice even a bit of slowdown. Even before the O(1) scheduler, 400 threads wasn’t too much for Linux, and Linux has been tops in the lmbench context-switch benchmarks almost forver. Second, the nice high-level thread abstractions do exist in Linux. QThread maps pretty well to BLooper. The main problem, as Vanders said, is that until recently, the big GUI toolkits on Linux were not thread-safe. Even Qt 3.x only has a lame globally shared lock for Qt, but it’s better than nothing…
“No trolls in this one?”
You spoke too soon.
The BeOS had a big issue with threads as far as I can tell, which was that it’s scheduler was pretty awful, hence the problem with ~400 threads. The OpenBeOS kernel will hopefully not have many of these issues. The big problem I see is that in BeOS and Win32, coders have been stuck with multithreading for a while and we’ve all got used to it (and in many cases would have trouble doing without it any more). In the unix world, threading support has been poorly documented and not ingrained in the way things are done. I partly blame this on the multitude of APIs available. In Win32 you have to code with win32 so you’re forced to use threads. In BeOS you have to use the BeOS api so you have to use threads. In Linux, if you’re not into the multithreading option there is always another way of doing it, so no-one developing for linux has HAD to use threads, so they avoid them. I think that multithreaded GUI stuff is essential as more and more Win32 coders (like myself) move to the linux platform and want to continue doing things (at a paradigm level, I’m not talking about specific apis) in the same way as they always have done.
I wish I understood all this Kernel lingo…