Linked by Thom Holwerda on Fri 11th Sep 2009 14:15 UTC
Mac OS X One of the main new features in Apple's new Snow Leopard operating system has been released as open source. Apple has released the code of the userland portion of its Grand Central Dispatch technology under the Apache License, version 2. Mac OS X also has kernel support for Grand Central Dispatch, which is also released as open source via the XNU project. While we're at it, let's take this opportunity to look into exactly what Grand Central Dispatch is.
Order by: Score:
Compared to .Net ThreadPool
by jpobst on Fri 11th Sep 2009 14:31 UTC
jpobst
Member since:
2006-09-26

I had assumed this Grand Central was something magical that addressed a lot of the issues of parallel programming like concurrency and data sharing. After reading this, it sounds like it is .Net's (and I'm sure other languages) ThreadPool with a lot of Apple marketing.

How is this different/better/revolutionary from ThreadPool? Just that fact that its at the OS level instead of the runtime level?

Reply Score: 1

RE: Compared to .Net ThreadPool
by kaiwai on Fri 11th Sep 2009 14:42 UTC in reply to "Compared to .Net ThreadPool"
kaiwai Member since:
2005-07-06

I had assumed this Grand Central was something magical that addressed a lot of the issues of parallel programming like concurrency and data sharing. After reading this, it sounds like it is .Net's (and I'm sure other languages) ThreadPool with a lot of Apple marketing.

How is this different/better/revolutionary from ThreadPool? Just that fact that its at the OS level instead of the runtime level?


So you ignore a 23 page article from Arstechnica; sufficient space spent to explaining GCD, and yet, you still see fit to whine on this forum that it is little more than 'marketing fluff'.

Reply Score: 2

Thom_Holwerda Member since:
2005-06-29

So you ignore a 23 page article from Arstechnica; sufficient space spent to explaining GCD, and yet, you still see fit to whine on this forum that it is little more than 'marketing fluff'.


Or, he's asking a legitimate question about what sets GCD apart.

Siracusa explains:

Those with some multithreaded programming experience may be unimpressed with the GCD. So Apple made a thread pool. Big deal. They've been around forever. But the angels are in the details. Yes, the implementation of queues and threads has an elegant simplicity, and baking it into the lowest levels of the OS really helps to lower the perceived barrier to entry, but it's the API built around blocks that makes Grand Central Dispatch so attractive to developers. Just as Time Machine was "the first backup system people will actually use," Grand Central Dispatch is poised to finally spread the heretofore dark art of asynchronous application design to all Mac OS X developers. I can't wait.

Reply Score: 10

buzievagy Member since:
2009-01-23

Exactly Thom. As in John Siracusa's article - the devil is in the details.

Edited 2009-09-11 15:02 UTC

Reply Score: 2

kaiwai Member since:
2005-07-06

So you ignore a 23 page article from Arstechnica; sufficient space spent to explaining GCD, and yet, you still see fit to whine on this forum that it is little more than 'marketing fluff'.

Or, he's asking a legitimate question about what sets GCD apart.


No, what ticked me off was this:

it sounds like it is .Net's (and I'm sure other languages) ThreadPool with a lot of Apple marketing.


And ignores what makes it unique when compared to other implementations of such an idea. He might as well trolled in saying, "nothing special, Apple is just copying Microsoft" for which his post could be summarised down to.

Reply Score: 2

Thom_Holwerda Member since:
2005-06-29

And ignores what makes it unique when compared to other implementations of such an idea. He might as well trolled in saying, "nothing special, Apple is just copying Microsoft" for which his post could be summarised down to.


Of course he ignored what makes it unique, as that was his very question!

Next time, try not to be so aggressive when someone is trying to ask an honest question.

Reply Score: 1

kaiwai Member since:
2005-07-06

Of course he ignored what makes it unique, as that was his very question!

Next time, try not to be so aggressive when someone is trying to ask an honest question.


Which is why I pointed him to the Arstechnica website.

If the answer to the question was incredibly difficult to find - then I'd cut him some slack but it isn't the case. The answer was sitting in a 23 page review.

Reply Score: 2

cb_osn Member since:
2006-02-26


Siracusa explains:

"Those with some multithreaded programming experience may be unimpressed with the GCD. So Apple made a thread pool. Big deal. They've been around forever. But the angels are in the details. Yes, the implementation of queues and threads has an elegant simplicity, and baking it into the lowest levels of the OS really helps to lower the perceived barrier to entry, but it's the API built around blocks that makes Grand Central Dispatch so attractive to developers. Just as Time Machine was "the first backup system people will actually use," Grand Central Dispatch is poised to finally spread the heretofore dark art of asynchronous application design to all Mac OS X developers. I can't wait.
"
I love working with closures/blocks, but I'm not sure why being able to execute them on a separate thread is so special. C# has been able to do this with anonymous methods since 2.0 and it hasn't brought any threading nirvana to the .NET world.

Developers avoid writing multithreaded code not because it is difficult to execute or schedule, but because it is difficult to synchronize shared resources. That is the hard problem. Solve that and then we reach threading nirvana.

Not to mention that the block implementation in Objective-C has some potentially serious memory leak issues when used without garbage collection enabled or on platforms where it is unavailable (iPhone/iTouch).

Reply Score: 3

RE[4]: Compared to .Net ThreadPool
by tobyv on Sat 12th Sep 2009 04:35 UTC in reply to "RE[3]: Compared to .Net ThreadPool"
tobyv Member since:
2008-08-25

It has been explained to me that it is possible to avoid locking using blocks+GCD, and that this could be done by creating threads by using the block syntax with IBDispatch calls, kind of 'in-line' threading.

This provides multithreading but also ensures that code can be structured in such a way as that shared resources will be accessed serially. So with careful planning (so the story goes) locking can be avoided entirely, or at least reduced.

Not likely to be developing OS X anytime soon, so I can safely say that this is truly threading nirvana, without having to worry about being bitten by my own words. ;)

Reply Score: 1

RE: Compared to .Net ThreadPool
by vivainio on Fri 11th Sep 2009 14:47 UTC in reply to "Compared to .Net ThreadPool"
vivainio Member since:
2008-12-26

After reading this, it sounds like it is .Net's (and I'm sure other languages) ThreadPool with a lot of Apple marketing.


dotnet certainly didn't invent threadpool ;-).

Threadpool as such is trivial. GCD manages the threadpool by looking at how busy the CPUs are, and is per-OS instead of per-application.

Reply Score: 5

RE: Compared to .Net ThreadPool
by galvanash on Fri 11th Sep 2009 15:20 UTC in reply to "Compared to .Net ThreadPool"
galvanash Member since:
2006-01-25

Its more than a threadpool, but I don't if its as big of a deal as it has been made out to be... It needs a few months as least so that the tires can get kicked so to speak.

Anyway, how it is different from a .net threadpool:

1. It isn't a class instantiated in your code. It is an always running background daemon.
2. It knows the topology of the machine it is running on, i.e. how many sockets and cores are available. The primary benefit of this is that you do not have to configure the size of the pool - you just let it handle that itself.
3. In the traditional threadpool approach each application manages its own (probably differing) implementation of a threadpool, and since they do not know of each others existence they often don't have enough information to make the best decisions for the system as a whole. GC is a global threadpool, and as such it has a much broader view of the system and can therefore make better decisions.
4. It is designed to execute closures (Apple calls them blocks, same thing). While this is a restriction, some people consider it a good one because it eliminates a lot of potential blocking issues.
5. Its completely dynamic and requires no pre-planning. By that I mean your code does not (and by definition should not) be concerned with how many threads will be needed or even if your code will actually end up running in parallel or synchronous.
5. It supports FIFO queuing (not unique, but not a common attribute of most threadpool implementations)
6. As I understand it (I may be wrong) it appears to be a M:N threading model, i.e. userspace threads. As such, the GC "threads" themselves are very lightweight.

The bad (or at least potentionally bad):
1. For other platforms, if it gets adopted, it is unclear at this point how things would work, since it requires support for closures, and there are no existing C compilers (besides Apples) that support closures.
2. I may or may not be possible to use it from VM based langauges (i.e. .net, java), but frankly for those types of environments it doesn't make all that much sense either. You would probably want something like it implemented within the VM, it makes more sense there.

In my opinion, it seems like a pretty solid system (at least on paper). I think the main attraction is going to be in using it for hiding blocking behavior in UI applications, i.e. pushing code that might block the main UI thread off to the background. I don't see it as a replacement for a threadpool exactly, there are probably times you want to have application control of thread count and such... But I think its simplicity will attract a lot of users, at least on OSX. Whether it will catch on for other platforms is hard to say.

Reply Score: 22

RE[2]: Compared to .Net ThreadPool
by ringham on Fri 11th Sep 2009 15:24 UTC in reply to "RE: Compared to .Net ThreadPool"
ringham Member since:
2006-03-23

This is an excellent summary of the differences between GCD and .NET's thread pool. I'm a .NET dev and I love the thread pool but I fully realize it's limitations when it comes to balancing tasks I submit to it with OS-wide task execution.

I do some OS X development for fun in my spare time and I'm looking forward to seeing what it's like. Just have to upgrade to Snow Leopard first.

Reply Score: 1

RE[2]: Compared to .Net ThreadPool
by tyrione on Fri 11th Sep 2009 17:53 UTC in reply to "RE: Compared to .Net ThreadPool"
tyrione Member since:
2005-11-21

Its more than a threadpool, but I don't if its as big of a deal as it has been made out to be... It needs a few months as least so that the tires can get kicked so to speak.

Anyway, how it is different from a .net threadpool:

1. It isn't a class instantiated in your code. It is an always running background daemon.
2. It knows the topology of the machine it is running on, i.e. how many sockets and cores are available. The primary benefit of this is that you do not have to configure the size of the pool - you just let it handle that itself.
3. In the traditional threadpool approach each application manages its own (probably differing) implementation of a threadpool, and since they do not know of each others existence they often don't have enough information to make the best decisions for the system as a whole. GC is a global threadpool, and as such it has a much broader view of the system and can therefore make better decisions.
4. It is designed to execute closures (Apple calls them blocks, same thing). While this is a restriction, some people consider it a good one because it eliminates a lot of potential blocking issues.
5. Its completely dynamic and requires no pre-planning. By that I mean your code does not (and by definition should not) be concerned with how many threads will be needed or even if your code will actually end up running in parallel or synchronous.
5. It supports FIFO queuing (not unique, but not a common attribute of most threadpool implementations)
6. As I understand it (I may be wrong) it appears to be a M:N threading model, i.e. userspace threads. As such, the GC "threads" themselves are very lightweight.

The bad (or at least potentionally bad):
1. For other platforms, if it gets adopted, it is unclear at this point how things would work, since it requires support for closures, and there are no existing C compilers (besides Apples) that support closures.
2. I may or may not be possible to use it from VM based langauges (i.e. .net, java), but frankly for those types of environments it doesn't make all that much sense either. You would probably want something like it implemented within the VM, it makes more sense there.

In my opinion, it seems like a pretty solid system (at least on paper). I think the main attraction is going to be in using it for hiding blocking behavior in UI applications, i.e. pushing code that might block the main UI thread off to the background. I don't see it as a replacement for a threadpool exactly, there are probably times you want to have application control of thread count and such... But I think its simplicity will attract a lot of users, at least on OSX. Whether it will catch on for other platforms is hard to say.


LLVM. End of story.

Reply Score: 2

Stratoukos Member since:
2009-02-11

1. For other platforms, if it gets adopted, it is unclear at this point how things would work, since it requires support for closures, and there are no existing C compilers (besides Apples) that support closures.

I don't think that compiler support is the a problem for GCD adoption outside OS X. In UNIX, gcc is king and I expect Apple to push the blocks/closures extension support into gcc soon. Also, any compiler vendor can just copy the implementation of LLVM since it's licensed under the BSD license.

As for Windows, Microsoft is going to do whatever it wants no matter what Apple does.

Reply Score: 2

vivainio Member since:
2008-12-26

Also, any compiler vendor can just copy the implementation of LLVM since it's licensed under the BSD license.


I don't think license is an issue - and IIUC apple already had patches that implement blocks for gcc. It's a different thing whether the C community really *wants* this feature, as it doesn't fit very well in C (C is still not intended to be a high level language).

C++ lambdas are probably a better closure alternative:

http://groups.google.com/group/comp.lang.c++.moderated/browse_threa...

Reply Score: 3

galvanash Member since:
2006-01-25

I don't think it will end up being a problem in the long run. But although the user space code is available, right now you'll have to use llvm to hack on it. I don't think you will see widespread use until GCC proper implements Apple's changes, and afaik that has not happened yet. It might happen quickly, but I doubt it. Ive already seen some bickering on the mailing lists about the ugly syntax...

Some developers are a bit upset because the existing inner function support (-fnested-functions) is already very close to blocks in functionality, and rather than implementing the rather strange syntax Apple uses for blocks they would prefer to expand inner functions to support full closures. I guess we'll see...

Reply Score: 2

RE[3]: Compared to .Net ThreadPool
by Kroc on Fri 11th Sep 2009 18:35 UTC in reply to "RE[2]: Compared to .Net ThreadPool"
Kroc Member since:
2005-11-10

Apple to push the blocks/closures extension support into gcc soon


Not unless it's over Richard Stallman's dead body. The whole reason for the existence of LLVM/Clang is that GCC was going nowhere Apple wanted and RMS was unwilling to add even trivial features any other modern-day compiler has.

As cynical as it is, I think it'll take some real convincing for GCC to accept Blocks. Maybe if it gets ratified into the standard, then there should be no reasonable reason for it to not be included.

Reply Score: 3

RE[4]: Compared to .Net ThreadPool
by flynn on Fri 11th Sep 2009 23:09 UTC in reply to "RE[3]: Compared to .Net ThreadPool"
flynn Member since:
2009-03-19

Not unless it's over Richard Stallman's dead body. The whole reason for the existence of LLVM/Clang is that GCC was going nowhere Apple wanted and RMS was unwilling to add even trivial features any other modern-day compiler has.


While RMS (co)wrote a lot of GNU software, how much control does he actually have these days? I doubt he still actively develops any of them. Any decisions about what features are and are not added to GCC are probably up to the GCC maintainers, not RMS.

Reply Score: 1

RE[4]: Compared to .Net ThreadPool
by CrLf on Sat 12th Sep 2009 13:49 UTC in reply to "RE[3]: Compared to .Net ThreadPool"
CrLf Member since:
2006-01-03

Not unless it's over Richard Stallman's dead body. The whole reason for the existence of LLVM/Clang is that GCC was going nowhere Apple wanted and RMS was unwilling to add even trivial features any other modern-day compiler has.


This reminds me of EGCS*, which came to be exactly because the GCC project was stale and unwilling to evolve in any meaningful way. This is not the case with LLVM and Clang.

GCC and LLVM/Clang have fundamentally different objectives.

GCC is supposed to be the "all architectures/all-languages" compiler (which is why it changed from "GNU C Compiler" to "GNU Compiler Collection"), which places and extra burden in anyone who wishes to make significant changes in the way it operates and the features it has: making significant changes to GCC implies that those changes have to apply to all architectures (from multi-core multi-GHz CPUs to tiny microcontrollers) or, at least, not cause them any harm.

This is fantastically hard. Even so, GCC seems to be making steady progress over the last few years, but within its limitations and without a (probably disastrous for the project's goals) major rewrite.

LLVM/Clang is the way to work beyond those limitations without caring if it only works for x86. There is no war between LLVM and GCC (and if there is any significant rivalry between the two projects, there shouln't be).

The two projects actually help each other.

The knowledge within the GCC project about the different architectures it supports can help LLVM from falling into the x86-only pit, and the more production-oriented rate of change can help LLVM integrate the more relevant features sooner and more consistently into its design.

On the other hand, LLVM can eventually reach the level of quality and architecture/language support that GCC currently has, allowing it to adopt GCC's goals and effectively become a replacement for GCC.

I don't think RMS has anything to do with this (and with today's GCC, actually), nor I think he would come forward criticizing LLVM if this should happen.


As cynical as it is, I think it'll take some real convincing for GCC to accept Blocks. Maybe if it gets ratified into the standard, then there should be no reasonable reason for it to not be included.


I think you are wrong in both counts.

On one hand you imply that GCC is slow in adopting changes when, in fact, GCC is often criticized for implementing too many extensions to standards. Many changes to language standards have actually been implemented first in GCC, which is the complete opposite of what you're stating.

On the other hand, GCC doesn't have to implement blocks just because Apple provides them with the necessary code to do it and they are required by something that currently has no expression whatsoever outside of OS X.

The real problem, like others have mentioned, is that blocks/closures may not be a good fit for C, and C developers may not like them.

This may be confusing to the people that use the term "C/C++", but C is still a separate, and living, language from C++ because of its simplicity (a real understatement when comparing to a language so feature-bloated and baroque as C++). C programmers may not look favorably to C becoming a little more like C++.



* http://en.wikipedia.org/wiki/GNU_Compiler_Collection#EGCS_fork

Reply Score: 3

puenktchen Member since:
2007-07-27

The knowledge within the GCC project about the different architectures it supports can help LLVM from falling into the x86-only pit,


llvm isn't x86-only:

An easily retargettable code generator, which currently supports X86, X86-64, PowerPC, PowerPC-64, ARM, Thumb, SPARC, Alpha, CellSPU, PIC16 MIPS, MSP430, SystemZ, and XCore.

http://llvm.org/Features.html

Reply Score: 2

RE[6]: Compared to .Net ThreadPool
by CrLf on Sun 13th Sep 2009 13:46 UTC in reply to "RE[5]: Compared to .Net ThreadPool"
CrLf Member since:
2006-01-03

I didn't say that it was. But supporting multiple architectures isn't their primary objective, unlike GCC, and on the Clang site there is mention of it being production quality for x86 and x86_64, although it supports other architectures.

Reply Score: 2

puenktchen Member since:
2007-07-27

clang isn't llvm. clang is the frontend, llvm the backend. clang doesn't support architectures, it supports languages. llvm does support the processor architectures. you can use gcc as frontend for llvm, but that doesn't change the supported architectures but the supported languages. and you can't use clang with gcc as backend. so i don't get what you're trying to say with your post.

Reply Score: 2

FellowConspirator Member since:
2007-12-13

FWIW - Apple's compiler isn't Apple's at all. It's GCC, and the blocks implementation is open-source; whether the GCC maintainers pick it up is a different story.

Reply Score: 1

galvanash Member since:
2006-01-25

FWIW - Apple's compiler isn't Apple's at all. It's GCC, and the blocks implementation is open-source; whether the GCC maintainers pick it up is a different story.


Apple uses a custom branch of GCC for OSX, and that branch supports blocks, but GCC mainline isn't necessarily going to accept the changes required to support it. Which is more or less what you just said ;) , just pointing out my use of the term "Apple's compiler" was meant to be read as "Apple's GCC branch".

Reply Score: 2

google_ninja Member since:
2006-02-05

re: C closures, I thought apple used gcc 4? do they have to make any changes available?

Reply Score: 2

RE: Compared to .Net ThreadPool
by adricnet on Fri 11th Sep 2009 16:56 UTC in reply to "Compared to .Net ThreadPool"
adricnet Member since:
2005-07-01

Blocks and closures in C, C++, ObjC if I had to guess.

Reply Score: 1

RE: Compared to .Net ThreadPool
by waid0004 on Fri 11th Sep 2009 17:33 UTC in reply to "Compared to .Net ThreadPool"
waid0004 Member since:
2009-06-19

Java has thread pools as well (in java.util.concurrent), since 1.5.

Reply Score: 1

RE: Compared to .Net ThreadPool
by google_ninja on Sat 12th Sep 2009 01:44 UTC in reply to "Compared to .Net ThreadPool"
google_ninja Member since:
2006-02-05

This isn't a thread pool, the .net equivilent would be this the TPL (and PLINQ) that is coming with .NET 4 http://msdn.microsoft.com/en-us/magazine/cc163340.aspx

I haven't looked into GCD or TPL enough to comment, but they both (supposedly) address the same issue. IMO this isn't something you can just shoehorn in, you need it from the ground up (like scala)

Reply Score: 2

galvanash Member since:
2006-01-25

I had not seen this before. You are right though, it looks to be very much the same basic concept as GCD. Thanks for pointing this out.

One thing that I notice immediately: I like the TPL syntax a lot better. For example, Parallel.For is very similar to dispatch_apply, but it doesn't require you to declare a queue. It looks much less alien...

Parallel.For(0, count, delegate(int i) {
result[ i ] = doSomething(something, i);
});


just reads a whole lot nicer than

dispatch_apply(count, dispatch_get_global_queue(0, 0), ^(size_t i){
result[ i ] = doSomething(something, i);
});


of course, that is just syntactic sugar - they both do the same thing...

Reply Score: 2

In Linux/BSD ...
by dindin on Fri 11th Sep 2009 15:27 UTC
dindin
Member since:
2006-03-29

Anything like this in linux or BSD systems? I guess now that they have released it as Open Source and under the Apache License, they will find a way into the other systems. Any problems adopting them into the kernel for GPL or BSD licensed systems?

Reply Score: 2

RE: In Linux/BSD ...
by vermaden on Fri 11th Sep 2009 16:12 UTC in reply to "In Linux/BSD ..."
vermaden Member since:
2006-11-18

Any problems adopting them into the kernel for GPL or BSD licensed systems?


No problem for BSD systems, a BIG problem for Linux since it uses GPL which is compatible only with itself.

Reply Score: 4

RE[2]: In Linux/BSD ...
by raboof on Fri 11th Sep 2009 16:52 UTC in reply to "RE: In Linux/BSD ..."
raboof Member since:
2005-07-24

No problem for BSD systems, a BIG problem for Linux since it uses GPL which is compatible only with itself.


Not entirely: the Apache 2.0 license is generally considered to be unidirectionally compatible with GPLv3. This means you may take Apache2.0-licensed code, combine it with GPLv3-licensed code, and publish the result under the GPLv3. You may not publish the result under the Apache2 license, though.

http://www.apache.org/licenses/GPL-compatibility.html
http://www.fsf.org/licensing/licenses/#apache2

However, only the libdispatch part is Apache2-licensed: the kernel part (xnu) is under 'APPLE PUBLIC SOURCE LICENSE', and I haven't checked whether that's GPL-compatible. Moreover, the Linux kernel is GPLv2, not GPLv3, and whether GPLv2 and Apache2 licenses are compatible afaik is under dispute.

So libdispatch can be included in GPLv3 userland applications, and xnu can be included in the kernel if the 'APPLE PUBLIC SOURCE LICENSE' is GPLv2-compatible, which is might be.

Reply Score: 3

RE[3]: In Linux/BSD ...
by ba1l on Fri 11th Sep 2009 17:04 UTC in reply to "RE[2]: In Linux/BSD ..."
ba1l Member since:
2007-09-08

The kernel component would be pretty specific to OS X anyway - it would have to be completely reimplemented anyway to run on Linux.

The compiler parts would be under the same license as the compilers - GPL3 for GCC, and the LLVM license for clang (looks like a standard BSD-style license). No problems here.

The rest of it is userspace. There would be no licensing problems at all with the daemon, since it's independent anyway. As for any library components that need to be linked to applications, the Apache license is compatible with GPL / LGPL (in one direction), so there wouldn't be any licensing problems with integrating it with glibc, or wherever it needs to go.

I think Apple pretty much did the right thing here. It just remains to be seen how useful it is, and whether anyone puts it to good use.

Reply Score: 3

RE[3]: In Linux/BSD ...
by kaiwai on Sat 12th Sep 2009 02:09 UTC in reply to "RE[2]: In Linux/BSD ..."
kaiwai Member since:
2005-07-06

Not entirely: the Apache 2.0 license is generally considered to be unidirectionally compatible with GPLv3. This means you may take Apache2.0-licensed code, combine it with GPLv3-licensed code, and publish the result under the GPLv3. You may not publish the result under the Apache2 license, though.


That is the point I think he/she was trying to get at; when he/she talks about compatibility, as you've explained, it is a one way transaction. Quite honestly it makes me laugh when GPL advocates talk about the evils of proprietary software and yet their own licence has the overtones of being proprietary given that the transaction between BSD/Apache/X11/etc is one way.

Reply Score: 3

RE[4]: In Linux/BSD ...
by sorpigal on Mon 14th Sep 2009 14:28 UTC in reply to "RE[3]: In Linux/BSD ..."
sorpigal Member since:
2005-11-02

You have a strange definition of proprietary.

The GPL has this one major goal: To make sure the user of the software can always view the source. It accomplishes this very well. Its goal is not to be the friendliest license for you to copy and paste from--that would be the BSD license. If all you want is to steal someone else's code, BSD is for you! If you want to be sure no one can steal your code, GPL is a much better choice.

For all its faults I don't know that I've heard many people talk about the evils of the BSD license.

Can we dispense with the anti-GPL hostility, please? I can be just as hostile toward the BSD license, but I generally don't turn my flamethrower on for that. I can respect the position of someone choosing to give away his code in that fashion. Can't you respect the position of someone who wishes to use the GPL?

Reply Score: 1

RE[5]: In Linux/BSD ...
by puenktchen on Tue 15th Sep 2009 14:06 UTC in reply to "RE[4]: In Linux/BSD ..."
puenktchen Member since:
2007-07-27

If all you want is to steal someone else's code, BSD is for you!


you have a strange definition of stealing.

Reply Score: 2

RE: In Linux/BSD ...
by vivainio on Fri 11th Sep 2009 16:30 UTC in reply to "In Linux/BSD ..."
vivainio Member since:
2008-12-26

Anything like this in linux or BSD systems? I guess now that they have released it as Open Source and under the Apache License, they will find a way into the other systems. Any problems adopting them into the kernel for GPL or BSD licensed systems?


IIUC they didn't release the kernel bits, just the userspace lib. In any case, GCD isn't really that special and it should be a piece of cake to implement from scratch, even.

Reply Score: 1

RE[2]: In Linux/BSD ...
by puenktchen on Fri 11th Sep 2009 16:59 UTC in reply to "RE: In Linux/BSD ..."
puenktchen Member since:
2007-07-27

IIUC they didn't release the kernel bits, just the userspace lib.


rtfa:

Mac OS X also has kernel support for Grand Central Dispatch, which is also released as open source via the XNU project.


@ raboof: apples public source licence isn't compatible with he gpl. but the xnu-code won't be of any use in kernels which aren't reated to xnu or at least mach anyway. the concepts used will probably be more interesting than the code.

Edited 2009-09-11 17:06 UTC

Reply Score: 5

RE[2]: In Linux/BSD ...
by PlatformAgnostic on Fri 11th Sep 2009 17:05 UTC in reply to "RE: In Linux/BSD ..."
PlatformAgnostic Member since:
2006-01-02

It looks like the language/compiler features are the hardest part of the project.

Reply Score: 2

RE[2]: In Linux/BSD ...
by testman on Sat 12th Sep 2009 00:47 UTC in reply to "RE: In Linux/BSD ..."
testman Member since:
2007-10-15

In any case, GCD isn't really that special and it should be a piece of cake to implement from scratch, even.

Get cracking then! :-)

Reply Score: 2

RE: In Linux/BSD ...
by clei on Sat 12th Sep 2009 09:44 UTC in reply to "In Linux/BSD ..."
clei Member since:
2008-10-04

Anything like this in linux or BSD systems? I guess now that they have released it as Open Source and under the Apache License, they will find a way into the other systems. Any problems adopting them into the kernel for GPL or BSD licensed systems?


Ummm...Why would they want to?

Reply Score: 1

Kudos to Apple
by Tuishimi on Fri 11th Sep 2009 16:06 UTC
Tuishimi
Member since:
2005-07-06

For releasing it so quickly.

Reply Score: 8

Double Edge Sword
by milles21 on Fri 11th Sep 2009 16:14 UTC
milles21
Member since:
2006-11-08

It seems that apple cannot win with some people, if the release the GCD technology open source it's no big deal. If the don't then all apple supposedly does is piggybacking off opensource.

I would think that any contributions to the community that could possibly yield some benefit would be a welcomed thing.

I mean it is not like, .net is opensource and Mono is filled with criticism, what is the happy medium, I just wish that sometimes people can appreciate it the contributions no matter how small they all form a larger opensource eco-system.

Reply Score: 9

RE: Double Edge Sword
by AaronD on Fri 11th Sep 2009 17:27 UTC in reply to "Double Edge Sword"
AaronD Member since:
2009-08-19

I agree. This is the internet though and opinions are cheap. It is next to impossible to even get agreement that the sky is blue.

Reply Score: 1

RE[2]: Double Edge Sword
by nickelbackro on Fri 11th Sep 2009 22:05 UTC in reply to "RE: Double Edge Sword"
nickelbackro Member since:
2009-04-12

Where I am it's grey ;)

The truth is though that as long as a company makes money off open source they will be derided by the community (not the whole community just the nutty fringe). Just ask Red Hat. They get it despite them contributing tons of code.

Reply Score: 1

Hrm...
by looncraz on Fri 11th Sep 2009 16:34 UTC
looncraz
Member since:
2005-07-24

Not a terribly bad idea, but I wonder exactly how multiple 'threads' of execution will be handled ( read: dependency branches ). If it is done at compile-time, then all is well -- so long as you don't plan on writing a portable piece of software.

I guess it would not be too hard to tell a decent compiler to ignore all of the ^{} blocks...

--The loon

Reply Score: 2

RE: Hrm...
by JonathanBThompson on Fri 11th Sep 2009 17:04 UTC in reply to "Hrm..."
JonathanBThompson Member since:
2006-05-26

IIRC you can attach dependency information between the various GC threads and the system figures out all that itself, making it a very nice little thing, but I've not read all the documentation myself yet.

Reply Score: 2

The fun of threading...
by JonathanBThompson on Fri 11th Sep 2009 17:14 UTC
JonathanBThompson
Member since:
2006-05-26

[quote]Parallel programming is extremely difficult, and even the most talented of programmers will run into race conditions, deadlocks, and other difficulties, because different threads require access to the same data and memory space.[/quote]

Er, the most talented of programmers run into those in other people's code and fixes those problems, if it is tractable, and reformulate the code when not, with only a few exceptions being stuck in the case where deadlocks are allowed to occur: the one that comes to mind most readily are databases as a very common piece of software that's prone to deadlocks, due to the nature of them, and then the database is (hopefully, correctly) coded in a way as to detect and break the deadlocks.

There are actually rather simple rules as to how you must design and code to avoid race conditions/deadlocks: maintain a consistent locking/unlocking order, and protect with critical sections all resources that are shared between threads. If you do a bit of proper diagramming ahead of time, and stop and think about what's required, you'll either fairly readily realize what needs to be done to do that, or that it simply can't be done reliably. Timeout periods (not available in all threading primitives) are an admission to 2 things:
1. People will eventually make mistakes in their locking order, so this provides a slightly more graceful exit out of their errors than remaining locked, only if they handle such error conditions properly: not doing so can introduce race conditions into the system.

2. Often times the timeouts are for access to resources that can take an indeterminate period of time, sometimes never completing its intended task, so not having a timeout value on that would be a nightmare that'd require killing threads and cleaning up the bodies.

#2 can't be worked around nicely, at least not without asynchronous I/O for I/O-related items, but #1 can almost always be prevented.

Reply Score: 2

RE: The fun of threading...
by vivainio on Fri 11th Sep 2009 17:28 UTC in reply to "The fun of threading..."
vivainio Member since:
2008-12-26

There are actually rather simple rules as to how you must design and code to avoid race conditions/deadlocks: maintain a consistent locking/unlocking order, and protect with critical sections all resources that are shared between threads.


Threading looks extremely simple on paper/design phase and when you are writing the code. The simplicity is over when you are trying to debug the code. Things like Python's Queue.Queue and GCD help, but once you start needing mutexes you are screwed (or, well, will be eventually).

It is somewhat (but not much) harder to write the same code in asynchronous fashion, but it pays off in the end as you'll get more deterministic code.

To quote Alan Cox, "A Computer is a state machine. Threads are for people who can't program state machines.".

Reply Score: 2

JonathanBThompson Member since:
2006-05-26

I do speak from firsthand experience: it's true that debugging threaded code can be a nightmare, especially in the sorts of contexts I have experience with, industrial machinery that cannot tolerate holding a thread at some point of execution while the hardware itself is running. For that, you need built-in facilities in the source code in the form of telemetry to help debug what the sequence of things are. Designing and implementing it isn't that overly complicated, unless you go bonkers: debugging it (and whatever mistakes you've made in general logic otherwise: you can have correct threading logic but other things will cause debugging to be required) is where the real fun comes in, as those conditions arise, if you can't lock down a system indefinitely while debugging lest vital things timeout.

State machines are definitely an easier way at times to do things, no question there, and that's why industrial machinery tends to use programmable ladder logic, a formalized and well-understood state machine that works the same everywhere. If more software was written in the same way, it'd be easier to debug.

Reply Score: 2

RE[2]: The fun of threading...
by Stratoukos on Fri 11th Sep 2009 17:50 UTC in reply to "RE: The fun of threading..."
Stratoukos Member since:
2009-02-11

To quote Alan Cox, "A Computer is a state machine. Threads are for people who can't program state machines.".

Well I couldn't agree more, but only if we are talking about one core. If you want to access more cores you need threads.

Things like Python's Queue.Queue and GCD help, but once you start needing mutexes you are screwed (or, well, will be eventually).

GCD is more than just a queue. It exists so you don't have to use mutexes. You just decide what parts of your code can be executed parallel to your main thread and GCD deals with race conditions itself

Reply Score: 1

RE[3]: The fun of threading...
by dotnick on Fri 11th Sep 2009 18:15 UTC in reply to "RE[2]: The fun of threading..."
dotnick Member since:
2009-07-28

It's too bad there isn't an "LVM" for CPU cores though, as there is for hard drives. Just pool the CPU's together and let the daemon do it's thing.

Reply Score: 1

RE[4]: The fun of threading...
by Stratoukos on Fri 11th Sep 2009 18:22 UTC in reply to "RE[3]: The fun of threading..."
Stratoukos Member since:
2009-02-11

Well, I don't know what exactly LVM does, but from what i've read about it from Wikipedia this is exactly what GCD is trying to do. You just say "I want these stuff to run separately" and the daemon does its thing.

And I guess there is/will be some way to use GCD with openCL, allowing the daemon to pool not only the CPUs but the GPUs too

Edited 2009-09-11 18:36 UTC

Reply Score: 1

RE[3]: The fun of threading...
by cb_osn on Sat 12th Sep 2009 02:45 UTC in reply to "RE[2]: The fun of threading..."
cb_osn Member since:
2006-02-26

GCD is more than just a queue. It exists so you don't have to use mutexes. You just decide what parts of your code can be executed parallel to your main thread and GCD deals with race conditions itself

No. GCD is nothing more than a thread pool that is managed by the OS rather than the language runtime. It's interesting and should lead to better allocation of CPU cores for multithreaded programs, but it does not solve any of the hard problems which revolve around shared resources, and it certainly doesn't remove the need for synchronization primitives like mutexes.

Reply Score: 3

RE[4]: The fun of threading...
by Stratoukos on Sat 12th Sep 2009 10:23 UTC in reply to "RE[3]: The fun of threading..."
Stratoukos Member since:
2009-02-11

Avoid using locks. The support provided by dispatch queues and operation queues makes locks unnecessary in most situations. Instead of using locks to protect some shared resource, designate a serial queue (or use operation object dependencies) to execute tasks in the correct order.


That is from Apple's Concurrency Programming Guide. I am sure that GCD is using some locking mechanisms inside, but it abstracts them from the developer.

Reply Score: 1

tyrione
Member since:
2005-11-21

Now you can all shove it about what does Apple contribute back. Between GCD and OpenCL the world of computing is far better off.

Reply Score: 4

BallmerKnowsBest Member since:
2008-06-02

So I can do whatever I want with applications developed by others and I'm automatically absolved of all wrongdoing, just as long as I release the code to completely unrelated applications that I've developed?

Awesome!

Reply Score: 2

concerns about blocks
by Anon9 on Fri 11th Sep 2009 22:20 UTC
Anon9
Member since:
2008-06-30

I think that blocks are a bad idea as it overlaps too much with C++ lambdas, which are much farther along in standardization. Also the syntax for a pointer to a block uses the ^ symbol. In C++/CLI, which I think is already standardized, ^ is used as a managed pointer. So there are two conflicts with existing practice here. Another concern is that block is already a defined term in C.

I'm not opposed to closures in general, I just think they should have gone more along the route of C++ lambdas.

Reply Score: 1

RE: concerns about blocks
by testman on Sat 12th Sep 2009 00:50 UTC in reply to "concerns about blocks"
testman Member since:
2007-10-15

I'm not opposed to closures in general, I just think they should have gone more along the route of C++ lambdas.

You're a C++ developer I take it?

Reply Score: 2

RE: concerns about blocks
by CrLf on Sat 12th Sep 2009 13:57 UTC in reply to "concerns about blocks"
CrLf Member since:
2006-01-03

I think that blocks are a bad idea as it overlaps too much with C++ lambdas, which are much farther along in standardization. Also the syntax for a pointer to a block uses the ^ symbol. In C++/CLI, which I think is already standardized, ^ is used as a managed pointer. So there are two conflicts with existing practice here.


Well, C and C++ have been separate languages for a while. Since C99, C++ is no longer a superset of C, although people still think it is.

Reply Score: 2

Kudos
by skingers6894 on Sat 12th Sep 2009 00:50 UTC
skingers6894
Member since:
2005-08-10

This is one of the headline features of their new OS just released weeks ago. To Open Source this so quickly is commendable. Well done to Apple on this.

Reply Score: 2

Am I getting this straight?
by 3rdalbum on Sat 12th Sep 2009 10:27 UTC
3rdalbum
Member since:
2008-05-26

I'm an extreme beginner in multithreaded programming. In Python, no less.

So this GCD thing means I can just say "Run this in a different thread" and that's it? I don't have to create locks to control access to resources that are shared between threads? Can my separate thread work safely with any data that just a normal function or method would be able to use? What would I have to do to tell the parent thread that the worker thread has finished?

It's confusing to follow whether GCD takes care of all the details for you, or if it's just a daemon that keeps track of what cores are available and says "no more, please" when they're all working to maximum capacity. If it's the latter, then so freaking what, even I could write something like that.

Reply Score: 2

RE: Am I getting this straight?
by vivainio on Sat 12th Sep 2009 11:18 UTC in reply to "Am I getting this straight?"
vivainio Member since:
2008-12-26

I'm an extreme beginner in multithreaded programming. In Python, no less.

So this GCD thing means I can just say "Run this in a different thread" and that's it?

Pretty much, with the twist that the OS optimizes the worker thread operation according to amount of cores and their workload.

What would I have to do to tell the parent thread that the worker thread has finished?


Use callbacks (where the blocks step in).

If it's the latter, then so freaking what, even I could write something like that.


This is trivial in Python (and other langs that support closures). Apple provided closures (blocks) for C and ObjC, and implemented OS level support for that.

Yes, it's not rocket science, but that doesn't mean providing such standard support is a bad idea ;-)

Reply Score: 2

RE: Am I getting this straight?
by galvanash on Sat 12th Sep 2009 19:32 UTC in reply to "Am I getting this straight?"
galvanash Member since:
2006-01-25

So this GCD thing means I can just say "Run this in a different thread" and that's it? I don't have to create locks to control access to resources that are shared between threads?


No, it isn't quite that simple. If you actually have shared state, you have to deal with it. The difference is that the methodology for dealing with shared state is always going to be uniform - you create a queue for it to gate access. Queues in essence replace locking constructs.

Can my separate thread work safely with any data that just a normal function or method would be able to use?


No. I would read this if you want to understand the implications of blocks as Apple has implemented them:

http://thirdcog.eu/pwcblocks/

What would I have to do to tell the parent thread that the worker thread has finished?


You would use dispatch_async and give it a callback to call on completion of the queue.

It's confusing to follow whether GCD takes care of all the details for you, or if it's just a daemon that keeps track of what cores are available and says "no more, please" when they're all working to maximum capacity. If it's the latter, then so freaking what, even I could write something like that.


Its definitely not just for tracking cores. The concept of queues is fundamental to it.

Reply Score: 2

moondevil
Member since:
2005-07-08

The C++ programmers have Intel Thread Building Blocks and Microsoft Parallel Pattern Library, with no need for language extensions.

Java programmers have java.util.concurrent.

.Net programmer have ThreadPools, Task Parallel Library and Parallel LINQ.

Grand Central Dispatch is only Apple catching up, while providing proprietary extensions to the C language. Which might take several years before any other vendor cares to implement them, even with the source available.

So for me, this is a good thing for Mac only developers, for all the others a nice curiosity, nothing more.

Reply Score: 1

There's only one thing that bugs me
by FealDorf on Sat 12th Sep 2009 11:45 UTC
FealDorf
Member since:
2008-01-07

I like how it was released open-source -- this shows that Apple can give as much as(?) it takes. I've done little multithreading, but I too can't see what makes this better than other threadpools. Nevertheless, The GCD relies on blocks. This is the distinguishing difference from most implementations, but it's also what bugs me.

Blocks would've been fine if there was no __block qualifier. What I understood is that it relies on reference counting, which would mean an invisible implementation in a language of much lower level.. I don't want C standard to accept this upon Apple's insistence. This is different from '&' in C++ because those are "sugary pointers" at a lower level. It is for that reason that the blocks in C++0x have such an ugly (albeit expressive) syntax for lambdas. If stdC includes blocks, then these would not only come in clash with upcoming C++ lambdas but also bring in more complexity.

Reply Score: 1