Linked by David Adams on Sun 5th Dec 2010 20:33 UTC, submitted by dylansmrjones
Oracle and SUN Oracle executives talked up on Thursday the planned Solaris 11 release due in 2011, with the Unix OS upgrade offering advancements in availability, security, and virtualization. The OS will feature next-generation networking capabilities for scalability and performance, said John Fowler, Oracle executive vice president of systems, at a company event in Santa Clara, Calif. "It's a complete reworking of [the] enterprise OS," he said. Oracle took over Solaris when the company acquired Sun Microsystems early this year.
Thread beginning with comment 452349
To view parent comment, click here.
To read all comments associated with this story, please click here.
RE[3]: Really...
by Kebabbert on Mon 6th Dec 2010 18:04 UTC in reply to "RE[2]: Really..."
Kebabbert
Member since:
2007-07-27

I think he is refering to Solaris 10. Solaris 11 is a complete rework, compared to S10.

For instance, S11 scales much better now, up to thousands of threads and cpus. S10 scaled very good earlier, the best of all OS - for instance, it could handle 512 cpus (each thread is treated as an individual cpu) in Sun T5440 without problems.

But Oracle will release an Solaris server in 2015 that has 16.384 threads, which need a rework of scaling, it will be driven by S11. No OS can scale to that many threads/cpus today.

For instance, recently IBM released their POWER7 cpu. Their biggest POWER7 server released a couple of months ago, has something like 256 cores or so. The mature IBM Enterprise Unix AIX, could not handle that many cores. AIX was rewritten (according to several articles on IT-web sites, such as theregister.co.uk) to handle the biggest POWER7 server with as little as 256 cores. The step from 256 cores, up to 16.384 cores is huge and needs a complete rework.

Linux supercomputers are basically a bunch of PCs on a fast network. That is a different kind of scaling which is easy to do. But to scale to thousands of threads in one single server, is difficult. No one can scale to that many cpus today.

Reply Parent Score: 4

RE[4]: Really...
by wigry on Tue 7th Dec 2010 12:02 in reply to "RE[3]: Really..."
wigry Member since:
2008-10-09

IMHO IBM does not use AIX on its heavy boxes but really big iron is controlled b specifically written z/OS

The big System Z is basically an array of smaller computers and then there is one z/OS on top of that which commands those smaller computers. Each small box however may contain its separate OS (like AIX or Linux if you will).

Now Solaris is targeted to the same level as z/OS is so it can handle lots of resources.

And then there is HPUX and thats pretty much all about mainframe business.

Reply Parent Score: 1

RE[5]: Really...
by Kebabbert on Wed 8th Dec 2010 09:29 in reply to "RE[4]: Really..."
Kebabbert Member since:
2007-07-27

IBM uses z/OS on their Mainframes, yes. But the Mainframe cpus are dog slow, much much much slower than x86 cpus.

In IBM's fastest boxes, they use POWER7 (which happens to be a really fast cpu in some workloads)

Reply Parent Score: 2

RE[4]: Really...
by TheOrqwithVagrant on Tue 7th Dec 2010 19:05 in reply to "RE[3]: Really..."
TheOrqwithVagrant Member since:
2007-08-16


For instance, S11 scales much better now, up to thousands of threads and cpus. S10 scaled very good earlier, the best of all OS - for instance, it could handle 512 cpus (each thread is treated as an individual cpu) in Sun T5440 without problems.


It seems a bit premature to proclaim as fact that Solaris 11 scales up to "thousands of threads and CPUs" when there aren't yet any 1000+ core servers on which Solaris will run (unless someone's gotten Solaris x86 to boot on an Altix UV - that'd be a very cool thing to see benchmarked).

Further, at least according to Oracle's product page, the largest configuration for the T5440 has 256 threads. Do you have a link to any benchmarks that show scaling up to 256-way SMP on a single solaris instance? It'd be really interesting to see.


Linux supercomputers are basically a bunch of PCs on a fast network. That is a different kind of scaling which is easy to do. But to scale to thousands of threads in one single server, is difficult. No one can scale to that many cpus today.


While MOST Linux supercomputers listed on the Top500 are indeed clusters, the Altix 3000 & 4000-series from SGI are not clusters but large NUMA machines (same "breed" of system as the big boxes from Oracle, IBM,etc) and scaled beyond a 1000 CPUs years ago. Setups with of up to 4096 CPUs in a single instance were sold by SGI as far back as 2006.

Linux scaling _used to be_ rather sad compared to the high end UNIXes and compared to Solaris and IRIX in particular. That rapidly started changing once SGI got into the game and decided to replace IRIX with Linux. Today, both Solaris and Linux scale very well. I believe Solaris still has the edge in scaling linearly up to the largest systems a single instance will run on, but Linux is supported on considerably larger setups, thanks to SGI.

Reply Parent Score: 1

RE[5]: Really...
by Kebabbert on Wed 8th Dec 2010 10:05 in reply to "RE[4]: Really..."
Kebabbert Member since:
2007-07-27

It seems a bit premature to proclaim as fact that Solaris 11 scales up to "thousands of threads and CPUs" when there aren't yet any 1000+ core servers on which Solaris will run (unless someone's gotten Solaris x86 to boot on an Altix UV - that'd be a very cool thing to see benchmarked).

Further, at least according to Oracle's product page, the largest configuration for the T5440 has 256 threads. Do you have a link to any benchmarks that show scaling up to 256-way SMP on a single solaris instance? It'd be really interesting to see.

Solaris 10 scales up to hundreds of cpus. Each thread is treated as a cpu, by Solaris. So if you see 256 threads, it is the same thing as 256 cpus.

Earlier there was at least one large Solaris server weighing 1000kg with 144 cpus, sold by Sun. Maybe it was the Enterprise 10000 model? Can not remember. But that was decades ago and driven by an old Solaris version which back then scaled to hundreds of cpus.

In 2015, Solaris 11 will drive this new server with 16.384 threads. That is one of the reasons S11 kernel was reworked, to scale further up from several hundreds of cpus, to many thousands of cpus (threads).


While MOST Linux supercomputers listed on the Top500 are indeed clusters, the Altix 3000 & 4000-series from SGI are not clusters but large NUMA machines (same "breed" of system as the big boxes from Oracle, IBM,etc) and scaled beyond a 1000 CPUs years ago. Setups with of up to 4096 CPUs in a single instance were sold by SGI as far back as 2006.

When I talked about large Linux servers just being a cluster, I was actually thinking about the SGI's ALTIX servers, and they are the large clusters I was referring to.

If you look at the impressive Altix benchmarks, you will see that it is the same kind of workload run on a cluster.
http://www.c0t0d0s0.org/archives/6751-Thank-you,-SGI.html

"Perhaps those benchmarks published by SGI finally deliver a few nails for the coffin of the reasoning of some fanboys that Linux scales better than Solaris, because there are systems with thousands of cores out there. Linux scales on this system exactly like a cluster. And that means for non clusterable tasks in one word: Not."


Linux scaling _used to be_ rather sad compared to the high end UNIXes and compared to Solaris and IRIX in particular. That rapidly started changing once SGI got into the game and decided to replace IRIX with Linux. Today, both Solaris and Linux scale very well. I believe Solaris still has the edge in scaling linearly up to the largest systems a single instance will run on, but Linux is supported on considerably larger setups, thanks to SGI.

Regarding Linux scaling. Here we have a bunch of Linux scaling experts that "dispells the FUD from Unix vendors that Linux does not scale well" in an article:
http://searchenterpriselinux.techtarget.com/news/929755/Experts-For...

"Linux has not lagged behind in scalability, [but] some vendors do not want the world to think about Linux as scalable. The fact that Google runs 10,000 Intel processors as a single image is a testament to [Linux's] horizontal scaling.

Today, Linux kernel 2.4 scales to about four CPUs

"-With the 2.6 kernel, the vertical scaling will improve to 16-way. However, the true Linux value is horizontal scaling [that is: clusters].
Q: Two years from now, where will Linux be, scalability-wise, in comparison to Windows and Unix?
A: It will be at least comparable in most areas"


According to the Linux experts, Linux scales to 10.000 cpus in one single image in the current v2.4, and in Linux 2.6 the kernel will improve to 16-way? Isn't that a bit strange?

The ALTIX machine sold in year 2006 with 4096 cores, was using Linux v2.6 (which had only been released 2 years earlier). I find it extremely hard to believe that in v2.4 Linux scaled bad (2-4 CPUs) and two years later it suddenly scales to 4096 cores in the ALTIX machine? It takes decades to scale well. The only conclusion is that ALTIX machine is a cluster, otherwise Linux would have not chance to scale to 4096 cores in two years.

Linux scales well on large clusters, yes. But that is not Big Iron. When people says Linux scales well (which it does) then they talk about clusters - that is scaling Horizontally.

In other words; Linux scales well HORIZONTALLY, but still not good at VERTICAL scaling (which Solaris excels at on the large Solaris servers).





The Linux kernel developers only have access to their desktop computers, with 1-2 cpus. They have no access to big servers weighing 1000kg and 100s of cpus and TB of RAM (who would pay that many million USD?), so it is difficult for them to test Linux on large configurations. Everything in Linux is tailored to 1-2 cpus and a few GB of RAM - it is a desktop OS.

This is taken from the ext4 main developer:
http://thunk.org/tytso/blog/2010/11/01/i-have-the-money-shot-for-my...

"Ext4 was always designed for the “common case Linux workloads/hardware”, and for a long time, 48 cores/CPU’s and large RAID arrays were in the category of “exotic, expensive hardware”, and indeed, for much of the ext2/3 development time, most of the ext2/3 developers didn’t even have access to such hardware. One of the main reasons why I am working on scalability to 32-64 nodes is because such 32 cores/socket will become available Real Soon Now, and high throughput devices such as PCIe attached flash devices will start becoming available at reasonable prices soon as well."

He says that soon 32-64 cores will become available (in a couple of years) so he started to work on trying to scale as high as 32-64 cores.

From 32 cores, it is a far stretch to 100s of cpus or cores. Let us not even start to talk about Altix 4096 cores.

The only big servers I know of, are old and mature Unix, Solaris with 144 cpus, and IBM big POWER servers with 64 cpus. None of the big Unix vendors has ever offered servers bigger than those. It sounds weird that SGI offers Linux servers in two years time, from 2-4 cores up to 4096 cores. Something is strange. Or, it is just a cluster.



Another reason Linux has problem, is that Linux cuts corners and cheats, just to win benchmarks. Linux does not obey correct Unix standards, which mean your data can be corrupt. This is a bad thing:
http://milek.blogspot.com/2010/12/linux-osync-and-write-barriers.ht...

"This is really scary. I wonder how many developers knew about it especially when coding for Linux when data safety was paramount. Sometimes it feels that some Linux developers are coding to win benchmarks and do not necessarily care about data safety, correctness and standards like POSIX. What is even worse is that some of them don't even bother to tell you about it in official documentation "

Reply Parent Score: 1