Linked by David Adams on Sun 5th Dec 2010 20:33 UTC, submitted by dylansmrjones
Oracle and SUN Oracle executives talked up on Thursday the planned Solaris 11 release due in 2011, with the Unix OS upgrade offering advancements in availability, security, and virtualization. The OS will feature next-generation networking capabilities for scalability and performance, said John Fowler, Oracle executive vice president of systems, at a company event in Santa Clara, Calif. "It's a complete reworking of [the] enterprise OS," he said. Oracle took over Solaris when the company acquired Sun Microsystems early this year.
Thread beginning with comment 452841
To view parent comment, click here.
To read all comments associated with this story, please click here.
RE[6]: Really...
by foobar on Wed 8th Dec 2010 22:26 UTC in reply to "RE[5]: Really..."
foobar
Member since:
2006-02-07

"It seems a bit premature to proclaim as fact that Solaris 11 scales up to "thousands of threads and CPUs" when there aren't yet any 1000+ core servers on which Solaris will run (unless someone's gotten Solaris x86 to boot on an Altix UV - that'd be a very cool thing to see benchmarked).

Further, at least according to Oracle's product page, the largest configuration for the T5440 has 256 threads. Do you have a link to any benchmarks that show scaling up to 256-way SMP on a single solaris instance? It'd be really interesting to see.

Solaris 10 scales up to hundreds of cpus. Each thread is treated as a cpu, by Solaris. So if you see 256 threads, it is the same thing as 256 cpus.

Earlier there was at least one large Solaris server weighing 1000kg with 144 cpus, sold by Sun. Maybe it was the Enterprise 10000 model? Can not remember. But that was decades ago and driven by an old Solaris version which back then scaled to hundreds of cpus.

In 2015, Solaris 11 will drive this new server with 16.384 threads. That is one of the reasons S11 kernel was reworked, to scale further up from several hundreds of cpus, to many thousands of cpus (threads).
"

Nice. We have to wait until 2015 to verify your claims. Vapor ware!

The more realistic explanation is that some data structures were reworked to accommodate a system with 16k threads. Then marketing ran with the big number.

This statement from the article is so nebulous that the great improvement might be as dull as support for RoCEE. WooHoo another driver that is meaningless without buying a ridiculously expensive pcie adapter.

The OS will feature next-generation networking capabilities for scalability and performance, said John Fowler, Oracle executive vice president of systems, at a company event in Santa Clara, Calif. "It's a complete reworking of [the] enterprise OS," he said. Oracle took over Solaris when the company acquired Sun Microsystems early this year.


"While MOST Linux supercomputers listed on the Top500 are indeed clusters, the Altix 3000 & 4000-series from SGI are not clusters but large NUMA machines (same "breed" of system as the big boxes from Oracle, IBM,etc) and scaled beyond a 1000 CPUs years ago. Setups with of up to 4096 CPUs in a single instance were sold by SGI as far back as 2006.

When I talked about large Linux servers just being a cluster, I was actually thinking about the SGI's ALTIX servers, and they are the large clusters I was referring to.

If you look at the impressive Altix benchmarks, you will see that it is the same kind of workload run on a cluster.
http://www.c0t0d0s0.org/archives/6751-Thank-you,-SGI.html

"Perhaps those benchmarks published by SGI finally deliver a few nails for the coffin of the reasoning of some fanboys that Linux scales better than Solaris, because there are systems with thousands of cores out there. Linux scales on this system exactly like a cluster. And that means for non clusterable tasks in one word: Not."

Linux scaling _used to be_ rather sad compared to the high end UNIXes and compared to Solaris and IRIX in particular. That rapidly started changing once SGI got into the game and decided to replace IRIX with Linux. Today, both Solaris and Linux scale very well. I believe Solaris still has the edge in scaling linearly up to the largest systems a single instance will run on, but Linux is supported on considerably larger setups, thanks to SGI.


Regarding Linux scaling. Here we have a bunch of Linux scaling experts that "dispells the FUD from Unix vendors that Linux does not scale well" in an article:
http://searchenterpriselinux.techtarget.com/news/929755/Experts-For...
"

Your arguments would be more convincing if people could read your references. Registering is a nonstarter.

"Linux has not lagged behind in scalability, [but] some vendors do not want the world to think about Linux as scalable. The fact that Google runs 10,000 Intel processors as a single image is a testament to [Linux's] horizontal scaling.

Today, Linux kernel 2.4 scales to about four CPUs

"-With the 2.6 kernel, the vertical scaling will improve to 16-way. However, the true Linux value is horizontal scaling [that is: clusters].
Q: Two years from now, where will Linux be, scalability-wise, in comparison to Windows and Unix?
A: It will be at least comparable in most areas"

According to the Linux experts, Linux scales to 10.000 cpus in one single image in the current v2.4, and in Linux 2.6 the kernel will improve to 16-way? Isn't that a bit strange?


You're mixing things up again. 10,000 cpus is for a cluster of machines. 16-way is for a single machine.

The ALTIX machine sold in year 2006 with 4096 cores, was using Linux v2.6 (which had only been released 2 years earlier). I find it extremely hard to believe that in v2.4 Linux scaled bad (2-4 CPUs) and two years later it suddenly scales to 4096 cores in the ALTIX machine? It takes decades to scale well. The only conclusion is that ALTIX machine is a cluster, otherwise Linux would have not chance to scale to 4096 cores in two years.


2.4 and 2.6 were developed in parallel for many years. Wikipedia has a nice timeline:

http://en.wikipedia.org/wiki/Linux_kernel

2.4 continues to be developed today. 2.5 development started in 2001. 2.6 development started in December 2003. So your 2 years number is pretty meaningless.

Also, one of the biggest improvements in 2.6 was to remove the 'big kernel lock', which helped scalability immensely.

Linux scales well on large clusters, yes. But that is not Big Iron. When people says Linux scales well (which it does) then they talk about clusters - that is scaling Horizontally.

In other words; Linux scales well HORIZONTALLY, but still not good at VERTICAL scaling (which Solaris excels at on the large Solaris servers).

The Linux kernel developers only have access to their desktop computers, with 1-2 cpus. They have no access to big servers weighing 1000kg and 100s of cpus and TB of RAM (who would pay that many million USD?), so it is difficult for them to test Linux on large configurations. Everything in Linux is tailored to 1-2 cpus and a few GB of RAM - it is a desktop OS.

This is taken from the ext4 main developer:
http://thunk.org/tytso/blog/2010/11/01/i-have-the-money-shot-for-my...

"Ext4 was always designed for the “common case Linux workloads/hardware”, and for a long time, 48 cores/CPU’s and large RAID arrays were in the category of “exotic, expensive hardware”, and indeed, for much of the ext2/3 development time, most of the ext2/3 developers didn’t even have access to such hardware. One of the main reasons why I am working on scalability to 32-64 nodes is because such 32 cores/socket will become available Real Soon Now, and high throughput devices such as PCIe attached flash devices will start becoming available at reasonable prices soon as well."

He says that soon 32-64 cores will become available (in a couple of years) so he started to work on trying to scale as high as 32-64 cores.


Ext2 was developed in 1993. Of course linux developers didn't have access to such hardware. It was only a few hobbyists then. Ext3 was developed in 2001 which is about the time that the corporate world started embracing linux. So Ted is right, ext2/3 didn't have access to such hardware.

Today things are different. There are many paid engineers. The story was here on osnews:

http://www.osnews.com/story/23636/Who_Really_Contributes_the_Most_t...

Weren't you paying attention? There is a nice table here:

http://news.cnet.com/8301-13505_3-10288910-16.html

Even your beloved Sun, Oracle are on the list. Obviously they have access to such hardware.

[q]From 32 cores, it is a far stretch to 100s of cpus or cores. Let us not even start to talk about Altix 4096 cores.

The only big servers I know of, are old and mature Unix, Solaris with 144 cpus, and IBM big POWER servers with 64 cpus. None of the big Unix vendors has ever

Reply Parent Score: 1

RE[7]: Really...
by Kebabbert on Thu 9th Dec 2010 09:11 in reply to "RE[6]: Really..."
Kebabbert Member since:
2007-07-27

The more realistic explanation is that some data structures were reworked to accommodate a system with 16k threads. Then marketing ran with the big number.

You know, on the public Oracle roadmaps there are servers with 16.384 threads and 64TB RAM. I dont think such servers would have been on the roadmap, without a reason. Solaris has been doing scalability for decades. 100s of cpus and threads for decades. Now it is time to move up to thousands of cpus and threads. No other cpu vendor has that many threads, except Oracle with Niagara cpus. Therefore Oracle sees the need for their next gen servers to rework Solaris now. You never know what Oracle have in their labs today.

Nice. We have to wait until 2015 to verify your claims. Vapor ware!

Oh, and how about BTRFS? The ZFS killer? Have you read the mail lists about all bugs and data corruptions? Not many, if any, of the ZFS features have been implemented in BTRFS yet. We will have to wait until 2020 to verify the BTRFS developers claims. Vapor ware!


Your arguments would be more convincing if people could read your references. Registering is a nonstarter.

Anyway, I did some quotes where the Linux experts claim that v2.6 is going to scale to 16 cores, from v2.4 with 2-4 cores. I dont see how Linux will scale up to thousands of cores in two years - especially as Linux developers dont have access to such big servers.


You're mixing things up again. 10,000 cpus is for a cluster of machines. 16-way is for a single machine.

I am not mixing anything. I am quoting the Linux experts. I also explained that Linux scales well in a cluster (horizontal scaling) but bad vertically (single big servers). I wrote that. I dont mix up anything, but Linux fans do.

They say that Linux scales to 10.000 cores (for instance Google) and concludes that Linux scales excellent. That is wrong conclusion, I say, because Linux scales well on a cluster. But there is no single server with 10.000 cores. Again, Linux scales well on a cluster. But on a single server, Linux scales good to 16-32 cores. Today. As Tso confirms below.

2.4 and 2.6 were developed in parallel for many years. Wikipedia has a nice timeline:
http://en.wikipedia.org/wiki/Linux_kernel
2.4 continues to be developed today. 2.5 development started in 2001. 2.6 development started in December 2003. So your 2 years number is pretty meaningless.

How is my "two years number meaningless"? 2.6 development started in early 2004 according to a graphical timeline I saw. Two years later ALTIX servers were sold. It is impossible to scale well from 16 cores up to 4096 cores in two years.


Also, one of the biggest improvements in 2.6 was to remove the 'big kernel lock', which helped scalability immensely.

Sure. And when did they remove the BKL? Wasnt it two months ago? Year 2010? So maybe Linux scales better today. How can the Linux experts claim Linux scaled excellent back in 2003 when the interview was made?

"Yes, Linux scales excellent! It scales to 10.000 cores on Google. It also scales to 2-4 cores in v2.4 and to 16 cores in v2.6. Yes, Linux scales perfect! 16 cores is remarkably good! Linux scaling FTW!!"

Of course, if the Linux experts thinks that 16 cores is excellent scaling, then I understand they think that 32 cores today is more than excellent.

Linux fans think "Linux have never crashed on my desktop - it must be perfectly stable. It also scales well on my dual cpu PC - it scales perfect" and then go out and rant about how good it scales.


Ext2 was developed in 1993. Of course linux developers didn't have access to such hardware. It was only a few hobbyists then. Ext3 was developed in 2001 which is about the time that the corporate world started embracing linux. So Ted is right, ext2/3 didn't have access to such hardware.

So how can Linux scale well if they dont have access to big servers?


Today things are different. There are many paid engineers. The story was here on osnews:
Even your beloved Sun, Oracle are on the list. Obviously they have access to such hardware.

I dont see how you make the jump:
"There are paid developers => they have access to such hardware"

At my company, not many have access to big servers, even though we have such servers.

Many Linux devs are concerned with user stuff, lag, etc.

Reply Parent Score: 2