OSNews

I was wondering the same thing, what's the software?!

What idiots write these articles anyway. It seems to me that the OSes running on the worlds fastest computers would be mentionable in an article about one of them!

Oh but Hawking will be using it! Woopedy-doo. I bet he could get a login to 90% of the high performance machines around the world with little more than a phone call (not that he'd need to, since Cambridge lets him use theirs ).

HPC computers always amaze me though because in one sense they're such a huge step forward: A bunch of processors running tons of data in parallel. At the same time, the really huge ones (like this one) sound like they're administered like a batch system: You get a time. I suppose they give out a lot fewer time slots on these and you get months instead of hours.
And they probably assign multiple people at the same time, most of the time. But still, it's reticent of a batch system to me.

If you find an article mentioning software (which would make this osnews worthy) please do post it!

yes, it runs linux.

the thing has more than 27 TeraFLOPs (27 x 1012 calculations per second) of raw computing power.

576 Dell PowerEdge 1950 compute servers in the cluster have two Dual-Core Intel Xeon 5160 processors at 3GHz and 8GB of memory.

all compute servers are interconnected by an InfiniBand network from QLogic InfiniPath host adapters and switches from SilverStorm Technologies. 60 TeraBytes of storage is provided by 18 Dell PowerVault MD1000 disk arrays connected to the cluster network over 10 Gigabit Ethernet links. the storage pool is managed by the TerraGrid parallel filesystem.

the cluster runs the ClusterVisionOS, ClusterVision's own Linux-based cluster software distribution which provides an integrated software environment for the effective use and management of the system.

the cluster comprises nine computational units (CUs), each with its own ClusterVisionOS Sub-Master. Each CU has 65 nodes for computation. the CUs are managed by two failover Sub-Masters. the CUs can run independently - if one CU fails the other CUs will keep on functioning. also, if one of the ClusterVisionOS Main Masters fails, the cluster will keep on running. user jobs, however, are not limited to a single CU and can span multiple CUs. The construction using independent CUs allows for scaling up to far larger clusters with tens of thousands servers. users log into one of the four DNS balanced ClusterVisionOS Login Nodes.

hope it helps.

Edited 2006-11-23 20:54

The original article made it sound like standard PCs were used with normal networking equipment, in which case it wouldn't really be a supercomputer in my eyes.
Thankfully with the details you provided here, it's clear that it really is a supercomputer.

Would be cool to have access to computing power like that, but I have no idea what I would use it for :p

Would be cool to have access to computing power like that, but I have no idea what I would use it for :p

well, the usual stuff, but really quickTM

;-D

Edited 2006-11-23 21:39

> Would be cool to have access to computing power like that, but I have no idea what I would use it for

The usual cool stuff: humiliate it by making it loose vs. a skilled Go player, big laugh plus great celebration of human mind superiority

>Would be cool to have access to computing power like >that, but I have no idea what I would use it for :p

seti@home

Edited 2006-11-23 22:28

I do. I'd switch back to Gentoo ;-)

The original article made it sound like standard PCs were used with normal networking equipment, in which case it wouldn't really be a supercomputer in my eyes.

So what you are saying, is that not all computers on top500 are supercomputers?

What matters is the types of applications a supercomputer needs to be super at. I guess a lot of money have been wasted on fancy networks
just to be on top500, while the real applications are embarrassingly parallel.

Would be cool to have access to computing power like that, but I have no idea what I would use it for

Duke Nukem Forever

out of curiousity are the silverstorm infiniband switches the new ones that are capabable of 20gbit/s per hca? and does the terragrid parallel filesystem run at a full 10gbit/s on the backend (over 10gige with tcp offload or infiniband?). also is each CU connected to a smallish switch as opposed to have a few 288port ib switches?

Edited 2006-11-23 22:25

"A bunch of processors running tons of data in parallel."

Yes, and "PC experts" think their poor quad core x86 stuff is really fast... :-)

It is really fast. That's why when they become more available, then we'll see clusters with them "A bunch of processors running tons and tons of data in parallel."

"It is really fast. That's why when they become more available, then we'll see clusters with them "A bunch of processors running tons and tons of data in parallel.""

We'll see if they're as reliable as "usual" multiprocessor systems that are built upon non-x86 hardware. BTW: I'd like to see some statistics that compare actual Intel and AMC CPUs to the ones built by Sun, DEC, HP, IBM or MIPS (if they still build CPUs). Usually, multiprocessor systems offer a better "throughput" than multicore (but singleprocessor) systems (e. g. dyadic processor vs. double processor). At the times the Intel Pentium 4 / 1700 MHz processor was "modern", the MIPS R14k / 500 MHz was more than doublefaster...

Yes, it runs Linux. The new machine isn't generally accessible yet (so I can't just log in to check the setup) but assuming the queuing system is much the same as on the previous hardware it'll be running a batch processing setup. The machine is partitioned into multiple queues, each with a number of processors allocated to it. Users select a number of processors and then submit their job to the queue. Priorities are assigned based on how long a job has been in the queue, how many jobs have been submitted by that user recently and whether the user has some degree of priority access. Once the job reaches the top of the queue, it's executed until it's either complete or a timeout (usually 24 hours) has passed. Once the completed job has been tidied up, another job is selected and run.

Working within these sorts of constrains makes programming slightly harder. A cluster isn't like an SMP machine - even if you're using something like infiniband to link the different nodes, the latency and bandwidth between the processors and memory on different nodes is much worse than between a CPU and local memory. The two general ways around that are to either split your dataset up into multiple smaller chunks and then have each processor run a discrete process, or to use something like MPI to explicitly pass information between the code running on separate CPUs.

The new Dell replaces two older machines - a 128-way Opteron cluster running Suse, and a ~900 way Sun running Solaris. Assuming it performs according to the specifications, it should be a useful upgrade.

They go to all the trouble of linking thousands of the latest dual core Intel chips and it doesn't even make the top 10? Sun's box has less systems connected yet is faster and has more storage.

Dude, they got a Dell.

Exactly. That's why I got modded down, and likewise, why he got modded to three for stating the wrong thing. That is how Osnews always worked, lol.

.. and a sad comment to go with it, hm. If you watch your linked video, you will figure that the system has 10480 CPUs, so if you fancy calling any of the systems "sad", it got to be the one you linked to. The "Dell" doesn't even have half as many. Gosh, they are both impressive for what they are - time flies in computing-development terms.

They go to all the trouble of linking thousands of the latest dual core Intel chips and it doesn't even make the top 10? Sun's box has less systems connected yet is faster and has more storage.

The current Top 500 has the Tsubame with 11088 processors at #9 with the Cambridge 2340 CPU system at #20.

Every other system "faster" than the Cambridge system has 4096 processors or more.

I think Cambridge is getting pretty good bang for the buck.

I think cost per GFLOPS would be a very interesting stat.

I notice that 341 of the Top 500 are x86 or x64 CPU's.

Edited 2006-11-25 02:09

Correction, Tsubame is #9 on the top500. And if you're going to call a 2 million pound 2340 processor cluster 'sad' compared to a 11088 processor cluster, then your 'faster' Sun is pretty sad compared to #6 which is a 9024 processor Dell cluster at NNSA. Sun used 4U 8-way boxes. How many of those can you put in a 42U rack? 60 2-way poweredge 1955 blades fit into the same space. Seems like Dell can scale out and perform a lot better at much lower cost than Sun, which makes is very attractive for HPC scenarios.