OSNews: http://www.osnews.com/story/20635/Inside_Tsubame_Japan_s_NVIDIA_GPU_Supercomputer Exploring the Future of Computing en-us Copyright 2001-2010, David Adams adam+nospam@osnews.com Tue, 16 Mar 2010 22:06:47 GMT http://www.osnews.com/images/osnews.gif OSNews.com http://www.osnews.com The heat dissipation must be a joy on the Power bill http://osnews.com/thread?339856 http://osnews.com/thread?339856 The cooling of that setup must be a delight. Thu, 11 Dec 2008 00:21:00 GMT donotreply@osnews.com (tyrione) Comments RE: The heat dissipation must be a joy on the Power bill http://osnews.com/thread?339877 http://osnews.com/thread?339877 Well, the GPUs only take like 4 racks. I assume they are using another 4 racks for the host computers. 8/10 racks is not that bad for a system that puts out 77 tflops. Thu, 11 Dec 2008 02:50:00 GMT donotreply@osnews.com (javiercero1) Comments RE[2]: The heat dissipation must be a joy on the Power bill http://osnews.com/thread?339880 http://osnews.com/thread?339880 4 racks? Are you kidding?

"Tsubame itself - once you move past the air-conditioners - is split across several rooms in two floors of the building and is largely made up of rack-mounted Sun x4600 systems. There are 655 of these in all, each of which has 16 AMD Opteron CPU cores inside it, and Clearspeed CSX600 accelerator boards.

The graphics chips are contained in 170 Nvidia Tesla S1070 rack-mount units that have been slotted in between the Sun systems. Each of the 1U Nvidia systems has four GPUs inside, each of which has 240 processing cores for a total of 960 cores per system."

More info on http://www.pcworld.com/article/155242/.html?tk=rss_news Thu, 11 Dec 2008 03:18:00 GMT donotreply@osnews.com (GatoLoko) Comments Not the future today http://osnews.com/thread?339893 http://osnews.com/thread?339893 NVidia's gpus really fall flat on double precision. The nvidia tesla system advertises 1TFlop SP and 80GFlop DP. That means 64bit doubles run at only 8% the speed of 32bit floats. And 32bit floats are just about totally worthless for general scientific processing. An 8 core powermac 2 years ago could churn ~40GFlop DP. That was 2 intel generations ago....

The new cell successor from IBM looks far more interesting but it seems that IBM isn't interested in selling it (same mistake DEC made). Thu, 11 Dec 2008 04:50:00 GMT donotreply@osnews.com (bnolsen) Comments RE[3]: The heat dissipation must be a joy on the Power bill http://osnews.com/thread?339914 http://osnews.com/thread?339914 They have 170 tesla cabinets, each is 1U rack unit. Each standard rack tends to be 42 units.

Simple math dictates that 4 racks could hold those 170Us.

I am not sure they packed them that much. But I was referring to the GPUs alone. Which is indeed a remarkable density of computing. Thu, 11 Dec 2008 07:40:00 GMT donotreply@osnews.com (javiercero1) Comments RE: Not the future today http://osnews.com/thread?339915 http://osnews.com/thread?339915 Please, refrain from making blanket statements like "simple precision FP is worthless" it turns out that for some applications, it is enough. Which I am sure that is what these machines are being targeted at. Thu, 11 Dec 2008 07:42:00 GMT donotreply@osnews.com (javiercero1) Comments RE[2]: Not the future today http://osnews.com/thread?339936 http://osnews.com/thread?339936 It was my understanding that nvidia was bringing native doubles with their latest hardware. The boards used were announced in 2007, while the vendor ClearSpeed is saying that they support single and double, this is with their api. Thu, 11 Dec 2008 10:31:00 GMT donotreply@osnews.com (nsrbrake) Comments RE: Not the future today http://osnews.com/thread?339992 http://osnews.com/thread?339992 Not all scientific applications require 64-bit floating point support. Some algorithms are resilient enough for the 32-bit FP support to be sufficient, even with the restricted rounding modes. When you absolutely need 64-bit support, you can mix it with 32-bit floats where 64-bit is not necessary. And when you absolutely need 64-bits, remember that all that memory bandwidth (141.7 GB/sec) is available. Your 8% figure is only about raw execution resources but says nothing about actual application performance, which is of course totally dependent on the application. Considering super computers are not exactly cheap, they must have taken this into account somewhere.

The new cell successor from IBM looks far more interesting but it seems that IBM isn't interested in selling it (same mistake DEC made).


True. Cell is still very expensive. IBM cannot compete with COTS nVidia solutions, since the Cell in the PS3 does not support DP, so the volumes must be quite small of the DP enabled Cell. Thu, 11 Dec 2008 15:42:00 GMT donotreply@osnews.com (drahca) Comments RE[2]: Not the future today http://osnews.com/thread?340033 http://osnews.com/thread?340033
Not all scientific applications require 64-bit floating point support.


True, but the majority (if you ask me for a number I'd say >95%) of scientific applications that require a super computer will need 64 bit floating point support. Thu, 11 Dec 2008 19:35:00 GMT donotreply@osnews.com (evangs) Comments Suse Linux Enterprise Server 9 http://osnews.com/thread?340037 http://osnews.com/thread?340037 Just in case anyone was wondering:

http://tinyurl.com/6998vg Thu, 11 Dec 2008 19:56:00 GMT donotreply@osnews.com (sbergman27) Comments RE[3]: Not the future today http://osnews.com/thread?340043 http://osnews.com/thread?340043 Be careful, pulling numbers like that out of one's derriere is a bit counter productive :-) Thu, 11 Dec 2008 21:28:00 GMT donotreply@osnews.com (javiercero1) Comments RE[3]: Not the future today http://osnews.com/thread?340046 http://osnews.com/thread?340046
True, but the majority (if you ask me for a number I'd say >95%) of scientific applications that require a super computer will need 64 bit floating point support.

Remember that this is a hybrid system. It also has 10,480 Opterons. Parts of the job which require higher precision can use the Opterons, while parts which can get by with lower precision, can use the GPUs. Also, as I understand it, GPUs are really great for vector processing, but really suck at most everything else. So they can be viewed as one more resource that the supercomputer application programmer has at hand which can be applied at his or her discretion, just like any other resource. Thu, 11 Dec 2008 21:52:00 GMT donotreply@osnews.com (sbergman27) Comments RE[2]: Not the future today http://osnews.com/thread?340093 http://osnews.com/thread?340093 Actually cell SPEs can do DP, butt also with a big hit in performance. The newer ibm PowerXCell boosts the double precision. Apparently a single cell with 8 SPEs could only do 14GFlops DP. PowerXCell can do 102GFlops with 8 SPEs...

In fact a super computer made up of these PowerXCell was (maybe still is?) the top super computer in the world.

http://www.ppcnux.com/?q=node/7144

What sucks is that cell should absolutely not be so expensive. Proof: sony who's claiming no hardware cost losses ~$399 per ps3 sale. That's with a helluva lot more hardware than just cell.

IBM wants their 10,000% markup, that's all. Fri, 12 Dec 2008 04:58:00 GMT donotreply@osnews.com (bnolsen) Comments RE: Not the future today http://osnews.com/thread?340108 http://osnews.com/thread?340108 Interesting to see people agree with this statement.

OK, the thing is this: There are computations that require double precision. And there then there are some which don't. Now guess what this is made for. Also, the machine doesn't consist of GPUs only. Btw, if you see the performance of NV GPUs within the folding at home project you will find that they easily outdo even the Cell. Again, it depends what you wanna do with it. I hope you don't believe the issue escaped them before they had built the thing... ;) Fri, 12 Dec 2008 10:07:00 GMT donotreply@osnews.com (Googol) Comments