A startup on Tuesday unveiled plans at the Apple Expo here to build a distributed supercomputer cluster that could use some thousands of Apple’s Xserve servers. The high-performance network will be installed along Europe’s internet backbone, startup Omneta said.
I’ll say two things:
1. Should have waited for the intel versions, they’ll be kicking themselves when the wattage and heat drops a ton on the intel servers
2. Install linux on them all to get the extra performance (they might do this anyway)
I’ll say two things;
1. They picked the hardware based on what they perceived their needs to be. The Intel switch is no secret. So, they probably won’t be kicking themselves.
2. Maybe Linux does not do what they need. News Flash. Linux is not the solution to everything. Like everything else in a comparison with OSX and Linux, clustering is much easier with OSX than Linux, and several large clustering solutions are performing great with OSX.
He may have a point though. Remember when Virginia Tech built their huge supercomputer out of G5 PowerMacs?
It was up for just long enough to make the top500 linst before they tore it all out and put in XServe cluster nodes. http://www.tcf.vt.edu/systemX.html
The cluster nodes run 2 2.3GHz G5 procs in a 1U form factor though, so they are not too shabby.
You can see it in the pic from the link above. The currently system is much smaller than the last one.
I suggested point 2 because of the recent arstechnica article on the massive performance boost of installing linux on G5 hardware.
I myself am a windows and OSX user and think linux isn’t the solution to anything other than firewalls ATM.
I think a super computer is used for scientific calculating Simulations that kind of thing. Linux is not going to make a difference in that. also the g5 is better at floating point than a pentium so for scientific computing a g5 is probably faster than a pentium.
It’s actually pretty well known within the HPC community that Linux 10% to 20% faster than OSX for most scientific apps; on the same hardware. The fact is that OSX’s memory management sucks. Even Kazushige Goto gave up trying to make a decent libgoto for OSX.
My own tests show 20% difference on the HPL.
I am not part of the hpc community i was just guessing but it seems than os-x still sucks less for scientific apps then it sucks for database apps
Install Linux?
Don’t you remeber it took NASA Days to do a Linux Cluster and there is a one page document on how to make an OSX Cluster, haha
Don’t you remeber it took NASA Days to do a Linux Cluster and there is a one page document on how to make an OSX Cluster, haha
True, but this is a supercomputer, every single flop counts. If you spend a few days installing linux/unix so that you can make a complex methematical model take three less days to compute because of performance gains then you do so.
You only have to install the cluster once, delays on mega computations continue constantly
why would it linux perform better on their setup than macosx? do you have any numbers to back this up?
why would it linux perform better on their setup than macosx? do you have any numbers to back this up?
http://www.osnews.com/story.php?news_id=11751
And that’s on the same hardware. Anybody who wants to run a supercomputer, or something with any sort of performance, needs a cat scan if they choose OS X. The
why would it linux perform better on their setup than macosx? do you have any numbers to back this up?
http://www.osnews.com/story.php?news_id=11751
And that’s on the same hardware. Anybody who wants to run a supercomputer, or something with any sort of performance, needs a cat scan if they choose OS X.
and how does this report extropolate to a clustered super computer? running mysql is a little different than running a supercomputer. but no, your right. i am sure the EU guys did not do any research into which hardware/OS is best for there needs. what they should have done is posted a question on Slashdot/OSNews “what should i use to built a supercomputer?”. because that’s where the real research and intelligence is at.
Why would computations be any slower in OS X than in Linux? Oh, they wouldn’t. After all, the ars review is comparing MySQL performance, not computing performance. The problems they made out where with creating threads (not something you’d want your supercomputer do all the time, you’d have a worker thread per CPU and that’s it), and with MySQL insisting the data be written to disk synchronously (which Linux just ignores). Anyway, who knows what they are trying to accomplish…
Why would computations be any slower in OS X than in Linux? Oh, they wouldn’t. After all, the ars review is comparing MySQL performance, not computing performance.
Err, MySQL performance on OS X, but also looking at the inherent limitations of the OS X as a resultof that. If you don’t think that’s an issue you need to seek professional help for your Mac addiction.
Super computing is about more than just computation, otherwise you could use Windows as a supercomputer. Not suprisingly, few people do.
OS X memory management has never been tuned and it performs poorly.
There was a company that built a prototype VM subsystem for Apple that doubled TPC scores and showed a significant inprovements on SPEC and other tests, but Apple never integrated this technology into the OS X.
(I think that Apple execs are idiots.)
Overall OS X has higher system call overhead, inefficient memory memory management and poor MP scalability.
All applications on OS X will run slower compared to other OSes, but IO bound apps would suffer the most.
Even for CPU bound applications Linux (or FreeBSD once PPC version become MP capable) will be a much better fit.
The funny part is that Omneta would do something like this without hiring someone experienced in the matter.
“I can build a better supercomputer” said US PC geek Joe “AC” Smith. “I mean, they’re using Macs and everyone knows that they just suck. Linux is so much better that it makes me t4L|< |_i|<3 teh //4N|<3R. And everyone knows that there was some post on some website that means for a specific database OS X threading is just like, crap, you know? So if that EU place wants to use MySQL they’re going to be sooo pissed! And I heard on the Internet that some company made some software for Macs that made them faster, and Apple never bought them out! Fools!!!11!one! LOL” (Joe actually said “exclamation mark, one, one exclamation mark, one, lol”)
Joe showed reporters a home-built PC he cobbled together mainly from parts he found in garbage. “I built this for only $57, and extrapolating from here, it’s simple to see how only an x86 chip running Linux can be the basis for a supercomputer. How can they even think of using Apple? It’s trivial to extend my techniques for building one machine to building two thousand of them. you just, like, do it more!”
When asked to point to the EU on a map, or name a single EU nation, Joe was a bit stumped. “They’re over near Korea aren’t they? You know, in that axis thingy who bombed us and shit.” (Joe points to Hawaii) “Yeah! We should just bomb those bastards! They don’t look so big compared to the US of A!”
Joe’s parents refused to comment.
lol
I needed that, its been a long day 🙂
I hope at least you enjoy your own sense of humor!
“The demonstration will initially involve just two sites, communicating information back and forth across the Interoute link. The test will involve 10 Xserves and two Xserve RAIDs, as observers closely eye network latencies.”
Seems to be, someone is doing a proof of concept with 12, count them, 12 machines.
Should be interesting to hear how they get on. That’s an awful lot of machines….
Its hilarious to see how just because I mention the name linux (though I am not a user or advocate even in the slightest), for the purposes of performance enhancement on the hardware, I’m instantly branded as a deluded linux zealout.
I think that says a lot about how linux users do their OS a big diservice.
These are supercomputing geeks, if they think linux is the better option to install then they will, as for hardware – Apple servers and OSX are good choices, I wouldn’t refute that.
Also forgot to add to that “branded as an american, when I live in the UK!”
Just goes to show how far a stereotype can be wrong.
why chose MySQL for an EU wide application and not oracle?
Yes MySQL is free…but so what?
This makes no sense…
it is only free for non-commercial use.
It’s interesting that in these posts there are always readers who try to counter facts with humor.
Never the less facts remain: OS X is not good at memory management. MySQL or not, there are micro and macro-benchmark suites that show this very clearly.
The G5 CPUs are really nice, but one has to run an OS other than OS X to get the most out of them.
Just as it is a fact the OS X is more user friendly than Linux.
It puzzles me that Apple would not spend a couple of million to integrate a technology that would double performance of IO intensive application and make machines more responsive, without any negative impact on performance and/or compatibility.
If anyone has been in software business for a while, you may know that Apple has a reputation of a company that is not worth dealing with.
Is there something unique to you, that you know all this and assume the people building the supercomputer don’t?
That’s the underlying assumption in your post, after all.
It is far more reasonable to assume that they’ve done their due diligence and understand their requirements far better than you.
If you want facts, look at real supercomputer implementations. VTechs supercomputer is G5s + Mac OSX. Gets the most performance per processor on the whole TOP500 list (save the other G5 Systems). The Barcelona system G5 system uses linux, no performance gain compared to OS X.
Conclusion: G5s have the best FLOP performance in supercomputer app largely because of single cycle Mult+Add.
OS X sports no penalty in this type of cluster application. Clustered supercomputers != a poorly architected database application that issues too many new thread calls.
VTech computers are not OSX. You just don’t know what you are saying.
What use does the EU have for a super computer anyway? Perhaps to use it to forcast their double digit unemployment rates? or the negative economic growth percentages? Or run synthetic models to support the Mankind enduced Global warming myth?
Maybe its too work out the aveageintelligence of the American people, unfortunatly they have used the wrong equipment for the purpose, a Abacus would have been good enough for the that job
Actually it will be used for climate and weather prediction, so they can send disaster relief teams to the US before the next Hurricane hits.
Or to give George W. a timely call to pull his head out of wherever it may be stuck.
Yeah that’s quite possible the first thing their models will find(i am sure)is that humans are contributing to Global Warming by means of respiration!. Tons of carbon dioxide released into the atmosphere by air breathing humans. I am sure this finding will result in a treaty to ban breathing air by Western Nations! let call it the Tokyo protocol! I am sure all the EU countries will be the first to sign it.
Do you have any education in this field or do you take your ideas from your lokal right wing newspaper, uhm I mean TV show?
Okay, I actually have both an Xserve G5 cluster and several Linux/x86 clusters, and here’s what I’ll say:
They both suck! I’ve been teetering between switching our Xserve G5 cluster to Linux for quite awhile now (been experimenting with YDL and Gentoo/ppc64) but for the time being we’re still sticking with OS X.
Getting Netboot going with OS X is a monsterous pain. I finally have it rolled out so the nodes reimage themselves on boot, but it was quite the headache.
I found OS X to be quite buggy. For example, the Broadcom drivers that shipped with Panther Server would simply flake out and all network communication to a node would be lost. 30 calls to AppleCare later, they were giving me debug versions to try (as this was a fairly well known and widespread issue). After sending debug data to their developer probably a good 30 times, and over a period of a few months, they managed to get the driver fixed. But still, a long, drawn out process.
Switching to Linux means I’d lose OS X’s awesome management tools though. They’re really quite pimp, especially Server Monitor.
Just out of curiosity (I’m not trolling here) what computing environment would you recommend if linux/x86 and osx/ppc aren’t up to par? Let’s assume there are no real budget constraints, but you would like to get the most performance and manageability for your money.
“So says the anonymous geek, who suddenly rushed out of his mum and dads basement to ask, “mum, and I as ignorant about life, business, the world and basically in a nutshell, reality, as this person on the forum says I am?!”
“To which the father says, “get away from that gun rack son, I’ve got a hootenanay, I don’t want ma good shot gun getting all smudged” to which the bare foot and pregnany wife scream, “shut up Cletus, we’ve gotta go to church now to hear about the evils of homosexuality, abortions and demn foreigners from brother Farwell!”
(ps. I’m sure someone can finish this story)
I am getting following error when I try to open the article –>
“Active Server Pages error ‘ASP 0126’
Include file not found
/component/display_module_group_component/0,1388,s%3D25986%26t %3D1,00.asp, line 198
The include file ‘/component/display_section_highlights_component/0,1389,s%3D701~*~ @~26ss%3D701%26mrc%3D8%26ast%3DBREAKING+NEWS%2 6dt%3D1,00.asp’ was not found. ”
Microsoft/eweek.com sucks! Big time. I am using FF 1.06.
I’m not going to make a comment either for or against Linux or Mac.
Yet, it is indisputable that Virginia Tech has this seriously awesome cluster of 1U XServes running 10.3.7 and they’re not using it to play Doom [at least, not all of the time]. There must be something in the package that persuades them the Apple kit is worth the bother.
And they’re not the only ones to think so.
Now this Euro company is going to try and see if they can get from it what they need. I’m not really convinced Apple is that bad a company to work with. They have been doing a lot of hard work in recent years to build reliable systems. It’s hard to tell from only using one Mac of course, but my experience points to a very usable and stable platform. Of course not everything is perfect, is there a system out there that is? Apple knows it needs to convince companies that they can deliver systems and software that get the job done, cutesy UI be damned. If they can’t prove they’ve got a winner they’re not going to sell systems, it doesn’t get simpler than that.
Not too many years ago you couldn’t put a gun to an IT guy’s head and force him to buy a Mac, let alone use it in the company. Today they’re building clusters with them, that’s not something even I would have seriously contemplated, not when their idea of a ‘server’ was a souped up desktop. Maybe we hear a bit of envy from the Linux crowd that sees Apple going places that is not their traditional domain. I think it’s a sign of their growing maturity, it certainly took them long enough. As far as specific performance issues are concerned, smaller operations will benefit from big systems being built because when a small player asks for a change it’s never going to be as urgent as when the guys from Virg Tech call and ask poignant questions about not-so-well-performing-drivers.
“It’s actually pretty well known within the HPC community that Linux 10% to 20% faster than OSX for most scientific apps; on the same hardware. The fact is that OSX’s memory management sucks. Even Kazushige Goto gave up trying to make a decent libgoto for OSX.
My own tests show 20% difference on the HPL.”
You comments suck……, you dont know what you are talking about. We are using osx and Xservers for our ice-sheet/climate modeling,and it works great. And yes we tried many other solutions before chosing the Xserve/OSX platform, we made performance test with nay other vendors running Linux or Windows. The Xserve/OsX platform was the best in term of performance/price ratio. Everyone who has built big clusters based on Xserve/OSX recemtly made the same thing. They compared with others platforms, in term of performance amd in term of price. And you can be sure that Linux was tested too.
Just think a little bit, why do you think that Virginia Tech, COLSA ( http://www.apple.com/science/profiles/colsa/ ), Turing
Xserve Cluster (http://www.cse.uiuc.edu/turing/),the Omneta (//www.omnetaplc.com/), the Dauger Research (http://daugerresearch.com/pr/JPLXServeCluster.html), the Compagnie Générale de Géophysique (CGG) which use 672 Xserves, have chosen OS X for running their cluster.
Just use your brain, do you think that if OsX or its memory management would really suck as you said, then those important organization would choose to run osx on their cluster. I mean those guys tested everything before, they evaluated others solutions, and obviously OSX gives plenty of satisfaction.
Anyway all your argument about osx are not relevant at all. i deal with high performance computing every day, i deal with many platform, and i can see that OsX is not only very stable, but is is very performant. You statement about memory management are not true at all, i know very well the osx kernel, so which side of the memory management are you talking about exactly? You just dont decribe what you are talking about, your statement is so general that it is irrelevent. I guess you dont know really what you are talking about. We dont have any of those problems, our numerical code is optimized for osx and runs very fast, as fast, actually faster than any other os. You should also ask biotechnology scientists who i guess will show how you osx is strong in their field.
Look at a tool like Genentech BLAST, Xeerve/OSX offer the highest performance in the market.
Moreover look at the LINPACK benchmark for double precision calculation, Xserve dual 2.3 ghz/OSX give 10.3 gigaflops. a dual 3.4 ghz Xeon running Linux gives something like 10.5 gigaflops, an Opteron running linux gives you 8.3 gigaflops. So i guess the Xserver/OSX result are quiet remarquable. Look at the performance of Virginia Tech, COLSA clusters, i dont invent anything. All of them performs very good running OS X. If OS X would be so bad as you try to convince people, well it would be noticed.
So what your point, stop staying boolshit, you really dont now what you are talking about. Its funny to see those Linux fan boys trying to convince people that Linux is the only OS for HPC or clustering. Its particularly funny when a new big cluster project do not use Linux, then for them it should be a mistake as anything else than Linux sucks. I guess that they try to defend a position where Linux is (only?) good at, but really OS X became a big player in this market, sadly for Linux fan boys….
“It’s actually pretty well known within the HPC community that Linux 10% to 20% faster than OSX for most scientific apps; on the same hardware. The fact is that OSX’s memory management sucks. Even Kazushige Goto gave up trying to make a decent libgoto for OSX.
My own tests show 20% difference on the HPL.”
You comments suck……, you dont know what you are talking about. We are using osx and Xservers for our ice-sheet/climate modeling,and it works great. And yes we tried many other solutions before chosing the Xserve/OSX platform, we made performance test with nay other vendors running Linux or Windows. The Xserve/OsX platform was the best in term of performance/price ratio. Everyone who has built big clusters based on Xserve/OSX recemtly made the same thing. They compared with others platforms, in term of performance amd in term of price. And you can be sure that Linux was tested too.
Just think a little bit, why do you think that Virginia Tech, COLSA ( http://www.apple.com/science/profiles/colsa/ ), Turing
Xserve Cluster (http://www.cse.uiuc.edu/turing/),the Omneta (//www.omnetaplc.com/), the Dauger Research (http://daugerresearch.com/pr/JPLXServeCluster.html), the Compagnie Générale de Géophysique (CGG) which use 672 Xserves, have chosen OS X for running their cluster.
Just use your brain, do you think that if OsX or its memory management would really suck as you said, then those important organization would choose to run osx on their cluster. I mean those guys tested everything before, they evaluated others solutions, and obviously OSX gives plenty of satisfaction.
Anyway all your argument about osx are not relevant at all. i deal with high performance computing every day, i deal with many platform, and i can see that OsX is not only very stable, but is is very performant. You statement about memory management are not true at all, i know very well the osx kernel, so which side of the memory management are you talking about exactly? You just dont decribe what you are talking about, your statement is so general that it is irrelevent. I guess you dont know really what you are talking about. We dont have any of those problems, our numerical code is optimized for osx and runs very fast, as fast, actually faster than any other os. You should also ask biotechnology scientists who i guess will show how you osx is strong in their field.
Look at a tool like Genentech BLAST, Xeerve/OSX offer the highest performance in the market.
Moreover look at the LINPACK benchmark for double precision calculation, Xserve dual 2.3 ghz/OSX give 10.3 gigaflops. a dual 3.4 ghz Xeon running Linux gives something like 10.5 gigaflops, an Opteron running linux gives you 8.3 gigaflops. So i guess the Xserver/OSX result are quiet remarquable. Look at the performance of Virginia Tech, COLSA clusters, i dont invent anything. All of them performs very good running OS X. If OS X would be so bad as you try to convince people, well it would be noticed.
So what your point, stop staying boolshit, you really dont now what you are talking about. Its funny to see those Linux fan boys trying to convince people that Linux is the only OS for HPC or clustering. Its particularly funny when a new big cluster project do not use Linux, then for them it should be a mistake as anything else than Linux sucks. I guess that they try to defend a position where Linux is (only?) good at, but really OS X became a big player in this market, sadly for Linux fan boys….
i don’t give a fuck if osx or linux is 20% faster. what seems more interesting to me is the kind of “problem” for which the computer will be build:
“The Omneta supercomputer, using advanced retrieval and pattern matching capabilities, can analyse all banking transactions, documents and communications over a seven year period – identifying activities that are more than one standard deviation outside of an established norm; potentially identifying insider trading and money laundering activities.
Omneta enables corporate electronic data (i.e. email, spreadsheets, documents, presentations, voice and video recordings, in addition to live-feed from video and telephone monitors) to be rapidly found, searched, categorised and retrieved, on demand.
Omneta has a powerful face and voice recognition capability that, once populated with the known details of staff, terrorists and/or wanted suspects, can provide unprecedented levels of security at ports, airports, office and other secure locations; enhancing individual, corporate, economic and community security.”
taken from their website: http://www.omnetaplc.com/