Create a working Linux cluster from many separate pieces of hardware and software. Part 1 of the series provides detailed instructions on setting up the hardware for the cluster. Part 2 describes configuring the management server and installing the nodes in the cluster. This provides an easy way to get started with large Linux clusters.
Quite technical. Anybody have ideas for stuff to do with clusters? Running one at work or home would be cool, but of course, you have to be a data-crunching scientist or company to justify one.
It depends on the clustering technology. If you use a MOSIX based system (which is what I would recomend, if you are not into scientific number crunching alone), you can load-balance seperate tasks quite nicely across a even heterogenous cluster (I have installed one for my sister who is not really a number cruncher, for parallising cd ripping to ogg files or render analog-to-digital videos from her camcorder at a process level. Very effective, even though the cluster employs several older machines). Mosix clusters are also known to work with scientific/mathematical software like (IIRC, please correct me if I’m wrong) octave or R.
I guess you could call me a professional number cruncher (computational physicist) and we use MPI (and sometimes even PVM) to parallise our integral equation solvers at code level (e.g. the programm has to implement the parallising structures), mostly due to the fact, that the equations kernels would never fit into a single machines memory.
If you programm in C/Fortran/C++ or Java, then playing with MPI to parallise existing code can be interesting and entertaining
EDIT: Some pointers for usage scenarios with openMosix, perhaps this gives you a better estimation of what general purpose applications can benefit from clusters.
[1] http://howto.x-tend.be/openMosix-HOWTO/x212.html
[2] http://povmosix.sourceforge.net/ [PovMosix = Povray + OpenMosix]
[3] http://www.imsc.res.in/~parapp/IMSCTalk.pdf
I have currently no access to my link collection, perhaps this three urls help a little
Edited 2007-01-29 18:13
Yeah, I actually did use MPI for a parallel computing class. Interesting stuff.
And I actually looked at getting a graduate degree in physics for the computational/quantum physics aspect. But I could never study well (lack of focus). If only I’d heard of adderall back then. Tell me some more about computational physics though!
I’m not the most focused guy myself, I’ve worked in several non-related fields (automatisation, high speed data acquisition, catering, student counceling, web design) during my undergraduate studies, that’s why it took me so long to finish it.
And the looming of the dark side (theoretical informatics, mathematics, clustering, web design) didn’t helped either :-).
Tell me some more about computational physics though!
Hm, its a rather interdisciplinary branch of physics, although sooner or later one has to specialise on one research topic nevertheless. At my university it is basically a “reversed” physics study insofar as one specialises first on the tools (numerical mathematics, algorithms, a little theoretical informatics and hardware/software/signal theory) and later learns the more advanced physics needed to do actual research. Or to make a long story short: computational physicsts try to solve problems that are too physical for persons with a standard informatic / telematic background and too demanding wrt numerical knowledge/methods for our non-specialised peers from the theoretical or experimental branch of physics.
Nice to see some interest in physics, normally people look at me as if I were an axe murder, when I tell them about my profession 🙂
Edited 2007-01-29 18:48
Clustering can be using with LTSP…So you can have a terminal server/node setup which uses clustering. That will be benificial as the server needs horsepower and memory to support the thinclients and in theory it should get that from each node as it logs in.
the best way for a home user to play around with this stuff is to use vmware. i setup an openmosix cluster in vmware just to learn.
I recommend the book on it by Karl Kopper rather highly:
http://www.oreilly.com/catalog/1593270364/
And once again I dissapointed of quality of articles from ibm.com, so many of them and still not one worth reading (ok maybe about the features of zsh was worth reading but thats it), whats the point of submitting them if they all suck?
I just wish they would get this running for the 2.6 kernel series. Seens like development is a bit slow. I dont really want to go back to running a 2.4 kernel but this would be wicked cool at home.