Hosting Provided By

Re: about cluster in debian

From: Carsten Aulbert <carsten.aulbert(at)aei.mpg.de>
Date: Tue Aug 14 2007 - 01:59:30 EDT

Hi Bruno (I'm sharing this to the list again).

Bruno Sales wrote:
> Hi Carsten,
> it will be multi-threaded,
> they will need to access central storage areas
> low number of users (just to begin, i can grow up later)

Let's see. Depending on the money and needs I would suggest the following:

If you have very few threads running simultaneously, go for a single or dual socket multi core system fitting your needs. E.g if you have at most 6 threads running, buy a dual socket quad core Xeon or wait a bit longer and buy the same from AMD. However, I strongly suggest, that you come up with a benchmark like tool and talk to different vendors and ask them to test the boxes. If you have some money to spend, I think it may happen.

If you need more parallelism, you have to look into network infrastructure. Is GB ethernet good enough for you, or do you need low latency high bandwidth connections like Infiniband? That's something only you can decide. Usually with more testing and looking into your budget. Rule of thumb: After taking away money for storage, image spending all the money on network and computers going for infiniband. Then do the same for GBit ethernet and see how many more compute nodes you could purchase .Then you have to evaluate which would be better suited for your needs.

If you need some help on Gbit ethernet I might help a bit.

Storage: If you vast amounts of money to spend, look into the SUN X4500 running Solaris, although on this list I would advocate for a more standard set-up with a 3ware 9560 or Areca 12xx RAID6 controller with a bunch of not too cheap hardware. Usually you should be able to get something which is quite fast and shoudl cost lss than 1000US$/TB net storage.

Software:

Do you need help?

We use the Debian package FAI to automatically install cluster nodes, and since we don't have jobs running across nodes we use a simple batch system called condor (http://www.cs.wisc.edu/condor/), but you may want to look into openpgs, sun grid engine, torque among others.

HTH Carsten

-- 
To UNSUBSCRIBE, email to debian-beowulf-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Received on Tue Aug 14 01:59:47 2007

This message: [ Message body ]
Next message: Farid Behnia: "Configuration Management and Monitoring of a Debian Etch Beowulf Cluster"
Previous message: Carsten Aulbert: "Re: about cluster in debian"
Maybe in reply to: Bruno Sales: "about cluster in debian"

Contemporary messages sorted: [ By Date ] [ By Thread ] [ By Subject ] [ By Author ] [ By messages with attachments ]

This archive was generated by hypermail 2.1.8 : Sun Oct 07 2007 - 07:58:55 EDT