RE: [AMBER] 10Gb Interconnect profile

From: Ross Walker <>
Date: Wed, 10 Mar 2010 07:07:27 -0800

Hi Alessandro,

> We are thinking about a new setup for a cluster with 2 Intel Quad-core
> processors per machine (~13 machines in the cluster) and a 10Gb
> ethernet (Dell racks).
> In amber website I couldn't find any benchmark for 10Gb interconnect.
> So, my question is: does anybody have any experience with a similar
> setup to share?

I think you will be extremely lucky to see any reasonable performance of
10Gb Ethernet. Especially if you plan to use TCP/IP over it. If you use
something like Myrinet 10G you can get reasonable performance although not
as good infiniband. Unfortunately I don't have benchmarks for 10Gb Ethernet,
just 10Gb Myrinet, which does not use TCP/IP. Although I don't have these to
hand at present. You might find it cheaper to use SDR infiniband (which is
getting cheap these days) in place of 10Gb Ethernet and have Dell configure
it to support TCP/IP over infiniband (for NFS), or ideally have a 1Gb
Ethernet management / NFS network. This will at least give you reasonable
performance, although still way short of optimal, especially if you push NFS
traffic over the IB fabric. Although this will be MUCH better than 10Gb

Good luck,

|\oss Walker

| Assistant Research Professor |
| San Diego Supercomputer Center |
| Tel: +1 858 822 0854 | EMail:- |
| | |

Note: Electronic Mail is not secure, has no guarantee of delivery, may not
be read every day, and should not be used for urgent or sensitive issues.

AMBER mailing list
Received on Wed Mar 10 2010 - 07:30:03 PST
Custom Search