Re: [AMBER] Running Amber on a SGI cluster

From: George Tzotzos <gtzotzos.me.com>
Date: Wed, 25 Sep 2013 12:26:20 -0300

Jason,

Many thanks. We have Infiniband running on the cluster. Is there another diagnostic to achieve better scaling?

Any suggestion will be much appreciated.

Regards

George


On Sep 25, 2013, at 8:52 AM, Jason Swails <jason.swails.gmail.com> wrote:

> On Tue, Sep 24, 2013 at 3:32 PM, George Tzotzos <gtzotzos.me.com> wrote:
>
>> Hi everybody,
>>
>> I'm trying to run Amber on a cluster of the following specs
>>
>> SGI Specs – SGI ICE X
>> OS - SUSE Linux Enterprise Server 11 SP2
>> Kernel Version: 3.0.38-0.5
>> 2x6-Core Intel Xeon
>>
>> 16 blades 12 cores each
>>
>> Environment
>> export AMBERHOME=/bio/georgios/MD/amber12
>> export
>> LD_LIBRARY_PATH=/opt/rpm_share/lib/lib64:/bio/george/amber12/lib:/bio/george/MD/amber12/AmberTools/lib
>>
>> Command line
>> mpirun -np 48 pmemd.MPI -O -i prod.in -o prod_12ns.out -p
>> 2erb_bis_solv.prmtop -c prod_10ns.rst -r prod_12ns.rst -x prod_12ns.mdcrd
>>
>> Question
>>
>> No advantage in increasing the number of nodes beyond -np 24. The
>> performance is reduced the more cores engaged. In fact it is similar or
>> worse to that on a OSX 2 x 3.06 GHz 6-Core Intel Xeon
>>
>> I'd be very grateful for any suggestions on what may be wrong
>>
>
> Your interconnect between nodes may be too slow. Also, the more cores you
> have on a single node, the more bandwidth you need between nodes to avoid a
> slow-down (this is why some supercomputers give faster timings for Amber
> when you do not utilize the whole node). This does not paint the whole
> picture (the topology of the inter-node connections also matters to some
> extent), but it's probably the most important part.
>
> You really need Infiniband (QDR is typical, I think) to see good scaling
> across nodes with Amber.
>
> HTH,
> Jason
>
> --
> Jason M. Swails
> BioMaPS,
> Rutgers University
> Postdoctoral Researcher
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber


_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Wed Sep 25 2013 - 08:30:04 PDT
Custom Search