Hi Uday,
Are you certain that you used the correct infiniband library for your
system. Check with 'which mpirun', 'which mpif90' etc. You need to be
certain that you are using the MPI that is designed to run over the IB
interface (likely an up to date version of mvapich) rather than over
ethernet which is what something like a basic openMPI or mpich2
installation would likely use.
Note hyper threads don't tend to help you much (since AMBER is almost
entirely floating point) so there is not much likely to be gained using 32
threads per node. I would use at most 16 and you might find that using
just 12 per node is possibly more efficient.
Note you will need to make sure your queuing system is correctly setup to
allocate cores on the nodes correctly and be sure it isn't running all the
threads on 1 node.
The fact that you don't give any arguments to your mpirun command
regarding a hostfile (such as $PBS_NODEFILE) or other settings related to
core and node counts makes me very suspicious of your configuration. You
might want to check with whoever administers your cluster to find out what
is the correct compilation approach, environment variables to set and MPI
launch commands and parameters for IB runs.
All the best
Ross
On 8/15/13 9:53 PM, "Uday Midya" <umidya.iitkgp.gmail.com> wrote:
>Hi
>I have installed the AMBER12 (serial and parallel) in our rock cluster
>with the InfiniBand communication among the nodes. Prodedure was as
>follows.
>
>tar xvf AmberTools12.tar.bz2
>tar xvfj Amber12.tar.bz2
>export AMBERHOME=/home/myname/amber12 (added to .bashrc file)
>$AMBERHOME/bin added to PATH
>cd $AMBERHOME
>./configure gnu
>make install
>make test
>
>At this point no failure massage.
>
>./configure -mpi gnu
>make install
>export DO_PARALLEL=˛mpirun -np 8˛
>make test
>
>No failure massage here.
>
>But when I try to benchmark the speed of samder.MPI, following are found
>
>#atom #step #cpu (thread) #node time
>
>84394 60000 16 2 04:00:00
> do do 32 2
>09:20:00
> do do 16 1
>03:36:00
> do do 32 4
>05:14:00
>
>
>
>It seem that when I use multiple nodes with more cpus the speed decreases.
>Can any one tell me why this is happening? Following is the betails of the
>cluster.
>
>Proccessor: 64-bit Intel Xeon processor.
>#proccessor/node =2
>#core/proccessor =8
>#thread/core =2
>total #thread/node=32
>OS: CentOS 6.3
>Cluster: rock
>Communication: InfiniBand
>
>Thanks in advanced
>Uday Sankar Midya
>Dept. of Chemistry
>IIT Kharagpur
>_______________________________________________
>AMBER mailing list
>AMBER.ambermd.org
>http://lists.ambermd.org/mailman/listinfo/amber
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Thu Aug 15 2013 - 23:00:03 PDT
