Dear Amber users,
I need urgently parallel version of NAB to use it for huge system (15000
atoms) analysis.
I found just a little about this topic in AmberTools manual pages 115-116.
Our platform is "SGI AltixXE 310" with "Intel Xeon Quad-core 5365"
processors.
I installed AmberTools (in serial using icc compiler) without any
problems and
also Amber using ifort compiler ( serial and also parallel with flag
-intelmpi).
Typical perfectly working script seems like this:
---------------START-------------------------------------
#!/bin/bash
#number of nodes to load into virtual machine
NODES=2
#number of processors per node
PPN=8
#total number of processors allocated for the given job
CPUS=16
.
.
.
#export NO_STOP_MESSAGE=" "
export EXE="/opt/amber/exe/sander.MPI"
#creation of virtual multiprocessor machine
mpdboot -f ~/.mpd.hosts -n $NODES
#run sander/pmemd
mpiexec -ppn $PPN -n $CPUS $EXE -O -i $IN -o $OUT -c $INPCRD -p $PRMTOP
-r $RST
#'destruction' of the virtual machine
mpdallexit
----------------END----------------------------------
I thaught that it will be easy to run like this also NAB in parallel but
it isn't :((
I tried something like this:
---------------START-------------------------------------
#!/bin/bash
NODES=1
PPN=8
CPUS=8
nab -O -o makej_MPI makej_MPI.nab
mpdboot -f ~/.mpd.hosts -n $NODES
mpiexec -ppn $PPN -n $CPUS ./makej_MPI > results.txt
mpdallexit
--------------END----------------------------------------
But it just loaded 8 serial jobs in one time nothing more.
The main problem is probably that I was not able to compile NAB in
parallel way.
I tried -mpi flag but using this option was absolutely useless it even
didn't add
the proper flags into config.h. When I tried "-openmp" flag situation was
a little
better and -openmp successfully propagated into config.h in this two lines:
-L$(MKL_HOME)/lib/em64t -lguide -lpthread -openmp
OCFLAGS=-wd117,177,266,880,1011 -O2 -DBINTRAJ -openmp -DOPENMP
$(AMBERBUILDFLAGS)
But I am still not sure if even my compilation "nab -O -o makej_MPI
makej_MPI.nab" create now
parallel binary file "makej_MPI" since this should be probably clear from
the c source makej_MPI.c and
I didn't find any "MPI" strings here. Of course that also examination of
my binary file was not going in parallel,
although I tried play with OMP_NUM_THREADS constant and tried to run my
executable makej_MPI with or without mpiexec command.
In my opinion the root of the problem is that -intelmpi flag is not
allowed in case of NAB parallel configuration apart from
the AMBER where is available and works perfectly. Is there any chance how
to bypass this problem for example edit config.h
created with -openmp and put everywhere instead this flag just -intelmpi
or something like this ?
If is really no chance/trick how to use -intelmpi in case of parallel NAB
compilation what could be a solution for me ?
Since we have not ScaLAPACK installed yet, probably only one chance is to
use -openmp.
If "-intelmpi way" (IntelMPI) is really unpossible could anyone who has
some experience with parallel compilation/run of NAB with -openmp (under
OpenMPI)
provide a small "How to ?" = if and how modify "nab -O -o makej_MPI
makej_MPI.nab" (compilation of nab programs), how modify "mpiexec -ppn
$PPN -n $CPUS ./makej_MPI > results.txt" ...
Last my question is this: Is there any dependence of the number of
parallelisable NAB tools on the given flag (-openmp, -mpi, -scalapack ).
After reading of page 116 I think that yes and all the important
functions: conjgrad(), newton( ), nmode() ... are available in parallel
version
only using -scalapack flag. Am I right ? Could anyone please make simple
list of NAB functions which are parallelisable in each particular case (
-openmp, -mpi, -scalapack ).
Thanks a lot in advance ! for any relevant info or link/s to the relevant
materials which describes "NAB in PARALLEL" story a little more in detail
than AmberTools manual.
Marek
--
Tato zpráva byla vytvořena převratným poštovním klientem Opery:
http://www.opera.com/mail/
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Fri Jan 30 2009 - 01:31:13 PST
