Dear Amber pro,
I have been struggling with my MD simulation using sander.MPI for many
days. Tried many ways, but still can not figure out why my parallel running
is not speed up. Using 2 nodes, is faster than using 1 node. But when I
used 8 nodes, the speed is similar as using 1 node. My system have 101,679
atoms. The following is the detail of my tests.
PBS script template:
#!/bin/csh
#PBS -V
#PBS -r n
#PBS -N ENZ_6ns
#PBS -v AMBERHOME=/home/appmgr/Software/Amber/amber11
#PBS -v PATH=$PATH:$AMBERHOME/exe:
#PBS -v LD_LIBRARY_PATH=/home/appmgr/Software/Openmpi/openmpi-1.4.3/exe/lib
#PBS -l select=8:ncpus=8
#PBS -l select=arch=linux
#PBS -l place=scatter
#PBS -m abe
#PBS -q verylong
#
set WRKDIR=/home/william/Work/ENZ/MD/KP94_ENZ
cd $WRKDIR
ulimit -a
export OMP_NUM_THREADS=64
##unset OMP_NUM_THREADS
mpirun -np 64 -machinefile $PBS_NODEFILE sander.MPI -O -i
ENZ_KP94_CnsP_4_6ns.in \
-o ENZ_KP94_CnsP_6ns.out \
-p K94_ENZ.top \
-c ENZ_KP94_CnsP_4ns.rst \
-x ENZ_KP94_CnsP_6ns.mdcrd \
-v ENZ_KP94_CnsP_6ns.mdvel \
-e ENZ_KP94_CnsP_6ns.mden \
-r ENZ_KP94_CnsP_6ns.rst \
-inf ENZ_KP94_CnsP_6ns.mdinfo
The following tests were done based on the script above:
-v: “-“ deleted all –v options from PBS script. “Y” has the options. Same
as the other options.
Test set 1: with #PBS -l select=8:ncpus=8
Options
-v
Y
Y
Y
Y
Y
unlimit -a
-
Y
Y
Y
-
export OMP_NUM_THREADS
-
-
Y
-
-
unset OMP_NUM_THREADS
-
-
-
Y
Y
Speed
0.3ns/day
271080.0 Sec/ns
0.3ns/day
254625.4 Sec/ns
0.3ns/day
255308.4 Sec/ns
0.3ns/day
262850.4 Sec/ns
0.3ns/day
252493.3 Sec/ns
Test set 2: with #PBS -l select=2:ncpus=8
Options
-v
Y
Y
Y
Y
Y
unlimit -a
-
Y
Y
Y
-
export OMP_NUM_THREADS
-
-
Y
-
-
unset OMP_NUM_THREADS
-
-
-
Y
Y
Speed
0.5ns/day
183578.9 Sec/ns
0.5ns/day
181977.6 Sec/ns
0.5ns/day
183625.0 Sec/ns
0.5ns/day
181734.3 Sec/ns
0.5ns/day
181799.4 Sec/ns
Test set 3: with #PBS -l select=1:ncpus=8
Options
-v
Y
Y
Y
Y
Y
unlimit -a
-
Y
Y
Y
-
export OMP_NUM_THREADS
-
-
Y
-
-
unset OMP_NUM_THREADS
-
-
-
Y
Y
Speed
0.3ns/day
276932.5 Sec/ns
0.3ns/day
278406.4 Sec/ns
0.3ns/day
276227.4 Sec/ns
0.3ns/day
276956.7 Sec/ns
0.3ns/day
274001.6 Sec/ns
Any help and clue is appreciated.
William
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Thu Oct 20 2011 - 09:30:03 PDT
