Hi Vijay,
A few other things to check as well:
1) Make sure you set OMP_NUM_THREADS=1 on ALL nodes. This is especially true
if you use MKL.
2) Make sure all the nodes are clean before you start. I would suggest
shutting the entire thing down and powering it back on to be sure. (if you
can)
3) Make sure pmemd compiled with the correct MPI libraries..
4) Try compiling a single cpu version and compare the performance of that
against sander to see how it does.
5) Then try repeating with 2, 4, 8 cpus etc to see where the problem starts
to occur.
6) Finally once you have this try using ntt=1 and see if that changes
anything. You might also want to write to the output files less frequently
especially if someone else on the machine is hammering the disk.
Good luck,
Ross
> -----Original Message-----
> From: amber-bounces.ambermd.org [mailto:amber-bounces.ambermd.org] On
> Behalf Of Carlos Simmerling
> Sent: Thursday, May 14, 2009 3:09 AM
> To: AMBER Mailing List
> Subject: Re: [AMBER] pmemd running very slow in amber10
>
> I suspect the thermostat, but to find out more you should look at one
> of the scaling benchmark runs that you did (hopefully) and check the
> bottom of the file where pmemd prints out the time used for various
> tasks. that will be very helpful.
>
>
>
> On Thu, May 14, 2009 at 1:41 AM, Vijay Manickam Achari
> <vjrajamany.yahoo.com> wrote:
> >
> > Dear amber users,
> >
> > I have installed amber10 and pmemd in altix4700.
> > All the installation went well without error after guidance from
> amber mailing list.
> >
> > If I run sander.MPI in amber10, it runs fine, but if I run pmemd it
> run very slow. I run my script not using PBS but directly.
> >
> >
> > Here I give my *.in file and the *.out file contents.
> >
> > *in file
> > =========Dynamic Simulation with Constant Pressure
> > &cntrl
> > imin=0,
> > irest=1, ntx=7,
> > iwrap = 1, ntxo=1,
> > scnb=1.0, scee=1.0,
> > ntt=3, gamma_ln = 1.0,
> > tempi = 300.0, temp0=300.0, tautp=0.2,
> > ntb = 2, ntp=1, taup=0.2,
> > ntf=2,ntc=2,
> > nstlim=100000, dt=0.001,
> > ntwe=100, ntwx=100, ntpr=100, ntwr=-50000,
> > ntr=0,
> > /
> >
> >
> >
> > *.out
> > ============
> >
> > -------------------------------------------------------
> > Amber 10 SANDER 2008
> > -------------------------------------------------------
> >
> > | PMEMD implementation of SANDER, Release 10
> >
> > | Run on 05/14/2009 at 13:32:03
> >
> > [-O]verwriting output
> >
> > File Assignments:
> > | MDIN: MD-betaMalto-lowLyo.in
> > | MDOUT: betaMalto-lowLyo-MD00-run0000.out
> > | INPCRD: e2malto-lowLyo-equi21.rst_100000
> > | PARM: maltose-lowLyo.top
> > | RESTRT: betaMalto-lowLyo-MD01-run0100.rst
> > | REFC: refc
> > | MDVEL: mdvel
> > | MDEN: mden
> > | MDCRD: betaMalto-lowLyo-MD00-run0000.traj
> > | MDINFO: mdinfo
> > |LOGFILE: logfile
> >
> >
> > Here is the input file:
> >
> > Dynamic Simulation with Constant Pressure
> > &cntrl
> > imin=0,
> > irest=1, ntx=7,
> > iwrap = 1, ntxo=1,
> > scnb=1.0, scee=1.0,
> > ntt=3, gamma_ln = 1.0,
> > tempi = 300.0, temp0=300.0, tautp=0.2,
> > ntb = 2, ntp=1, taup=0.2,
> > ntf=2,ntc=2,
> > nstlim=100000, dt=0.001,
> > ntwe=100, ntwx=100, ntpr=100, ntwr=-50000,
> > ntr=0,
> > /
> >
> >
> >
> >
> >
> > | Conditional Compilation Defines Used:
> > | MPI
> > | SLOW_INDIRECTVEC
> > | PUBFFT
> > | MKL
> >
> > | Largest sphere to fit in unit cell has radius = 24.102
> >
> > | New format PARM file being parsed.
> > | Version = 1.000 Date = 06/23/08 Time = 17:06:22
> > | Duplicated 0 dihedrals
> >
> > | Duplicated 0 dihedrals
> >
> > ---------------------------------------------------------------------
> -----------
> > 1. RESOURCE USE:
> > ---------------------------------------------------------------------
> -----------
> >
> > getting new box info from bottom of inpcrd
> >
> > NATOM = 23736 NTYPES = 9 NBONH = 14776 MBONA = 9216
> > NTHETH = 27648 MTHETA = 12032 NPHIH = 45312 MPHIA = 21248
> > NHPARM = 0 NPARM = 0 NNB = 123552 NRES = 1256
> > NBONA = 9216 NTHETA = 12032 NPHIA = 21248 NUMBND = 9
> > NUMANG = 14 NPTRA = 20 NATYP = 9 NPHB = 1
> > IFBOX = 1 NMXRS = 81 IFCAP = 0 NEXTRA = 0
> > NCOPY = 0
> >
> > | Coordinate Index Table dimensions: 18 9 10
> > | Direct force subcell size = 5.1696 5.3561 5.0569
> >
> > BOX TYPE: RECTILINEAR
> >
> > ---------------------------------------------------------------------
> -----------
> > 2. CONTROL DATA FOR THE RUN
> > ---------------------------------------------------------------------
> -----------
> >
> >
> >
> > General flags:
> > imin = 0, nmropt = 0
> >
> > Nature and format of input:
> > ntx = 7, irest = 1, ntrx = 1
> >
> > Nature and format of output:
> > ntxo = 1, ntpr = 100, ntrx = 1, ntwr
> = -50000
> > iwrap = 1, ntwx = 100, ntwv = 0, ntwe
> = 100
> > ioutfm = 0, ntwprt = 0, idecomp = 0,
> rbornstat= 0
> >
> > Potential function:
> > ntf = 2, ntb = 2, igb = 0, nsnb
> = 25
> > ipol = 0, gbsa = 0, iesp = 0
> > dielc = 1.00000, cut = 8.00000, intdiel = 1.00000
> > scnb = 1.00000, scee = 1.00000
> >
> > Frozen or restrained atoms:
> > ibelly = 0, ntr = 0
> >
> > Molecular dynamics:
> > nstlim = 100000, nscm = 1000, nrespa = 1
> > t = 0.00000, dt = 0.00100, vlimit = 20.00000
> >
> > Langevin dynamics temperature regulation:
> > ig = 71277
> > temp0 = 300.00000, tempi = 300.00000, gamma_ln= 1.00000
> >
> > Pressure regulation:
> > ntp = 1
> > pres0 = 1.00000, comp = 44.60000, taup = 0.20000
> >
> > SHAKE:
> > ntc = 2, jfastw = 0
> > tol = 0.00001
> >
> > | Intermolecular bonds treatment:
> > | no_intermolecular_bonds = 1
> >
> > | Energy averages sample interval:
> > | ene_avg_sampling = 100
> >
> > Ewald parameters:
> > verbose = 0, ew_type = 0, nbflag = 1, use_pme
> = 1
> > vdwmeth = 1, eedmeth = 1, netfrc = 1
> > Box X = 93.053 Box Y = 48.205 Box Z = 50.569
> > Alpha = 90.000 Beta = 90.000 Gamma = 90.000
> > NFFT1 = 96 NFFT2 = 50 NFFT3 = 54
> > Cutoff= 8.000 Tol =0.100E-04
> > Ewald Coefficient = 0.34864
> > Interpolation order = 4
> >
> > | PMEMD ewald parallel performance parameters:
> > | block_fft = 1
> > | fft_blk_y_divisor = 4
> > | excl_recip = 0
> > | excl_master = 0
> > | atm_redist_freq = 320
> >
> > ---------------------------------------------------------------------
> -----------
> > 3. ATOMIC COORDINATES AND VELOCITIES
> > ---------------------------------------------------------------------
> -----------
> >
> >
> > begin time read from input coords = 4012.500 ps
> >
> > Number of triangulated 3-point waters found: 1000
> >
> > Sum of charges from parm topology file = 0.00000000
> > Forcing neutrality...
> >
> > | Dynamic Memory, Types Used:
> > | Reals 908174
> > | Integers 1893246
> >
> > | Nonbonded Pairs Initial Allocation: 168376
> >
> > | Running AMBER/MPI version on 64 nodes
> >
> >
> > ---------------------------------------------------------------------
> -----------
> > 4. RESULTS
> > ---------------------------------------------------------------------
> -----------
> >
> > ---------------------------------------------------
> > APPROXIMATING switch and d/dx switch using CUBIC SPLINE
> INTERPOLATION
> > using 5000.0 points per unit in tabled values
> > TESTING RELATIVE ERROR over r ranging from 0.0 to cutoff
> > | CHECK switch(x): max rel err = 0.3338E-14 at 2.509280
> > | CHECK d/dx switch(x): max rel err = 0.8261E-11 at 2.768360
> > ---------------------------------------------------
> >
> > NSTEP = 100 TIME(PS) = 4012.600 TEMP(K) = 301.34 PRESS
> = -45.6
> > Etot = 72323.5211 EKtot = 16896.3909 EPtot =
> 55427.1302
> > BOND = 4331.7919 ANGLE = 12660.8451 DIHED =
> 3557.8100
> > 1-4 NB = 5543.6944 1-4 EEL = 112668.1311 VDWAALS = -
> 10250.7744
> > EELEC = -73084.3679 EHBOND = 0.0000 RESTRAINT =
> 0.0000
> > EKCMT = 1136.7441 VIRIAL = 1359.7065 VOLUME =
> 226657.6514
> > Density =
> 1.0897
> > Ewald error estimate: 0.1325E-03
> > --------------------------------------------------------------------
> ----------
> >
> >
> > NSTEP = 200 TIME(PS) = 4012.700 TEMP(K) = 300.36 PRESS
> = 51.9
> > Etot = 72308.4304 EKtot = 16841.3962 EPtot =
> 55467.0342
> > BOND = 4223.5093 ANGLE = 12742.0334 DIHED =
> 3598.7024
> > 1-4 NB = 5569.0962 1-4 EEL = 112679.4301 VDWAALS = -
> 10228.2235
> > EELEC = -73117.5137 EHBOND = 0.0000 RESTRAINT =
> 0.0000
> > EKCMT = 1124.6180 VIRIAL = 870.8898 VOLUME =
> 226443.8438
> > Density =
> 1.0907
> > Ewald error estimate: 0.8322E-04
> > --------------------------------------------------------------------
> ----------
> >
> >
> >
> > I dont know what is the error.
> > Can anyone help?
> >
> > Thank you in advance.
> >
> >
> >
> > Vijay Manickam Achari
> > (Phd Student c/o Prof Rauzah Hashim)
> > Chemistry Department,
> > University of Malaya,
> > Malaysia
> > vjramana.gmail.com
> >
> >
> >
> >
> > _______________________________________________
> > AMBER mailing list
> > AMBER.ambermd.org
> > http://lists.ambermd.org/mailman/listinfo/amber
> >
>
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Wed May 20 2009 - 16:29:45 PDT
