Is there another MPI you can use on your system for mpi4py? MMPBSA.py.MPI
uses virtually no form of communication, so no one MPI will work better than
another in this case. However, it's difficult for me to diagnose the
problem without being able to investigate things myself.
Are you sure that mpicc comes from the same mpich2 that you're using?
Because as far as I know mpi4py automatically grabs the first "mpicc" in
your PATH, and I'm not familiar enough with Python's build tools to know how
to modify it.
Good luck!
Jason
On Tue, Jan 11, 2011 at 12:25 PM, amit dong <dongamit123.gmail.com> wrote:
> Thanks Jason for the instant reply. I re-compiled mpi4py and mmpbsa.py
> module with mpich2. But still it uses single processor. Do you have
> any other suggestions that I can troubleshoot.
>
>
> >
> > > 2. Is there any way to know that the snapshots are actually distributed
> > to
> > > different nodes? Even though I submitted the job (4 snapshots) to 2
> nodes
> > ,
> > > the log file says job submitted to 1 processor. I am copying the log
> file
> > > and the submission script below.
> > >
> >
> > MMPBSA.py.MPI uses MPI.Get_size() to get the number of threads running,
> > which is exactly what the MPI communicator will see. If it says 1
> > processor, then it only sees 1 processor. You can see that this is the
> > case
> > by looking at your output below. You see the garbled output of 8
> > MMPBSA.py.MPI jobs trying to run all at the same time as though they are
> > they running alone.
> >
> > On the bright side, this means that you successfully compiled mpi4py and
> > properly put it in your PYTHONPATH (as I see in your submission script),
> > otherwise MMPBSA.py.MPI would have quit in error. I have seen this kind
> of
> > behavior often with MPI clashes. By this, I mean that the MPI
> > implementation used to build mpi4py is NOT the same implementation from
> > which you're using mpiexec/mpirun. It's possible that the build process
> of
> > mpi4py inadvertently grabbed an mpicc in a directory that is in your PATH
> > before your MPI_HOME/bin directory (for instance, if there is some stray
> > mpicc in /usr/bin or /usr/local/bin). Or perhaps you changed MPIs at
> some
> > point?
> >
> > One final note -- if this MPI confusion did not occur, the simulation
> would
> > have failed. MMPBSA.py.MPI parallelizes by assigning an equal number of
> > frames to each processor (some threads may have 1 extra if it doesn't
> > divide
> > the frames evenly). However, if there are MORE threads than you have
> > frames, then MMPBSA.py.MPI will tell you this is not allowed and quit.
> > You've asked for 8 threads, yet only 4 frames.
> >
> > I hope this helps,
> > Jason
> >
> >
> > > MMPBSA.py.MPI being run on 1 processors
> > > ptraj found! Using /amber11/exe/ptraj
> > > sander found! Using /amber11/exe/sander
> > >
> > > Preparing trajectories with ptraj...
> > > MMPBSA.py.MPI being run on 1 processors
> > > ptraj found! Using /amber11/exe/ptraj
> > > sander found! Using /amber11/exe/sander
> > >
> > > Preparing trajectories with ptraj...
> > > MMPBSA.py.MPI being run on 1 processors
> > > ptraj found! Using /amber11/exe/ptraj
> > > sander found! Using /amber11/exe/sander
> > >
> > > Preparing trajectories with ptraj...
> > > MMPBSA.py.MPI being run on 1 processors
> > > ptraj found! Using /amber11/exe/ptraj
> > > sander found! Using /amber11/exe/sander
> > >
> > > Preparing trajectories with ptraj...
> > > MMPBSA.py.MPI being run on 1 processors
> > > ptraj found! Using /amber11/exe/ptraj
> > > sander found! Using /amber11/exe/sander
> > >
> > > Preparing trajectories with ptraj...
> > > 4 frames were read in and processed by ptraj for use in calculation.
> > >
> > > Starting calculations
> > >
> > > Starting gb calculation...
> > >
> > > calculating ligand contribution...
> > > calculating receptor contribution...
> > > MMPBSA.py.MPI being run on 1 processors
> > > ptraj found! Using /amber11/exe/ptraj
> > > sander found! Using /amber11/exe/sander
> > >
> > > Preparing trajectories with ptraj...
> > > MMPBSA.py.MPI being run on 1 processors
> > > ptraj found! Using /amber11/exe/ptraj
> > > sander found! Using /amber11/exe/sander
> > >
> > > Preparing trajectories with ptraj...
> > > 4 frames were read in and processed by ptraj for use in calculation.
> > >
> > > Starting calculations
> > >
> > > Starting gb calculation...
> > >
> > > calculating ligand contribution...
> > > calculating receptor contribution...
> > > calculating complex contribution...
> > > calculating complex contribution...
> > >
> > > Calculations complete. Writing output file(s)...
> > >
> > >
> > > Submission script:
> > >
> > > #!/bin/sh
> > > #PBS -l walltime=24:00:00
> > > #PBS -l nodes=2:ppn=4
> > > #PBS -V
> > > #PBS -N decomp.py
> > >
> > > export AMBERHOME=/amber11
> > >
> > > export WORK_DIR=/home/abc
> > > export PYTHONPATH=/home/abc/lib/python2.7/site-packages\:$PYTHONPATH
> > >
> > > cd $WORK_DIR
> > > export NPROCS=`wc -l $PBS_NODEFILE |gawk '//{print $1}'`
> > > echo $NPROCS
> > > mpdboot -n 5 -f $HOME/mpd.hosts -v
> > > mpdtrace -l
> > > mpiexec -n $NPROCS $AMBERHOME/bin/MMPBSA.py.MPI -O -i input.in -o
> > > FINAL_RESULTS_MMPBSA.dat -sp complex_wat.prmtop -cp complex.prmtop -rp
> > > receptor.prmtop -lp lig.prmtop -y md.x.gz > output.log
> > > mpdallexit
> > >
> > >
> > > Thanks!!
> > > AD
> > > _______________________________________________
> > > AMBER mailing list
> > > AMBER.ambermd.org
> > > http://lists.ambermd.org/mailman/listinfo/amber
> > >
> >
> >
> >
> > --
> > Jason M. Swails
> > Quantum Theory Project,
> > University of Florida
> > Ph.D. Graduate Student
> > 352-392-4032
> > _______________________________________________
> > AMBER mailing list
> > AMBER.ambermd.org
> > http://lists.ambermd.org/mailman/listinfo/amber
> >
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
--
Jason M. Swails
Quantum Theory Project,
University of Florida
Ph.D. Graduate Student
352-392-4032
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Wed Jan 12 2011 - 07:30:02 PST
