I'll chime in here really quick... If you're using AmberTools 1.5, then I
think (according to Dave's email), I adjusted the configure script to
extract MPI_HOME from the mpif90 that's being used to compile the rest of
pmemd. This, I think, is the proper way to do it, since then the mpi.h file
included in the CUDA portions will come from the same distribution as the
mpif.h/mpi.h included in the *.f(90) and *.c files.
Furthermore, I think Ross may have misinterpreted and/or misread what
Fabricio was saying -- the standard way of installing MPIs is to use the
--prefix option at configure time (or live with /usr/local as the default),
which will not build as /usr/lib/openmpi, /usr/include/openmpi, and
/usr/bin/openmpi, but rather put a bin/, include/, and lib/ directory in
/usr/lib/openmpi. As long as "which mpif90" points to the one in
/usr/lib/openmpi/bin/mpif90, we should be fine here. However, if Ross was
right about what was happening, and I'm the one that misread, then this is
certainly going to cause issues (although it's not complaining about not
finding mpi.h, but rather some functions that it can't find, which I believe
indicates that it's an MPI-2 issue described below).
One problem with using OpenMPI, though, is that many versions excepting the
most recent ones, does NOT have (full) MPI-2 support, which means that it
CAN'T work for pmemd.cuda. I'm almost positive that I've verified this with
all OpenMPI 1.4.x versions (though OpenMPI 1.5.x may work). My suggestion
is to try another MPI implementation (I know that MPICH2 works, and is easy
to install). Just make sure to install MPICH2 in a proper way such that it
doesn't trample over your OpenMPI installation, and that "which mpif90"
points to the one in the directory you just installed.
May I suggest
On Sat, Jul 2, 2011 at 12:31 PM, Ross Walker <ross.rosswalker.co.uk> wrote:
> Hi Fabricio,
>
> > I have taken your advice and set MPI_HOME=/usr/lib/openmpi . It seems
> > to have worked partially since now the error comes at a later stage of
> > the "make" process. The error comes out as described below. Sorry for
> > the size of the paste below, but I wasn't sure where the error began.
> > Any suggestions? Thank you in advance
>
> Ah ha... Here in lies your problem I think. Your MPI installation would
> seem
> to be in an unusual place. Here you have /usr/lib/openmpi. I assume this
> contains the mpi library files and then you also have /usr/bin/openmpi and
> /usr/include/openmpi. This unfortunately will cause all sorts of problems
> since the configure script expects the paths to be of the form:
>
> $MPI_HOME/bin
> $MPI_HOME/include
> $MPI_HOME/lib
>
> In most cases your mpif90 / mpicc scripts will just take care of this but
> in
> the cuda compilation you have to use NVCC which means the MPI include
> directory needs to be manually specified.
>
> You will have something like the following in your config.h
>
> PMEMD_CU_INCLUDES=-I$(CUDA_HOME)/include -IB40C -IB40C/KernelCommon
> -I/usr/lib/openmpi/include
>
> Which I suspect is NOT where your mpi.h resides. You can edit config.h and
> change the -I to -I/usr/include/openmpi and it will probably work.
>
> The other option is to install MPI in a more 'normal' location. Ultimately
> though I guess we need to update the configure script to properly deal with
> these situations. There probably needs to be an MPI_INCLUDE environment
> variable or some kind of more advanced passing of mpif90 -show to deal with
> this situation since it is not compatible with the MPI_HOME approach or
> simply looking for the location of mpif90 which is what is currently used.
>
> On another note, if you are running on more than one node over IB I
> strongly
> recommend you use MVAPICH2 instead of openmpi since you will likely get
> much
> better performance.
>
> All the best
> Ross
>
> /\
> \/
> |\oss Walker
>
> ---------------------------------------------------------
> | Assistant Research Professor |
> | San Diego Supercomputer Center |
> | Adjunct Assistant Professor |
> | Dept. of Chemistry and Biochemistry |
> | University of California San Diego |
> | NVIDIA Fellow |
> | http://www.rosswalker.co.uk | http://www.wmd-lab.org/ |
> | Tel: +1 858 822 0854 | EMail:- ross.rosswalker.co.uk |
> ---------------------------------------------------------
>
> Note: Electronic Mail is not secure, has no guarantee of delivery, may not
> be read every day, and should not be used for urgent or sensitive issues.
>
>
>
>
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
--
Jason M. Swails
Quantum Theory Project,
University of Florida
Ph.D. Candidate
352-392-4032
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Sat Jul 02 2011 - 13:00:03 PDT
