On Thu, Apr 12, 2018 at 1:14 PM, Ryan Novosielski <novosirj.rutgers.edu> wrote:
>
> My question is what is the recommendation for what to do about MPI, and subsequently, the modules system (which is out of scope here, but someone may have solved the problem). What I’d done so far is compiled everything with the same Intel compiler, including the MVAPICH2 stack, and this had worked. If I use the GNU compiler for the GPU portions, I suppose I’ll have to use MVAPICH2 compiled with the GNU compiler as well, which I suppose them complicates hierarchical modules/prevents one from using the MPI-enbabled CPU and GPU versions together, if that’s something anyone ever needs to do.
In practice, modules actually make supporting the separate CPU/GPU
builds of Amber simple. What I do for our local cluster is build the
CPU version with Intel compilers and either Intel MPI or Mvapich built
with Intel compilers, and the GPU version with GCC5/Mvapich (GNU)/CUDA
8. Then, to simplify things for users I create Amber and AmberGPU
modules in the core modulefiles directory that take care of loading in
the required modules, so that all users have to do is e.g.
module load AmberGPU
to get the GPU build of Amber (instead of 'module load gcc/5.4.0
mvapich2 cuda' etc). This also makes it easy when you're updating the
Amber build since the top-level module name doesn't ever change.
I'm a big fan of using GCC with the GPU build of Amber. I found that
when doing burn-in testing of GPUs with pmemd.cuda I always get
reproducibly consistent results with GNU/CUDA, whereas with Intel/CUDA
sometimes I get two sets of results (on the same GPUs). Of course this
was probably with Intel 16 compilers (I don't remember exactly when I
did these tests but it was a year or so back) so YMMV with newer
versions.
Hope this helps,
-Dan
--
-------------------------
Daniel R. Roe
Laboratory of Computational Biology
National Institutes of Health, NHLBI
5635 Fishers Ln, Rm T900
Rockville MD, 20852
https://www.lobos.nih.gov/lcb
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Fri Apr 13 2018 - 08:30:02 PDT
