Re: [AMBER] any working singularity or docker recipe? (esp for CUDA and MPI)

From: Karl Kirschner <k.n.kirschner.gmail.com>
Date: Mon, 24 Jan 2022 09:20:57 +0100

Hi Mike and Gerald,

  I'm not sure if using the docker container for teaching is a goal of
yours, but in case it is, I can provide a bit of feedback.

  A student of mine was able to make a docker contain that included
AmberTools for teaching purposes only when our university went to online
classes. This was okay, but ended up not being used much. We also provided
an alternative solution to the students, which was to use a virtual machine
with a "customized Linux OS" that provided an environment that included
AmberTools among other things. A colleague and I have used this in our
remote courses (natural science and computer science students), and this
seems to be the preferred way by the students. (We also provided the third
option of having the students download and AmberTools directly, but didn't
provide help with that due to the very diverse OS that the students were
using - which would chew up my time Then also then had to install on their
own the other software that we wanted them to learn.) We are currently
writing this up, and a preprint is available at
https://doi.org/10.26434/chemrxiv.14356970.v1.

Best regards,
Karl

On Sun, Jan 23, 2022 at 10:43 PM Michael Coleman <mcolema5.uoregon.edu>
wrote:

> Hi Gerald,
>
> Thanks very much for your quick response. I'm realizing that my email
> lacked a key detail. My primary goal is just to get a working build of any
> sort, rather than to use containers in production (which complicates
> debugging). The documented Amber build procedure isn't working for us, and
> I was hoping that seeing a working version would provide some hints.
>
> The Amber license complicates things, but it seems like just a Singularity
> def wouldn't offend too badly.
>
> In any case, the good news is that looking again at the Spack recipe this
> weekend (last attempt a few months ago), it appears to work now!
> Definitely yes for CUDA. Still having some multi-node MPI startup issues,
> but that might be more about Spack than Amber.
>
> As a breadcrumb for others struggling to build Amber, here's my current
> best try:
>
> spack install -j 56 amber.20 %gcc +x11 +mpi +cuda ^cuda.10.2.89
> ^openmpi schedulers=slurm +cuda fabrics=auto
>
> This is on the 'devel' Spack branch, pulled a couple of days ago.
>
> This build passes some of the simple test cases for single-node MPI
> driving multiple K80s. (I haven't yet run the complete test suite, but I'm
> optimistic.)
>
> The 'cuda.10.2.89' works around an exasperating undocumented behavior of
> the Amber build, which is that GPU code is generated or not generated for
> different cards depending on the CUDA version number. In particular, K80
> code is not generated for CUDA 11, even though CUDA 11 supports K80s. The
> Amber code doesn't fail obviously for lack of this code, but rather one
> gets an obscure diagnostic.
>
> I'll update that 'spack install' command if I get a better one.
>
> Mike
>
>
> -----Original Message-----
> From: Gerald Monard <gerald.monard.gmail.com>
> Sent: Sunday, January 23, 2022 3:39 AM
> To: AMBER Mailing List <amber.ambermd.org>
> Subject: Re: [AMBER] any working singularity or docker recipe? (esp for
> CUDA and MPI)
>
> Also, just for the record and sharing my experience with docker and
> singularity containers here.
> What I have found so far is that these containers are not really as
> portable as one may think of.
> Basically, for the serial version of Amber, that's OK. You can take a
> container and go wherever you want (laptop to HPC center) to use it. It's
> fun!
> But for the CUDA version and the MPI version, it can get very tricky.
> For the CUDA version, you can end up with conflicts between the CUDA
> version inside the container and the driver on the host.
> For the MPI version, it depends whether you want to use the MPI version of
> the HPC center, or your own. And you can easily end up with
> incompatibility issues (see
>
> https://urldefense.com/v3/__https://sylabs.io/guides/latest/user-guide/mpi.html__;!!C5qS4YX3!S2zvzLtfCq2WYc44bGX4Xk4yX8jZNs3t3m7FCaVk77f-YtrNCDhtm3SN-T0gcjgVUEA$
> ).
> So sometimes, to get the best performance for Amber, it is better to create
> yourself a container that would suit best the platform on which you want to
> run than to rely on a pre-defined container made elsewhere.
> My 2 cents :-)
>
> Gerald.
>
>
> On Sun, Jan 23, 2022 at 8:31 PM Gerald Monard <gerald.monard.gmail.com>
> wrote:
>
> > Hello,
> >
> > I have developed singularity and docker containers to build and test the
> > Amber source code for different linux flavors. This is how the
> >
> https://urldefense.com/v3/__https://ambermd.org/InstSingularity.php__;!!C5qS4YX3!S2zvzLtfCq2WYc44bGX4Xk4yX8jZNs3t3m7FCaVk77f-YtrNCDhtm3SN-T0g-qqlW7A$
> page is generated. But these
> > containers only serve to _build_ Amber, they don't _contain_ Amber per
> se.
> > I am working on recipes to include the binaries after the build stage
> into
> > the containers. Hopefully, these will be included in the next Amber
> release.
> > In the meantime, I can share this with you (off the list) if you are
> > interested to beta test + give some feedback.
> >
> > best regards,
> > Gerald
> >
> > On Sat, Jan 22, 2022 at 2:05 PM Michael Coleman <mcolema5.uoregon.edu>
> > wrote:
> >
> >> Hi,
> >>
> >> Does anyone have a suggestion for a Singularity or Docker script to
> build
> >> a GPU- and MPI-enabled build of Amber? (and AmberTools?)
> >>
> >> It seems like there once was a Singularity script, 'amberity', which
> >> probably encapsulated a Singularity recipe, but it no longer seems
> >> accessible.
> >>
> >> This page mocks me, as it seems to be using such containerization. No
> >> source, though, so no help.
> https://urldefense.com/v3/__https://ambermd.org/InstSingularity.php__;!!C5qS4YX3!S2zvzLtfCq2WYc44bGX4Xk4yX8jZNs3t3m7FCaVk77f-YtrNCDhtm3SN-T0g-qqlW7A$
> >>
> >> I'm on RHEL 7, so the solution has to work on Linux generally. Beyond
> >> that, I'm all ears.
> >>
> >> Thank you,
> >> Mike
> >>
> >>
> >> Michael Coleman
> >> Computational Scientist
> >> Research Advanced Computing Services (HPC)
> >> University of Oregon
> >>
> >> _______________________________________________
> >> AMBER mailing list
> >> AMBER.ambermd.org
> >>
> https://urldefense.com/v3/__http://lists.ambermd.org/mailman/listinfo/amber__;!!C5qS4YX3!S2zvzLtfCq2WYc44bGX4Xk4yX8jZNs3t3m7FCaVk77f-YtrNCDhtm3SN-T0gzM7ylc4$
> >>
> >
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
>
> https://urldefense.com/v3/__http://lists.ambermd.org/mailman/listinfo/amber__;!!C5qS4YX3!S2zvzLtfCq2WYc44bGX4Xk4yX8jZNs3t3m7FCaVk77f-YtrNCDhtm3SN-T0gzM7ylc4$
>
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Mon Jan 24 2022 - 00:30:02 PST
Custom Search