In a world where DGX-1 wasn't entirely for derp...
In an age where NVIDIA remembered that something other than derp mattered...
One app, one single codebase, could scale PME on DGX-1.
Sadly, that's some parallel universe not ours...
On Mon, Feb 6, 2017 at 12:56 PM, Ross Walker <ross.rosswalker.co.uk> wrote:
> Hi Chris,
>
> While AMBER will run 8 way P2P you won't see performance improvement for
> PME runs. Only large scale GB runs such as the 20K atom nucleosome test
> will scale to 8 GPUs. PME is blocked waiting on a decent FFT library from
> NVIDIA that can scale to multiple GPUs for the typical size of FFT grids
> AMBER uses. The only advantage you would get from going to an 8 way P2P
> system is the flexibility it would give you in scheduling 2 way P2P runs -
> you could use any 2 GPUs rather than having to make sure they were on the
> same PCI-E domain, or better on the same PLX switch. Note even things like
> the 10 and 20 P2P systems (cocolink etc) use PLX switches so although the
> P2P domain spans all GPUs there are still bottlenecks between banks of
> them. See:
>
> http://exxactcorp.com/blog/exploring-the-complexities-of-
> pcie-connectivity-and-peer-to-peer-communication/ <
> http://exxactcorp.com/blog/exploring-the-complexities-of-
> pcie-connectivity-and-peer-to-peer-communication/>
>
> My advice would be to stick to the existing 8 way boxes and run 8 x 1 GPU
> or 4 x 2 GPU runs (4 way P2P is possible but not very efficient - better to
> run 2 x 2 GPU runs). Note REMD runs will happily run across 8 GPUs if you
> have at least 8 replicas.
>
> All the best
> Ross
>
> > On Feb 6, 2017, at 15:31, Chris Neale <candrewn.gmail.com> wrote:
> >
> > Dear Users:
> >
> > We would like to build a node with 8 Pascal GTX-1080 GPUs capable of
> > running peer-to-peer for a single job (pmemd) utilizing all 8 GPUs. I see
> > an Exxact stystem linked on the amber homepage:
> > https://exxactcorp.com/index.php/solution/solu_detail/207 but I am not
> > entirely clear whether "Supports up to 8x Double Width GPUs" implies
> 8-way
> > peer-to-peer GPU communication.
> >
> > We have already one node with 8 GPUs but it is arranged as 2x4 such that
> a
> > single job can only use 4 GPUs efficiently and I am not sure if there are
> > yet motherboards that can handle 8-way GPU runs (or if they are yet
> > somewhat economical).
> >
> > Also, were such a machine available, will the same pmemd code that runs
> > well on 4 GPUs also run well on 8 GPUs or would code changes be required?
> >
> > Thank you for your help,
> > Chris.
> > _______________________________________________
> > AMBER mailing list
> > AMBER.ambermd.org
> > http://lists.ambermd.org/mailman/listinfo/amber
>
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Mon Feb 06 2017 - 13:00:04 PST
