I'll also add that the peer-to-peer bandwidth test that he's referring to
is multi-threaded not multi-process and that makes it irrelevant.
On Jan 27, 2017 6:54 AM, "Ross Walker" <ross.rosswalker.co.uk> wrote:
> Hi Matt,
>
> Unfortunately the way AMBER does P2P does not work with the GPUs in
> process exclusive mode. I would suggest going to the sysadmin and having
> them modify the queuing system to allow either the user to specify the mode
> in the job submission script or have some logic in there to set the mode
> depending if the user is requesting more than 1 GPU or if they add some
> specific request - e.g. P2P or some other keyword in the job submission
> script. We used to do this at SDSC and it worked great. It can go in the
> prolog of the queuing system which runs as root and thus can issue
> nvidia-smi -c XXX commands. Eva Hocks had this working well at SDSC with
> SGE, your sysadmin may want to try contacting her and see if she'll share
> the config scripts.
>
> All the best
> Ross
>
> > On Jan 26, 2017, at 17:12, Matthew Harrigan <harrigan.stanford.edu>
> wrote:
> >
> > Hi all,
> >
> >
> > I'm trying to run a multi-gpu simulation. Our cluster's GPUs are set to
> process exclusive mode. Per http://archive.ambermd.org/201405/0145.html
> it is not surprising that this fails with the message
> >
> >
> > cudaDeviceEnablePeerAccess all CUDA-capable devices are busy or
> unavailable
> > cudaDeviceEnablePeerAccess all CUDA-capable devices are busy or
> unavailable
> >
> >
> > However, our cluster administrator doesn't want to set the compute mode
> to default *but also* points out that p2p works in general in process
> exclusive mode. Specifically, if I run p2pBandwidthLatencyTest from the
> CUDA SDK samples, it shows p2p is enabled
> >
> >
> > $ ./p2pBandwidthLatencyTest
> > [P2P (Peer-to-Peer) GPU Bandwidth Latency Test]
> > Device: 0, Tesla K80, pciBusID: 93, pciDeviceID: 0, pciDomainID:0
> > Device: 1, Tesla K80, pciBusID: 94, pciDeviceID: 0, pciDomainID:0
> > Device=0 CAN Access Peer Device=1
> > Device=1 CAN Access Peer Device=0
> > <snip>
> >
> >
> > Is there some way amber can be modified to use p2p in process exclusive
> mode?
> >
> >
> > Matt
> >
> >
> > _______________________________________________
> > AMBER mailing list
> > AMBER.ambermd.org
> > http://lists.ambermd.org/mailman/listinfo/amber
>
>
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Fri Jan 27 2017 - 09:00:02 PST
