Re: [AMBER] "cudaDeviceEnablePeerAccess all CUDA-capable devices are busy or unavailable"

From: Ross Walker <ross.rosswalker.co.uk>
Date: Thu, 08 May 2014 00:00:46 -0700

Hi Ahmed,

You still didn't mention the types of calculations you are running. Are
these single GPU or are they multi-GPU?

I don't know why anyone would set EXCLUSIVE THREAD mode since I don't know
any applications that support it properly or rely on it.

To make a long short:

Single GPU runs with AMBER 14: If you want auto selection of GPUs as well
protection against a user overloading a single GPU then set "EXCLUSIVE
PROCESS" mode 'nvidia-smi -c 3'

Multi-GPU runs with AMBER 14: If you want to run multi-GPU runs with AMBER
14 since we now use peer to peer for communication you MUST set the GPUs
to 'Default' mode - no exceptions 'nvidia-smi -c 0'

My suggestion if you don't want to keep changing things and plan to run
some multi-GPU runs is to set everything to 'Default" and just make sure
your queuing system if you have one allocates GPUs correctly and then
threaten any users with starvation of their coffee supply if they override
the queuing system or when running manually don't religiously set
CUDA_VISIBLE_DEVICES correctly so GPUs are not overloaded.

There no method for running peer to peer in any mode other than default
and I don't expect this to change anytime soon.

All the best
Ross


On 5/7/14, 11:28 PM, "ahmed.sajid.stfc.ac.uk" <ahmed.sajid.stfc.ac.uk>
wrote:

>Hi,
>
>Sorry guys. I'm using Amber 14.
>
>I have the latest driver and I have compiled Amber using Cuda 6.
>
>Scott, All the GPUs that we have are setup for EXCLUSIVE THREAD mode. So
>I don't want to set them to compute mode.
>(http://archive.ambermd.org/201403/0127.html)
>
>Thanks for the input.
>
>Regards,
>Ahmed.
>--
>Scanned by iCritical.
>
>_______________________________________________
>AMBER mailing list
>AMBER.ambermd.org
>http://lists.ambermd.org/mailman/listinfo/amber



_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Thu May 08 2014 - 00:30:02 PDT
Custom Search