Re: [AMBER] Problem with two jobs on one GPU

From: Ross Walker <>
Date: Mon, 6 Jun 2011 09:23:08 -0700

Hi Marek,

> some time ago I asked here question, as a part of some discussion,
> about
> the possibility to overload GPU card (here TESLA 2050) with
> calculations
> of two
> simultaneous jobs. If I remember Ross answered that it should be OK
> although,
> performance will decrease significantly comparing to just one job
> calculation.

I VERY MUCH doubt I said it like that. If I did I was probably answer emails
after one too many martinis. Yes you can run 2 jobs at the same time on a
single GPU. Do I recommend it? NO I CERTAINLY DO NOT. It is pointless for
anything other than testing. The performance will completely suck, maybe
each job will run about 10% of the speed it would normally. Thus you run 2
jobs at once you get about 20% of the total throughput as you would running
a single job. So consider you want to run 2 jobs each of 5 ns long and you
get 10ns/day for each job on its own. If you run:

Job 1 to completion followed by job 2 to completion. Time = 12 hours for job
1 + 12 for job 2 = 24 hours

However, if you run both together then you get about 10% performance for
each so Job 1 will get about 1ns/day, same for job 2. But they run
concurrently so the time needed is that for a single job = 5 days!

So you are ALWAYS better off running the jobs sequentially.
> Today I tried to put 2 jobs on one Tesla 2050 and when I wanted to
> launch
> the second job
> there it was crashed with this error message:
> -----------------------------------------------------------------------
> ------------------------
> cudaMemcpyToSymbol: SetSim copy to cSim failed all CUDA-capable devices
> are busy or unavailable
> -----------------------------------------------------------------------
> ------------------------
> From the memory point of view it should be OK (enough for both jobs).

It is possible NVIDIA changed the way things work in the driver meaning it
won't let you do this anymore. Or maybe it lets you but you have to look for
busy signals in your code and retry. This is something the AMBER code does
not have and is something that will NOT be added. So, my simple suggestion
is just to run 1 job at a time per GPU.

All the best

|\oss Walker

| Assistant Research Professor |
| San Diego Supercomputer Center |
| Adjunct Assistant Professor |
| Dept. of Chemistry and Biochemistry |
| University of California San Diego |
| NVIDIA Fellow |
| | |
| Tel: +1 858 822 0854 | EMail:- |

Note: Electronic Mail is not secure, has no guarantee of delivery, may not
be read every day, and should not be used for urgent or sensitive issues.

AMBER mailing list
Received on Mon Jun 06 2011 - 09:30:02 PDT
Custom Search