Hi Ross,
to run GPUs it is essential to put them in PCI-E X16 slots.
On a normal motherboard there are only 1 or 2 PCI-E X16 slots available, but
there are some special motherboards on the market with additional chip sets,
which supply more PCI-E X16 slots.
We did some benchmarks on our HP SL390 system.
This node is equipped with an additional chipset and supplies 4 PCI-E X16
slots, one for the onboard QDR Infiniband and the other to be used by 3 GPUs.
The performance degration was less than 2% when we ran 3 separated GPU jobs
and one non-GPU pmemd.MPI with 8 cores in parallel on this dual-CPU 6-core
system. In principal, it is a good idea, not to use all cores, normally
core-1 handles all interrupts.
On our system, the GPU utilization was only 90% but the CPU runs with 100%,
so using faster CPUs (with higher clock rate than our 2.67 GHz CPUs) may be
useful.
All the best
Peter
Dr. Peter Stauffert
Boehringer Ingelheim Pharma GmbH & Co. KG
-----Ursprüngliche Nachricht-----
Von: Ross Walker [mailto:ross.rosswalker.co.uk]
Gesendet: Dienstag, 13. September 2011 18:54
An: 'AMBER Mailing List'
Betreff: Re: [AMBER] Sufficient CPU cores/GPU ratio ?
Hi Marek,
I would say that the answer is, with caveats, not really. The reason for
this is not a core to GPU ratio argument per se but actually one of memory
bandwidth. For a single socket motherboard there is no way that it can drive
4 PCI-E sockets flat out at x16 speed. Hence you are always going to get
contention. Typically I would say never go beyond 2 GPUs per node for a
single socket system. 4 GPUs per node is pushing it for a dual socket system
even with the highest CPU FSB speed you can get so on a single socket system
with a low end CPU it is definitely not going to cut it.
Now if you only plan to run single GPU jobs. I.e. 4 independent jobs at once
with reasonably large values for ntpr, ntwx etc then it is probably not an
issue since the memory bandwidth only really comes into play at each output
file or trajectory write for most types of calculations. However, trying to
run in parallel where a single job spans GPUs it really all comes down to a
combination of the aggregate memory bandwidth of the main CPU memory per GPU
and the PCI-E bandwidth to each GPU when all are communicating flat out at
the same time. You might be able to get speedup across 2 of the GPUs if you
leave the other two idle. However, running two sets of dual GPU runs or
running a single 4 GPU run is likely to cause so much contention that see
little or no speedup.
So it really comes down to a case of the type of jobs you want to run.
I hope that helps.
All the best
Ross
> -----Original Message-----
> From: Marek Maly [mailto:marek.maly.ujep.cz]
> Sent: Tuesday, September 13, 2011 7:19 AM
> To: amber.ambermd.org
> Subject: [AMBER] Sufficient CPU cores/GPU ratio ?
>
> Hi all,
>
> just a very quick technical question. I would like to know if 4 core
> CPU
> "Intel Core i7-960" might be sufficient for
> 4 GPU machine (motherboard: Asus P6T7 WS SuperComputer - Intel X58 ).
> I
> mean if ratio 1 CPU core/ 1 GPU might
> be here sufficient also considering some requirements (on CPU) for
> managing operating system etc.
>
> Thanks in advance for any relevant comment !
>
> Best wishes,
>
> Marek
>
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Tue Sep 13 2011 - 11:00:05 PDT
