Hi Ross, Jason and all,
Thanks for your answers and advices!
I interest what are the GTX580 benchmark test values? Could someone provide some numbers?
Thanks in advance!
I saw that previously it was mentioned 28.7ns/day for JAC, but NPT, NVE?
Regards,
Filip
--- On Sun, 4/10/11, Ross Walker <ross.rosswalker.co.uk> wrote:
> From: Ross Walker <ross.rosswalker.co.uk>
> Subject: Re: [AMBER] Amber CUDA calcualtion on GeForce GTX 590 ?
> To: "'AMBER Mailing List'" <amber.ambermd.org>
> Date: Sunday, April 10, 2011, 7:29 AM
> > Hi Ross,
> > Is it possible to run pmemd.cuda.MPI with only 3
> cards?
>
> Not with the current release version. You have to use a
> power of 2 GPUs. This is being addressed for the next
> release and then you will be able to use any number of GPUs.
> For the moment though this code is not publically available
> I am afraid.
>
> > I tried to do that but received this error:
> > application called MPI_Abort(MPI_COMM_WORLD, 1) -
> process 0
> > APPLICATION TERMINATED WITH THE EXIT STRING: Hangup
> (signal 1)
> >
> > The problem is that I observed today that the 4th GPU
> of the second
> > GTX590 in fact worse the results or give only up to 5%
> contribution.
> > For instance, the scaling between the docking program
> Hex and Amber is
> > almost the same -1.48x between 2GPU's of GTX590. When
> I used the 3th
> > GPUs in Hex the scale is fine 1.87x compared to 1GPU
> (Also similar to
> > ACEMD results -1 to 3 cards gives nearly 2x speed up).
> However, the 4th
> > GPUs worse the calculations in Hex and I suppose also
> in Amber will be
> > the same. The results in NAMD2.8b1 are similar and the
> 4th GPU gives
> > only 5% contribution.
> > Thus if I am able to use only 3GPUs in Amber this will
> give me
> > significant advantage with the current drivers.
>
> I actually think the GTX590 is doing a remarkable job here.
> What is amazing is the scaling you see for 2 GPUs on the
> same card. Considering they are sharing a single interface
> this is very good. I don't think you will get much better
> than that. Your best bet is to try running two x 2 GPU runs,
> making sure you put the threads on the correct GPUs. You
> might be able to run 2 jobs at once without too much slow
> down.
>
> Unfortunately scaling between the GPUs is always going to
> suck and there really is no magic bullet fix for this. It is
> just the laws of physics getting in the way unfortunately
> and I suspect one will have to wait for PCI-E Gen 3 to make
> an appearance before things get any better.
>
> Sorry I can't help much more than that but I would try to
> smile because a single GTX590 with 2 GPUs is doing a lot
> better than I would have suspected.
>
> > I'd like also to ask you about your opinion what could
> be the reason
> > for such bad/terrible scaling? I mean 1.3-4x is
> somehow ok, but
>
> 1.3x to 1.4x is awesome scaling in my opinion giving the
> woefully slow interconnect between the GPUs. They are
> sharing a single PCI-E 16x channel so even seeing some
> scaling is amazing.
>
> > 1.1x..Only driver problem or I am missing something?
>
> Where is the 1.1x coming from? On 4 GPUs? I would give up
> trying to use all 4 and just run 2 jobs by 2 GPUs each. Note
> unless you have a dual core board with some kick butt X
> series processors in there I doubt both your PCI-E slots are
> running at full speed anyway. Note normally one would have 4
> x x16 slots with 4 GPUs in. Here you have 2 x x16 slots with
> 4 GPUs in. So each GPU is on an 8x slot. Then when you try
> to run all 4 in parallel all hell breaks loose. 3 GPUs
> probably 'just' works because one of those GPUs gets an x16
> slot to itself.
>
> All the best
> Ross
>
> /\
> \/
> |\oss Walker
>
> ---------------------------------------------------------
> |
> Assistant Research Professor
> |
> | San Diego
> Supercomputer Center
> |
> |
> Adjunct Assistant Professor
> |
> | Dept. of Chemistry
> and Biochemistry
> |
> | University of
> California San Diego
> |
> |
> NVIDIA Fellow
>
> |
> | http://www.rosswalker.co.uk | http://www.wmd-lab.org/ |
> | Tel: +1 858 822 0854 | EMail:- ross.rosswalker.co.uk
> |
> ---------------------------------------------------------
>
> Note: Electronic Mail is not secure, has no guarantee of
> delivery, may not be read every day, and should not be used
> for urgent or sensitive issues.
>
>
>
>
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Mon Apr 11 2011 - 22:30:02 PDT
