Thanks for the quick reply, these benches are much better!
However that leads me to another question: I can't find any information on SPXP... is it `dangerous` to use SPXP results? Also what is SPXP? single precision eXtreme point?
Parker
________________________________________
From: Scott Le Grand [varelse2005.gmail.com]
Sent: Sunday, November 23, 2014 7:32 PM
To: AMBER Mailing List
Subject: Re: [AMBER] 4x GTX 980
Also you need to compile to SPXP (not the default SPFP)...
We're working on this... This is a real curveball they threw us this
round. The worst-case scenario is stabilizing SPXP and leaving SPFP as
is...
On Sun, Nov 23, 2014 at 4:31 PM, Scott Le Grand <varelse2005.gmail.com>
wrote:
> Turn off SLI...
>
>
> On Sun, Nov 23, 2014 at 4:22 PM, Parker de Waal <Parker.deWaal.vai.org>
> wrote:
>
>> Hi Scott,
>>
>> After putting together the machine and running the benchmark suite I've
>> noticed that my performance is significantly worse than your numbers.
>> Specifically, while running 2x cards in SLI mode the FactorIX-NVE I'm only
>> getting:
>>
>> | Average timings for last 1000 steps:
>> | Elapsed(s) = 5.19 Per Step(ms) = 5.19
>> | ns/day = 33.28 seconds/ns = 2596.11
>>
>> Interestingly I get the almost the same number when running with a single
>> gpu.
>>
>> I'm wondering if I set this up correctly as I've never built an SLI
>> machine before. However, I have noticed that having SLI on ($
>> nvidia-xconfig --sli=on) disables gpup2p (according the gpup2p checker).
>>
>> Are there settings you are supposed to specify to get that kind of
>> performance?
>>
>> Best,
>> Parker
>> ________________________________________
>> From: Scott Le Grand [varelse2005.gmail.com]
>> Sent: Monday, November 17, 2014 5:42 PM
>> To: AMBER Mailing List
>> Subject: Re: [AMBER] 4x GTX 980
>>
>> GTX 980 is well on its way to provide blowout AMBER numbers despite the
>> claims of its hopeful competitors (Really, will they ever learn?
>> Nah...)...
>>
>> It didn't do so out the door because the performance a of key HW
>> instruction was crippled relative to Kepler (llrintf).
>>
>> In the near-term future, either NVIDIA will address the above or we will
>> adopt a workaround to stabilize SPXP and make it the preferred operational
>> mode for GTX9xx cards. In fact, if you're using a thermostat, I *think*
>> you can run JAC NVE in SPXP today at ~185 ns/day out of the box. This is
>> of course the *correct* mode for directly comparing performance of
>> unstable
>> crappy MD code, but at least our crappy unstable mode is deterministic.
>>
>> Scott
>>
>>
>>
>>
>>
>> On Sun, Nov 16, 2014 at 8:16 PM, James Maier <jimbo.maier.gmail.com>
>> wrote:
>>
>> > **Please note that all experiences I've cited are for GTX 780s.
>> >
>> > On Sun, Nov 16, 2014 at 11:13 PM, James Maier <jimbo.maier.gmail.com>
>> > wrote:
>> >
>> > > Hi Parker,
>> > >
>> > > For the case, you can't go wrong with a HAF or almost anything with a
>> fan
>> > > on the side. We went with the "Thermaltake Overseer" for our cluster;
>> > it's
>> > > a bit cheaper than the HAF and has a built-in SATA dock. It's a pretty
>> > neat
>> > > case.
>> > > Not that it matters, but the blue lights look exquisite against the
>> green
>> > > GTX logo.
>> > >
>> > >
>> http://scanmail.trustwave.com/?c=129&d=wvzy1IeuIEIHX-mZ248yeHqXVVxmdEgC71HNbs9p4A&u=http%3a%2f%2fwww%2enewegg%2ecom%2fProduct%2fProduct%2easpx%3fItem%3dN82E16811133194
>> > >
>> > > We actually moved the fan that's pre-installed on the front to the
>> side,
>> > > and put this "bgears b-Blaster" fan in the front instead:
>> > >
>> > >
>> http://scanmail.trustwave.com/?c=129&d=wvzy1IeuIEIHX-mZ248yeHqXVVxmdEgC71XIas1l4A&u=http%3a%2f%2fwww%2enewegg%2ecom%2fProduct%2fProduct%2easpx%3fitem%3dN82E16835132022
>> > >
>> > > 120 mm, 103 CFM, not too expensive. We've not had any problems with
>> them
>> > > so far after about 9 months.
>> > >
>> > > As for the GPU, we've found the reference blower design to be more
>> > > effective when trying to squeeze four GPUs right next to each other. I
>> > > believe the blower style can generate higher (negative) pressure or
>> > > something, because the ACX, while great on its own, seems to do a
>> > terrible
>> > > job when something's directly occluding it. Our ACX GPUs were also
>> higher
>> > > clocked though, so that's an important caveat to anything I wish I
>> could
>> > > tell you more definitively.
>> > >
>> > > But don't take my word for it; consult some BitCoin mining
>> forums---they
>> > > seem to know all about this stuff. I believe a forum like that is
>> where I
>> > > first read something about the reference style being better for >2
>> GPUs,
>> > > which seems to match our experience.
>> > >
>> > > As Scott has cautioned, "superclocked is a 'super' way to get
>> inaccurate
>> > > results." Sticking with a normal clockspeed will likely save
>> headaches,
>> > > esp. as the 980s are a new design.
>> > >
>> > > The other concern is power; you need a power supply with a rail with
>> > > enough wattage for all cards or with separate rails that all can
>> drive a
>> > > card. I'm not sure if there is a single PSU that can "technically" (on
>> > > paper, following all stats to the letter) drive four GTX780s (require
>> 42
>> > > Amps each, I've only been able to find +12v with 133.3 Amps, plus
>> about
>> > 5 V
>> > > through each PCI slot). If someone knows of an adequate PSU, please
>> > correct
>> > > me.
>> > >
>> > > In that regard, the GTX 980s look promising due to their lower power
>> > draw.
>> > > There are only 2 issues I can think of atm:
>> > >
>> > > (1) 980s are "provisionally" supported by AMBER. This may not be an
>> > issue,
>> > > and according to
>> http://scanmail.trustwave.com/?c=129&d=wvzy1IeuIEIHX-mZ248yeHqXVVxmdEgC7wPBa500tw&u=http%3a%2f%2fambermd%2eorg%2fgpus
>> there's an optimization coming.
>> > > (2) 980s are immature and thus the silicon is not likely as stable as,
>> > > say, a 780 or a Titan.
>> > >
>> > > I'm guessing Ross, Scott, or anyone who has gotten their hands on some
>> > > 980s can chime in more about the above points.
>> > >
>> > > HTH,
>> > >
>> > > James
>> > >
>> > > On Sun, Nov 16, 2014 at 9:28 PM, Parker de Waal <
>> Parker.deWaal.vai.org>
>> > > wrote:
>> > >
>> > >> Hi Everyone,
>> > >>
>> > >> I apologize for the off topic post, however I'm currently building a
>> 4x
>> > >> GTX 980 machine and was wondering if anyone had experience or
>> insight on
>> > >> picking a proper case/fans. I'm worried that 4 cards will lead to
>> > >> overheating in most traditional cases.
>> > >>
>> > >> Additionally I'm wondering about the different fans on the GTX 980
>> > cards,
>> > >> specifically the reference fan blower vs. the ACX 2.0 fans on the
>> EVGA
>> > >> cards.
>> > >>
>> > >> Any insight would be extremely valuable.
>> > >>
>> > >> Best,
>> > >> Parker
>> > >> _______________________________________________
>> > >> AMBER mailing list
>> > >> AMBER.ambermd.org
>> > >>
>> http://scanmail.trustwave.com/?c=129&d=wvzy1IeuIEIHX-mZ248yeHqXVVxmdEgC71LJbcA35w&u=http%3a%2f%2flists%2eambermd%2eorg%2fmailman%2flistinfo%2famber
>> > >>
>> > >
>> > >
>> > _______________________________________________
>> > AMBER mailing list
>> > AMBER.ambermd.org
>> >
>> http://scanmail.trustwave.com/?c=129&d=wvzy1IeuIEIHX-mZ248yeHqXVVxmdEgC71LJbcA35w&u=http%3a%2f%2flists%2eambermd%2eorg%2fmailman%2flistinfo%2famber
>> >
>> _______________________________________________
>> AMBER mailing list
>> AMBER.ambermd.org
>>
>> http://scanmail.trustwave.com/?c=129&d=wvzy1IeuIEIHX-mZ248yeHqXVVxmdEgC71LJbcA35w&u=http%3a%2f%2flists%2eambermd%2eorg%2fmailman%2flistinfo%2famber
>> _______________________________________________
>> AMBER mailing list
>> AMBER.ambermd.org
>> http://scanmail.trustwave.com/?c=129&d=wvzy1IeuIEIHX-mZ248yeHqXVVxmdEgC71LJbcA35w&u=http%3a%2f%2flists%2eambermd%2eorg%2fmailman%2flistinfo%2famber
>>
>
>
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://scanmail.trustwave.com/?c=129&d=wvzy1IeuIEIHX-mZ248yeHqXVVxmdEgC71LJbcA35w&u=http%3a%2f%2flists%2eambermd%2eorg%2fmailman%2flistinfo%2famber
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Sun Nov 23 2014 - 17:00:03 PST
