For a fat node, the plex chip may be enough to drive GPUDirect/P2P. I've
more or less given up on MPI 2.0 so that's the only step forward from
here. Currently verifying this with nvidia though before proceeding
On Jun 22, 2013 1:27 PM, "ET" <sketchfoot.gmail.com> wrote:
> Thanks for that Ross, I hadn't consider that.Some more food for thought.
>
>
> On 22 June 2013 21:12, Ross Walker <rosscwalker.gmail.com> wrote:
>
> > You need to be looking at DUAL socket motherboards if you want 4-way x16
> > support. A single CPU does not have enough bandwidth (pins) to support
> true
> > 4 by x16 - they cheat using pxii switches.
> >
> > All the Exxact machines on the Amber website are dual socket by quad slot
> > x16. Of course this makes them more expensive than the budget single
> socket
> > systems you can build for sub $1000 (exc GPU cost).
> >
> > All the best
> > Ross
> >
> >
> >
> > On Jun 22, 2013, at 12:45, ET <sketchfoot.gmail.com> wrote:
> >
> > > Looks like the Asus P9X79-E WS is for you then Scott! :) Haven't seen
> > many
> > > (if any!) boards with that anount of bandwidth so far.
> > >
> > > br,
> > > g
> > >
> > >
> > > On 22 June 2013 19:31, Scott Le Grand <varelse2005.gmail.com> wrote:
> > >
> > >> It may be overkill now but I'm planning to revisit the Multi GPU code
> in
> > >> the near future and that's why I need a motherboard that can really
> take
> > >> advantage of it.
> > >> On Jun 22, 2013 10:12 AM, "ET" <sketchfoot.gmail.com> wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> Thanks v much for you specs Divi. :) I've been debating with myself
> as
> > my
> > >>> board as it looks good and has a very nice spec. From what I've read,
> > the
> > >>> only problems with it is the higher than average power draw.
> > >>>
> > >>> .Scott. I believe the board runs in x8/x8/x8/x8 for a 4 GPU config -
> so
> > >>> effectively PCI2 2.0 x16 rate. Would this present any problems, if
> you
> > >>> were running the serial GPU code, From what I read on the AMBER GPU
> > >>> hardware page, this is more important for the parallel GPU code?
> > Though,
> > >> I
> > >>> imagine having 4x serial ruins going simultaneously would also tax
> the
> > >> GPU
> > >>> to CPU interface, though how much I'm not sure.
> > >>>
> > >>> Apparently, if you are going intel, you can only acheive PCIe 3.0
> using
> > >> at
> > >>> least a Sandy Bridge-E or ivy bridge CPU in a socket 155. Please
> > correct
> > >> me
> > >>> if I have understood this incorrectly though.
> > >>>
> > >>> http://www.enthusiastpc.net/articles/00003/3.aspx
> > >>>
> > >>>
> > >>> A socket 2011 proposition would be the Asus P9X79-E WES which has 2x
> > PLX
> > >>> PEX 8747 chips so can run at x16/x16/x16/x16 with four GPUs
> > >>>
> > >>> https://www.asus.com/Motherboards/P9X79E_WS/#specifications
> > >>>
> > >>> However, I'm unsure whether this is overkill for running 4xGPUs doing
> > >> AMBER
> > >>> serial code.
> > >>>
> > >>> What do you guys think?
> > >>>
> > >>> br,
> > >>> g
> > >>>
> > >>>
> > >>>
> > >>> On 22 June 2013 16:15, Scott Le Grand <varelse2005.gmail.com> wrote:
> > >>>
> > >>>> Does this MB support full p2p at 16x PCIE Gen 3 speeds between all 4
> > >>> GPUs?
> > >>>> On Jun 21, 2013 4:09 PM, "Divi/GMAIL" <dvenkatlu.gmail.com> wrote:
> > >>>>
> > >>>>>
> > >>>>> ET:
> > >>>>> I am using GA-Z77X-UP7 that has PLX chipset and supports 3rd Gen
> > >>>> LGA1155
> > >>>>> socket. Bought together with 2 TITANS sometime in March.
> > >>>>> It has been running pretty stable 24/7 since then. I thought of
> > >>> buying
> > >>>>> two more titans later to fill all four slots. With so much mess
> > >> going
> > >>> on
> > >>>>> with TITANS, I put off that plan until the dust settles. You might
> > >>> want
> > >>>> to
> > >>>>> check new 4th GEN cpus and supporting motherboards as the Hardware
> > >> keep
> > >>>>> changing pretty rapidly these days.
> > >>>>>
> > >>>>> I have i5-processor with 16 GB ram and 256 GB SSD. All four
> PCI-E
> > >>>> lanes
> > >>>>> are X-16. It also has native X-16 link directly "hardwired" to
> > >>> CPU-lanes
> > >>>>> that will bypass PLX chipset, in case if you run single GPU. This
> > >> might
> > >>>>> reduce a bit of latency but not much. I get 35ns/day on FIX/NVE
> > >>> benchmark
> > >>>>> bypassing PLX chipset, but get about 34ns/day using PLX chipset (on
> > >>> TITAN
> > >>>>> of
> > >>>>> course!!). Not a deal breaker..
> > >>>>>
> > >>>>> Link below:
> > >>>>>
> > >>>>> http://www.gigabyte.com/products/product-page.aspx?pid=4334#ov
> > >>>>>
> > >>>>> HTH
> > >>>>> Divi
> > >>>>>
> > >>>>> -----Original Message-----
> > >>>>> From: ET
> > >>>>> Sent: Thursday, June 20, 2013 8:18 PM
> > >>>>> To: AMBER Mailing List
> > >>>>> Subject: [AMBER] Anyone running machines with Quad GPU setups
> > >>>>>
> > >>>>> Hi all,
> > >>>>>
> > >>>>> I was looking at getting a new mobo to run a quad GPU system. I was
> > >>>>> wondering if anyone has done this. If you could post the model &
> make
> > >>> of:
> > >>>>>
> > >>>>> 1) motherboard
> > >>>>> 2) CPU
> > >>>>> 3) RAM
> > >>>>> 4) Case
> > >>>>> 5) The aggregate estimate of ns in simulation you have run on your
> > >>> setup
> > >>>>> without issue,
> > >>>>>
> > >>>>> I would be much obliged! :)
> > >>>>>
> > >>>>> br,
> > >>>>> g
> > >>>>> _______________________________________________
> > >>>>> AMBER mailing list
> > >>>>> AMBER.ambermd.org
> > >>>>> http://lists.ambermd.org/mailman/listinfo/amber
> > >>>>>
> > >>>>>
> > >>>>> _______________________________________________
> > >>>>> AMBER mailing list
> > >>>>> AMBER.ambermd.org
> > >>>>> http://lists.ambermd.org/mailman/listinfo/amber
> > >>>>>
> > >>>> _______________________________________________
> > >>>> AMBER mailing list
> > >>>> AMBER.ambermd.org
> > >>>> http://lists.ambermd.org/mailman/listinfo/amber
> > >>>>
> > >>> _______________________________________________
> > >>> AMBER mailing list
> > >>> AMBER.ambermd.org
> > >>> http://lists.ambermd.org/mailman/listinfo/amber
> > >>>
> > >> _______________________________________________
> > >> AMBER mailing list
> > >> AMBER.ambermd.org
> > >> http://lists.ambermd.org/mailman/listinfo/amber
> > >>
> > > _______________________________________________
> > > AMBER mailing list
> > > AMBER.ambermd.org
> > > http://lists.ambermd.org/mailman/listinfo/amber
> >
> > _______________________________________________
> > AMBER mailing list
> > AMBER.ambermd.org
> > http://lists.ambermd.org/mailman/listinfo/amber
> >
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Sat Jun 22 2013 - 14:00:02 PDT
