Hey Jason, I found the root cause and checked in a fix.
If I put a printf into any CUDA kernel, that's the end of its performance.
Printf is a hog. It's only useful for debugging purposes. The beginning
of this thread has links to the data to repro this behavior. It's weird
that we never saw this before.
Just initialializing PMEFloat dr does the trick. The bug is fixed.
On Thu, Jan 29, 2015 at 9:42 AM, Jason Swails <jason.swails.gmail.com>
wrote:
> On Thu, 2015-01-29 at 08:51 -0800, Scott Le Grand wrote:
> > I think I found the problem. There's something I can fix that will make
> > the manifestation disappear, but the underlying bad news is that I think
> > your Born Radius parameters are invalid and that's triggering an
> > uninitialized variable, the equivalent of a single threaded race
> condition
> > because God only knows the order of computation and what ends up in that
> > uninitialized variable.
> >
> > But the thing is that that variable should *never* see a situation where
> it
> > doesn't get a value. It doing so is a sign there's something wrong with
> > your Born Radii so I am somewhat tempted to leave the code as is because
> > there's no way I'm putting a printf warning in an innermost loop on the
> GPU.
> >
> > David, Ross et al., is there a valid situation in the Born Radius
> > calculation where dr should not get a value? Or should it *always* fall
> > into one of the clauses as I previosuly believed?
>
> Do any of the intrinsic RADII look bad? Do you know why none of the
> clauses in kCalculateGBBornRadii.h are being hit?
>
> Can you put the printf warning into the inner GB loop and see if that
> warning is triggered for any of the CUDA tests? I *think* it should
> always fall into one of the clauses you specify in that file, but I
> haven't looked closely enough to tell for certain. (btw -- would this be
> something along the line of adding a raw else clause ~line 171 of that
> file?)
>
> If you send me the prmtop, I can try and look to see if anything looks
> unusual.
>
> All the best,
> Jason
>
> --
> Jason M. Swails
> BioMaPS,
> Rutgers University
> Postdoctoral Researcher
>
>
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Thu Jan 29 2015 - 18:30:02 PST
