BTW, seems to me that what Ilyas is reporting (Thanks !!!) and what
Scott and Ross found agree with each other, that the race condition is
triggered when some GB radii are not defined.
adrian
On 1/30/15 2:22 PM, Ilyas Yildirim wrote:
> Hi Carlos,
>
> I might have pinpointed the problem. Here is the current tests I've
> done and its results:
>
> 1. When I use amber14 and load my frcmod/lib files over ff12SB force
> field, and test AMBER14's pmemd.cuda_DPFP, I am getting exactly the
> same energies in individual runs. Here are the 5 results:
>
> -7.8062E+04
> -7.8062E+04
> -7.8062E+04
> -7.8062E+04
> -7.8062E+04
>
> PS: See prmtop.1 and inpcrd located at
> http://www-wales.ch.private.cam.ac.uk/ilyas/
>
> 2. When I use amber12 and load my frcmod/lib files over ff12SB force
> field, and test AMBER14's pmemd.cuda_DPFP, I am getting different
> energy results in individual runs. Here are the 5 results:
>
> -7.7839E+04
> -7.7833E+04
> -7.7841E+04
> -7.7835E+04
> -7.7833E+04
>
> PS: See prmtop.2 and inpcrd located at
> http://www-wales.ch.private.cam.ac.uk/ilyas/
>
> 3. When I use amber12 and load ONLY ff12SB force field, and test
> AMBER14's pmemd.cuda_DPFP, I am not getting different energy results
> in individual runs. Here are the 5 results:
>
> -7.9274E+04
> -7.9274E+04
> -7.9274E+04
> -7.9274E+04
> -7.9274E+04
>
> PS: See prmtop.3 and inpcrd located at
> http://www-wales.ch.private.cam.ac.uk/ilyas/
>
> 4. When I use amber14 and load ONLY ff12SB force field, and test
> AMBER14's
> pmemd.cuda_DPFP, I am not getting different energy results in
> individual runs. Here are the 5 results:
>
> -7.9274E+04
> -7.9274E+04
> -7.9274E+04
> -7.9274E+04
> -7.9274E+04
>
> PS: See prmtop.4 and inpcrd located at
> http://www-wales.ch.private.cam.ac.uk/ilyas/
>
> It seems that when new atom types are used in amber14 and
> prmtop/inpcrd files are created with amber14's Leap, pmemd.cuda_DPFP
> gives the same energies. When, however, the prmtop/inpcrd files are
> created with amber12's Leap, this is not the case. If ff12SB by itself
> was used to create prmtop/inpcrd files, the energies come out to be
> the same at each run.
>
> The difference between prmtop.1 and prmtop.2 are under the %FLAG RADII
> part. Here are a couple of lines for this difference:
>
> iy222.quartz:~$ diff prmtop.1 prmtop.2 | more
> 1c1
> < %VERSION VERSION_STAMP = V0001.000 DATE = 01/30/15 16:49:27
> ---
>> %VERSION VERSION_STAMP = V0001.000 DATE = 01/15/15 16:36:45
> 119991c119991
> < 1.55000000E+00 1.70000000E+00 1.20000000E+00 1.55000000E+00
> 1.70000000E+00
> ---
>> 1.55000000E+00 2.20000000E+00 1.20000000E+00 1.55000000E+00
> 1.70000000E+00
> 120004c120004
> < 1.70000000E+00 1.20000000E+00 1.70000000E+00 1.20000000E+00
> 1.70000000E+00
> ---
>> 2.20000000E+00 1.20000000E+00 1.70000000E+00 1.20000000E+00
> 1.70000000E+00
> 120010c120010
> < 1.70000000E+00 1.20000000E+00 1.55000000E+00 1.70000000E+00
> 1.70000000E+00
> ---
>> 2.20000000E+00 1.20000000E+00 1.55000000E+00 1.70000000E+00
> 1.70000000E+00
> 120016c120016
> < 1.20000000E+00 1.55000000E+00 1.70000000E+00 1.20000000E+00
> 1.55000000E+00
> ---
>> 1.20000000E+00 1.55000000E+00 2.20000000E+00 1.20000000E+00
> 1.55000000E+00
> 120029c120029
> .
> .
> .
>
> It seems that the RADII for the new atom types are set to 2.2 Ang
> while the old values are 1.7 Ang and this 2.2 Ang seems to be causing
> the problem. Maybe I need to specify these atom types somewhere in the
> code? It seems that unitio.c file, line # 6149 has Carbon atom radii
> set to 2.2. The new atom types I have are named as C1, C2, C3, and C4.
> Is there any simple way to make sure the correct radii are used for
> the new atom types without changing the code? Thanks.
>
> Best,
>
> PS: Jason, it seems that we uploaded the wrong prmtop files before. I
> re-uploaded them to the following address:
>
> http://www-wales.ch.private.cam.ac.uk/ilyas/
>
> Ilyas Yildirim, Ph.D.
> -----------------------------------------------------------
> = Department of Chemistry | University of Cambridge =
> = Lensfield Road (Room # 380) | Cambridge, UK CB2 1EW =
> = Ph.: +44-1223-336-353 | E-mail: iy222.cam.ac.uk =
> = Website: http://ilyasyildirim.wordpress.com =
> = ------------------------------------------------------- =
> = http://www.linkedin.com/in/yildirimilyas =
> = http://scholar.google.com/citations?user=O6RQCcwAAAAJ =
> -----------------------------------------------------------
>
>
> On Fri, 30 Jan 2015, Carlos Simmerling wrote:
>
>> Hi Ilyas,
>> Can you be more specific about what you mean when you say "new frcmod
>> /lib"
>> files? Also the selection of intrinsic Born radii may matter, as well as
>> the specific igb value (GB model). Are there specific sets of
>> frcmod/atom
>> type that don't mix well with particular intrinsic radii set and
>> specific
>> gb model?
>> Thanks
>> Carlos
>> On Jan 30, 2015 10:11 AM, "Ilyas Yildirim" <iy222.cam.ac.uk> wrote:
>>
>>> To MODERATOR: I've sent the same email today early morning with some
>>> attachments, but the size of the attachments were too big, which
>>> required
>>> the moderator to decide to post or not. I am re-sending the email
>>> without
>>> the attachments; thus, please discard and do not send the previous
>>> email to
>>> the list.
>>>
>>> Dear Jason, Scott, Dave, and Ross,
>>>
>>> I have been reading this thread since Rosie described the problem
>>> with the
>>> energies coming out different at each new calculation. Scott and
>>> Jason were
>>> talking about the problem arising due to the GB model, or the method
>>> it is
>>> being used by cuda. So, I provided Rosie with a new prmtop/inpcrd
>>> input set
>>> directly created with ff12SB force field. The new set seems to work
>>> fine.
>>> The previous one was created with new frcmod/lib files loaded over
>>> ff12SB.
>>> The new frcmod has new atom types in it, and it seems that those are
>>> the
>>> ones causing the problem (though it is not clear yet to me why
>>> energies are
>>> different at the end). You can check out the prmtop.notworking and
>>> prmtop.working from the following address:
>>>
>>> http://www-wales.ch.cam.ac.uk/rosie/new_input/
>>>
>>> While creating the prmtop.notworking file, I used the following leap
>>> script to create it.
>>>
>>> ---------- xleap.in -----------
>>> source leaprc.ff12SB
>>> addPdbAtomMap {
>>> { "OP1" "O1P" } { "OP2" "O2P" }
>>> { "HO5'" "H5T" } { "HO3'" "H3T" }
>>> { "H5'" "H5'1" } { "H5''" "H5'2" }
>>> { "H2'" "H2'1" } { "H2''" "H2'2" }
>>> }
>>> addAtomTypes {
>>> { "DH" "H" "sp3" }
>>> { "C1" "C" "sp2" }
>>> { "C2" "C" "sp2" }
>>> { "C3" "C" "sp2" }
>>> { "C4" "C" "sp2" }
>>> { "C5" "C" "sp2" }
>>> { "C6" "C" "sp2" }
>>> { "C7" "C" "sp2" }
>>> { "C8" "C" "sp2" }
>>> { "CI" "C" "sp3" }
>>> { "OZ" "O" "sp3" }
>>> }
>>> loadamberparams ./libraries/frcmod.parmCHI
>>> loadamberparams ./libraries/frcmod.ionsjc_tip3p
>>> loadoff ./libraries/dna.parm99chi.parmbsc.off
>>> loadoff ./libraries/ions08.lib
>>> set default PBradii mbondi2
>>> mol = loadpdb model.pdb
>>> saveAmberParm mol prmtop.notworking inpcrd
>>> quit
>>> ------------------------------------------------
>>>
>>> If someone wants to use a GB model on a modified system (or let's say a
>>> model with revised force field) I am not sure what else needs to be
>>> defined
>>> in the leap input file. In the cuda version of pmemd, is there a
>>> specific
>>> file that has info on the GB model which needs to be modified in
>>> order make
>>> sure that the new atom types are recognized by AMBER 14 DPFP runs
>>> (so that
>>> the GB model won't complain about it)? It seems that usage of new atoms
>>> types somehow creates this whole problem. Thanks.
>>>
>>> Best regards,
>>>
>>> Ilyas Yildirim, Ph.D.
>>> -----------------------------------------------------------
>>> = Department of Chemistry | University of Cambridge =
>>> = Lensfield Road (Room # 380) | Cambridge, UK CB2 1EW =
>>> = Ph.: +44-1223-336-353 | E-mail: iy222.cam.ac.uk =
>>> = Website: http://ilyasyildirim.wordpress.com =
>>> = ------------------------------------------------------- =
>>> = http://www.linkedin.com/in/yildirimilyas =
>>> = http://scholar.google.com/citations?user=O6RQCcwAAAAJ =
>>> -----------------------------------------------------------
>>>
>>>
>>> On Thu, 29 Jan 2015, Jason Swails wrote:
>>>
>>> On Thu, Jan 29, 2015 at 9:14 PM, Scott Le Grand
>>> <varelse2005.gmail.com>
>>>> wrote:
>>>>
>>>> Hey Jason, I found the root cause and checked in a fix.
>>>>>
>>>>> If I put a printf into any CUDA kernel, that's the end of its
>>>>> performance.
>>>>> Printf is a hog. It's only useful for debugging purposes.
>>>>>
>>>>
>>>>
>>>> That's what I was suggesting it be used for. Basically what I was
>>>> curious
>>>> about was whether or not any of the GB tests in our test suite hit the
>>>> code
>>>> path where dr didn't get initialized. If it did, that would seem
>>>> (to me)
>>>> to be clear evidence that that code path is not illegal/unusual (i.e.,
>>>> indicative of bad parameters). If that printf is never triggered
>>>> during
>>>> the test suite, the parameters may be unusual. I certainly wasn't
>>>> suggesting it be left in permanently. I'll take a look if I have
>>>> time.
>>>>
>>>>
>>>>
>>>> The beginning
>>>>> of this thread has links to the data to repro this behavior. It's
>>>>> weird
>>>>> that we never saw this before.
>>>>>
>>>>> Just initialializing PMEFloat dr does the trick. The bug is fixed.
>>>>>
>>>>>
>>>> I saw the commit log :).
>>>>
>>>> Thanks,
>>>> Jason
>>>>
>>>> --
>>>> Jason M. Swails
>>>> BioMaPS,
>>>> Rutgers University
>>>> Postdoctoral Researcher
>>>> _______________________________________________
>>>> AMBER mailing list
>>>> AMBER.ambermd.org
>>>> http://lists.ambermd.org/mailman/listinfo/amber
>>>
>>>
>>> _______________________________________________
>>> AMBER mailing list
>>> AMBER.ambermd.org
>>> http://lists.ambermd.org/mailman/listinfo/amber
>>>
>>>
>> _______________________________________________
>> AMBER mailing list
>> AMBER.ambermd.org
>> http://lists.ambermd.org/mailman/listinfo/amber
>
>
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
--
Dr. Adrian E. Roitberg
Professor.
Department of Chemistry
University of Florida
roitberg.ufl.edu
352-392-6972
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Fri Jan 30 2015 - 11:30:03 PST
