Hi Fernando,
I still believe that this is simply a statistical sampling issue. If you
are looking at the averages pmemd by default only calculates the average
every NTPR steps. So if you have a 100,000 step run with ntpr=1 you get
the average over 100,000 values while if you have ntpr=1000 you get the
average over only 1000 values. These two values will only converge in the
limit of nstlim -> infinity. This is compounded by the fact that when you
set ntpr=1 it calculates the energy on every step while when you have it
set to 1000 it calculates the energy every 1000 steps. This involves a
different number of mathematical operations so you will get different
rounding leading to divergence of the trajectory. This will be more
pronounced at low values of ntpr. I.e. for 1000 going to 10000 the 'extra'
calculations are carried out rarely while going from ntpr=1 to ntpr=10 you
change how often the 'extra' calculations by a lot so the rounding
difference will be more pronounced.
There are a number of things you can play with. If you want you can
request that the averages be calculated every step irrespective of ntpr.
This is done by using the keyword ene_avg_sampling in the cntrl name list.
This is related to the following code in pme_force.fpp
call update_time(pme_misc_timer)
if ((mod(irespa, ene_avg_sampling) .eq. 0) .or. (mod(irespa, ntpr) .eq.
0)) then
call gpu_pme_ene(ew_coeff, uc_volume, pot_ene, virial, ekcmt)
else
call gpu_pme_force(ew_coeff, uc_volume, virial, ekcmt)
end if
call update_time(nonbond_time)
In addition to other code. This will cause the code to calculate the
energies on every step and average them rather than every ntpr step. This
will however slow down the calculation, although arguably less than ntpr=1
since you don't have the extra i/o. It should also address issues with
divergence of the trajectory since it will force calls to the full energy
routine on every step irrespective of ntpr.
This is somewhat artificial though, both trajectories are correct since
Newton's equations of motion are, by definition, chaotic. Hence what you
see is, I believe, simply a statistical convergence issue.
Hope that helps.
All the best
Ross
On 11/27/12 3:46 AM, "Fernando Martín García" <fmgarcia.cbm.uam.es> wrote:
> Hi Ross,
>
> Sorry for the delay. What you saw have sense. Only one question about
> the suggestion. Amber is now running over a Fedora 13 and as I see in
> CUDA 4.2, it is available for fedora 14. Could I use that version or
> should I need to upgrade my Fedora version?. We are finding the same
> problem with a Tesla 2090 in CentOS, with CUDA 4.2
>
> Here there are two outputs only varying the ntpr value (1 and 20,
> respectively). These simulations where made with GB, but same case
> happens with explicit waters in ntp ensemble. They were made with cuda
> 4.2 and a CentOS 6.3. As I said, if you use, for ntpr 1000 and 10000, no
> differences are observed.
>
> A V E R A G E S O V E R 2000 S T E P S
> vs A V E R A G E S O V E R 100 S T E P S
>
>
> NSTEP = 2000 TIME(PS) = 5.500 TEMP(K) = 300.26 PRESS =
> 0.0 NSTEP = 2000 TIME(PS) = 5.500 TEMP(K) =
> 300.61 PRESS = 0.0
> Etot = -1292.3285 EKtot = 1849.9964 EPtot =
> -3142.3248 Etot = -1292.5560 EKtot = 1852.1738
> EPtot = -3144.7298
> BOND = 465.9291 ANGLE = 1462.0663 DIHED =
> 815.1164 BOND = 464.2868 ANGLE = 1460.7239 DIHED
> = 817.2981
> 1-4 NB = 525.3658 1-4 EEL = 7885.1834 VDWAALS =
> -1173.7162 1-4 NB = 525.6464 1-4 EEL = 7880.5082
> VDWAALS = -1174.9187
> EELEC = -483.0180 EGB = -12639.2514 RESTRAINT =
> 0.0000 EELEC = -486.9575 EGB = -12631.3171
> RESTRAINT = 0.0000
>
>
>--------------------------------------------------------------------------
>----
>
>
>--------------------------------------------------------------------------
>----
>
> Best regards.
>
> Fer
>
> On Fri, 23 Nov 2012 17:27:29 -0800, Ross Walker wrote:
>> Hi Fernando,
>>
>> It would help to see select excerpts from your output files in order
>> to be
>> able to understand exactly what you are describing here. Firstly why
>> the
>> GPU code is deterministic on identical hardware this is not
>> necessarily
>> true when changing the value of ntpr. The reason for this is that the
>> code
>> only calculates the energy when it is needed for printing. Normally
>> it
>> just calculates the gradients. Hence when you change ntpr you change
>> the
>> amount of work being done and this leads to different rounding
>> differences
>> that will lead to natural divergence of the MD trajectory. This is
>> perfectly fine since you should always be running sufficient length
>> runs
>> to converge the properties you are interested in.
>>
>> Thus what you are reporting is probably reasonable but it would need
>> a lot
>> more info to understand fully.
>>
>> Another note though. You mention CUDA 4.0. Currently AMBER 12 with
>> the
>> latest patches, including a number of bug fixes for GPU only supports
>> CUDA
>> 4.2. This suggests to me that you are running with unlatched code. I
>> would
>> suggest running configure to apply the latest patches and then
>> recompiling
>> from scratch with cuda 4.2.
>>
>> All the best
>> Ross
>>
>>
>>
>> On 11/23/12 3:14 AM, "Fernando Martín García" <fmgarcia.cbm.uam.es>
>> wrote:
>>
>>> Dear Amber users,
>>>
>>> I have a doubt about ntpr option and GPU (and maybe CPU). I usually
>>> run
>>> my simulations with ntpr =1000. But we have been trying with
>>> different
>>> values (1, 500, 1000 and 10000). The system is running on Amber 12,
>>> GPU
>>> c2070, cuda 4.0.
>>>
>>> We observed that not difference in energy values between ntpr = 1000
>>> and 10000, but they change with ntpr = 1 and 500 (between them and
>>> with
>>> the other values). I would like to know if it is caused because the
>>> frequency in writing the energies what provoke the changes in them
>>> (proportional to the frequency). I think I read something about that
>>> while energies are been printed, leap-frog algorithm continues with
>>> the
>>> calculation of the coordinates, what would change dt.
>>>
>>> Thanks
>>>
>>> Fernando
>>>--
>>> ==============================================
>>> Fernando Martín García
>>> Molecular Modelling Group - Lab 312.1
>>> Molecular Biology Center "Severo Ochoa"
>>> C/ NICOLáS CABRERA, 1.
>>> UAM University. Cantoblanco, 28049 Madrid. Spain.
>>> TEL: (+34) 91-196-4662 FAX: (+34) 91-196-4420
>>> Web: http://fertoledo.wordpress.com/
>>> ==============================================
>>>
>>>_______________________________________________
>>>AMBER mailing list
>>>AMBER.ambermd.org
>>>http://lists.ambermd.org/mailman/listinfo/amber
>>
>>
>>
>> _______________________________________________
>> AMBER mailing list
>> AMBER.ambermd.org
>> http://lists.ambermd.org/mailman/listinfo/amber
>
>--
> ==============================================
> Fernando Martín García
> Molecular Modelling Group - Lab 312.1
> Molecular Biology Center "Severo Ochoa"
> C/ NICOLáS CABRERA, 1.
> UAM University. Cantoblanco, 28049 Madrid. Spain.
> TEL: (+34) 91-196-4662 FAX: (+34) 91-196-4420
> Web: http://fertoledo.wordpress.com/
> ==============================================
>
>_______________________________________________
>AMBER mailing list
>AMBER.ambermd.org
>http://lists.ambermd.org/mailman/listinfo/amber
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Tue Nov 27 2012 - 11:00:02 PST
