Just to clarify on this thread, since I think it's gotten confused (or at
the very least disorganized)... There are two errors that were being
discussed here. The first occurred when not all of the bug fixes were
applied (specifically bugfix.12 that fixes issues with cut != 8). In this
instance, many fields in the mdout file were filled with NaN. This is
specific to GPUs.
The other error, which is GPU-independent, is a shortcoming of a formatted
ASCII file as has been described in thorough detail numerous times in the
past (and in this post). This error occurs when various programs fail to
read and process a restart file (i.e. sander/pmemd when restarting a
simulation, VMD when trying to visualize, etc.) and is caused by
************s filling one or more of the fields. Because this occurs after
long periods of simulation (predictably), and GPUs enable longer time-scale
simulations, this effect has been erroneously attributed to GPUs on
occasion.
To fix the first one: be sure that all Amber 11 bug fixes are successfully
applied. Also beware that consumer video cards designed for gaming (GTX
series) are designed specifically for that -- gaming. If the card messes up
a calculation here or there, it's barely noticed during a game. In a
simulation of a chaotic system it can be catastrophic. The cards were not
intended to sustain the load that MD simulations subject them to for days on
end, and the new Fermi cards run very hot. This is ESPECIALLY true for
overclocked cards and some cheaper cards that tend to cut corners.
To fix the second one: set iwrap=1 in your input file, at least until a
NetCDF restart file is implemented in Amber (I second Ross's support!)
Hope this helps someone,
Jason
On Tue, Jan 25, 2011 at 12:45 AM, Ross Walker <ross.rosswalker.co.uk> wrote:
> Hi Peker,
>
> The iwrap option should NOT affect the energy or forces, it is simply a
> visualization option in that it wraps water molecules back into the central
> box as they translate out rather than storing the unimaged coordinates.
> Before people could routinely run long simulations it did not matter either
> way. Now people are routinely running 50ns+ simulations one sees the
> situation where a water molecule can translate sufficient box lengths away
> that its coordinates exceed the ASCII space defined in the restart file.
> Hence iwrap becomes a MUST for long simulations.
>
> You can safely turn it on in the middle of a set of simulations and it will
> not cause any issues. Just when you come to visualize you will suddenly see
> all the water jump back to the central box. But then again you probably
> want
> to reimage, center and strip water using ptraj before visualizing long
> trajectories anyway.
>
> Please turn on iwrap=1 and see if the problem goes away.
>
> I cast my vote for those listening for netcdf based restart files which
> will
> solve this problem more elegantly.
>
> If the problem persists with iwrap=1 set then we should try to investigate
> further.
>
> All the best
> Ross
>
> > -----Original Message-----
> > From: peker milas [mailto:pekermilas.gmail.com]
> > Sent: Monday, January 24, 2011 9:30 PM
> > To: AMBER Mailing List
> > Subject: Re: [AMBER] NaN error in .rst files
> >
> > Than you Ross for your quick reply.
> >
> > First thing to say, i am not using this flag. But as far as i know in
> > the manual it says that flag/option doesn't has no effect on either
> > force or energy calculations. That is why i didn't use it. I can turn
> > it on and check if same problem appears. But at the same time, i am
> > just in the middle of a long calculation, so then another question
> > appears in my mind. If i continue with iwrap flag/option, should i
> > expect any important difference in the simulation results ? You
> > definitely know much more than me about amber, can you tell me if it
> > is safe ?
> >
> > best
> > peker
> >
> > On Mon, Jan 24, 2011 at 7:54 PM, Ross Walker <ross.rosswalker.co.uk>
> > wrote:
> > > Hi Peter,
> > >
> > > Before you check into this any further can I just confirm that you have
> > > iwrap=1 set in the &cntrl namelist. If not then this is the problem and
> it
> > > is a pmemd / sander issue, completely unrelated to cuda. It will, in
> > > principal go away if we move to netcdf restart files.
> > >
> > > If you can confirm this only occurs with PMEMD.cuda and you have
> iwrap=1
> > set
> > > then it warrants investigating and the next step will be to see if
> anyone
> > > has seen this problem on any of the Tesla cards or it is confined to
> GTX
> > > series cards.
> > >
> > > All the best
> > > Ross
> > >
> > >> -----Original Message-----
> > >> From: peker milas [mailto:pekermilas.gmail.com]
> > >> Sent: Monday, January 24, 2011 2:42 PM
> > >> To: AMBER Mailing List
> > >> Subject: [AMBER] NaN error in .rst files
> > >>
> > >> Dear Amber coders/users
> > >>
> > >> We had a discussion about this problem couple days ago. Briefly at
> > >> some point of simulations pmemd.cuda creates NaN s in the restart
> > >> files. Then the next step of simulation messes up. This is sort of a
> > >> problem which happens in totally random way. As some users already
> > >> pointed out, part of it can be related with heating problem because
> > >> GTX 470/480 cards were not designed for this kind of intense
> > >> calculations. Although i totally agree with this, i couldn't find an
> > >> easy way of observing this behavior. In fact there is only nvidia-smi
> > >> command which shows me the current state of the card (like its
> > >> temperature, etc...). So i really want to go little bit deep with this
> > >> problem and i would like to know if anybody knows a good indicator for
> > >> checking cards behavior in detail. It can be a command or any sort
> > >> additional package/program for monitoring it.
> > >>
> > >> Any help will be appreciated
> > >>
> > >> regards
> > >> peker milas
> > >>
> > >> _______________________________________________
> > >> AMBER mailing list
> > >> AMBER.ambermd.org
> > >> http://lists.ambermd.org/mailman/listinfo/amber
> > >
> > >
> > > _______________________________________________
> > > AMBER mailing list
> > > AMBER.ambermd.org
> > > http://lists.ambermd.org/mailman/listinfo/amber
> > >
> >
> > _______________________________________________
> > AMBER mailing list
> > AMBER.ambermd.org
> > http://lists.ambermd.org/mailman/listinfo/amber
>
>
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
--
Jason M. Swails
Quantum Theory Project,
University of Florida
Ph.D. Graduate Student
352-392-4032
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Mon Jan 24 2011 - 22:00:09 PST
