- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]

From: bishop <bishop.tulane.edu>

Date: Wed, 21 Jan 2004 09:14:45 -0600

What Bob said is true. And what Andreas said it true.

But the question remains how does one determine significance of the

differences?

This all depends on what you are looking for: hardware, software or

system (model) dependencies in your results.

1) Differences due to numerical computing (order of operations type

issues) should be in the 5th or 6th decimal place and affected by

physical nature of the machine, compiler, etc....

(amber is single precision right?).

These are significant if you are of a hardware mindset

2) Differences due to numerical methods, e.g the integration step size,

PME considerations, etc... all have an effect on the fluctuations of

solutions. These fluctuations may be equal to or significantly greater

than numerical precision.

These are significant if you are of a software/methods mindset.

3) Differences due to initial assignment of velocities (random number

seed issues) should be on the order the kinetic energy fluctuations

observed during run time in your system. This has to be at least as

great as the fluctuations due to numerical methods, but should vary with

(Square root ?) of the number of particles in the system. The

dependence of the fluctuation on the nature of the system is what makes

these fluctuations different from 2).

These are significant to the biologists.

I expect differences in 2) to be greater than differences in 1) but it

depends on your choice of simulation parameters. I expect 3) > 2) but

it depends on the details of your system.

If your "significant result" is biological shouldn't it depend on 3) and

not 2) or 1)?

If the significant result only depends on 3) then you can run multiple

simulations or longer simulations to determine a probability of it

happening as a measure of significance. You can also perturb your result

by changing not only the temperature but charge assignments, bond

strenghts, etc... (as in FEP calculation) to find the underlying source

for the event.

Differences due to 1) and 2) are often ignored and simply replaced by a

rule of thumb (like 2fs timestep with shake) but only if 1) and 2) are

excluded does 3) carry any significance. Granted isolating each of

these is no small undertaking.

Just my $.03, which is admittedly $.01 more than warranted.

Please add/subtract/agree/disagree with me.

TOm

Robert Duke wrote:

*> It should also be noted that same executable + same random seed + same
*

*> architecture WON'T produce the same results on parallel runs (ie., under
*

*> mpi). This has to do with order of operations issues in the networking, at
*

*> a minimum, and loadbalancing issues, where it is used in programs like
*

*> pmemd.
*

*> Regards - Bob
*

*> ----- Original Message -----
*

*> From: "Andreas Svrcek-Seiler" <svrci.tbi.univie.ac.at>
*

*> To: <amber.scripps.edu>
*

*> Sent: Wednesday, January 21, 2004 5:40 AM
*

*> Subject: Re: AMBER: Do I need to repeat MD simulation?
*

*>
*

*>
*

*>
*

*>>Hi,
*

*>>
*

*>>> A general question.Do I need to repeat my simulation several times to
*

*>>>prove the changes of protein I observed significant, not some artifical
*

*>>>things?
*

*>>>
*

*>>...Definitely, if the change is "irreversible".
*

*>>If the changes are "reversible", like repeated folding and unfolding
*

*>>(that would be a cool observation)
*

*>>or repeated conversions between well defined distinct conformations,
*

*>>you might simply continue the existing trajectory for a much longer time.
*

*>>Generally, don't accept the outcome of MD simulations as proof for
*

*>>anything (or be *very* cautious at least).
*

*>>
*

*>>
*

*>>
*

*>>>And will two trajectory files be identical if I run twice with
*

*>>>identical input files?
*

*>>>
*

*>>...Same executable + same random seed + same architecture: yes.
*

*>>Executable compiled with different compiler options (or another
*

*>>compiler): trajectories might slowly diverge from each other.
*

*>>Different random seeds (i.e. identical coordinates but not completely
*

*>>identical input files): No.
*

*>>However, relevant results must be independent of the random seed.
*

*>>
*

*>>...just my 0.02%
*

*>>regards
*

*>>Andreas
*

*>>
*

*>> )))))
*

*>> (((((
*

*>> ( O O )
*

*>>-------oOOO--(_)--OOOo----------------------------------------------------
*

*>>
*

*> -
*

*>
*

*>> o Wolfgang Andreas Svrcek-Seiler
*

*>> o (godzilla)
*

*>> svrci.tbi.univie.ac.at
*

*>> .oooO Tel.:01-4277-52733
*

*>> ( ) Oooo.
*

*>>-------\
*

*>>
*

*> ---( )--------------------------------------------------------
*

*>
*

*>> \_) ) /
*

*>> (_/
*

*>>
*

*>>
*

*>>
*

*>>-----------------------------------------------------------------------
*

*>>The AMBER Mail Reflector
*

*>>To post, send mail to amber.scripps.edu
*

*>>To unsubscribe, send "unsubscribe amber" to majordomo.scripps.edu
*

*>>
*

*>>
*

*>>
*

*>
*

*>
*

*>
*

*> -----------------------------------------------------------------------
*

*> The AMBER Mail Reflector
*

*> To post, send mail to amber.scripps.edu
*

*> To unsubscribe, send "unsubscribe amber" to majordomo.scripps.edu
*

*>
*

Date: Wed, 21 Jan 2004 09:14:45 -0600

What Bob said is true. And what Andreas said it true.

But the question remains how does one determine significance of the

differences?

This all depends on what you are looking for: hardware, software or

system (model) dependencies in your results.

1) Differences due to numerical computing (order of operations type

issues) should be in the 5th or 6th decimal place and affected by

physical nature of the machine, compiler, etc....

(amber is single precision right?).

These are significant if you are of a hardware mindset

2) Differences due to numerical methods, e.g the integration step size,

PME considerations, etc... all have an effect on the fluctuations of

solutions. These fluctuations may be equal to or significantly greater

than numerical precision.

These are significant if you are of a software/methods mindset.

3) Differences due to initial assignment of velocities (random number

seed issues) should be on the order the kinetic energy fluctuations

observed during run time in your system. This has to be at least as

great as the fluctuations due to numerical methods, but should vary with

(Square root ?) of the number of particles in the system. The

dependence of the fluctuation on the nature of the system is what makes

these fluctuations different from 2).

These are significant to the biologists.

I expect differences in 2) to be greater than differences in 1) but it

depends on your choice of simulation parameters. I expect 3) > 2) but

it depends on the details of your system.

If your "significant result" is biological shouldn't it depend on 3) and

not 2) or 1)?

If the significant result only depends on 3) then you can run multiple

simulations or longer simulations to determine a probability of it

happening as a measure of significance. You can also perturb your result

by changing not only the temperature but charge assignments, bond

strenghts, etc... (as in FEP calculation) to find the underlying source

for the event.

Differences due to 1) and 2) are often ignored and simply replaced by a

rule of thumb (like 2fs timestep with shake) but only if 1) and 2) are

excluded does 3) carry any significance. Granted isolating each of

these is no small undertaking.

Just my $.03, which is admittedly $.01 more than warranted.

Please add/subtract/agree/disagree with me.

TOm

Robert Duke wrote:

-- http://dna.cbr.tulane.edu Center for Bioenvironmental Research Tulane and Xavier Universities 1430 Tulane Avenue SL-3 New Orleans, LA 70112 Phone: 504-988-6203 Fax: 504-585-6428 bishop.tulane.edu ----------------------------------------- Alternate Contact: Center for Computational Sciences Voice: (504) 862-8391 ----------------------------------------- ----------------------------------------------------------------------- The AMBER Mail Reflector To post, send mail to amber.scripps.edu To unsubscribe, send "unsubscribe amber" to majordomo.scripps.eduReceived on Wed Jan 21 2004 - 15:53:01 PST

Custom Search