Sorry. Please ignore the previous email.
On 10/23/2014 05:03 PM, koushik wrote:
> On Hai's comment on "converting to netcdf file again using cpptraj
> seems very slow". Writing the trajectories directly to the current
> working directory slows down the writing process because all the
> programs are simultaneously accessing almost the same disk area. This
> is an even bigger problem if you are extracting temperature
> trajectories from multiple simulations, which is usually the case, at
> the same time.
>
> If the script we are running is I/O intensive, its a really good
> practice to copy the data to the storage space on local node, change
> to the local node directory, execute the script, and copy back the
> files. The following links explains this process.
>
> http://www3.imperial.ac.uk/bioinfsupport/help/cluster_usage/datastaging
> https://ncisf.org/training/userguide/tmpdir
> https://www.nersc.gov/users/computational-systems/edison/file-storage-and-i-o/
>
> The /cavern uses the Lustre file system. The link 3 above briefly
> summarizes scratch data handling on Lustre file system.
>
> To share my experience, I was writing the temperature trajectories
> from 56 REMD simulations on Laufer cluster and it took almost 10 hours
> for all the jobs to finish (this also slowed down the jobs of others).
> When I used the TMPDIR feature and ran the same number of scripts
> submitting all at the same time, all of them finished within 15 minutes.
>
> Another observation regarding Netcdf is that, if you use the "go" word
> on line 6 of the code below, the writing speed of netcdf is really
> slow. I have observed that cpptraj writes the Netcdf file twice if you
> use the "go" word. When I ran the script without the "go" keyword
> below, it runs fast. The "go" word is in a lot of analysis scripts on
> the wiki, hence, if you are using the "go" keyword in your scripts, it
> slows down the process of writing.
>
> TEMPLIST=`cat remd/temperatures.dat`
> for T in $TEMPLIST; do
> /cavern/koushik/amber2013-02-15/bin/cpptraj tc5b.top <<EOF
> trajin remd/trj/remd.trj.001 remdtraj remdtrajtemp $T
> trajout remd/temptraj/remd.Ttraj.$T netcdf
> *go*
>
> EOF
> done
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Thu Oct 23 2014 - 14:30:03 PDT
