Re: [AMBER] cudaMemcpy GpuBuffer::Download failed unspecified launch failure

From: Hailin Huang <hailin.huang.my.liu.edu>
Date: Wed, 17 Jul 2013 19:37:35 -0400

I put the files on my skydrive since they are too large to be sent through
email.

https://skydrive.live.com/#cid=2C2D4445A2C3B850&id=2C2D4445A2C3B850!131


On Wed, Jul 17, 2013 at 7:28 PM, Scott Le Grand <varelse2005.gmail.com>wrote:

> I guess Ross should gather the input files and file a bug...
>
>
>
> On Wed, Jul 17, 2013 at 4:22 PM, Hailin Huang <hailin.huang.my.liu.edu
> >wrote:
>
> > This is a GTX 690..
> >
> > |------------------- GPU DEVICE INFO --------------------
> > |
> > | CUDA Capable Devices Detected: 2
> > | CUDA Device ID in use: 1
> > | CUDA Device Name: GeForce GTX 690
> > | CUDA Device Global Mem Size: 2047 MB
> > | CUDA Device Num Multiprocessors: 8
> > | CUDA Device Core Freq: 1.02 GHz
> > |
> > |--------------------------------------------------------
> >
> >
> >
> > On Wed, Jul 17, 2013 at 7:19 PM, Scott Le Grand <varelse2005.gmail.com
> > >wrote:
> >
> > > This is a GTX 780 right?
> > >
> > >
> > > On Wed, Jul 17, 2013 at 4:01 PM, Hailin Huang <hailin.huang.my.liu.edu
> > > >wrote:
> > >
> > > > Hi Ross,
> > > >
> > > > Yes we are using the version 12.3 of the GPU code.
> > > >
> > > > |--------------------- INFORMATION ----------------------
> > > > | GPU (CUDA) Version of PMEMD in use: NVIDIA GPU IN USE.
> > > > | Version 12.3
> > > > |
> > > > | 04/24/2013
> > > > |
> > > > | Implementation by:
> > > > | Ross C. Walker (SDSC)
> > > > | Scott Le Grand (nVIDIA)
> > > > | Duncan Poole (nVIDIA)
> > > > |
> > > > | CAUTION: The CUDA code is currently experimental.
> > > > | You use it at your own risk. Be sure to
> > > > | check ALL results carefully.
> > > > |
> > > > | Precision model in use:
> > > > | [SPFP] - Mixed Single/Double/Fixed Point Precision.
> > > > | (Default)
> > > > |
> > > > |--------------------------------------------------------
> > > >
> > > > |----------------- CITATION INFORMATION -----------------
> > > > |
> > > > | When publishing work that utilized the CUDA version
> > > > | of AMBER, please cite the following in addition to
> > > > | the regular AMBER citations:
> > > > |
> > > > | - Romelia Salomon-Ferrer; Andreas W. Goetz; Duncan
> > > > | Poole; Scott Le Grand; Ross C. Walker "Routine
> > > > | microsecond molecular dynamics simulations with
> > > > | AMBER - Part II: Particle Mesh Ewald", J. Chem.
> > > > | Theory Comput., 2012, (In review).
> > > > |
> > > > | - Andreas W. Goetz; Mark J. Williamson; Dong Xu;
> > > > | Duncan Poole; Scott Le Grand; Ross C. Walker
> > > > | "Routine microsecond molecular dynamics simulations
> > > > | with AMBER - Part I: Generalized Born", J. Chem.
> > > > | Theory Comput., 2012, 8 (5), pp1542-1555.
> > > > |
> > > > | - Scott Le Grand; Andreas W. Goetz; Ross C. Walker
> > > > | "SPFP: Speed without compromise - a mixed precision
> > > > | model for GPU accelerated molecular dynamics
> > > > | simulations.", Comp. Phys. Comm., 2013, 184
> > > > | pp374-380, DOI: 10.1016/j.cpc.2012.09.022
> > > > |
> > > > |--------------------------------------------------------
> > > >
> > > >
> > > > I am sending the input files to your email.
> > > >
> > > > Thanks!
> > > > Richard
> > > >
> > > >
> > > > On Wed, Jul 17, 2013 at 5:53 PM, Ross Walker <ross.rosswalker.co.uk>
> > > > wrote:
> > > >
> > > > > Hi Richard,
> > > > >
> > > > > Can you confirm that you are running version 12.3 of the GPU code
> > > please.
> > > > > Check the mdout file where it writes info about the GPUs in use and
> > > > > citation information. This corresponds to AMBER 12 up to bugfix.18.
> > If
> > > > you
> > > > > have an earlier version please upgrade and this should fix the
> > problem.
> > > > If
> > > > > you still see the problem with the latest version of the code
> please
> > > post
> > > > > your input files for the run that takes the shortest time to crash
> > and
> > > > > we'll try and debug it.
> > > > >
> > > > > All the best
> > > > > Ross
> > > > >
> > > > >
> > > > >
> > > > > On 7/17/13 2:23 PM, "Hailin Huang" <hailin.huang.my.liu.edu>
> wrote:
> > > > >
> > > > > >Dear Amber Users,
> > > > > >
> > > > > >I am simulating 4 variations of the protein kinase A (PKA)
> tetramer
> > > > using
> > > > > >explicit solvent. One of them has 1 Mg ion between each R-C
> > subunits.
> > > > > >First
> > > > > >I did the minimization with 10 kcal/mol restraint weight on the
> > > > backbone.
> > > > > >Then, when I tried to run the heating simulation, the calculation
> > > > stopped
> > > > > >at step 0, without leaving any error messages in the output file.
> > > > > >
> > > > > >The end of the output file looks like this:
> > > > > >
> > > > > >NSTEP = 0 TIME(PS) = 0.000 TEMP(K) = 0.00
> > PRESS =
> > > > > >-3062.5
> > > > > > Etot = -409445.2805 EKtot = 0.0000 EPtot =
> > > > > >-409445.2805
> > > > > > BOND = 49371.8329 ANGLE = 1469.1848 DIHED =
> > > > > >10100.8048
> > > > > > 1-4 NB = 2093.2049 1-4 EEL = 24175.6479 VDWAALS =
> > > > > >97254.9978
> > > > > > EELEC = -593910.9535 EHBOND = 0.0000 RESTRAINT =
> > > > > >0.0000
> > > > > > EKCMT = 0.0000 VIRIAL = 70377.8192 VOLUME =
> > > > > >1064336.6826
> > > > > > Density =
> > > > > >0.8515
> > > > > >
> > > > >
> > > >
> > >
> >
> >--------------------------------------------------------------------------
> > > > > >----
> > > > > >
> > > > > > NMR restraints: Bond = 0.000 Angle = 0.000 Torsion =
> > > > 0.000
> > > > >
> > > >
> > >
> >
> >==========================================================================
> > > > > >=====
> > > > > >
> > > > > >
> > > > > >In the log file, I got the message "cudaMemcpy GpuBuffer::Download
> > > > failed
> > > > > >unspecified launch failure". This problem did not occur when I run
> > the
> > > > > >other 3 variations of PKA structures (one with 2 Mg ions, one with
> > no
> > > Mg
> > > > > >ions and one with no ADPs).
> > > > > >
> > > > > >I have tried to run the simulation using GTX 680, 690, 780, and on
> > the
> > > > > >Stampede supercomputers that use Tesla K20m, I got the same
> result.
> > > > > >
> > > > > >Then I changed the size of the solvent box (from 12A to 10A), the
> > > > problem
> > > > > >went away and the calculation completed successfully. However,
> same
> > > > > >problem
> > > > > >happened again in the later NVT equilibration simulation. This
> time
> > > the
> > > > > >calculation stopped in the middle:
> > > > > >
> > > > > >NSTEP = 1599000 TIME(PS) = 3598.000 TEMP(K) = 299.24
> > PRESS =
> > > > > >22.1
> > > > > > Etot = -432512.9372 EKtot = 90851.5391 EPtot =
> > > > > >-523364.4763
> > > > > > BOND = 3740.6596 ANGLE = 10689.2110 DIHED =
> > > > > >21485.9623
> > > > > > 1-4 NB = 4601.7696 1-4 EEL = 47865.1134 VDWAALS =
> > > > > >74622.8763
> > > > > > EELEC = -687021.9668 EHBOND = 0.0000 RESTRAINT =
> > > > > >651.8984
> > > > > > EAMBER (non-restraint) = -524016.3746
> > > > > > EKCMT = 38312.0157 VIRIAL = 37612.0350 VOLUME =
> > > > > >1465745.9675
> > > > > > Density =
> > > > > >1.0384
> > > > > >
> > > > >
> > > >
> > >
> >
> >--------------------------------------------------------------------------
> > > > > >----
> > > > > >
> > > > > >Same message in the log file "cudaMemcpy GpuBuffer::Download
> failed
> > > > > >unspecified launch failure".
> > > > > >
> > > > > >
> > > > > >When I was simulating 4 variations of the heterodimer of PKA (half
> > > size
> > > > of
> > > > > >the tetramer), same error occurred in the heating stage for the
> > > > structure
> > > > > >with no Mg ions and the structure with no ADPs, while the ones
> with
> > 1
> > > Mg
> > > > > >and 2 Mg ions are running just fine. So I changed the size of
> > solvent
> > > > box
> > > > > >from 10A to 8A and this got rid of the problem again. Now I am
> > running
> > > > the
> > > > > >NVT equilibration and "expecting" the calculations to stop in the
> > > middle
> > > > > >just like what happened to the tetramer with 1 Mg (hopefully not).
> > > > > >
> > > > > >
> > > > > >I would be really grateful if anyone could give me an idea what is
> > > going
> > > > > >on
> > > > > >and possibly with a solution to this annoying "cudaMemcpy
> > > > > >GpuBuffer::Download failed unspecified launch failure" problem.
> > > > > >
> > > > > >
> > > > > >Thank you!
> > > > > >
> > > > > >Best,
> > > > > >Richard
> > > > > >_______________________________________________
> > > > > >AMBER mailing list
> > > > > >AMBER.ambermd.org
> > > > > >http://lists.ambermd.org/mailman/listinfo/amber
> > > > >
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > AMBER mailing list
> > > > > AMBER.ambermd.org
> > > > > http://lists.ambermd.org/mailman/listinfo/amber
> > > > >
> > > > _______________________________________________
> > > > AMBER mailing list
> > > > AMBER.ambermd.org
> > > > http://lists.ambermd.org/mailman/listinfo/amber
> > > >
> > > _______________________________________________
> > > AMBER mailing list
> > > AMBER.ambermd.org
> > > http://lists.ambermd.org/mailman/listinfo/amber
> > >
> > _______________________________________________
> > AMBER mailing list
> > AMBER.ambermd.org
> > http://lists.ambermd.org/mailman/listinfo/amber
> >
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Wed Jul 17 2013 - 17:00:03 PDT
Custom Search