Hi, Chris
You can use the patch here
https://github.com/pfreezer/AmberPatch
Actually it is not a problem of REMD or general with pressure coupling, it
is a problem within the NFE method when using GPUs.
But if you just use the T-REMD without any biasing forces, I believe this
problem can be avoided by minor changes to the codes.
You can start from CPUs and tell me if the results are good. If so I will
work on the GPUs.
Feng
On Fri, Dec 23, 2016 at 11:02 AM, Chris Neale <candrewn.gmail.com> wrote:
> Dear Feng.
>
> It would be great to have this. I am not sure which way I will proceed, but
> options are always good.
>
> Can you elaborate about problems with NPT on GPUs? I presume you're talking
> about a REMD-specific issue and not some possible problem in general with
> pressure coupling and GPUs...
>
> Thank you,
> Chris.
>
> On Thu, Dec 22, 2016 at 7:47 PM, Feng Pan <fpan3.ncsu.edu> wrote:
>
> > Hello, Chris
> >
> > This module now is not officially in pmemd, but I made a third-party
> patch
> > to put it in. If you want I can give you.
> >
> > For pmemd.cuda on GPUs, the NPT ensemble may not work well. I found there
> > is a problem when I stream forces
> > between CPUs and GPUs. I will look into it, it could be fixed but I am
> not
> > sure.
> >
> > Best
> > Feng
> >
> > On Fri, Dec 23, 2016 at 1:43 AM, Chris Neale <candrewn.gmail.com> wrote:
> >
> > > Thank you Feng!
> > >
> > > Am I correct that (a) this module is not compatible with pmemd and
> > > therefore that (b) it's going to be a lot slower on GPUs than running
> > > pmemd?
> > >
> > > Thanks again,
> > > Chris.
> > >
> > > On Thu, Dec 22, 2016 at 9:35 AM, Feng Pan <fpan3.ncsu.edu> wrote:
> > >
> > > > Hi, Chris
> > > >
> > > > You can try the &bbmd module for sander.MPI.
> > > >
> > > > The &bbmd is for ABMD with T-REMD and H-REMD. But you can run only
> > T-REMD
> > > > by setting the mode='ANALYSIS'. Basically the T-REMD here use
> > > > different code from the -rem=1 one, so it should work with NPT
> > ensemble.
> > > To
> > > > be honest, I don't know if the results will be valid but it is worth
> > > > trying.
> > > >
> > > > you can check the details of &bbmd here http://ambermd.org/doc12/nfe
> .
> > pdf
> > > > in
> > > > section 22.6.5
> > > >
> > > > Best
> > > > Feng Pan
> > > >
> > > > On Wed, Dec 21, 2016 at 1:09 AM, Neale, Christopher Andrew <
> > > > cneale.lanl.gov>
> > > > wrote:
> > > >
> > > > > Dear developers:
> > > > >
> > > > > I have finished including PV work in the T-REMD exchange criterion
> in
> > > > > amber16. I still have an open question about PBC changing too much
> > and
> > > > the
> > > > > fact that "The GPU code does not automatically reorganize grid
> > cells",
> > > > and
> > > > > I'll update this thread if any issues pop up. Note that I did not
> > > enable
> > > > > pressure coupling Hamiltonian exchange and I completely turned off
> 2D
> > > > REMD
> > > > > so that I didn't have to worry about passing the other variables
> > > around,
> > > > > but if you're interested in allowing pressure coupling with REMD
> then
> > > > this
> > > > > code probably represents a plausible start.
> > > > >
> > > > > I made modifications to the following files:
> > > > > src/pmemd/src/mdin_ctrl_dat.F90
> > > > > src/pmemd/src/runmd.F90
> > > > > src/pmemd/src/remd_exchg.F90
> > > > > src/pmemd/src/cuda/gpu.cpp
> > > > >
> > > > >
> > > > > $ diff amber16/src/pmemd/src/mdin_ctrl_dat.F90
> > > > > amber16_pressremdPROPER/src/pmemd/src/mdin_ctrl_dat.F90
> > > > > 2152,2157c2152,2157
> > > > > < if (ntp .gt. 0) then
> > > > > <
> > > > > < write(mdout, '(a,a)') error_hdr, 'REMD cannot be run with
> > ntp >
> > > > 0!'
> > > > > < inerr = 1
> > > > > <
> > > > > < end if
> > > > > ---
> > > > > > ! if (ntp .gt. 0) then
> > > > > > !
> > > > > > ! write(mdout, '(a,a)') error_hdr, 'REMD cannot be run with
> > ntp
> > > >
> > > > > 0!'
> > > > > > ! inerr = 1
> > > > > > !
> > > > > > ! end if
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > $ diff amber16/src/pmemd/src/runmd.F90
> amber16_pressremdPROPER/src/
> > > > > pmemd/src/runmd.F90
> > > > > 895c895,896
> > > > > < si(si_kin_ene) / fac(1),
> > > > > print_exch_data, &
> > > > > ---
> > > > > > si(si_kin_ene) / fac(1),
> > pres0, &
> > > > > > si(si_volume),
> > print_exch_data, &
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > $ diff amber16/src/pmemd/src/remd_exchg.F90
> > > amber16_pressremdPROPER/src/
> > > > > pmemd/src/remd_exchg.F90
> > > > > 68c68,69
> > > > > < use pmemd_lib_mod, only : strip
> > > > > ---
> > > > > > use pmemd_lib_mod, only : strip,mexit
> > > > > >
> > > > > 158,159c159,162
> > > > > < call temperature_exchange(atm_cnt, vel, remd_ptot, my_dim,
> > > > > remd_size, &
> > > > > < actual_temperature, .true.,
> mdloop)
> > > > > ---
> > > > > > write(mdout, '(/,a)') ' PV for 2D-REMD not implemented.
> > > Exiting.'
> > > > > > call mexit(mdout, 1)
> > > > > > !call temperature_exchange(atm_cnt, vel, remd_ptot, my_dim,
> > > > > remd_size, &
> > > > > > ! actual_temperature, .true.,
> > mdloop)
> > > > > 314a318
> > > > > > actual_pressure, actual_volume, &
> > > > > 330a335,336
> > > > > > double precision, intent(in) :: actual_pressure
> > > > > > double precision, intent(in) :: actual_volume
> > > > > 340a347,348
> > > > > > double precision :: real_pres
> > > > > > double precision :: real_vol
> > > > > 347c355
> > > > > < integer, parameter :: SIZE_EXCHANGE_DATA = 6 ! for mpi_gather
> > > > > ---
> > > > > > integer, parameter :: SIZE_EXCHANGE_DATA = 8 ! for mpi_gather
> > > > > 354a363
> > > > > > double precision :: pressurevolumedelta
> > > > > 392a402,403
> > > > > > my_exch_data%real_pres = actual_pressure
> > > > > > my_exch_data%real_vol = actual_volume
> > > > > 460a472,480
> > > > > > ! * 0.0602214 / 4184 converts units to kcal/mol
> > > > > > pressurevolumedelta = &
> > > > > > (((ONEKB / my_exch_data%temp0) * &
> > > > > > my_exch_data%real_pres) - &
> > > > > > ((ONEKB / exch_data_tbl(neighbor_rank+1)%temp0) * &
> > > > > > exch_data_tbl(neighbor_rank+1)%real_pres)) * &
> > > > > > (my_exch_data%real_vol - exch_data_tbl(neighbor_rank+1)
> > > > %real_vol)
> > > > > * &
> > > > > > 0.0602214 / 4184.0
> > > > > >
> > > > > 465c485,486
> > > > > < (my_exch_data%temp0 *
> exch_data_tbl(neighbor_rank+1)
> > > > > %temp0)
> > > > > ---
> > > > > > (my_exch_data%temp0 *
> exch_data_tbl(neighbor_rank+1)
> > > > %temp0)
> > > > > &
> > > > > > - pressurevolumedelta
> > > > > 485,487c506,515
> > > > > < write(mdout,'(a8,E16.6,a8,E16.6,a12,f10.2)') &
> > > > > < "Metrop= ",metrop," delta= ",delta," o_scaling=
> ", &
> > > > > < 1 / my_exch_data%scaling
> > > > > ---
> > > > > > write(mdout,'(a8,E16.6,a8,E16.
> 6,a8,E16.6,a5,E16.6,a5,E16.6,
> > > > > a5,E16.6,a5,E16.6,a5,E16.6,a5,E16.6,a12,f10.2)') &
> > > > > > "Metrop= ",metrop," delta= ",delta, &
> > > > > > " pvwrk= ", pressurevolumedelta, &
> > > > > > " T1= ", my_exch_data%temp0, &
> > > > > > " T2= ", exch_data_tbl(neighbor_rank+1)%temp0, &
> > > > > > " P1= ", my_exch_data%real_pres, &
> > > > > > " P2= ", exch_data_tbl(neighbor_rank+1)
> %real_pres,
> > &
> > > > > > " V1= ", my_exch_data%real_vol, &
> > > > > > " V2= ", exch_data_tbl(neighbor_rank+1)%real_vol,
> &
> > > > > > " o_scaling= ", 1 / my_exch_data%scaling
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > $ diff amber16/src/pmemd/src/cuda/gpu.cpp
> > amber16_pressremdPROPER/src/
> > > > > pmemd/src/cuda/gpu.cpp
> > > > > 6306,6325c6306,6325
> > > > > < if (skin <= 0.5)
> > > > > < {
> > > > > < printf("ERROR: Calculation halted.");
> > > > > < printf(" Periodic box dimensions have changed too much
> > from
> > > > > their initial values.\n");
> > > > > < printf(" Your system density has likely changed by a
> large
> > > > > amount, probably from\n");
> > > > > < printf(" starting the simulation from a structure a long
> > way
> > > > > from equilibrium.\n");
> > > > > < printf("\n");
> > > > > < printf(" [Although this error can also occur if the
> > > simulation
> > > > > has blown up for some reason]\n");
> > > > > < printf("\n");
> > > > > < printf(" The GPU code does not automatically reorganize
> > grid
> > > > > cells and thus you\n");
> > > > > < printf(" will need to restart the calculation from the
> > > > previous
> > > > > restart file.\n");
> > > > > < printf(" This will generate new grid cells and allow the
> > > > > calculation to continue.\n");
> > > > > < printf(" It may be necessary to repeat this restarting
> > > > multiple
> > > > > times if your system\n");
> > > > > < printf(" is a long way from an equilibrated
> density.\n");
> > > > > < printf("\n");
> > > > > < printf(" Alternatively you can run with the CPU code
> until
> > > the
> > > > > density has converged\n");
> > > > > < printf(" and then switch back to the GPU code.\n");
> > > > > < printf("\n");
> > > > > < exit(-1);
> > > > > < }
> > > > > ---
> > > > > > // if (skin <= 0.5)
> > > > > > // {
> > > > > > // printf("ERROR: Calculation halted.");
> > > > > > // printf(" Periodic box dimensions have changed too much
> > > from
> > > > > their initial values.\n");
> > > > > > // printf(" Your system density has likely changed by a
> > large
> > > > > amount, probably from\n");
> > > > > > // printf(" starting the simulation from a structure a
> long
> > > way
> > > > > from equilibrium.\n");
> > > > > > // printf("\n");
> > > > > > // printf(" [Although this error can also occur if the
> > > > > simulation has blown up for some reason]\n");
> > > > > > // printf("\n");
> > > > > > // printf(" The GPU code does not automatically
> reorganize
> > > grid
> > > > > cells and thus you\n");
> > > > > > // printf(" will need to restart the calculation from the
> > > > > previous restart file.\n");
> > > > > > // printf(" This will generate new grid cells and allow
> the
> > > > > calculation to continue.\n");
> > > > > > // printf(" It may be necessary to repeat this restarting
> > > > > multiple times if your system\n");
> > > > > > // printf(" is a long way from an equilibrated
> > density.\n");
> > > > > > // printf("\n");
> > > > > > // printf(" Alternatively you can run with the CPU code
> > until
> > > > > the density has converged\n");
> > > > > > // printf(" and then switch back to the GPU code.\n");
> > > > > > // printf("\n");
> > > > > > // exit(-1);
> > > > > > // }
> > > > > _______________________________________________
> > > > > AMBER mailing list
> > > > > AMBER.ambermd.org
> > > > > http://lists.ambermd.org/mailman/listinfo/amber
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Feng Pan
> > > > Ph.D. Candidate
> > > > North Carolina State University
> > > > Department of Physics
> > > > Email: fpan3.ncsu.edu
> > > > _______________________________________________
> > > > AMBER mailing list
> > > > AMBER.ambermd.org
> > > > http://lists.ambermd.org/mailman/listinfo/amber
> > > >
> > > _______________________________________________
> > > AMBER mailing list
> > > AMBER.ambermd.org
> > > http://lists.ambermd.org/mailman/listinfo/amber
> > >
> >
> >
> >
> > --
> > Feng Pan
> > Ph.D. Candidate
> > North Carolina State University
> > Department of Physics
> > Email: fpan3.ncsu.edu
> > _______________________________________________
> > AMBER mailing list
> > AMBER.ambermd.org
> > http://lists.ambermd.org/mailman/listinfo/amber
> >
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
--
Feng Pan
Ph.D. Candidate
North Carolina State University
Department of Physics
Email: fpan3.ncsu.edu
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Thu Dec 22 2016 - 19:30:03 PST
