Hector,
Thank so much for your response! I will definitely try the Ahlrichs pVDZ, TZV or DZV bases and I will report that! I am not confident enough on deleting the f functions from the basis set yet and I will need to read more to understand what the drawbacks could be.
Best regards,
Fernando
________________________________
From: Hector A. Baldoni <hbaldoni.unsl.edu.ar>
Sent: Thursday, March 7, 2024 5:47 PM
To: Montalvillo, Fernando <Fernando.Montalvillo.UTDallas.edu>
Subject: Re: [AMBER] Quick QM/MM fails with GPUs but not with CPUs
Hi Fernando,
My installation has Amber 22 SANDER+cuda QUICK-22.03 via API. but
unfortunately I don't have any of my own examples at hand with
TM+sander+quick.cuda.
You can try two things:
1) include in the quick base set the Ahlrichs pVDZ, TZV or DZV bases
from
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.basissetexchange.org%2F&data=05%7C02%7CFernando.Montalvillo%40utdallas.edu%7C266cbda051c044c0eb0708dc3f00ec85%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638454520412469155%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=jdT5tJ0F2%2FLS9YLS8nMhOtla9b7Js6Kih%2Fx1IzEnQeU%3D&reserved=0<
https://www.basissetexchange.org/> because they don't have F
functions.
2) Alternatively try a less conserved approach as follows:
a) from your path .../amber22/AmberTools/src/quick/basis
b) mv 6-31GDP.BAS 6-31GDP.BAS_Orig
c) vi 6-31GDP.BAS
d) delete the Cu F gaussian function:
F 1 1.00
0.8000000000D+00 1.0000000
(NOTE: This is a valid procedure when instabilities occur in the orbital
integration with Gaussian orbital exponentials < 0.5. It is not our case
but we are close.)
f) run the calculation conservatively, example:
/
&quick
keywords = "DFT BLYP D3 BASIS=6-31G** SCF=20 ALLOW_BAD_SCF
OPTIMIZE=10 GTOL=1.0E00 ETOL=1.0E00 COARSEIN ",
/
g) control the geometry of the coordination sphere. Cu(I) is a d10,
closed shell ion, and the most common coordination geometry is four. If
everything goes well, delete the extra keys.
h) let us know what happened
Greeting,
Hector.
El 2024-03-07 19:10, Montalvillo, Fernando escribió:
> Hi Hector thanks for your response,
>
> The problem with those basis sets is that they have f functions and
> that is not yet compatible with the current version of GPU-accelerated
> QUICK in Ambertools23. However, according to Dr. Goetz that capability
> will be released soon with Amber24. So, my dilemma here will be either
> doing Umbrella Sampling with another QM/MM interface in the meantime
> or wait till April and do TI with the new QUICK but I am little bit
> constrained with time since we are in the last steps of the project.
>
>
> Thanks,
> Fernando
>
> -------------------------
>
> From: Hector A. Baldoni <hbaldoni.unsl.edu.ar>
> Sent: Thursday, March 7, 2024 3:52 PM
> To: Montalvillo, Fernando <Fernando.Montalvillo.UTDallas.edu>; AMBER
> Mailing List <amber.ambermd.org>
> Subject: Re: [AMBER] Quick QM/MM fails with GPUs but not with CPUs
>
> Dear Fernando,
>
> Regarding, the LANL2DZ basis set, and assuming there are no
> significant
> objections, I recommend employing a homogeneous 6-31G* basis set at
> the
> Hartree-Fock (HF) level for the QM region. This choice is particularly
>
> compatible with the derivation of amber charges. Let me rmember you
> that
> Cu+ is an d10 ion, so charge=+1 and multiplicity=zero.
> In the event that the Cu+ coordination sphere remains undistorted and
> all aspects progress smoothly, I further suggest utilizing the B3LYP
> functional with a 6-31G* (or 6-31G**) basis set and incorporating the
> D3BJ dispersion correction. A cautious approach provides fewer
> complications.
>
> Greeting,
> Hector.
>
> El 2024-03-07 15:31, Montalvillo, Fernando via AMBER escribió:
>> Thanks for the help Andy and Madu,
>>
>> Regarding the mpi support,
>> sander.MPI can run when the system is all MM (but not with QM/MM)
>> pmemd.cuda.MPI can run (so I can do serial with GPUs)
>> sander.quick.cuda.MPI cannot run. (But the executable can be found
> and
>> run with one GPU)
>>
>> So, I think there might be 2 problems:
>> 1st I installed Amber20 with AmberTools23. Could that be an issue?
>> 2nd Maybe the installation I did of openmpi was not good, but I
> doubt
>> it since all the other mpi executables can run fine.
>>
>> If you think the MPI installation support is the problem, then I
> will
>> stop bothering you because that is something I will have to fix with
>> IT.
>>
>> Regarding, the LANL2DZ basis set,
>> Is there a resource you are aware of for selecting a good basis set
>> for TM ions? Or what should be the characteristics of a good basis
> set
>> for TM ions for quick? Do you recommend any basis set for TM ions
>> (Cu+) for quick?
>>
>> I am trying to calculate Absolute Free Binding Energy through TI, so
>> that is why sander.quick.cuda.MPI would be perfect!
>>
>> Best regards and many thanks,
>> Fernando
>>
>> ________________________________
>> From: Goetz, Andreas <awgoetz.ucsd.edu>
>> Sent: Thursday, March 7, 2024 4:31 AM
>> To: Montalvillo, Fernando <Fernando.Montalvillo.UTDallas.edu>
>> Cc: AMBER Mailing List <amber.ambermd.org>; Manathunga
> Mudiyanselage,
>> Madushanka <manathun.msu.edu>
>> Subject: Re: [AMBER] Quick QM/MM fails with GPUs but not with CPUs
>>
>> Hi Fernando,
>>
>> The LANL2DZ basis set is for valence electrons only. It should be
> used
>> together with an effective core potential (ECP) for the core
>> electrons. QUICK does not yet implement ECPs and should thus only be
>> used with all-electron basis sets. While the calculations will work
>> technically, the results will be questionable if you use LANL2DZ
>> without its ECP.
>>
>> All the best,
>> Andy
>>
>> —
>> Dr. Andreas W. Goetz
>> Associate Research Scientist
>> San Diego Supercomputer Center
>> Tel: +1-858-822-4771
>> Email: agoetz.sdsc.edu
>> Web:
> https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.awgoetz.de%2F&data=05%7C02%7CFernando.Montalvillo%40utdallas.edu%7C266cbda051c044c0eb0708dc3f00ec85%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638454520412480881%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=PnCYRUhI%2BmUNZQZHHbZ5cfCY4sD%2BHpqwK700V65RqVk%3D&reserved=0<http://www.awgoetz.de/>
> [1]
>>
>> On Mar 6, 2024, at 7:45 PM, Montalvillo, Fernando
>> <Fernando.Montalvillo.UTDallas.edu> wrote:
>>
>> Andy,
>>
>> Thanks for your response! Based on this information I decided to
>> upload the LANL2DZ basis set which has no f functions and it is one
> of
>> the most recommended for TM ions. Seems to be working so far! So
>> again, thank you!
>>
>> Also, I am not sure if this is a good question. But I don't know how
>> to run sander.quick.cuda.MPI with GPUs. It seems I can only request
>> CPUs. The HPC I uses SLURM manager.
>>
>> #SBATCH --ntasks=4 # Number of mpi tasks requested
>> #SBATCH --gres=gpu:4 # Number of gpus requested
>> I also requested the maximum amount of memory of the node, just in
>> case,
>>
>> That should request 4 GPUs and 4 CPUs and then,
>>
>> mpirun -np 4 sander.quick.cuda.MPI -O .... (rest of the options)
>>
>> I have run the sander.MPI before so I know I am exporting all the
>> librairies correctly, but it seems my job only runs on CPUs. Do you
>> know what I am doing wrong? Or could you let me know how you did it
>> for your publication with Vinicius W. Cruzeiro, et al.?
>>
>> #!/bin/bash
>>
>> #SBATCH --job-name=QM-LANL # Job name
>> #SBATCH --output=error.out
>> #SBATCH --nodes=1 # Total number of nodes requested
>> #SBATCH --ntasks=4 # Number of mpi tasks requested
>> #SBATCH --gres=gpu:4 # Number of gpus requested
>> #SBATCH -t 48:00:00 # Run time (hh:mm:ss) - 48 hours
>> #SBATCH --partition=torabifard
>> #SBATCH
>>
> --mail-user=fxm200013.utdallas.edu<mailto:mail-user=fxm200013.utdallas.edu>
>> #SBATCH --mail-type=all
>>
>> module load cuda
>> source
> /mfs/io/groups/torabifard/Amber20-mpi-Fernando/amber22/amber.sh
>> export
>>
> PATH=/mfs/io/groups/torabifard/Amber20-mpi-Fernando/amber22_src/bin:$PATH
>> export
>>
> LD_LIBRARY_PATH=/mfs/io/groups/torabifard/Amber20-mpi-Fernando/amber22_src/lib:$LD_LIBRARY_PATH
>>
>> mpirun -np 4 sander.quick.cuda.MPI -O -i 01_Min1.in -o 01_Min1.out
> -p
>> afCopAE1.prmtop -c Min0_lipids.rst -r Min1.rst -ref Min0_lipids.rst
>>
>> Best regards,
>> Fernando
>>
>> ________________________________
>> From: Goetz, Andreas <awgoetz.ucsd.edu<mailto:awgoetz.ucsd.edu>>
>> Sent: Tuesday, March 5, 2024 3:44 PM
>> To: Montalvillo, Fernando
>>
> <Fernando.Montalvillo.UTDallas.edu<mailto:Fernando.Montalvillo.UTDallas.edu>>;
>> AMBER Mailing List <amber.ambermd.org<mailto:amber.ambermd.org>>
>> Cc: Manathunga Mudiyanselage, Madushanka
>> <manathun.msu.edu<mailto:manathun.msu.edu>>
>> Subject: Re: [AMBER] Quick QM/MM fails with GPUs but not with CPUs
>>
>> Hi Fernando,
>>
>> The QUICK version that ships with AmberTools 23 does not support f
>> functions. 6-31G(d), cc-pVTZ and def2-SVPD basis sets contain f
>> functions for Cu. The error message that should be generated has
>> inadvertently been deactivated. You would have to use a basis set
> that
>> does not contain f functions.
>>
>> F functions will be supported with the next release, QUICK 24.03 and
>> AmberTools 24. Initially only for closed shells (e.g. Cu+ but not
>> Cu2+) on GPUs, closed and open shells on CPUs.
>>
>> All the best,
>> Andy
>>
>> —
>> Dr. Andreas W. Goetz
>> Associate Research Scientist
>> San Diego Supercomputer Center
>> Tel: +1-858-822-4771
>> Email: agoetz.sdsc.edu<mailto:agoetz.sdsc.edu>
>> Web:
>>
> https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.awgoetz.de%2F&data=05%7C02%7CFernando.Montalvillo%40utdallas.edu%7C266cbda051c044c0eb0708dc3f00ec85%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638454520412485955%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=iUOYeCedEgDFeXwJo4dA0j4WZX1SBj%2B6arcetGmiXeI%3D&reserved=0<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2F%2Fwww.awgoetz.de__%3B!!Mih3wA!CRnfR_hoJZ82u3Ibs6SdynPv7p9wiDOGyzYBwvwegTd91Y4UrR7sLvY2obyfPZxcoO4Ng63z6W6S5d151eTTw8MgJ6Y6lFVs%24&data=05%7C02%7CFernando.Montalvillo%40utdallas.edu%7C266cbda051c044c0eb0708dc3f00ec85%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638454520412490730%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=hre6DpOoah7C8istXCz1C9L15ppgvNT0xpfYXHlM0kc%3D&reserved=0><http://www.awgoetz.de/>
> [1]
>>
>> On Mar 5, 2024, at 8:58 PM, Montalvillo, Fernando via AMBER
>> <amber.ambermd.org<mailto:amber.ambermd.org>> wrote:
>>
>> Hi,
>>
>> I am trying to run some QM/MM calculations with Sander.quick.cuda,
> but
>> the energy minimization of the QM region is being troublesome.
>>
>> I did a MM minimization of the lipids and solvent molecules of my
>> system (restraining protein atoms and Cu ion) due to lipids always
>> having very bad clashes.
>>
>> The next minimization is already with QM/MM. I have used multiple
>> basis set with HF, B3LYP and O3LYP and the results are as follows:
>>
>>
>> Regardless of HF or the other DFT methods I have used, QM energy
>> explodes when using GPUs and higher accuracy basis sets suchs as
>> 6-31G(d), cc-pVDZ, def2-SVPD. But if I use lower accuracy methods
> such
>> as 6-31G or 3-21G, it can run on GPUs and energies look fine (QM
> atoms
>> don't seem to be manipulated during the minimization only MM atoms).
>>
>>
>> Using CPUs, I can run with the more accurate basis sets, but it is
>> slow, and the QM atoms also don't seem to be manipulated during the
>> minimization. So, when I use the restart file to continue next step
>> with GPUs (maybe heating or another minimization), the energies
>> explode again despite using the same method and basis set that had
>> been used in the minimization with CPUs.
>>
>> Can you point at what I am doing wrong or what should I check? This
> is
>> my first QM/MM simulation, so I don't know what I am doing.
>>
>> Thanks for your invaluable help and time.
>>
>> Fernando
>> _______________________________________________
>> AMBER mailing list
>> AMBER.ambermd.org<mailto:AMBER.ambermd.org>
>>
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2F%2Flists.ambermd.org%2Fmailman%2Flistinfo%2Famber__%3B!!Mih3wA!GOclVpA2BKKnXDQFyIBlRRVPGA4eH66BOYV4wji5yPhpoZlwiJtuaiiZmnya62nsM1iekNbT8PeTQw%24&data=05%7C02%7CFernando.Montalvillo%40utdallas.edu%7C266cbda051c044c0eb0708dc3f00ec85%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638454520412495603%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=rYL8cJoZIWa1y6TDk8dbc3Xhls%2Fz6ARz33AeHiSHbMA%3D&reserved=0<https://urldefense.com/v3/__http://lists.ambermd.org/mailman/listinfo/amber__;!!Mih3wA!GOclVpA2BKKnXDQFyIBlRRVPGA4eH66BOYV4wji5yPhpoZlwiJtuaiiZmnya62nsM1iekNbT8PeTQw$>
> [2]
>>
>> _______________________________________________
>> AMBER mailing list
>> AMBER.ambermd.org
>>
> https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.ambermd.org%2Fmailman%2Flistinfo%2Famber&data=05%7C02%7CFernando.Montalvillo%40utdallas.edu%7C266cbda051c044c0eb0708dc3f00ec85%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638454520412500686%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=rOD0fS%2BpzfNAmCxc1jQ9bT0cAN%2Fi0aVd7es3dNJLsHw%3D&reserved=0<http://lists.ambermd.org/mailman/listinfo/amber>
> [3]
>
> --
> --------------------------------------
> Dr. Hector A. Baldoni
> Profesor Titular (FQByF-UNSL)
> Investigador Adjunto (IMASL-CONICET)
> Area de Quimica General e Inorganica
> Universidad Nacional de San Luis
> Chacabuco 917 (D5700BWS)
> San Luis - Argentina
> hbaldoni at unsl dot edu dot ar
> Tel.:+54-(0)266-4520300 ext. 6157
> --------------------------------------
>
>
> Links:
> ------
> [1] https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.awgoetz.de%2F&data=05%7C02%7CFernando.Montalvillo%40utdallas.edu%7C266cbda051c044c0eb0708dc3f00ec85%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638454520412505659%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=Lc7m5U7LSBRb1uZTFQkiUOj%2F0OB32%2Fbx0RlQ5RzL2Rk%3D&reserved=0<http://www.awgoetz.de/>
> [2]
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2F%2Flists.ambermd.org%2Fmailman%2Flistinfo%2Famber__%3B!!Mih3wA!GOclVpA2BKKnXDQFyIBlRRVPGA4eH66BOYV4wji5yPhpoZlwiJtuaiiZmnya62nsM1iekNbT8PeTQw%24&data=05%7C02%7CFernando.Montalvillo%40utdallas.edu%7C266cbda051c044c0eb0708dc3f00ec85%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638454520412510520%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=4VejEjHZuxJc5Z6qMWi423PgNUktPiOZ%2FA05mFI3tVY%3D&reserved=0<https://urldefense.com/v3/__http://lists.ambermd.org/mailman/listinfo/amber__;!!Mih3wA!GOclVpA2BKKnXDQFyIBlRRVPGA4eH66BOYV4wji5yPhpoZlwiJtuaiiZmnya62nsM1iekNbT8PeTQw$>
> [3] https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.ambermd.org%2Fmailman%2Flistinfo%2Famber&data=05%7C02%7CFernando.Montalvillo%40utdallas.edu%7C266cbda051c044c0eb0708dc3f00ec85%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638454520412515484%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=h8n0p5%2BddYP8%2BNjwLmaFhpvJsFRnad3X1dHDDy782xM%3D&reserved=0<http://lists.ambermd.org/mailman/listinfo/amber>
--
--------------------------------------
Dr. Hector A. Baldoni
Profesor Titular (FQByF-UNSL)
Investigador Adjunto (IMASL-CONICET)
Area de Quimica General e Inorganica
Universidad Nacional de San Luis
Chacabuco 917 (D5700BWS)
San Luis - Argentina
hbaldoni at unsl dot edu dot ar
Tel.:+54-(0)266-4520300 ext. 6157
--------------------------------------
2024 - Año de la Defensa de la Vida, la Libertad y la Propiedad.
Decreto 55/2024 GN
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Fri Mar 08 2024 - 09:00:01 PST
