Hi All,
Yes the changes are simple.
1) Use CUDA 11.8 and a new enough driver.
2) Edit cmake/CudaConfig.cmake and ensure it allows CUDA 11.8 and includes SM89 (RTX4XXX) and SM90 (H100).
Note there are additional optimizations that could be made for SM89 and SM90 chips, size of kernel launch bounds etc but that will involve some effort.
On the RTX4090 front I had a chance to personally test one of these: https://www.tomshardware.com/news/rtx-4090-blower-aims-to-compete-with-quadros <https://www.tomshardware.com/news/rtx-4090-blower-aims-to-compete-with-quadros> and it works really well, only about 5% slower than the 3.5 slot monster. So these are probably the ideal option for anyone looking to put more than 2 in a box. You'll need 240V though to handle the power requirements of more than two so unlikely to be a good solution for a workstation but certainly good for 2U x 4GPU servers. Of course we'll see how long they stay around for before they get 'Banned By NVIDIA(tm)'.
All the best
Ross
> On Dec 5, 2022, at 09:33, Gould, Ian R via AMBER <amber.ambermd.org> wrote:
>
> Dear Dave and Jim,
>
> I don’t have access to an H100 but have got 8 4090’s and the required change to the configuration files are very limited, basically allowing for the detection and use of cuda11.8 compilers, plus you need the most up to date drivers on your machine, I am running 520.56.06.
>
> I ran Ross’s benchmarks and get the same numbers for the 4090’s give or take slight noise as you’d expect. They really fly on large system sizes our 1.5M atom system goes 2.2 faster than on 3090’s. Another observation is that they run significantly cooler than the 3090’s, this could be due to their large size, more surface area to get effective cooling over. Here is a screenshot of one of our machines with 2x 4090’s at full throttle.
>
>
> +-----------------------------------------------------------------------------+
>
> | NVIDIA-SMI 520.61.05 Driver Version: 520.61.05 CUDA Version: 11.8 |
>
> |-------------------------------+----------------------+----------------------+
>
> | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
>
> | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
>
> | | | MIG M. |
>
> |===============================+======================+======================|
>
> | 0 NVIDIA Graphics... Off | 00000000:05:00.0 Off | Off |
>
> | 39% 62C P2 306W / 450W | 1652MiB / 24564MiB | 83% Default |
>
> | | | N/A |
>
> +-------------------------------+----------------------+----------------------+
>
> | 1 NVIDIA Graphics... Off | 00000000:0A:00.0 Off | Off |
>
> | 30% 58C P2 305W / 450W | 1652MiB / 24564MiB | 84% Default |
>
> | | | N/A |
>
> +-------------------------------+----------------------+----------------------+
>
>
>
> +-----------------------------------------------------------------------------+
>
> | Processes: |
>
> | GPU GI CI PID Type Process name GPU Memory |
>
> | ID ID Usage |
>
> |=============================================================================|
>
> | 0 N/A N/A 2447 C ...amber/bin/pmemd.cuda_SPFP 1650MiB |
>
> | 1 N/A N/A 2823 C ...amber/bin/pmemd.cuda_SPFP 1650MiB |
>
> +-----------------------------------------------------------------------------+
>
> As Ross said in his original post on the 4090’s you can only sensibly get 2 of them in a case, they are large and take 3.5 slots plus you need 4 PCIE cables per card. The UK fortunately has 240 volt/ 13 Amp AC so these do not strain our normal sockets and power circuits.
>
> Cheers,
> Ian
>
> Mediocrity knows nothing higher than itself, but talent instantly recognizes genius.
> Sir Arthur Conan Doyle
> --
> Professor Ian R Gould, FRSC.
> Professor of Computational Chemical Biology
>
> MSRH
> Imperial College White City Campus
> 80 Wood Lane
> London W12 0BZ
>
> E-mail i.gould.imperial.ac.uk
> https://urldefense.com/v3/__http://www.imperial.ac.uk/people/i.gould__;!!Mih3wA!AcS3R5myffgx-RbS-zzutPsoNl0qnUZx7zU2ZWRZsIzaUBc-tE-4TXzLlp0t0MrlqyIg-NrDxUc$
> Tel +44 (0)207 594 5809
>
>
> From: David A Case via AMBER <amber.ambermd.org>
> Date: Monday, 5 December 2022 at 14:16
> To: jimkress_58.kressworks.org <jimkress_58.kressworks.org>, AMBER Mailing List <amber.ambermd.org>
> Subject: Re: [AMBER] H100 & RTX4090 Benchmarks
>
> *******************
> This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender.
> If you trust the sender, add them to your safe senders list https://urldefense.com/v3/__https://spam.ic.ac.uk/SpamConsole/Senders.aspx__;!!Mih3wA!AcS3R5myffgx-RbS-zzutPsoNl0qnUZx7zU2ZWRZsIzaUBc-tE-4TXzLlp0t0MrlqyIg79qdwF4$ to disable email stamping for this address.
> *******************
> On Sat, Dec 03, 2022, James Kress via AMBER wrote:
>
>> Is there a pricing comparison? If I put and H100 in my system will the
>> current incarnation of amber and amber tools support the H100?
>
> Pricing estimates were in the original email (below). The "managed to get
> Amber up and running" comment suggests that some changes were needed to the
> configuration scripts.
>
> ....dac
>
>> -----Original Message-----
>> From: Ross Walker via AMBER <amber.ambermd.org>
>> Subject: [AMBER] H100 & RTX4090 Benchmarks
>>
>> TLDNR: The 4090 (MSRP $1,599) consistently beats the H100 (MSRP ~$20,000) by
>> around 20% - so the advice for running AMBER is the same as it has been
>> since the birth of AMBER on GPUs - save your money and get 12 RTX4090s
>> instead of a single H100.
>
>
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> https://urldefense.com/v3/__http://lists.ambermd.org/mailman/listinfo/amber__;!!Mih3wA!AcS3R5myffgx-RbS-zzutPsoNl0qnUZx7zU2ZWRZsIzaUBc-tE-4TXzLlp0t0MrlqyIgN0v0Olw$
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> https://urldefense.com/v3/__http://lists.ambermd.org/mailman/listinfo/amber__;!!Mih3wA!AcS3R5myffgx-RbS-zzutPsoNl0qnUZx7zU2ZWRZsIzaUBc-tE-4TXzLlp0t0MrlqyIgN0v0Olw$
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Mon Dec 05 2022 - 07:30:02 PST
