It's unsurprising that 8X cards would run slower--the thing simply can't
scale beyond two or four, and the optimizations we've introduced in 18 and
the upcoming patch may even make the scaling problem more severe. In time,
there may be a more general solution to the scaling problem on multiple
GPUs--there are tricks that I know how to implement, and some of the
further optimizations I am working on, if they pan out, will lend
themselves to better decomposition. However, I don't see these scaling
problem being taken on, by me at least, in the next year or two. For now,
NVLink is better than SLI or PCIE, but GTX-1080Ti will not support it (need
GP100 or Tesla cards in the Pascal line and newer, and even in the Volta
line Titan-V does not support NVLink).
Just run concurrent simulations, one per card. Best use of the resources
and hardly slower than anything you could get with 2x or 4x cards devoted
to one calculation.
Dave
On Sun, Apr 29, 2018 at 6:16 AM, 汪盛 <shengwang.hust.edu.cn> wrote:
> Dear,
>
> I ran a simulation with 8X GTX 1080Ti GPU card, but I found the speed of
> 8XGPU is much slower that just 1XGPU:
>
> With command: mpirun -np 8 pmemd.cuda.MPI ................, I found 8
> threads with top command in linux, but with nvidia-smi, I found:
>
> +-----------------------------------------------------------
> ------------------+
> | Processes: GPU
> Memory |
> | GPU PID Type Process name Usage
> |
> |===========================================================
> ==================|
> | 0 10792 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 445MiB |
> | 0 10793 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 0 10794 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 0 10795 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 1 10792 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 1 10793 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 445MiB |
> | 1 10794 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 1 10795 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 2 10792 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 2 10793 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 2 10794 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 475MiB |
> | 2 10795 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 3 10792 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 3 10793 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 3 10794 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 3 10795 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 475MiB |
> | 4 10796 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 475MiB |
> | 4 10797 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 4 10798 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 4 10799 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 5 10796 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 5 10797 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 475MiB |
> | 5 10798 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 5 10799 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 6 10796 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 6 10797 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 6 10798 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 475MiB |
> | 6 10799 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 7 10796 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 7 10797 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 7 10798 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 151MiB |
> | 7 10799 C /DATABAK/amber16/bin/pmemd.cuda.MPI
> 475MiB |
> +-----------------------------------------------------------
> ------------------+
>
> There are 32 threads for the same task. And for my test case it took
> almost 31 seconds for every 5000 steps. But if only one card was used, it
> just took just 6 seconds for every 5000 steps.
>
> I have installed ompenmpit 3.0 on CENTOS 6.9 64 bit, and the system is
> 7048 GR-TR. How about this problem?
>
>
>
>
> Thank you very much!
>
>
>
>
> Dr. Sheng Wang
> _______________________________________________
> AMBER mailing list
> AMBER.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Sun Apr 29 2018 - 06:00:06 PDT
