Hi there,
Our scientists were excited to upgrade to Amber 14 because of the expected better performance of GPU calculations. We built ours using the Intel 14.0.3 compiler, CUDA 5.5, and are using MVAPICH2 2.0. Recently, one tried to run a job that he’d run with Amber 12 (which appears to have used CUDA 4.2, and the Intel 12.1 compiler). The machine has 4 NVIDIA Tesla K20m’s in it. The results are as follows:
"MD specifics
It is a system of 305,541 atoms - protein and water box.
We run it as a production run with NVT (constant volume) from step md1 to md2
Amber 12
4x1gpu cuda 6.67 ns/day - 99% GPU load (as reported by nvidia-smi)
2GPU cuda.MPI 7.67ns/day - 70-75% "
4GPU cuda.MPI 10.05 ns/day - 50-63% "
Amber 14
4x1gpu cuda 8.08 ns/day - 99% GPU load (as reported by nvidia-smi)
2GPU cuda.MPI 12.10 ns/day - 97-99% "
3GPU cuda.MPI 6.45 ns/day - 35-43% "
4GPU cuda.MPI 7.37 ns/day - 35-43% "
For amber 14 : 4x1gpu and 2GPU cuda.MPI runs with 97-99% GPU load while switching to 3 or more GPUs downgrades performance to 35-43%
In both 2GPU and 4GPU, but not 3GPU, nvidia-smi shows that jobs are split into 2 components as we saw before - one big job and another a 64MB job. It did not appear in 3GPU job and never for amber12."
We are seeing roughly linear scaling on Amber 12 but different behavior on 14. Can anyone explain what we might do to try to improve things, and why we see 8 compute processes in nvidia-smi when running a 2 or 4 GPU MPI job? I figure this might be design as we see one using 64MB RAM on each GPU and one using another variable amount in the hundreds of MiB.
Let me know if there’s any more info that would be helpful.
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
|| \\UTGERS |---------------------*O*---------------------
||_// Biomedical | Ryan Novosielski - Senior Technologist
|| \\ and Health | novosirj.rutgers.edu - 973/972.0922 (2x0922)
|| \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
`'
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Fri Dec 26 2014 - 13:30:02 PST
