Hi Bob,
I don't know what to say ...
Today I made a test to investigate SANDER/PMEMD scaling on our cluster
after reinstaling SANDER (using ifort 11) and with old compilation of
PMEMD (10.1.012)
which just uses new cc/MKL dynamic libraries.
I have obtained this results:
SANDER
1node 197,79 s
2nodes 186,47 s
4nodes 218 s
PMEMD
1node 145 s
2nodes 85 s
4nodes 422 s
1 node = 2 x Xeon Quad-core 5365 (3,00 GHz ) = 8 CPU cores
Molecular test system has 60 000 atoms (explicit wat) with cut-off 10A,
1000 time steps.
As you can see regarding to SANDER scaling is bad and is really wasting of
CPUs to use more than 1 node per job.
Regarding to PMEMD situation is different. As you see there are two pretty
strange opposite extreems.
Extreemly nice result for 2 nodes and extreemly bad for 4 nodes. I
repeated and check twice ...
All tests was performed using the same 4 empty nodes.
This behaviour is pretty strange but if I use for whole job = SANDER
(minimisation + heating + density), PMEMD (NPT equil, NVT production)
2 nodes I could have pretty fine results at least from the time point of
view. But there is another issue regarding to reliability
of computed data, not just here but also later when I will compute binding
energ. MM_PBSA/NAB.
So I do prefer your suggestion to recompile whole Amber (including PMEMD)
with ifort v. 10.1.021 as you suggested.
Could you please advise me where is possible to download this old version
(if possible with relevant MKL and cc libraries) ?
Thank you very much in advance !
Best,
Marek
Dne Fri, 08 May 2009 02:11:25 +0200 Robert Duke <rduke.email.unc.edu>
napsal/-a:
> Marek,
> There has been a whole other thread running about how ifort 11, various
> versions, will hang if you try to use it to compile pmemd (actual mails
> on the reflector right around yours...). I have recommended using ifort
> 10.1.021 because I know it works fine. As far as ifort 11.*, I have no
> experience, but there are reports of it hanging (this is a compiler bug
> - the compiler is defective). I also have coworkers that have tried to
> build gaussian 03 with ifort 11.*, and it compiles, but the executables
> don't pass tests. I think German Ecklenberg (I am guessing at the name
> - I unfortunately cleaned up some mail and the may amber reflector
> postings are not available yet) did get some version of 11 to work
> (might have been 11.0.084, but we are dealing with a very dim
> recollection here), but I would still prefer to just trust 10.1.021...
> Boy, you are getting to hit all the speed bumps... These days I would
> not trust any software intel releases for about 6 months after it is
> released - let other guys do the bleeding on the bleeding edge... Ross
> concurs with me on this one.
> Best Regards - Bob
> ----- Original Message ----- From: "Marek Malý" <maly.sci.ujep.cz>
> To: "AMBER Mailing List" <amber.ambermd.org>
> Sent: Thursday, May 07, 2009 7:59 PM
> Subject: Re: [AMBER] Error in PMEMD run
>
>
> Dear Ross and Bob,
>
> first of all thank you very much for your time and effort
> which really brought good result however some problem
> is still present ...
>
> OK,
>
>
> Our admin installed today ifort version 11 including corresponding cc,
> MKL.
>
> Here is actual LD_LIBRARY_PATH :
>
> LD_LIBRARY_PATH=/opt/intel/impi/3.2.0.011/lib64:/opt/intel/mkl/10.1.0.015/lib/em64t:/opt/intel/cc/11.0.074/lib/intel64:/opt/intel/fc/11.0.074/lib/intel64::/opt/intel/impi/3.2/lib64
>
> Of course first of all I tried just to compile pmemd with this new
> settings but I didn't succeeded :((
>
> Here is my configuration statement:
>
> ./configure linux_em64t ifort intelmpi
>
>
> Compilation started fine but after some time it "stopped" it means just
> progress stopped but not
> the compilation process it means after cca 1 hour the process is still
> alive see this part
> of the "top" list:
>
> 30599 mmaly 20 0 61496 16m 7012 R 50 0.1 58:51.64 fortcom
>
>
> But almost whole hour it was got jammed here:
>
> .....
>
> runmin.f90(465): (col. 11) remark: LOOP WAS VECTORIZED.
> runmin.f90(482): (col. 3) remark: LOOP WAS VECTORIZED.
> runmin.f90(486): (col. 3) remark: LOOP WAS VECTORIZED.
> /lib/cpp -traditional -P -I/opt/intel/impi/3.2/include -DPUBFFT
> -DBINTRAJ
> -DMPI -DDIRFRC_EFS -DDIRFRC_COMTRANS -DDIRFRC_NOVEC -DMKL
> -DFFTLOADBAL_2PROC veclib.fpp veclib.f90
> ifort -c -auto -tpp7 -xP -ip -O3 veclib.f90
> ifort: command line remark #10148: option '-tp' not supported
> gcc -c pmemd_clib.c
> /lib/cpp -traditional -P -I/opt/intel/impi/3.2/include -DPUBFFT
> -DBINTRAJ
> -DMPI -DDIRFRC_EFS -DDIRFRC_COMTRANS -DDIRFRC_NOVEC -DMKL
> -DFFTLOADBAL_2PROC gb_alltasks_setup.fpp gb_alltasks_setup.f90
> ifort -c -auto -tpp7 -xP -ip -O3 gb_alltasks_setup.f90
> ifort: command line remark #10148: option '-tp' not supported
> /lib/cpp -traditional -P -I/opt/intel/impi/3.2/include -DPUBFFT
> -DBINTRAJ
> -DMPI -DDIRFRC_EFS -DDIRFRC_COMTRANS -DDIRFRC_NOVEC -DMKL
> -DFFTLOADBAL_2PROC pme_alltasks_setup.fpp pme_alltasks_setup.f90
> ifort -c -auto -tpp7 -xP -ip -O3 pme_alltasks_setup.f90
> ifort: command line remark #10148: option '-tp' not supported
> /lib/cpp -traditional -P -I/opt/intel/impi/3.2/include -DPUBFFT
> -DBINTRAJ
> -DMPI -DDIRFRC_EFS -DDIRFRC_COMTRANS -DDIRFRC_NOVEC -DMKL
> -DFFTLOADBAL_2PROC pme_setup.fpp pme_setup.f90
> ifort -c -auto -tpp7 -xP -ip -O3 pme_setup.f90
> ifort: command line remark #10148: option '-tp' not supported
> pme_setup.f90(145): (col. 17) remark: LOOP WAS VECTORIZED.
> pme_setup.f90(159): (col. 22) remark: LOOP WAS VECTORIZED.
> pme_setup.f90(80): (col. 8) remark: LOOP WAS VECTORIZED.
> pme_setup.f90(80): (col. 8) remark: LOOP WAS VECTORIZED.
> /lib/cpp -traditional -P -I/opt/intel/impi/3.2/include -DPUBFFT
> -DBINTRAJ
> -DMPI -DDIRFRC_EFS -DDIRFRC_COMTRANS -DDIRFRC_NOVEC -DMKL
> -DFFTLOADBAL_2PROC get_cmdline.fpp get_cmdline.f90
> ifort -c -auto -tpp7 -xP -ip -O3 get_cmdline.f90
> ifort: command line remark #10148: option '-tp' not supported
> /lib/cpp -traditional -P -I/opt/intel/impi/3.2/include -DPUBFFT
> -DBINTRAJ
> -DMPI -DDIRFRC_EFS -DDIRFRC_COMTRANS -DDIRFRC_NOVEC -DMKL
> -DFFTLOADBAL_2PROC master_setup.fpp master_setup.f90
> ifort -c -auto -tpp7 -xP -ip -O3 master_setup.f90
> ifort: command line remark #10148: option '-tp' not supported
> /lib/cpp -traditional -P -I/opt/intel/impi/3.2/include -DPUBFFT
> -DBINTRAJ
> -DMPI -DDIRFRC_EFS -DDIRFRC_COMTRANS -DDIRFRC_NOVEC -DMKL
> -DFFTLOADBAL_2PROC pmemd.fpp pmemd.f90
> ifort -c -auto -tpp7 -xP -ip -O3 pmemd.f90
> ifort: command line remark #10148: option '-tp' not supported
> <<<<-HERE IS THE LAST LINE OF THE COMPILATION PROCESS
>
> after this one hour I killed compilation and obtained this typical
> messages:
>
> make[1]: *** Deleting file `pmemd.o'
> make[1]: *** [pmemd.o] Error 2
> make: *** [install] Interrupt
>
> I really do not understand how it is possible that compiler is using 50%
> of CPU for one hour and get jammed in one line ...
>
> I have to say that compilation with old version of ifort package was
> question of some minutes.
>
> It seems to me as an typical case of "infinity" loop ...
>
> But nevertheless then I got the idea just use the old pmemd compilation
> with the new installed libraries (cc ...) and it works :)) !!!
>
> Another situation was with SANDER but after compleet recompilation of
> Amber Tools and Amber, everything is OK ( at least now :)) ).
>
> So I think that my problem is solved but still is here some strange
> question about impossiblity to finish in real time compilation
> of the PMEMD with ifort11 package. Just to be complex I have to say, that
> I tried pmemd instalation with original "configure" file
> but also with Bob's late night one. The result is the same. Fortunately
> it
> is not crucial problem for me now ...
>
> So thank you both again !!!
>
> Best,
>
> Marek
>
>
>
>
>
>
>
>
>
>
>
>
> Dne Thu, 07 May 2009 01:04:10 +0200 Robert Duke <rduke.email.unc.edu>
> napsal/-a:
>
>> Oh, very good find Ross; I have not had the experience of mixing these,
>> but I bet you are right! - Bob
>> ----- Original Message ----- From: "Ross Walker" <ross.rosswalker.co.uk>
>> To: "'AMBER Mailing List'" <amber.ambermd.org>
>> Sent: Wednesday, May 06, 2009 5:53 PM
>> Subject: RE: [AMBER] Error in PMEMD run
>>
>>
>>> Hi Marek,
>>>
>>>> here is the content of the LD_LIBRARY_PATH variable:
>>>>
>>>> LD_LIBRARY_PATH=/opt/intel/impi/3.1/lib64:/opt/intel/mkl/10.0.011/lib/e
>>>> m64t:/opt
>>>> /intel/cce/9.1.043/lib:/opt/intel/fce/10.1.012/lib::/opt/intel/impi/3.1
>>>> /lib64
>>>
>>> I suspect this is the origin of your problems... You have cce v9.1.043
>>> defined and fce v10.1.012 defined. I bet these are not compatible. Note
>>> there is a libsvml.so in /intel/cce/9.1.043/lib/ and this comes first
>>> in
>>> your LD path so will get picked up before the Fortran one. This is
>>> probably
>>> leading to all sorts of problems.
>>>
>>> My advice would be to remove the old cce library spec from the path so
>>> it
>>> picks up the correct libsvml. Or upgrade your cce to match the fce
>>> compiler
>>> version - this should probably always be done and I am surprised Intel
>>> let
>>> you have mixed versions this way but alas..... <sigh>
>>>
>>> All the best
>>> Ross
>>>
>>>
>>> /\
>>> \/
>>> |\oss Walker
>>>
>>> | Assistant Research Professor |
>>> | San Diego Supercomputer Center |
>>> | Tel: +1 858 822 0854 | EMail:- ross.rosswalker.co.uk |
>>> | http://www.rosswalker.co.uk | PGP Key available on request |
>>>
>>> Note: Electronic Mail is not secure, has no guarantee of delivery, may
>>> not
>>> be read every day, and should not be used for urgent or sensitive
>>> issues.
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> AMBER mailing list
>>> AMBER.ambermd.org
>>> http://lists.ambermd.org/mailman/listinfo/amber
>>>
>>
>>
>>
>> _______________________________________________
>> AMBER mailing list
>> AMBER.ambermd.org
>> http://lists.ambermd.org/mailman/listinfo/amber
>>
>> __________ Informace od NOD32 4051 (20090504) __________
>>
>> Tato zprava byla proverena antivirovym systemem NOD32.
>> http://www.nod32.cz
>>
>>
>
--
Tato zpráva byla vytvořena převratným poštovním klientem Opery:
http://www.opera.com/mail/
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Wed May 20 2009 - 15:10:56 PDT
