Hi,
This page is a decent presentation of some LD_LIBRARY_PATH issues
http://www.visi.com/~barr/ldpath.html
Based on it, we should probably be doing more to set the runtime
library search path.
Piotr, I added this to the config.h produced by configure ifort:
LOADLIB= -Wl,-rpath /opt/intel_fc_80/lib
The resulting sander ran without setting LD_LIBRARY_PATH.
We should investigate getting the right paths as arguments to -rpath
in our configure script. However, even merely using -rpath with
the canonical path is a big step.
A few responses below ...
On Tue, 15 Feb 2005, Robert Duke wrote:
> Okay, guys, I am a little fuzzy on this stuff, but as I recollect:
> 1) LD_LIBRARY_PATH is first critical during linking, and only used in
> loading (by ld.so) when the information needed to find a shared object is
> not present in the executable.
ifort does not use LD_LIBRARY_PATH to find libs in /opt/intel_fc_80/lib
Using ifort -v shows that ifort invokes ld with -L/opt/intel_fc_80/lib.
Using -Wl,-verbose,-M as LOADLIB gets the details on ld's actions, eg:
attempt to open /opt/intel_fc_80/lib/libm.so failed
attempt to open /opt/intel_fc_80/lib/libm.a failed
attempt to open /usr/lib/libm.so succeeded
ifort -V
Intel(R) Fortran Compiler for 32-bit applications, Version 8.0 Build 20031016Z Package ID: l_fc_p_8.0.034
> 2) I actually don't set LD_LIBRARY_PATH at all on the machines other than
> the compile machine; I just make sure that all the machines have a version
> of the intel compiler libraries visible in the same place (via the same
> path).
Consistency counts, but a general user may not have control over that.
> 3) It may be necessary to get the path to the ifort libs into
> /etc/ld.so.cache by using ldconfig. Please read the man pages for ldconfig
> (the dynamic linking config tool), ld.so (the dynamic loader), and ld (the
> linker) for clarifications on all this wonderful stuff.
I cannot execute ldconfig as non-root
ldconfig: Can't create temporary cache file /etc/ld.so.cache~: Permission denied
Thus, this also may not be a general solution.
Scott
> 4) I WOULD be 100% behind making everything static, but there were huge
> problems with statically linked executables on RH 3 a while back, associated
> with the fact that the static threads libraries were essentially broke
> (whereas the dynamic loaded ones are okay). One could dink around with
> loader options and perhaps manage to dynamically load only certain things,
> but I never got to the bottom of the problem, and the OS and compiler were
> both moving targets.
>
> Bottom line -
> 1) I continue to load everything dynamically because I have not sorted out
> this grief.
> 2) I use LD_LIBRARY_PATH in compiling, but also use ldconfig to insure the
> correct library paths are in /etc/ld.so.conf. Please read the ldconfig man
> page.
> 3) I insure the intel libraries are visible by the same path. I have no
> idea whether 2 + 3 are both necessary; I would not think so, but it works.
>
> 4) Another common environment gotcha, especially for pmemd, but potentially
> for sander, is the need to put "limit stacksize unlimited" in your .login
> and .cshrc files (it should only be needed in the first, but it IS needed in
> the second; I would regard this as a linux bug).
>
> I think I wrote up some stuff that is out there on the amber web page about
> all this stuff; David Konerding dug deeper than I did and there was some
> mail on the reflector; I used to be fascinated by all this stuff when I was
> a systems guy, but got over it and just want to get the job done.
>
> Regards - Bob
Date: Tue, 15 Feb 2005 11:42:45 -0800
From: Scott Brozell <sbrozell.scripps.edu>
Reply-To: amber.scripps.edu
To: amber.scripps.edu
Subject: Re: AMBER: problems with running sander in parallel on linux
Hi,
Verify that LD_LIBRARY_PATH is the problem:
copy libimf.so to your working directory and rerun mpirun.
Use the -v option to mpirun to get more details.
Try a different libimf.so; copy one from another machine.
Here are libimf.so details:
1504 -rwxr-xr-x 1 root root 1533098 Oct 29 2003 libimf.so
Intel(R) Fortran Compiler for 32-bit applications, Version 8.0 Build
20031016Z Package ID: l_fc_p_8.0.034
What happens with the serial sander ?
What ifort version are you using ?
good luck,
Scott Brozell
> ----- Original Message -----
> From: "Ross Walker" <ross.rosswalker.co.uk>
> To: <amber.scripps.edu>
> Sent: Tuesday, February 15, 2005 1:50 PM
> Subject: RE: AMBER: problems with running sander in parallel on linux
>
>
> > Hi Piotr,
> >
> >> I compiled mpi sander on my linux box using ifort intel compiler
> >> (under most recent mpi/mpd).
> >> When I tried to run it (in this case on 2 processors) I got
> >> the following error message:
> >>
> >> sander: error while loading shared libraries: libimf.so:
> >> cannot open shared object file: No such file or directory
> >> sander: error while loading shared libraries: libimf.so:
> >> cannot open shared object file: No such file or directory
> >
> > What mpi wrapper are you using? It is possible that the mpirun command is
> > running rsh to your machine and this is not sourcing your login scripts
> > correctly. Hence it doesn't pick up the LD_LIBRARY_PATH and so can't find
> > the libraries. See if you can find out how your mpirun is spawning
> > processes
> > and in particular find out what shell it is running. The default shell may
> > be different to "your" shell.
> >
> > The best solution to this is to compile everything static. You can compile
> > a
> > static version of lam by adding the flag -all-static to the configure
> > script. Then recompile lam and also compile amber statically. You can also
> > compile mpich statically but I can't remember the options for this.
-----------------------------------------------------------------------
The AMBER Mail Reflector
To post, send mail to amber.scripps.edu
To unsubscribe, send "unsubscribe amber" to majordomo.scripps.edu
Received on Tue Feb 15 2005 - 21:53:00 PST
