LIBPATH... How Does It Work?
So in the last entry, I described the differences between PATH and LIBPATH, but I didn't really explain how LIBPATH is actually used. I'm going to rectify that in this entry. First, though, we need to take a detour and learn how programs get loaded.
Loading 101
When you execute a program on Unix systems, this flows through a function called exec(). This special function will cause the program specified to replace the currently executing program. Before the new program can start its execution, it must first get loaded in to memory. There's two pieces that coordinate to load a program:
- the loader in the kernel (SLIC)
- the linker (ld)
The loader will load the program objects in to memory, while the linker will find the shared libraries that the program depends on and has them loaded as well as a bunch of other things that we don't care about for this. In AIX/PASE there's also the Runtime Dynamic Linker, but we can ignore that for now as well.
Drafting our MVP
So to start, let's look at a "simple" example. This is a program which doesn't link to any shared libraries. One might call it an "MVP": Minimum Viable Program. You might think this basic program looks like this:
int main()
{
int x = 6 + 25;
return x;
}
However, if we build this program it will link to the libc shared library, but I said we didn't want to link to any shared libraries! How can we do that? For that we need to make our "simple" example somewhat more complicated by going to assembly. Here's the code:
.file "example.s"
.csect .text[PR]
.toc
.csect .text[PR]
.align 2
.globl __start
.globl .__start
.csect __start[DS]
__start:
.llong .__start, TOC[tc0], 0
.csect .text[PR]
.__start:
li 9,6
li 10,25
add 3,9,10
This is practically the smallest program you can make. It's so small that some might argue the applicability of using the term "viable" to describe it, since it doesn't contain a complete instruction stream and simply crashes after the last instruction:
-bash-4.3$ ./example1
Illegal instruction (core dumped)
However, it does load and execute so it's definitely a program... Just not a very good one! 😉
Dumping the MVP
So our MVP got in to a bad crash and broke his leg, punctured his spleen, and lost half his blood. What to do with him? It may seem heartless, but I think we have to dump him:
-bash-4.3$ dump -X64 -H example1
example1:
***Loader Section***
Loader Header Information
VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x00000001 0x00000002 0x00000010
#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000001 0x00000070 0x0000000a 0x00000080
***Import File Strings***
INDEX PATH BASE MEMBER
0 /usr/lib:/lib
The AIX/PASE dump command can tell you quite a bit about binaries/executables, shared libraries, and object files. Here we're using the -X64 flag to tell it we're looking at a 64-bit object and the -H flag to tell it to dump the object's header. The important part to look at is below the ***Import File Strings***. There is only 1 index defined and it is index 0, which is special and always exists. Since there are no other indexes defined this means the program is not linked to any shared libraries.
Since this program is not linked to any shared libraries, the linker doesn't have anything to do and so LIBPATH is not used at all. What about a program that links to some shared libraries? Let's that first C example above and dump it:
-bash-4.3$ dump -X64 -H example2
example2:
***Loader Section***
Loader Header Information
VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x0000000a 0x0000002a 0x00000021
#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000002 0x000003c8 0x0000007b 0x000003e9
***Import File Strings***
INDEX PATH BASE MEMBER
0 /usr/lib:/lib
1 libc.a shr_64.o
Aha! Notice now we have an index 1 whose "base" is libc.a. This is the AIX/PASE C library (libc). Shared libraries are a bit funky on AIX/PASE compared to other UNIX environments. On most other UNIX platforms, a "shared library" is synonymous with a "shared object", but this is not the case in AIX/PASE. Here, a shared library can contain my shared objects. In this case we're linked to shr_64.o, which is the conventional name for AIX 64-bit shared objects (shr.o is the conventional name for 32-bit shared objects).
Ok, so now we execute the program: Our shell calls exec() which tells the SLIC loader to load the example2 binary in to memory and then the linker gets to load our shared libraries. In this case it sees that it needs to load shr_64.o from the libc.a archive, but how does it go about finding libc.a?
Index 0, You're My Hero!
I have skipped over index 0 previously, but you may have noticed it looks an awful lot like PATH or LIBPATH. Well, that's because it is! Index 0 is a special index that specifies the executable's library search path. It's technically not LIBPATH, but you can think of it like that. The linker will look through each of these paths until it find a file named libc.a and try to load shr_64.o from it. If we look, indeed there is a /usr/lib/libc.a in PASE with a shr_64.o member:
-bash-4.3$ ar -X64 t /usr/lib/libc.a | grep shr
shr_64.o
404: Library Not Found
So what happens if the linker looks through all the directories on the application's search path and still can't find a library? Well, then you get linker errors:
-bash-4.3$ ./example3
exec(): 0509-036 Cannot load program ./example3 because of the following errors:
0509-150 Dependent module libc.a(shr_64.o) could not be loaded.
0509-022 Cannot load module libc.a(shr_64.o).
0509-026 System error: A file or directory in the path name does not exist.
-bash-4.3$ dump -X64 -H example3
example3:
***Loader Section***
Loader Header Information
VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x0000000a 0x0000002a 0x00000023
#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000002 0x000003c8 0x0000007b 0x000003eb
***Import File Strings***
INDEX PATH BASE MEMBER
0 /does/not/exist
1 libc.a shr_64.o
In this case the library search path only contains a dummy directory /does/not/exist which doesn't contain libc.a. You would get a similar error if the library exists, but does not contain the shr_64.o member:
-bash-4.3$ ./example3
exec(): 0509-036 Cannot load program ./example3 because of the following errors:
0509-150 Dependent module /does/not/exist/libc.a(shr_64.o) could not be loaded.
0509-152 Member shr_64.o is not found in archive
In either case, we can use the LIBPATH environment variable to tell the linker where to find libc.a:
-bash-4.3$ LIBPATH=/usr/lib:/lib ./example3
-bash-4.3$ echo $?
31
Use LIBPATH to specify extra directories to search for shared libraries in addition to the application's built in search path. These directories are searched before the directories in the application's search path. In the above example, the application's library search path is /does/not/exist and the LIBPATH is /usr/lib/:/lib, so the resulting search path is /usr/lib:/lib:/does/not/exist. Since libc.a is found in /usr/lib and contains shr_64.o, all is well!
Of course, all of that mess could have just been avoided if our application told the linker where the libc.a actually is. Then we wouldn't have needed to specify LIBPATH at all! So why doesn't our application have the right library path specified? In the next entry I'll explain how the application's library search path gets set and how to set it yourself so you shouldn't normally have to specify LIBPATH.