In order to address issues in applications, the author has listed what needs to be done whenever you install a third party application on RHEL systems. The steps are almost the same for other Linux flavours.
Before going any further, let’s take a look at certain aspects of applications:
- Shared libraries
- How linking shared libraries with applications works
- How applications really find shared libraries
- Real-time troubleshooting relating to application issues.
For the purposes of this article, I have used RHEL (Red Hat Enterprise Linux) as the base OS. However, the steps are almost the same for other Linux flavours.
Most Linux applications are dynamically linked with the shared libraries they use. Shared libraries are built in a way that allows their functions to be mapped into an application’s memory when the application is executed. Since the required code is provided by the library at runtime, it is not copied to the application itself.
Linking shared libraries with the application gives the following benefits:
- The application binary executables are smaller, since the library code is not copied into them.
- The library can be shared with multiple, simultaneously executing programs. This takes up less system memory.
- Shared libraries make it easier to fix bugs in the library code. Most fixes to a shared library do not require every application linked with it to be rebuilt.
Linking with shared libraries
When an application is built, it is linked with the shared libraries that provide the functions that it uses. The compiler checks the libraries for the required symbols and confirms they exist.
Identifying information about each required library is embedded in the executable. This can be the absolute path name of the library but it is more commonly the DT_SONAME field of the shared library. The DT_SONAME includes the name and version of the shared library, so that the application will use the correct version at runtime. This field is set when the shared library is created with the –soname option to the ln linker. The objdump command includes the field when it displays information about the shared library. You can view the information about libpthread.so.0 in Figure 1.
The shared library has a symbolic link that points to it, which matches its DT_SONAME field, as shown in Figure 2.
When an application executes, the runtime linker identifies the required shared libraries and maps them into the program’s memory space. On Red Hat Enterprise Linux 7, /lib64/ld-linux-x86-64.so.2default, which is the 64-bit version of the runtime linker, is provided by the glibc.x86_64 package. The 32-bit version of the glibc library provides the 32-bit version of the runtime linker when it is installed.
The runtime linker uses the DT_SONAME names embedded in the application to determine which versions of the libraries to use. It then searches the system for those libraries, using the following steps:
- It searches the directories specified in the LD_LIBRARY_PATH environment variable. This step is ignored for setUID and setGID applications. This environment variable is often used when developing or testing an application, where development libraries are to be used.
- Next, the /etc/ld.so.cache file is consulted. It contains a compiled list of candidate libraries previously found. The ldconfig -p command prints the list of libraries mapped to the /etc/ld.so.cache. You can refer to Figure 3 for the same.
- Finally, it searches the directories in the default library paths. The 64-bit shared library loader points to the 64-bit versions of the default location — that is /lib64, then /usr/lib64, then /lib, and then /usr/lib (Figure 4).
The ldd command displays the shared libraries that are required by the specified application at runtime. Each library DT_SONAME name is displayed, followed by the path name to the library on the system, as shown in Figure 5.
Note: The absolute path name of a library is embedded in a program when the shared library does not have the DT_SONAME field. In the previous example (refer to Figure 5), the /lib64/ld-linux-x86-64.so.2 references point to the runtime linker itself.
Troubleshooting library dependency issues
Library dependency problems can occur on Red Hat Enterprise Linux systems when some third party software is installed without using yum or RPM. Also, using the —force option or —nodeps option with rpm can cause library dependency issues, which are easy to identify. When an application references a shared library that is not available, the runtime linker displays a distinctive error message. It displays the name of the first library it cannot find, and then exits the application with an exit status of 127.
When installing some third party application on RHEL without yum or with the —nodeps or —force option, you can check whether the libraries are available or not by executing the ldd command as follows:
# ldd /usr/bin/application
You will get a message like xyz >> library not found. So, from this output, the missing libraries can be identified. Once these are identified, they need to be installed on the system. This is a relatively easy task when they are provided by an RPM package. RPM packages include shared library dependency information within the package meta data.
The yum whatprovides command can identify the package that provides a particular library. Let’s suppose the name of the missing library is libpthread.so.0. Then execute the following command:
# yum whatprovides libpthread.so.0
The output of the above command will be somewhat similar to what’s shown below:
glibc-2.17-196.el7_4.2.i686 : The GNU libc libraries Repo : rhel-7-server-rpms Matched from: Filename : /lib/libpthread.so.0
The output above indicates that the glibc package contains the library that is being looked for.
If the shared library is not packaged and distributed by Red Hat, the third party vendor will have to be called to see if it can be provided
Note: Whenever a shared library is installed or removed from the system, make sure to run the ldconfig command without any argument. This will update the runtime linker cache /etc/ld.so.cache.
I have summarised below what needs to be performed whenever you are installing a third party application on RHEL systems:
- Execute the ldd command to find out what libraries are missing:
# ldd /usr/sbin/app
- Note down the names of missing libraries from the above output.
- Now find the name of the package that provides that library by using the yum whatprovides command:
# yum whatprovides <missing-library-name>
Note down the name of the package. Let us suppose the name of the package is abc.
- Now install the abc package so that you can get the respective libraries.
- The final step is to execute the ldconfig command without any arguments to update the runtime linker cache.
