A tale of woe: LD_LIBRARY_PATH, SETUID and q

Whilst trying to enforce restrictions around how services run on a RHEL6 server, I thought it would be good to make use of the SETUID/SETGID permission bits, rather than relying on the development team to to make us of the su command to launch their applications as a specific user.

This turned out to be a really bad idea!  And caused a major headache.

Well… the idea to restrict what someone can do on a Linux server and to reduce their exposure to the system is generally NOT seen as a bad idea.  BUT due to the requirements of the application (an internal app developed with KDB+ q and reliant upon a third party application for connectivity to an Oracle database) this became quite problematic.

Overview

At this point I had considered rambling on about the hardware and OS and application versions, but for the most part that is a waste of time.

The problem

When we launched a q based application, which relied upon a 3rd party ODBC driver for Oracle, we started to see some very odd errors which we struggled to explain.

[unixODBC][Easysoft][Oracle]Could not load oracle libraries

Now at this point we checked the obvious things.

  • Does the ODBC drivers own tools connect to the required database without issue?  Yes
  • Are we missing any environment variables, such as ORACLE_HOME? No
  • Is the PATH variable defined correctly? Yes
  • Do we need to add anything to the LD_LIBRARY_PATH? Yes we do

So, from the above list the only thing we hadn’t set was the LD_LIBRARY_PATH, but in fairness this didn’t seem to be needed by the ODBC Driver.  We were able to use its own tools to check everything is working, and everything worked as expected and a connection was made without issue.

At this point I delved into the multiple scripts which are used to start the service.  After adding some carefully placed;

env > /some/textfile

At the beginning, middle and end of each script we found that the variables were being passed without issue.  It was just inside the application that we seemed to loose the LD_LIBRARY_PATH variable.

So I then started looking at the executable in a standalone capacity.  This also exhibited the same behaviour, whereby LD_LIBRARY_PATH was defined in my bash shell but was not being exposed to the executable.

The executable had been set with the setuid bit enabled and as a result it would run under the user that owned it.  So we start looking at the user specifically to see if there is anything wrong with it’s environment.  Once again, everything looked as it should except within q.  All very frustrating!

The cause

In the end, it would appear that this behaviour is by design on the grounds of security.  Because LD_LIBRARY_PATH is used to override the the systems default paths to dynamic libraries, it was seen that passing this variable into a application which runs as a different user, could be used to introduce less desirable libraries or to introduce weaknesses which in turn could compromise security.

For a more in-depth view on what is happening and why went using SETUID and/or SETGID, take a look at the following URL;

https://wiki.cac.washington.edu/display/~ktower%40washington.edu/Library+Linking+and+setuid+executables

The solution

It is now up to the developers to re-engineer their application to enforce that a their application runs as a specific user, for example using su.

An alternative would have also been for me to re-install the ODBC driver, which ultimately was the one struggling to work as it couldn’t find the variables it required to talk to the Oracle Instant Client, but at the time of installation I would need to specify that I wanted the installation to use the system preferred directories for libraries.  But time and environmental constraints meant this wasn’t an option this time round.

Thanks to the guys at KX for providing the link above which help to clear this issue up!

Leave a Reply