Обсуждение: 8.3 to 8.4 - Can't load dynamic shared library
I'm migrating from Postgres 8.3.10 to 8.4.4, and also from Fedora 9 to Ubuntu 10.04. On 8.3.10, I have a C extension that worked on 8.3, but now refuses to load on 8.4: CREATE OR REPLACE FUNCTION chmoogle_session_id() RETURNS integer AS '/usr/local/pgsql/lib/libchmoogle.so', 'chmoogle_session_id' LANGUAGE 'C' VOLATILE; ERROR: could not load library "/usr/local/pgsql/lib/libchmoogle.so": /usr/local/pgsql/lib/libchmoogle.so: undefined symbol:_ZN9OpenBabel4etabE So it finds the first shared library, libchmoogle.so. But the symbol _ZN9OpenBabel4etabE is from a second library, the OpenBabel(chemistry) shared library, which is also located in /usr/local/pgsql/lib/libopenbabel.so On the Postgres 8.3 system, I simply put libopenbabel.so into the /usr/local/pgsql/lib directory, and everything worked well. On 8.4, I can't seem to get it to load libopenbabel.so. Other programs that use this library have no trouble findingit. What (if anything) has changed from 8.3 to 8.4 regarding how it finds and links dynamic libraries? What else should I lookfor? I'm stumped on this one. Thanks, Craig
On Thu, 2010-07-15 at 11:49 -0700, Craig James wrote: > I'm migrating from Postgres 8.3.10 to 8.4.4, and also from Fedora 9 to Ubuntu 10.04. > > On 8.3.10, I have a C extension that worked on 8.3, but now refuses to load on 8.4: > > CREATE OR REPLACE FUNCTION chmoogle_session_id() RETURNS integer > AS '/usr/local/pgsql/lib/libchmoogle.so', 'chmoogle_session_id' > LANGUAGE 'C' VOLATILE; > ERROR: could not load library "/usr/local/pgsql/lib/libchmoogle.so": /usr/local/pgsql/lib/libchmoogle.so: undefined symbol:_ZN9OpenBabel4etabE > > So it finds the first shared library, libchmoogle.so. But the symbol _ZN9OpenBabel4etabE is from a second library, theOpenBabel (chemistry) shared library, which is also located in > > /usr/local/pgsql/lib/libopenbabel.so > > On the Postgres 8.3 system, I simply put libopenbabel.so into the /usr/local/pgsql/lib directory, and everything workedwell. On 8.4, I can't seem to get it to load libopenbabel.so. Other programs that use this library have no troublefinding it. > > What (if anything) has changed from 8.3 to 8.4 regarding how it finds and links dynamic libraries? What else should Ilook for? I'm stumped on this one. > Studid question, but did you recompile the library against 8.4 first? > Thanks, > Craig > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579 Consulting, Training, Support, Custom Development, Engineering
Craig James <craig_james@emolecules.com> writes:
> On the Postgres 8.3 system, I simply put libopenbabel.so into the /usr/local/pgsql/lib directory, and everything
workedwell. On 8.4, I can't seem to get it to load libopenbabel.so. Other programs that use this library have no
troublefinding it.
> What (if anything) has changed from 8.3 to 8.4 regarding how it finds
and links dynamic libraries?
Nothing. I think this more likely is a consequence of your platform
switch from Fedora to Ubuntu. I'm not sure how the dynamic linker's
search path is defined on Ubuntu, but it might be different. Or
possibly it's the same but you forgot about making an /etc/ld.so.conf.d
entry on the new machine, or didn't run ldconfig afterwards.
regards, tom lane
On 7/15/10 12:52 PM, Tom Lane wrote: > Craig James<craig_james@emolecules.com> writes: >> On the Postgres 8.3 system, I simply put libopenbabel.so into the /usr/local/pgsql/lib directory, and everything workedwell. On 8.4, I can't seem to get it to load libopenbabel.so. Other programs that use this library have no troublefinding it. > >> What (if anything) has changed from 8.3 to 8.4 regarding how it finds > and links dynamic libraries? > > Nothing. I think this more likely is a consequence of your platform > switch from Fedora to Ubuntu. I'm not sure how the dynamic linker's > search path is defined on Ubuntu, but it might be different. Or > possibly it's the same but you forgot about making an /etc/ld.so.conf.d > entry on the new machine, or didn't run ldconfig afterwards. We don't use /etc/ld.so.conf.d, and never have. On 8.3.10 and earlier, Postgres would ONLY load this library if we put it in /usr/local/pgsql/lib, so I figured it was asecurity restriction, that Postgres altered the LD_LIBRARY_PATH to ensure that only legitimate, admin-approved librariescould be loaded. The Postgres documentation sort of hints at this but doesn't give details. Is this true, that Postgres restricts the dynamic linker in some way? And if so, is there documentation? Craig
Craig James <craig_james@emolecules.com> writes: > On 8.3.10 and earlier, Postgres would ONLY load this library if we put it in /usr/local/pgsql/lib, so I figured it wasa security restriction, that Postgres altered the LD_LIBRARY_PATH to ensure that only legitimate, admin-approved librariescould be loaded. The Postgres documentation sort of hints at this but doesn't give details. > Is this true, that Postgres restricts the dynamic linker in some way? No, it is not. Library searching behavior is whatever the platform's dynamic linker defines it to be. On some platforms we try to build the executables with an "rpath" pointing at the PG lib directory, so that you don't need to explicitly configure anything to load our libraries --- but I don't believe that that causes the linker to *only* search that directory on any of those platforms. (If it did, you'd have big problems with libc.so for instance.) As for LD_LIBRARY_PATH, the only part of our code that touches that is the regression test wrapper pg_regress. You might or might not find this relevant: http://archives.postgresql.org/pgsql-hackers/2010-05/msg00248.php My guess is that the reason you had to put libopenbabel.so into /usr/local/pgsql/lib before is precisely that you hadn't modified /etc/ld.so.conf, and so only /usr/local/pgsql/lib would be searched (as a result of the postgres executable's rpath property) plus the standard system libraries (but unless you ran ldconfig, the linker wouldn't know about any additions there). The Ubuntu situation may or may not be similar --- in particular, I don't know whether their packages even set the rpath property at all. I think some distros deprecate using rpath and instead expect the PG lib directory to be part of the ldconfig configuration, in which case *no* nonstandard library is gonna get found unless you ask ldconfig to pretty please find it. Or maybe Ubuntu has yet another way. You really ought to ask for help about this in an Ubuntu-specific forum. regards, tom lane
On 7/15/10 2:14 PM, Tom Lane wrote: > Craig James<craig_james@emolecules.com> writes: >> On 8.3.10 and earlier, Postgres would ONLY load this library if we put it in /usr/local/pgsql/lib, so I figured it wasa security restriction, that Postgres altered the LD_LIBRARY_PATH to ensure that only legitimate, admin-approved librariescould be loaded. The Postgres documentation sort of hints at this but doesn't give details. > >> Is this true, that Postgres restricts the dynamic linker in some way? > > No, it is not. Library searching behavior is whatever the platform's > dynamic linker defines it to be. On some platforms we try to build the > executables with an "rpath" pointing at the PG lib directory, so that > you don't need to explicitly configure anything to load our libraries > --- but I don't believe that that causes the linker to *only* search > that directory on any of those platforms. (If it did, you'd have big > problems with libc.so for instance.) > > As for LD_LIBRARY_PATH, the only part of our code that touches that is > the regression test wrapper pg_regress. You might or might not find > this relevant: > http://archives.postgresql.org/pgsql-hackers/2010-05/msg00248.php > > My guess is that the reason you had to put libopenbabel.so into > /usr/local/pgsql/lib before is precisely that you hadn't modified > /etc/ld.so.conf, and so only /usr/local/pgsql/lib would be searched > (as a result of the postgres executable's rpath property) plus the > standard system libraries (but unless you ran ldconfig, the linker > wouldn't know about any additions there). The Ubuntu situation may > or may not be similar --- in particular, I don't know whether their > packages even set the rpath property at all. I think some distros > deprecate using rpath and instead expect the PG lib directory to > be part of the ldconfig configuration, in which case *no* nonstandard > library is gonna get found unless you ask ldconfig to pretty please > find it. Or maybe Ubuntu has yet another way. You really ought to > ask for help about this in an Ubuntu-specific forum. Thanks, you answer helped but in an unexpected way. Knowing it had nothing to do with dynamic loading made me look elsewhere,so I dug around with nm(1) and realized that the missing symbol really was missing. The reason is because of changes in pgxs.mk, the makefile that's supposed to help build extensions. In 8.3, I could overridethe link step and use g++ rather than gcc, but in 8.4 the pgxs.mk ignored my link step, and was using gcc for thelinking. Since the OpenBabel chemistry library is written in C++, you have to use g++ to do the link step for the Postgresextension. I ended up hand crafting a Makefile and not using pgxs.mk at all, and now everything works. Craig
On Thu, 2010-07-15 at 11:49 -0700, Craig James wrote: > I'm migrating from Postgres 8.3.10 to 8.4.4, and also from Fedora 9 to Ubuntu 10.04. > > On 8.3.10, I have a C extension that worked on 8.3, but now refuses to load on 8.4: > > CREATE OR REPLACE FUNCTION chmoogle_session_id() RETURNS integer > AS '/usr/local/pgsql/lib/libchmoogle.so', 'chmoogle_session_id' > LANGUAGE 'C' VOLATILE; > ERROR: could not load library "/usr/local/pgsql/lib/libchmoogle.so": /usr/local/pgsql/lib/libchmoogle.so: undefined symbol:_ZN9OpenBabel4etabE > > So it finds the first shared library, libchmoogle.so. But the symbol _ZN9OpenBabel4etabE is from a second library, theOpenBabel (chemistry) shared library, which is also located in > > /usr/local/pgsql/lib/libopenbabel.so > > On the Postgres 8.3 system, I simply put libopenbabel.so into the /usr/local/pgsql/lib directory, and everything workedwell. On 8.4, I can't seem to get it to load libopenbabel.so. Other programs that use this library have no troublefinding it. > > What (if anything) has changed from 8.3 to 8.4 regarding how it finds and links dynamic libraries? What else should Ilook for? I'm stumped on this one. > Studid question, but did you recompile the library against 8.4 first? > Thanks, > Craig > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579 Consulting, Training, Support, Custom Development, Engineering