Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Override bloom dh_shlibdeps behavior? Issues pip dependencies that bundle shared libraries #110

Open
villapx-path opened this issue Dec 8, 2023 · 0 comments

Comments

@villapx-path
Copy link

I am using catkin_virtualenv 0.6.1 (this is the only version that's currently published for ROS Melodic and Noetic, see this issue), and some of my pip dependencies bundle shared libraries in them.

After successfully building my package with catkin_virtualenv as a dependency, I am attempting to use Bloom to create a rosdebian package:

bloom-generate rosdebian \
  --ros-distro melodic \
  --os-name ubuntu \
  --os-version bionic

debian/rules binary

The resulting debian/rules Makefile is throwing errors from dh_shlibdeps for all of my pip dependencies that bundle shared libraries, for example:

#24 42.65 dpkg-shlibdeps: error: cannot find library libgfortran-ed201abd.so.3.0.0 needed by debian/ros-melodic-redacted-pkg-name/opt/ros/melodic/share/redacted_pkg_name/venv/lib/python3.6/site-packages/numpy/.libs/libopenblasp-r0-34a18dc3.3.7.so (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')

But libgfortran-ed201abd.so.3.0.0 is correctly bundled in the numpy pip package:

# ls -l install/share/redacted_pkg_name/venv/lib/python3.6/site-packages/numpy/.libs/
total 30028
-rwxr-xr-x 1 root root  1023960 Dec  7 21:58 libgfortran-ed201abd.so.3.0.0
-rwxr-xr-x 1 root root 29724672 Dec  7 21:58 libopenblasp-r0-34a18dc3.3.7.so

I think what's tripping up dh_shlibdeps is the fact that pip doesn't publicly set an LD_LIBRARY_PATH in the venv nor an RPATH in the "consuming" shared lib (libopenblasp in this case), so if you ldd libopenblasp-r0-34a18dc3.3.7.so it will say that it can't find libgfortran.

But this is expected, as pip does its own internal shared library resolution at runtime.


What is the proper way to get around this in my CI pipeline? Basically in my bloom-generate call (or after it), I need to either turn off dh_shlibdeps, or at least tell it not to worry about shared libs bundled within the packages in the venv.

I feel like this has to be something folks have run into before when using catkin_virtualenv with Bloom, especially since many ROS applications work with sensor data and the like, and thus depend on pip packages that bundle shared libraries.

I appreciate the help!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant