[CoinBinary] [BuildTools] BuildTools issues for debian packages

[Miles Lubin]

On Mon, Sep 9, 2013 at 1:16 PM, Ted Ralphs <ted at lehigh.edu> wrote:
> I've been working on updating the debian packages for Clp, Cbc, and
>>
>> related Coin projects, which are out of date and no longer of interest
>> to the previous maintainer. I wanted to raise a few issues with the
>> build system which I've run into during this process:
>
>
> It's great that someone is picking this up. We have been working with
> someone at Fedora and have packages going through the approval process
> there. It would be great to have the same for Debian. We'e made a lot of
> changes over the past few years to accommodate this kind of packaging and
> I'd be happy to help you through the process.

Thanks. I saw the thread about the Fedora packages. The coin packages
are already in Debian, so at least we won't need to go through an
approval process.


>> 1) I'm unable to regenerate the autotools-generated files. Debian
>> prefers to rerun autotools so that newer architectures like ARM64 are
>> properly supported. I've tried using the `run_autotools` script, but
>> the libtool version used (1.5.22) is no longer available within
>> debian. The older versions of autoconf (2.59) and automake (1.9.6) are
>> okay since they are still actively maintained as debian packages.
>
>
> It's pretty easy to build the proper version of libtool from source, but I
> guess the issue is not that you can't get that version of libtool---it's
> just that it's not supported as a debian package, right? It is on our TODO
> list to get the build system working with more recent version of the auto
> tools, but we have a lot of patches and fixes that are specific to those
> particular version of the auto tools and it looks like some work to unwind
> all of that. We do plan to do this, however. Finding the time is another
> thing...

Right, all build-time dependencies (and autotools would be rerun at
build time) have to be available as a debian package, given how the
debian binary build system works. This issue falls a bit on the side
of pedantry for most practical purposes, so I wouldn't prioritize it.


>> 2) "make install" fails when DESTDIR is not the same as the
>> installation prefix (and pkg-config is installed). To reproduce,
>> building OSI, for example:
>> $ ./configure
>> $ make
>> $ mkdir installdir
>> $ make DESTDIR=`pwd`/installdir install
>> ...
>> make  install-data-hook
>> make[4]: Entering directory `/home/mlubin/osi/Osi-0.106.1/CoinUtils'
>>
>> PKG_CONFIG_PATH=/home/mlubin/osi/Osi-0.106.1/lib64/pkgconfig:/home/mlubin/osi/Osi-0.106.1/lib/pkgconfig:/home/mlubin/osi/Osi-0.106.1/share/pkgconfig:
>> \
>>     pkg-config --libs coinutils >
>>
>> /home/mlubin/osi/Osi-0.106.1/installdir/home/mlubin/osi/Osi-0.106.1/share/coin/doc/CoinUtils/coinutils_addlibs.txt
>> Package coinutils was not found in the pkg-config search path.
>> Perhaps you should add the directory containing `coinutils.pc'
>> to the PKG_CONFIG_PATH environment variable
>> No package 'coinutils' found
>>
>>
>> Debian packages are built using DESTDIR, so I've had to patch this to
>> remove this code. I might suggest simply not generating the
>> *_addlibs.txt files, since these are redundant to pkg-config.
>
>
> We didn't want to assume pkg-config was available and in fact, using
> pkg-config in the build process seems to cause a lot of problems. For
> examples, on Windows, if there are spaces in the path, this causes some
> problems. It's also sort of a pain figuring out whether the version of
> pkg-config you'reworking with understands POSIX paths or Windows paths.
> Overall, I would say that the best thing would be just to disable the use of
> pkg-config during configuration on all platforms. This is one more reason to
> do that. If you use the option "--disable-pkg-config", does that fix the
> problem?

pkg-config is convenient for finding installed coin dependencies when
these are built independently, so I'd prefer to keep using it during
configure. I've done a bit more digging and seem to have fixed it:

The lines in Osi/Makefile.am:

if COIN_HAS_PKGCONFIG
    PKG_CONFIG_PATH=@COIN_PKG_CONFIG_PATH@ \
    $(PKG_CONFIG) --libs osi > $(addlibsdir)/osi_addlibs.txt
else

should be:

if COIN_HAS_PKGCONFIG
    PKG_CONFIG_PATH=$(DESTDIR)/$(pkgconfiglibdir) \
    $(PKG_CONFIG) --libs osi > $(addlibsdir)/osi_addlibs.txt
else


Same change for all of the coin projects. Hopefully this doesn't break anything.


>> 3) Shared libraries do not properly list their runtime dependencies;
>> e.g. libClp.so does not, but should, depend on libCoinUtils.so,
>> libbz2.so, liblapack.so, etc., as reported by the ldd tool. This has
>> been raised a few times:
>> http://list.coin-or.org/pipermail/clp/2012-June/001265.html,
>> http://list.coin-or.org/pipermail/coin-discuss/2008-October/003480.html,
>> and in a private communication with Stefan Vigerske this past
>> December. While this isn't too difficult to deal with when developing
>> compiled applications, it significantly complicates using the shared
>> library from within dynamic languages like Python or Julia (relatedly,
>> I am the maintainer of the Clp and Cbc interfaces for Julia). The
>> Debian package system also uses shared library dependencies to infer
>> package dependencies, and this procedure is broken for coin libraries.
>
>
>  Yes, this is a problem for the Fedora packing as well and we had on our
> TODO list to deal with this, but it will also require some work. For Python,
> I parse the _addlibs.txt files to find dependencies and this works fine. If
> you look at the setup script for Dippy, you will see how it's done:
>
> https://projects.coin-or.org/Dip/browser/trunk/Dip/setup.py

Parsing the _addlibs.txt at runtime is nice trick, albeit not so
pretty. I'm no autotools expert, but what is needed to make this work?
Perhaps I can help.

Thanks,
Miles