[BuildTools] [Coin-tlc] Help with build system

[Matěj Týč]

Hello,
thank you for your reply!
On Sat, 2010-05-29 at 13:02 +0200, Stefan Vigerske wrote:
> Hi,
> 
> I don't know, the macro you suggest seem to assemble the compiler flags
> at the time where configure is setup, but it should be at the time when
> configure is executed.

The idea is that you will know the flags at autoreconf time and you may
or may not use them at configure time.
You can use a similar logic with sourcing shell scripts, but m4sugar has
those nice macros that make it easier :-)

> Also there is already a file that contains dependency and compiler flags
> information: xxx-uninstalled.pc.
> 
> Why not implement the following?
> 1. If pkg-config is available, use it to collect compiler and linker
> flags. Here it does not make a difference, if a project has been
> installed or not yet.

Well, personally I wouldn't say that there is no difference. As I see
it, if a project is installed, it has a .pc file in the "standard"
directory, whereas the not-yet-installed project has it in its source
tree. So as I see it, you have to invoke pkg-config in a different ways
those two times.

> 2. If pkg-config is not available, use gnulib to collect compiler and
> linker flags of all installed dependencies. Since they are installed,
> the compiler flags are in some way easy (-I<installdir>/include).

Yeah.

> 3. If the first two fails (no pkg-config available and dependencies are
> not installed yet), try using gnulib to find linker flags in the build
> directories of the dependencies and assemble compiler by parsing the
> xxx-uninstalled.pc file (as already implemented).

I would say that we don't need to use the .pc file format since we don't
need all of its features. I think that a primitive shell script (just
two lines of assignments) that would be sourced in a "parent" script
would do the job, too.
And those macros I have provided probably would handle recursive
dependencies too, like when you want to build library A, you need to
access include files both from libraries B and C, where C depends only
on B, not on A. I don't know how a big issue is this and whether it is
not solved, too...
However, I think that the future would be to get rid of situations like
these anyway.

> If that fails, try parsing the xxx-uninstalled.pc file to assemble both
> compiler and linker flags.

Moreover, if I understand correctly, if we intend to use a
xxx-uninstalled.pc file, it means that we are dealing with our (=COIN)
stuff that is not installed yet, right? 
If this is the case, it should be safe to assume that we are able to
provide all *FLAGS correctly since we, the creators, know best. Which
would mean that we can use all the info from the file
(xxx-uninstalled.pc or whatever) right away (if I have understood
correctly, now it is like take one piece, guess the rest and if it
fails, only then take all).

> 4. Test the compiler and linker flags by attempting a link.

By the way, finding libraries with gnulib includes this step.

> 
> Stefan
> 
> Matěj Týč wrote:
> > On Wed, 2010-05-19 at 23:38 +0200, Stefan Vigerske wrote: 
> >> Hi,
> >>
> >> Matěj Týč wrote:
> >>> On Mon, 2010-05-17 at 10:50 +0200, Stefan Vigerske wrote:
> >>>> Hi,
> >>>>>
> >>> OK I hope that I understand that you would like to make things more
> >>> simpler than remembering all different -I flags that are necessary
> >> now.
> >>
> >> Yes.
> >>
> >>> So I have another suggestion - all projects can store their headers
> >> that
> >>> are going to be installed in a separate directory. 
> >>> It would make things very simple from more point of views:
> >>> - It would be immediately visible what files are going to be
> >> installed
> >>> - All subprojects would have to pass only one -I flag
> >>>
> >>> So it could look like this
> >>>
> >>> coin-cbc/
> >>> ... 
> >>> public_include_two.h
> >>> some_clp_source.cpp
> >>> some_clp_header.hpp
> >>> ...
> >>>
> >>> The Makefile.am could contain something like
> >>> AM_CPPFLAGS += -I includes
> >>>
> >>> and the source files would contain
> >>> #include "coin/public_include_two.h"
> >>>
> >>> And the Cbc project's Makefile.am would contain
> >>> AM_CPPFLAGS += -I $(CLPOBJDIR)/src/includes
> >>
> >> But I don't think that one can convince all (CoinAll related) project
> >> managers to change their directory structure, I mean to tell them
> >> where
> >> they have to store header files.
> >> I could more imagine that a project stores their include paths
> >> somewhere
> >> where another projects configure can find it. Currently we use the
> >> xxx-uninstalled.pc files for this.
> >> Then the fallback for the case that pkg-config is not there parses the
> >> xxx-uninstalled.pc for the compiler flags.
> >> Problem are hidden dependencies here again. If XXX depends on YYY, and
> >> YYY depends on ZZZ, then XXX may find the flags to use YYY from the
> >> yyy-uninstalled.pc file, but it may also require the compiler flags
> >> from
> >> ZZZ. That makes the configure.ac files a bit messy for the fallback
> >> currently.
> > 
> > I think that nobody will be able to resist to a smarter solution :-)
> > Well, I have some other ideas how the directory tree could be
> > reorganized, so I will post them with the appropriate reasoning and
> > maybe project managers will like that.
> > On the other hand, SVN does not handle file moves very well, so there
> > may be a problem even if the reorganization idea is perfect.
> > The other possibility that I can think of would be using another
> > solution than a .pc file.
> > The benefit of pkg-config is that they provide a standard way of
> > checking for CFLAGS and LDFLAGS. We need only CPPFLAGS and other
> > dependencies and we need it only for our purposes, nobody cares whether
> > we use pkg-config or not there.
> > So I suggest using a file containing only CPPFLAGS and dependencies.
> > Then there could be a script (or a m4 macro) that would go through all
> > dependencies recursively and fetching CPPFLAGS from them.
> > It could look like this (copy&paste to your shell, all will happen in a
> > subdirectory):
> > http://pastebin.com/raw.php?i=rgkBXCL0
> > Of course the calls of m4_append within cppflags.m4 should be wrapped,
> > but this should be OK as a demonstration.
> > 
> >>>>> it should be OK. We can't use libtool here, but
> >>>>> pkg-config would not help us anyway.
> >>>> It does, I can just ask pkg-config about the compiler flags for a
> >> package.
> >>>
> >>> I meant it like that if a package is not installed, it is
> >> troublesome to
> >>> locate its .pc file.
> >>
> >> Since we know the base directory of the subproject and we assume that
> >> the xxx-uninstalled.pc file is in this directory, it's not so
> >> difficult
> >> to locate.
> > 
> > Yes, you are right, although it means that you have to do things in a
> > way that users are not used to.
> > There's nothing fundamentally wrong with it, it just tends to make the
> > build system more cryptic.
> > 
> >>> Moreover a "standard" .pc file is usually processed by the configure
> >>> script and relates to the state after installation.
> >>> Having a hand written .pc file for the "subproject build" does not
> >> sound
> >>> bad, though.
> >>> Am I right that this is that what you have meant?
> >>
> >> Yes, this is what we do. :-)
> >> Btw, there is a kind of small documentation of the pkg-config setup at
> >> https://projects.coin-or.org/CoinTLC/wiki/BuildSystemUsingPkgConfig
> > 
> > I have not studied it line by line but, I will do that as soon as I have
> > more time. 
> > What comes to my mind when I see that is that it would be nice to stick
> > to "standard" macros more than creating custom ones if it is possible.
> > I know that my macro contribution doesn't really show that I am a
> > supporter of this approach, but at least let's not forget advantages of
> > simplicity :-)
> > 
> >> I think that gnulib has some interesting features that are missing in
> >> the pkg-config setup. The main hurdle from my point of view is that
> >> the
> >> .la files do not communicate header paths.
> >> I agree that this may be ok for building against installed projects
> >> where the directories of the headers can be deducted from the library
> >> paths. But for building against an uninstalled project, something is
> >> missing.
> >> I don't know, maybe one should abandon the idea of building against
> >> uninstalled projects, at least with gnulib? The long term goal is to
> >> get
> >> rid of this anyway, if I remember correctly.
> > 
> > As I say, it would be nice to have it. Imagine that you have a distro
> > that doesn't have any COIN stuff in their repos. And imagine the pain of
> > a user trying to compile some of it and therefore having to deal with
> > all of COIN dependencies... 
> > I think that the macro that I have provide a good start.
> > 
> >>
> >> Stefan
> > 
> > Sorry for my lack of flexibility, but I have been busy above average the
> > past week, it should settle down a bit now.
> > Regards,
> > Matej
> > 
> > 
> > 
>