[Ipopt] Suitability of IPOPT for bundle adjustment problems in open source computer vision

[Keir Mierle]

Summary: IPOPT looked perfect for libmv[1]'s minimization needs, but
was not due to the CPL. This is not the first time this has happened
to IPOPT [2]. Can the licensing problems be resolved?

The longer story:

I'm interested in using IPOPT for solving bundle adjustment problems
in computer vision. Essentially, this involves solving simultaneously
the positions and orientations of a set of cameras and points in the
world that are visible in the cameras. The typical application of this
is matchmoving in the special effects industry [1]. The cost function
is usually a sum of squared reprojection errors; each error represents
the difference between the measured (2d) position for a point in an
image, and the position obtained by projecting a (3d) point into a
camera (where the 3d point and camera have been recovered via
structure from motion techniques).

The traditional method for solving bundle problems is to exploit the
fixed sparsity pattern in the Jacobian and run Levenberg-Marquardt to
optimize the camera and point positions [3]. However, this requires a
strict configuration of cameras and points. I would like to support
more general setups, and potentially constraints such as the cameras
must stay in a plane, or a subset of the points are on a sphere, etc.

At first IPOPT looked ideally suited for this, but after further
investigation I see it will not work because of licensing problems. I
am mailing in case the IPOPT developers have ideas. The two issues are
as follow:

1. The CPL is not GPL compatible. This is a problem because our
eventual plan is to link with Blender (GPL'd) [4] and provide a
matchmoving solution for them. Note that there are no open source
matchmovers, so this is a big deal. While the CPL is open source
according to the OSI definition, it is closed off to the expansive
ecosystem of GPL projects. That is unfortunate.

Would the IPOPT team consider relicensing under a more compatible open
source license like the LGPL, Apache 2.0, BSD, or MIT?

2. The sparse indefinite solvers that IPOPT requires also don't have a
suitable license. This may be a larger problem than 1.

Are there packages the IPOPT team is aware of that would be suitable
for use in IPOPT but are also liberally licensed? UMFPACK might be
sufficient, but I am not sufficiently qualified to say. I understand
that this may require writing an interface, and volunteer to do so.

Thanks for your input.

Keir

[1] libmv: http://code.google.com/p/libmv
[2] IPOPT passed over for Google's Summer of Code 2009:
http://www.nabble.com/-SoC09-Info--An-IPopt-interface-for-R-td22151556.html
[3] SBA: http://www.ics.forth.gr/~lourakis/sba/
[4] Blender: http://www.blender.org/