[ptx] Conserving verticals with many control points

[Rik Littlefield]

Damien,

I wrote parts of the panotools optimizer, and I think that hugin still 
uses the same approach.

There are two main classes of problems.

Class 1.  Your images stitch together nicely with each other via 
ordinary t0 control points, but the whole panorama happens to come out 
not "level".  In theory, and usually in practice, this can be entirely 
corrected by only two vertical controls, preferably about 90 degrees 
apart.  This is because the error contributed by t0 control points is 
not affected by overall rotation of the sphere.  If you have many 
ordinary control points, then the vertical controls are relatively weak, 
but still they are usually strong enough that the optimizer needs only a 
few of them.

Class 2.  Your images do *not* stitch together nicely with each other, 
so it is not enough to just level the whole panorama.  This is often due 
to parallax, misplaced control points, or uncorrected lens distortions.  
It can also be caused by subject motion, such as movement of trees or 
clouds that are often chosen by automatic control point pickers. 

If you find yourself needing lots of separate vertical controls, then 
you probably have class 2.  That is unfortunate.  Class 2 is very hard, 
because you are trying to force an attractive balance of fairly large 
errors, rather than find parameter values that will push all the errors 
very close to zero like in class 1.

The best advice is to avoid class 2.  When taking pictures, be sure that 
your camera rotates properly around the "no-parallax point" (entrance 
pupil of the lens).  Avoid placing control points on features that could 
possibly have moved between images.  Be sure that you do not have any 
misplaced control points.  (Study the lists of errors -- individual 
control points that have large errors are likely to be misplaced.)  Do 
not use "straight line" controls (type t3+).

If you cannot avoid class 2, then there is not a lot that the software 
can do to help you.  As you suggest, the optimizer could be extended to 
include explicit weights.  This would reduce the run time a little, but 
it would not change the underlying problem that class 2 really requires 
optimizing a human judgement "subjective" function (attractiveness) 
rather than a purely computational "objective" one (stitching error and 
leveling).

The one exception to all this is if you have long skinny panoramas, say 
20 or 30 horizontal frames in a single row.  In that case, small t0 
errors can accumulate to produce significant waves, even when the 
panorama is level overall.  But this is usually handled well by adding 
just one or two vertical controls per image.  From your description, I 
would guess that you don't have this situation.

Again, best advice is to avoid class 2.  If you shoot carefully and 
place ordinary control points carefully, you should need only two 
vertical controls..

Hope this is helpful,
--Rik

PS. Just to be sure the basics are covered...  To get proper leveling, 
you must allow the optimizer to adjust pitch, roll, and yaw of all 
images.  You can set the yaw of one "anchor image", but you must 
optimize pitch and roll even for that anchor. 

Damien Douxchamps wrote:

>Hi all,
>
>First post here, so hello again ;-)
>
>I've been getting mixed results with Hugin but finally I think I nailed
>it. My problem is always the same: how can I keep the verticals in a
>panorama? IOW, how can I avoid the panorama to look like a big wavy
>line?
>
>Yes, I know about the vertical/horizontal-line control points ("intra"
>points) ;-) But it seems ineffective when the number of classic control
>points ("inter" points) is comparatively large. In my case, I typically
>use 20 "inter" control points and 5 "intra" control points per image. If
>I use only 5 inter points and 5 intra points the result is straighter
>but the fit is poor.
>
>One quick hack I found is to simply duplicate the intra control points
>many times (editing the .PTO file). This nearly solved the problem:
>verticals are now much better, the panorama is not as wavy and at the
>same time the fit is good.
>
>This brings two questions:
>
>- do we have to use both H and V intra control points to keep the
>panorama straight (from a theoretical point of view)? Or can we afford
>to use only one type? In my panorama no horizontal feature can be used;
>this may lead to the ineffectiveness of the intra control points.
>
>- would it be useful to add some weighting in the solver? Duplicating
>points can work but when you have 5000 points it gets a bit slow.
>Besides, it's not very elegant... I don't know which method is used in
>Hugin (or panotools) but I know some linear and non-linear fitting
>techniques allow that.
>
>Thanks for your help,
>
>Damien
>
>  
>