[ptx] Conserving verticals with many control points

[Damien Douxchamps]

Hello Rik,

On Sat, 2006-04-15 at 09:46 -0700, Rik Littlefield wrote:
> Damien,
> 
> I wrote parts of the panotools optimizer, and I think that hugin still 
> uses the same approach.
> 
> There are two main classes of problems.
> 
> Class 1.  Your images stitch together nicely with each other via 
> ordinary t0 control points, but the whole panorama happens to come out 
> not "level".  In theory, and usually in practice, this can be entirely 
> corrected by only two vertical controls, preferably about 90 degrees 
> apart.  This is because the error contributed by t0 control points is 
> not affected by overall rotation of the sphere.  If you have many 
> ordinary control points, then the vertical controls are relatively weak, 
> but still they are usually strong enough that the optimizer needs only a 
> few of them.

This appears to be my case since the average error is .16 pixels (the
max is .73) for about 270 control points. The panorama looks very well
stitched too (except for the wavy horizon)

> Class 2.  Your images do *not* stitch together nicely with each other, 
> so it is not enough to just level the whole panorama.  This is often due 
> to parallax, misplaced control points, or uncorrected lens distortions.  
> It can also be caused by subject motion, such as movement of trees or 
> clouds that are often chosen by automatic control point pickers. 

My pano consists mainly in buildings so there is no problem like motion
here. Perspective can be tricky in the near field but carefully choosing
the points is not too difficult in my case.

> If you find yourself needing lots of separate vertical controls, then 
> you probably have class 2.  That is unfortunate.  Class 2 is very hard, 
> because you are trying to force an attractive balance of fairly large 
> errors, rather than find parameter values that will push all the errors 
> very close to zero like in class 1.

What I usually do is pick many points, then run the optimized and delete
the ones that yield a bad match. Then re-run the optimizer, ... This way
the bad points a filtered almost automatically.

> The best advice is to avoid class 2.  When taking pictures, be sure that 
> your camera rotates properly around the "no-parallax point" (entrance 
> pupil of the lens).  Avoid placing control points on features that could 
> possibly have moved between images.  Be sure that you do not have any 
> misplaced control points.  (Study the lists of errors -- individual 
> control points that have large errors are likely to be misplaced.)  Do 
> not use "straight line" controls (type t3+).

Its seems we do the same thing regarding the lists of errors ;-) With
hugin my control points are all t0 and t1 so I guess I'm not using
t3(+).

No problems with motion given the nature of the panorama.

> If you cannot avoid class 2, then there is not a lot that the software 
> can do to help you.  As you suggest, the optimizer could be extended to 
> include explicit weights.  This would reduce the run time a little, but 
> it would not change the underlying problem that class 2 really requires 
> optimizing a human judgement "subjective" function (attractiveness) 
> rather than a purely computational "objective" one (stitching error and 
> leveling).

Yeah, being familiar with camera calibration I would not even try to
deal with class 2 in the first place ;-)

> The one exception to all this is if you have long skinny panoramas, say 
> 20 or 30 horizontal frames in a single row.  In that case, small t0 
> errors can accumulate to produce significant waves, even when the 
> panorama is level overall.  But this is usually handled well by adding 
> just one or two vertical controls per image.  From your description, I 
> would guess that you don't have this situation.

I have 11 photos (4MP each) in a single row for a 360° panorama. The
vertical FOV for each photo is 36°. So maybe I'm having this problem
after all...

Maybe a solution would be to stitch photos 2 by 2 and then put these
bigger blocks together?

> Again, best advice is to avoid class 2.  If you shoot carefully and 
> place ordinary control points carefully, you should need only two 
> vertical controls..

I just tried that but it's still wavy. Duh. Is there a way to verify the
adequacy of vertical control points? (for example, which line angle was
detected) 

> Hope this is helpful,

Sure, I'm learning every day :-)

> --Rik
> 
> PS. Just to be sure the basics are covered...  To get proper leveling, 
> you must allow the optimizer to adjust pitch, roll, and yaw of all 
> images.  You can set the yaw of one "anchor image", but you must 
> optimize pitch and roll even for that anchor. 

I first adjust the position/attitude, then everything. The default
setting in Hugin is indeed to fix one yaw, and that's what I'm doing.

Thanks for your help,

Damien

> Damien Douxchamps wrote:
> 
> >Hi all,
> >
> >First post here, so hello again ;-)
> >
> >I've been getting mixed results with Hugin but finally I think I nailed
> >it. My problem is always the same: how can I keep the verticals in a
> >panorama? IOW, how can I avoid the panorama to look like a big wavy
> >line?
> >
> >Yes, I know about the vertical/horizontal-line control points ("intra"
> >points) ;-) But it seems ineffective when the number of classic control
> >points ("inter" points) is comparatively large. In my case, I typically
> >use 20 "inter" control points and 5 "intra" control points per image. If
> >I use only 5 inter points and 5 intra points the result is straighter
> >but the fit is poor.
> >
> >One quick hack I found is to simply duplicate the intra control points
> >many times (editing the .PTO file). This nearly solved the problem:
> >verticals are now much better, the panorama is not as wavy and at the
> >same time the fit is good.
> >
> >This brings two questions:
> >
> >- do we have to use both H and V intra control points to keep the
> >panorama straight (from a theoretical point of view)? Or can we afford
> >to use only one type? In my panorama no horizontal feature can be used;
> >this may lead to the ineffectiveness of the intra control points.
> >
> >- would it be useful to add some weighting in the solver? Duplicating
> >points can work but when you have 5000 points it gets a bit slow.
> >Besides, it's not very elegant... I don't know which method is used in
> >Hugin (or panotools) but I know some linear and non-linear fitting
> >techniques allow that.
> >
> >Thanks for your help,
> >
> >Damien
> >
> >  
> >
> 
> 
-- 
  _    Damien 'Takahara' Douxchamps, PhD
 ('-   Post-doctoral investigator
 //\   Image Processing Group, NAIST
 V_/_  http://chihara.aist-nara.ac.jp/