[ptx] Band Blending [was: Hugin wishlist, RFC]

[Andrew C Mihal]

Here is the direct link to the Burt & Adelson paper:

http://www-bcs.mit.edu:16080/people/adelson/pub_pdfs/spline83.pdf

Citeseer usually has links to the full article text in the upper right 
hand corner of the page.

> > I even managed to find an implementation in the (old) Utah Raster Toolkit.
>
> Did You prepare makefiles for linux, so some peoples could take a look on
> it (me included).

No. I installed the source RPM that I found on rpmfind.net. I didn't let 
RPM rebuild the package, I just did an install of the source. This puts a 
tar and a couple patches in your /usr/src/RPM/SOURCES directory.

There is a binary RPM too but it will segfault because of the malloc/free
bug. The bug is in the librle portion of the URT package that deals with
manipulating the RLE file format, not in pyrmask itself I think.

If you want to repeat what I did to make it work, do the following:

1) unpack the tar that comes out of the source rpm.
2) apply the urt-config.patch and the urt-fixes.patch that come with the 
rpm. Do not apply the other patches.
3) Edit lib/rle_hdr.c. Go to the rle_header_clear function and comment out 
the three function calls to free.
4) run "./Configure config/urt".
5) run "make".

That should do it. You will get a pyrmask.out executable in the tools 
directory.

To make the rle files from tiff files I used tifftopnm and pnmtorle from 
the netpbm package. 

You also need to supply a mask file which indicates the transition line 
between the two images. I made this by hand in gimp. This is a black and 
white image of the same size as the two input images. The mask should be 
all white where the first image has greater weight, and all black where 
the second image has greater weight. The actual 50% transition line will 
be at the border between black and white. You want the border to be as far 
away as possible from the edge of the region where the images intersect. 
A good start would be to put it right down the middle of the transition 
region. I probably also makes sense to make the line be far away from 
locations in the overlap region where the stitching is poor. These are the 
places you see in gimp when you use the difference overlap method.

The second paper I referenced talks about caluclating the ideal transition 
mask - but doesn't consider moving the transition away from stitching 
imperfections.

Once you get rles for the left, right, and mask images you can run 
pyrmask. Then convert the result back to pnm with rletopnm. gimp can read 
the pnms directly. But one of these conversion steps loses the alpha 
channel and you get a black background.

Actually, if not for this black background I probably wouldn't have 
noticed a big problem. Two images we get back from hugin typically overlap 
on one edge but not exactly - for example two images side-by-side that are 
not perfectly aligned vertically. Say the left image is higher than the 
right. Then there is a little bit of the left image that sticks above the 
overlap region and a little bit of the right image that sticks below the 
overlap region. Now it is impossible to draw the transition line between 
these images without coming close to these areas that don't overlap. So 
the pyramid blender will still blend those little bits, but only one image 
contributes data here - the other is supposed to be transparent. Since the 
converters swap black for transparent, the result is that these parts of 
the picture get darkened because the blender is blending against pure 
black.

This is not a big deal for single-row images since you're probably going
to crop these regions, but for multi-row it may become a problem. It will
think those areas that were blended to black are really what the picture
is supposed to be there. I think you have to treat the transparent regions
of the input images separately, and not just blend the alpha channel the
same way as the rgb channels. The blender must identify pixels outside the
intersection of the two images but inside the blending region defined by
the mask and delete them so they don't mess up the next row.

What I'd like to do is reimplement pyrmask so that it works with tiffs 
directly. Then I'll make it deal with the alpha channel correctly. Goal 
three is to do automatic mask calculation using Medial Axis Transform on 
the input image boundaries, and hopefully consider locations where the 
stitching is suboptimal too. I can probably put in an evening or two a 
week, I'll keep you posted.

Andrew

---------------------------------
Andrew Mihal
www-cad.eecs.berkeley.edu/~mihal
mihal at eecs.berkeley.edu