[ptx] Super-resolution images from many registered digicam images

[John Bäckstrand]

Sebastian Nowozin wrote:

>Hello,
>
>
>I am dabbling with the idea to implement a resolution increase/fusion
>algorithm to combine two or more digicam images to one higher resolution
>image.
>
>  A friend of mine suggested the following simple procedure: 1. Do a subpixel
>registration of all the images,  2. Transform them (using nona),  3. Upscale
>them using a good interpolation algorithm,  4. Merge them by averaging.
>
>Now two questions.  First, do you think this procedure would practically work?
>I.e. is the interpolation algorithm nona uses precise enough?  Second, does
>nona support sub-pixel control point coordinates?  (With autopano-sift I can
>easily output subpixel coordinates for the keypoints).
>
>
>I found some papers (http://ivrgwww.epfl.ch/publications/vsv04.pdf and
>http://citeseer.ist.psu.edu/675638.html), but their algorithm is targeted at
>aliasing-artefact removal and they conclude:
>
>  "The optical systems that are mostly used nowadays limit the application of
>   this super-resolution technique in digital cameras. Because aliasing
>   artifacts are very disturbing, the optical system of most digital cameras
>   is designed such that all high frequencies are removed. The entire image is
>   slightly blurred, but no more aliasing is visible. This of course causes
>   our algorithm to fail, because it relies exactly on this high frequency
>   information."
>
>
>Other papers are (keyword "super resolution"):
>http://citeseer.ist.psu.edu/patti97super.html
>http://citeseer.ist.psu.edu/nguyen00numerical.html
>http://citeseer.ist.psu.edu/farsiu03fast.html
>http://citeseer.ist.psu.edu/zomet01robust.html
>http://citeseer.ist.psu.edu/borman98superresolution.html
>http://citeseer.ist.psu.edu/319998.html
>http://citeseer.ist.psu.edu/zomet01superresolution.html
>
>
>However, I would like an expert here point me at the current state-of-the-art
>for creating super-resolution images from normal digicam images (ie. without
>any special equipment besides a tripod).
>  
>
I am currently writing a review on super-resolution techniques as a 
project in school, but Im afraid most of the math is much beyond me. I 
havent really started doing the real work for this, but I have read 
several SR papers before, and I have also thought about using 
autopano-sift for sub-pixel registering, however I didnt know if the 
algorithm allowed that at all.

The amount of SR software available is very low, and that is something 
which keeps surprising me, since the research on this has been going on 
for quite some time. I know of ccdtools, registax 2 and imagesplus, 
though neither of them is specifically aimed at enhacning the spatial 
resolution.

And yes, anti-aliasing filters is of course a problem: for example: imaging a very high-frequency pattern with my digicam produces a flat grey, which of course makes it impossible to extract the pattern again. The filter is not perfect, however, and it allows for subtle aliasing. Though there are cameras without filter, notably Sigma SD9.

I dont really understand the math, as I said, so I dont know if sub-pixel shifts really can produce anything new at all if the anti-aliasing filter is "perfectly" blocking aliasing frequencies (I do realise almost no cam has this).

---
John Bäckstrand