The purpose of this project was to extract and align three images, representing three color channels, to form a single color image. First, I implemented a single-scale version, which uses the SSD metric to score the displacement between images. I then aligned both the red and the green images to the blue one using this error metric. Then, because this exhaustive search method is very slow for high-quality TIF images, I implemented coarse-to-fine pyramid speedup, which continually halved the image size and looped over larger displacements for smaller image sizes. I then added together all the optimal displacements obtained at each level, weighted more strongly for the smaller image sizes.
I then observed that this method doesn't quite work for images such as Emir and selfie, because the images have different brightness values. To remedy this, I decided to apply a canny edge detector to the images before aligning them, which means the alignment is between two binary images. This fixed the problem and made Emir and selfie align properly. The difference for selfie is demonstrated near the bottom.
Furthermore, I implemented the techniques of automatic contrasting, automatic white balancing, and automatic border cropping. The images that follow have the contrasting and white balancing applied, but not the cropping. More explanation of my implementation of these extra tasks is given near the bottom.
1. Better features: use canny edge detection before SSD displacement calculation, instead of using raw pixel values.
2. Automatic contrasting: I determine the 5th and 95th percentile pixel values for each color channel, then set pixels below the 5th percentile to 0.0 and above the 95th percentile to 1.0. I then scale all other pixels appropriately.
3. Automatic white balancing: I first tried a white world approach, but that didn't work too well. I then utilized a gray world approach, where I calculated the gain of the red pixels as average green intensity over average red intensity, and gain of the blue pixels as average green intensity over average blue intensity. I then multiplied the red pixel intensities by the red gain, and similarly for the blue pixels.
4. Automatic border cropping: I again used a canny edge detector to obtain an edge map of the image. With this, I considered the first 10% of rows/columns from each edge, and found the highest-intensity horizontal/vertical pixel strip closest to the center of the image, and used this as the crop point. Unfortunately, a shortcoming of this algorithm is that if the border noise is not axis-aligned, no corresponding straight-line intensity will be very high. Also, if the noise is not very noticeable, an edge detector might not pick up on it.
