CMPUT 206 - Introduction to Digital Image Processing- Assignment 3: Bayer Filter, Dithering and Geometric Operations
CMPUT 206 (LEC B1 Winter 2023): Assignment 3: Bayer Filter, Dithering and Geometric Operations
Assignment 3: Bayer Filter, Dithering and Geometric Operations Opened: Saturday, 11 February 2023, 12:00 AM Due: Monday, 27 February 2023, 11:59 PM
For this assignment, you have to implement your code in the provided template python scripts. These perform the loading of images and plotting the outputs for you. Make sure you complete all of the TODO statements, and that your output matches the expected.
In this assignment, you will perform color image reconstruction using Bayer pattern and color reproduction using Dithering. (I) BasicBayer (25%): In this part we reconstruct this RGB color image (this original color image is only for your reference, you will not use it inside your code) given a corresponding Bayer pattern image (your code takes the Bayer pattern as input). The Bayer pattern used is GRBG. In this part, the task is to reconstruct green, red and blue channels from the Bayer pattern image. The reconstruction of channels should be done using these Green, Red, and Blue image masks, where black color has been used for valid entries, and the white for empty entries. These masks together form the Bayer color pattern, viz., GRBG.
You should complete the provided template python script. Interpolate the empty entries in the reconstructed green, red and blue channels independently of each other. Call the channels IG, IR, andIB respectively. Next, combine these channels into a full RGB color image and display this image.
The details about Bayer filter and the reconstruction process are provided in the presentation Bayer Filter and Demosaicing.pdf. You can also refer to the Wikipedia pages on Bayer filter and demosaicing. In case you are interested in even more details, here's a nice reference: http://www.site.uottawa.ca/~edubois/lslcd/
Note: In this assignment, you will implement a modified version of the interpolation algorithm, where values from all valid neighbours (shaded pixels in the mask) are averaged.
As a simple scheme of pixel interpolation, you can follow these rules:
- For reconstruction of the green channel IG, note with respect to the pattern Green that the green channel value at location B can be interpolated as:B = (A+C+F)/3. Similarly, D = (C+H)/2. For interpolating the green channel value at location G, use G = (F+C+H+K)/4, etc.
- For reconstruction of the red channel IR, with respect to the pattern Red, use these rules of interpolation: C = (B+D)/2, F = (B+J)/2, G = (B+D+L+J)/4, etc. Note that for the red channel here, the first column and the last row are entirely empty. You need to fill them in by copying the second column and the second last row, respectively.
- For reconstruction of the blue channel IB with respect to the pattern Blue, use these rules:F = (E+G)/2,I = (E+M)/2,J = (E+G+O+M)/4, etc. Notice that here the last column and the first row are entirely empty. So, fill them by copying to the second row and the last but one column, respectively. Expected output:
(II) Floyd-Steinberg dithering (25%): Complete the provided template python script.to read this RGB image and implement a Floyd-Steinberg dithering to change the representation of this image.
The original algorithm uses the error distribution on the left, but you need to implement the one on the right:
The template script:
- Dynamically calculates an N-colour palette for the given image (in the notebook the variable nColours determines the number of colours in the palette).
- Uses the KMeans clustering algorithm to determine the best colors.
- Makes a kd-tree palette from the provided list of colors.
- Applies the Floyd-Steinberg method to change the representation of the image. More details about Dithering techniques can be found in the Wikipedia pages on Dither and Palette. Expected output:
(III) Affine Transformation (25%): Rotation and Scaling are the specializations of affine transformation that can be applied to an image. In this assignment, you will perform image rotation, scaling, and skewing.
You should complete the provided template python script. Read this color image 1. Create a Rotation transformation matrix T_r (75 degrees counter-clockwise) 2. Create a Scale transformation matrix T_s which scales the placement of the points by 1.5 times in the x-direction and 2.5 times in the ydirection. 3. Create a Skew transformation with parameter = 0.2 in both directions
- Combine the transformations: The neat thing about affine transformations being essentially linear transformations is that you can combine the transformations and apply them in one step. To demonstrate this you should apply the dot product (matrix multiplication) of the previous three transformation matrices. Apply the combined transformation in one step to the spatial domain of the image data.(Do not use any built-in function to apply transformation). Plot the image after applying the transformation. The result obviously diminishes the quality of the original image as you can easily see discontinuities in the pixel intensities. 5. Perform backwards mapping of the pixel coordinates in the transformed image using the inverse of the combined transformation matrix T to find the corresponding coordinates in the original image. Use this mapping to obtain the pixel values in the transformed image through bilinear interpolation. (You can use any built-in function for this such as skimage.transform.warp) Expected output:
(IV) Use skimage-Python to merge two pictures (25%): In this exercise, you will need to write the code that can perform image stitching. You should complete this code. Download im1 and im2 for your use. There are a standard procedure and algorithm for image stitching, which can be briefly summarized as the following steps:
Step 1) Detect local features with BRIEF or ORB in the two images I1 and I2. You can also use any other feature extractors like SIFT or SURF. Step 2) Perform feature matching between I1 and I2, based on (approximate) nearest neighbour search, to generate a set of putative matching feature pairs. Usually, the similarity or distance between two feature descriptors is considered as the metric for matching. Apply further constraint, either geometrical or non-geometrical, to exclude outliers (wrong matches) and retain inliers (true matches). Step 3) Compute the homography matrix between I1 and I2 based on the true matches. A homography matrix is a 3-by-3 matrix that defines the transformation between the two images, i.e., H12 [x1, y1, 1]T = w [x2, y2, 1]T where (x1, y1) and (x2, y2) are the image coordinates of the matching features in the two images, H12 is the homography matrix and w is a scaler. Step 4) Perform image stitching based on the homography matrix. For this, you will need to calculate the new image coordinates for every pixel in I1, with respect to I2 coordinate system. If H12 is calculated correctly, you should expect that after transformation, the features in I1 now have the same coordinates to their correspondences in I2, i.e., they are translated to the matching places.
The resulting stitched image should look similar to this.
