1. Homepage
  2. Programming
  3. MAEG5720 Computer Vision in Practice - Mini-Project2: Panorama

MAEG5720 Computer Vision in Practice - Mini-Project2: Panorama

Engage in a Conversation
CUHKMAEG5720Computer Vision in Practicepanorama

MAEG5720 Computer Vision in Practice CourseNana.COM

Mini-Project2 CourseNana.COM

Aim: The field of view of the image is limited by your lens. However, we could solve this issue by combining multiple images together to form a panorama. The aim of this assignment is to automatically stitch the images acquired by a panning camera.
CourseNana.COM

Image 1 Image 2 CourseNana.COM

Stitched Image CourseNana.COM

The Algorithm outline: CourseNana.COM

MAEG5720 Computer Vision in Practice CourseNana.COM

  1. Choose one image as the reference frame CourseNana.COM

  2. Estimate the homography between reference image and the other image CourseNana.COM

    • -  Detect the local features and extract the feature descriptor for each feature point in both images CourseNana.COM

    • -  Matches the feature descriptors between two images CourseNana.COM

    • -  Robustly estimate homography using RANSAC CourseNana.COM

    • -  Warp the image using the homography into the reference frame. CourseNana.COM

      Tips and detailed description of the algorithm:
      (1) Choose the reference image:
      Since we are working with two images, you are free to choose the CourseNana.COM

      reference frame from any image. CourseNana.COM

      Possible extension :In case you would like to extend this algorithm to multiple images in your spare time, you can choose the middle frame as the reference frame to minimize the distortion of the images. CourseNana.COM

Also, to ensure you have larger area of overlapping, you can use chain homographies. For CourseNana.COM

separate it into two steps. i.e. H = H H 24 34 23 CourseNana.COM

example: If you wish to calculate the homography between image 2 to image 4, you can CourseNana.COM

(2) Estimation of homography CourseNana.COM

(a) Detecting local features in each image CourseNana.COM

To extract the local features of the image, one of the common methods is to use SIFT descriptor. Since SIFT descriptor is a patented technology and it is not available in MATLAB, please download the library from VLFeat.org (https://www.vlfeat.org/download.html) and follow the instruction on the installation page (https://www.vlfeat.org/install-matlab.html). CourseNana.COM

Once the setup is done, you can use the following code for SIFT descriptor extraction. CourseNana.COM

   I = single(rgb2gray(I));

[f,d] = vl_sift(I) ; where CourseNana.COM

please refer to the Shape Descriptor lecture notes for detail. CourseNana.COM

f is a 4 × 𝑛𝑛𝑛𝑛𝑛𝑛_𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑 matrix consists of 𝑥𝑥, 𝑦𝑦, 𝜎𝜎, 𝜃𝜃 of each SIFT point d is a 128 × 𝑛𝑛𝑛𝑛𝑛𝑛_𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑 matrix CourseNana.COM

MAEG5720 Computer Vision in Practice CourseNana.COM

(b) Matching the descriptors between images CourseNana.COM

Once the SIFT descriptors are extracted, you have to find the pairs of descriptors between images that looks similar. This will probably be corresponding each others. One of the method to do is to compare the sum of square distance between two descriptors and adopt the near neighbour distance ratio (NNDR) of 1.5 CourseNana.COM

𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 = 𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑 𝑜𝑜𝑜𝑜 𝑆𝑆𝑑𝑑𝑑𝑑𝑜𝑜𝑑𝑑𝑑𝑑 𝑀𝑀𝑑𝑑𝑑𝑑𝑑𝑑h 𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑 𝑜𝑜𝑜𝑜 𝐹𝐹𝑑𝑑𝐹𝐹𝑑𝑑𝑑𝑑 𝑀𝑀𝑑𝑑𝑑𝑑𝑑𝑑h CourseNana.COM

The pseudo code below gives you some hints on the algorithm outline: CourseNana.COM

function [matches scores] = match_descriptor(d1, d2) CourseNana.COM

%Define the threshold (for example 1.5) CourseNana.COM

Num_of_matches =0 CourseNana.COM

for each i in d1 for j each d2 CourseNana.COM

find the best match and second best match if NNDR>threshold CourseNana.COM

add the i and j into matches[num_of_match]
add the
𝑆𝑆𝑆𝑆𝑁𝑁(𝑑𝑑𝑑𝑑, 𝑑𝑑𝑝𝑝)score into scores[num_of_match] num_of_match=num_of_matches+1 CourseNana.COM

end CourseNana.COM

end end CourseNana.COM

return matches and scores
(c) Robust Estimation of the homography by RANSAC CourseNana.COM

Random samples of 4-points to compute each homography hypothesis. You will need to write the following function CourseNana.COM

𝐻𝐻 = 𝐹𝐹𝑑𝑑𝑛𝑛𝑑𝑑𝐻𝐻𝑑𝑑𝑛𝑛𝑑𝑑𝐹𝐹𝑑𝑑𝐹𝐹𝑑𝑑h𝑦𝑦(𝑥𝑥1, 𝑦𝑦1, 𝑥𝑥2, 𝑦𝑦2, 𝑥𝑥3, 𝑦𝑦3, 𝑥𝑥4, 𝑦𝑦4, 𝑥𝑥𝑑𝑑1, 𝑦𝑦𝑑𝑑1, 𝑥𝑥𝑑𝑑2, 𝑦𝑦𝑑𝑑2, 𝑥𝑥𝑑𝑑3, 𝑦𝑦𝑑𝑑3, 𝑥𝑥𝑑𝑑4, 𝑦𝑦𝑑𝑑4) CourseNana.COM

Which takes the 4 SIFT points from image 1 and 4 corresponding SIFT points from image 2. In order to compute the elements of 𝐻𝐻, You will need to setup the linear system of equations (i.e. Ah = 0). There are totally 8 elements for the 3x3 homography matrix. Each pair of CourseNana.COM

MAEG5720 Computer Vision in Practice CourseNana.COM

correspondence gives you 2 equations and therefore you need totally 4 pairs of correspondences. The A matrix is of the form below: CourseNana.COM

A=[
-x1 -y1 -1 0 0 0 x1*xp1 y1*xp1 xp1;
CourseNana.COM

0 0 0 -x1 -y1 -1 x1*yp1 y1*yp1 yp1; : CourseNana.COM

-x4 -y4 -1 0 0 0 x4*xp4 y4*xp4 xp4; CourseNana.COM

0 0 0 -x4 -y4 -1 x4*yp4 y4*yp4 yp4]; CourseNana.COM

The solution can be obtained using SVD of A. The solution is the right singular vector corresponding to the least singular value. CourseNana.COM

[U,S,V] = svd(A);
H=V(:,end);
H=reshape(H,3,3)’;
Please refer to Lecture note 9 for more detail.
CourseNana.COM

For the RANSAC, you can just use a very simple implementation with a fixed number of sampling iteration is sufficient. You should output a single homography matrix H with the most number of inliners. The pseudo code for the RANSAC is given below: CourseNana.COM

function [bestH, num_of_inliners] = RANSAC ( f1, f2, matches)
define the parameters (PixelError, num_Iteration, num_pts_cal_H=4) max_inliner =0;
CourseNana.COM

for each i in num_Iteration
random select 4 points and compute H_hypothesis with 𝐹𝐹𝑑𝑑𝑛𝑛𝑑𝑑𝐻𝐻𝑑𝑑𝑛𝑛𝑑𝑑𝐹𝐹𝑑𝑑𝐹𝐹𝑑𝑑h𝑦𝑦 function CourseNana.COM

inliner =inliner+1 CourseNana.COM

reset inliner ;
for each pair in the matches list CourseNana.COM

find_out the coordinate for each matches 𝑃𝑃1 𝐹𝐹𝑛𝑛𝑑𝑑 𝑃𝑃
CourseNana.COM

CourseNana.COM

MAEG5720 Computer Vision in Practice CourseNana.COM

if inliner>max_inliner
bestH = H_hypothesis number_of_inliners = inliner CourseNana.COM

end CourseNana.COM

end CourseNana.COM

(d) Warp the image using the best homography into the reference frame CourseNana.COM

Once you obtained the best homography, the last step is to warp the image and stich the image together. You will need to extend the images in the reference frame to accommodate the other image. One of the methods is to use padarray function in MATLAB to pad zeros into the reference frame. CourseNana.COM

Then we use backward warping projection. This could avoid holes in the warped image. For each pixel in the reference image, calculate the corresponding pixel location in the second image using the best homography obtained by the RANSAC function. You can compare the value of the wrapped pixel with the corresponding value in the other image and simply pick the pixel value with maximum value from both images. This tends to produce less artifacts than taking the average of the warped images. CourseNana.COM

MAEG5720 Computer Vision in Practice CourseNana.COM

function stitchedImage = stitch(im1, im2, homography) copy the reference image (im1) to the stitchedImage pad the stitchedImage with zeros
for each i column in stitchedImage) CourseNana.COM

for each j row in stitchedImage
calculate the corresponding position in im2 using homography p2 = homography* p1
if (p2 is within range of stitchedimage)
CourseNana.COM

Take the higher intensity out of the p1 and p2 as the pixel value end CourseNana.COM

end; end; CourseNana.COM

Crop the image to remove extra boundary which are black(pixel value=0) CourseNana.COM

MAEG5720 Computer Vision in Practice CourseNana.COM

The final result should look like this. CourseNana.COM

The stitched Image CourseNana.COM

Bonus Question: (20%) CourseNana.COM

You could use the above algorithm to stitch more than two images. Please select your own three or more images for stitching and select the best reference image. Please use another main_multiple.m file for this bonus question for clarity issue. CourseNana.COM

The stitched Image (3 images) CourseNana.COM

CourseNana.COM

What to submit: CourseNana.COM

MAEG5720 Computer Vision in Practice CourseNana.COM

Please submit a compress file with the following: CourseNana.COM

  • -  main.m for the main function and read in the image for processing. (20%) CourseNana.COM

  • -  main_multiple.m for the main function and read image for processing multiple CourseNana.COM

    images(20%) -bonus CourseNana.COM

  • -  match_descriptor.m for matches the descriptors (20%) CourseNana.COM

  • -  FindHomography.m for solving the homography using 4 correspondences (20%) CourseNana.COM

  • -  RANSAC.m for robust estimation of the best homography between two images (20%) CourseNana.COM

  • -  Stitch.m for stitching the images together.(20%) CourseNana.COM

  • -  A pdf file for the result and discussion you would like to share. CourseNana.COM

  • -  Images you are using. CourseNana.COM

    Please ensure your program is executable! Also compress your files into a zip file and use your student id as the name. CourseNana.COM

    You will have two weeks for this assignment and tutorial will be arranged to help you do this assignment. Good luck!  CourseNana.COM

Get in Touch with Our Experts

QQ QQ
Wechat WeChat
Whatsapp Whatsapp
CUHK代写,MAEG5720代写,Computer Vision in Practice代写,panorama代写,CUHK代编,MAEG5720代编,Computer Vision in Practice代编,panorama代编,CUHK代考,MAEG5720代考,Computer Vision in Practice代考,panorama代考,CUHKhelp,MAEG5720help,Computer Vision in Practicehelp,panoramahelp,CUHK作业代写,MAEG5720作业代写,Computer Vision in Practice作业代写,panorama作业代写,CUHK编程代写,MAEG5720编程代写,Computer Vision in Practice编程代写,panorama编程代写,CUHKprogramming help,MAEG5720programming help,Computer Vision in Practiceprogramming help,panoramaprogramming help,CUHKassignment help,MAEG5720assignment help,Computer Vision in Practiceassignment help,panoramaassignment help,CUHKsolution,MAEG5720solution,Computer Vision in Practicesolution,panoramasolution,