CS 6475: Computational Photography - A2: Pyramid Blending
Pyramid Blending
Synopsis
In this assignment, you will be putting together a pyramid blending pipeline that will allow you to combine separate images into a seamlessly blended image. The technique is based on the Burt & Adelson paper A Multiresolution Spline With Application to Image Mosaics. You are required to use your own unique images and design a custom mask that best supports your image blend.
To help you with this assignment, you should carefully read the two required technical papers, and watch the associated modules:
- “The Laplacian Pyramid as a Compact Image Code" (Burt and Adelson; 1983)
- “A Multiresolution Spline With Application to Image Mosaics” (Burt and Adelson; 1983)
- Module 04-01: Fourier Transforms
- Module 04-02: Blending
- Module 04-03: Pyramids
- Module 04-04: Cuts
Instructions
1. Implement the functions in the blending.py file.
returnYourName: return your Gradescope official namereduce_layer: Blur and subsample an input imageexpand_layer: Upsample and blur an input imagegaussPyramid: Construct a gaussian pyramid from the image by repeatedly reducing the inputlaplPyramid: Construct a laplacian pyramid by taking the difference between gaussian layersblend: Combine two laplacian pyramids through a weighted sumcollapse: Flatten a blended pyramid into a final image
🔴 You CANNOT use cv2.pyrUp or cv2.pyrDown anywhere in this assignment. Using forbidden functions will result in a significant deduction.
Notes
-
The blending procedure splits the input images into their blue, green, and red channels and blends each channel separately. You do not have to worry about dealing with three channels; you can assume your functions take in grayscale images.
-
You can execute the code that you write in
blending.pyby runningpython main.py. The script will look inside each subfolder underimages/source, looking for folders that have images with filenames that end with 'white', 'black' and 'mask'. For each such folder it finds, it will apply the blending procedure to them, and save the output to a folder with the same name as the input inimages/output/. For example,images/source/samplewill produce output inimages/output/sample. -
You may edit
main.pyas you wish to help you complete and troubleshoot your code. You will NOT submitmain.py, it is only here to aid you in completing this assignment. -
Along with the output blended image,
main.pywill create visualizations of the Gaussian and Laplacian pyramids for the blend. You may use or refer to these to answer questions in your report, in which case, please add them to the report in the Appendix. However, they are not required.
🟡 The docstrings of each function contains instructions, so read them carefully. You have a limited number of submissions for each project, so you are strongly encouraged to write your own unit tests. Do NOT try to use the autograder as your test suite. The test_blending.py file is provided to get you started. Your code will be evaluated on input and output type (e.g., uint8, float, etc.), array shape, and values. Be careful regarding arithmetic overflow!
2. Generate your own blended images using ORIGINAL inputs
You are REQUIRED to take and create your OWN images that are suitable for blending - you will need an ORIGINAL "black" and "white" image to combine, and an ORIGINAL mask.
The black and white images should retain EXIF for exposure, aperture and ISO, at a minimum.
Your images do not need to be large. We recommend that you keep each black and white image around 1 MB so that your code will not take forever to run, and so that your report and resources will stay under the size limit (see submission sections). You can use larger images, but they will be slow.
Mask Requirement
Create your own custom, non-trivial, "mask" image that defines the regions to combine. Your mask MUST be UNIQUE and should show that you have put thought into it. UNIQUE here means that you created the mask yourself. Do NOT submit a mask that you found online. Do NOT use the same or similar mask like the one we provided, in other words, simple masks such as masks that are split horizontally or vertically, or masks that are basic geometric shapes (ex: circle or oval), are not acceptable. Do NOT create a mask that you think looks cool (e.g. checkerboard), but it makes no sense with your images because you will be penalized. We expect the mask to relate to the images WELL. Run your images through the pipeline and include the results in your report.
-
You may use photo editing software to design your mask, and only your mask. Example programs are Gimp (open source, free), Photoshop, or many others. In the report, you will be asked to discuss the steps that you took to create your mask, so if you used software, make note of the steps you took and take screenshots if necessary.
-
If you implement your own code to create your mask, you must discuss your implementation in your report.
-
The remainder of this project must be done by you using your python functions, and NOT using any software.
🔴 Do NOT get any of your images (black, white, mask) off of the internet and try to submit them as your own. This is considered plagiarism and will be reported to OSI. Also, images in the images/source/sample directory are provided for testing -- do not use these images in your report.
3. Complete the LaTex Report
Use the A2-Pyramid_Blending_template.zip report template provided in the class repository. The template specifies all of the images, data, and questions you must answer. Follow the instructions in the Course Setup to upload it in Overleaf. If you have a different favorite LaTeX program, you may use it; the template should be generally compatible.
Report Requirements
- The total size of your report+resources must be less than 20MB for this assignment. If your submission is too large, you can reduce the scale of your images or report. You may resize your images using
cv2.resize. You can compress your report using Smallpdf. There will be an additional deduction if the 20MB total size limit is not met.- The total length of your Report is 5 pages maximum. This report is expected to be four pages for most students, unless you decide to submit extra observations and images in the appendix. (No extra credit, but the TA staff enjoys interesting work!)
- You are required to use the LaTex format for your report. Different report organization or changing section titles and questions will receive an additional deduction. You may add additional relevant information or images within the structure of the report, as long as all requirements are met.
- You are required to provide References. Everyone is expected to have at least two references; these should include StackOverflow answers that you relied on for specific commands, valuable tutorials, blogs, helpful Ed posts, and textbook references. Provide URLs and site or page names, typical book information and page numbers, etc. We should be able to easily view your references if needed.
- Give well thought out, detailed answers for your report discussions. For example, a one or two sentence answer about your mask creation is not acceptable. Put effort into your report answers. The quality of your report is a significant part of the assignment grade.
- Provide all required images in your report. Although we require submission of images in your
resources.zip, we also require you to present images in your report. If there are any missing images in your report, there will be a deduction. - Save your report as
report.pdfand make sure to submit it to BOTH Gradescope and Canvas. See Section 4 below.
