CS 180 Project 2

Two finite difference kernels were implemented as NumPy arrays to compute partial derivatives:

• dx = np.array([[1, -1]]) for horizontal changes
• dy = np.array([[1], [-1]]) for vertical changes

These kernels were applied to the original image using scipy.signal.convolve2d with the parameter mode='same', resulting in two images representing partial derivatives in the x and y directions.

To create a single edge image, the gradient magnitude was calculated at each pixel using:

np.sqrt(dx ** 2 + dy ** 2)

This operation effectively computes the L2 norm of the gradient vector formed by corresponding pixel values from the two partial-derivative images, producing the final edge-detected image.

Outputs

I took a different approach to edge detection by pre-processing the Gaussian kernels:

1. First, I convolved the Gaussian kernels with the dx and dy finite difference kernels. This produced two new kernels: gaussian_dx and gaussian_dy.
2. These new kernels effectively represent the partial derivatives of the Gaussian kernel with respect to x and y.
3. Next, I applied these modified kernels to the original image through convolution. This step generated two partial derivative images.
4. Finally, I combined these partial derivative images into a single edge image by calculating the magnitude of the gradient at each pixel.

This technique allowed me to integrate the Gaussian smoothing and differentiation steps, potentially offering a more efficient edge detection process. I also experimented with different values for kernel size and sigma and visualized the differences below:

Outputs

As expected, the results of the finite difference operator and derivative of Gaussian filter were quite similar in goal. The key difference was that the smoothing prevented additional artifacts and improved edge detection. For concrete differences, observe the "sky" in both pictures. Without smoothing, there are some white spots. Also, the edge corresponding to the back of the cameraman is captured in greater detail with smoothing.

Here's how I sharpened an image:

1. Blur the original image by convolving it with a Gaussian kernel.
2. Extract high-frequency components: edges = img - img_blur
3. Enhance the image: sharpened = img + alpha * edges

Where alpha is a constant that controls the sharpening intensity. I experimented with the value of alpha until it looked aesthetically pleasing.

Sharpening

For the first set of pictures, I set ksize=10 and sigma=2 when blurring and used alpha=1. This was done step-by-step, and the results were similar when using the filter.

Custom Images

I found this segment particularly cool and picked two distinct pictures. The first is of me on a sailboat, and the second is a picture of Cambridge, UK, I had taken this year.

Resharpening

When working with the original image, the effects of sharpening were prominent. As the details outline indicates, we are adding emphasis to various bricks of the Taj Mahal as well as the lined bushes near it. If we blur the original image, we lose some information, and so we cannot expect to reconstruct details we didn't have access to. Still, the filter does a good job at highlighting the prominent edges.

The approach here was to blur im1 to obtain a low pass filtered version of it, with sigma=sigma1. Then, I constructed the laplacian of im2 by computing the difference between the original and a blurred im2, with sigma=sigma2.

Outputs

Cats are great. Both Nutmeg and a random cat from the internet formed a pretty nice hybrid image with my LinkedIn picture.

I also wanted to experiment with pictures taken moments apart to see if I could communicate motion. The swing up by Big C provided a perfect opportunity. After a bit of alignment and tuning of sigma1 and sigma2, the results were:

Failure

When combining the flowers below, I had higher hopes. I thought the edges would be captured well in the high-pass filter, and the intricacy could provide the perception of two different flowers.

Fourier Analysis

I chose to visualize the Fourier transforms of the swing pictures below:

Art?

As a quick bonus, I attempted to combine these two pictures from within and outside a water tower. It didn't work as expected, but with some normalization and blurring, it felt Picasso-esque.

Here, I built up the gaussian and laplacian stacks like so:

This pseudocode:

1. Initialized the gaussian list with the input img.
2. Looped through the depth range, performing the following steps:
   • Blurred the previous gaussian image to get the current gaussian image.
   • Calculated the laplace image as the difference between the previous and current gaussian images.
   • Normalized the laplace image and stored it in the norm_laplace list.
3. Appended the final gaussian image to the laplace list.
4. Returned the gaussian, laplace, and norm_laplace lists.

I did not include the bottom-most blurred gaussian layer in the normalized laplace stack while displaying, though it is needed to reconstruct the original image with the laplacian stack.

Outputs

I defined my blend function such that the inputs were two image filenames, a 2D mask, and several optional parameters to control the size, blurring, and the output format. Rather than deal with alignment or cropping, I worked with all square images and performed resizing before blending.

Big picture, my function did the following:

1. Load and process the two input images to resize them to a common size and convert them to the appropriate data format.
2. Generate Gaussian, Laplace, and normalized Laplace image stacks for both input images using the generate_stacks function.
3. Resize the 2D mask to the same size as the input images and generate a Gaussian pyramid of the mask.
4. Blend the Laplace image stacks of the two input images using the Gaussian mask pyramid, creating a combined Laplace stack.
5. Collapse the combined Laplace stack into a single output image by summing the Laplace images and clipping the result to the valid pixel value range.
6. Optionally, return the output image and the normalized combined Laplace stack.

The key idea behind this function is to blend the two input images by decomposing them into Gaussian and Laplace image pyramids, and then using a mask to selectively combine the Laplace components of the two images. This allows for a more seamless and natural-looking blend, especially around the edges and boundaries defined by the mask.