3D Photos - Pumpkins

In this post, I want to show that you can use my line of software to generate tweeners for lenticular imaging even if the input stereo pair is large (3591x2432 pixels). We're gonna create a left and right (full size) depth map with dmag5, improve the quality of the depth maps with dmag9b, and then generate the intermediate frames (the tweeners) with fsg6 using the full size images. Here, I am using dmag5 as the automatic depth map generator but I could have used (without much difference) dmag2 or dmag6.

Left image of a stereo pair taken with a Fuji W1 by Mike.

Right image of a stereo pair (also taken with a Fuji W1 by Mike).

This original stereo pair was aligned with SPM (StereoPhotoMaker) but if you load the 2 images in Gimp as layers and switch between the two, you can see that corresponding features in the images are not moving horizontally. This is not good for automatic depth map generation which relies on matches being along horizontal lines (scanlines). As a matter of principle, I always rectify my stereo pairs prior to generating a depth map. Here, I am using a modified version of er9b which is not as aggressive (This version will be released soon, hopefully.)

Rectified left image.

Rectified right image.

Left and right depth maps generated by dmag5 using the following parameters:

image 1 = ../image_l.png
image 2 = ../image_r.png
min disparity for image 1 = -70
max disparity for image 1 = 6
disparity map for image 1 = depthmap_l.jpg
disparity map for image 2 = depthmap_r.jpg
occluded pixel map for image 1 = occmap_l.jpg
occluded pixel map for image 2 = occmap_r.jpg
radius = 32
alpha = 0.9
truncation (color) = 30
truncation (gradient) = 10
epsilon = 255^2*10^-4
disparity tolerance = 0
radius to smooth occlusions = 9
sigma_space = 9
sigma_color = 25.5
downsampling factor = 2

Notice the downsampling factor equal to 2 (enables a large cpu gain without much loss in accuracy) and the relatively large radius (the large the images, the larger the radius should be).

I think we can improve on the depth maps, especially as far as the object boundaries are concerned. We are gonna use dmag9b to improve the depth maps. Note that we could also have used dmag2 (or the other edge preserving smoothers) but I like the "destructive" nature of dmag9b.

Left and right depth maps generated by dmag9b using the following parameters (for the right image):

reference image = ../../image_r.png
input disparity map = ../depthmap_r.jpg
sample_rate_spatial = 32
sample_rate_range = 8
lambda = 0.25
hash_table_size = 100000
nbr of iterations (linear solver) = 25
sigma_gm = 1.4142
nbr of iterations (irls) = 32
radius (confidence map) = 12
gamma proximity (confidence map) = 12
gamma color similarity (confidence map) = 12
sigma (confidence map) = 2
output depth map image = depthmap_r_dmag9b.jpg

Now that we are somewhat satisfied with the depth maps, it's time to generate the in-between frames for lenticular creation. We're gonna use fsg6 which uses both left and right depth maps. Be careful though as the right depth map must be inverted.

Inverted right depth map (needed by fsg6).

This is an animated gif showing 12 frames in between (and including) the left and right images generated by fsg6 using the following parameters:

Left image = ../../../image_l.tiff
Right image = ../../../image_r.tiff
Left disparity map = ../depthmap_l_dmag9b.tiff
Right disparity map = ../depthmap_r_dmag9b_inv.tiff
Minimum disparity = -70
Maximum disparity = 6
Number of frames = 12

The frames were reduced to a more manageable size of 600x406 pixels to produce the animated gif.

It should be noted that one could have used a frame sequence generator like fsg4 to produce the frames (on the left and right side of the left image using the left depth map). The problem with this type of frame sequence generators which only makes use of a single depth map is that the inpainting that occurs in the frames looks heavily blurred. The only advantage is that the parallax (3d effect) can be increased as much as desired artificially.