3D Photos - Hard at work

This article will demonstrate what Depth Map Automatic Generator 7 (DMAG7) and Edge Preserving Smoothing 7 (EPS7) can do. Recall that DMAG7 is based upon "Fast Bilateral-Space Stereo for Synthetic Defocus" by Jonathan T. Barron, Andrew Adams, YiChang Shih, and Carlos Hernández and that EPS7 is based upon "Domain transform for edge-aware image and video processing" by Eduardo Gastal and Manuel Oliveira.

The following stereo pair was provided by Jon Barron (and Google). The dimensions are 2432x1728. The min and max disparities are 0 and 25, respectively. This is not a large disparity range for a stereo pair of that size, meaning that there won't be a whole lot of occlusions to possibly worry about. It was probably taken with a small interaxial stereo camera.

Left image.

Right image.

Disparity map obtained with DMAG7 with spatial sample rate = 8, range sample rate = 32, radius = 12, lambda = 0.01, max number of iterations = 1000, hash table size = 10000.

Let's examine now the effects of applying edge preserving smoothing (with EPS7) on the depth map by playing with the filter spatial standard deviation and the filter range standard deviation. In the following, the spatial standard standard deviation is fixed (at 1000) and the range standard deviation is decreased logarithmically from 1000 to 10.

Filtered depth map with spatial standard deviation = 1000 and range standard deviation = 1000.

Filtered depth map with spatial standard deviation = 1000 and range standard deviation = 100.

Filtered depth map with spatial standard deviation = 1000 and range standard deviation = 10.

Now, let's fix the range standard deviation (at 1000) and decrease the spatial range deviation.

Filtered depth map with spatial standard deviation = 1000 and range standard deviation = 1000.

Filtered depth map with spatial standard deviation = 100 and range standard deviation = 1000.

Filtered depth map with spatial standard deviation = 10 and range standard deviation = 1000.

I personally kinda like the spatial standard deviation = 100 and range standard deviation = 1000 best, so this is the one we are gonna use to produce a wiggle using Wiggle Maker. Before we do that, it's probably a good idea to reduce the size of the images (left image and depth map) because you really don't want to create animated gifs from large images.

Animated gif (wiggle) obtained with Wiggle Maker.

To put things in perspective, below is the depth map obtained by Depth Map Automatic Generator 5 (DMAG5). I used to think stereo matching local methods (like DMAG5) could compete with global methods but not anymore. Up to now, the smoothness term used in global methods was really simplistic. You issue a penalty when the disparity of a given pixel differs from the disparities at immediately neighboring pixels but only if the disparity difference is small. A large disparity difference is assumed to be a legit depth change, which is a bit of stretch. Thanks to Fast Bilateral-Space Stereo, the smoothness term makes a lot more sense now since it is tied to a bilateral filter that takes into account location and color of neighboring pixels. This makes global methods really hard to beat now.

Depth map obtained with DMAG5 using radius = 12, alpha = 0.9, truncation (color) = 20, truncation (gradient) = 10, and epsilon = 1.