Monday, September 23, 2013

Depth Map Automatic Generator 3 (DMAG3)

DMAG3 is an automatic depth/disparity map generator based on the Graph Cuts methodology. It is an implementation of the algorithm described in "Computing Visual Correspondence with Occlusions using Graph Cuts" by Vladimir Kolmogorov and Rabin Zabih. Please refer to Stereo Matching and Graph Cuts and Stereo Matching and Graph Cuts (Part 2) for details about the implementation of such an algorithm, which, by the way, would have not been possible without the help of "Kolmogorov and Zabih's Graph Cuts Stereo Matching Algorithm" by Vladimir Kolmogorov, Pascal Monasse, and Pauline Tan and the accompanying code. I didn't code the actual mincut/maxflow solver. It comes straight from Vladimir Kolmogorov's software library (big thanks to him for providing that).

I recommend having a look at Depth Map Generation using Graph Cuts to get a better grasp of what DMAG3 does and, if nothing else, the meaning of the parameters that control DMAG3's behavior.

The input to DMAG3 is a pair of rectified images as well as the minimum and maximum disparities. The minimum and maximum disparities can be obtained manually with DF2 or automatically with ER9b. I always recommend using ER9b because it rectifies the images but you can also use Stereo PhotoMaker to align the images (although it won't be as good). The output of DMAG3 is the left and right depth maps as well as the left and right occlusion maps.

I think the only parameters to worry about are K and lambda. K is the occlusion penalty. Lambda is the multiplier for the smoothness penalty. For starters, I recommend letting DMAG3 compute K and lambda by setting them both to 0. These 2 parameters should then be modified (if needed) based upon how the left depth map and left occlusion map look. If the occlusion maps look a bit too busy (occluded pixels should only show up at object boundaries), increase K. If the depth maps looks a bit too noisy, increase lambda. If the images are large, I recommend using ds_factor (downsampling factor) = 2 (or more) in order to speed up DMAG3 (tremendously).

Here is an example:


Rectified left image of stereo pair called IMAG0013.


Left depth map obtained by DMAG3.


Left occlusion map obtained by DMAG3.

Here's another example:


Rectified left image of stereo pair called IMAG0020.


Left depth map obtained by DMAG3.


Left occlusion map obtained by DMAG3.

The windows executable (guaranteed to be virus free) is available for free via the 3D Software Page. Please, refer to the 'Help->About' page in the actual program for how to use it.

8 comments:

  1. I get a crash right upon the first attempt... using WinXP 32 SP3. Loading PNG images.
    Computing the disparity map for image 1... then a crash, windows debugger pops up.

    ReplyDelete
    Replies
    1. Make sure the image size is not too big, especially if you have an older computer with not a whole lot of ram on board and/or swap space.

      Delete
  2. The .exe application doesn't have a slot for the input images. How are the examined images loaded ?

    Thanks

    ReplyDelete
    Replies
    1. "File" in upper left of the window, next to "About", then File->open image 1 and open image 2

      Delete
  3. Excellent program! I was looking for something that could do this.

    ReplyDelete
  4. Hi,
    I know that the program is not open source but ,Can I ask you about the datacost and smooth term ? I am trying to use the gco matlab a library for optimizing multi-label energieshttp://vision.csd.uwo.ca/code/gco-v3.0.zip I already code the datacost and the smooth matrices but the disparity map is wrong and I am wondering why so Can you help me ?

    Thanks

    ReplyDelete
    Replies
    1. the data cost is the same cost used in the paper about Adaptive Support Weights. It's basically a cost based on the bilateral filter. You can pretty much use whatever cost you want, for example, an SAD cost. For the smoothing cost, it's also based on a bilateral filter: high cost if similar color and close in space but with a different disparity. Again, you can use whatever you want as long as it behaves like a smoothing cost.

      Delete