Friday, June 26, 2015

Depth Map Automatic Generator 8 (DMAG8)

Depth Map Automatic Generator 8 (DMAG8) is a multi-view stereo automatic depth map generator. Its input is a set (any number) of non-rectified (non-aligned) images, typically extracted from a video taken with a single lens camera.

Because multi-view stereo relies on Structure from Motion (and Bundle Adjustment) which I know nothing about (not sure if I want to invest the time to learn their secrets), DMAG8 needs the output of a third-party program called VisualSFM, which creates a sparse 3d reconstruction from a set of images. So, to be able to use DMAG8, you will need to install VisualSFM prior. No worries as it's free. VisualSFM is quite cool on its own as it can also create dense 3d reconstruction (DMAG8 doesn't need that part though). There is a bit of a learning curve though to get a good 3d sparse reconstruction (for our purposes).

Even if you can get a sparse 3d reconstruction from VisualSFM, it doesn't mean it's good enough for DMAG8. Depth Finder 8 (DF8) can assist you in determining if the nvm file is ok by drawing the epipolar line on the target image that corrsponds to a given pixel in the reference image. By checking it against the matching pixel in the target image, one can tell if the sparse 3d reconstruction is good or bad. Of course, it doesn't have to be perfect but it may explain deficiencies in the depth map produced by DMAG8 (no use wasting time adjusting parameters then).

I don't know what makes VisualSFM produce good or bad (for DMAG8's purposes) sparse 3d reconstructions. The problem is that it's key to DMAG8's success (or failure). Anyways, if you use it, let me know what works for you regarding VisualSFM.

The manual for DMAG8 (dmag8_manual.pdf) is in the directory where you decompressed ugosoft3d-8-x64.rar.

The following is a sample depth map produced by DMAG8 using a set of two non-rectified images taken with a single lens camera.


Image 0 (reference image).


Image 1.


Sparse 3d reconstruction in VisualSFM.


Just for fun, this is the dense 3d reconstruction using Yasutaka Furukawa's CMVS tool within VisualSFM.


Depth map produced by DMAG8 using near plane depth = 0, far plane depth = 0, number of planes = 0, spatial sample rate = 8, range sample rate = 32, radius = 12, lambda = 0.01, max iterations = 1000, and hash table size = 10000.

Here's another example of a depth map produced by DMAG8! This time I used a video shot with a single lens camera. Extracted 8 images from the video using ImageGrab which were then fed to VisualSFM to get the sparse 3d reconstruction (input to DMAG8).


Video and depth map produced by DMAG8 using near plane depth = 9, far plane depth = 80, number of planes = 0, spatial sample rate = 8, range sample rate = 32, radius = 12, lambda = 0.1, max iterations = 1000, and hash table size = 10000.

The windows executable (guaranteed to be virus free) is available for free via the 3D Software Page.