Depth Map Automatic Generator 4 (DMAG4)

DMAG4 is a semi-automatic 2D to 3D image converter. DMAG4 is an implementation of what's explained in Semi-Automatic 2D to 3D Image Conversion using Random Walks.

The sparse depth map (which the program requires) is a transparent image partially filled with brush strokes in various shades of gray to indicate (approximate) depth. This is the non-automatic part. So, how do you create that sparse depth map? In Gimp or Photoshop, you open the reference image and create a new transparent layer on top of the reference image. This is where you are gonna draw the sparse depth map. The brush strokes should be done with the pencil tool using a 100% hard brush so that any painted pixel is fully opaque (no anti-aliasing). To verify that you are not generating semi-transparent pixels, you may want to zoom in on your painted brush strokes and look at the edges. They should be hard and look jagged (no anti-aliasing involved). In case semi-transparent pixels get created for whatever reason (maybe you used a tool with anti-aliasing turned on), they can be easily removed prior to saving the sparse depth map by doing the following (in Gimp): click on "Layer"->"Transparency"->"Threshold Alpha ...", and click OK. After you are done scribbling on the sparse depth map layer, save the layer (only the layer) in the png format (lossless and preserves transparency). To save the layer only, you can simply make the reference image not visible by clicking off the eye icon in the layer dialog box.

Let's go through the parameters DMAG4 uses. Then, we will introduce the notion of an "edge image" to simplify the process.

The maximum number of iterations is a parameter used in the linear equation solver (which happens to be Lis, a Library of Iterative Solvers for Linear Systems). If you see that the depths don't seem to propagate correctly across areas of similar color (in the reference image), it's probably because the number of iterations is set too low.

The beta parameter is explained in Semi-Automatic 2D to 3D Image Conversion using Random Walks. The lower the beta, the more smooth the depth map is going to be although the more bleeding may occur across object boundaries (if no "edge image" is used). In other words, the lower the beta, the easier it is for depths to propagate and the easier it is for depths to bleed across object boundaries (if no "edge image" is used). If an "edge image" is used, beta can be lowered quite a bit because bleeding is prevented by the existence of the "edge image". With a low beta, depths propagate very easily within an object as the bilateral filter (that's really what DMAG4 is) becomes a mere Gaussian filter.

The number of scales controls the size of the scale space. DMAG4 can build a scale space in order to mitigate the effects of noise or fine texture in the reference image (at the expense of respect of object boundaries). The more scales, the smoother the depth map is going to be, the more bleeding is going to occur, the longer it is going to take and the more memory is going to be used. If you use an "edge image", setting this parameter to 1 should be just fine.

When the level of graph connection within a scale (con_level) is set to 1, each pixel is connected (by a graph edge) to four adjacent pixels (left, right, top, and bottom). If con_level is set to 2, it is connected to eight adjacent pixels (the four from level 1 and the four more situated diagonally). When the level of graph connection across scales (con-level2) is set to 1, each pixel is connected to one pixel on the previous and next scale. If it is set to 2, each pixel is connected to five pixels on the previous and next scale. If it is set to 3, each pixel is connected to nine pixels on the previous and next scale. The more connections, the more the depths get propagated. Using con_level = 1 and con_level2 = 1 is probably a good idea and there's really no need to change that.

Now, let's talk about the "edge image" which is really supposed to make everything easier at not much of a cost for the user.

In order to eliminate the bleeding of depths across objects of similar colors, it is possible to give a so-called "edge image" to DMAG4. An edge image is basically a trace over the object boundaries of the reference image. Just like for the sparse depth map, it is created by adding a transparent layer over the reference image and tracing over the object boundaries. The tracing on the edge image should be 1 pixel wide (in most places). The presence of an edge image enables to be less conservative regarding possible bleeding. Thus, beta can be lowered (say, from 90. to 10.) and the number of scales increased with less fear. Also, the sparse depth map probably doesn't need to be as dense. Note that the edge image does not need to be complete, that is, only areas where bleeding occurs can be traced over. If you do use one an edge image, make sure con_level and con_level2 are set to 1. To sum things up and be perfectly clear, DMAG4 does not need an edge image but if colors are similar between foreground and background, it is probably a good idea to create an edge image to avoid bleeding in some areas.

What is the best way to trace over object boundaries to create the "edge image"? In Gimp, I like to use the "Paths Tool" and draw a path along the object boundaries by left-clicking. When done drawing the path, click on "Stroke Path" in the "Tool Options" of the "Paths Tool", click on "Stroke line", click on "Solid color", uncheck the "Antialiasing" box, choose a "Line width" of 1.0 pixel, and click on "Stroke". When you zoom in on the stroke, it should 1 pixel wide with no antialiasing. The color of the stroke does not matter although I like to use red in most cases. It is very easy to do.

An alternative for creating the "edge image" is to simply use the pencil tool with a hard brush (no anti-aliasing) and set the brush size to the minimum. Press the shift key and trace along the object boundaries to create the edge image. Pressing the shift key while left-clicking allows you draw straight line segments with the pencil tool.

If DMAG4 gives an error about the pixels being semi-transparent, it's because you probably used a tool that's anti-aliased (soft edges) at some point. To fix this in Gimp (Photoshop is probably similar), click on "Layer"->"Transparency"->"Threshold Alpha ...", and click OK. All your semi-transparent pixels are now either transparent or opaque and DMAG4 will thank you for it.

Here's a post that shows the process: 3D Image Conversion - Top Gun.

Here's a video tutorial for DMAG4 used with an "edge image" (I am so sorry there is no sound):


A few words to the wise now ... It is recommended not to use large images. I would not use images that are bigger than 1 mega-pixels. If you do use large images, the max number of iterations should be increased. The cpu time will increase though as a result. So keep your images small! If you use an "edge image", you should use a low beta, like beta = 10.

If you have any problem with DMAG4, feel free to send your reference image, sparse depth map, edge image (if any), and dense depth map (if you have gotten that far) to the email that should be somewhere in the right sidebar.

The windows executable (guaranteed to be virus free) is available for free via the 3D Software Page. Please, refer to the 'Help->About' page in the actual program for how to use it.

Source code: DMAG4 on github.

UPDATE (03/03/2020): I now recommend using 2d to 3d Image Conversion Software - the3dconverter2 for 2d to 3d conversion over DMAG4.