3D Image Conversion - Top Gun

In this post, we are gonna look at 2d to 3d image conversion using DMAG4.

That's our good friend Tom Cruise in Top Gun. That's the 2d reference image we are going to "3d-fy".

As you probably know, the input to DMAG4 is the 2d image, a sparse depth map, and possibly what I call an "edge image". The purpose of the edge image is to separate different objects in the scene. DMAG4 will not propagate depths past an edge in the edge image, which means you don't have to worry about the beta parameter.

This is the "edge image". White is actually transparent background.

"Edge image" shown on top of the 2d image.

I drew the "edge image" in Gimp using the "Paths" tool and stroking the path once done. Use the "Stroke Line" option, turn the anti-aliasing off, and choose 1 pixel for the width when stroking the path. It takes about 5 minutes to get the "edge image". Because I use an "edge image", DMAG4 can be allowed to propagate regardless of color similarity and therefore it is ok to use a small beta (makes the bilateral filter in DMAG4 behave like a regular Gaussian filter). Having an "edge image" makes things much easier when it is time to draw the sparse depth map and give the depth clues to DMAG4.

This is the sparse depth map. White is actually transparent background.

Sparse depth map shown on top of the "edge image" and 2d image.

I took a pretty minimalist approach when drawing the sparse depth map, which means it is very sparse. To draw the sparse depth map in Gimp, I use the "Pencil" tool with an hard edge brush (no anti-aliasing). It takes another 5 minutes to draw the sparse depth map. When creating the sparse depth map, the important thing is the relationship between the various depths, not the actual depths. Make sure that when you zoom on the sparse depth map brushed on areas, the edges look jagged, which means there is no anti-aliasing applied, which is what we want.

We now have everything we need to launch DMAG4 and get the dense depth map.

Dense depth map obtained by DMAG4.

I used the following parameters in DMAG4:
beta = 10
maxiter = 5000
scale_nbr = 1
con_level = 1
con_level2 = 1

Again, because I used an "edge image", I really don't have to worry about depths propagating across objects. This means that I can use a low beta (here, equal to 10). When beta is low, DMAG4 behaves pretty much like a classic Gaussian filter. If beta is large, DMAG4 behaves like a bilateral filter, in other words, it propagates depths only along similar colors, which can be a real problem in some cases. The idea behind using a bilateral filter is that things that are not of the same color probably don't belong to the same object and are probably at different depths. Of course, this is not ideal because you can clearly have different colors within an object. Because of that, if you don't use an "edge image", the sparse depth map may need to be not so sparse at all and a lot of time is wasted drawing the sparse depth map, running DMAG4, and fixing the sparse depth map before going through another iteration. Conclusion: use an "edge image" and a low beta!

Wiggle/wobble created with wigglemaker.