3D Photos - Pebbles

The stereo pair we're gonna work on comes from Peter Simcoe, who was kind enough to let me experiment with his Fuji W3 pictures.

This is the left image.

That is the right image. This is a stereo pair from which a good depth map should be easily obtained.

Let's get a depth map using Depth Map Automatic Generator 7 (DMAG7).

This is the depth map generated by DMAG7 using the following parameters: min disparity = -32, max disparity = 6, spatial sample rate = 8, range spatial rate = 32, patch radius = 12, lambda = 0.01, max L-BFGS iterations = 500, hash table size = 10,000. You might be able to get a better depth map by fiddling around with the parameters but that's good enough for me as is. For the min and max disparities, I didn't even bother using Disparity Finder 2 (DF2), I just put some values until I got white for whatever is the most in front and black for whatever is the most in the back. Since that was taken with a Fuji W3 without parallax fussing around, the foreground is probably at around 0 disparity and the background at -width/30 = -27 (width = 800).

This is the 3d wiggle generated by Wiggle Maker.

2D to 3D Conversion - Ludovic Butelle

Ludovic Butelle is the goalkeeper of Ligue 1's SCO Angers football club.

Here is diving against Olympique Marseille. We are gonna use Depth Map Automatic Generator 4 (DMAG4) to convert that image into a 3d image via a depth map.

This is my first attempt at the sparse depth map, input to DMAG4. Nothing too fancy: a gradient for the ground, everything past the goalposts is background (black), and each player in its own depth plane (old school viewmaster cartoon style).

This what DMAG4 produces. A little too much bleeding in my opinion. This time, instead of going back to the sparse depth map, I am gonna work from the dense depth map, erase the areas I don't like, and redo the scribbles locally. When you load up the reference image and put the dense depth map on top (saved as png), you're gonna need to add an alpha channel by clicking on "Transparency"->"Add alpha channel", otherwise when you erase, you're not gonna see the reference image that's underneath.

This is my second attempt at the depth map we are gonna feed to DMAG4. Clearly, it's not sparse anymore.

That's the dense depth map DMAG4 gives us. There is a slight problem where the right sleeve of the Marseille player (on the left) meets the goal post. Both objects are white and that's causing a whole lot of unwanted bleeding. Easy to fix.

This is my final attempt at the depth map to be fed to DMAG4. Again, I used the previous dense depth map (the one produced by DMAG4), erased what I didn't like, and redid the scribbles locally.

That's the dense depth map produced by DMAG4. We're gonna stop here. Time to go to Wiggle Maker and generate a 3d wiggle for everybody's entertainment.

2D to 3D Conversion - Queen's Freddie Mercury and Brian May at Live Aid

That's Freddie Mercury and Brian May of Queen at Live Aid, back in the 80s. We're gonna try to do a 2d to 3d conversion of that iconic image using Depth Map Automatic Generator 4 (DMAG4), the premier semi-automatic 2d to 3d image converter.

This is the sparse depth map, scribbled in Gimp.

You can create a gradient in DMAG4 with just a starting color and an ending color (in the sparse depth map) but only if what's between the two (in the reference image) doesn't have any color change. For example, for Brian's guitar body, you can't just scribble a depth for the bottom and top because there's quite a bit of color changes between the bottom and top of the guitar body. Recall that in DMAG4, depths propagate only within similar colors. That explains all the gradients in my scribbled sparse depth map. I didn't scribble this sparse depth map in one go. I usually go back and forth between fixing up the sparse depth map, generating the dense depth map with DMAG4, and visualizing the result in Wiggle Maker, the premier 3d wiggle gif creation tool. Another way of doing things is to create a crude dense depth map from a crude sparse depth map, erase the areas that are wrong, scribble some new depth clues in those now transparent areas, and regenerate the dense depth map (iterate on that process until satisfied with the dense depth map).

This is the depth map that DMAG4 generated from my sparse depth map. I think it's good enough for our purposes.

This is the 3d wiggle created with Wiggle Maker.

2D to 3D Conversion - Jackie and JFK 1959

I got this nice 2d photo from Reddit's Old School Cool sub-reddit. No idea who the photographer is but I am claiming use fair use to demonstrate the use of Depth Map Automatic Generator 4 (DMAG4), the premier semi-automatic 2d to 3d image conversion program for windows pc.

This is my first attempt at a sparse depth map.

This is the corresponding depth map as obtained by DMAG4. We can do better.

This is my second attempt at a sparse depth map. To refine the depths, I use the "free select tool" to select rectangular regions and then the "blend" tool to fill them. This kinda establish a nice gradient of depths from which I can work using the "color picker tool".

This is the corresponding depth map as obtained by DMAG4. We can do better.

This is my third attempt at a sparse depth map. I just continue refining the depth scribbles.

This is the corresponding depth map as obtained by DMAG4. We can do better.

This is my fourth attempt at a sparse depth map. I just continue refining the depth scribbles.

This is the corresponding depth map as obtained by DMAG4. I think I am gonna stop here as far scribbling the sparse depth map and running DMAG4. Now is a good time to apply some edge preserving smoothing with Edge Preserving Smoothing 5 (EPS5).

This is the smoothed depth map. It's finally time to create a 3d wiggle with Wiggle Maker.

This is the animated gif (3d wiggle) as produced by Wiggle Maker.

If you want to generate an anaglyph (or other things like a crosseyed view) out of the 2d image and the depth map, mi buen amigo Pepe Baeza recommends using 2to3, a totally free pc windows program. It seems to work ok although anytime I press the preview buttons, it crashes on me. If you press the "save" buttons, it works just fine and as advertised.

This is the anaglyph I got from 2to3. Unfortunately, you do need those fancy red/cyan glasses to see one of those.

3D Photos - At the park

In this post, I want to demonstrate how you can easily fix a depth map with Depth Map Automatic Generator 4 (DMAG4), the depth map generator that's usually used for 2d to 3d image conversion.

Left image.

Right image.

This is the depth map produced by Depth Map Automatic Generator 7 (DMAG7) using the following parameters: min disparity = -15, max disparity = 0, spatial sample rate = 8, range spatial rate = 16, patch radius = 7, lambda = 0.01, max L-BFGS iterations = 1,000, hash table size = 10,000.

This is the depth map smoothed by Edge Preserving Smoothing 7 (EPS7) using the following parameters: spatial standard deviation = 100, range spatial deviation = 100, and number of iterations = 3.

This is the animated gif produced by Wiggle Maker. As you can plainly see, the depth map has a few problems, like where the fencing is located in the background. We're gonna erase the bad areas and have DMAG4 "fill in the blanks" (the easy way and the slightly less easy way).

Easy way:

This is the depth map after having loosely erased problem areas. This was done in gimp using the eraser tool (with hard edge checked). Make sure that when you erase, you can see the checkerboard pattern, which indicates that the depth map has a transparency channel. If you don't see the checkerboard pattern, export the depth map in a format that allows for a transparency channel like png and work on that depth map instead.

This is the depth map produced by Depth Map Automatic Generator 4 (DMAG4).

Slightly less easy way:

We are gonna scribble some depths in the transparent areas of the depth map prior to giving it to DMAG4.

This the depth map scribbled over.

This is how the scribbling looks in Gimp. The process is pretty simple: 1) load up the reference image (left image), load up the depth map as new layer (it will go on top and we're gonna be able to see the reference image underneath in the transparent areas), scribble hint depths in the transparent areas of the depth map, and export the depth map layer.

This is the depth map produced by Depth Map Automatic Generator 4 (DMAG4).

This is the animated gif produced by Wiggle Maker. Much better, I think.

3D Photos - Sign with flowers

Left image (after rectification).

Right image (after rectification).

I usually use Stereo PhotoMaker to align the stereo pair but sometimes, it is less than satisfactory. Here, I have used Quasi-Euclidean Epipolar Rectification by Pascal Monasse to rectify the images.

This above is the depth map obtained by Depth Map Automatic Generator 7 (DMAG7) with the following parameters: min disparity = -32, max disparity = +32, spatial sample rate = 16, range spatial rate = 8, patch radius = 12, lambda = 0.1, max L-BFGS iterations = 2,000, hash table size = 10,000. It would probably be a good idea to smooth the depth map while preserving the edges, just to make it less harsh.

This above is the depth map smoothed by Edge Preserving Smoothing 7 (EPS7) with the following parameters: spatial standard deviation = 100, range spatial deviation = 100, and number of iterations = 3.

This the animated 3d gif obtained using Wiggle Maker. You can clearly see just by looking at the wiggle gif that there's something not quite right with the tree on the left and the bus. Not a big deal as those problems can be easily fixed by lightly editing the depth map.

Just for fun and also for the interest of science, let's see what depth map we would have gotten from DMAG7 using the left and right images as aligned/rectified by Stereo PhotoMaker. I should probably mention that they were aligned using the automatic alignment option in Stereo PhotoMaker.

Left image (aligned by Stereo PhotoMaker).

Right image (aligned by Stereo PhotoMaker).

Those 2 images are not very well aligned, especially as you go toward the top. Misaligned left and right images are always a problem for any automatic depth map generator that assumes that the epipolar lines as horizontal knowing that matching is always performed along the epipolar line. Most of them make that assumption to make the problem of stereo matching easier.

This above is the depth map obtained by DMAG7 using the same parameters as before. Clearly, the poor alignment gives problems to the depth map automatic generator.

2D to 3D Conversion - Marilyn Monroe in dance class

The 2d photo I am gonna start with comes from Marilyn Monroe in dance class, 1949.. I have no idea who took the original photo but I am gonna claim fair use for its usage here.

Here's the 2d photo of a rather young Marylin that we are gonna attempt to convert to 3d:

First I load it up into Gimp and draw a sparse depth map on top:

This is how the sparse depth map looks like all by itself:

Then I let Depth Map Automatic Generator 4 (DMAG4), the semi-automatic 2d-3d image converter do its magic:

Here's the corresponding 3d animated gif (wiggle) thanks to Wiggle Maker:

The reason why I drew those 3 gradients on the floor is because of the shadows. I first ran DMAG4 without them and the the gradation on the floor was not too satisfactory. So, I added them using a selection and the gradient tool (easy as pie).

Here's a tutorial video that explains the whole process of converting the 2d image into a 3d wiggle using DMAG4: