Friday, October 11, 2013

An example of 2D to 3D conversion using Gimpel3D

Lately, I've been looking into ways to convert 2d flat images into 3d scenes, either directly or via a depth map. One can certainly create depth maps using Photoshop or Gimp but I have always been intrigued by 3d scene reconstruction. One program (free and runs on windows) that can do that has caught my eye: Gimpel3D. Gimpel3D was originally written to convert video streams but it can certainly be used to convert a single frame or image (although it might be a tad overkill just to get a depth map).

The documentation for Gimpel3D is not the best but you can pretty much figure out what all the tools do thanks to the html "help" inside the program. There are some things that are not very well explained though, in particular, the "Alignment Options" and "Gap Fill".

Something that should be stressed: Whatever you do in Gimpel3D in terms of modeling, when you look straight on at the scene from the virtual camera (the red dot when Views->Show Camera is checked), you will always see the 2d picture you loaded as if the scene were flat. This explains why things may look weird when you look at the 3d scene from different view points.

In the following youtube video, I've tried to reconstruct the Mona Lisa (actually, a close-up) using Gimpel3D:


Here's the corresponding depth map:


As you can see, there are some problems: the nose is a bit on the manly side and the eye sockets are set way too deep. There's also a depth discontinuity between the face and the hair on both sides. It's not really Gimpel3D's fault as the model file for the human head is probably not the greatest for our Mona Lisa.

Here's another youtube video where I attempt to model the nose of the Mona Lisa using 5 planes (layers) and the "orientation" tool:


Here's the depth map:


I guess I could have also used the "scale" tool but kinda forgot about it when I made the video. In theory, one could model the whole face manually using a bunch of facial planes (see books about drawing the human head). That would certainly be a bit tedious, especially knowing that there's the capability to project onto models. If you don't have a model file handy for an object in the foreground you want to render precisely, I am afraid that there might not be much of a choice. One could also probably use the "contour extrusion" tool and the "anchor points" tool to speed up the "dimensionalization" of an object. I've tried those and they work quite well (as described), but obviously these tools have their own limitations.

6 comments:

  1. what are the output file format of Gimpel3D?

    ReplyDelete
    Replies
    1. you can take snapshots depending on camera position (jpeg, png, etc). I believe I explain the process of taking snapshots to make a 3d animated gif in another post.

      Delete
  2. Hello!
    I have the need to use software gimple3d, very eager to be trained to use. I am willing to pay for training from the software organization. Please reply to messages when received
    Thank you

    ReplyDelete
    Replies
    1. you would need to contact the creator of the software (René Gimpel) yourself.

      Delete
  3. When i use Single Frame>Load Fresh Frame every image is all white in 2D mode (where i need to edit) but it shows up correctly in 3D/2D mode and 3D mode.

    ReplyDelete
    Replies
    1. Sorry Chris, I really don't know why it does that. I have used Gimpel a handful of times a long time ago.

      Delete