Thursday, May 14, 2020

Fun fact: the3dconverter2 can be used to edit depth maps

It is possible to use the3dconverter2 to edit depth maps relatively easily.

Here is the process:

1) Load (in Gimp) the reference image, typically, the left image of a stereo pair.

2) Load (as a layer) the depth map that needs to be edited.

3) Erase the input depth map in places where the depths do not appear to be correct.

4) If, for at least one area that's been erased, there is a depth discontinuity in the reference image, create a new layer and draw along the discontinuities using the pencil tool (size = 1 pixel). What you are doing is drawing a so-called "edge image".

4) Export the reference image as "content_image.png".

5) Export the partially erased input depth map as "sparse_depthmap_rgba_image.png".

6) Export the edge image as "edge_rgba_image.png". If there was no need to draw an edge image, then do nothing in this step.

7) Run the3dconverter2.

Here is an example:


Reference image aka "content_image.png".


Input depth map that needs to be edited.


Partially erased depth map aka "sparse_depthmap_rgba_image.png". Pure white pixels are actually transparent.

Note that I could have done a much better job erasing stuff on the right side (upper right, to be precise) as I clearly missed a few pixels. No big deal but it's gonna show up in the output depth map. If you are a perfectionist, you should really erase them.


Edge image aka "edge_rgba_image.png".

Here, I simply draw along what looks like a paddle since the paddle is clearly not at the same depth as the background. That's what I mean by depth discontinuity. You only need to draw within the confines of what's been erased in the input depth map, nothing more.


Output from the3dconverter2.

Remember those dark pixels I didn't erase on the right side (upper right) of the input depth map... I told you they would show up.

Here is a video tutorial that shows how the3dconverter2 can be used to edit depths maps:


Here is another video tutorial about the3dconverter2 but more in the context of 2d to 3d image conversion:

6 comments:

  1. Replies
    1. how helpful in improving the depth map can be dmag 11b

      Delete
    2. I think that many dmag packages can be used alternately to improve the quality of the depth map



      Delete
  2. how helpful in improving the depth map can be dmag 11b ?

    ReplyDelete
    Replies
    1. I think it can be used effectively to make the depth map closely follow object boundaries. You can use dmag11b to generate a depth map with transparent areas which you can then feed to dmag11 or dmag4. You can also use dmag9b which is very good at improving depth map quality if the input depth map is already quite good. the3dconverter2 is probably better for when the depth map is very wrong in some areas and no amount of fine tuning can make it better.

      Delete