Tuesday, March 14, 2017

Frame Sequence Generator 11 (FSG11)

FSG11 generates synthetic views given input images and depth maps. If a single image and depth map is provided, it behaves very much like FSG4 but a with a subtle difference: FSG11 gets rid of outliers prior to inpainting. If more than one image and depth map is provided, the additional images and depth maps are directly used to inpaint, avoiding the all too familiar blurring effect of most frame sequence generators (FSG4 included).

Generating the synthetic views using a single (main) image and depth map:


First (main) image


First (main) depth map.


Animated gif showing the synthetic views generated by FSG11.

Yep, it's definitively blurry but I actually do think it looks pretty good (because it's silky smooth). Does inpainting the actual background gives better results? To do that, you need a background image and its associated depth map, which can be fed to FSG11.

Generating the synthetic views using an additional image and depth map (for the background):


Second image.


Second depth map.


Animated gif showing the synthetic views produced by FSG11.

I felt the need to write FSG11 in the context of 2d to 3d image conversion. Since you have to spend time to generate the (main) depth map, why not spend a tad more time generating a second image and a depth map for the background. I use the clone tool and eat away at the foreground to generate the background image. The associated background depth map is usually quite simple since the foreground objects are supposed to be gone. In some cases, it can even be pure black for a background at infinity.

The windows executable (guaranteed to be virus free) is available for free via the 3D Software Page.