Thursday, June 6, 2013

Frame Sequence Generator 2 (FSG2)

Frame Sequence Generator 2 (FSG2) generates synthesized intermediate frames given a left image (image 1), a disparity/depth map, and the min and max disparities (needed to compute the actual disparity from the depth map). The basic idea is, for each frame, to shift the pixels in the left image according to the disparity map. This frame will have "holes" (disoccluded pixels) which are filled by an ultra simplistic in-painting algorithm. When the foreground of an image stays stationary and the background shifts (to the right), disoccluded areas are created which can be filled, in a very simplistic manner, by copying the rightmost visible (not disoccluded) pixel. For a more sophisticated method, check out Frame Sequence Generator 4 (FSG4)..

If imageL.xxx is the left image (image 1), the frames generated will be named frame1.xxx, frame2.xxx, etc. The last frame corresponds to the right image. Animated gifs or lenticular prints (see Lenticular Printing - Interlacing with SuperFlip) can be made with imageL.xxx, frame1.xxx, frame2.xxx, etc (don't use the actual right image). The program automatically creates an animated gif of those images after generating and saving the frames.

The disparity/depth map should be a grayscale image where white indicates closest foreground object and black indicates furthest background object. The darkest value in the depthmap corresponds to the minimum disparity and the brightest value corresponds to the maximum disparity. The disparity/depth map doesn't need to come from any of my programs.

Here's an example:


Left image of the "Art" stereo pair (part of 2005 Middlebury dataset).


Depth/disparity map.


Disoccluded areas ("holes") which result from the shifting of the reference image (left image) to get the "right" image (which you get when you ask to generate just 1 frame).


Pseudo-right image obtained with FSG2 (you get the "right" image when you generate just 1 frame).

The windows executable (guaranteed to be virus free) is available for free via the 3D Software Page. Please, refer to the 'Help->About' page in the actual program for how to use it.

9 comments:

  1. I m trying to use this program(fsg2.exe), but the result is not accurate as fsg.exe. It look like already combined 2 pictures in 1 file.

    ReplyDelete
    Replies
    1. it's using the right image when there's an occlusion or unreliable match on the left image. if the right image can't be used either, then it's doing some "in-painting" or filling.

      Delete
    2. it's not like doing 'in-painting' or filling. It seem like making a stereo image in a single frame.

      Delete
  2. within this dropbox i have enclosed what i have test. Can you please give some comment?https://www.dropbox.com/sh/dzww25ntrgwhl06/FjbOUU4c_l

    ReplyDelete
    Replies
    1. I'll try to have a look as soon as I can. It may take some time though. Thanks for the test case.

      Delete
    2. You are welcome. As a 3D fans, I also like to get some good tool to create good 3D picture.

      Delete
    3. i quickly looked at the depth maps and they don't seem to be very good to begin with. I am working on something else at the moment. once i am done, i will get back to dmag2/fsg2 and see if i can do anything about it.

      Delete
    4. i had a bit more time to look at your test case. i've updated ugosoft3d-2 today so get that first. then, in dmag2, make sure the picture called 29 is loaded as image 1 (left image) and picture called 00 is loaded as image 2 (right image), put -50 as min disparity and 10 as max disparity, you should get a relatively correct depth map to work with, then you can use fsg2.

      Delete