Frame Sequence Generator 3 (FSG3)

Frame Sequence Generator 3 (FSG3) generates synthesized frames given a left image (image 1), a disparity/depth map, and the min and max disparities (needed to compute the actual disparity from the depth map). The basic idea is, for each frame, to shift the pixels in the left image according to the disparity map. This shifted image contains "holes" (areas made up of pixels that have become disoccluded or disoccluded areas, in short) which need to be inpainted (inpainting is a common theme in computer vision. It's pretty easy to understand why you have "holes" showing up when you shift a reference image according to a disparity/depth map. Consider a picture with an object in the foreground with disparity 0 and a background. When you shift the background, "holes" will form on the immediate right of the foreground object. FSG4 fills the "holes" by propagating the pixels to the immediate right of the disoccluded areas using a methodology similar to Semi-Automatic 2D to 3D Image Conversion using Random Walks. This type of inpainting is quite a bit fancier than what's being done in Frame Sequence Generator 2 (FSG2) and therefore, FSG3 should be used in lieu of FSG2.

If imageL.xxx is the left image (image 1), the frames generated will be named frame1.xxx, frame2.xxx, etc. The last frame corresponds to the right image. Animated gifs or lenticular prints (see Lenticular Printing - Interlacing with SuperFlip) can be made with imageL.xxx, frame1.xxx, frame2.xxx, etc (don't use the actual right image). The program automatically creates an animated gif of those images after generating and saving the frames.

The disparity/depth map should be a grayscale image where white indicates closest foreground object and black indicates furthest background object. The darkest value in the depthmap corresponds to the minimum disparity and the brightest value corresponds to the maximum disparity. The disparity/depth map doesn't need to come from any of my programs.

The quality of the inpainting heavily depends on the accuracy of the depth map at object boundaries.

Here's an example:

Left image of the "Art" stereo pair (courtesy of Middlebury).

Depth/disparity map.

Disoccluded areas ("holes") which result from the shifting of the reference image (left image) to get the "right" image (generate 1 frame).

Right image obtained with FSG3.

The windows executable (guaranteed to be virus free) is available for free via the 3D Software Page. Please, refer to the 'Help->About' page in the actual program for how to use it.