The idea is to give pixels in a window centered on a reference pixel a weight that depends on spatial proximity and color similarity. This allows the use of a fixed window size unlike the usual adaptive-window methods. You can find the very same methodology in bilateral filtering which is commonly used as an edge preserving blurring/smoothing tool in image processing.
Given a reference pixel p and a neighboring pixel q, this is how the support weight w(p,q) is computed:
Clearly, support weights will vary between 0 and 1. A weight is closer to the value 1 (maximum weight) if its color is similar to that of the reference pixel and it is near the reference pixel. A weight is closer to the value 0 (minimum weight) if its color is not similar to that of the reference pixel and/or it is far from the reference pixel.
Let's look at how the weight distribution looks like in real-life examples (those are from the paper):
As you can see, the brighter pixels are the pixels which are similar in color to the reference pixel (marked by a rectangle). As you move away from the reference pixel, pixels that are similar in color get darker. Recall that brighter pixels have the heaviest weight and contribute the most to the matching cost. It's like adapting the window contour to the object the reference pixel is a part of. As the color similarity parameter (gamma_c) increases, the bright part of the window gets larger, in other words, more of the window carries a significant weight. It's a bit like increasing the window size in the usual non weight-based methods.
Given a pixel p and a disparity d, the matching cost for a given window size is given by:
This matching cost based on weighted correlation windows can be used in local methods (Winner Takes All approach) but also in global methods (as the data term).
As an update to this little write-up (updated 01/14/2014), I would like to go a bit deeper into the color similarity parameter gamma_c and the proximity parameter gamma_p. When looking at the neighborhood of a given pixel in the context of stereo matching, it is kind of assumed that the neighboring pixels should all be at the same depth, in other words, the pixels within a window/block should all be at the same depth. Obviously, we don't know that beforehand since that's we are trying to determine (the depth). So, it is often assumed that neighboring pixels with different colors are likely to be at different depths. It is clearly not always the case (neighboring pixels with different colors could be at the same depth and neighboring pixels with similar colors could be at different depths). It is however not a bad assumption to make when you want to avoid the possible fattening effect ("foreground-fattening") at boundaries between neighboring zones at different depths, a pretty common occurrence in stereo matching algorithms. So, if you look into a neighborhood or window, you want to see weights that are high (close to 1) when the neighboring pixel is similar in color to the center pixel and weights that are low (close to 0) when the neighboring pixel is not similar in color to the center pixel. This is controlled by the color similarity parameter gamma_c. What you also want is a reduction of the weight as you go away from the center pixel. This is controlled by the proximity parameter gamma_p.
Let's look at the following neighborhood window/block (the reference pixel is at the center), play around with gamma_c and gamma_p, and see what kind of effects it has on the pixel weights:
Let's first focus on the color similarity parameter gamma_c. If you set gamma_p to some very large value, it has no effect on the weights (the weights solely come from the color similarity term.) In the following series of pictures, I am only changing gamma_c and see the effect it has on the weights (black means weight equals 0, white means weight equals 1.) What you wanna see is white where there is red and various shades of gray anywhere else (the further away the color is from red, the blacker it should be):
radius=12, gamma_p=1,000,000, gamma_c=200
radius=12, gamma_p=1,000,000, gamma_c=100
radius=12, gamma_p=1,000,000, gamma_c=50
radius=12, gamma_p=1,000,000, gamma_c=20
radius=12, gamma_p=1,000,000, gamma_c=5
Let's now focus on the proximity parameter gamma_p. If you set gamma_c to some very large value, it has no effect on the weights (the weights come solely from the proximity term.) In the following series of pictures, I am only changing gamma_p and see the effect it has on the weights (black means weight equals 0, white means weight equals 1.) What you wanna see is white at the reference pixel and darker shades of gray as you move away from the center:
radius=12, gamma_c=1,000,000, gamma_p=200
radius=12, gamma_c=1,000,000, gamma_p=100
radius=12, gamma_c=1,000,000, gamma_p=50
radius=12, gamma_c=1,000,000, gamma_p=20
radius=12, gamma_c=1,000,000, gamma_p=5
It is worth noting that if both gamma_c and gamma_p are set to large values, all weights are equal to 1 and the method is basically a classic window-based or block-based stereo matching method.
This last picture shows what happens when you combine the color similarity and proximity effects (color similarity weights multiplied by the proximity weights):
radius=12, gamma_c=100, gamma_p=20
No comments:
Post a Comment