Non Photorealistic Rendering - Separated Orientation-Aligned Bilateral Filter (Separated OABF)

Bilateral filtering is a staple in the Computer Vision diet. It is used a lot to smooth images while preserving edges. The problem is that it is, in its naive implementation at least, quite slow especially if it needs to be iterated. As you probably know, the bilateral filter is a mix of two Gaussian filters, one that considers the spatial distance between the pixel under consideration and the neighboring pixel (in the convolution window) as the argument to the Gaussian function and one that considers the color distance. In contrast, the Gaussian filter is quite fast because it is separable, that is, you can apply a one-dimensional Gaussian filter along the horizontal and then apply another one-dimensional Gaussian filter along the vertical instead of applying a true two-dimensional Gaussian filter.

So, the bilateral filter is not separable. What can we do about that? Well, you can still separate the bilateral filter if you want to but it won't be the same. For some applications, it doesn't really matter if the implementation of the bilateral filter is not exact. So, you can separate the two-dimensional bilateral filter by applying a one-dimensional bilateral filter along the horizontal and then a one-dimensional bilateral filter along the vertical. If you have an Edge Tangent Flow (ETF) vector field handy, you can separate the bilateral filter by applying the one-dimensional bilateral filter along the gradient direction and the tangent direction alternatively. Recall that the gradient direction is perpendicular to the tangent direction and goes across the edge. This separation along the gradient and tangent is referred to as "Separated Orientation-Aligned Bilateral Filter" (Separated OABF) in "Image Abstraction by Structure Adaptive Filtering" by Jan Eric Kyprianidis and Jürgen Döllner. It is supposed to behave better at preserving shape boundaries than the non-oriented version (See "Flow-Based Image Abstraction"by Henry Kang et al.)

Before showing some examples of Separated OABF, it is probably a good idea to give the formula for the bilateral filter. You control the way the bilateral filter behaves by the variance "sigma_d" of the spatial Gaussian function and the variance "sigma_r" of the color (aka tonal or range) Gaussian function. The larger sigma_d, the larger the influence of far away (in the spatial sense) sampling points. The larger sigma_r, the less edge-preserving the filter is going to be.

The bilateral filter weights and formula.

Input rgb image (640x422).

Output from the "Separated Orientation-Aligned Bilateral Filter" (Separated OABF) using sigma_d = 16 and sigma_r = 16.