Non Photorealistic Rendering - Separated Flow-based Difference of Gaussians (Separated FDoG)

One can use the Difference of Gaussians (DoG) to extract edges from an intensity image. As you know, the Gaussian filter is a low-pass filter, meaning that it removes high frequency artifacts in an image (e.g. noise). When you take the Gaussian of an image and you subtract a larger sigma Gaussian from it, you essentially have a band-pass filter that rejects high frequency as well as low frequency intensities. That's perfect to detect edges! It should be noted that the edges we want when stylizing images are not necessarily the same edges one might want in other Computer Vision field. Indeed, we want relatively thick well-defined edges, not skinny edges like those you usually get with the Canny edge detector, for example.

To get started with computing the edge image, a one-dimensional DoG filter is applied along the gradient direction for every pixel in the input image (in CIE-Lab color space). It is customary to choose the variance of the 2nd Gaussian filter to be 1.6 times the variance of the 1st Gaussian. This is done so that the Difference of Gaussians (DoG) approximates the Laplacian of Gaussians (LoG).

Difference of Gaussians (DoG) filter applied in the gradient direction.

Now, in order to have nice flowing edges that would not be out of place in a Sunday morning paper funnies section, you need to smooth the edges along ... well, the edges. The problem is that it is those very nice flowing edges that we are trying to define. A chicken and egg problem, no doubt. Kang et al. in "Flow-based Image Abstraction" are the first ones to come up with a solution. They use the Edge Tangent Flow (ETF) vector field to smooth the edges. What you do is take the edge image coming from the DoG filter and convolve it along the flow curves of the ETF using a bilateral filter. It is very similar to the convolution used to visualize the ETF in Line Integral Convolution (LIC). For more info about ETF and LIC, check Non Photorealistic Rendering - Edge Tangent Flow (ETF).

We are not quite done yet as thresholding is applied to the edge image so that the grayscale values are (pretty much) either 0 (black) or 1 (white). A smooth step function function is used to control how sharp the edges are going to be.

Thresholding via smoothed step function.

Input rgb image.

Edge image obtained with Separated Flow-based Difference of Gaussians (Separated FDoG).