Non Photorealistic Rendering - Edge Tangent Flow (ETF)

Given an input rgb image, given a pixel of that input rgb image, the eigenvectors of the so-called "structure tensor" (aka "2nd moment tensor") define the directions of extreme (minimum and maximum) rate of change (intensity rate of change). The eigenvector associated with the smallest eigenvalue defines the direction of minimum rate of change. The eigenvector associated with the largest eigenvalue defines the direction of maximum rate of change. If there is an edge at a given pixel, it is the eigenvector associated with the smallest eigenvalue that gives the direction of the edge. Indeed, if you go along the edge, the rate of change (in intensity) is minimum. So, for each pixel of the image, if you consider the eigenvector associated with the smallest eigenvalue, you have got yourself a vector field which is referred to as the Edge Tangent Flow (ETF) vector field. The name Edge Tangent Flow (ETF) comes from "Flow-Based Image Abstraction" by Henry Kang, Seungyong Lee, and Charles K. Chui. To get good results, the structure tensor should be smoothed a little using the usual Gaussian filter.

To visualize an Edge Tangent Flow (ETF) vector field, you can use Line Integral Convolution (LIC). What you do is start with a noise image where each pixel is given a random grayscale value. Then, for each pixel of the noise image, you follow the flow/stream line that passes through the pixel going forward and backward so that the pixel is in the middle of the flow/stream line under consideration. You convolve (with a one-dimensional bilateral filter) the values along that stream line and assign the resulting average value to the pixel. In the output grayscale image, the actual grayscale values have no meaning, that is, whiter does not mean a stronger edge.

Input rgb image.

Edge Tangent Flow (ETF) visualized by Line Integral Convolution (LIC).

With a good imagination, one can kinda see a Vincent Van Gogh or Edvard Munch style in the LIC visualization of an ETF. ETFs are used quite a bit in Non-Photorealistic Rendering (NPR), in particular, in Stroke-Based Rendering (SBR) or Painterly Rendering as they provide a simple way to orient brush strokes. ETFs are also used in image abstraction and stylization.