Tuesday, July 17, 2018

The Watercolorist - Françoise Hardy in the movie "Grand Prix"


Source photograph: still from the movie "Grand Prix" starring James Garner.

I ran "The Cartoonist" with the default settings to get an abstracted image. That abstracted image is going to be the input to "The Watercolorist".


Abstracted version of the source photograph.

You can play around with parameter "quant_levels" to change the cel shading before watercolorizing it.

Let's run "The Watercolorist" using the default setting. That's our reference.

Input to "The Watercolorist":

image = abstracted_image_after_quantize.png
paper texture = ../texture/free-vector-watercolor-paper-texture.jpg
beta (paper texture) = -2
beta (turbulent flow) = -4
gradient convolution kernel size (edge darkening) = 5
beta (edge_darkening) = -4
output image = watercolor_image.png

Rendered image:


Let's change the paper texture beta from -2.0 to -1.0. This will make the paper texture effect less pronounced.

Input to "The Watercolorist":

image = abstracted_image_after_quantize.png
paper texture = ../texture/free-vector-watercolor-paper-texture.jpg
beta (paper texture) = -1
beta (turbulent flow) = -4
gradient convolution kernel size (edge darkening) = 5
beta (edge_darkening) = -4
output image = watercolor_image.png

Rendered image:


Let's go back to the default settings and change the paper texture from -2.0 to -3.0. This will make the paper texture effect more pronounced.

Input to "The Watercolorist":

image = abstracted_image_after_quantize.png
paper texture = ../texture/free-vector-watercolor-paper-texture.jpg
beta (paper texture) = -3
beta (turbulent flow) = -4
gradient convolution kernel size (edge darkening) = 5
beta (edge_darkening) = -4
output image = watercolor_image.png

Rendered image:


Let's go back to the default settings and change the turbulent flow beta from -4.0 to -2.0. This will make the turbulent flow effect weaker.

Input to "The Watercolorist":

image = abstracted_image_after_quantize.png
paper texture = ../texture/free-vector-watercolor-paper-texture.jpg
beta (paper texture) = -2
beta (turbulent flow) = -2
gradient convolution kernel size (edge darkening) = 5
beta (edge_darkening) = -4
output image = watercolor_image.png

Rendered image:


Let's go back to the default settings and change the turbulent flow beta from -4.0 to -6.0. This will make the turbulent flow effect stronger.

Input to "The Watercolorist":

image = abstracted_image_after_quantize.png
paper texture = ../texture/free-vector-watercolor-paper-texture.jpg
beta (paper texture) = -2
beta (turbulent flow) = -6
gradient convolution kernel size (edge darkening) = 5
beta (edge_darkening) = -4
output image = watercolor_image.png

Rendered image:


Let's go back to the default settings and change the gradient convolution kernel size (edge darkening) from 5 to 21. This will make the edge darkening effect less steep (more smooth).

Input to "The Watercolorist":

image = abstracted_image_after_quantize.png
paper texture = ../texture/free-vector-watercolor-paper-texture.jpg
beta (paper texture) = -2
beta (turbulent flow) = -4
gradient convolution kernel size (edge darkening) = 21
beta (edge_darkening) = -4
output image = watercolor_image.png

Rendered image:


Let's go back to the default settings and change the edge darkening beta from -4.0 to -2.0. This will make the edge darkening effect weaker.

Input to "The Watercolorist":

image = abstracted_image_after_quantize.png
paper texture = ../texture/free-vector-watercolor-paper-texture.jpg
beta (paper texture) = -2
beta (turbulent flow) = -4
gradient convolution kernel size (edge darkening) = 5
beta (edge_darkening) = -2
output image = watercolor_image.png

Rendered image:


Let's go back to the default settings and change the edge darkening beta from -4.0 to -6.0. This will make the edge darkening effect stronger.

Input to "The Watercolorist":

image = abstracted_image_after_quantize.png
paper texture = ../texture/free-vector-watercolor-paper-texture.jpg
beta (paper texture) = -2
beta (turbulent flow) = -4
gradient convolution kernel size (edge darkening) = 5
beta (edge_darkening) = -6
output image = watercolor_image.png

Rendered image:


I think it's probably a good idea to soften the edge darkening effect on Miss Hardy's face.

Monday, July 16, 2018

The Cartoonist - Ford GT40 at Le Mans 24 hours race in 1968


Input photograph: Lucien Bianchi/Pedro Rodriguez Ford GT40 number 9 at the 1968 24 hours of Le Mans.

Let's do a base run using the default parameters.

Input to "The Cartoonist":

input rgb image = Le_Mans_1968_1.jpg
tensor_sigma = 3
n_e = 2
n_a = 4
sigma_d = 3
sigma_r = 4.25
fdog_n = 1
fdog_sigma_e = 1
fdog_tau = 0.99
fdog_sigma_m = 3
fdog_phi = 2
phi_q = 3
quant_levels = 8
output rgb image = cartoon_image.jpg

Detected edges:


Resulting rendered image:


Let's change n_e from 2 to 4 and n_a from 4 to 8, which means a lot more bilateral filtering is going to occur.

Input to "The Cartoonist":

input rgb image = Le_Mans_1968_1.jpg
tensor_sigma = 3
n_e = 4
n_a = 8
sigma_d = 3
sigma_r = 4.25
fdog_n = 1
fdog_sigma_e = 1
fdog_tau = 0.99
fdog_sigma_m = 3
fdog_phi = 2
phi_q = 3
quant_levels = 8
output rgb image = cartoon_image.jpg

Detected edges:


Resulting rendered image:


Let's go back to the default settings and change fdog_n from 1 to 2. This should make the edges much more defined.

Input to "The Cartoonist":

input rgb image = Le_Mans_1968_1.jpg
tensor_sigma = 3
n_e = 2
n_a = 4
sigma_d = 3
sigma_r = 4.25
fdog_n = 2
fdog_sigma_e = 1
fdog_tau = 0.99
fdog_sigma_m = 3
fdog_phi = 2
phi_q = 3
quant_levels = 8
output rgb image = cartoon_image.jpg

Detected edges:


Resulting rendered image:


Let's go back to the default setting and change fdog_sigma_e from 1 to 0.5. This should make the edges thinner.

Input to "The Cartoonist":

input rgb image = Le_Mans_1968_1.jpg
tensor_sigma = 3
n_e = 2
n_a = 4
sigma_d = 3
sigma_r = 4.25
fdog_n = 1
fdog_sigma_e = 0.5
fdog_tau = 0.99
fdog_sigma_m = 3
fdog_phi = 2
phi_q = 3
quant_levels = 8
output rgb image = cartoon_image.jpg

Detected edges:


Resulting rendered image:


Personally, I like the rendered image using fdog_n = 2 best, but that's just me.

Friday, July 13, 2018

The Painter - Blade Runner

I am going to use "The Painter" to simulate digital painting. The input photo is from the movie "Blade Runner", the original, of course.


Original photograph we are gonna turn into a digital painting.

Input to the "The Painter":

source rgb image = blade-runner-sean-young-rachael-hd-wallpaper.jpg
brush radius, opacity strength, and bumpity strength = 64 0.5 1.0 32 0.5 1.0 16 0.5 1.0 8 0.5 1.0 4 0.5 1.0 2 0.5 1.0
f_sigma = 0.1
error_threshold = 60
canvas rgb image = canvas_rgb_image.png
canvas bumpity image = canvas_bumpity_image.png

The two values after each brush radius are the opacity strength and the bumpity strength. The bumpity strength is not a factor in digital painting since there is no height associated with the paint. In digital painting, you usually want the colors to blend together so a value of 0.5 for the opacity is chosen (initially). Depending on the size of the input photo, you may want to get rid of the smaller brushes right off the bat. The larger the input photo, the less you want the smaller brushes to be included (unless you dramatically reduce their associated opacity).

Resulting painting:


Painting 1.

I really don't like Deckard's rendering as it is way too coarse. Also, the small paint brushes left smidgens of paint on Rachel's face, which doesn't look good. Let's get rid of the small brushes (radius 4 and 2) and make the rendered painting finer by decreasing error_threshold (from 60.0 to 20.0). Hopefully, we will get a better render of Deckard's face. We'll worry about Rachel later. Note that instead of getting rid of the smaller brushes, I could have decreased their opacity (use an opacity of 0.1 instead of 0.5).

Input to "The Painter":

source rgb image = blade-runner-sean-young-rachael-hd-wallpaper.jpg
brush radius, opacity strength, and bumpity strength = 64 0.5 1.0 32 0.5 1.0 16 0.5 1.0 8 0.5 1.0
f_sigma = 0.1
error_threshold = 20
canvas rgb image = canvas_rgb_image.png
canvas bumpity image = canvas_bumpity_image.png

Resulting painting:


Painting 2.

I like Deckard's rendering but I think Rachel's could be a little bit coarser. Let's raise error_threshold from 20.0 to 30.0 leaving the brushes alone.

Input to "The Painter":

source rgb image = blade-runner-sean-young-rachael-hd-wallpaper.jpg
brush radius, opacity strength, and bumpity strength = 64 0.5 1.0 32 0.5 1.0 16 0.5 1.0 8 0.5 1.0
f_sigma = 0.1
error_threshold = 30
canvas rgb image = canvas_rgb_image.png
canvas bumpity image = canvas_bumpity_image.png

Resulting painting:


Painting 3.

Now, I like Rachel's rendering but I think Deckard's rendering is too coarse. This happens often but there is an easy solution: simply combine the two paintings into one. How do you do that? In Gimp, you load the two paintings as layers, turn on layer transparency (click on "Layer", "Transparency", and "Add Alpha Channel"), erase what you don't like on the top layer with a soft brush, and export the result.

This a combo of painting 2 and painting 3 (I put painting 3 on top of painting 2 and erased Deckard's face to reveal the one underneath):


Combo painting 2 and painting 3.

Just for giggles, this a combo of painting 1 and painting 2 (I put painting 1 on top of painting 2 and erased Deckard's face to reveal the one underneath):


Combo painting 1 and painting 2.

Personally, I like the coarser look on Rachel better (although I should have removed or muted those small brushes) but it's really a matter of taste.

Monday, July 9, 2018

Non Photorealistic Rendering Software - The Watercolorist

"The Watercolorist" is software to watercolorize a photograph. "The Watercolorist" simulates/fakes wet-on-dry watercolor effects. See Non Photorealistic Rendering - Watercolor rendering (watercolorization) for the methodology that's behind it.

When you extract thewatercolorist-x64.rar, you should get something like this:


The directory "barn1024" shows you how to use the software. The "texture" directory contains the watercolor paper textures.

When you go into the directory "barn1024", you should see something like this:


The file called "thewatercoloris.bat" is the one you will double-click to run the software. It tells windows where the executable is. You need to edit the file so that the location of the executable is the correct one for your installation. I use notepad++ to edit text files but I am sure you can use notepad or wordpad with equal success.

The file "thewatercolorist_input.txt" is the input file for the software. It should look like this:

abstracted_image_after_quantize.png
../texture/free-vector-watercolor-paper-texture.jpg
-2.0
-4.0
5
-4.0
watercolor_image.png

Line 1: input image. The input image should be the abstracted image called "abstracted_image_after_quantize.png" that comes from Non Photorealistic Rendering Software - The Cartoonist. You could also use "abstracted_image_after_oabf2.png". It's quite possible to use something other than "The Cartoonist" to generate a good abstracted image. As a matter of fact, you should be able to use gimp or photoshop to get it.

Line 2: paper texture. I conveniently placed a few watercolor paper textures in the "texture" directory. You can use those or your own. There should be no spaces in the file name.

Line 3: paper texture beta (here, -2.0)

Line 4: turbulent flow beta (here, -4.0)

Line 5: edge darkening gradient convolution kernel size (here, 5)

Line 6: edge darkening beta (here, -4.0)

Line 7: output image (here, "watercolor_image.png"). There should be no spaces in the image name.

"The Watercolorist" simulates the following watercolor effects: watercolor paper texture (watercolor paint pigments have a tendency to deposit into the cavities of the paper), turbulent flow (in watercolors, there is a low frequency pigment density variation whenever waterlogged paint pigments are deposited on the paper), and edge darkening (paint pigments have a natural tendency to migrate to edges and deposit there in greater numbers). All those watercolor effects are simulated using the same technique: the current color is darkened/lightened according to a grayscale image that's supposed to represent pigment density. The higher the pigment density, the darker the color is going to be as more pigments get deposited onto the paper. The parameter "beta" (there is one for each watercolor effect) is used to scale that grayscale image so that the effect can be amplified/muted. The lower the beta, the more extreme the darkening/lightening will be.

"The Watercolorist" outputs images after each watercolor effect has been applied:
- image_after_paper_texture.png. This is the rendered image after the paper texture watercolor effect has been applied.
- image_after_turbulent_flow.png. This is the rendered image after the turbulent flow watercolor effect has been applied.
- image_after_edge_darkening.png. This is the rendered image after the edge darkening watercolor effect has been applied. It should the same as the output image by the way (here, watercolor_image.png).


image_after_paper_texture.png


image_after_turbulent_flow.png


image_after_edge_darkening.png

Posts that feature "The Watercolorist":
- The Watercolorist - Françoise Hardy in the movie "Grand Prix"

Non Photorealistic Rendering Software - The Cartoonist

"The Cartoonist" is software to abstract and stylize an input photograph in such a way that the output photograph kinda looks like a cartoon with thick lines and cel shading (toon shading). Details of the methodology can be found in Non Photorealistic Rendering - Image Abstraction by Structure Adaptive Filtering as well as a relatively thorough description of the parameters used by "The Cartoonist". The code comes from a "Processing" implementation by Thomas Lindemeier which I ported to C++ and to which I added support for basic image formats: jpeg, png, and tiff. At some point, I may re-implement the whole thing as I don't particularly like using other people's code.

When you extract thecartoonist-x64.rar, you should see something like this:


The directory "barn1024" shows how to use the software. If you go into "barn1024", you should see something like this:


The file called "thecartoonist.bat" is the one you will double-click to run the software. It tells windows where the executable is. You need to edit the file so that the location of the executable is the correct one for your installation. I use notepad++ to edit text files but I am sure you can use notepad or wordpad with equal success.

The file that controls the input to the software is "thecartoonist_input.txt". It should look like this:

barn1024.jpg
3.
2
4
3.
4.25
1
1.
0.99
3.
2.
3.
8
cartoon_image.png

Line 1: name of the input photograph (here, barn1024.jpg). There should be no spaces in the image name.

Line 2: tensor_sigma (here, 3.)

Line 3: n_e (here, 2)

Line 4: n_a (here, 4)

Line 5: sigma_d (here, 3.)

Line 6: sigma_r (here, 4.25)

Line 7: fdog_n (here, 1)

Line 8: fdog_sigma_e (here, 1.)

Line 9: fdog_tau (here, 0.99)

Line 10: fdog_sigma_m (here, 3.)

Line 11: fdog_phi (here, 2.)

Line 12: phi_q (here, 3.)

Line 13: quant_levels (here, 8)

Line 14: name of the output image (here, cartoon_image.png). There should be no spaces in the image name.

The meaning of the parameters that control the "The Cartoonist" are explained in Non Photorealistic Rendering - Image Abstraction by Structure Adaptive Filtering.

The Cartoonist outputs a bunch of images during to the course of its execution. They are (in alphabetical order):
- abstracted_image_after_oabf.png. This is after the "Separated Orientation Aligned Bilateral Filter" (aka "Separated OABF" or simply OABF) has been applied for n_e iterations to the currently rendered image.
- abstracted_image_after_oabf2.png. This is after the "Separated Orientation Aligned Bilateral Filter" (aka "Separated OABF" or simply OABF) has been applied for n_a-n_e iterations to the currently rendered image (after the edges have been detected).
- abstracted_image_after_quantize.png. This is after the currently rendered image has been color quantized. This is a very important image as it can be used in other software such as "The Watercolorist" as input photograph.
- edges_image.png. This image shows the detected edges by the "Separated Flow-based Difference of Gaussians" (aka "Separated FDoG" or simply FDoG).
- tangent_flow.png. This image shows the local orientation aka the Edge Tangent Flow (ETF). It was obtained using Line Integral Convolution (LIC).


abstracted_image_after_oabf.png


abstracted_image_after_oabf2.png


abstracted_image_after_quantize.png


edges_image.png


tangent_flow.png

Here is the output produced by "The Cartoonist" (cartoon_image.png), a blend of the abstracted image after color quantization and the edges image:


Posts featuring "The Cartoonist":
- The Cartoonist - Ford GT40 at Le Mans 24 hours race in 1968

Sunday, July 8, 2018

Non Photorealistic Rendering Software - The Painter

"The Painter" is software to painterly render a photograph. Details of the methodology can be found in Non Photorealistic Rendering - Stroke-Based Rendering (SBR).

When you extract thepainter-x64.rar (windows 64 bit), you will end up with something like this:


The directories "barn1024_brushmarks", "barn1024_digital", and "barn1024_impasto" show how to use the software to get painting effects that kinda simulate oil painting with visible brush marks, digital painting, and impasto oil painting, respectively. The directory "texture" contains the brush textures (opacity and height aka bumpity) for various brush radii.

If you go into "barn1024_impasto", you should see something like this:


The file called "thepainter.bat" is the one you will double-click to run the software. It tells windows where the executable is. You need to edit the file so that the location of the executable is the correct one for your installation. I use notepad++ to edit text files but I am sure you can use notepad or wordpad with equal success.

The file that controls the input to the software is "thepainter_input.txt". It should look like this:

barn1024.jpg
64 1.0 1.0 32 1.0 1.0 16 1.0 1.0 8 1.0 1.0 4 1.0 1.0 2 0.5 1.0
0.1
60.0
canvas_rgb_image.png
canvas_bumpity_image.png

Line 1 contains the name of the input photograph you want to render (here, barn1024.jpg). There should be no spaces in the image name.

Line 2 contains the list of brush radii (here, 64, 32, 16, 8, 4, and 2) you want to use for the painting. Each brush radius is followed by the opacity strength (between 0.0 and 1.0) and the bumpity strength (between 0.0 and 1.0) for that brush radius.

Line 3 contains the Gaussian blur sigma factor (here, 0.1). For the input photo not to be smoothed at all, use a small value like 0.00001.

Line 4 contains the color error threshold (here, 60.0). The larger the color error threshold, the coarser the painting will look.

Line 5 contains the name of the output canvas image (here, canvas_rgb_image.png). There should be no spaces in the image name.

Line 6 contains the name of the output height map aka bumpity map (here, canvas_bumpity_image.png). There should be no spaces in the image name.

For each brush radius XXX, there needs to be a file called textureXXX.txt. Here, because the brush radii are 64, 32, 16, 8, 4, and 2, the files texture064.txt, texture032.txt, texture016.txt, texture008.txt, texture004.txt, texture002.txt must be present. These texture files contain the textures to be used for the brushes. Let's look, for example, at texture064.txt. It should look like this:

../texture/064/opacity04.png
../texture/064/bumpity04.png

Line 1 is the image that describes the opacity of the brush.

Line 2 is the image that describes the height (aka bumpity) of the brush.

You can use as many opacity/bumpity texture combos you want. The software will pick a texture at random among the ones that are present in the texture file. For example, you could have "texture064.txt" looking like this:

../texture/064/opacity04.png
../texture/064/bumpity04.png
../texture/064/opacity04_var1.png
../texture/064/bumpity04_var1.png
../texture/064/opacity04_var2.png
../texture/064/bumpity04_var2.png

The software would then pick at random, for each brush stroke to paint on the canvas, either the combo opacity04.png/bumpity04.png, opacity04_var1.png/bumpity04_var1.png, or opacity04_var2.png/bumpity04_var2.png. Of course, you would have to create those texture variations yourself.

The brush textures are stored in the "texture" directory. What does a typical brush texture look like? Let's have a look at the brush texture images for brush radius = 64.


Brush texture opacity image "opacity04.png".

As you build up layers of paint on the canvas with brush strokes, if the opacity is high (grayscale value is close to white), the paint that's underneath the current brush stroke will not be seen by transparency. If the opacity is low (grayscale value is close to black), the paint that's underneath the current brush stroke will be seen by transparency. The lower the opacity, the more the paint that's underneath will show. If the opacity is pure black, the current brush stroke will not deposit any paint on the canvas.

The opacity value coming from the brush texture opacity image is multiplied by the opacity strength to give the actual opacity value that's gonna be used. So, for example, if the opacity strength is set at 0.5, the opacity that is actually used is half the opacity that's in the brush texture opacity image. This enables to change the opacity quite easily without having to change the brush texture opacity image.


Brush texture bumpity image "bumpity04.png".

I am calling the brush height map the brush bumpity map because it rhymes with brush opacity map. It's really a brush bump map which is applied to the canvas bump map (aka canvas height map or canvas bumpity map). The brush bumpity map describes the height of the paint that's deposited by the brush. If the value is middle gray (grayscale value = 128), there is no bump and the paint is laid flat. The white the value, the more height the paint will have when it is deposited by the brush stroke. Here, we are simulating impasto oil painting which is characterized by thick layers of paint.

Just like with opacity, the actual bumpity used is the bumpity in the brush texture bumpity image times the bumpity strength.


This is the canvas rgb image.


This is the canvas bumpity image.

To apply the canvas bumpity map to the canvas rgb image, I use gimp:

- Open as layers the canvas rgb image and the canvas bumpity image
- Activate the layer containing the canvas rgb image and click on Filters->Map->Bump Map
- For the Bump Map, choose the canvas bumpity image
- You may want to adjust the Depth Parameter to adjust the height
- Click OK and then export the bumpmapped image


This is the bumpmapped canvas rgb image.

Posts about "The Painter":
The Painter - Blade Runner

Thursday, July 5, 2018

3D Photos - Pink Flamingos


Left image (courtesy of Masuji Suto).


Right image (courtesy of Masuji Suto).

I used er9b to rectify/align the images.

er9b outputs the mean vertical disparity error and the min/max disparities which can be used as input to the depth map generators. I will typically use the min/max disparities given in the second line as it slightly expands the range given in the first line and gets rid of outliers.

Mean vertical disparity error = 0.256639
Min disp = -68 Max disp = 6
Min disp = -75 Max disp = 13


Rectified left image.


Rectified right image.

Note that, in this case, there is really not much difference between the input images and the rectified images. This means the input images were well aligned to begin with. Always a good thing.

I used dmag5 to get the depth map as it behaves the best, in general.

Input to dmag5:
min disparity for image 1 = -75
max disparity for image 1 = 13
disparity map for image 1 = depthmap_l.png
disparity map for image 2 = depthmap_r.png
occluded pixel map for image 1 = occmap_l.png
occluded pixel map for image 2 = occmap_r.png
radius = 16
alpha = 0.9
truncation (color) = 30
truncation (gradient) = 10
epsilon = 255^2*10^-4
disparity tolerance = 0
radius to smooth occlusions = 9
sigma_space = 9
sigma_color = 25.5
downsampling factor = 2

Note that I used the min/max disparities given by er9b in my input to dmag5.


Depth map produced by dmag5.

I used dmag9b to improve the depth map.

Input to dmag9b:
sample_rate_spatial = 32
sample_rate_range = 8
lambda = 0.25
hash_table_size = 100000
nbr of iterations (linear solver) = 25
sigma_gm = 1
nbr of iterations (irls) = 32
radius (confidence map) = 12
gamma proximity (confidence map) = 12
gamma color similarity (confidence map) = 12
sigma (confidence map) = 32


Depth map improved by dmag9b.

If you want to further smooth out the disparities, you are welcome to use any of the edge preserving smoothers I have on offer. Maybe eps7 or eps9 would make things smoother. I don't think it's necessary to do more smoothing though.

I used wigglemaker to generate the 3d gif aka wigglegram aka wobbler.


3d gif created by wigglemaker.

Wigglemaker uses a single image and depth map to produce the wigglegram. If you want 3d gifs that make use of both images and depth maps (left and right), you can use fsg6. The advantage of using both images and depth maps is that there is virtually no inpainting.