Computer Vision: How To Cartoonize Video Footage

Computer vision is an interdisciplinary scientific field that deals with how computers can be made to gain a high-level understanding of digital images or videos. From the perspective of engineering, it seeks to automate tasks that the human visual system can do.

There are many popular libraries and tools that are used, so we can do different operations in order to find solutions for the problems of this area. One of the most popular libraries is OpenCV (learn how to blend images using OpenCV).

Today we are going to use it, to create something cool and interesting. We are going to cartoonize live video footage. Keep in mind that you can not only cartoonize video footage but you can cartoonize an image as well.

The problem:

For this problem, or better for this case we are going to use Python programming language because it is easy to use and is really understandable for everybody, and OpenCV methods and functions are easy to implement in Python.

import cv2
import numpy as np

The first step is to import the libraries. We need the OpenCV (cv2) library and the NumPy library which is used for converting our image into NumPy arrays because some of the methods of OpenCV require an image to be a NumPy array.

Copy to Clipboard

def cartoonize_image(img, ds_factor=4, sketch_mode=False):

# Convert image to grayscale
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Apply median filter to the grayscale image
img_gray = cv2.medianBlur(img_gray, 7)

# Detect edges in the image and threshold it
edges = cv2.Laplacian(img_gray, cv2.CV_8U, ksize=5)
ret, mask = cv2.threshold(edges, 100, 255, cv2.THRESH_BINARY_INV)

# 'mask' is the sketch of the image
if sketch_mode:
    return cv2.cvtColor(mask, cv2.COLOR_GRAY2BGR)

# Resize the image to a smaller size for faster computation
img_small = cv2.resize(img, None, fx=1.0/ds_factor, fy=1.0/ds_factor, interpolation=cv2.INTER_AREA)
num_repetitions = 10
sigma_color = 5
sigma_space = 7
size = 5

#Apply bilateral filter the image multiple times
for i in range(num_repetitions):
img_small = cv2.bilateralFilter(img_small, size, sigma_color, sigma_space)

img_output = cv2.resize(img_small, None, fx=ds_factor, fy=ds_factor, interpolation=cv2.INTER_LINEAR)

dst = np.zeros(img_gray.shape)

# Add the thick boundary lines to the image using 'AND' operator

dst = cv2.bitwise_and(img_output, img_output, mask=mask)
return dst

This is the function that we implemented and it is going to help us to cartoonize the live video footage. There are comments on almost every line of code, so you can understand what is happening, but we are going to explain to you every single line anyway.

Firstly the function takes three arguments, the image, the blur factor, and the sketch mode. The first argument is pretty self-explanatory, the second is blurring the image on order for the image to get the effect like it is painted, and the third is going to determine if the footage has color or not (if it is set to “False”, the footage is going to have color).

Copy to Clipboard

First, we are converting the image to gray in order to use some of the required operations and for faster execution time.

Copy to Clipboard

Then we are applying a median filter in order to remove any noise from the footage.

Copy to Clipboard

After that, we are converting the image to RGB if the sketch mode has value “True”.

Copy to Clipboard

Then we are doing the final resizing and thickening the boundary lines of the image.

Copy to Clipboard

These lines are used to activate the laptop camera or your PC camera. The next two lines are variables that are going to help us to switch between sketching modes by pressing different keys on the keyboard.

Copy to Clipboard

Then we check what we pressed on the keyboard and call the cartoonie_image function. In the two different calls the sketch_mode attribute has different values and produces two different outputs (we mentioned what the output will look like earlier in this post).