Computer vision is an interdisciplinary scientific field that deals with how computers can be made to gain a high-level understanding from digital images or videos. From the perspective of engineering, it seeks to automate tasks that the human visual system can do.

There are many popular libraries and tools that are used, so we can do different operations in order to find solutions for the problems of this area. One of the most popular libraries is OpenCV.

Today we are going to use it, to create something cool and interesting. We are going to cartoonize live video footage. Keep in mind that you can cartoonize not only video footage, but an image as well.

Problem

For this problem, or better for this case we are going to use Python programming language because it is easy to use and is really understandable for everybody, and OpenCV methods and functions are easy to implement in Python.

import cv2
import numpy as np

The first step is to import the libraries. We need the OpenCV (cv2) library, and the numpy library which is used for converting our image into numpy arrays because some of the methods of OpenCV require an image to be a numpy array.

def cartoonize_image(img, ds_factor=4, sketch_mode=False):

# Convert image to grayscale
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Apply median filter to the grayscale image
img_gray = cv2.medianBlur(img_gray, 7)

# Detect edges in the image and threshold it
edges = cv2.Laplacian(img_gray, cv2.CV_8U, ksize=5)
ret, mask = cv2.threshold(edges, 100, 255, cv2.THRESH_BINARY_INV)

# ‘mask’ is the sketch of the image
if sketch_mode:
    return cv2.cvtColor(mask, cv2.COLOR_GRAY2BGR)

# Resize the image to a smaller size for faster computation
img_small = cv2.resize(img, None, fx=1.0/ds_factor, fy=1.0/ds_factor, interpolation=cv2.INTER_AREA)
num_repetitions = 10
sigma_color = 5
sigma_space = 7
size = 5

#Apply bilateral filter the image multiple times
for i in range(num_repetitions):
img_small = cv2.bilateralFilter(img_small, size, sigma_color, sigma_space)

img_output = cv2.resize(img_small, None, fx=ds_factor, fy=ds_factor, interpolation=cv2.INTER_LINEAR)

dst = np.zeros(img_gray.shape)

# Add the thick boundary lines to the image using ‘AND’ operator

dst = cv2.bitwise_and(img_output, img_output, mask=mask)
return dst

This is the function that we implemented and it is going to help us to cartoonize the live video footage. There are comments on almost every line of code, so you can understand what is happening, but we are going to explain to you every single line anyway.


Firstly the function takes three arguments, the image, the blur factor and the sketch mode. The first argument is pretty self-explanatory, the second is blurring the image on order for the image to get the effect like it is painted, and the third is going to determine if the footage has color or not (if it is set to “False”, the footage is going to have color).

# Convert image to grayscale
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

First, we are converting the image to gray in order to use some of the required operations and for faster execution time.

# Apply median filter to the grayscale image
img_gray = cv2.medianBlur(img_gray, 7)

Then we are applying a median filter in order to remove any noise from the footage.

# Detect edges in the image and threshold it
edges = cv2.Laplacian(img_gray, cv2.CV_8U, ksize=5)
ret, mask = cv2.threshold(edges, 100, 255, cv2.THRESH_BINARY_INV)

The next step is detecting the edges and threshold them in order to enhance their color.

# ‘mask’ is the sketch of the image
if sketch_mode:

return cv2.cvtColor(mask, cv2.COLOR_GRAY2BGR)

After that we are converting the image to RGB if the sketch mode has value “True”.

# Resize the image to a smaller size for faster computation
img_small = cv2.resize(img, None, fx=1.0/ds_factor, fy=1.0/ds_factor, interpolation=cv2.INTER_AREA)
num_repetitions = 10
sigma_color = 5
sigma_space = 7
size = 5

#Apply bilateral filter the image multiple times
for i in range(num_repetitions):

img_small = cv2.bilateralFilter(img_small, size, sigma_color, sigma_space)

Next is resizing the image for faster computation, and applying bilateral filter. We are using the bilateral filter to preserve the edges and reduce the noise.

img_output = cv2.resize(img_small, None, fx=ds_factor, fy=ds_factor, interpolation=cv2.INTER_LINEAR)

dst = np.zeros(img_gray.shape)

# Add the thick boundary lines to the image using ‘AND’ operator
dst = cv2.bitwise_and(img_output, img_output, mask=mask)
return dst

Then we are doing the final resizing and thickening the boundary lines of the image.

if __name__==‘__main__’:

cap = cv2.VideoCapture(0)

cur_char = -1
prev_char = -1

while True:
ret, frame = cap.read()
#frame = cv2.resize(frame, None, fx=0.5, fy=0.5, interpolation=cv2.INTER_AREA)

c = cv2.waitKey(1)
if c == 27:

break

if c > -1 and c != prev_char:
cur_char = c
prev_char = c

if cur_char == ord(‘s’):

cv2.imshow(‘Cartoonize’, cartoonize_image(frame, sketch_mode=True))

elif cur_char == ord(‘c’):

cv2.imshow(‘Cartoonize’, cartoonize_image(frame, sketch_mode=False))

else:

cv2.imshow(‘Cartoonize’, frame)

cap.release()
cv2.destroyAllWindows()

The code above is the main part of the script.

cap = cv2.VideoCapture(0)
cur_char = -1
prev_char = -1

These lines are used to activate the laptop camera, or your PC camera. The next two lines are variables that are going to help us to switch between sketching modes by pressing different keys on the keyboard.

c = cv2.waitKey(1)
if c == 27:

break

Then we check if “Esc” is pressed and if it is the recording is stopped.

if c > -1 and c != prev_char:

cur_char = c

prev_char = c

if cur_char == ord(‘s’):

cv2.imshow(‘Cartoonize’, cartoonize_image(frame, sketch_mode=True))

elif cur_char == ord(‘c’):

v2.imshow(‘Cartoonize’, cartoonize_image(frame, sketch_mode=False))

else:

cv2.imshow(‘Cartoonize’, frame)

Then we check what we pressed on the keyboard and call the cartoonie_image function. In the two different calls the sketch_mode attribute has different values and produces two different outputs (we mentioned what the output will look like earlier in this post).

Results

Cartoonize Video Footage Using Computer Vision

Image 1: Screenshot of the original video footage

 

Cartoonize Live Video Footage Using Computer Vision

Image 2: Screenshot of sketch video (sketch_mode = True)

 

Cartoonize Live Video Footage Using Computer Vision

Image 3: Screenshot of sketch video (sketch_mode = False)

 

Conclusion

This post is just an example of how can you use computer vision to create some interesting stuff in your free time.

We hope that we spark a little interest in you so you will be learning more about the real-world applications of computer vision.

Like with every post we do, we encourage you to continue learning, trying and creating.

Facebook Comments