Face recognition is a type of biometric identification that uses unique characteristics of a person’s face to verify their identity. This can be done through various methods, including analyzing the shape, size, and features of a person’s face, as well as the patterns and colors of their skin and hair.
There are various applications for face recognition technology, including security and surveillance, access control, and personal identification. It is often used in conjunction with other identification methods, such as fingerprints or iris scans, to provide a more reliable means of identification.
Face recognition systems can be designed to operate in real-time, allowing for the immediate identification of individuals as they appear in front of a camera. These systems can be used in a variety of settings, including airports, banks, and other public or private facilities where security is a concern.
While face recognition technology has the potential to improve security and efficiency in many areas, it also raises concerns about privacy and civil liberties. Some people worry that the use of face recognition technology may infringe on their right to privacy, or that it could be used to track and monitor individuals without their knowledge or consent. It is important for organizations that use face recognition technology to have clear policies in place to ensure that it is used ethically and responsibly.
The Face recognition systems fall under the Computer Vision (CV) umbrella. As we’ve already discussed in previous articles CV is one of the Artificial Intelligence’s (AI) fields that has the most real life implementations that are improving the day to day life of the users. As such an important branch of AI, it is one of the most desired fields for research and employment in the industry.
In this article we are going to show you how to implement face emotion recognition using the OpenCV library and then how to create a neural network model with Keras, that will allow us to increase the range of emotions that we can detect on a human face.
Implement Face Emotion Recognition Using Python
Example 1: In this example, we first load the three cascade classifiers for detecting faces, eyes, and smiles. These classifiers are trained to detect specific features in an image, such as edges, lines, and patterns, and are useful for detecting objects in images. The cascades can be found here. You can download them, and use them.
Then, we read an input image and convert it to grayscale, as the cascade classifiers work best with grayscale images. The input images should be downloaded on your local machine. The image used in this example is below the code.
Next, we use the face_cascade classifier to detect faces in the image, and draw rectangles around them. For each detected face, we extract the region of interest (ROI) and convert it to grayscale.
Then, we use the eye_cascade classifier to detect eyes in the ROI, and draw rectangles around them. Finally, we use the smile_cascade classifier to detect smiles in the ROI, and draw
import cv2
# Load the cascade classifier for detecting faces
face_cascade = cv2.CascadeClassifier(‘haarcascade_frontalface_default.xml’)
# Load the cascade classifier for detecting eyes
eye_cascade = cv2.CascadeClassifier(‘haarcascade_eye.xml’)
# Load the cascade classifier for detecting smiles
smile_cascade = cv2.CascadeClassifier(‘haarcascade_smile.xml’)
# Read the input image
image = cv2.imread(‘image2.jpg’)
# Convert the image to grayscale
gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# Detect faces in the image
faces = face_cascade.detectMultiScale(gray_image, 1.3, 5)
# Iterate over the faces
for (x, y, w, h) in faces:
# Draw a rectangle around the face
cv2.rectangle(image, (x, y), (x+w, y+h), (255, 0, 0), 2)
roi_gray = gray_image[y:y+h, x:x+w]
roi_color = image[y:y+h, x:x+w]
# Detect eyes in the face
eyes = eye_cascade.detectMultiScale(roi_gray)
# Iterate over the eyes
for (ex, ey, ew, eh) in eyes:
# Draw a rectangle around the eye
cv2.rectangle(roi_color, (ex, ey), (ex+ew, ey+eh), (0, 255, 0), 2)
# Detect smiles in the face
smiles = smile_cascade.detectMultiScale(roi_gray)
# Iterate over the smiles
for (sx, sy, sw, sh) in smiles:
# Draw a rectangle around the smile
cv2.rectangle(roi_color, (sx, sy), (sx+sw, sy+sh), (255, 0, 255), 2)
# Show the output image
cv2.imshow(‘Image’, image)
cv2.waitKey(0)
cv2.destroyAllWindows()
The image named ‘image2.jpg’ used in this example
The result of the program:
As you can see the code drew a smile over the eye as well (the purple color is for smiles). This is tied to the quality of the dataset we got. So in order to have better results, you need higher quality data.
Example 2: In this example, we first create a TensorFlow model for emotion recognition. Then we load the weights. You can find them here. Thanks and KUDOS to atulapra for the model and the weights.
Then, we load the cascade classifier for detecting faces in the image and convert the image to grayscale.
Next, we use the cascade classifier to detect faces in the image and draw rectangles around them. For each detected face, we crop it from the image, resize it to the input size of the model, and expand its dimensions.
Finally, we use the model to predict the emotion of the face, and display the predicted emotion on the image using the cv2.putText function.
import cv2
import numpy as np
# Label the emotions
from keras import Sequential
from keras.layers import Conv2D, MaxPooling2D, Dropout, Flatten, Dense
EMOTIONS = [‘angry’, ‘disgust’, ‘fear’, ‘happy’, ‘sad’, ‘surprise’, ‘neutral’]
# Create the model
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), activation=‘relu’, input_shape=(48,48,1)))
model.add(Conv2D(64, kernel_size=(3, 3), activation=‘relu’))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(128, kernel_size=(3, 3), activation=‘relu’))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(128, kernel_size=(3, 3), activation=‘relu’))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(1024, activation=‘relu’))
model.add(Dropout(0.5))
model.add(Dense(7, activation=‘softmax’))
# Load the weights
model.load_weights(‘model.h5’)
# Load the cascade classifier for detecting faces
face_cascade = cv2.CascadeClassifier(‘haarcascade_frontalface_default.xml’)
# Read the input image
image = cv2.imread(‘image4.jpg’)
# Convert the image to grayscale
gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# Detect faces in the image
faces = face_cascade.detectMultiScale(gray_image, 1.3, 5)
# Iterate over the faces
for (x, y, w, h) in faces:
# Draw a rectangle around the face
cv2.rectangle(image, (x, y), (x + w, y + h), (255, 0, 0), 2)
# Crop the face from the image
face = gray_image[y:y + h, x:x + w]
# Resize the face to the input size of the model
face = cv2.resize(face, (48, 48))
# Expand the dimensions of the face to (1, 48, 48, 1)
face = np.expand_dims(face, axis=0)
face = np.expand_dims(face, axis=–1)
# Predict the emotion of the face
emotion_index = np.argmax(model.predict(face))
emotion = EMOTIONS[emotion_index]
# Put the predicted emotion text on the image
cv2.putText(image, emotion, (x, y – 10), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)
# Show the output image
cv2.imshow(‘Image’, image)
cv2.waitKey(0)
cv2.destroyAllWindows()
Here are the results we got from the second example:
This image is the image we’ve used in the first example. As you can see the results are the same. The model in the first example detected a smile, and here we have a happy face.
In the second example we can demonstrate a list of emotions, so here is the result when we feed the mode an image with a sad face.
Conclusion
I hope you’ve found the examples useful and interesting. You can download more of those cascade classifiers and make your own spin on the examples we’ve provided.
If you are interested in learning more about Computer Science, we’ve created tons of examples and even a few Computer Science curriculums from different free sources like the MIT OpenCourseWare program, YouTube, etc.. you should check out the rest of our articles.
- FREE Courses from Harvard University to Become Data Scientist
- Learn How To Do K-Means Clustering On An Image
- Top 10 Machine Learning Books That You Should Read Before You Start With It
- Top 40 COMPLETELY FREE Coursera Artificial Intelligence and Computer Science Courses
Like with every post we do, we encourage you to continue learning, trying and creating.