261
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Marzo 2020
MUSIC RECOMMENDATION SYSTEM BASED ON FACIAL
EMOTION RECOGNITION
Deny John Samuvel
Department of Electronics and Communication Engineering,
Kalasalingam Academy of Research and Education,
Krishnankoil, Virudhunagar Dt., (India).
E-mail: deny.j@klu.ac.in ORCID: http://orcid.org/0000-0001-6515-3575
B. Perumal
Department of Electronics and Communication Engineering,
Kalasalingam Academy of Research and Education,
Krishnankoil, Virudhunagar Dt., (India).
E-mail: palanimet@gmail.com ORCID: https://orcid.org/0000-0003-4408-9396
Muthukumaran Elangovan
Department of Electronics and Communication Engineering,
Dr. B. R. Ambedkar Institute of Technology,
Pahargaon, Port Blair, (India).
E-mail: reachmkumaran@gmail.com ORCID: https://orcid.org/0000-0002-0763-9902
Recepción:
05/12/2019
Aceptación:
17/12/2019
Publicación:
23/03/2020
Citación sugerida:
Samuvel, D. J., Perumal, B., y Elangovan, M. (2020). Music recommendation system based on facial
emotion recognition. 3C Tecnología. Glosas de innovación aplicadas a la pyme. Edición Especial, Marzo 2020,
261-271. http://doi.org/10.17993/3ctecno.2020.specialissue4.261-271
Suggested citation:
Samuvel, D. J., Perumal, B., & Elangovan, M. (2020). Music recommendation system based on facial
emotion recognition. 3C Tecnología. Glosas de innovación aplicadas a la pyme. Edición Especial, Marzo 2020,
261-271. http://doi.org/10.17993/3ctecno.2020.specialissue4.261-271
262 http://doi.org/10.17993/3ctecno.2020.specialissue4.261-271
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Marzo 2020
ABSTRACT
Face recognition technology has widely attractedattention due to its enormous application
value and marketpotential. It is being implemented in various elds like securitysystem,
digital video processing, and many such technologicaladvances. Additionally, music is
the form of art, which isknown to have a greater connection with a person’s emotion.
Ithas got a unique ability to lift up one’s mood. Relatively, thispaper focuses on building
an ecient music recommendationsystem which determines the emotion of user using
FacialRecognition techniques. The algorithm implemented wouldprove to be more
procient than the existing systems. Moreover, on a larger dimension, this would render
salvage oftime and labor invested in performing the process manually. The overall concept
of the system is to recognize facial emotionand recommend songs eciently. The proposed
system will beboth time and cost ecient.
KEYWORDS
Recognition, Articial intelligence, OpenCV Application.
263 http://doi.org/10.17993/3ctecno.2020.specialissue4.261-271
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Marzo 2020
1. INTRODUCTION
Articial intelligence, an extensive, prominent andimperative domain that has attracted a
lot of researchers andprograms in recent times. This particular domain has takenover the
world in very short notice. It is incorporated in over daily life in the form of chatbots, digital
assistants like Siriand several other technology-based systems. One of the most prominent
powers up of articialintelligence is face recognition techniques. The basicexample of its
usage is the grouping of Google Photos of aparticular person.
There are many existing systems that could recognize facialemotions. On the other hand,
there are systems thatrecommend music. Bringing together, a system which willrecommend
music by recognizing the mood of the user fromfacial emotions is the overall concept
described in the paper. Emotion recognition would have larger scope in the nearfuture
in elds like robotics for ecient sentimentalanalysis without the involvement of another
human.
2. RELATED WORK
A few methodologies have been proposed and embraced to group human feelings
successfully. Most of themethodologies laid their emphasis on seven essential feelings which
are steady over age culture or dierent characters.
Describes the advantages of using OpenCV, especially the Adaboost algorithm, in the
process of face recognition. Detecting and recognition of face in complicated color images
can be achieved using a combination of a particular algorithm with AdaBoost algorithm. It
also talks about the disadvantages of using a timer in face detection.
Proposes on utilizing Support Vector Machines (SVM) as the primary characterization
technique to order eight facial feelings. The faces distinguished utilizing channels in
OpenCV and changed over to Greyscale. The paper likewise explains on robotized constant
coding of outward appearances in nonstop video gushing, which is feasible forapplications
in which frontal perspectives can be accepted utilizing webcam.
The creator proposed a calculation to produce a subset ofa unique playlist or a custom
playlist related to the feeling perceived. The picture to be prepared was acquired from
264 http://doi.org/10.17993/3ctecno.2020.specialissue4.261-271
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Marzo 2020
aweb camera or the hard circle itself. The picture is expose to improvements, where a few
mapping and upgrade procedures are connected to reestablish required dierentiation of
the picture. Preparing and arrangement are maintained by “one versus all” approach of
SVM to encourage multi-class characterization.
Proposes on the utilization of profound convolutional neural networks. It depends on solid
face acknowledgment convolutional systems, which can be eectively tweaked toplay out
the feeling acknowledgment task. Visual models are supplemented with sound highlights for
better face acknowledgment.
Aids in the music suggestion framework which is additionally a signicant module of the
proposed framework. It discusses highlights to be removed from the music to characterize
its mind-set.
The paper depicts utilizing Thayer’s model of mind-sets toperceive the state of mind of the
music piece. The edge level of a music piece is resolved and the feeling it brings is perceived
via prepared neural systems.
3. METHODOLGY
Compared to other algorithms used in previous systems, the proposed algorithm is procient
enough to battle large pose variations. Large pose variations tend to disrupt the eciency of
pre-existing algorithms. To reduce this Standard image input format is taken. Few systems
detect the faces rst and then locate them. On the other hand, rarely, some other algorithms
detect and locate the faces at the same time. Every face detection algorithm usually has
common steps. First, to achieve a response time, then to perform data dimension. Focusing
on data dimension a few algorithms extract facial measurements and the next react certain
relevant facial region. Advantages of the proposed algorithm Using the static image gives
a great advantage on the defect of pose variations. The three most faced problems are
the presence of unidentied elements like glasses or beard, quality of static images and
unidentiable facial gesture. Face Feature Extraction Pictures are spoken to as weight
edeigen vectors that are consolidated and known as “Eigenfaces”. One of the focal points
taken by Eigen faces is the comparability between the pixels among pictures by methods for
their covariance network.
265 http://doi.org/10.17993/3ctecno.2020.specialissue4.261-271
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Marzo 2020
Following are the means required to perceive the outwardappearances utilizing this
Eigenfaces approach:
Let X={x
1
,x
2
,...,x
n
}x
i
∊R
d
Here X be a random vector with observations.
1. Calculate the mean µ:
∑
n
n
µ=
x
i
1
i=1
2. Calculate the covariance matrix S:
∑
n
n
S=
(x
i
-µ)(x
i
-µ)
T
1
i=1
3. Compute the eigenvectors v
i
and eigenvalues λ
i
of S:
Sv
i
= λ
i
v
i
, i=1, 2,..., n
4. The eigenvectors are arranged by their egeinvalue in descending order:
y= W
T
(x-µ)
5. Calculate eigenfaces.
Eigen Faces: Not all the parts of the face are important for emotion recognition. This
key fact is considered to be important anduseful. Face recognition techniques focus on
recognizingeyes, nose, cheek and forehead and how the change with respect to each other.
Overall, the areas with maximumchanges, mathematically, areas with high variations are
targeted. When multiple faces are considered, they are compared by detecting these parts of
the faces becausethese parts are the most useful and important parts of a face. They tend to
catch the maximum change among faces, specically, the change that helps to dierentiate
one face from the other. This is how Eigen Faces face recognizer works.
266 http://doi.org/10.17993/3ctecno.2020.specialissue4.261-271
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Marzo 2020
4. SYSTEM ARCHITECHURE
Face Input
Image
Image
processing
Play Music
according to
emotion
SVM
Classifier
Emotion
Recognizer
Feature
Extraction
Song
Database
Music
data
Music
Mood
recognition
User
preferences
Application Play the song
Process
Emotion
Recognition
Graphic 1. Block Diagram.
The proposed framework is rst prepared to distinguish a face from a static picture. When
the information picture is perceived, the picture is handled. The picture isexposed to
SVM classiers for subtleties to perceive thefeeling displayed by the face. The subtleties
recuperated from the image are utilized by the feeling classier to discover feeling.
The song database and feature extraction module function simultaneously. The songs are
disintegrated into several music pieces and the mood of the song is recognized. The songs
are stored based on the mood detected. Once the emotion recognizer reports the mood, the
songs pertaining to the mood are played by the music player.
5. MODULE IDENTIFICATION
Face Detection and Recognition: Facial expressions are powerful reections of the
emotional state of a person. In this section, we will discusshow images with human faces
can be processed in order to detect the emotions presented in them. Various algorithms are
used for face recognition. Here we are using the OpenCV to detect the face in the image.
Eigenfaces algorithm is usedto recognize the face. The algorithms used for local feature
extraction are Local Binary Patterns, Direct Cosines Transform, and Gabor Wavelets.
To depict progressively trademark highlights of thespecic chose face most noteworthy
Eigenvalues of the Eigenvector will be picked as the ideal eigenface. Most noteworthy
Eigenfaces with low Eigenvalues could be discarded since they coordinated just a little piece
of trademark highlights of the countenances.
267 http://doi.org/10.17993/3ctecno.2020.specialissue4.261-271
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Marzo 2020
Face Input
Image
Image
processing
Play Music
according to
emotion
SVM
Classifier
Emotion
Recognizer
Feature
Extraction
Song
Database
Music
data
Music
Mood
recognition
User
preferences
Application Play the song
Process
Emotion
Recognition
Graphic 2. Module Explanation.
Music Feature: Music can be recommended based on available information such as the album
and artist. Another way of classifying the mood based on pitch and rhythm. Unfortunately,
this will lead to predictable recommendations. For example, recommending songs basedon
the artists that the user is known to enjoy is not particularly useful. With developing
procedures, the utilization of Neural Networks has turned out to be progressively famous.
We utilize an Articial Neural Network (ANN) to arrange the melodies in individual classes.
The dataset we utilized for preparing the model is Million Song Dataset given by Kaggle.
The information comprises of two records: metadata document and triplet document. The
metadat_le contains the title, song_id, artist_name, andrelease_by. Thetriplet_le contains
user_id, song_id and listen time.
268 http://doi.org/10.17993/3ctecno.2020.specialissue4.261-271
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Marzo 2020
Start
Read
Image
Face
detected?
Extract
Face
Emotion
Related
song
found?
Play
Song
Database
Yes
Yes
No
No
Graphic 3. Flow diagram of the proposed system.
6. CONCLUSION AND FUTURE ENHANCEMENT
A simple system is proposed here for the music recommendation using face emotion
recognition. It suggests music by extracting dierent facial emotion of a person: Happy, anger,
surprise, neutral. There is a degree for further upgrades and enhancements. Progressively
eective approaches to incorporate dierent highlights and functionalities should, in any
case, be investigated due to the lopsided nature of each element set. It is additionally seen
that to improve the exactness of the arrangement framework the informational collection
used to construct the grouping model could be expanded further.
269 http://doi.org/10.17993/3ctecno.2020.specialissue4.261-271
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Marzo 2020
REFERENCES
Bhat, A. S., Amith, V. S., Prasad, N. S., & Mohan, M. (2014). An Ecient Classication
Algorithm For Music Mood Detection In Western and Hindi Music Using Audio
Feature Extraction. 2014 Fifth International Conference on Signal and Image Processing, 359-
364. https://doi.org/10.1109/ICSIP.2014.63
Chew, L. W., Seng, K. P., Ang, L. M., Ramakonar, V., & Gnanasegaran, A. (2011),
Audio-Emotion Recognition System using Parallel Classiers and Audio Feature
Analyzer. In Proceedings of the 2011 Third International Conference on Computational
Intelligence, Modelling & Simulation (pp. 210-215), USA. IEEE. https://doi.org/10.1109/
CIMSim.2011.44
Deny, J., & Sundararajan, M. (2014). Survey of Texture Analysis UsingHistogram
in Image Processing. International Journal of Applied Engineering Research, 9(26), 8737-
8739. https://www.researchgate.net/publication/304091674_Survey_of_Texture_
Analysis_Using_Histogram_in_Image_Processing
Deny, J., & Sundhararajan, M. (2015). Multi Modal Biometric Security for
MANET Military Application-Face and Fingerprint. Journal of Computational
and Theoretical Nanoscience, 12(12), 5949-5953. https://www.researchgate.net/
publication/304380954_Multi_Modal_Biometric_Security_for_MANET_
Military_Application-Face_and_Fingerprint
Deny, J., Muthukumaran, E., Ramkumar, S., & Kartheesawaran, S. (2018).
Extraction Of Respiratory Signals And Motion Artifacts From PPG Signal
Using Modied Multi Scale Principal Component Analysis. International Journal
of Pure and Applied Mathematics, 119(12), 13719-13727. https://www.researchgate.
net/publication/325465991_Extraction_Of_Respiratory_Signals_And_
Motion_Artifacts_From_PPG_Signal_Using_Modified_Multi_Scale_Principal_
Component_Analysis
Dureha, A. (2014). An Accurate Algorithm for Generating a Music laylist based on
Facial Expressions. International Journal of Computer Applications, 100(9). https://pdfs.
semanticscholar.org/312b/2566e315dd6e65bd42cfcbe4d919159de8a1.pdf
270 http://doi.org/10.17993/3ctecno.2020.specialissue4.261-271
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Marzo 2020
Fan, X., Zhang, F., Wang, H., & Lu, X. (2012). The System of Face Detection Based on
OpenCV. In 24th Chinese Control and Decision Conference (CCDC), Taiyuan, China. IEEE.
https://doi.org/10.1109/CCDC.2012.6242980
Gilda, S., Zafar, H., Soni, C., & Waghurdekar, K. (2017). Smart Music Player Integrating
Facial Emotion Recognition and Music Mood Recommendation. In 2017 International
Conference on Wireless Communications, Signal Processing and Networking (WiSPNET), Chennai,
India. IEEE. https://doi.org/10.1109/WiSPNET.2017.8299738
Gossi, D., & Gunes, M. H. (2016). Lyric-based music recommendation. In Cheri
H., Gonçalves B., Menezes R., Sinatra R. (eds.) Complex Networks VII. Studies in
Computational Intelligence, vol. 644. Springer, Cham. https://doi.org/10.1007/978-3-
319-30569-1_23
Gupta, S. (2018). Facial emotion recognition in real-time and static images. In 2nd
International Conference on Inventive Systems and Control (ICISC), Coimbatore, India. IEEE.
https://doi.org/10.1109/ICISC.2018.8398861
Knyazev, B., Shvetsov, R., Efremova, N., & Kuharenko, A. (2018). Leveraging large
face recognition data foremotion classication. In 13th IEEE International Conference
on Automatic Face & Gesture Recognition (FG 2018), Xi’an, China. IEEE. https://doi.
org/10.1109/FG.2018.00109
Levi, G., & Hassner, T. (2011). Emotion Recognition in the Wild via Convolutional Neural
Networks and Mapped Binary Patterns. In Proceedings ACM International Conference on
Multimodal Interaction (ICMI), Seattle. https://doi.org/10.1145/2818346.2830587
Muthukumaran, E., Deny, J., Perumal, B., Suseendran, G., & Akila, D. (2015).
A narrative Non-Invasive Diagnostic loom Based by the side of correlation of
Nasal set Rhythm in addition to customary Three Radial Pulses Measurement.
Journal of Physics: Conference Series, 1228(1). https://iopscience.iop.org/
article/10.1088/1742-6596/1228/1/012075
271 http://doi.org/10.17993/3ctecno.2020.specialissue4.261-271
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Marzo 2020
Sankaranarayanan, S., & Deny, J. (2013). Securable Image Compression Using SPIHT
Algorithm. International journal of Electronics and Communication Engineering & Technology,
4(4), 96-100. https://fdocuments.in/document/securable-image-compression-
using-spiht-algorithm.html
Tzanetakisand, G., & Cook, P. (2002). Musical Genre Classication of Audio Signals.
IEEE Transactions on Speech and Audio Processing, 10(5). https://doi.org/10.1109/
TSA.2002.800560
Youssif, A. A. A., & Wesam, A. A. A. (2011). Automatic Facial Expression Recognition
System based on Geometric and Appearance Features. Computer and Information Science,
4(2), 115-124. https://doi.org/10.5539/cis.v4n2p115
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009). A survey of aect recognition
methods: Audio, visual, and spontaneousexpressions. IEEE Transactions on Pattern
Analysis and Machine Intelligence, 31(1). https://doi.org/10.1109/TPAMI.2008.52
