What is Emotion AI ?

Emotion AI is the process of identifying human emotions.Emotion AI is a subset of artificial intelligence that measures, understands, simulates, and reacts to human emotions.It replicates the way humans think. The technology uses multi-modality for emotion recognition.The key elements used are facial expressions, voice tonality, written text, body postures.

Empathy is Central to Emotional Intelligence

Empathy is central to identifying the true emotion of a person. It means understanding and feeling the under person with our frame of understanding.

Empathy is of three types

a) Cognitive empathy : This means understanding the other person’s feeling and state of mind.

b) Somatic empathy : This means the ability to respond with appropriate emotions.

c) Affective empathy : This considers physical response to the empathy process.

The art of understanding of these phases and responses through AI constitute a true Emotion AI system.

How emotions are identified ?

a. Voice :

For any form of audio dataset, There are plenty of useful features which are extracted from 3 different domains for data viz Time domain, Frequency domain and time-frequency domain.

Time domain features (temporal features) are easy to compute like energy of signal, zero crossing rate, maximum amplitude etc.

Frequency domain features (spectral features) are used to understand the pitch, melody of the audio. Conversion of audio to frequency domain is done using Fourier transform on time domain data. Features include spectral centroid, spectral flux, spectral rolloff etc.

Time-frequency domain (Mel-spectrogram) is used to compute the most widely used feature for audio analysis Mel-Frequency Cepstral Coefficients (MFCCs). MFCC is observed to be the exact replication for human hearing mechanism.

Techniques: LSTM, CNN, SVM, MLP, Decision tree(xgboost) are adopted architectures for emotion recognition using features mentioned above.

b. Face:

Features: Facial gestures. A better facial expression recognizer followed by a classifier proves to be a better approach.

Techniques: CNN, SVM, xgboost. etc

c. Text: Strong word and plenty of text data can be used to create a satisfactory emotion model. Text embedding can be done using various techniques like Tf-IDF, Tokenizer, BERT, WORD2VEC. Semantic labelling for the previous context is also used for Text emotion recognition.

Architectures -> LSTM models, LSTM-CNN models, SVM etc.

Decision making through Emotion AI

Emotion AI does not limit itself to emotion recognition. Probabilistic ways of taking decisions to change the Negative conversation to Positive direction is one of the achievable objectives. AI plays an important role in making interactions more efficient. AI powered customer interactions and recommendation systems are in a good position if the system itself is capable of reacting to different emotions of customers.

In the above example, you can see the extracted emotions from the voice of the person, the assistant on the basis of these inputs gives a befitting perfect reply to the person.

Business use cases of Emotion AI.

a. Advertisement : Emotion AI can be used heavily in advertisement. We can show products to a customer ,better our service if the overall emotion of a customer is going negative. We can track the emotion by analysing feedback, comments, facial expressions of a user. For e.g. if any user has rated badly and given negative feedback to a product , we have given him better discounts and advertised better deals.

b. Call centers : Empathy is very important in call centers and agents always need to be polite and to the point. Emotion AI plays a big role in tracking the performance and ratings of call center agents and overall improves the customer service experience.

c. Assistive services : Voice – assistants like Siri,Cortana,Alexa,Google home use emotion detection a lot. They try to understand the emotion of the user and accordingly reply to him. The tonality and empathy in their voice adjust according to the emotion of the talking person.

d. Patient care :  Tracking the digital well of a person is very important these days.A nurse bot not only reminds older patients on long-term medical programs to take their medication but also converses with them about emotional and mental well being by detecting the emotion of the person.