Implementation of a Multisensory Strategy for Emotion Recognition

Multisensorial emotion recognition methods require several sensors to collect relevant data from expressions, as these systems are more complex than a single sensor in terms of number and diversity of sensors involved, and computational complexity of data-interpreting algorithms. This work presents a multisensorial integration strategy for emotions recognition, with three methods of integration implemented, which are Decision-Level, Feature-Level and Hybrid Level. The advantage of such multisensorial system was the three sensors (eye tracker, Kinect and thermal camera) combined lead to a better and varied analysis of emotional aspects, allowing the evaluation of focal attention, valence and arousal detection, and emotion recognition. This system also presents the potential to analyze people’s emotions by facial features using contactless sensors in semi structured environments, such as clinics, laboratories, or classrooms.