Double Domain Converter Transformer For Improving EEG-Based Emotion Recognition from Video to Game Scenarios

Abstract

Emotion recognition (ER) plays an important role in the field of modern technology and human-computer interaction. Traditional emotion recognition approaches usually utilize videos as stimuli. However, watching video lacks interaction. Recently, more and more game-related stimuli have been used. To explore emotion recognition in a more natural state, this study pioneers the use of Genshin Impact game and gameplay-related videos to evoke positive and neutral emotions. As Electroencephalogram (EEG) feature distributions of video and game scenarios are very different, we propose the Double Domain Converter Transformer network (DDCT) to enhance EEG-based cross-scenario emotion recognition, making full use of information from different domains. Our model has two converters that can convert input data into the source domain and the target domain, diminishing distribution difference in two feature spaces. Experimental results demonstrate that our model achieves a remarkable prediction accuracy of 84.95% in video-game mix-scenario emotion recognition and 74.76% in video to game cross-scenario emotion recognition. Our code is available at https://github.com/AlbertMentat/DDCT.