Emotion Recognition in Comics: The Effect of Visual Morphemes in Visual Narrative Contexts

Abstract

In comics, the term “visual morphemes” refers to one type of graphic structure that can be combined with other graphic elements to generate diverse meaning. For instance, the visual morphemes of whirlwind-shaped lines indicate confusion if they are placed above a person’s head. Prior empirical research has shown that such emotive visual morphemes do in fact help comic readers recognize the emotions of comic characters. However, there has been little evidence of the effect of emotive visual morphemes when emotion recognition is required in narrative contexts where multiple images are arranged to form a story, as opposed to when in solitary images of character-morpheme dyads. This study thus examined how emotive visual morphemes affect the identification of character’s emotions in narrative contexts consisting of three image panels. Results showed that emotion recognition was slower when the visual morphemes were not corresponding to the emotions of characters than when they were corresponding or when they were not provided at all. The findings thus add to our understanding of visual morpheme processing by providing empirical support for the emotive visual morpheme effects in the visual narrative structure.