Video Screen Commentary System Supporting Online Learning of Visually Impaired Students

Abstract

Because visually impaired students cannot use visual materials in a non-face-to-face lecture system, their understanding of the class content decreases, leading to an infringement of their right to learn. In this paper, we therefore propose a service that automatically provides voice commentary on a video lecture screen to increase the class understanding of visually impaired students. First, to provide an appropriate description of each screen, the screen switching point and the slide number in the video are identified using a similarity analysis algorithm applied between the frame and image. Subsequently, commentary files are created around the text, image, and table in each slide, and commentary voice files are generated using text-to-speech technology. Finally, the original lecture and commentary videos are merged according to their order of appearance on the slide. We conducted a usability evaluation experiment by organizing an assessment using blind students and non-disabled students who had their vision blocked. As the results indicate, the understanding of the lecture content increased when the students heard the explanation on the screen when using the proposed service. Through this evaluation, we confirmed that the proposed service assists in the learning of visually impaired students. © 2022, Springer Nature Switzerland AG.