Look Here, Click Me: Improving Older Adults' Perception of Manipulable User Interface Components through AI-Based Perceptual Guidance

초록

Older adults often struggle to identify manipulable components in UIs due to low familiarity with digital devices. Despite existing digital literacy programs, they mainly rely on memorizing step-by-step operations in specific applications, offering limited support for adapting to new digital environments. This study proposes an AI-based perceptual training program that guides users to easily detect manipulable UI components through perceptual information, color and location. Our AI system extracts perceptual information from UI components on a screen and generates textual guidance. We conducted a user study with 13 older adults, including a training session and pre-/post-interviews, and measured their detection performance and confidence. Results showed that basic UI detection training improved participants' detection abilities, and our perceptual guidance helped older adults with low confidence better recognize UI components. We expect that our program enables older adults to fundamentally understand digital environments beyond rote memorization, supporting their long-term digital adaptation.

키워드