Development of an eye-tracking and voice command interface to facilitate GUI operation for people with disabled upper limbs

Abstract

Purpose Eye-tracking technology can be used for interaction functions involving the manipulation of various types of web content by replacing the functions of conventional input devices, such as mouse and keyboard. However, eye-tracking technology can also cause serious errors in pointing and executing pointed objects when using a web browser. Owing to the small size of executable objects, pointer execution errors cause the unintended execution of objects and difficulty in pointing. Methods We propose an interface that solves these two types of pointer execution errors for people with disabled upper limbs to control a graphical user interface. The proposed interface dynamically magnifies the tracked execution objects and executes objects pointed at by eye-tracking using voice commands. We develop a Chrome extension for this interface and use the interface by embedding the program in the browser. Results We verify the effect of this interface by conducting experiments on the error reductions of pointer execution and on proper magnification ratio identification. The results show that the eye-voice interface lowered the error rate of pointer execution and proved effective. Conclusion We confirmed that an eye-tracking interface combined with voice commands is effective in executing menus and execution objects in a web browser.