Implementation of Eye Movement-based Digital Healthcare Communication Board: Accurate Gaze Estimation Using Triplet Loss Function

Abstract

Effective communication is important in healthcare, especially for quadriplegia patients who often face barriers in expressing their needs. Traditionally, these patients rely on healthcare communication boards to interact with doctors and nurses. However, this method is often slow, cumbersome, and requires the assistance of another person which makes it an inconvenient solution. Recent advancements in gaze estimation technology, which predicts the direction of an individual's eye movements, can provide a promising alternative. This research examines the potential of gaze estimation to develop a digital healthcare communication board driven by eye movements. Such a system would empower quadriplegia patients to communicate independently which enhances both the speed and efficiency of interactions. The proposed system processes eye images to predict a gaze vector which represents the direction in which an individual is currently looking. To improve the system's accuracy, we introduce a conjugate ability-based loss function. Additionally, the proposed approach was applied to a digital healthcare system to demonstrate its feasibility and effectiveness in real-world scenarios. The system achieved an angular error of 8.7 on a public gaze estimation dataset which surpasses previous state-of-the-art methods.