SmartEye: A Machine Learning Approach to Enhance Mobility for the Visually Impaired through Depth Estimation and Object Detection

Abstract

Visually impaired individuals encounter significant challenges in their daily lives, particularly when navigating streets and public spaces. Walking in urban environments poses unique difficulties, as they must contend with obstacles, uneven surfaces, and traffic, all of which can create hazardous situations. With the growing prevalence of visual impairment, it is increasingly important to develop effective methods and technologies that can assist these individuals in safely and confidently navigating their surroundings. To address this issue, we propose SmartEye, a machine learning-based mobility assistant system that utilizes depth estimation and object detection. The system features a compact camera module mounted on the user’s glasses, which captures the environment in front of them. Through object detection and depth estimation algorithms, SmartEye analyzes the surroundings in real time, identifying obstacles and their distances. The outputs from both the object detection and depth estimation processes are then integrated to provide a comprehensive understanding of the user’s environment. This information is communicated to the individual through a speaker attached to the glasses, offering essential guidance and enhancing their mobility and safety while navigating public spaces. The proposed system achieved an absolute relative error of 0.068 and a mean average precision of 57.5 on a public dataset. Additionally, we conducted a real-world study by applying the SmartEye system to real-world street scenarios. The results demonstrated the system’s feasibility and effectiveness in assisting visually impaired individuals in navigating complex environments.