Viability of Automating Annotation through MediaPipe’s Effectiveness on Pose Points Model Accuracy

Abstract

This research investigates the effectiveness of automated annotation using MediaPipe for human motion recognition tasks, comparing its performance against manually annotated data from the MP2 dataset. Through the evaluation of three machine learning models—Generative Adversarial Network (GAN), Dense model, and Transformer model—applied to both datasets, we assess the impact of dataset quality and annotation methods on model performance. The findings indicate that while models trained on MediaPipe-annotated data generally outperformed those trained on MP2, the overall accuracy remained low across all models, highlighting challenges in generalization. The study identifies the need for high-quality automated annotations that approach the granularity of manual annotations to improve performance. Moreover, it suggests that environmental factors such as lighting, background, and camera angles, which can affect joint detection accuracy, contribute to performance inconsistencies. The research also emphasizes the importance of data preprocessing, augmentation, and the potential for combining multi-modal data to enhance annotation precision. Ultimately, this study demonstrates that automated annotation offers scalability for large-scale projects but requires refinement in handling complex, dynamic environments to fully realize its potential in machine learning applications.