Dual-Modality Drone Control: Vision-Based Hand Gesture Recognition and Autonomous Path Following in CoppeliaSim

Abstract

This paper presents a dual-modality approach for controlling quadcopter drones in two distinct simulation scenarios: vision-based hand gesture recognition and autonomous path-following within the CoppeliaSim simulation environment. The proposed solution leverages MediaPipe's hand-tracking framework to enable real-time gesture-based control, providing an intuitive interface for users to manipulate the drone's position without the need for complex remote controllers. Simultaneously, an autonomous path-following algorithm, utilizing PID-based control and dynamic path interpolation, ensures precise and stable navigation along predefined trajectories. A comprehensive simulation setup demonstrates the system's capabilities: in the first scenario, a quadcopter drone responds to hand gestures for human control, while in the second scenario, it autonomously navigates complex paths. Experimental results highlight the system's robustness, responsiveness, and adaptability, suggesting potential applications in autonomous surveillance, human-drone interaction, and robotics education. © 2025 IEEE.