Gesture based teleop for UFactory Lite 6

Using google mediapipe to get hand landmarks ratio & mapping them to End Effector Rate Control

This project implements gesture-driven teleoperation for UFactory Lite6 using MediaPipe hand tracking on Windows and ROS2 MoveIt Servo in Docker, where test.py maps gestures to velocity commands over UDP and ros2_relay.py publishes them to /servo_server/delta_twist_cmds, with optional Unity mirroring via /joint_states for fast, hardware-independent iteration.

One of the problem with gesture based teleop is hand fatigue and safe cancellation of commands. Here the movement of left hand outside of cam view acts as immedeate stop and as clutch reset mechanism which we often see in handheld telop

The displacement of wrist position from neutral marker is used to calc the velocites, and the number of fingers is used to chose the plane of effect. This effectively removes the need of depth sensor

With one hand, operator can move the end effector in all 6 domains, X,Y,Z,RX,RY,RZ with very precise movements due to rate control delta

Full setup, prerequisites, usage steps, and troubleshooting: GitHub project page