Mouse Control with Eye and Hand Tracking

I worked on this project while at a machine learning academy. 

For the eye tracking I tried using several APIs but ended up using an already existing application called gaze pointer that after calibration with just a webcam could roughly estimate where you were looking at on a screen. For the hand tracking I used google’s mediapipe library that allowed me to track specific points on a hand. I combined these two programs so that the mouse cursor would move to where you are looking on the screen and so that the mouse either left clicks, right clicks or scrolls based on your hand gestures. 

You can find my code here: 

https://github.com/farev/Gaze-Estimation-and-Hand-Tracking-Control 

In the script hand_tracking.py in the lines 82-83, I calculate the distance between specific tracking points on the thumb, index, and middle fingers which I later compare to a threshold in lines 107-123 to trigger a mouse click when your fingers get close enough.