An intelligent HCI system that replaces your mouse using real-time hand gesture recognition and voice commands — no extra hardware required.
See how hand gestures and voice commands control your PC in real time — no mouse, no keyboard.
Gesture control demo
Voice assistant demo
Move your cursor by simply pointing — speed is proportional to your hand movement velocity.
Left click, right click, and double click using distinct finger gestures detected in real time.
Horizontal and vertical scrolling via pinch gestures. Speed proportional to pinch distance.
Search Google, navigate files, control volume, and more — all with spoken commands via Nova assistant.
Adjust system brightness and audio volume dynamically using pinch gesture movements.
Pick up and move files between folders using a dedicated drag gesture — no mouse needed.
OpenCV opens your webcam and captures a live video feed at real-time frame rates. No special camera needed — any standard webcam works.
Google's MediaPipe ML model detects 21 key hand landmark points per frame, mapping the exact 3D position of each finger joint.
A custom gesture classifier analyzes the landmark positions to identify which gesture is being performed — click, scroll, drag, etc.
The recognized gesture is mapped to a system action using PyAutoGUI — moving the cursor, firing a click, adjusting volume, and more.
In parallel, the Nova voice assistant listens via your microphone using SpeechRecognition and executes commands like search, file navigation, and more.
Requires Python 3.8.5 and Anaconda. Windows only. Setup instructions included in the README.
Hand gesture recognition module only. Control your cursor, clicks, scroll, drag & drop, brightness and volume — no microphone needed.
Download gesture-only.zipComplete package with both gesture recognition and the Nova voice assistant. Full hands-free PC control experience with all features enabled.
Download full-package.zip