Virtual Mouse Using Hand Gesture Recognition

Mini Project Report

Submitted by: [Your Name]

Department of Computer Science
Academic Year: 2025

Under the Guidance of

Faculty Name
College / Institution Name
Abstract
This mini project presents a Virtual Mouse system that enables users to control the cursor and
perform mouse operations through hand gestures captured via a webcam. The project leverages
computer vision and gesture recognition to provide a touch-free interface. With the use of
MediaPipe for hand tracking and PyAutoGUI for mouse control, the system recognizes gestures for
movement, clicks, drag, scroll, volume, and brightness control. The goal is to enhance
human-computer interaction and provide an accessible, hygienic alternative to physical input
devices.
I. INTRODUCTION
Human–Computer Interaction (HCI) has evolved significantly, moving from physical input devices to
more natural interaction methods. Traditional devices like keyboards and mice are being replaced
or supplemented by touch, voice, and gesture-based systems. This project focuses on creating a
virtual mouse using real-time hand gesture recognition. By using a webcam as the input device and
MediaPipe for tracking hand landmarks, the system detects specific gestures and maps them to
corresponding mouse actions. This technology is particularly relevant for touch-free control in
healthcare, AR/VR systems, and accessibility applications.
II. RELATED WORK
Several research works and open-source projects have demonstrated the potential of computer
vision for gesture recognition. Hand gesture detection has been widely used for sign language
interpretation, robot control, and AR/VR interfaces. Using tools like OpenCV and MediaPipe,
developers have built systems capable of detecting real-time gestures with high accuracy. Our
system builds on this foundation and integrates gesture recognition with mouse control to create an
efficient, real-time virtual mouse.
III. SYSTEM OVERVIEW
The proposed system consists of real-time webcam capture, hand landmark detection, gesture
classification, and mapped control actions. The system is divided into modules for input processing,
feature extraction, gesture recognition, and system interaction. Specific gestures such as single
finger pointing, pinky raise, and fist are mapped to mouse actions like left click, right click, and drag
respectively. Additional pinch gestures control system volume and brightness.
IV. KEY FEATURES

• Real-time hand tracking using MediaPipe.

• Touch-free mouse operations (move, click, scroll, drag).

• Pinch gesture control for volume and brightness.

• Region of Interest (ROI) control for stable pointer movement.

• Smooth gesture interpretation with dead zone filtering.

V. SYSTEM ARCHITECTURE
The system architecture consists of five modules: video acquisition, hand tracking, gesture
detection, gesture-to-action mapping, and system interaction. The camera captures live video,
which is processed through MediaPipe to identify hand landmarks. Based on the detected finger
combinations, gestures are classified and translated into corresponding PyAutoGUI mouse actions.
Additional modules handle volume and brightness control using pinch gestures.
VI. ALGORITHM
Step 1: Initialize camera and load MediaPipe hand tracking module.
Step 2: Capture frame and convert to RGB.
Step 3: Detect hand landmarks and track finger positions.
Step 4: Classify gestures based on finger patterns.
Step 5: Map gestures to corresponding actions:
• INDEX finger – Left click
• PINKY finger – Right click
• V SIGN – Cursor movement
• FIST – Drag
• THUMB – Scroll
• Pinch gestures – Volume & brightness
Step 6: Execute PyAutoGUI commands.
Step 7: Repeat until user exits.
VII. IMPLEMENTATION
The system was implemented in Python 3. The key libraries used were OpenCV for video capture,
MediaPipe for gesture recognition, PyAutoGUI for controlling mouse functions, Pycaw for audio
volume control, and ScreenBrightnessControl for brightness adjustment. A simple dead zone was
applied to reduce jittery cursor movement. The program runs on standard hardware with a built-in or
external webcam.
VIII. EXPERIMENTAL SETUP
The experiments were conducted on a Windows 10 laptop with 8GB RAM and an Intel i5 processor.
A 720p webcam was used to capture hand gestures. Tests were performed in different lighting
conditions to check stability. The system was able to perform basic actions like move, click, drag,
and scroll smoothly with minimal delay.
IX. RESULTS AND OBSERVATIONS
The system successfully detected and executed mouse operations using hand gestures in
real-time. The average response time was low, and gestures were accurately recognized in well-lit
environments. Volume and brightness controls were smooth with pinch gestures. Future
improvements may focus on optimizing performance in low light.
X. FUTURE WORK

• Improve accuracy in varying lighting conditions.

• Add advanced gestures like zoom, swipe, and rotate.

• Integrate with AR/VR systems.

• Enhance system performance using ML gesture classification.

• Develop mobile and cross-platform support.

XI. CONCLUSION
The Virtual Mouse mini project demonstrates how gesture recognition can provide a practical,
touch-free alternative to conventional input devices. By using computer vision and MediaPipe, the
system accurately detects hand gestures and maps them to mouse actions. The project opens
possibilities for wider applications in accessibility, AR/VR, and future human–computer interaction
systems.
XII. REFERENCES

• OpenCV Documentation - https://opencv.org

• MediaPipe Framework - https://google.github.io/mediapipe/

• PyAutoGUI Library - https://pyautogui.readthedocs.io

• Pycaw Library for Audio Control

• ScreenBrightnessControl Library for Brightness Adjustment