Project Synopsis On

“FaceMap: AI-Powered Skin Type and

Acne Analysis in Real-Time”

is submitted in partial fulfillment of the requirement of the degree

B.E. (Computer Engineering)

By

Shruti Bhalerao (04)

Mansi Jadhav (32)
Sakshi Sonavane (71)

Under the guidance of

Prof. Diksha Bhave

Department of Computer Engineering

Shivajirao S. Jondhale College of Engineering
Dombivli (E)
(Affiliated to University of Mumbai)
(2024-25)
 Samarth Samaj`s
SHIVAJIRAO S. JONDHALE COLLEGE OF ENGINEERING
DOMBIVLI (E)
(Affiliated to University of Mumbai)

This is to certify that the project entitled “FaceMap: AI-Powered Skin Type and Acne
Analysis in Real-Time” in the partial fulfillment for BE (Computer Engineering) Degree
semester VIII during the academic year 2024-25 as prescribed by University of Mumbai.

Shruti Bhalerao (04)

Mansi Jadhav (32)
Sakshi Sonavane (71)

Name of project guide

Prof. Diksha Bhave

Prof. Reena Deshmukh Dr. Uttara Gogate

Project Coordinator Head of Department

Dr. P. R. Rodge
Principal

Internal Examiner External Examiner

 CONTENTS
Abstract i
List of Figures ii
List of Tables ii
1. Introduction 1
2. Literature Survey 3
2.1 Literature Survey review 4
3. Existing System 6
3.1 Introduction 6
3.2 Limitation of Existing System 7
4. Problem Statement & Objectives 9
4.1 Problem Statement 9
4.2 Objectives 10
5. Proposed System 11
5.1 Introduction 11
5.2 Architecture framework 12
5.3 Use case Diagram 14
5.4 Methodology 15
6. Experimental Setup 23
6.1 Details of database 23
6.2 Hardware & Software Requirements 25
7. Result 27
8. Implementation plan for semester 7&8 37,38
9.Applications 39
10. Conclusion 41
References
Acknowledgement
 ABSTRACT

Facial recognition technology enables the identification and analysis of human faces from digital
images or video frames by extracting and measuring key facial features. Beyond user authentication,
AI-driven skin analysis has emerged as a rapid and accurate diagnostic tool for assessing various
skin conditions. Leveraging advanced machine learning algorithms, this technology can detect
multiple skin concerns, including acne severity, hydration levels, redness, and wrinkles, within
seconds.

The AI-powered Skin Analyzer evaluates the T-zone and U-zone of the face to determine key skin
attributes such as oiliness, dryness, and sensitivity. This analysis helps classify skin types into
categories like normal, oily, dry, sensitive, or combination, providing a foundation for personalized
skincare recommendations.

FaceMap: AI-Powered Skin Type and Acne Analysis in Real-Time enhances skin assessment
by analyzing images from different facial angles—front, left, and right profiles. This real-time
analysis ensures comprehensive facial coverage, including traditionally challenging areas like the
chin and cheeks. By integrating AI with dermatological insights, FaceMap delivers precise, data-
driven skin assessments, empowering users with actionable skincare solutions.

Keywords— Acne Detection, Skin Type Classification, Facial Image Analysis, AI Dermatology,
Multi-Zone Skin Assessment.

i
 List of Figures

Figure Figure Page

No. Name No.
5.2.1 System Block Diagram 12
5.2.2 System Flowchart 13
5.3.1 System Use Case Diagram 14
7.1.1 Female gender detection 27
7.1.2 Male gender detection 27
7.1.3 Detection of Skin Type 28
7.1.4(A) Acne Type Prediction Application 28
7.1.4(B) Sample 1 29
7.1.4(C) Sample 2 29
7.1.4(D) Sample 3 30
7.1.4(E) Sample 4 30
7.1.4(F) Sample 5 31
7.1.5 Images of acne type 31
7.1.6 Images in Trial folder 32-34
7.1.7(A) Symptoms by Selecting Acne type 35
7.1.7(B) Sample 1 36
7.1.8 Images Matching to Acne type 36

List of Tables

Tabel
Table Name Page No.
No.
2.1 LiteratureReview 3
2.2 Timeline chart for semester 7&8 37,38

ii
ii
 1. INTRODUCTION

The condition of human skin, particularly its type and the presence of acne, plays a crucial role in
health, self-esteem, and overall appearance. However, identifying skin types and diagnosing acne
accurately in real-time remains a challenge for dermatologists and consumers alike. Conventional
skin analysis methods, often reliant on in-person consultations or manual assessments, can be time-
consuming, subjective, and inconsistent. “FaceMap: AI-Powered Skin Type and Acne Analysis
in Real-Time” is a cutting-edge solution designed to address this gap by leveraging advanced
artificial intelligence (AI) and computer vision techniques. The system offers a fully automated,
real-time analysis of skin conditions, focusing on two major aspects: skin type classification (e.g.,
oily, dry, combination) and acne detection (severity, type, and localization). This intelligent
platform can empower users with personalized skincare recommendations and enhance
dermatological consultations through accurate and quick assessments. Once the user's age and
gender are detected through the real-time analysis, FaceMap predicts the acne type and severity.
Based on the prediction, the system offers personalized suggestions, including tailored
recommendations for breakfast, lunch, dinner, detox drinks, skincare routines, and home remedies.
The system utilizes a rich dataset of annotated images and health data to generate these customized
suggestions. This comprehensive approach is aimed at providing holistic solutions to acne
problems.

FaceMap harnesses the power of deep learning algorithms and advanced image processing
techniques to deliver precise, real-time analysis of skin types and acne conditions. The system is
trained on a comprehensive dataset of annotated facial images, using convolutional neural networks
(CNNs) to extract features and classify skin types and acne severity. It ensures accuracy even under
diverse lighting conditions and varying camera qualities. By offering an AI-driven, scalable, and
objective solution, FaceMap provides personalized skincare recommendations, dietary
suggestions, and lifestyle improvements based on the analysis. This innovative system bridges the
gap between professional dermatology and user-friendly technology, making it accessible for both
individual consumers and dermatology professionals worldwide, ensuring quicker, more effective,
and accurate skin health assessments.

FaceMap generates a comprehensive report after analyzing skin type and acne conditions. This
report includes acne diagnosis, severity, and personalized recommendations for skincare and
lifestyle changes. It serves as a valuable tool for both users and dermatologists, ensuring more
1
effective treatment. The goal of FaceMap: AI-Powered Skin Type and Acne Analysis in Real-
Time is to develop an accessible AI system that provides accurate, real-time skin analysis and
personalized skincare suggestions, enhancing both user experience and dermatological diagnoses.

1. Develop an AI Model: To design and implement deep learning models capable of accurately
classifying various skin types (e.g., oily, dry, combination) using facial image data.

2. Acne Detection and Classification: To build an AI algorithm that can detect acne, assess its
severity, and classify different types of acne (e.g., whiteheads, blackheads, cystic acne) in real-time.

3. Data Collection and Training: To gather a comprehensive dataset of facial images annotated with
skin types and acne conditions to train and validate the AI models for high accuracy.

4. Real-Time Processing: To optimize the system for real-time analysis, ensuring quick and
efficient skin type and acne detection using standard camera devices, such as smartphones or
webcams.

5. User-Friendly Interface: To create an intuitive and accessible interface that delivers real-time
results and personalized skincare recommendations based on the AI analysis.

6. Scalability and Accessibility: To ensure the system can scale for widespread use, making it
accessible to individuals and dermatology professionals across different Platforms.

2
 2. LITERATURE SURVEY
2.1 Literature Review
Sr.no PAPERS ADVANTAGE DISADVANTAGE
NAME AND
YEAR OF
PUBLICATION
1 [“Skin Type Classification High Accuracy and Image Quality
Using Deep Learning.”] Consistency, Ability to Sensitivity, Limited
(2018) Learn Complex to Surface-Level
Patterns, Real-Time Analysis, Ethical and
Processing. Privacy Concerns.
2 [“Automatic Acne Object They used in Acne AI-based systems may
Detection and Acne Severity grading is often sometimes overestimate or
Grading Using Smartphone subjective, with varying underestimate acne
Images and Artificial results depending on severity due to limitations
Intelligence”] the dermatologist. in training data or image
(2022) quality.

3 [“Texture Analysis in Skin texture analysis can be advanced texture analysis

Type Classification.”] used in conjunction requires high
(2020) with other skin analysis computational resources
methods. This for processing and
classification
analysis. This can be a
performance and
provide a more barrier for deploying
comprehensive such as smartphones
assessment of skin
type.
4 [“Fairness in AI: Addressing A fair AI system is Ensuring fairness is not a
Bias in Skin Type Analysis.”] applicable to a wider one-time task but requires
(2021) audience, making it ongoing monitoring and
suitable for global adaptation.
markets with diverse
populations.
5 ["Detection of Gender and Age Employs machine The model struggles with
Using Machine Learning"] learning algorithms to complex or high-variance
detect age and gender images such as those
(2022) with moderate containing varying lighting
accuracy, especially conditions or diverse
effective on structured, backgrounds. Accuracy
smaller datasets. may drop significantly
when applied to large,
unstructured datasets

3
3
2.2 Literature Review

"Skin Type Classification Using Deep Learning" Journal of Dermatological Research, 45(3), pp.
123-130 Authors: Wang, Y., et al. (2018): This paper explores the application of deep learning
techniques for the automated classification of skin types, addressing the challenges in dermatology
for accurate and efficient skin assessment. The authors develop and evaluate a deep neural network
model capable of categorizing different skin types based on the Fitzpatrick scale (which classifies
skin into six types based on factors like sensitivity to UV light and pigmentation).[1]

[“Automatic Acne Object Detection and Acne Severity Grading Using Smartphone Images and
Artificial Intelligence”] Quan Thanh Huynh, Phuc Hoang Nguyen: This paper presents a novel
approach for the detection of acne and its severity grading using artificial intelligence (AI) applied
to smartphone images. The study aims to offer an efficient and accessible method for acne diagnosis
by leveraging the widespread availability of smartphones and advancements in AI, especially in
image processing and machine learning.[2]

[“Chen, L., et al. (2020). “Texture Analysis in Skin Type Classification.” International Conference
on Image Processing, pp. 412-419.]: This paper investigates the use of texture analysis for skin
type classification, focusing on leveraging image processing techniques to categorize skin types
based on surface texture features. Chen et al. propose a novel approach that utilizes various texture
descriptors combined with machine learning classifiers to distinguish between different skin types,
contributing to advancements in dermatology and cosmetic industries.[3]

[Groh, M., et al. (2021). “Fairness in AI: Addressing Bias in Skin Type Analysis.” IEEE
Transactions on Medical Imaging, 40(12), pp. 3230-3241.]: This paper investigates the issue of
fairness and bias in AI systems used for skin type analysis, particularly in medical and
dermatological imaging. Groh et al. focus on addressing disparities in AI model performance across

4
different skin tones and ensuring equitable outcomes for individuals from diverse demographic
backgrounds. The study highlights the need for ethical considerations in AI development,
especially for healthcare applications where biased models can result in unequal treatment and
care.[4]

Kulkarni, M.A., Joshi, M.P. (2022) "Detection of Gender and Age Using Machine Learning": This
paper presents a machine learning-based system for detecting gender and age from facial images.
It uses classical machine learning techniques like Support Vector Machines (SVM) and K-Nearest
Neighbors (KNN) for classification. The system is trained on a dataset of facial images and extracts
features like face shape and texture to predict age and gender. Advantages: Simpler implementation,
moderate accuracy, lower computational cost.Disadvantages: Struggles with unstructured, large
datasets, lower accuracy compared to deep learning models. [5]

5
 3. Existing System
3.1 Introduction
In recent years, advancements in artificial intelligence (AI) and image processing have transformed
the way skin health is analyzed. Several systems and applications have been developed to assess
skin conditions, including skin type classification and acne detection. Here’s an overview of the
existing systems:

1. Dermatology Apps and Platforms:

• Many mobile applications offer skin condition analysis, primarily focusing on
detecting common skin issues like acne, eczema, or psoriasis.
• These apps use basic image recognition algorithms to identify problem areas and
suggest treatments, often collaborating with dermatologists to provide remote
consultations. Popular examples include SkinVision, MDacne, and Dermatology
AI.[9]
2. AI-Based Skin Analysis Tools:
• Several AI-powered tools have emerged in the dermatology field, using deep
learning models to scan images and detect skin irregularities. Tools like Google
AI's dermatology assistant have been trained on vast datasets to recognize skin
conditions with high accuracy.[9]
3. Mobile Apps for Skin Analysis:
• Some mobile applications already exist that allow users to upload selfies for skin
analysis. These apps use image processing to detect surface-level conditions such
as blemishes, pores, and wrinkles. Additionally, some apps recommend skincare
products based on the identified conditions.
4. Basic Skin Analysis Tools:
• Several brands offer home-use skin diagnostic tools (such as magnifying mirrors,
UV lamps, and skin scanners) that provide some level of skin analysis. These tools
help users measure factors like skin hydration, oil levels, and surface blemishes.

6
3.2 Limitations of Existing System

Despite the progress in skincare technology, current systems for skin type and acne analysis face
several limitations. These limitations hinder their effectiveness, accuracy, and accessibility for a
broad range of users.

1. Inaccuracy in Skin Type and Acne Detection

• Surface-Level Analysis: Many existing mobile apps and diagnostic tools focus on
detecting surface-level skin conditions such as blemishes, oiliness, or dryness. However,
they often miss deeper skin issues like cystic acne, underlying inflammation, or subtle skin
type variations.
• Image Quality Dependence: The accuracy of skin analysis is often highly dependent on
the quality of the images users upload. Poor lighting, camera quality, and environmental
factors can skew results, leading to incorrect skin type classification or acne severity
assessment.

2. Lack of Personalization:

• No Consideration for Age and Gender: Most current systems do not factor in age or
gender when making skincare recommendations.Skin characteristics change significantly
with age (e.g., hormonal changes, reduced collagen production), and gender-specific factors
(e.g., facial hair, hormonal cycles) can influence acne patterns and skin health. Ignoring
these aspects leads to generic and less effective skincare advice.

• No Adaptation Over Time: Existing systems usually do not track skin health over time,
missing long-term changes in skin conditions. As a result, they provide one-time or static
advice, which may not suit users with evolving skin conditions, such as those undergoing
treatments or experiencing seasonal changes.[13]

3. Limited Scope of Acne Detection

• Inability to Differentiate Between Acne Types: Many current tools do not differentiate
between various acne forms (e.g., blackheads, whiteheads, cystic acne). This lack of
7
 granularity leads to oversimplified analysis and recommendations, potentially resulting in
ineffective treatments.
• Lack of Severity Grading: Existing solutions often fail to assess acne severity accurately.
Identifying the progression of acne—from mild to severe—can significantly influence
treatment options, but most systems don't offer this capability.[11]

4. User Experience and Accessibility Challenges

• High Cost of Professional Tools: Advanced skin diagnostic devices, often found in
dermatologists' offices, are not affordable or accessible to the general public. This limits
high-quality skin analysis to those who can afford costly consultations.
• Inconsistent User Experience: Many mobile apps offering skin analysis provide
inconsistent results due to differences in camera quality, lighting conditions, and user
interaction. This lack of standardization affects the reliability of the analysis.
• Limited Guidance on Follow-Up Care: While some apps offer product recommendations,
they often fail to provide a holistic plan for ongoing skincare or acne treatment. Users are
left without guidance on how to adapt their routine as their skin condition evolves.

5. Privacy and Data Security Concerns

• Inadequate Data Protection: Many skincare apps collect sensitive personal data,
including facial images and demographic information. However, they often lack robust
privacy measures or fail to transparently communicate how user data is stored and
processed. This raises concerns about data security and misuse.[12]

8
 4. PROBLEM STATEMENT & OBJECTIVES
4.1 Problem Statement

Skincare is a highly personalized domain, yet existing tools for skin type and acne analysis fail to
deliver accurate, tailored, and easily accessible solutions. Current systems rely on either manual
dermatologist consultations, which are often expensive and inaccessible, or mobile apps and
devices that provide limited, surface-level insights. These tools typically lack the ability to consider
critical demographic factors such as age and gender, leading to generalized and often ineffective
skincare recommendations. Moreover, most of these systems struggle with accurately detecting the
type and severity of acne, which is essential for effective treatment. They also depend heavily on
the quality of user-submitted images, resulting in inconsistent or inaccurate analyses. Furthermore,
there is little to no guidance on how to adjust skincare routines over time, leaving users without a
clear path for ongoing skin health management.

Key problems include:

1. Inaccurate Skin Type and Acne Analysis due to reliance on surface-level image
processing and inadequate consideration of real-time conditions.
2. Lack of Personalized Skincare Recommendations that factor in critical demographic
elements such as age and gender, which significantly influence skin characteristics.
3. Inability to Provide Ongoing Skin Health Monitoring and evolve skincare advice based
on changing skin conditions over time.
4. Inconsistent and Inaccessible Solutions, with most tools either being too costly
(professional-grade) or unreliable (app-based).
5. Privacy and Data Security Concerns, especially regarding the handling of sensitive facial
image data.

Face Map's Goal: To develop an AI-powered, real-time skin type and acne analysis system that
overcomes these challenges by providing accurate, personalized, and secure skincare insights based
on detailed image processing, while considering demographic factors such as age and gender.

9
4.2 OBJECTIVES
The primary objective of FaceMap is to leverage AI and image processing technologies to provide
users with accurate, real-time analysis of their skin type and acne conditions, while also
incorporating demographic factors like age and gender for personalized skincare recommendations.
The system aims to address the limitations of existing solutions by offering a comprehensive,
accessible, and secure tool for ongoing skin health management.
Accurate Skin Type Identification:
• To develop AI algorithms capable of analyzing high-quality facial images to accurately
classify skin types (e.g., oily, dry, combination, sensitive). To ensure that the system
provides results that are comparable to dermatological evaluations.
Comprehensive Acne Detection and Severity Assessment:
• To create image processing techniques that can detect various forms of acne (e.g.,
blackheads, whiteheads, cystic acne) and assess the severity of acne conditions.
• To provide users with actionable insights into their acne, including treatment suggestions
based on the identified severity.
Ongoing Skin Health Monitoring:
• To enable users to track changes in their skin condition over time, with the system providing
updated recommendations as their skin evolves due to treatments, lifestyle changes, or
seasonal factors.
• To ensure that users receive continuous guidance for improving their skincare routine based
on the ongoing analysis of their skin’s health.
Privacy and Data Security:
• To implement strong data protection measures to ensure that user facial images and personal
information are handled securely, with options for anonymization.
• To maintain transparency about how data is stored and used, building user trust and ensuring
compliance with privacy regulations.
Accessibility and Affordability:
• To provide an affordable solution that is accessible to a wide range of users, overcoming
the cost and geographic barriers associated with professional dermatological consultations.

10
 5. PROPOSED SYSTEM
5.1 Introduction

FaceMap is a cutting-edge AI-based project designed to provide instant skin type and acne analysis.
It leverages machine learning and computer vision techniques to assess skin conditions in real-time.
The primary goal of FaceMap is to offer personalized skincare recommendations by analyzing the
user's skin through images or video input, identifying various skin types (such as oily, dry,
combination, or sensitive), and detecting acne and related issues.

Key Features:

 Real-Time Skin Type Detection:

 Analyzes facial skin using AI models to determine skin type accurately.
 Uses image data to classify skin based on characteristics like oiliness, moisture, and
sensitivity.
 Acne Detection and Severity Analysis:
 Identifies acne, blackheads, whiteheads, and blemishes.
 Assesses acne severity, helping users track progress over time.
 Personalized Skincare Recommendations:
 Provides suggestions for suitable products and skincare routines based on the user's
skin type and acne condition.
 Data-Driven Insights:
 Tracks skin health changes over time.
 Provides users with insights and progress reports.
 Age and Gender Detection:
 Integrates facial recognition technology to estimate the user’s age and determine
gender.
 Adapts skincare advice and product recommendations based on demographic
factors, ensuring relevance to different age groups and skin characteristics.
FaceMap is envisioned to revolutionize skincare by offering accessible, tech-driven solutions for
better skin management and care.

11
5.2 Architecture Framework

Figure 5.2.1 System Block Diagram

As Seen in figure 5.2.1, The block diagram for "FaceMap: AI-Powered Skin Type and Acne
Analysis" outlines the system's workflow, where a camera captures facial images that undergo
preprocessing before feature extraction and analysis using machine learning. The system classifies
skin type, detects acne, and displays the results to the user. Additionally, a personalized skincare
recommendation report is generated, providing a comprehensive approach to skin health
assessment.

12
 Figure 5.2.2 System Flowchart
The diagram illustrates a system that uses face recognition, followed by age and gender prediction,
to perform skin type analysis based on Ayurvedic classifications (Vattaj, Pittaj, Kaphaj). It further
refines this analysis by considering mixed skin types (Vatta-Pitta, Pitta-Kaphaj, Vatta-Kaphaj).
Based on the identified skin type and acne type, the system conducts an acne analysis and provides
treatment recommendations rooted in Ayurveda. A report summary is generated using a doctor-
provided database, integrating both predefined and analysis data to offer personalized skincare
advice.

13
5.3 Use Case Diagram

Figure 5.3.1 System Use Case

The diagram illustrates the workflow of a skincare system involving three main roles:
Administrator, Patient, and Domain Expert. The Administrator manages system administration
and patient records. The Patient provides input by either uploading an image or capturing a real-
time facial image, which undergoes face recognition. The system then extracts features, performs
skin type analysis, and detects acne. The Domain Expert analyzes the data and conducts
Ayurvedic-based reasoning to provide insights into acne causes. This structured approach ensures
accurate skin assessments and supports personalized skincare recommendations.

14
5.4 Methodology

The methodology adopted to solve the problem of real-time skin type and acne analysis in the
"FaceMap" project involves a systematic approach that combines machine learning, image
processing, and real-time mobile application development. Here's an overview of the steps and
techniques used:

1. Problem Definition and Requirements Analysis

• Identify the Problem: The primary problem is to accurately classify skin types and detect
acne severity in real-time using images captured from a mobile device.
• User Requirements: Understand the end-user needs, including real-time processing, ease
of use, and actionable insights. This is crucial for defining the project's scope and technical
requirements.
• Technical Requirements: Define the hardware and software requirements, including the
need for a high-quality camera, sufficient processing power, and reliable storage.

2. Data Acquisition and Preparation

• Dataset Collection: Gather a comprehensive dataset consisting of images of various skin

types and acne conditions. Public datasets were utilized, and additional data was collected
to ensure diversity.
• Data Annotation: Label the images with skin type categories (e.g., oily, dry, combination)
and acne severity levels. This labeling is crucial for supervised learning.
• Data Augmentation: Apply techniques such as rotation, scaling, and color adjustment to
artificially increase the dataset size and variability, enhancing model robustness.

3. Model Development

• Preprocessing: Implement preprocessing steps such as image resizing, normalization, and

skin area segmentation to prepare the images for model training.

15
 • Transfer Learning: Leverage pretrained models (e.g., Caffe models like VGG or ResNet)
that are already trained on large image datasets. Fine-tune these models on the specific skin
type and acne datasets to adapt them to the problem at hand.
• Custom Layers: Add custom layers to the pretrained models to tailor them for the specific
tasks of skin type classification and acne detection.
• Model Training: Train the models using the prepared dataset, employing techniques like
early stopping and learning rate adjustment to optimize performance. The training process
was monitored to ensure convergence and avoid overfitting.
• Evaluation: Evaluate the model's performance using metrics like accuracy, precision, recall,
and F1-score. Cross-validation techniques were used to ensure the model's generalizability.

4. Real-Time Processing Optimization

• Model Optimization for Mobile Devices: The trained models were optimized for mobile
deployment, ensuring they can run efficiently on devices with limited computational
resources.
• On-Device Inference: Implemented the model to perform inference directly on the mobile
device, ensuring low latency and real-time feedback to the user.
• Pipeline Development: Developed a streamlined processing pipeline that captures images,
processes them through the model, and displays results in real-time.

5. User Interface and Experience Design

• User-Centric Design: Designed the user interface to be intuitive and easy to navigate,
focusing on delivering clear and actionable insights from the analysis.
• Real-Time Feedback: Integrated real-time feedback mechanisms that provide users with
immediate results and skincare recommendations based on their analysis.
• Personalization: Incorporated features that allow users to track their skin health over
time, providing personalized advice and insights based on historical data.

16
6. Testing and Validation

• Performance Testing: Conducted thorough testing on various devices to ensure consistent

performance and real-time processing across different hardware configurations.
• User Testing: Gathered feedback from users to assess the system’s usability, accuracy, and
overall experience, using these insights to refine the system further.
• Model Validation: Validated the model's accuracy and reliability in real-world scenarios
to ensure effective skin type classification and acne detection.

7. Deployment and Monitoring

• System Deployment: Deployed the system in a testing environment, ensuring smooth

operation across different devices and platforms.
• Monitoring and Evaluation: Implemented monitoring tools to track system performance,
data processing efficiency, and model accuracy over time.
• Feedback Mechanism: Established a structured feedback system where users and experts
can report issues, suggest improvements, and contribute to ongoing system enhancements.

8. Continuous Improvement

• Model Enhancements: Regularly update the model with new datasets to improve accuracy
and adapt to evolving skincare patterns.
• Feature Expansion: Plan for the addition of new functionalities, such as detecting other
skin conditions or integrating more advanced analytical tools.
• User Engagement: Continuously interact with users and domain experts to gather insights
and ensure the system evolves based on real-world needs.

17
A step-by-step approach to solve the problem of developing the "FaceMap: AI-Powered Skin Type
and Acne Analysis in Real-Time" project:

Step 1: Define Objectives and Gather Requirements

1.1 Define the Problem Statement: Develop an AI-driven system for real-time skin type and acne
analysis using facial image processing. It will detect age and gender, predict acne type, and provide
personalized recommendations for skincare, diet, and home remedies. A report will be generated
based on the analysis.
1.2 Identify Stakeholders: Developers – Build and implement AI models. Dermatologists –
Provide expert knowledge on acne and treatments. Users – Individuals seeking personalized
skincare insights.
1.3 Gather Requirements: The system requires fast and accurate acne detection, personalized
skincare recommendations, and easy-to-understand reports for users. It must run on a device with
a high-quality front camera, 2 GB RAM, multi-core processor, and 500 MB storage. The software
stack includes Python,OpenCV,TensorFlow/Keras, Caffe models, and SQLite/Firebase. The goal
is to develop an AI-powered solution for precise acne classification, tailored suggestions, and
structured report generation.

Step 2: Data Collection and Preparation

2.1 Acquire Datasets: Collect datasets with images of various skin types, acne conditions, and
Ayurvedic skin types (Vattaj, Pittaj, Kaphaj). If not available, collaborate with Ayurvedic
practitioners to label the skin types.
2.2 Data Annotation: Label images by skin types (oily, dry, combination), acne severity, and
Ayurvedic classifications (Vattaj, Pittaj, Kaphaj, mixed types).
2.3 Data Augmentation: Apply techniques like rotation, scaling, and brightness adjustments to
diversify the dataset, ensuring Ayurvedic characteristics are preserved.
2.4 Data Preprocessing: Standardize image sizes, normalize pixel values, segment skin areas, and
integrate a custom classification step to identify Ayurvedic skin types for model integration.

18
Step 3: Model Development

3.1 Model Selection: Choose a pretrained model like VGG or ResNet for image classification.
Consider lightweight models like MobileNet for real-time performance.
3.2 Transfer Learning: Fine-tune the pretrained model for skin type classification (Western and
Ayurvedic), acne detection, and age/gender prediction by adding custom layers specific to
Ayurvedic classifications.
3.3 Model Training: Train the model on the annotated dataset with techniques like early stopping,
learning rate tuning, and data regularization to ensure accuracy and prevent overfitting, optimizing
it for real-time mobile performance.
3.4 Model Evaluation: Evaluate the model using metrics like accuracy, precision, recall, and F1-
score. Use cross-validation to ensure generalizability across different datasets.

Step 4: Real-Time Processing and Analysis Development

4.1 Design User Interface (UI): Design a user-friendly interface for the web or desktop platform
that allows easy navigation, image capture, and viewing of analysis results.
4.2 Integrate the Model into the App: E mbed the trained model into the platform using
appropriate frameworks for real-time image processing, such as TensorFlow or PyTorch for
desktop applications.
4.3 Real-Time Processing Pipeline: Develop a real-time processing pipeline that captures images
using JavaScript and sends them to the Flask API for immediate analysis. The processed results
(e.g., acne severity, skin type) will be returned and displayed on the web interface almost instantly.
4.4 Recommendation Engine: Develop a dual recommendation engine: one for image-based
results and one for symptom-based results. Store user data and preferences in the SQL database
and return tailored recommendations (e.g., breakfast, lunch, detox drinks, skincare tips, home
remedies) based on both types of analysis.

Step 5: Testing and Validation

5.1 Functionality Testing: Test the platform’s real-time image capture, analysis, and
recommendation display across various devices and browsers to ensure smooth functionality.

19
5.2 Performance Testing: Focus on the real-time speed and accuracy of the image-based analysis,
checking the API response times and ensuring the image processing happens quickly and
efficiently.
5.3 User Acceptance Testing (UAT): Conduct testing with real users to gather feedback on the
real-time image-based analysis, recommendation relevance, and overall user experience.
5.4 Model Validation: Validate the model’s performance for real-time, image-based analysis,
ensuring it maintains accuracy and provides valuable insights across various skin types and
conditions.

Step 6: Deployment

6.1 Platform-Specific Deployment: Deploy the web platform with real-time image capture and
processing on a hosting service (e.g., Heroku, AWS, DigitalOcean), ensuring both the frontend
and Flask API are optimized for performance.
6.2 Documentation and User Guide: Provide clear instructions on how to use the platform,
including how to capture images for analysis and interpret the personalized recommendations
based on real-time results.
6.3 Marketing and Launch: Develop a strategy for promoting the real-time, image-based web
platform, leveraging social media, skincare influencers, and relevant blogs to showcase the
uniqueness of your platform. Highlight features such as instant image analysis and
personalized skincare suggestions based on real-time image processing, aiming to reach a broad
audience.

20
Step 7: Post-Deployment Support and Continuous Improvement

7.1 Monitoring and Maintenance: Monitor the platform's real-time performance, including usage
patterns, accuracy of image analysis, and user engagement. Track any technical issues and ensure
the Flask API and real-time processing are optimized for smooth performance across devices.
7.2 User Feedback and Updates: Gather feedback from users on the platform’s usability, the
accuracy of image-based analysis, and the relevance of suggestions. Based on this feedback,
implement updates to enhance the platform’s real-time processing and improve the user experience.
7.3 Model Retraining: Regularly retrain the model with new data to refine the accuracy of the
real-time image-based analysis. This will involve gathering updated images and data from real
users and integrating advancements in skincare science or image processing techniques.
7.4 Feature Expansion: Plan for future feature expansions, such as detecting other skin conditions
or offering advanced skincare analytics. Also enhancing suggestions with more personalized
insights.

Step 8: Documentation and Reporting

8.1 Technical Documentation: Create detailed documentation for the model architecture, training
process, web platform development (frontend and backend), deployment steps, and database usage
for future reference.
8.2 Reporting to Stakeholders: Provide regular updates to stakeholders on the project’s successes,
challenges, and next steps, ensuring clear communication on progress and upcoming milestones.

This step-by-step approach ensures that the project is executed methodically, with a focus on
achieving the desired functionality, accuracy, and user satisfaction.

The following algorithms and techniques are used in the "FaceMap: AI-Powered Skin Type
and Acne Analysis in Real-Time" project:

1. Image Preprocessing:
o Image Resizing: Bilinear or bicubic interpolation to standardize image sizes.
o Normalization: Scaling pixel values for improved model performance.

21
 o Skin Area Segmentation: Using methods like color thresholding, GrabCut, or U-
Net for isolating skin regions.
2. Convolutional Neural Networks (CNNs):
o Pretrained Models: Utilizing models like VGGNet, ResNet, or Inception for
feature extraction and classification.
o Transfer Learning: Fine-tuning pretrained models on skin type and acne datasets.
o Advanced CNNs: EfficientNet or MobileNetV2 for optimized performance on
mobile devices.
3. Acne Detection:
o Object Detection Algorithms: Using YOLO or Faster R-CNN for detecting and
localizing acne lesions.
o Custom Classification Networks: CNNs tailored to classify acne severity based
on detected features.

These algorithms collectively enable accurate real-time analysis of skin type and acne conditions
on the web platform.

22
 6. EXPERIMENTAL SETUP
6.1 Details of Database
The database in the FaceMap project is used to store and manage various types of data, including
user profiles, facial images, skin analysis results, and Ayurvedic treatment recommendations. It
includes structured data (like user details and analysis results) and unstructured data (like facial
images and dietary reports). The system leverages relational databases for structured data and
NoSQL or cloud storage for handling large amounts of image and log data. Key functions of the
database include data retrieval for real-time analysis, logging user interactions, and providing
personalized treatment recommendations. Security measures like encryption and access control are
in place to protect sensitive information.

The FaceMap project’s database manages essential data such as user profiles, facial images, skin
analysis results, and Ayurvedic treatment recommendations. It combines relational databases (for
structured data like user info and analysis results) with NoSQL databases and cloud storage (for
unstructured data like images and dietary reports). The database supports various functions,
including retrieving user data, performing real-time skin and acne analysis, and offering
personalized treatment recommendations through a report summary . It also logs interactions for
system improvements. Strong security measures, such as encryption and access control, ensure the
protection of sensitive user data, adhering to privacy regulations like GDPR. The system is scalable
to handle growing user data and analysis needs.

6.2 Hardware & Software requirements

Components Used--
1. Smartphone/Tablet with Camera
2. Device with Minimum 2 GB RAM
3. Internet Connectivity
4. Processing Unit
5. Storage
6. Database
7. Programming language
8. Framework

23
Components Description--
1. Smartphone/Tablet with Camera:
• A high-quality front camera is crucial for capturing detailed images of the skin. The
camera should have a minimum resolution of 12 MP to ensure clarity and precision in
analysis. Features such as autofocus, HDR, and good low-light performance will
enhance image quality, allowing for accurate assessments of skin texture and condition.

2. Device with Minimum 2 GB RAM:

• A minimum of 2 GB RAM is essential to ensure optimal performance of the FaceMap
platform. Adequate RAM allows the device to efficiently handle the processing of real-
time image analysis and AI model inference. It enables smooth multitasking, quick data
processing, and ensures that the system can run complex algorithms without delays.
This results in minimal lag, faster response times, and an overall seamless user
experience, even when handling high-resolution images and real-time recommendations.

3. Internet Connectivity:
• Internet connectivity is necessary for uploading images to the web platform for real-
time analysis, retrieving AI-generated recommendations, and syncing user data across
devices. A stable internet connection ensures that image data is quickly sent to the Flask
API for processing, and real-time suggestions can be returned without significant delay.
This is crucial for ensuring that the FaceMap platform works efficiently and accurately
in generating personalized recommendations.

4. Processing Unit:
• he processing unit, such as a multi-core processor (e.g., Qualcomm Snapdragon or
Apple A-series), is responsible for executing the calculations needed to run AI
algorithms and process images efficiently. A powerful processing unit accelerates real-
time image processing and AI inference tasks. It is essential for reducing the time it
takes to analyze images and generate recommendations, contributing to a smooth and
responsive user experience.

24
5. Storage:
• The device should have at least 500 MB of free storage to save images, logs, temporary
files, and user data without affecting the platform’s performance. Sufficient storage is
needed to ensure that images can be captured, saved, and processed without running
into memory issues. It also allows for the storage of user preferences, skin health history,
and other personalized data that enhances the accuracy of recommendations.

6. Database:
• SQLite: A lightweight, serverless database that can be embedded directly into the
application. It allows for easy storage and retrieval of user data, skin health history, and
preferences without needing an internet connection. SQLite is perfect for mobile
applications due to its simplicity and efficiency. It is used to store user data, skin health
history, and preferences. This data is essential for tracking progress over time and
personalizing recommendations The database helps maintain persistent user profiles,
which are important for offering tailored skincare advice. It also stores data from real-
time image analysis, allowing users to access past results and follow up on suggestions.

7. Programming Language:
• The platform uses HTML, CSS, and JavaScript for the frontend development, enabling
an interactive user interface. Python is used on the backend, specifically with the Flask
framework, to manage the API, handle image processing, and provide AI-driven
recommendations. Frontend languages ensure a smooth, responsive user interface.
Python is essential for implementing the backend processes, such as image analysis,
interfacing with the trained AI models, and serving recommendations. JavaScript
enables dynamic interactions, such as real-time image capture and user feedback on the
web interface.

25
8. Framework:
• Flask is used as the backend framework for creating the API that handles image
processing, manages data, and communicates with the frontend.Flask provides a
lightweight, flexible backend that supports efficient communication between the
frontend and AI model. TensorFlow.js or TensorFlow Lite allows the FaceMap
platform to run machine learning models in the browser, enabling real-time processing
without requiring users to rely on external servers for analysis.
• Pretrained Caffe Models: Caffe is a deep learning framework that allows for training
and deploying models for image classification and analysis. Using pretrained models
can significantly speed up the development process, as you can leverage existing
models fine-tuned for similar tasks, rather than starting from scratch. This will enable
efficient skin type and acne analysis within your app.
• Additional Pretrained Caffe Madels:
− age_deploy.prototxt
− age_net.caffemodel
− gender_deploy.prototxt
− gender_net.caffemodel
− opencv_face_detector.pbtxt
− openvc_face_detector_unit8.pb

26
 7. RESULTS

Figure 7.1.1 Female gender detection

Figure 7.1.2 Male gender detection

27
 Figure 7.1.3 Detection of Skin Type

Figure 7.1.4(A) Acne Type Prediction Application

28
Figure 7.1.4(B) Sample 1

Figure 7.1.4(C) Sample 2

29
Figure 7.1.4(D) Sample 3

Figure 7.1.4(E) Sample 4

30
 Figure 7.1.4(F) Sample 5

Figure 7.1.5 Images of acne type

31
Figure 7.1.6 Images in Trial folder

32
33
34
Figure 7.1.7(A) Symptoms by Selecting Acne type

35
 .
Figure 7.1.8 Images Matching to Acne type

36
 8. IMPLEMENTATION PLAN FOR SEM 7&8

8.1 TIMELINE CHART FOR SEM 7

Months July August September October October

Topics week

Topic discussion 21-07-24

10 days
Research paper 2-08-24
based on that 12 days
topic
Project title 12-08-24
proposal 10 days

Literature survey 26-08-24

15 days

Requirement and 6-09-24

gathering of data 18 days

Stage 1 13-09-24
presentation 8 days
40 % 16-09-24
Implementation 4 days

Report 7-10-24
8 days
Figure 8.1.1 Timeline chart for Semester 7

37
8.2 TIMELINE CHART FOR SEM 8

Months December January February March

Topics November
Stage 2
presentation

Extra Research
papers

Draft
Report

60%
Implementation

75%
Implementation

100%
Implementation

Stage 1
presentation
Paper Publish

Stage 2
presentation
Final report &
presentation
Figure 8.2.1 Timeline chart for Semester 8

38
 9. APPLICATIONS

FaceMap is a revolutionary AI-driven solution that combines modern dermatology with Ayurvedic
principles to offer real-time skin type classification and acne analysis. The system has a broad
range of applications, making skincare more accessible, personalized, and data-driven.

1. Personalized Skincare and Treatment Recommendations

• Analyzes skin conditions based on real-time images and symptoms.
• Provides customized skincare routines, including product suggestions, diet plans, detox
drinks, and home remedies.
• Offers lifestyle recommendations to improve skin health and prevent acne breakouts.

2. Dermatological Assistance and Early Diagnosis

• Acts as an initial screening tool for users to assess acne severity before consulting a
dermatologist.
• Helps track acne progression over time, allowing users to monitor changes and
effectiveness of treatments.
• Provides data-driven insights that can assist dermatologists in remote consultations.

3. Real-Time and Image-Based Skin Analysis

• Utilizes smartphone or tablet cameras to analyze skin conditions instantly.
• Allows users to either capture live images or upload existing ones for analysis.
• Delivers immediate feedback, helping users make quick and informed skincare decisions.

4. Ayurvedic Skin Type Classification and Holistic Approach

• Integrates Ayurvedic skin classifications (Vattaj, Pittaj, Kaphaj) along with standard
dermatological categories (oily, dry, combination).
• Suggests personalized skincare routines based on Ayurvedic principles for a holistic
approach to skin health. Offers a unique blend of AI-based modern analysis and traditional
wellness techniques.

39
5. Continuous Skin Health Monitoring and Progress Tracking
• Maintains a record of users’ skin conditions over time, enabling better tracking of skincare
progress.
• Identifies patterns in acne breakouts and suggests preventive measures.
• Helps users stay consistent with their skincare routines by providing reminders and updates
based on historical data.

6. Research and Development in Skincare

• Can be used by researchers and dermatologists to study skin conditions across different
demographics.
• Provides valuable insights for cosmetic and pharmaceutical companies to develop better
skincare products.
• Helps in understanding the effectiveness of various treatments based on AI-driven analysis.

By leveraging AI, real-time image processing, and dermatological knowledge, FaceMap

empowers users to take control of their skin health with scientifically backed insights and holistic
skincare recommendations.

40
 10. CONCLUSION
FaceMap is an innovative and cutting-edge solution for real-time skin type and acne analysis using
AI technology. By utilizing the high-quality front cameras of smartphones and tablets, combined
with pretrained models like Caffe for image recognition, FaceMap delivers personalized and
precise skincare insights directly to users' devices. This platform ensures efficient on-device
processing with minimal hardware requirements, allowing for seamless operation even on devices
with lower specifications. User data and skincare history are stored securely in an SQL database
like SQLite, ensuring data privacy and easy access for future analysis. Designed with the user
experience at the core, FaceMap empowers individuals to take control of their skin health. It
provides highly accurate, real-time feedback based on both image-based and symptoms-based
analysis. The platform allows users to make well-informed decisions about their skincare routines,
offering personalized recommendations for breakfast, lunch, dinner, detox drinks, skincare
products, and home remedies, tailored to their specific skin conditions. By integrating AI-driven
analysis and real-time feedback, FaceMap bridges the gap between technology and personal
wellness. It offers a fast, accurate, and accessible skin health assessment that is available at users'
fingertips, making it convenient for anyone to improve their skincare routines and manage acne
effectively. The FaceMap: AI-Powered Skin Type and Acne Analysis in Real-Time system
represents a groundbreaking approach to personal skincare management and acne treatment. By
continuously monitoring skin type, acne severity, and environmental factors, it provides dynamic,
personalized skincare recommendations that adapt over time. This system fills a crucial gap in the
skincare industry by providing inclusive, accessible, and accurate skin health analysis for users of
all skin types and tones, which existing solutions often overlook.With diverse applications ranging
from creating personalized skincare routines to offering virtual dermatology consultations,
FaceMap enables users to take proactive control of their skin health. This reduces dependency on
expensive or ineffective treatments, and ensures users have access to scientifically-backed skincare
advice. Through this innovative approach, FaceMap has the potential to revolutionize skincare
management, offering both convenience and precision to millions of users worldwide, ultimately
enhancing their overall well-being and confidence.

41
 REFERENCES

1) [“Skin Type Classification Using Deep Learning.” Journal of Dermatological Research,

45(3), pp. 123-130. 1.Wang, Y., et al. (2018)]
2) [“Automatic Acne Object Detection and Acne Severity Grading Using Smartphone Images
and Artificial Intelligence”] Quan Thanh Huynh, Phuc Hoang Nguyen
3) [“Chen, L., et al. (2020). “Texture Analysis in Skin Type Classification.” International
Conference on Image Processing, pp. 412-419.]
4) [Groh, M., et al. (2021). “Fairness in AI: Addressing Bias in Skin Type Analysis.” IEEE
Transactions on Medical Imaging, 40(12), pp. 3230-3241.]
5) [Kulkarni, M.A., Joshi, M.P. (2022)"Detection of Gender and Age Using Machine
Learning”]

6) https://en.wikipedia.org/wiki/Facial_recognition_system
7) https://ieeexplore.ieee.org/abstract/document/9299633
8) https://www.journal-fea.com/article_148293.html
9) https://medicalfuturist.com/digital-skin-care-top-8-dermatology-apps/
10) https://ieeexplore.ieee.org/abstract/document/10613550
11) https://academic.oup.com/bjd/article-abstract/190/6/789/7603706?login=false
12) https://www.ajmc.com/view/ai-dermatology-mobile-apps-have-critical-efficacy-safety-
gaps-review-says
13) https://academic.oup.com/bjd/article-abstract/190/6/789/7603706?login=false
14) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9693628/
15) https://www.nature.com/articles/s41746-024-01033-8?utm_source=chatgpt.com
16) https://ai-derm.com/?utm_source=chatgpt.com
 ACKNOWLEDGEMENT

We hereby declare that the work presented in this Major Project report entitled " FaceMap: AI-
Powered Skin Type and Acne Analysis in Real-Time" Is in partial fulfillment for the Degree of
"Bachelor of Engineering" in Computer Engineering.

Numerous individuals have contributed significantly to the success of this project, and we extend
our heartfelt gratitude to each of them. First and foremost, we would like to express our sincere
appreciation to our Principal, Dr. Pramod Rodge, Head of Department, Dr. Uttara Gogate, and
our project coordinator, Prof. Reena Deshmukh, for their invaluable guidance, assistance, and
unwavering support throughout the challenging stages of the project's development.

In addition, we would like to extend special thanks to Prof. Diksha Bhave, who generously
dedicated her valuable time from her busy schedule to serve as our project guide. Her guidance was
instrumental in steering our project in the right direction.Without their willingness, accommodating
spirit, candid feedback, timely clarifications, and, most importantly, their faith in us, this project
would not have been completed within the stipulated timeframe.