Class: CMU-CS 447 AIS

Team Member:
1. Nguyễn Thành Danh - 2221125770
2. Bùi Đức Dương - 24211216092
3. Từ Mỹ Ngân – 2320117977
4.

Ứng dụng: FaceMe

I. CONCEIVE
1. Project overview
a. Project Definition

● The ability to recognize and imitate facial expressions of emotion is crucial

to establishing interpersonal connections early in life. Recognition of facial
expressions is one of the primary signals used to understand the feelings
and intentions of others, and it has been argued that the ability to recognize
basic emotional expressions may be universal. Yet many researches have
shown that some groups of autism patients display difficulties in such tasks,
which usually led to their detachment from social life.
● The answer to this problem lies in the question itself. People with autism
need a tool that can work as a translator hub between them and the non-
verbal signs of the other party of a conversation. And while those signs
include many such as body language, environment context, etc. facial
expressions are reasonably the most important factor.
● That’s why in this project we will develop a mobile app that can detect the
emotional change of a person in real-time and give subtle alerts to the user,
assuming someone with autism, so that they can try to navigate the
conversation to a positive end.
b. Business Need (phạm vi của dự án)

● Detect change in a person’s emotion through their facial expression.

● Send an alert to the user, using the system’s capacity to notify the
flow of the conversation.
2. Prior Art
In this section we present the SWOT analysis of our own application and some
other available ones on the market.
FaceMe
FaceMe is a mobile application developed as a tool specifically designed to
assist people with trouble recognizing facial expressions in daily life
conversations.
Strengths Weaknesses
Use CNN model to detect emotional Can only help recognizing emotional
expression. expression.
Useful and subtle design. Developer team is inexperienced.
Low budget.
Opportunities Threats
There’s not much competition. Might be treated as a breach of privacy
of other people.

AffdexMe
AffdexMe demonstrates the use of Affectiva's Affdex Software Developer Kit
(SDK), which lets developers emotionally-enable their apps and digital
experiences.

The app analyzes and responds to facial expressions of emotion in real time
using the built-in camera on your Android/iOS device. To use, simply start the
app and you will see your own face on the screen with metrics at the top
measuring expressions and emotions and the most likely emoji icon that
matches your facial expression.

Strengths Weaknesses

The app analyzes and responds to facial Limited emotional awareness

expressions of emotion in real time using Most of the time emotions are wrongly
the built-in camera on your Android/IOS classified to Joy. If teeth are visible the
device. emotion is detected as Joy irrespective of
actual real joy.
Seemingly still in development.
Accuracy is not high.

Opportunities Threats
The first pioneer app for free. Didn't really get the point.
Uses new technology. Crashed many times.
Has many negative feedbacks.

3. System Context (Biểu đồ ngữ cảnh của ứng dụng)

 System Context Description
Camera:
Capturing real-time video of the other party in the conversation.
FaceMe:
Receive and transfer data to the CNN model then send alerts to the user
using system’s capacity such as ringtone to notify the changes.
CNN model:
Analysis and detect the emotion from the person’s emotional expression.
Send the result to the system.
User:
Receive notification from their device.
● Cost Estimate

Cost Person/Hour
Resource Name Type Initials Max. Units Std. Rate Ovt. Rate

Nguyen Thanh Danh Work NTD 100% $1.00/hr $2.00/hr

Tu My Ngan Work TMN 100% $1.00/hr $2.00/hr
Bui Duc Duong Work DP 100% $1.00/hr $2.00/hr
 Dang Van Phuoc Work DVP 100% $1.00/hr $2.00/hr

Total cost estimate

N Criteria Pric Amoun Total
o e t (USD)

1 Working $1 421 $421

hours

2 Other cost $100 1 $100

Total cost $521

Description Amoun Unit

t

Number of members 4 Perso

n

Number of working hours per day 4 Hour

Number of workdays/weeks 5 Day

The duration of the project 120 Day

The number of working hours 421 Hour

Other costs (Internet, foods, drinks, meetings, 100 USD

fuel…)
● Master Plan
Ví dụ:

N Task Name Duratio Start Finish

O n
26/08/202 06/09/202
1 Start-Up 15 days
1 1
Project’s Kick-off 26/08/202 27/08/202
1.1 2 days
Meeting 1 1
28/15/202 06/09/202
1.2 Create Document 13 days
1 1
 07/09/202 22/10/202
2 Development 45 days
1 1
07/09/202 21/09/202
2.1 Sprint 1 15 days
1 1

22/09/202 11/10/202
2.2 Sprint 2 20 days
1 1

12/05/202 22/05/202
2.3 Sprint 3 10 days
1 1

23/05/202 23/05/202
3 Final release 1 day
1 1

4. Business plan (Kế hoạch kinh doanh)

The application brings a new beginning for people with autism, helping autistic
people to recognize the emotions of the opposite person through the application
developed by our team. Giving a little help to someone with autism can be more
fun and especially social. Because this is a project to help people with autism, our
team decided to do it for a non-profit purpose with the hope that those who are
suffering from the disease can have fun and mingle with everyone and especially it
will be back to normal soon

5. Công nghệ sử dụng

Main Programming Language: Python

The system uses a CNN (Convolutional Neural Network) with TensorFlow to

detect human emotions from facial expressions with great accuracy, then
integrates that model with OpenCV to process real-time video.

The application is developed completely in python, then compile to android

friendly app using BeeWare and VOC, which is a transpiler - it takes Python
source code, compiles it to CPython Bytecode, and then transpiles that bytecode
into Java-compatible bytecode. The end result is that your Python source code
files are compiled directly to a Java .class file, which can be packaged into an
Android application.

PROJECT MANAGEMENT
Trello, GitHub
TOOLS
DOCUMENT TOOLS Google Docs

UML TOOLS Astah Professional 8.2.0, [Link]

Visual Studio Code 1.44.2, Android Studio

IDES
2.3.2, BeeWare

DEPLOYMENT SERVER AWS S3, Heroku, MongoDB Atlas

DBMS MongoDB Compass 1.20.5

DATABASE MongoDB 4.2.0

SOURCE CODE VERSION

Git 2.26 (Hosted on GitHub)
CONTROL

TESTING TOOL Postman 5.5.4

COMMUNICATION TOOLS Zoom, Facebook Messenger, Google Hangout

6. Quality attribute

ID Type Description Solution

QA1 Performance A conversation is usually fast and Applying CNN
continuous, requiring the system to model.
return detection results in the latest
of 1 second to avoid bad timing and
misunderstanding.
QA2 Availability The system needs to run smoothly
during a short or long period of time
depending on the length of the
conversation with equal precision.
In case of crashing or memory
overloading, reboot the system.

7. Other (importance)
Problem definition: The use of an off-system camera with an unnoticeable
camera serves the purpose of making the conversation seem normal. But in
some cases it can be treated as a breach of privacy to the other party.

The solution to this is decided that the application will only work on real-time
video records and save no aftermath data.
8. References

1. Imitation and recognition of facial emotions in autism: a computer vision

approach
[Link]
00430-0

2. Developing Android app using Python

[Link]
completely-using-python

3. The Development of Emotion Recognition in Individuals with Autism

[Link]

4. Detect Emotions with Convolutional Neural Networks

[Link]

5. Detect Emotions in Real-Time with OpenCV

[Link]