Smart Food Sorting Using Deep Learning

Gasbaoui Mohammed el Amin1, Benkrama Soumia2, Bendjima Mostefa3

Laboratory of TIT
1,2,3
Mathematics and Computer Science Department
Faculty of Exact Sciences
Tahri Mohammed University, Bechar, Algeria
1
[email protected]
2
[email protected]
3
[email protected]

Abstract—Food quality detection refers to the process of allows for the evaluation of fruit freshness without the need to
2024 3rd International Conference on Advanced Electrical Engineering (ICAEE) | 979-8-3503-4935-1/24/$31.00 ©2024 IEEE | DOI: 10.1109/ICAEE61760.2024.10783364

evaluating and assessing the characteristics, properties, and cut, peel, or otherwise physically alter the fruit. Traditional
safety of food products to determine their overall quality. It methods of checking fruit freshness, such as cutting open the
ensures that the food is free from harmful contaminants, such as fruit to examine its color, texture, or aroma, can lead to waste
bacteria and viruses. Traditional methods involve various testing
and reduce the shelf life of the fruit. Non-destructive
techniques to identify potential issues related to safety and
nutritional value. However, these methods often require manual techniques, on the other hand, enable fruit producers,
labor, resulting in extended detection times and laborious distributors, and retailers to assess the quality of the fruit
processes. Food fruit sorting techniques, on the other hand, are without compromising its integrity, allowing them to make
methods used to categorize and separate fruits based on informed decisions about its handling, storage, and
attributes such as size, color, shape, ripeness, and quality. In this distribution. Among these techniques, we find imaging, which
article, we present an embedded system remotely connected to a is based on cameras, and computer vision algorithms that can
server-trained model for fruit image classification using the analyze visual characteristics like color, shape, and surface
Convolutional Neural Network (CNN) deep learning algorithm. features to determine freshness. This article introduces an
The accuracy of the model reached 94% in the validation set.
embedded system that is connected remotely to a server-
The embedded system is based on a Raspberry Pi
microcontroller equipped with a camera, enabling real-time trained model for fruit image classification. The rest of the
sorting of fruits into six categories: fresh orange, fresh banana, paper is structured as follows: The related work in this axis of
fresh apple, rotten banana, rotten apple, and rotten orange. research is presented in the second section. Third we propose
Additionally, it incorporates a weight sensor to determine the a system that is divided into two main parts, one for creating a
total weight of each category.
model to classify the food fruit using CNN, the second for
proposing a smart embedded system to sort these foods, we
Keywords—food sorting, convolution neural network, embedded also propose the algorithms for the communication scenario
system, Raspberry Pi, quality detection and controlling the servo motors for triggering the actions by
the microcontroller, the fourth section is dedicated to the
I. INTRODUCTION discussion and results, In the final section, we present the
According to the World Food Summit’s definition, the conclusions drawn from our work, along with discussing
objective of food safety is to ensure that every individual has potential perspectives and aspirations for future endeavors.
access to safe, nutritious, and sufficient food that meets their
dietary requirements and preferences, promoting a healthy and II. RELATED WORKS
active lifestyle [1]. The key factor influencing public health Machine learning has the potential to harness vast and
and social security is food safety [2]. Around the world, an emerging data sets effectively, leading to enhanced food
estimated 600 million almost 1 in 10 people fall ill after eating supply safety and effective mitigation of food safety incidents
contaminated food each year, resulting in 420 000 deaths and [5]. Machine learning powered computer vision systems can
the loss of 33 million healthy life years [3]. The presence of accurately inspect and grade food products, ensuring that only
hazards in the food or feed supply chain can pose risks to the high-quality items reach consumers. This helps reduce the
safety of food products, potentially leading to adverse impacts chances of contaminated or substandard products being sold.
on human and animal health. It is crucial to monitor and The authors in [6] proposed an automatic mango grading
identify potential food safety hazards throughout the entire technique that categorizes mangoes based on their quality
supply chain to ensure the effective functioning of food safety using key fruit features, including size, shape, color, and
management systems [4]. A non-destructive way to detect surface pixel characteristics. To achieve this, they employed
fruit freshness refers to a method or technique of assessing the the Naive Bayes and Support Vector Regression (SVR)
freshness or ripeness of fruits without causing any damage or algorithms, which enable efficient sorting of mangoes
altering the fruit’s quality. In other words, it is a method that according to their respective quality levels. The same thing in

979-8-3503-4935-1/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: JSS Academy of Technical Education-Bangalore. Downloaded on August 19,2025 at 11:10:26 UTC from IEEE Xplore. Restrictions apply.
 [7], the authors investigated the correlation between the RGB 2. Propose a smart embedded system to sort the fruit
values of orange fruits and their sweetness. The findings food.
revealed that among the red, green, and blue colors, the red
color significantly influenced the prediction of sweetness in A. Creating a Model to Classify Food Fruit Using CNN
orange fruits. Employing the logistic regression model Model
algorithm resulted in the highest level of accuracy, achieving We utilized a dataset from Kaggle [11] to train a model
an impressive 97% accuracy rate in predicting the sweetness capable of classifying fruit freshness. The dataset contains six
of the fruits. A parallel SVM (Support Vector Machine) model classes as follows:
was employed in [8] to develop a risk assessment model for  fresh apples
dairy production. The researcher incorporated data from
 fresh banana
various sources, including analytical data concerning dairy
products, such as the concentration levels of relevant factors  fresh oranges
like protein and sodium. The model reached an accuracy of  rotten apples
93%. The study in [9] presents a novel early warning model  rotten banana
for vegetable diseases utilizing a decision tree approach. This  rotten apples
intelligent model effectively assesses temperature, humidity,
The train folder has 10901 images and 2698 for the test step,
and soil conditions within the greenhouse to provide timely
figure 1 shows some sample dataset images. The proposed
early warning results for potential disease outbreaks. In [10], a
CNN architecture has nine layers: three for convolution and
comparison of three machine learning algorithms, SVM,
six for the fully connected layer. Table 1 shows the details of
Naive Bayes, and Random Forest (RF) was employed to
the architecture.
differentiate between pure beef and mixed beef by analyzing
beef features, including temperature and humidity, using
electronic nose data. Among the three methods, the Random
Forest approach yielded the highest accuracy in distinguishing
between the two types of beef samples. In [2], a novel and
non-destructive approach is introduced to detect fruit
freshness effectively. The proposed method utilizes the
Convolutional Neural Network (CNN) machine learning
algorithm. To enhance the performance and accuracy of the
detection, the study incorporates Residual Networks (ResNets)
and Densely Connected Convolutional Networks (DenseNets)
for better results. Most of the cited work stopped at training
the model, focusing on either prediction or classification. Our
contribution lies in proposing an embedded system that
triggers actions after result classification using the CNN Fig. 1 Some sample images from the dataset
algorithm. Our system is intended for the food industry (FI)
and manufacturing to automate sorting processes and reduce
labor. TABLE 1
THE ARCHITECTURE OF THE PROPOSED CNN
III. PROPOSED APPROACH
This paper proposes an intelligent fruit-sorting system layer Filter size Size after Activation
combining artificial intelligence and an embedded system. the Filter function
The integration of these hardware and software components input (64,64,3) - -
complements each other to enhance various aspects of life, Conv1 (3,3,32) (62,62,32) relu
such as making more precise decisions and reducing labor Max pooling (2,2,32) (31,31,32) -
efforts. The embedded system establishes a remote connection Conv2 (3,3,32) (29,29,32) relu
with a server-trained model designed for classifying fruit Max pooling (2,2,32) (14,14,32) -
images through the Convolutional Neural Networks (CNN) Conv3(flatten) - (6272,1) -
algorithm. The system is based on a Raspberry Pi FC1 - 32 units relu
microcontroller equipped with a camera that categorizes fruits FC2 - 64 units relu
into six distinct classes. To add to its capabilities, the system FC3 - 128 units relu
also integrates a weight sensor that accurately measures the FC4 - 256 units relu
collective weight of each category. We divide this part into FC5 - 256 units relu
two main subsections:
FC6 - 6 units softmax
1. Creating a model to classify food fruit using a
CNN model.

979-8-3503-4935-1/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: JSS Academy of Technical Education-Bangalore. Downloaded on August 19,2025 at 11:10:26 UTC from IEEE Xplore. Restrictions apply.
 To avoid the overfitting problem, we applied some data B. The Proposed Embedded system To Sort The Fruit Food
augmentation techniques: The fruit food passes through a conveyor channel, and the
 Rescaling the image between (0–1) by dividing each camera identifies the fruit, triggering an action in the
pixel by 255. appropriate servo motor. The motor moves accordingly,
 Random shearing images. turning left for fresh categories and right for rotten ones. The
 Random zooming images. system is equipped with a weight sensor to track the total
weight of each group. Each time a fruit is identified, the
 Horizontal flip images. weight category is updated with the latest value. All
After we trained the model, we used the Flask framework recognized fruits are collected in specific baskets according to
to deploy the model in a web application and expose the their category. If the system cannot identify an object, it is
classification service as a Restful API using a POST request. placed in an undefined category. Figure 2 illustrates the
Reaching this point, we finished the creation of the CNN proposed embedded system.
model.

Fig. 2 the proposed embedded system

2) Material And Tools

1) Rest API Communication Scenario
We used a microcontroller Raspberry Pi 4 with 4 GB of
The system sends a POST request to the remote server-
RAM, three servo motors, a weight sensor model HX711, a
trained model for fruit food classification. The image (64,64,3)
camera, and a power supply.
is transformed into a matrix with three channels (RGB). The
server receives the request and utilizes the matrix as input for
the trained model. Finally, the server sends the response back
to the system in string format. Figure 3 shows the
communication scenario.

Fig. 4 connecting components to a Raspberry Pi

3) Algorithm Initialization Part

We employed a dictionary data structure based on key
values to store the weight of each category and also initialize
Fig. 3 The communication scenario between the remote server and the the URL of the remote server and motor list.
embedded system

979-8-3503-4935-1/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: JSS Academy of Technical Education-Bangalore. Downloaded on August 19,2025 at 11:10:26 UTC from IEEE Xplore. Restrictions apply.
 ListFruit=["fresh banana","fresh orange", IV. DISCUSSION AND RESULTS
"fresh apple","rotten banana","rotten orange",
For the training model, after choosing batch sizes of 12 and
"rotten apple"]
50 epochs, we got the results as shown in figures 5 and 6.
ListMotor= ["M1","M2","M3"]
url= ’http://localhost:5000’
for i in range 6 :
weight[ListFruit[i]]=0

4) Algorithm Processing Part

The conveyor channel moves at a constant speed of v, and
there is a distance of d between each object in the channel.
The system operates in an infinite loop and captures an image
at regular intervals of t time. The following snapshot
illustrates the crucial steps for receiving a response from a
server.
While true :
For each t seconds time :
Image=captureTheImage() Fig. 5 training and validation accuracy
R,G,B=transformIntoThreeChannel(image)
Request=postRequest(url , (R ,G ,B))
Response=getResponse()
triggerAction(Response)

5) Trigger Action Servo Motor Algorithm

After receiving a response from the server, we actuate the
appropriate motor on either the left side for the fresh category
or the right side for the rotten one. We increment the weight
of the corresponding group with the latest value obtained
from the sensor. The following snapshot shows the body of
the trigger action function.

triggerAction(Response) :
Fig. 6 training and validation loss
match response:
case "fresh banana":
weight["response"] +=getDataWeightSensor() The final results of the model in the table 2.
rotateMotor(ListMotor[0], "left") TABLE 2
case "rotten banana": THE RESULTS MODEL AFTER 50 EPOCHS
weight["response"]+= getDataWeightSensor()
loss accuracy
rotateMotor(ListMotor[0], "right")
Training set 0.22 0.92
case "fresh orange":
Validation set 0.13 0.94
weight["response"]+= getDataWeightSensor()
rotateMotor(ListMotor[1], "left")
We chose a CNN architecture that yielded acceptable
case "rotten orange":
results and did not require an extensive training process. The
weight["response"]+= getDataWeightSensor()
training took approximately 35 minutes in the Kaggle lab. The
rotateMotor(ListMotor[1], "right")
Raspberry Pi is our chosen single-board computer because it
case "fresh apple":
offers built-in Wi-Fi capabilities without needing other
weight["response"]+= getDataWeightSensor() modules. The widespread Linux distribution is the Raspberry
rotateMotor(ListMotor[2], "left") Pi OS, which makes programming tasks straightforward and
case "rotten apple": accessible using Python. We deploy the trained model on
weight["response"]+= getDataWeightSensor() another computer (server) because we tend to use modular
rotateMotor(ListMotor[2], "right") programming software design techniques. By dividing the
functionality into smaller, manageable chunks, we achieve
better results, and the system becomes easier to maintain and
more extensible for future development.

979-8-3503-4935-1/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: JSS Academy of Technical Education-Bangalore. Downloaded on August 19,2025 at 11:10:26 UTC from IEEE Xplore. Restrictions apply.
 The embedded system’s microcontroller is also designed Therefore, we separated the trained model from the embedded
for low-resource computation and lacks strong computational system in this article.
power. Therefore, we separate the trained model and run it on For future development and progress, some challenges
the server to offload the computation from the embedded need to be addressed. One of them is food waste, which can
system. REST APIs are a mechanism that enables an be mitigated by proposing a strategy that starts with farm
application or service to access a resource within another monitoring. This strategy involves not only identifying
application or service. Therefore, we chose to use a Restful diseases but also utilizing robots to trigger appropriate actions,
API for communication between the embedded system and such as cutting infected parts or applying specific components
the server. We primarily utilize POST requests instead of as a cure. Additionally, for transportation and preservation,
GET requests to enhance security and avoid revealing we need to propose a modified environment designed to meet
sensitive data in the URL. For the synchronization problem the specific needs of the transported food using AI and all
between the conveyor channel and motor actions and the time available means.
taken for HTTP requests, we propose using a DC motor
(Direct Current) to control the conveyor. Each time the REFERENCES
embedded system requests data from the server, we stop the [1] Q. Zhou, H. Zhang, and S. Wang, ‘Artificial intelligence, big data, and
conveyor channel. Once the result arrives and we know the blockchain in food safety’, International Journal of Food Engineering,
vol. 18, no. 1, pp. 1–14, Jan. 2022, doi: 10.1515/ijfe-2021-0299.
conveyor’s speed and the distance between the camera and
[2] J. Han, T. Li, Y. He, and Q. Gao, ‘Using Machine Learning
the motor, we can trigger an action in time ’t’ using the Approaches for Food Quality Detection’, Mathematical Problems in
formula: t = d/v. Engineering, vol. 2022, pp. 1–9, Dec. 2022, doi:
A dictionary is a data structure that allows us to store and 10.1155/2022/6852022.
[3] ‘Food Safety’. Accessed: Jul. 23, 2023. [Online]. Available:
organize data in key-value pairs, enabling fast retrieval of
https://www.who.int/health-topics/food-safety
values based on their keys. In our system, we used a [4] X. Wang, Y. Bouzembrak, A. O. Lansink, and H. J. Fels‐Klerx,
dictionary to record the total weight for each category. To ‘Application of machine learning to the monitoring and prediction of
enhance the system and obtain the total number of fruits for food safety: A review’, Comp Rev Food Sci Food Safe, vol. 21, no. 1,
pp. 416–434, Jan. 2022, doi: 10.1111/1541-4337.12868.
each basket, we can create another dictionary variable and [5] X. Deng, S. Cao, and A. L. Horn, ‘Emerging Applications of Machine
increment the corresponding key value by one each time the Learning in Food Safety’, Annu. Rev. Food Sci. Technol., vol. 12, no. 1,
camera detects a fruit. pp. 513–538, Mar. 2021, doi: 10.1146/annurev-food-071720-024112.
[6] D. Pise and G. D. Upadhye, ‘Grading of Harvested Mangoes Quality
and Maturity Based on Machine Learning Techniques’, in 2018
V. CONCLUSION International Conference on Smart City and Emerging Technology
(ICSCET), Mumbai: IEEE, Jan. 2018, pp. 1–6. doi:
In this article, we trained a model for fruit classification 10.1109/ICSCET.2018.8537342.
quality using a CNN and proposed an embedded system based [7] M. A. J. Al-Sammarraie et al., ‘Predicting Fruit’s Sweetness Using
Artificial Intelligence—Case Study: Orange’, Applied Sciences, vol. 12,
on three servo motors for sorting. We addressed the issue of no. 16, p. 8233, Aug. 2022, doi: 10.3390/app12168233.
synchronization and HTTP request consumption time by [8] Y. Ma, Y. Hou, Y. Liu, and Y. Xue, ‘Research of food safety risk
suggesting using another DC motor directly connected to the assessment methods based on big data’, in 2016 IEEE International
conveyor channel. This motor remains inactive until a Conference on Big Data Analysis (ICBDA), Hangzhou, China: IEEE,
Mar. 2016, pp. 1–5. doi: 10.1109/ICBDA.2016.7509812.
response is received from the server via the Raspberry Pi. [9] F. Wu, N. Li, J. Zhang, and S. Guo, ‘The design about early warning of
However, there are limitations due to the microcontroller's vegetable disease in facility agriculture’, J. Phys.: Conf. Ser., vol. 1168,
resources. It does not allow us to connect a high-quality p. 032052, Feb. 2019, doi: 10.1088/1742-6596/1168/3/032052.
camera for image capture. [10] S. A. Laga and R. Sarno, ‘Temperature effect of electronic nose
sampling for classifying mixture of beef and pork’, IJEECS, vol. 19, no.
Additionally, the illumination environment can sometimes 3, p. 1626, Sep. 2020, doi: 10.11591/ijeecs.v19.i3.pp1626-1634.
affect the classification results. The modular programming [11] ‘Fruits fresh and rotten for classification’. Accessed: Mar. 09, 2024.
software design approach is necessary for the complex [Online]. Available: https://www.kaggle.com/datasets/sriramr/fruits-
systems demanded in the recent pioneering industry. Dividing fresh-and-rotten-for-classification
the main problem into smaller, manageable chunks facilitates
maintenance and helps avoid interrupting the entire service.

979-8-3503-4935-1/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: JSS Academy of Technical Education-Bangalore. Downloaded on August 19,2025 at 11:10:26 UTC from IEEE Xplore. Restrictions apply.