2020 International Conference on Computer Science and Its Application in Agriculture (ICOSICA)

Chilli Quality Classification using Deep Learning

Sudianto Yeni Herdiyeni
Dept of Computer Science Dept of Computer Science
IPB University IPB University
Bogor, Indonesia Bogor, Indonesia
[email protected] [email protected]

Anggra Haristu Medria Hardhienata

Dept of Computer Science Dept of Computer Science
IPB University IPB University
Bogor, Indonesia Bogor, Indonesia
[email protected] [email protected]

Abstract—Chilli peppers (Capsicum annuum) are potential to 316.75 thousand hectares in 2019. The projection for the
commodities in Indonesia that have relatively high economic harvested area until 2023 is expected to continue to increase
value. Sorting and grading of chilli peppers after harvesting is at an average rate of 2.66% per year [2]. This gives the chilli
very important for separating the post-harvest results to commodity in Indonesia a tremendous economic potential.
determine whether the chillies are in good or bad conditions.
Currently, these activities are still done manually using visual Maximize the existing potential, chilli production and
observation. The problem of using this manual method is that marketing must go hand in hand, especially after harvest.
the process often becomes time-consuming. Another drawback Curly red chilli is a perishable commodity, in this case post-
of utilizing manual methods is that it has often inconsistent harvest handling cannot be separated from production
results. This study aims to develop a decision support system for activities. Several factors to consider in post-harvest
classifying the quality of chillies using deep learning. The processing is the sorting of chilli. Sorting is done to separate
development of the science and technology of digital image healthy red chilli that is identified using attributes related to
processing makes it possible to sort agricultural and plantation shape, health and freshness. Delays in the sorting process
products automatically. In this research, we employ “You Only increases the opportunity for rot.
Look Once” (YOLO) method which is one technique of the deep
learning methods that is widely used for image recognition in The conditions in the field regarding the process of sorting
real-time. Simulation results from the custom model show that chilli are generally done manually and involve humans as
the classification accuracy obtained from the simple test is decision-makers. This standard process is used by various
99.4% and the challenging test is 75.6%. processing industry companies such as exporters and chilli
farmers who plant curly red chillies. The manual identification
Keywords—chilli pepper, classification, deep learning, image process has many weaknesses: including a relatively long time
processing, yolo taken; humans tend to tire and become bored when doing
monotonous activities; differences in perceptions of quality;
I. INTRODUCTION
variations in product are also obtained due to limitations with
Horticulture is a potential commodity in Indonesia that has human visual capacity; and sorting decisions are influenced
high economic value. One type of horticultural commodity by the observer's psychological condition. This can also result
that has potential and is already established is the red chilli in inconsistencies in the selection process. The manual
pepper; especially curly red chilli because it is needed by method is too time-consuming, so if applied on a large
households [1]. The chilli family of plants (Capsicum industrial scale, machine assistance is needed during the
annuum) in Indonesia consists of various types including: two process.
large chilli varieties - one with large red chillies (cabai besar),
and another with curly red chilli (cabai keriting). As well, Various technologies have contributed to the sorting
there is a cayenne variety consisting of green cayenne and red process as a decision-making about chilli quality. Among
cayenne. Of the various types, curly red chilli is the most those technologies based on image processing are detecting
commonly consumed chilli in Indonesia [1]. the quality of chillies with maturity levels using YCbCr Color
Transformation [3]; classification of chilli pepper maturity
National chilli consumption per annum for all major cities based on image HSV features with Fuzzy K-Nearest Neighbor
with a population of 1 million or more is around 800,000 tons [4]; classification chilli quality with Convolutional Neural
per year; or 66,000 tons per month. The need for chilli on Networks (CNN) [5]; bell pepper quality based on NIR-HSI
religious holidays and the festive season has increases demand [6]; and sorting on red chilli based on color and texture with
by approximately 10-20% from normal needs. The projected Artificial Neural Network (ANN) [7]. Importantly, from the
demand for chilli commodity is expected to rise until 2023 research that has been conducted, detecting the quality of
with an average annual growth of around 3.36%. While the chilli is not yet real-time because there is a delay when
chilli consumption growth is projected to increase from 1.52 processing information sourced from sensors or cameras.
thousand tons in 2018 to 1.67 thousand tons in 2023 with the
average growth per year is calculated at 2.16%. It is predicted In this study, the solution offered applies the YOLO (You
that production will begreater than consumption, thus creating Only Look Once) method, because YOLO is an object
a surplus of chilli until 2023. The surplus will increase from detection method that has better speed and accuracy than other
571.95 thousand tons in 2018 to 786.96 thousand tons in 2023 methods such as R-FCN, SSD321 [10]. Moreover, you can
[2]. On the other hand, the chilli harvest area in 2019 is easily tradeoff between speed and accuracy simply by
estimated to increase by 2.66% or 8,203 hectares. This is changing the size of the model, no retraining required. YOLO
compared to the change in 2018 of 308.56 thousand hectares is a deep learning algorithm that utilizes convolutional neural

978-1-7281-6907-1/20/$31.00
Authorized ©2020
licensed use limited to: Auckland University IEEE
of Technology. Downloaded on December 23,2020 at 16:54:42 UTC from IEEE Xplore. Restrictions apply.
 2020 International Conference on Computer Science and Its Application in Agriculture (ICOSICA)

Truth
networks (CNN) for detecting objects. YOLO’s algorithm will Conf(class) = Pr(class) × IOU Pred (1)
divide the image according to an s × s grid, then each grid
square will predict bounding boxes and class maps. If a grid is
predicted by an object, then the bounding box will surround Where ( ) is the probability of an object
Truth

the object. The confidence value will be calculated on each appearing in a region and IOU is the Intersection Over
Pred

bounding box which will then be selected based on the values Union ratio between the prediction box and the ground
obtained [8]-[10]. truth box. Pred is the area in the prediction box, Truth is
the area in the ground truth. The greater the IOU value,
Therefore, the purpose of this study is to create a decision the higher the detection accuracy level [11].
support system for classifying the quality of chillies in real-
time based on the sensing. The chillies that are detected will • Each bounding box has five information values namely
then be classified based on information from expert data in x, y, w, h, and c. The values of x and y are the coordinates
order to determine the quality of chilli classified into proper of the predicted midpoint of the bounding box, the values
and improper categories. Subsequently, from this research, it of w and h are the ratio of the width and height relative
is expected to provide a solution for chilli farmers or traders to the grid, and c is the value of the confidence bounding
who need to determine the quality of chilli before being box.
distributed to the market or buyer. • In the YOLO algorithm, each grid will predict the
This paper is described in the following order: the material probability class value for the object selected within
section and the method that explains the steps of the YOLO equation 2. During testing, YOLO will multiply the class
method; information about experts; dataset type and pre- probability value with the confidence value of the
training configuration; the results section that reviews the bounding box [8].
performance and evaluation of the system that we are At test time, the model designates each box with its
proposing; simple testing and challenging testing; the class-specific confidence score by multiplying the
conclusions and recommendations section reports the findings confidence score of each box with its respective class-
for future research.
conditional probabilities, these scores evaluate the
II. MATERIAL AND METHOD probability of the class appearing in the box, and how
precise the box coordinates are.
The material used in this study comprises chilli image data
that is adjusted with information from experts.
Pr (Classi )|Object) × Pr(Object) × IOU = Pr(Classi) × IOU (2)
Truth Truth
Pred Pred

A. The YOLO Method

The general purpose of object detection is to be fast,
B. Expert Information
accurate and able to recognize various objects. YOLO is one
of the well-established detection algorithms today, and all The expert data that we refer to is based on SNI No. 01-
processing is done by GPU [13]. In this study we use 4480-1998 [12] (Table I) with categories: (1) A Quality chilli,
YOLOv3. YOLOv3 uses several tricks to improve training whole, clean, healthy curly red chilli with normal shape,
and improve performance, including multi-scale prediction without defects. (2) B quality chilli: curly red chilli with small
and better backbone classifier [10]. Therefore, these size (length <10 cm) / abnormal shape and has damage & rot
characteristics are the main consideration for the YOLOv3 [12]. In this study the parameters we used are shape and color.
algorithm used in this study. In the illustration below, we TABLE I
present how to facilitate the creation of the system (Fig. 1). EXPERT INFORMATION ABOUT THE QUALITY OF CHILLI
Requirements
Type of Test Unit
Quality I Quality II Quality III
Training Color Uniformity % Red ≥ 95 Red ≥ 95 Red ≥ 95
Camera
(Detection object, Classification)

Uniformity of Form % Uniform 98 Uniform 96 Uniform 95

Dataset
Big Red Chilli
User Interface
Build Model

Fruit Length cm 12 - 14 9 - 11 <9

Dataset

Trained Model Central Base Line cm 1.5 - 1.7 1.3 - 1.5 < 1.3
User

(Knowledge base) Curly Red Chilli

Fruit Length cm 12 - 17 10 - 12 < 10
Central Base Line cm 1.2 - 1.5 1.0 - 1.3 < 1.0
Testing Dirt Level % 1 2 5
Dataset Interface Engine Damage & Decay Rate
Big Red Chilli % 0 1 2
Curly Red Chilli % 0 1 2
Fig 1. Architecture of the development of a model of chilli quality
classification
C. Dataset
The dataset we used in this project was a chilli image
YOLO’s algorithm for detecting an object through several dataset that we photographed with a Samsung Note
stages can be described in three steps [8]: smartphone. The dataset contained 100 photos with a
• Divide the image into regions/grids of size s × s. The comparison of 50 quality A chilli photos (Fig. 2, left-hand
grids will detect objects. Each grid will be predicted with side). and 50 photos of quality B chilli (Fig. 2, right-hand
a bounding box and a confidence value. This confidence side).
value represents how sure the bounding box contains the
object and how accurate the prediction is. The
confidence value is obtained through the following
equation:

Authorized licensed use limited to: Auckland University of Technology. Downloaded on December 23,2020 at 16:54:42 UTC from IEEE Xplore. Restrictions apply.
 2020 International Conference on Computer Science and Its Application in Agriculture (ICOSICA)

Fig 3. Results in the average loss in 10,000 iterations of the model created

Fig 2. Quality of chilli (quality a, left-hand side) (quality b, right-hand side)

B. Validation of YOLO
The scope of this research is assumed to work on the
D. Pre-Training Model conveyor with the same lighting so that the model created can
sort out the chillies that are feasible (quality A) and unfit
In this research, we use a virtual machine (VM) from (quality B). Sorting is done based on the detection of chilli and
Google, namely Colab Notebook with Python 3 programming not chilli objects. Then if chilli is detected, it will be classified
language. Google Virtual Machine provides GPU facilities as with the quality of the chilli with a label. Tests that have been
a better and faster computational media, this is our conducted by us, show the number of steps affects the level of
consideration when deciding to use a virtual machine. Next, accuracy of detecting and classifying quality. Of the 10
we divided the composition of our dataset into 80% training iterations, the accuracy value calculation aims to find out how
data and 20% testing data. Other stages before creating a much the accuracy of the YOLO method changes in response.
custom model are as follows: Tests conducted show the number of steps has effect on the
(i) Labeling, provide ground truth labeling and box in the accuracy for detection and classification. Next step, this
image dataset using labelImg. Then the file will be saved validation is divided into two parts: (1) validation of accuracy
in txt format annotations. The naming of the label is in detecting objects as True or Not matching (False) (see Table
adjusted to the class that matches the information from III). We then calculate the accuracy, to calculate the accuracy
the expert source, namely chilli quality A and chilli in detecting the accuracy of chilli or not. We use equation 3;
quality B. (2) the validation of the classification chillis object is
successful as the quality of chilli A and the quality of chilli B
(ii) Cloning and Building Darknet, clone the darknet data on (Table IV).
the virtual machine. Then run the darknet repository and
download the yolov3.cfg file.
correct results
(iii) Configuration file, we use Yolov3 configuration file in Accuracy= x 100% (3)
total amount of data
this study. Before configuring the file: first, download
yolov3.weight, and run it on a virtual machine; Second,
do the settings in the file as downloaded previously, TABLE III
namely yolov3.cfg renamed to yolov3-custom.cfg then PRECISION IN DETECTING OBJECT
Detection of True, Threshold 0.3
set it as shown in Table II. Number
Chilli Chilli Chilli Chilli Accuracy
of batches Chilli 1
2 3 4 5
TABLE II
1000 T F F F T 40%
Yolov3.cfg FILE SETTINGS TO MAKE YOLO CUSTOM MODELS 2000 T T T T T 100%
Attribute Value 3000 T T T T T 100%
batch 64 4000 T T T T T 100%
subdivisions 16 5000 T T T T T 100%
classes 2 6000 T T T T T 100%
filters ((2+5) * 3) 21 7000 T T T T T 100%
learning_rate 0.001 8000 T T T T F 80%
max_batches 10000 9000 T T T T T 100%
10000 T T T T T 100%
We conduct training for up to 10000 steps and filtering Note: T: True; F: False; Chilli 1;5: Quality B. Chilli 2;3;4: Quality A.
21 5 ∗ 3 with two types of classes. TABLE IV
ACCURACY VALUE WHEN CLASSIFICATION WITH YOLO MODELS
III. RESULT AND DISCUSSION Detection Value True (%), Treshold 0.3
Number Accuracy
Chilli Chilli Chilli Chilli
A. System Performance of batches Chilli 1
2 3 4 5 (AVR %)
1000 60 0 0 32 34 25.2
Based on the filtering configuration performed on the 2000 95 84 89 99 65 86.4
yolov3-custom.cfg file, this YOLO model was trained for 3000 100 92 99 100 97 97.6
10,000 iterations with an average loss of 0.0498 (Fig. 3). The 4000 99 84 100 100 99 96.4
average loss is treated as a tolerance limit of the accuracy of 5000 100 98 98 100 98 98.8
the performance of the custom model made. The results of the 6000 100 97 98 100 77 94.4
7000 100 100 100 100 53 90.6
model's performance have been able to detect and classify 8000 100 100 100 100 40 88
objects accordingly (Fig. 4). 9000 100 100 100 100 97 99.4
10000 100 99 100 100 98 99.4
Note: Chilli 1;5: Quality B. Chilli 2;3;4: Quality A.

Authorized licensed use limited to: Auckland University of Technology. Downloaded on December 23,2020 at 16:54:42 UTC from IEEE Xplore. Restrictions apply.
 2020 International Conference on Computer Science and Its Application in Agriculture (ICOSICA)

1 2 3 5
4

Fig 4. Results of real-time detection and classification of chilli quality

Average accuracy value (AVR %)

120
97.6 96.4 98.8 99.4 99.4
94.4 90.6
100 86.4 88

80
value

60
40 25.2

20
0
0 2000 4000 6000 8000 10000 12000
Iteration

Fig 5. The average accuracy in the classification of five chilli objects

1
2 4
5

Fig 6. Classification of chilli quality in overlapping conditions

TABLE V
VALUE ACCURACY CONDITIONS WITH CHILLI OVERLAPPING

Number of Detection Value True (%), Treshold 0.3 Accuracy

Chilli 1 Chilli 2 Chilli 3 Chilli 4 Chilli 5
batches (AVR %)
(B) (A) (B) (A) (A)
9000 36 100 42 100 100 75.6
10000 39 99 0 100 100 67.6
Note : Chilli 1; 3: Quality B. Chilli 2; 4; 5: Quality A.

Of the 10 iterations described in Table IV, the trained The actual conditions curly red chilli has clustered and
models tested at steps 9000 to 10,000 produce the greatest overlapping characteristics, this is a very possible outcome in
accuracy value. The iteration steps both have an average the processing environment. Therefore, we tested the results
accuracy value of 99.4%. Steps 9000 and 10,000 are the most of the model against the overlapping chilli conditions as
stable trained models for classifying the quality of chilli shown in Fig. 6. We use the best iteration results in previous
precisely. tests, namely iteration 9000 and 10,000. From the tests we

Authorized licensed use limited to: Auckland University of Technology. Downloaded on December 23,2020 at 16:54:42 UTC from IEEE Xplore. Restrictions apply.
 2020 International Conference on Computer Science and Its Application in Agriculture (ICOSICA)

have conducted, the model with iteration 9000 has better [3] Ananto, I.D, Murinto. 2015. Aplikasi Pengolahan Citra Mendeteksi
accuracy than the model with iteration 10,000. Because in the Kualitas Cabai Berdasarkan Tingkat Kematangan Menggunakan
Transformasi Warna YCBCr. Jurnal Informartika UAD, 9(2).
9000 iterations, the model can be accessed and object
[4] Liantoni, F. and Annisa, F.N., 2018. Fuzzy K-Nearest Neighbor Pada
classification precisely. The accuracy obtained at iteration Klasifikasi Kematangan Cabai Berdasarkan Fitur HSV Citra. JIPI
step 9000 has an average accuracy of 75.6% (Table V). (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika), 3(2).
[5] Purwaningsih, T., Anjani, I.A. and Utami, P.B., 2018, August.
IV. CONCLUSION Convolutional Neural Networks Implementation for Chilli
In this research, we have developed a decision-making Classification. In International Symposium on Advanced Intelligent
Informatics (SAIN) (pp. 190-194). IEEE.
system with deep learning. The method we use is YOLO. We
make models for chilli quality classification. Simulation [6] Rahman, A., Faqeerzada, M.A., Joshi, R., Lohumi, S., Kandpal, L.M.,
Lee, H., Mo, C., Kim, M.S. and Cho, B.K., 2018. Quality Analysis of
results show that the number of iterations influences the level Stored Bell Peppers Using Near-Infrared Hyperspectral Imaging.
of accuracy of detection and the success of chilli Transactions of the ASABE, 61(4), pp.1199-1207.
classification. The iteration value of 9000 is the most perfect [7] Khuriyati, N., Nugroho, D.A. and Wicaksono, N.A., 2020, January.
trained models to detect and classify chilli according to the Quality assessment of chillies (Capsicum annuum L.) by using a
best detection accuracy values of 99.4%. However, for smartphone camera. In IOP Conference Series: Earth and
Environmental Science (Vol. 425, No. 1, p. 012040). IOP Publishing.
classification with overlapping curly red chilli conditions, the
accuracy value obtained from iterative step 9000 is the most [8] Redmon, J., Divvala, S., Girshick, R. and Farhadi, A., 2016. You only
look once: Unified, real-time object detection. In Proceedings of the
accurate at 75.6%. IEEE conference on computer vision and pattern recognition (pp. 779-
788).
Suggestions for the next study, the model can do a
comprehensive sorting of chilli quality (all regions of chilli). [9] Redmon, J. and Farhadi, A., 2017. YOLO9000: better, faster, stronger.
In Proceedings of the IEEE conference on computer vision and pattern
recognition (pp. 7263-7271).
ACKNOWLEDGMENT
[10] Redmon, J. and Farhadi, A., 2018. Yolov3: An incremental
We would like to thank the Department of Computer improvement. arXiv preprint arXiv:1804.02767.
Science, IPB University and PasTi Scholarship for supporting [11] Lan, W., Dang, J., Wang, Y., & Wang, S. 2018. Pedestrian Detection
this research financially. Based on YOLO Network Model. Proceedings of 2018 IEEE
International Conference on Mechatronics and Automation, 1547-
REFERENCES 1551.
[12] BSN. 1998. Standar Nasional Cabai Merah Segar: SNI 01-4480-1998.
[1] Yanuarti, AR. and Asfari, MD. 2016. Profil Komoditas Barang
Jakarta.
Kebutuhan Pokok dan Barang Penting Komoditas Cabai.
https://ews.kemendag.go.id/download.aspx?file=BK_CABAI_16-03- [13] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y.
2018-SP2KP.pdf&type=publication. Accessed june 18, 2020. LeCun. Overfeat: Integrated recognition, localization and detection
using convolutional networks. CoRR, abs/1312.6229, 2013.
[2] [PDSIP] Pusat Data dan Sistem Informasi Pertanian. 2019. Outlook
Cabai – Komoditas Pertanian Subsektor Hortikultura. Jakarta [ID].
Kementerian Pertanian. http://epublikasi.setjen.pertanian.go.id/arsip-
outlook/76-outlook-hortikultura/694-outlook-cabai-2027. Accessed
june 13, 2020.

Authorized licensed use limited to: Auckland University of Technology. Downloaded on December 23,2020 at 16:54:42 UTC from IEEE Xplore. Restrictions apply.