International Conference on Intelligent Computing and Control Systems

ICICCS 2017

Content Based Image Retrieval using

Feature Extraction with Machine Learning
Aasia Ali Sanjay Sharma
M.Tech Student Assistant Professor
Department of computer science and engineering Department of computer science and engineering
Shri Mata Vaishno Devi University Katra J&K Shri Mata Vaishno Devi University Katra J&K
[email protected] [email protected]

Abstract- Content-Based image Retrieval (CBIR) is a computer system (low-level) perception. The human
technique of image retrieval which uses the visual brain is capable in performing the complex visual
features of an image such as color, shape and texture perception at a very faster rates but a computer system
in order to search the user based query images from cannot. In CBIR, visual image contents are represented
the large databases. CBIR depends on feature as image features, which are extracted using
extraction of an image which are the visual features automatically done feature extraction methods. Thus
and these features are extracted automatically i,e human intervention is eliminated. The feature extraction
without human interaction. In this paper SIFT methods are computationally expensive and have also
feature extraction algorithm is used for feature large dimensionality and these methods tend to be
extraction, which basically gives us the key point in domain specific. A lot of research scope for simplifying
an image. SIFT image feature algorithm give a set of these algorithms need to be addressed [6]. CBIR system
image features that are not valuable so we use the have two main steps, the first step involves the per-
optimization technique BFOA (Bacteria foraging processing of images from the image database. The
optimization algorithm) to reduce the complexity, image features are extracted from the database images
cost, energy and Time consumptions. Then for and are stored as a feature database of images. This step
similarity check a deep neural network is trained and involves a large number of calculation and is a
then the validation and texting phases are done challenging step as how to extract the image features [7].
accordingly which lead to a better performance as Image feature extraction is usually done on the bases of
compared to previously done techniques. The color, shape, texture that are low-level feature
accuracy rates are relatively excellent in the proposed representing the image and many methods have been
technique. implemented for the same. The second step involves the
similarity measures, which is an important step as our
Keywords- Content Based Image Retrieval (CBIR), results are retrieved after the similarity measure check.
Scale invariant feature transformation (SIFT), The user query image features are matched with the
Bacteria foraging optimization algorithm (BFOA), stored feature database that gives us the best possible
Deep neural network (DNN). match according to the user query image. Several
similarity matching methods have been proposed which
I. INTRODUCTION usually calculate the distance between the query image
and the database images. The basic use of SIFT
With the use of internet and WWW tremendous increase algorithm is that this method gives us the better results
in digital data like images, videos and audios is being than other feature extraction techniques. This algorithm
generated per second. A very large amount of data is is invariant to rotation, scaling methods so we need not to
available and with each passing day gigabytes of worry about these issues the only thing of concern is the
information is generated [1]. A user is much interested to illumination and camera point of view. The SIFT
find their relevant information such as images from those algorithm give us the best distinctive points in an image.
multimedia databases. In order to handle such a large The similarity is done with the training of DNN (deep
amount of images an efficient image retrieval technique neural network) which has initially a population size. In
need to be addressed [2]. The problem of acquiring, training section, calculate the reduce features passed in
transmitting and storing of images is very old and is a training set which is calculate based on the target. In
very well-known problem [3 4]. The methods used to testing phase, we check the image in the query training
manipulate, index, sort, filter, summarize, or search set and detect or retrieve the image and calculate the
through image database lack a well-defined procedure, so performance parameters. Evaluate the performance
in order to have better results of retrieval we need to go parameters like accuracy, false acceptance rate, false
through. The previous technique uses the text or rejection rate and mean square error rate etc. In propose
annotation based approach in order to retrieve images work, we compare the performance parameters i.e
[5]. Based on the similarity of textual annotations given accuracy and existing work in performance parameters.
to images the retrieval was done were the performance of
retrieval was poor. So content based image retrieval is II. RELATED WORK
introduced.
Lei Zhu, Jialie Shen, Liang Xie, 2016 [8] defined a
A content Based Image Retrieval (CBIR) is an interface novel unsupervised visual hashing approach to
between the semantic gaps which is described as the effectively perform visual hashing learning with
difference between a human brain (high-level) and a

978-1-5386-2745-7/17/$31.00 ©2017 IEEE 1048

 International Conference on Intelligent Computing and Control Systems
ICICCS 2017

semantic assistance. The key idea in their proposed work algorithm, which gives us the best possible key points in
is to extract semantics automatically from the noisy an image that are invariant to scale and rotation.
associated texts to enhance the discriminative capability The SIFT method extracts the visual features but with the
of hash codes, and thus improving the performance of problem of time complexity and memory usage. So we
visual hashing. The hash codes are formulated first in a introduce an optimization algorithm (BFOA) which gives
unified unsupervised framework, which otherwise are us the optimum and best feature match from a set of
learned by simultaneously preserving visual similarity of population by means of mutation. The similarity check is
images. The optimized method used here is based on done by training the deep neural network and thus the
Augmented Lagrangian Multiplier (ALM), which parameters like accuracy, false rejection rate, false
iteratively calculate the optimal solution. acceptance rate and also the mean square error are
calculated.
Sadegh Fadaei, Rassoul Amir Fattahi, Mohammad The main idea of our proposed CBIR method is shown as
Reza Ahmadzadeh, 2016 [9] The CBIR schema in this below:
paper is proposed based on the optimized combination of
the colour and texture feature to evaluate the parametric
calculations. The main focus in the work done is on
partitioning schema which is applied in the HSV colour
space to extract the dominant colour descriptors features.
The wavelet and curvelet transformations on texture
feature are calculated and the colour feature is used in the
proposed work. A combination of colour feature and
texture feature is combined optimally using particle
swarm optimization algorithm.

Alireza Pourreza, Kourosh Kiani, 2016 [10] proposed

a use of colour based SIFT matching algorithm for partial
duplicate image retrieval, which extracts the most
dominant colour out of the content as a silent region. The
image key points are detected by applying SIFT
algorithm on the images from the database. The colour
histograms are used in combination with the SIFT
algorithm. The data set used are IPDID and INSTRE
Fig 1.Proposed flowchart
with the excellent performance of retrieval.

Gholam Ali Montazer, Davar Giveki, 2015 [11] the

basis of the proposed methods is built upon scale
Algorithm
invariant feature transform (SIFT) algorithm. After
extracting image features using SIFT, k-means clustering
1. Read the image dataset.
is applied on feature matrix extracted by SIFT, and then
two new kinds of dimensionality reductions are applied 2. Extract the features using SIFT method.
to make SIFT features more efficient and realistic for 3. Optimize the feature set by using
image retrieval problem. The proposed work also deals optimization algorithm (BFOA).
with the dimensionality reduction. The dataset used 4. Classify the optimized data using DNN.
belongs to two popular databases, Caltech 101 and Li 5. Extract the features of the query image
database, with the superiority and efficiency of the
(RGB).
results.
6. Optimize the query image features.
Bindita Chaudhuri, Begüm Demir, Lorenzo 7. Classify features from the feature database.
Bruzzone, Subhasis Chaudhuri, 2016 [12] the 8. Categorize the calculations based on both the
proposed method involves a novel unsupervised graph images query image and database image.
theoretical approach in the framework of region based 9. The images with best possible match are
retrieval of remote sensing images. The proposed method
involves modeling of images by means of graph, which
provides the region based approach located on both local
and spatial features distribution and then, the query
IV. IMPLEMENTATION AND RESULTS
image is retrieved base on the evaluation of the graph
based similarity.
The data set used in the proposed method is Caltech-101
III. PROPOSED METHODOLOGY image dataset, which contains the images of humans,
animals, butterflies and flowers [14]. These images are
As stated earlier CBIR is a retrieval technique of images divided into categories. The GUI uses the MATLAB
from large image database based on visual features of an programming interface.
image such as colour, shape and texture. The basic
concept used in this paper involves the color feature,
shape feature and also texture feature for retrieval of
images. These features are extracted using SIFT

978-1-5386-2745-7/17/$31.00 ©2017 IEEE 1049

 International Conference on Intelligent Computing and Control Systems
ICICCS 2017

The testing phase now involves steps upload test image,

covert it into black and white, calculate edges based on
regions and check the noise level and filter it. The
parameter calculation mean square error, accuracy, false
acceptance rate, false rejection rate and precision is
shown below in figure.

Figure 2 .Main figure.

The database used is Caltech_101 database consisting of

different images categorized into 101 classes. The
images are shown in below figure.

Figure 6. False acceptance rate.

The above figure defined that the false acceptance rate

Figure 3. Images in Dataset
means how much wrong data or images acceptable in the
original image features.
The edge detection is calculated firstly by using canny
method, which is a multistage algorithm gives the wide
range of edges in image. The image filtering is done and
the feature extraction algorithm is applied which extracts
the features based on steps like rotation, scaling,
assignment and orientation and key point distributor.

The bacteria foraging optimization algorithm is used to

reduce the extracted feature matrix, which shows the
results shown in below figure.

Figure 7. False Rejection rate.

The above figure defines that the false rejection rate

means how much true data is rejected in the original data
Figure 4. Bacteria foraging algorithm results graph. set.
The neural network is trained is trained which starts with Table no 1. False acceptance rate and false rejection rate
initialization of the dataset defined and is based on 100
is shown in below table:
iterations but input process in 3 iterations, time
consumed is 2 seconds, performance is 37.4,gradient
value is 9.12 and mutation 1.0, validation check 2 No of Images FAR FRR
process out of 6. The below graph shows the best Animal 0.00106 0.00531
validation performance is 20.2439 at epoch 1. Butterfly 0.00218 0.0111
Facial 0.00329 0.0164
Flower 0.00547 0.027

Figure 5. Validation performance.

978-1-5386-2745-7/17/$31.00 ©2017 IEEE 1050

 International Conference on Intelligent Computing and Control Systems
ICICCS 2017

differences. It is called the mean squared error as you are

Parameters - FAR and FRR finding the average of a set of errors.

0.03
FAR and FRR

0.02

0.01 FAR
FRR
0

Figure 8. FAR and FRR Graphical view. Figure 11. Precision.

The above figure depicts that false acceptance rate which Figure represents that the precision that the nearest of
is defined how much our wrong is accepted correctly. two or more measurements to each other.
The false rejection rate depicts the how much true data is
rejected in our system. Table no 2. Showing the precision and accuracy.

No of Images Accuracy Precision

Animal 17 19
Butterfly 38 39
Facial 57 69
Flower 95 99

Performance Parameter - Accuracy and

Precision
Accuracy and Precision (%)

Figure 9. Accuracy.

100
The above figure defined that the accuracy which means
how accurate the system is based on DNN. 50
Accuracy
0
Precision
Facial
Butterfly

Flower
Animal

Figure 12. Performance parameters of accuracy and

precision.

The above figure defined that the accuracy means

accurate the system based on DNN. The False rejection
rate is less than the improved performance of the images
Figure 10. Mean square error rate. of the dataset. It represents that the precision of the
nearest of two or more measurements to each other.
The above figure represents that the training state error
and testing state error sum is equal to mean square error
rate. The close a regression line is to a set of points. It
does this by taking the distances from the points to the
regression line (these distances are the “errors”) and
squaring them. The squaring is necessary to remove any
negative signs. It also gives more weight to larger

978-1-5386-2745-7/17/$31.00 ©2017 IEEE 1051

 International Conference on Intelligent Computing and Control Systems
ICICCS 2017

results show a considerable improvement in terms of

FRR, FAR and accuracy of image retrieval.
Future scope include implementing the CBIR system
considering more low-level image descriptors and highly
efficient deep learning neural network, that may possibly
verify to be quite fast as well as precise one. This work
can be extended by integrating with Fuzzy C-means
clustering algorithm for better efficiency.

REFERENCES

[1] Wan, Shouhong, et al. "Incorporating Spatial Distribution

Feature with Local Patterns for Content-Based Image
Retrieval." Chinese Journal of Electronics 25.5 (2016): 873-
879.

[2] Mohamadzadeh, Sajad, and Hassan Farsi. "Content-based

Figure 13. Accuracy between proposed and existing image retrieval system via sparse representation." IET Computer
work. Vision 10.1 (2016): 95-102.

The above figure represents that the comparison between [3] Kekre, H., S.D. Thepade, and A. Maloo, Extended
accuracy, performance parameters based on proposed Performance Appraise of Image Retrieval Using the Feature
and existing work). We improve the performance of Vector as Row Mean of Transformed Column Image.
accuracy with proposed (DNN and BFOA).
[4] Kekre, H., et al., Improved Shape Content Based Image
Table no 3. Comparison of accuracy between the Retrieval Using Multilevel Block Truncation Coding.
proposed and existing work. [5] Chun.Y, Kim.N, Jang.I,"Content-Based Image Retrieval
Using Multiresolution Color and Texture Features," IEEE
No of Accuracy Accuracy (proposed) Transactions On Multimedia, Vol. 10, No. 6, October 2008, pp.
Images (Base)
Animal 50 60 [6] Jain, Mihir. TOWARDS EFFICIENT AND SCALABLE
Butterfly 70 76.7 VISUAL PROCESSING IN IMAGES AND VIDEOS. Diss.
International Institute of Information Technology Hyderabad,
Facial 88 97 2010.
Flower 96 98
[7] Amores, J., et al. Boosting contextual information in content-
based image retrieval in Proceedings of the 6th ACM SIGMM
The table 3 defined that the number of images to enhance international workshop on Multimedia information retrieval.
the accuracy based on deep neural network and bacteria 2004. ACM.
foraging optimization algorithm.
[8] Zhu, Lei, Jialie Shen, and Liang Xie. "Unsupervised Visual
Hashing with Semantic Assistant for Content-based Image
V. CONCLUSION AND FUTURE SCOPE Retrieval." IEEE Transactions on Knowledge and Data
Engineering (2016).
In this research paper, the problem of image retrieval will
be solved using combination of SIFT, BFOA and DNN. [9] Fadaei, Sadegh, Rassoul Amirfattahi, and Mohammad Reza
Firstly feature extraction will be done using SIFT. Then, Ahmadzadeh. "New content-based image retrieval system based
at first, the neural network is trained based on the on optimised integration of DCD, wavelet and curvelet
features." IET Image Processing (2016).
features of images in the database. In BFOA algorithm
used to optimize the feature set. This algorithm is an [10] Pourreza, Alireza, and Kourosh Kiani. "A partial-duplicate
initialize the set of size i.e., called population. Problem image retrieval method using color-based SIFT." Electrical
Solutions from individual population are used and Engineering (ICEE), 2016 24th Iranian Conference on. IEEE,
reserved to new population. This is hope, that the novel 2016.
population would be better than previous one. Results
which are particular to form novel solution i.e. data [11] Montazer, Gholam Ali, and Davar Giveki. "Content based
stream bits are selected with the help of best cost image retrieval system using clustered scale invariant feature
function, the suitable phases they have to regenerate. transforms." Optik-International Journal for Light and Electron
Optics 126.18 (2015): 1695-1699.
BFOA techniques, to solve an optimization issue by
repetition the following three Phases: [12] Chaudhuri, Bindita, et al. "Region-Based Retrieval of
Remote Sensing Images Using an Unsupervised Graph-
- Rotation Theoretic Approach." IEEE Geoscience and Remote Sensing
- Dispersal and eliminate Letters 13.7 (2016): 987-991.
- Reproduction
[13] Rashno, Abdolreza, Saeed Sadri, and Hossein
The image features considered here are average value, SadeghianNejad. "An efficient content-based image retrieval
min value and max Value. The training is carried out with ant colony optimization feature selection schema based on
using DNN algorithm. This trained when presented with wavelet and color features." Artificial Intelligence and Signal
Processing (AISP), 2015 International Symposium on. IEEE,
a query image retrieves and displays the images which
2015.
are relevant and similar to query from the database. The

978-1-5386-2745-7/17/$31.00 ©2017 IEEE 1052

 International Conference on Intelligent Computing and Control Systems
ICICCS 2017

[14] Anandh, A., K. Mala, and S. Suganya. "Content based [16] Gonzalez & Woods,“Digital Image Processing using
image retrieval system based on semantic information using MATLAB” Pearson Education, Third Edition,2009 reprint.
color, texture and shape features." Computing Technologies and
Intelligent Data Engineering (ICCTIDE), International [17] Caltech-101 Image Database. [Online]. Available:
Conference on. IEEE, 2016. http://www.vision.caltech.edu/Image_Datasets/Caltech101/.

[15] Jiang, W., et al., Similarity-based online feature selection in

content based image retrieval. Image Processing, IEEE
Transactions on, 2006. 15(3): p. 702-712.

978-1-5386-2745-7/17/$31.00 ©2017 IEEE 1053