HAND WRITTEN PATTERN RECOGNITION USING

CONVOLUTIONAL NEURAL NETWORK

By

Shibani Khatua (1641014201)

DEPARTMENT OF ELECTRICAL AND

1
 HAND WRITTEN PATTERN RECOGNITION USING
CONVOLUTIONAL NEURAL NETWORK

Project report submitted

By

Shibani Khatua (1641014201)

DEPARTMENT OF ELECTRICAL AND ELECTRONICS

ENGINEERING
Institute of Technical Education and Research
SIKSHA ‘O’ ANUSANDHAN (Deemed to be) UNIVERSITY
Bhubaneswar, Odisha, India
(May,2020)

2
CERTIFICATE

This is to certify that the project titled “ Hand Written Pattern Recognition using

convolutional neural Network“ submitted by Shivani Kumari and Shibani

Khatua to the Institute of Technical Education and Research, SIKSHA‘O’

ANUSANDHAN(Deemed to be) University, Bhubaneswar for the partial

fulfilment for the degree of Bachelor of Technology in Electrical and Electronics

Engineering is a record of original bonafide work carried out by them under my

/ our supervision and guidance. The project work, in my / our opinion, has reached

the requisite standard, fulfilling the requirements for the degree of Bachelor of

Technology.

The results contained in this thesis have not been submitted in part or full to

any other university or institute for the award of any degree or diploma.

Dr Niranjan Nayak Mrutyunjaya Sahani

ITER ITER

3
 CONTENTS

CHAPTERS PAGES

Chapter 1: INTRODUCTION 7-8

Chapter 2: LITERATURE SURVEY 9-12

Chapter 3: CONCEPT GENERATION & SELECTION 13-16

Chapter 4: PROJECT MODELLING AND SIMULATION 17-19

Chapter 5: RESULTS 20-21

Chapter 6: CONCLUSION & SCOPE OF FUTURE WORK 21-22

Chapter 7: INDIVIDUAL AND GROUP LEARNING 22

Chapter 8: REFERENCES 23

4
 DECLARATION

We hereby declare that this written submission report represents our ideas in our own
words and where other’s ideas and words have been included. We have adequately
cited and referenced the original sources. We also declare that we have adhered to all
the principles of academic honesty and integrity and have not misrepresented or
fabricated or falsified any idea / source / fact in our submission.

We understand any violation of the above will be cause for disciplinary action
by the University and can also evoke penal action from the sources which have thus
not been properly cited or from whom proper permission have not been taken when
needed.

Shibani Khatua (1641014201)

Shivani Kumari (1641014027)

Date: 18.05.20

5
 REPORT APPROVAL
This project report entitled “Hand Written Pattern Recognition using convolutional

Neural Network “ by (Shibani Khatua and Shivani Kumari) is approved for the
degree

Bachelor of Technology in Electrical and Electronics Engineering.

Examiners

Supervisors

Mr. Mrutyunjaya Sahani

H.O.D.

Dr. Niranjan Nayak

Date: 18.5.20 Place: Bhubaneswar

ABSTRACT

6
 Handwritten character recognition has been one of the active and challenging

research areas in the field of image processing and pattern recognition. A neural

network is a feed forward neural network used for classification and recognition of

handwritten characters using numerous steps . It involve numerous applications

which are reading aid for blind, bank cheques and conversion of any hand written

document into structural text form . An attempt is made to recognize handwritten

characters for English alphabets without feature extraction using multilayer Feed

Forward neural network. Each character data set contains alphabets. Fifty different

character data sets are used for training the neural network. In the proposed

system, each character has been resized into 30x20 which is directly subjected to

training. That is, each resized character has 600 pixels and these pixels are taken as

features for training the neural network . The results show that the proposed

system yields good recognition rates which are comparable to that of feature

extraction based schemes for handwritten character recognition.

INTRODUCTION:-.

7
Hand written characters are easy to understand by humans as they have the ability to

learn. This ability has been fed to the machines by artificial intelligence and machine

learning . And the field that deals with these characters are known as OCR ( optical

character recognition ) . Character recognition ,an art of detecting, segmenting and

identifying characters from image .An ultimate objective of hand written character

recognition is to simulate the human reading capabilities so that the computer can read

, understand, edit and work as human do with text. Handwriting recognition has been

one of the most fascinating and challenging research areas in field of image processing

and pattern recognition in the recent years .It contributes immensely to the

advancement of automation process and improves the interface between man and

machine in numerous applications. Several research works have been focusing on new

techniques and methods that would reduce the processing time while providing higher

recognition accuracy. Character recognition is mainly of two types online and offline.

In online character recognition, data is captured during the writing process with the

help of a special pen on electronic surface. In offline recognition, prewritten data

generally written on a sheet of paper is scanned. Offline Character Recognition:

Generally all printed or type-written characters are classified in Offline mode. Off-line

handwritten character recognition refers to the process of recognizing characters in a

document that have been scanned from a surface such as a sheet of paper and are

stored digitally in gray scale format. The storage of scanned documents have to be

bulky in size and many processing applications as searching for a content, editing,

maintenance are either hard or impossible.

The online mode of recognition is mostly used to recognize only handwritten

characters. In this the handwriting is captured and stored in digital form via different

8
means. Usually, a special pen is used in conjunction with an electronic surface. As the

pen moves across the surface, the two- dimensional coordinates of successive points

are represented as a function of time and are stored in order. Recently, due to

increased use of handheld devices online handwritten recognition attracted attention of

worldwide researchers. This online handwritten recognition aims to provide natural

interface to users to type on screen by handwriting on a pad instead of by typing using

keyboard. The online handwriting recognition has great potential to improve user and

computer communication. Several applications including mail sorting, bank

processing, document reading and postal address recognition require offline

handwriting recognition systems. As a result, the off-line handwriting recognition

continues to be an active area for research towards exploring the newer techniques that

would improve recognition accuracy .

FIG NO-1 PATTERN RECOGNITION PROCESS

9
LITERATURE SURVEY
Character recognition is a fundamental, but most challenging in the field of pattern

recognition with various number of useful applications. It has been a vast area of

research since the early days of computer science due to it being a natural way of

interactions between computers and humans. More precisely Character recognition

is the process of detecting and recognizing characters from the input image and

converts it into ASCII or other equivalent machine editable form. Various

techniques have been proposed to for character recognition in handwriting

recognition system. Even though, sufficient studies and papers describes the

techniques for converting textual content from a paper document into machine

readable form. In coming days, character recognition system might serve as a key

factor to create a paperless environment by digitizing and processing existing

paper documents.

10
Working principle:-

Handwritten recognition is normally divided into six phases which are image

Acquisition, pre-processing, segmentation, feature extraction, classification and

post processing. The block diagram of the basic character recognition is shown :-

Image Acquisition

Pre-processing

Segmentation

Feature Extraction

Classification

Post Processing

A. Image Acquisition-- Digital Image is initially taken as input. The most common of

these devices is the electronic tablet or digitizer. These devices use a pen that is digital

in nature. Input images for handwritten characters can also be taken by using other

methods such as scanners, photographs or by directly writing in the computer by using

a stylus.

B. Pre-processing-- Pre-processing is the basic phase of character recognition and it is

crucial for good recognition rate. The main objective of pre-processing steps is to

normalize strokes and remove variations that would otherwise complicate recognition

11
and reduce the recognition rate. These distortions include the irregular size of text,

missing points during pen movement collections, jitter present in irregular size of text,

missing points during pen movement collections, jitter present in text, left or right

bend in handwriting and uneven distances of points from adjacent positions. Pre-

processing includes five common steps, namely, size normalization and centering,

interpolating missing points, smoothing, slant correction and re-sampling of points.

C. Segmentation-- Segmentation is done by separation of the individual characters of

an image. Generally document is processed in a hierarchical way. At first level lines

are segmented using row histogram. From each row, words are extracted using column

histogram and finally characters are extracted from words.

D. Feature Extraction --The main aim of feature extraction phase is to extract that

pattern which is most pertinent for classification . Feature extraction techniques like

Principle Component Analysis , Linear Discriminant Analysis , Chain Code , Scale

Invariant Feature Extraction , zoning, Gradient based features, Histogram might be

applied to extract the features of individual characters. These features are used to train

the system.

E. Classification-- When input image is presented to the system, its features are

extracted and given as an input to the trained classifier like artificial neural network or

support vector machine. Classifiers compare the input feature with stored pattern and

find out the best matching class for input.

F. Post Processing-- Refers to the procedure of correcting misclassified results by

applying linguistic knowledge. Post – processing is processing of the output from

shape recognition. Language information can increase the accuracy obtained by pure

shape recognition. For handwriting input, some shape recognizers yield a single string

of characters, while others yield a measure of confidence for each alternative.

12
Comparision between different techniques :-
Method Accuracy Purpose
OCR for cursive 88.8% for lexicon To implement
handwriting. size 40,000 segmentation and
recognition
algorithms for
cursive handwriting.
Recognition of 95% for Hindi and The aim is to utilize
handwritten 98.4% for English the fuzzy technique
numerals based upon numerals overall to recognize
fuzzy model handwritten
numerals for Hindi
and English
numerals.
Combining decision 89.6% overall. To use a reliable and
of multiple an efficient
connectionist technique for
classifiers for classifying
Devanagari numeral numerals.
recognition. [
Hill climbing 93% for uppercase To implement hill
algorithm for letters. climbing algorithm
handwritten for selecting feature
character subset.
recognition.
Optimization of 88% for numbers To apply a method
feature selection for and 70% for of selecting the
recognition of letters. features in an
Arabic characters optimized way.
99.56% for To find out the
Handwritten Devanagari, recognition rate for
numeral recognition 98.99% for the six popular
for six popular Bangla, 99.37% Indian scripts.
Indian scripts. for Telugu,
98.40% for
Oriya, 98.71% for
Kannada and
98.51% for Tamil
overall.

13
CONCEPT GENERATION:-

Handwritten pattern recognition played a big role in the technology world then. It also

Played an important role in the storage and in the recovery of critical handwriting

Information. This handwritten recognition ensured an accurate medical care and it also

reduced storage costs. It ensured that an essential field of research remains available to

students in the future. Various applications of handwritten pattern recognition were

there like National ID number recognition, postal office automation with code number

recognition on envelope, automatic license plate recognition and bank automation In

this era of globalization technologies continue to improve and improve more in no

time.

TRADITIONAL TECHNIQUES

(a) CHARACTER EXACTRACTION

Off-line character recognition often involved scanning a form or document

written sometimes in the past. This means the individual characters contained

in the scanned image would need to be extracted. Tools existed that were

capable of performing this step. However, there were several common

imperfections in this step. The most common was when characters that

connected were returned as a single sub-image containing both characters. This

caused a major problem in the recognition stage. Yet many algorithms were

available that reduced the risk of connected characters.

14
 (b) CHARACTER RECOGNITION

After the extraction of individual characters occurred, a recognition engine that

used to identify the corresponding computer character. Several different

recognition techniques were currently available.

(c) FEATURES EXTRACTION

Feature extraction worked in a similar fashion to neural network recognizers.

However, programmers must manually determine the properties they feel were

important. This approach gave the recognizer more control over the properties

used in identification. Yet any system using this approach required

substantially more development time than a neural network because the

properties were not learned automatically.

Modern techniques

Where traditional techniques focused on segmenting individual characters for

recognition, modern techniques focused on recognizing all the characters in a

segmented line of text. Particularly they focused on machine learning techniques

which were able to learn visual Features, avoiding the limiting feature engineering

previously used. State-of-the-art methods used convolutional network to extract visual

features over several overlapping windows of a text line image which an RNN used to

produce character probabilities .

15
Online recognition :-

On-line handwriting recognition involved the automatic conversion of text as it is

written on a special digitizer or PDA, where a sensor picked up the pen-tip movements

as well as pen-up/pen-down switching. This kind of data known as digital ink and can

be regarded as a digital representation of handwriting. The obtained signal then

converted into letter codes which were usable within computer and text-processing

applications. The elements of an on-line handwriting recognition interface typically

include: (a) a pen or stylus for the user to write with.

(b) a touch sensitive surface, which may be integrated with, or adjacent to, an output

display.

(c) a software application which interpreted the movements of the stylus across the

writing.

(d) Surface , translating the resulting strokes into digital text. And an off-line

recognition is the problem.

General process

The process of online handwriting recognition can be broken down into a few general

steps:

(a)pre-processing,

(b)feature extraction

(c)classification

16
The purpose of preprocessing was to discard irrelevant information in the input data,

that could negatively affect the recognition. This concerned speed and accuracy.

Preprocessing usually consists of binarization, normalization, sampling, smoothing

and denoising. The second step was feature extraction. Out of the two- or more-

dimensional vector field received from the preprocessing algorithms, higher-

dimensional data was extracted. The purpose of this step was to highlight important

information for the recognition model. This data might include information like pen

pressure, velocity or the changes of writing direction. The last big step was

classification. In this step various models were used to map the extracted features to

different classes and thus identifying the characters or words the features represent.

17
 CONCEPT SELECTION :-

Character recognition from handwritten images has received greater attention in

research area of pattern recognition due to vast applications and ambiguity in the

of learning methods learning methods. Primarily, two steps including character

recognition and feature extraction are required based on some classification

algorithm for handwritten pattern recognition. Previous schemes exhibit lack of

high accuracy and low computational speed for handwritten pattern recognition

process. The aim of the proposed attempt was to make the path toward

digitalization should be clearer and provide high accuracy and faster

computational for recognizing the handwritten pattern. The present research

employed convolutional neural network .As classifier, MNIST as dataset with

suitable parameters for training and testing and deep learning framework for

handwritten pattern recognition. The convolutional neural network system

successfully imparts accuracy up to 99.21% which is higher than formerly

18
 proposed schemes. In addition, the proposed system reduces computational time

significantly for training and testing due to which algorithm becomes efficient .

PROJECT MODELLING:-

19
 FIG NO-2 OVERVIEW OF CONVOLUTIONAL NEURAL
NETWORK FRAMEWORK

(a) Convolutional layer (CNL)

Convolutional neural network (CNL) is the first layer in convolutional neural

network which memorizes the features of input image which covers its entire

20
 region during scanning through vertical and horizontal sliding filters. It adds a bias

for every region followed by evaluation of scalar product of both filter values and

image regions. For thresholding element-wise activation function, such as

max(0,x), sigmoid and tan(h), is applied to output of this layer via rectified linear

unit.

(a) Pooling layer (PL)

At second, there comes pooling layer which is also called as max pooling layer or sub

Sampling. In pooling layer (PL), shrinkage in the volume of data takes place for the

easier and faster network computation. Max pooling and average pooling are main

tools for implementing pooling. This layer obtains maximum value or average value

for each region of the input data by applying vertical and horizontal sliding filters

through input image and reduces the volume of data .

(b) Fully connected layer or dense layer

At last , there is fully connected layer after convolution and pooling layer in the

standard neural network (separate neuron for each pixel) which is comprised of n

numbers of neurons, where n is the predicted class number.

For example,

There are ten neurons for ten classes (0–9) in digit character classification

problem. However, there should be 26 neurons for 26 classes (a–z) for English

character Classification Problem However, deep neural network architecture

consists of many nonlinear hidden layers with a enormous number of connections

and parameters. Therefore, to train the network with very less amount of samples

is a very difficult task. In convolutional neural network , only few set of

parameters are needed for training of the system. So, convolutional neural network

21
is the key solution capable map correctly datasets for both input and output by

varying the trainable parameters and numbers of hidden layers with high accuracy.

Hence, in this work , convolutional neural network architecture with deep learning

framework had been considered as the best fit for the character recognition from

the handwritten pattern images . For the experiments and the verification of the

system’s performance , the normalized standard MNIST dataset is utilized.

FIG NO-3 Architecture of the convolutional neural network

22
FIG NO-4 Control flow diagram

23
RESULTS :-
These are the required outputs of the given MATLAB code proposed by us :-

24
25
CONCLUSION

The aim of our project is to make an interface that can be used to recognize user

Hand written characters .We approached our problem using Convolutional neural

Networks in order to get a higher accuracy .Using modern day techniques like

neural networks to implement deep learning to solve basic tasks which are done

with a blink of an eye by any human like text recognition is just scratching the

surface of the potential behind machine learning. There are infinite possibilities

and application of this technology. Traditional optical character recogniser used to

work similar to biometric device. Photo sensor technology was used to gather the

match points of physical attributes and then convert it into database of known

types.

But with the help of modern-day techniques like convolution neural networks we

are able to scan and understand words with an accuracy never seen before in

history.

Numerous application of handwritten pattern recognition using convolutional

neural network are there like reading postal addresses, bank check amounts, and

forms convolutional neural networks have accomplished astonishing

achievements across a variety of domains, including medical research, and an

increasing interest has emerged in radiology.

26
INDIVIDUAL AND GROUP LEARNING:-

Through this project I learned as an individual team member that

 should actively, participate in meetings and shares knowledge, expertise, ideas

and information.

 I should be enthusiastic.

 I should carefully work .

 I should respect other’scontribution.

 I should be committed to team objectives.

 I should carry out assignments between meetings such as collecting data,

observing processes, charting data and writing reports.

 Through this project I learned as a team.

 When a group of individuals works together, compared to one person working alone,

they promote a more efficient work output and are able to complete tasks faster due to

many minds intertwined on the same goals and a particular objectives . Working in a

team enables us to learn from one another’s mistakes. one can able to avoid future

errors, gain insight from differing perspectives, and learn new concepts from more

experienced teammate

27
REFERENCES :-

 Surya Nath R S M, Afseena “Handwritten Recognition- A review”,

International Journal of Scientific and Research Publications

 Anita Pal and Davashankar Singnh,”Handwritten English Character

Recognition Using Neural Network”, International Journal of Computer

Science and Communication

 Manoj Sonkusare and Narendra Sahu “A survey on handwritten character

recognition (hcr) techniques for english alphabets” Advances in Vision

Computing: An International Journal (AVC)

 Kalchbrenner N, Grefenstette E, Blunsom P. A convolutional neural network

for modelling sentences. Paper (XIV:1404.2188). Apr 8, 2014.

 Ciresan DC, Meier U, Gambardella LM, Schmidhuber J. “Convolutional

neural network committees for handwritten character classification”. Paper

(1135-1139). Sep 18,2011 IEEE.

 Nisha Sharma et al, “Recognition for handwritten English letters: A Review”

International Journal of Engineering and Innovative Technology (IJEIT)

Volume 2, Issue 7, January 2013.

28