2022 IEEE Fourth International Conference on Advances in Electronics, Computers and Communications (ICAECC), 2022
Nowadays, Storing information from handwritten documents for future use is becoming necessary. An... more Nowadays, Storing information from handwritten documents for future use is becoming necessary. An easy way to store information is to capture handwritten documents and save them in image format. Recognizing the text or characters present in the image is called Optical Character Recognition. Text extraction from the image in the recent research is challenging due to stroke variation, inconsistent writing style, Cursive handwriting, etc. We have proposed CNN and BiLSTM models for text recognition in this work. This model is evaluated on the IAM dataset and achieved 92% character recognition accuracy. This model is deployed to the Firebase as a custom model to increase usability. We have developed an android application that will allow the user to capture or browse the image and extract the text from the picture by calling the firebase model and saving text in the file. To store the text file user can browse for the appropriate location. The proposed model works on both printed and handwritten text.
2021 5th International Conference on Computing Methodologies and Communication (ICCMC), 2021
Image captioning is a fascinating and challenging task combining two interesting fields in comput... more Image captioning is a fascinating and challenging task combining two interesting fields in computer science - Natural Language Processing and Computer Vision. Several approaches have been tried for this task. Many of these approaches are based on a combination of Convolutional Neural Networks(CNN’s) as encoders and Recurrent Neural Networks(RNN’s) as decoders. This paper compares two CNN-based encoders – (Vector Geometry Group) VGG16 and InceptionV3, which are used as encoders for encoding the input image features. The proposed research work uses the popular Flickr8k dataset and also compares among two different loss functions used with each of the above architectures- Categorical Cross Entropy loss function and Kullback-Leibler Divergence. Our main goal is to study the effect of the image encoder and the loss function on the image captioning task while keeping all other parameters the same.
2016 11th International Conference on Industrial and Information Systems (ICIIS), 2016
An Encoder-Decoder Neural Network Architecture is combined with a novel strategy to improve globa... more An Encoder-Decoder Neural Network Architecture is combined with a novel strategy to improve global label consistency, to come with an improved image segmentation model. Label Distribution predictions extracted from the SegNet Network is investigated and used in the project for image labeling. An algorithm called Region Consistency Activation (RCA) to improve the global label consistency is implemented. RCA is based on a novel transformation between Ultra metric Contour Map (UCM) and the Probability of Regions Consistency (PRC). These algorithms were rigorously tested on the CamVid dataset. Superior performances were achieved compared with the state-of-the-art methods on this dataset.
The massive influx of text, images, and videos to the internet has recently increased the challen... more The massive influx of text, images, and videos to the internet has recently increased the challenge of computer vision-based tasks in big data. Integrating visual data with natural language to generate video explanations has been a challenge for decades. However, recent experiments on image/video captioning that employ Long-Short-Term-Memory (LSTM) have piqued the interest of researchers studying its possible application in video captioning. The proposed video captioning architecture combines the bidirectional multilayer LSTM (BiLSTM) encoder and unidirectional decoder. The innovative architecture also considers temporal relations when creating superior global video representations. In contrast to the majority of prior work, the most relevant features of a video are selected and utilized specifically for captioning purposes. Existing methods utilize a single-layer attention mechanism for linking visual input with phrase meaning. This approach employs LSTMs and a multilayer attention...
The massive addition of data to the internet in text, images, and videos made computer vision-bas... more The massive addition of data to the internet in text, images, and videos made computer vision-based tasks challenging in the big data domain. Recent exploration of video data and progress in visual information captioning has been an arduous task in computer vision. Visual captioning is attributable to integrating visual information with natural language descriptions. This paper proposes an encoder-decoder framework with a 2D-Convolutional Neural Network (CNN) model and layered Long Short Term Memory (LSTM) as the encoder and an LSTM model integrated with an attention mechanism working as the decoder with a hybrid loss function. Visual feature vectors extracted from the video frames using a 2D-CNN model capture spatial features. Specifically, the visual feature vectors are fed into the layered LSTM to capture the temporal information. The attention mechanism enables the decoder to perceive and focus on relevant objects and correlate the visual context and language content for produci...
2022 IEEE Fourth International Conference on Advances in Electronics, Computers and Communications (ICAECC), 2022
RAFT is a deep network architecture for the detection of optical flow in the images. The RAFT mod... more RAFT is a deep network architecture for the detection of optical flow in the images. The RAFT model relates the per pixel motion between images even for minor changes in the position of the objects. It also updates the flow of field through recurrent units that perform lookups on the performance of the model. RAFT also works well with different datatypes and also it has better efficiency, training speed and count of parameters. Experiments were performed by using different parameters and also by changing certain values in the model itself. One cycle learning was also used to find the best parameters for the model. We also found that the RAFT model performs better than most of the other existing models for optical flow calculation in to images.
Communications in Computer and Information Science, 2021
Video Captioning is the task of describing the content of a video in simple natural language. Enc... more Video Captioning is the task of describing the content of a video in simple natural language. Encoder-Decoder architecture is the most widely used architecture for this task. Recent works exploit the use of 3D Convolutional Neural Networks (CNNs), Transformers or by changing the structure of basic Long Short-Term Memory (LSTM) units used in Encoder-Decoder to improve the performance. In this paper, we propose the use of a sentence vector to improve the performance of the Encoder-Decoder model. This sentence vector acts as an intermediary between the video space and the text space. Thus, it is referred to as semantic cross embedding that bridges the two vector spaces, in this paper. The sentence vector is generated from the video and is used by the Decoder, along with previously generated words to generate a suitable description. We also employ the use of a skip-connection in the Encoder part of the model. Skip-connection is usually employed to tackle the vanishing gradients problem in deep neural networks. However, our experiments show that a two-layer LSTM with a skip-connection performs better than the Bidirectional LSTM, for our model. Also, the use of a sentence vector improves performance considerably. All our experiments are performed on the MSVD dataset.
2021 2nd International Conference on Secure Cyber Computing and Communications (ICSCCC), 2021
Semantic image segmentation is the essential task of computer vision. It requires dividing visual... more Semantic image segmentation is the essential task of computer vision. It requires dividing visual input into different meaningful interpretable categories. In this work image attribution and segmentation approach is proposed. It can identify complex objects present in an image. The proposed model starts with superpixelization using Simple Linear Iterative Clustering (SLIC). A Multi Heat Map Slices Fusion model (MSF) produces an object seed heat map, and a Saliency Edge Colour Texture (SECT) model generates pixel-level annotations. Lastly, the PSPNet model for developing the final semantic segmentation of the object. The proposed model was implemented, and compared with the earlier work, it excelled the performance score.
2021 2nd International Conference on Smart Electronics and Communication (ICOSEC), 2021
Understanding covid-19 became very important since large scale vaccination of this was not possib... more Understanding covid-19 became very important since large scale vaccination of this was not possible. Chest X-ray is the first imaging technique that plays an important role in the diagnosis of COVID-19 disease. Till now in various fields, great success has been achieved using convolutional neural networks(CNNs) for image recognition and classification. However, due to the limited availability of annotated medical images, the classification of medical images remains the biggest challenge in medical diagnosis. The proposed research work has performed transfer learning using deep learning models like Resnet50 and VGG16 and compare their performance with a newly developed CNN based model. Resnet50 and VGG16 are state of the art models and have been used extensively. A comparative analysis with them will give us an idea of how good our model is. Also, this research work develops a CNN model as it is expected to perform really good on image classification related problems. The proposed research work has used kaggle radiography dataset for training, validating and testing. Moreover, this research work has used another x-ray images dataset which have been created from two different sources. The result shows that the CNN model developed by us outperforms VGG16 and Resnet50 model.
2021 5th International Conference on Computing Methodologies and Communication (ICCMC), 2021
Integration of visual content with natural language for generating images or video description ha... more Integration of visual content with natural language for generating images or video description has been a challenging task for many years. Recent research in image captioning using Long Short term memory (LSTM) recently has motivated its possible application in video captioning where a video is converted into an array of frames, or images, and this array along with the captions for the video are used to train the LSTM network to associate the video with sentences. However very little is known about using fine tuning techniques such as batch normalization or Stacked LSTMs models in video captioning and how it affects the performance of the model.For this project, we want to compare the performance of the base model described in [1] with batch normalization and stacked LSTMs with base model as our reference.
2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT), 2021
Automatically generating descriptions for an image has been one of the trending topics in the fie... more Automatically generating descriptions for an image has been one of the trending topics in the field of Computer Vision. This is due to the fact that various real-life applications like self-driving cars, Google image search, etc. are dependent on it. The backbone of this work is the encoder-decoder architecture of deep learning. The basic image captioning model has CNN as an encoder and RNN as a decoder. Various deep CNNs like VGG-16 and VGG-19, ResNet, Inception have been explored but despite the comparatively better performance, Xception is not that familiar in this field. Again for the decoder, GRU is not been used much, despite being comparatively faster than LSTM. Keeping these things in mind, and being attracted by the accuracy of Xception and efficiency of GRU, we propose an architecture for image captioning task with Xception as encoder and GRU as decoder with an attention mechanism.
2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT), 2021
We propose to model complex visual scenes using a non-parametric Bayesian model learned from weak... more We propose to model complex visual scenes using a non-parametric Bayesian model learned from weakly labelled images abundant on media sharing sites such as Flickr. Given weak image-level annotations of objects and attributes without locations or associations between them, our model aims to learn the appearance of object and attribute classes as well as their association on each object instance. Once learned, given an image, our model can be deployed to tackle a number of vision problems in a joint and coherent manner, including recognising objects in the scene (automatic object annotation), describing objects using their attributes (attribute prediction and association), and localising and delineating the objects (object detection and semantic segmentation). This is achieved by developing a novel Weakly Supervised Markov Random Field Stacked Indian Buffet Process (WS-MRF-SIBP) that models objects and attributes as latent factors and explicitly captures their correlations within and across superpixels. Extensive experiments on benchmark datasets demonstrate that our weakly supervised model significantly outperforms weakly supervised alternatives and is often comparable with existing strongly supervised models on a variety of tasks including semantic segmentation, automatic image annotation and retrieval based on object-attribute associations.
2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT), 2021
Video Captioning task involves generating descriptive text for the events and objects in the vide... more Video Captioning task involves generating descriptive text for the events and objects in the videos. It mainly involves taking a video, which is nothing but a sequence of frames, as data from the user and giving a single or multiple sentences (sequence of words) to the user. A lot of research has been done in the area of video captioning. Most of this work is based on using Long Short Term Memory (LSTM) units for avoiding the vanishing gradients problem. In this work, we purpose to implement a video captioning model using Gated Recurrent Units(GRU's), attention mechanism and word embeddings and compare the functionalities and results with traditional models that use LSTM's or Recurrent Neural Networks(RNN's). We train and test our model on the standard MSVD (Microsoft Research Video Description Corpus) dataset. We use a wide range of performance metrics like BLEU score, METEOR score, ROUGE-1, ROUGE-2 and ROUGE-L to evaluate the performance.
2021 6th International Conference for Convergence in Technology (I2CT), 2021
In modern days, extensive amount of data is produced from videos, because most of the populations... more In modern days, extensive amount of data is produced from videos, because most of the populations have video capturing devices such as mobile phone, camera, etc. The video comprises of photographic data, textual data, and auditory data. Our aim is to investigate and recognize the visual feature of the video and to generate the caption so that users can get the information of the video in an instant of time. Many technologies capture static content of the frame but for video captioning, dynamic information is more important compared to static information. In this work, we introduced an Encoder-Decoder architecture using Deep-Long Short-Term Memory (Deep-LSTM) and Bahdanau Attention. In the encoder, Convolution Neural Network (CNN) VGG16 and Deep-LSTM are used for deducing information from frames and Deep-LSTM combined with attention mechanism for describing action performed in the video. We evaluated the performance of our model on MSVD dataset, which shows significant improvement as compared to the other video captioning model.
2021 6th International Conference for Convergence in Technology (I2CT), 2021
Emotion detection is the process of detecting a human being's emotions based on various facia... more Emotion detection is the process of detecting a human being's emotions based on various facial cues and visual information. This field has gained much traction since the popularity of deep learning. Emotion detection has also given rise to many applications that had not been thought of before. One of the areas that are heavily associated with emotions is music. Music can invoke particular emotions of the listener, and a person feeling a certain emotion would look for a similar song. We use our emotion detection model to associate these emotions with a music player that plays music that accompanies user experiences. The model we designed includes two convolutional neural networks (CNN) models: a five-layer model and a global average pooling (GAP) model. We combined these CNN models with transfer-learning models. For our transfer-learning models, we used three pre-trained models: ResNet50; SeNet50; VGG16. Our results are comparable with the state-of-the-art models; however, our models are more efficient in performance.
2014 International Conference on Control, Instrumentation, Communication and Computational Technologies (ICCICCT), 2014
Mobile ad-hoc networks are more vulnerable to security attacks due to their unique characteristic... more Mobile ad-hoc networks are more vulnerable to security attacks due to their unique characteristics such as dynamic topology, no fixed infrastructure, resource limitations and multi-hop scenario. In MANETs one the dangerous attack is packet dropping attack. This packet dropping attack is two types: 1. Black hole and 2. Gray hole. In both the attacks, an attacker sends the false reply to source node that it is having the shortest route to the destination. In black hole attack, attackers drops all the packets received from source node and in gray hole attack, attacker drops packets selectively or forwards packets selectively or forwarding packets but not data packets. In this paper, we have proposed mechanism to detect and isolate these attacks in network.
2014 International Conference on Parallel, Distributed and Grid Computing, 2014
ABSTRACT This paper presents a different approach for parallelizing the Doolittle Algorithm with ... more ABSTRACT This paper presents a different approach for parallelizing the Doolittle Algorithm with the help of Intel Threading Building Blocks (TBB) allowing the users to utilize the power of multiple cores present in the modern CPUs. Parallel Doolittle Algorithm (PDA) has been divided into 3 parts: Decomposing the data, Parallely processing the data, finally Composing the data. Using the PDA we can solve the linear system of equations in considerably lesser amount time as compare to Serial Doolittle Algorithm (SDA). The PDA has been implemented in C++ using TBB library which makes it highly efficient, cross-platform compatible, and scalable. The efficiency of PDA over SDA has been verified by comparing the running time on different order of matrices. Experiments proved that PDA outperformed SDA by utilizing all the cores present in the CPU.
2022 IEEE Fourth International Conference on Advances in Electronics, Computers and Communications (ICAECC), 2022
Nowadays, Storing information from handwritten documents for future use is becoming necessary. An... more Nowadays, Storing information from handwritten documents for future use is becoming necessary. An easy way to store information is to capture handwritten documents and save them in image format. Recognizing the text or characters present in the image is called Optical Character Recognition. Text extraction from the image in the recent research is challenging due to stroke variation, inconsistent writing style, Cursive handwriting, etc. We have proposed CNN and BiLSTM models for text recognition in this work. This model is evaluated on the IAM dataset and achieved 92% character recognition accuracy. This model is deployed to the Firebase as a custom model to increase usability. We have developed an android application that will allow the user to capture or browse the image and extract the text from the picture by calling the firebase model and saving text in the file. To store the text file user can browse for the appropriate location. The proposed model works on both printed and handwritten text.
2021 5th International Conference on Computing Methodologies and Communication (ICCMC), 2021
Image captioning is a fascinating and challenging task combining two interesting fields in comput... more Image captioning is a fascinating and challenging task combining two interesting fields in computer science - Natural Language Processing and Computer Vision. Several approaches have been tried for this task. Many of these approaches are based on a combination of Convolutional Neural Networks(CNN’s) as encoders and Recurrent Neural Networks(RNN’s) as decoders. This paper compares two CNN-based encoders – (Vector Geometry Group) VGG16 and InceptionV3, which are used as encoders for encoding the input image features. The proposed research work uses the popular Flickr8k dataset and also compares among two different loss functions used with each of the above architectures- Categorical Cross Entropy loss function and Kullback-Leibler Divergence. Our main goal is to study the effect of the image encoder and the loss function on the image captioning task while keeping all other parameters the same.
2016 11th International Conference on Industrial and Information Systems (ICIIS), 2016
An Encoder-Decoder Neural Network Architecture is combined with a novel strategy to improve globa... more An Encoder-Decoder Neural Network Architecture is combined with a novel strategy to improve global label consistency, to come with an improved image segmentation model. Label Distribution predictions extracted from the SegNet Network is investigated and used in the project for image labeling. An algorithm called Region Consistency Activation (RCA) to improve the global label consistency is implemented. RCA is based on a novel transformation between Ultra metric Contour Map (UCM) and the Probability of Regions Consistency (PRC). These algorithms were rigorously tested on the CamVid dataset. Superior performances were achieved compared with the state-of-the-art methods on this dataset.
The massive influx of text, images, and videos to the internet has recently increased the challen... more The massive influx of text, images, and videos to the internet has recently increased the challenge of computer vision-based tasks in big data. Integrating visual data with natural language to generate video explanations has been a challenge for decades. However, recent experiments on image/video captioning that employ Long-Short-Term-Memory (LSTM) have piqued the interest of researchers studying its possible application in video captioning. The proposed video captioning architecture combines the bidirectional multilayer LSTM (BiLSTM) encoder and unidirectional decoder. The innovative architecture also considers temporal relations when creating superior global video representations. In contrast to the majority of prior work, the most relevant features of a video are selected and utilized specifically for captioning purposes. Existing methods utilize a single-layer attention mechanism for linking visual input with phrase meaning. This approach employs LSTMs and a multilayer attention...
The massive addition of data to the internet in text, images, and videos made computer vision-bas... more The massive addition of data to the internet in text, images, and videos made computer vision-based tasks challenging in the big data domain. Recent exploration of video data and progress in visual information captioning has been an arduous task in computer vision. Visual captioning is attributable to integrating visual information with natural language descriptions. This paper proposes an encoder-decoder framework with a 2D-Convolutional Neural Network (CNN) model and layered Long Short Term Memory (LSTM) as the encoder and an LSTM model integrated with an attention mechanism working as the decoder with a hybrid loss function. Visual feature vectors extracted from the video frames using a 2D-CNN model capture spatial features. Specifically, the visual feature vectors are fed into the layered LSTM to capture the temporal information. The attention mechanism enables the decoder to perceive and focus on relevant objects and correlate the visual context and language content for produci...
2022 IEEE Fourth International Conference on Advances in Electronics, Computers and Communications (ICAECC), 2022
RAFT is a deep network architecture for the detection of optical flow in the images. The RAFT mod... more RAFT is a deep network architecture for the detection of optical flow in the images. The RAFT model relates the per pixel motion between images even for minor changes in the position of the objects. It also updates the flow of field through recurrent units that perform lookups on the performance of the model. RAFT also works well with different datatypes and also it has better efficiency, training speed and count of parameters. Experiments were performed by using different parameters and also by changing certain values in the model itself. One cycle learning was also used to find the best parameters for the model. We also found that the RAFT model performs better than most of the other existing models for optical flow calculation in to images.
Communications in Computer and Information Science, 2021
Video Captioning is the task of describing the content of a video in simple natural language. Enc... more Video Captioning is the task of describing the content of a video in simple natural language. Encoder-Decoder architecture is the most widely used architecture for this task. Recent works exploit the use of 3D Convolutional Neural Networks (CNNs), Transformers or by changing the structure of basic Long Short-Term Memory (LSTM) units used in Encoder-Decoder to improve the performance. In this paper, we propose the use of a sentence vector to improve the performance of the Encoder-Decoder model. This sentence vector acts as an intermediary between the video space and the text space. Thus, it is referred to as semantic cross embedding that bridges the two vector spaces, in this paper. The sentence vector is generated from the video and is used by the Decoder, along with previously generated words to generate a suitable description. We also employ the use of a skip-connection in the Encoder part of the model. Skip-connection is usually employed to tackle the vanishing gradients problem in deep neural networks. However, our experiments show that a two-layer LSTM with a skip-connection performs better than the Bidirectional LSTM, for our model. Also, the use of a sentence vector improves performance considerably. All our experiments are performed on the MSVD dataset.
2021 2nd International Conference on Secure Cyber Computing and Communications (ICSCCC), 2021
Semantic image segmentation is the essential task of computer vision. It requires dividing visual... more Semantic image segmentation is the essential task of computer vision. It requires dividing visual input into different meaningful interpretable categories. In this work image attribution and segmentation approach is proposed. It can identify complex objects present in an image. The proposed model starts with superpixelization using Simple Linear Iterative Clustering (SLIC). A Multi Heat Map Slices Fusion model (MSF) produces an object seed heat map, and a Saliency Edge Colour Texture (SECT) model generates pixel-level annotations. Lastly, the PSPNet model for developing the final semantic segmentation of the object. The proposed model was implemented, and compared with the earlier work, it excelled the performance score.
2021 2nd International Conference on Smart Electronics and Communication (ICOSEC), 2021
Understanding covid-19 became very important since large scale vaccination of this was not possib... more Understanding covid-19 became very important since large scale vaccination of this was not possible. Chest X-ray is the first imaging technique that plays an important role in the diagnosis of COVID-19 disease. Till now in various fields, great success has been achieved using convolutional neural networks(CNNs) for image recognition and classification. However, due to the limited availability of annotated medical images, the classification of medical images remains the biggest challenge in medical diagnosis. The proposed research work has performed transfer learning using deep learning models like Resnet50 and VGG16 and compare their performance with a newly developed CNN based model. Resnet50 and VGG16 are state of the art models and have been used extensively. A comparative analysis with them will give us an idea of how good our model is. Also, this research work develops a CNN model as it is expected to perform really good on image classification related problems. The proposed research work has used kaggle radiography dataset for training, validating and testing. Moreover, this research work has used another x-ray images dataset which have been created from two different sources. The result shows that the CNN model developed by us outperforms VGG16 and Resnet50 model.
2021 5th International Conference on Computing Methodologies and Communication (ICCMC), 2021
Integration of visual content with natural language for generating images or video description ha... more Integration of visual content with natural language for generating images or video description has been a challenging task for many years. Recent research in image captioning using Long Short term memory (LSTM) recently has motivated its possible application in video captioning where a video is converted into an array of frames, or images, and this array along with the captions for the video are used to train the LSTM network to associate the video with sentences. However very little is known about using fine tuning techniques such as batch normalization or Stacked LSTMs models in video captioning and how it affects the performance of the model.For this project, we want to compare the performance of the base model described in [1] with batch normalization and stacked LSTMs with base model as our reference.
2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT), 2021
Automatically generating descriptions for an image has been one of the trending topics in the fie... more Automatically generating descriptions for an image has been one of the trending topics in the field of Computer Vision. This is due to the fact that various real-life applications like self-driving cars, Google image search, etc. are dependent on it. The backbone of this work is the encoder-decoder architecture of deep learning. The basic image captioning model has CNN as an encoder and RNN as a decoder. Various deep CNNs like VGG-16 and VGG-19, ResNet, Inception have been explored but despite the comparatively better performance, Xception is not that familiar in this field. Again for the decoder, GRU is not been used much, despite being comparatively faster than LSTM. Keeping these things in mind, and being attracted by the accuracy of Xception and efficiency of GRU, we propose an architecture for image captioning task with Xception as encoder and GRU as decoder with an attention mechanism.
2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT), 2021
We propose to model complex visual scenes using a non-parametric Bayesian model learned from weak... more We propose to model complex visual scenes using a non-parametric Bayesian model learned from weakly labelled images abundant on media sharing sites such as Flickr. Given weak image-level annotations of objects and attributes without locations or associations between them, our model aims to learn the appearance of object and attribute classes as well as their association on each object instance. Once learned, given an image, our model can be deployed to tackle a number of vision problems in a joint and coherent manner, including recognising objects in the scene (automatic object annotation), describing objects using their attributes (attribute prediction and association), and localising and delineating the objects (object detection and semantic segmentation). This is achieved by developing a novel Weakly Supervised Markov Random Field Stacked Indian Buffet Process (WS-MRF-SIBP) that models objects and attributes as latent factors and explicitly captures their correlations within and across superpixels. Extensive experiments on benchmark datasets demonstrate that our weakly supervised model significantly outperforms weakly supervised alternatives and is often comparable with existing strongly supervised models on a variety of tasks including semantic segmentation, automatic image annotation and retrieval based on object-attribute associations.
2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT), 2021
Video Captioning task involves generating descriptive text for the events and objects in the vide... more Video Captioning task involves generating descriptive text for the events and objects in the videos. It mainly involves taking a video, which is nothing but a sequence of frames, as data from the user and giving a single or multiple sentences (sequence of words) to the user. A lot of research has been done in the area of video captioning. Most of this work is based on using Long Short Term Memory (LSTM) units for avoiding the vanishing gradients problem. In this work, we purpose to implement a video captioning model using Gated Recurrent Units(GRU's), attention mechanism and word embeddings and compare the functionalities and results with traditional models that use LSTM's or Recurrent Neural Networks(RNN's). We train and test our model on the standard MSVD (Microsoft Research Video Description Corpus) dataset. We use a wide range of performance metrics like BLEU score, METEOR score, ROUGE-1, ROUGE-2 and ROUGE-L to evaluate the performance.
2021 6th International Conference for Convergence in Technology (I2CT), 2021
In modern days, extensive amount of data is produced from videos, because most of the populations... more In modern days, extensive amount of data is produced from videos, because most of the populations have video capturing devices such as mobile phone, camera, etc. The video comprises of photographic data, textual data, and auditory data. Our aim is to investigate and recognize the visual feature of the video and to generate the caption so that users can get the information of the video in an instant of time. Many technologies capture static content of the frame but for video captioning, dynamic information is more important compared to static information. In this work, we introduced an Encoder-Decoder architecture using Deep-Long Short-Term Memory (Deep-LSTM) and Bahdanau Attention. In the encoder, Convolution Neural Network (CNN) VGG16 and Deep-LSTM are used for deducing information from frames and Deep-LSTM combined with attention mechanism for describing action performed in the video. We evaluated the performance of our model on MSVD dataset, which shows significant improvement as compared to the other video captioning model.
2021 6th International Conference for Convergence in Technology (I2CT), 2021
Emotion detection is the process of detecting a human being's emotions based on various facia... more Emotion detection is the process of detecting a human being's emotions based on various facial cues and visual information. This field has gained much traction since the popularity of deep learning. Emotion detection has also given rise to many applications that had not been thought of before. One of the areas that are heavily associated with emotions is music. Music can invoke particular emotions of the listener, and a person feeling a certain emotion would look for a similar song. We use our emotion detection model to associate these emotions with a music player that plays music that accompanies user experiences. The model we designed includes two convolutional neural networks (CNN) models: a five-layer model and a global average pooling (GAP) model. We combined these CNN models with transfer-learning models. For our transfer-learning models, we used three pre-trained models: ResNet50; SeNet50; VGG16. Our results are comparable with the state-of-the-art models; however, our models are more efficient in performance.
2014 International Conference on Control, Instrumentation, Communication and Computational Technologies (ICCICCT), 2014
Mobile ad-hoc networks are more vulnerable to security attacks due to their unique characteristic... more Mobile ad-hoc networks are more vulnerable to security attacks due to their unique characteristics such as dynamic topology, no fixed infrastructure, resource limitations and multi-hop scenario. In MANETs one the dangerous attack is packet dropping attack. This packet dropping attack is two types: 1. Black hole and 2. Gray hole. In both the attacks, an attacker sends the false reply to source node that it is having the shortest route to the destination. In black hole attack, attackers drops all the packets received from source node and in gray hole attack, attacker drops packets selectively or forwards packets selectively or forwarding packets but not data packets. In this paper, we have proposed mechanism to detect and isolate these attacks in network.
2014 International Conference on Parallel, Distributed and Grid Computing, 2014
ABSTRACT This paper presents a different approach for parallelizing the Doolittle Algorithm with ... more ABSTRACT This paper presents a different approach for parallelizing the Doolittle Algorithm with the help of Intel Threading Building Blocks (TBB) allowing the users to utilize the power of multiple cores present in the modern CPUs. Parallel Doolittle Algorithm (PDA) has been divided into 3 parts: Decomposing the data, Parallely processing the data, finally Composing the data. Using the PDA we can solve the linear system of equations in considerably lesser amount time as compare to Serial Doolittle Algorithm (SDA). The PDA has been implemented in C++ using TBB library which makes it highly efficient, cross-platform compatible, and scalable. The efficiency of PDA over SDA has been verified by comparing the running time on different order of matrices. Experiments proved that PDA outperformed SDA by utilizing all the cores present in the CPU.
Uploads
Papers by Dinesh Naik