Sumit Agarwal iamagarwalsumit@gmail.

com
Senior Software Engineer Github | LinkedIn | Website
Samsung Research India, Bangalore +91-7278334160

Education

Indian Institute of Technology, Kharagpur, India Jul 2013 – May 2017

Bachelor of Technology (Hons.) in Computer Science and Engineering
Cumulative GPA : 9.75/10.0
Department Rank : 3 out of 105 students
Thesis Advisor : Prof. Niloy Ganguly

Publications

Bhushan Kulkarni*, Sumit Agarwal*, Abir De*, Sourangshu Bhattacharya, Niloy Ganguly. “SLANT+: A
Nonlinear Model for Opinion Dynamics in Social Networks” IEEE International Conference on Data Mining
(ICDM). 2017 (Poster, pdf, code) (Journal version to be submitted in ACM Transactions on Social
Computing) (* - equal contrib.)
Ashish Sharma*, Jatin Arora*, Pritam Khan*, Sidhartha Satapathy*, Sumit Agarwal*, Satadal Sengupta,
Sankarshan Mridha, Niloy Ganguly. “CommBox: Utilizing sensors for real-time cricket shot identification and
commentary generation” IEEE International Conference on Communication Systems and Networks
(COMSNETS). 2017 (Workshop, pdf, code) (* - equal contrib.)
Somnath Basu Roy Chowdhury*, Biswarup Bhattacharya*, Sumit Agarwal*. “Location Optimization of
ATM Networks.” arXiv Preprint 1706.09243. 2017 (pdf) (* - equal contrib.)

Work Experience

Samsung Research India, Bangalore

Senior Software Engineer (Bixby, Voice Intelligence R&D) Jun 2017 - Present
Multi Device Voice Assistant Experience
• Working towards enabling consistent voice assistant experience for users across devices like speaker,
mobile, television, refrigerator.
• Through intelligent device selection help the user smoothly transition among devices. For eg, when the
user asks the speaker to “What’s the weather”, the speaker can not only speak out the weather but also
show the detailed view on the television.
Bixby Capsule Developer
• Developing voice enabled apps called capsule for Bixby, capable of teaching capabilities to perform a
variety of tasks. (https://bixbydevelopers.com)
• Currently working on clock, messages and settings capsule for mobile and speaker devices and achieved
an accuracy of over 95% consistently on production usage.
Dynamic conversation drivers
• Implemented a system to suggest dialogs to drive the conversation forward with the user. For example,
suggesting the user to “Book an Uber” when he asks his “Direction to work”.
• Implemented the system as a multi label classification problem which used the user’s utterance as input
to the sequence to sequence model where every step of the decoder was used to generate labels/next
intent of the user. These intents were further used to suggest dialogs.
• Training data being less, achieved an improvement in F1-score of 5% on using pre-trained BERT as
encoder and fine-tuning it with the training data.
Data augmentation for goal oriented dialog systems
• To achieve the goal of generating variant utterances under the same semantic frames, the input
utterance is converted into its delexicalised form with its slot values replaced by labels, and then
generating its delexicalised variances with a seq2seq model. Finally, surface realization is carried out to
convert the delexicalised form into the raw utterance.
• Used BERT for next step classification to remove false positives, by matching the intents of the output
with the input and achieved multiplication rate of 2, i.e., generating 2 utterances for every utterance in
the training set.
Low corpus based classification for voice assistants
• Used a three-step classification pipeline, domain classification, then intent classification followed by slot
labelling using BERT.
• Fine tuned the model with low corpus data from Bixby across 10 different domains and achieved an
accuracy of 95% on domain and intent classification, and 85% on slot tagging.
Identifying data clashes in Bixby
• To identify the classification clashes across Bixby capsules, extracted similar utterances trained across
capsules using tf-idf and cosine similarity.
• Listed the most important words in each capsule through tf-idf scores to help developers get a deeper
insight in their capsule’s training data, balancing and modifying the data wherever necessary.
On-device light weight NLU for voice assistants
• Wrote a fast-text based text classification for dialog systems with identification of domain, intent and
slot values which was light-weight meant to be run on devices like television, refrigerator, washing
machine.
• The model had an accuracy over 90% and performed better than the rule-based technique used before.
Internship Experience

Samsung Research India, Bangalore

Software Engineering Intern (Samsung Pay) May 2016 - Jun 2016
• Worked on sms mining and extracting information from transaction messages using basic classification
and NLP techniques.
• Worked with the server team to integrate the UPI(United Payments Interface) service, a real-time
system made by the Government of India, with the Samsung Pay wallet.
• Secured a full time opportunity for the role of Software Engineer from the company.

National Digital Library of India, IIT Kharagpur

Software Engineering Intern May 2015 - Jun 2015
Advisor : Prof. Sudeshna Sarkar
• Worked on building a search engine with multi-lingual support for the National Digital Library using
Apache Solr as the backbone.
• Configured a multilingual frontend having phonetic and keyboard input using Google API’s.
• Using stemmers enabled searching and retrieving documents across ten Indian languages.
• Worked on language detection that needed to be checked before applying the respective stemmer.
• The project was sponsored by the Ministry of Human Resources Department, Govt. of India.
Projects

Bachelor’s Thesis, Indian Institute of Technology, Kharagpur, India

Shaping Opinion Dynamics using Recurrent Neural Networks Spring 2017
Advisor : Prof. Niloy Ganguly [Project]
Predicting future opinions of users in a social network by considering the influence of its friends. Used
network-guided recurrent neural networks to model the time and frequency of the opinions of the users as
opinion dynamics has an inherent recursive structure depending on past opinions. Achieved an
improvement of 10% in the MSE of the predicted opinions over previous state-of-the-art models as the
non-linearities of RNNs captured the opinion exchange process concretely.

Indian Institute of Technology, Kharagpur, India

Abstract generation for Scientific Papers Autumn 2016
Advisor : Prof. Pawan Goyal [Project]
Worked on a text summarisation task for automatic abstract generation for scientific documents.
Converted the paper to paragraph embeddings using para2vec and trained a bidirectional sequence to
sequence LSTM network with attention to capture the long-term dependencies in scientific papers.
Achieved ROUGE-1 scores of 0.5. Although, results weren’t stunning, could show that LSTMs have the
potential to work for long documents but require more computational power.

Automatic commentary for Cricket Shots Autumn 2016

Advisor : Prof. Niloy Ganguly [Project]
 Aim to generate automatic commentary through detection of shots real-time via sensors attached on
cricket bats/cricketer’s wrist. Learning a SVM classifier from the data collected from accelerometer,
gyroscope sensors and achieved a k-fold cross validation of 0.9 over 300 data-points.
Detecting vehicular environments for autonomous driving Autumn 2016
Advisor : Prof. Pabitra Mitra [Project]
Worked on a system to help drivers or autonomous vehicles decide whether to turn left, right or go
straight by the images that it captures from its front camera. Due to the lack of labeled data, used frames
from the popular racing game Road Rash. Implemented the model by segmenting the road information
from the image through Blind Color Decomposition followed by an Alex-net based CNN classifier and
achieved at par results with state-of-the-art models.

Citation Recommendations for Citation contexts of Scientific Papers Spring 2016

Advisor : Prof. Pawan Goyal and Prof. Animesh Mukherjee [Project]
An information retrieval system to fetch papers which can be cited given a citation context as a query.
Compared the search results across various combinations of weights assigned to different parts of the
paper (title, abstract, citation context) and used Tf-Idf and BM-25 to retrieve the papers. Used query
expansion by expanding the query with similar words using word2vec. BM-25 with title given the highest
priority performed the best among all with a MRR (Mean Reciprocal Rate) of 0.2 over a test-set of 1000
documents.

Tracking buses in college campus using a mobile application Spring 2015

Advisor : Prof. Pabitra Mitra
Application to help students track buses plying in the campus and know the times of buses at various
bus-stops through the bus’ GPS information. Duplicated the bus with a mobile application as a
substitute for GPS device and made another device to track the location of these mobiles (or buses) to
simulate the project in a classroom environment.
Multi-lingual stemmers for Indian languages Summer 2015
Advisor : Prof Sudeshna Sarkar
Built rule-based and learning-based stemmers for 10 different Indian languages including Hindi, Bengali,
Tamil, Telugu, Oriya, Gujarati. These stemmers along with Tf-Idf & BM-25 were used to enable search in
these languages across a digital library of 1 lakh documents.

Tiny C : A compiler for a shorter version of C Autumn 2014

Advisor : Prof Partha Pratim Das [Project]
Built a compiler for a shorter version of C written in C++ capable of compiling codes like merge sort,
binary search. Used flex and bison for parsing the code and then translated to 3-address code. This was
then used for generating assembly level code for programs targeted at x86_64 architecture.

Awards and Achievements

Samsung Professional Certification 2019

Received Samsung Professional Certification which has been successfully completed by only 20% of employees
Promotion to Senior Software Engineer, Samsung Research India, Bangalore 2019
Promoted to Senior Software Engineer position in the Bixby team, Voice Intelligence R&D for consistent
performance
TechGlanz, annual fest of Voice Intelligence, Samsung Research India, Bangalore 2018-19
Participated and represented our team for two consecutive years at the annual fest and showcased our work
including Bixby in action and Multi Device Experience in a smart home.
Samsung Spot Award, Samsung Research India, Bangalore 2018
Received Samsung Spot Award in the 3rd quarter of 2018 given to high-performing employees in the company
Department rank of 3, Computer Science & Engineering, IIT Kharagpur 2017
Graduated with a Department Rank of 3 out of 105 students
Silver, Data Analytics, General Championships, IIT Kharagpur 2017
Stood 2nd in Data Analytics . Proposed a novel model for location optimisation of ATM networks
Best Academic Demo Award, COMSNETS’17 2017
Received the Best Academic Demo award for presenting “CommBox: Utilizing sensors for real-time cricket
shot identification and commentary generation” at COMSNETS ’17
Best Term Project for Smartphone Computing 2016
Marked Best Term Project for smartphone computing and applications course for the project on automatic
cricket commentary generation
Technology Alumni Association (TAA) Scholarship 2015
Among the 10 students who received TAA scholarship, out of 320 students, for academic excellence
4th, HP Think-A-Thon 2015
Came 4th out of 2000 participants in a nationwide coding contest organised by HP
Robotics Winter Workshop, Technology Robotic Society, IIT Kharagpur 2014
Attended a 15 day workshop for Robotics and successfully developed a line following robot
All-India-Rank 3350 - IIT JEE (Joint Entrance Examination) - Advanced 2013
Secured an All-India-Rank of 3350 in JEE Advanced 2013 amongst 150,000 candidates.
All-India-Rank 2466 - IIT JEE (Joint Entrance Examination) - Mains 2013
Secured an All-India-Rank of 2781 in JEE Mains 2013 amongst 800,000 candidates.

State-Rank 44 - West Bengal Joint Entrance Examination 2013

Secured a rank of 44 in the State Engineering Entrance examination out of 300,000 candidates
Sanmarg Hindi Prize 2013
Received prize for securing the highest marks in Hindi in the Higher Secondary Examination by Sanmarg, the
most prestigious Hindi newspaper of the state of West Bengal
6th, Kolkata District, Secondary Examination 2011
Received prize from the Chief Minister of the State for securing the 6th rank in the Kolkata district of West
Bengal in Secondary Examinations

Talks

MDE in a smart home using voice assistants Dec 2018

Discussed various use cases of voice assistants in a smart home environment and how through intelligent
device selection, these assistants can transition across devices giving a consistent experience to end users
BERT : The breakthrough in NLP Aug 2019
Discussed about the importance of BERT as a turning point in NLP and how it can be used in solving
complex NLP problems related to voice assistants

Technical Skills
Languages : C++, Python, Javascript, Java, HTML5, CSS, PHP, MySQL
Libraries : Tensorflow, NumPy
Tools & Frameworks : Bixby Studio, Spring, Android Studio, Django, Jenkins, ElectronJs, Apache Solr
Extra Curricular Activities and Social & Cultural Service

Internship Mentor at Samsung 2019

Mentored 2 undergraduate interns on two projects based on applications of deep learning in voice assistants
and helped them secure pre-placements offers from the company
Captain of the Dramatics & Entertainment Cup, General Championships 2016 - 17
Captain of the Dramatics Cup team of Lal Bahadur Shastri Hall of Residence, IIT Kharagpur. Successfully
directed a team of 30 and participated in Hindi Dramatics ending up in the 5th position. Successfully led a
team of 2 and participated in Hindi Elocution ending up in the 3rd position
Group Leader, Cultural Team, National Service Scheme, IIT Kharagpur 2013 - 15
As a group leader of the cultural team led a team of 60 people through active social and cultural activities in
villages in the outskirts of the college campus including tree plantation, street play, organising art competition
in village schools
Teacher, Sandip Physics Classes 2014
Taught mathematics to a batch of 20 students appearing for the IITJEE’15 examination