Intelligent bot for HealthCare
Team Members:
1. Prakash Raj S (710716205030)
2. Praveen D (710716205031)
3. VishnuVarthan T(710716205058)
4. Ragul S (15IT041RO19)
Guided By : Mr.T.VijayaKumar AP/IT
TABLE OF CONTENTS:
2
Introduction
Existing System / Literature Survey
Problem Statement
Objectives
Proposed System
Result & Discussion
Conclusion
Future Enhancement
Reference
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
3 Introduction
Machine Learning Vs Traditional Programming
Traditional programming is a manual process—meaning
a person (programmer) creates the program. But without
anyone programming the logic, one has to manually
formulate or code rules.
If we're writing a game we can usually figure out the
rules. For example, if the ball hits the brick then the brick
should vanish and the ball should rebound.
But if the ball falls off the bottom of the screen then
maybe the player loses their life
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
Machine Learning
Machine learning rearranges this diagram
where we put answers and data in and then
we get rules out.
So instead of us as developers figuring out
the rules, take bunch of examples for what we
want to see and then have the computer figure
out the rule.
• So consider this example, activity
recognition.
• So consider this example, activity
recognition. If I'm building a device that
detects if somebody is say walking and I have
data about their speed,
• I might write code like this and if they're
running well that's a faster speed so I could
adapt my code to this and if they're
biking, well that's not too bad either. But then
I have to do golf recognition too, now my Machine Learning
concept becomes broken.
4 Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
So instead of trying to express the problem as rules when often that isn't even
5 possible.
The new paradigm is that I get lots and lots of examples and then I have labels on
those examples and I use the data to say this is what walking looks like,this is what
running looks like, this is what biking looks like and yes, even this is what golfing
looks like.
So, then it becomes answers and data in with rules being inferred by the machine. A
machine learning algorithm then figures out the specific patterns in each set of data
that determines the distinctiveness of each.
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
DeepLearning
Deep Learning is a subfield of
machine learning concerned with
algorithms inspired by the structure
and function of the brain called
artificial neural networks
Andrew Ng from Coursera and
Chief Scientist at Baidu Research
formally founded Google Brain
that eventually resulted in the
productization of deep learning
technologies across a large number
of Google services.
6 Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
7
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
Existing System / Literature Survey
8
Doc Product: Medical Q&A with Deep Language Models
Quality medical information is valuable to everyone, but it's not always
readily available. Doc Product aims to fix that.
Doc Product has you covered with up to date information and unique
AI-generated advice to address your medical concerns.
We wanted to use TensorFlow 2.0 to explore how well state-of-the-art
natural language processing models like BERT and GPT-2 could
respond to medical questions by retrieving and conditioning on relevant
medical data, and this is the result.
we drew inspiration for our design from various different areas of
machine learning. By combining the power of Transformer
architecture,latent vector search, negative sampling and generative pre-
training within TensorFlow 2.0's flexible deep learning framework,
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
700,000 medical questions and answers scraped from Reddit, Health
Tap, WebMD, and several other sites
9 Fine-tuned TF 2.0 BERT with pre-trainef BioBERT weights for
extracting representations from text
Network heads for mapping question and answer embeddings to
metric space, made with a Keras . Model feedforward network
Over a terabyte of CSV data
What it does
BERT has been trained to encode medical questions and medical
information. A user can type in a medical question, and our model
will retrieve the most relevant medical information to that question.
Data
They created datasets from several medical question and answering
forums. The forums are WebMD, HealthTap, eHealthForums,
iClinic, Question Doctors, and Reddit.com/r/AskDocs
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
10
Architecture
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
11 Problem Statement
Doctor Source is a biomedical app that lets you chat with an
automated general practice doctor and get a quick diagnosis and
treatment plan for whatever your ailment is.
The App is built for both iOS and Android using the Flutter
framework, and enables you to use your voice to communicate
with the AI doctor and Book appointment near by specialist
doctor.
It's trained on lots of Reddit, HealthTap, and WebMD medical
questions and answers, over 700K of them.
How does BioBERT work
How to build a Flutter plugin (iOS + Android)
How to chain together multiple machine learning models for a
robut result
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
12 Surveillance
(12 Sep 2019) Our paper has been accepted to Bioinformatics (
link). Please update your BibTex.
(19 June 2019) Check out our BioBERT based multi-type NER
and normalization model (BERN) with its paper!
(16 May 2019) A new version of BioBERT (v1.1) is now
available.
(15 May 2019) Updated code and dataset for BioASQ 7ed3d7a.
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
13 Objectives
Voice Recognition plugin (custom channel, iOS Speech API,
Android Speech Recognizer)
Building a Flutter app
Increases the accuracy of BERT Model
Integrating pretrained model into our flutter app
How to Improve the App
Named Entity Recognition
All Voice
More animations in the interface
Continous training pipeline, keep getting more data from users,
feed it back in
Robust edge case detection
Affiliate deals with drug companies
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
14 Proposed System
How does BioBERT work
First of all, when Google released BERT, it was like the ImageNet
moment in computer vision, but this time for natural language processing.
It was a big deal.
Now everyone can use the weights file that BERT learned after training on
millions of words as a starting point for their specific NLP task, be it
Q&A, sentence classfication, sentence generation, sentiment analysis, etc.
Published 5 months ago
It's a pre-trained biomedical language representation model. The full name
is "BiDirection Encoder Reprensentions from Transformers for
Biomedical Text Mining". -BioBERT outperforms plain old BERT on 3
tasks; Biomedical Named Entity Recognition, Biomedical Relation
Extraction, and Biomedical Question Answering
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
15
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
16
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
17
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
18 Software & Framework
IBM Watson Studio
Google Colabs
Flutter Plugin
Speech Recognition API
Visual Studio Code / Android Studio
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
How to build a Flutter plugin (iOS + Android)
19 Tools Used
App building (Flutter)
User Accounts (Firebase)
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
Voice Recognition plugin (custom channel, iOS Speech
20 API, Android Speech Recognizer)
https://pub.dev/packages/speech_recognition
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
21 Installation
Depend on it Add this to your package's pubspec.yaml file:
dependencies:
speech_recognition: "^0.3.0"
Install it You can install packages from the command line:
$ flutter packages get
Import it Now in your Dart code, you can use:
import 'package:speech_recognition/speech_recognition.dart';
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
22
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
23 Result & Discussion
https://colab.research.google.com/drive/1xZBvVG-
pQDy8HsEIqDyEKzLKWST4gAYy#scrollTo=Ul_tuaqe0u98
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
24 Conclusion
AI Model(BioBERT , input question into retrained bioBERT to
convert into an embedding. Input the embedding into a fully
connected neural network, output an embedding for similarity
lookup. Most similar Q&A's used by GPT-2 to generate an
answer. )
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
Future Enhancement
Develop a model , which monitor
ECG level , glucose bottle level and
patient behaviour using Computer
vision
Develop a model, which
automatically create and update
patient details and treatment for
future references using
Image recognition
25 Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
26 Reference
BASE PAPER
This repository provides fine-tuning codes of BioBERT, a
language representation model for biomedical domain:
https://github.com/dmis-lab/biobert
REFERENCE PAPER
Visualizing and Understanding Neural Models in NLP
Enriching BERT with knowledge Graph Embeddings for
Document Classification
Build it for Android and iOS specifically, then call it from
Dartviacreatingamessagechannel.https://medium.com/joytunes/de
ploying-a-tensorflow-model-to-android-69d04d1b0cba
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
