Scribd
Upload
167 views11 pages

Deep Learning for NLP Course

This document provides information about a course on deep learning for natural language processing. It will survey the use of deep learning techniques for applications including text classification, sequence transduction, question answering, and linguistic structure. The course will not provide a general introduction to NLP. It has prerequisites in math, machine learning, and programming. The course will cover topics like recurrent neural networks, text classification, sequence transduction, speech processing, machine translation, and linguistic structure through lectures and practical sessions.

Uploaded by

Hrittik Roy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
167 views11 pages

Deep Learning for NLP Course

This document provides information about a course on deep learning for natural language processing. It will survey the use of deep learning techniques for applications including text classification, sequence transduction, question answering, and linguistic structure. The course will not provide a general introduction to NLP. It has prerequisites in math, machine learning, and programming. The course will cover topics like recurrent neural networks, text classification, sequence transduction, speech processing, machine translation, and linguistic structure through lectures and practical sessions.

Uploaded by

Hrittik Roy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

Deep Learning for Natural Language Processing

Phil Blunsom
[email protected]
Why take this course?

Artificial Intelligence is one of the most interesting fields of


research today, and language is the most compelling manifestation
of intelligence.
Course Information

Website www.cs.ox.ac.uk/teaching/courses/2016-2017/dl
Textbooks No specific text,
• a good DL reference is:
Goodfellow, Bengio, and Courville,
Deep Learning. www.deeplearningbook.org
• for a general background in ML:
Machine Learning: A Probabilistic Perspective
Bishop, Pattern Recognition and Machine Learning
Lectures 4–6pm Tuesday and Thursday
No lectures in week 2!
Practicals 7 lab sessions, Weeks 2-8
Demonstrators: Brendan Shillingford, Yishu Miao, and
Yannis Assael
Assessment take home exam.
Provisional Lecture Schedule

Week 1 1. Introduction Phil Blunsom (Oxford and DM) and Wang Ling (DM)
2. Lexical Semantics Ed Grefenstette (DM)
Week 2 No lectures
Week 3 3&4. RNNs and Language Modelling Phil Blunsom

Week 4 5. Text Classification Karl Moritz Hermann (DM)


6. RNNs and GPUs Jeremy Appleyard (nvidia)
Week 5 7&8. Sequence Transduction Chris Dyer (CMU and DM)

Week 6 9&10. Speech Andrew Senior (DM)

Week 7 11. Question Answering Karl Moritz Hermann

12. Memory Ed Grefenstette


Week 8 13. Linguistic Structure Chris Dyer

14. Conclusion Phil Blunsom


Prerequisites

Maths • Linear Algebra,


• Calculus,
• Probability.
Machine Learning
• Evaluating ML models
(train/validation/test split, cross validation etc.),
• overfitting, generalisation, and regularisation,
• optimisation
(objective functions, stochastic gradient descent),
• linear regression and classification, neural networks
(common non-linearities, backpropagations etc.).
Programming
Knowledge of, or ability to learn quickly, a NN toolkit
(e.g. Torch, TensorFlow, Theano, DyNet etc.)
What this course is, and is not, about

This course will survey the use of Deep Learning techniques for a
range of Natural Language Processing applications.

This is not a general course on NLP. There is a lot more to


language and computational linguistics, and many interesting
paradigms beyond deep learning, than we will cover.
Language Understanding

CNN article:
Document The BBC producer allegedly struck by
Jeremy Clarkson will not press charges
against the “Top Gear” host, his lawyer
said Friday. Clarkson, who hosted one of
the most-watched television shows in the
world, was dropped by the BBC Wednesday
after an internal investigation by the
British broadcaster found he had subjected
producer Oisin Tymon “to an unprovoked
physical and verbal attack.” . . .
Query Who does the article say will not press
charges against Jeremy Clarkson?
Answer Oisin Tymon
Speech Processing and Machine Translation

Speech Recognition (ASR)

Les chiens aiment les os


Machine Translation (MT)

Dogs love bones


Text to Speech (TTS)
Image Understanding
How many slices of pizza are there?
de of? Is this a vegetarian pizza?

ompany? Does it appear to be rainy?


? DoesWhat
this person
is thehave
man20/20 vision?
holding?
of free-form, open-ended Does itquestions
appear to becollected
raining? for
on Mechanical Turk. Note that commonsense
Does this man have 20/20 vision?
ed along with a visual understanding of the scene
estions.
Agrawal et al., VQA: Visual Question Answering, ICCV 2015.
Linguistic Structure
Example

Dependency parsing

Sense

I saw her duck

Idioms
He kicked a goal
He kicked the ball
He caught the ball
He kicked the bucket
Reference
The ball did not fit in the box because it was too
[big/small].
etc.
Next lecture, Wang Ling:
Deep Neural Networks Are Our Friends

You might also like