Scribd
Upload
40 views18 pages

Lecture 1 Course Introduction

This document provides an introduction to an advanced machine learning course. It outlines that basic machine learning is often inadequate for many applications, while advanced techniques usually provide better solutions. The course aims to characterize data inputs and outputs, identify problems and appropriate ML solutions, and critically review research papers. Topics will include neural network architectures, optimization techniques, training deep networks, attention mechanisms, probabilistic models, and more. Students will complete coding exercises and assignments, and be evaluated based on assignments, exams, and a final project. The document uses examples to illustrate challenges in advanced ML, such as handling large/high-dimensional data, transferring learning across domains, and utilizing unlabeled data.

Uploaded by

Utkarsh Bhalode
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
40 views18 pages

Lecture 1 Course Introduction

This document provides an introduction to an advanced machine learning course. It outlines that basic machine learning is often inadequate for many applications, while advanced techniques usually provide better solutions. The course aims to characterize data inputs and outputs, identify problems and appropriate ML solutions, and critically review research papers. Topics will include neural network architectures, optimization techniques, training deep networks, attention mechanisms, probabilistic models, and more. Students will complete coding exercises and assignments, and be evaluated based on assignments, exams, and a final project. The document uses examples to illustrate challenges in advanced ML, such as handling large/high-dimensional data, transferring learning across domains, and utilizing unlabeled data.

Uploaded by

Utkarsh Bhalode
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

Advanced Machine Learning

Course Introduction
Amit Sethi
Electrical Engineering, IIT Bombay
Why an advanced ML course

• Basic ML inadequate for many applications

• Advanced ML usually gives better solutions

• True innovations are in advanced ML


Learning outcomes for the course

• Characterize data input and desired output

• Identify problems and design ML solutions

• Read and critically compare research papers

• Code complex learning frameworks


How we will accomplish our goals

• Lectures for frameworks

• Coding exercises for implementation

• Reading assignments for concepts and


mathematics
Topics to be covered
• Revision of basic ML • Learning with less data
– Transfer
• NN architectures – semi-supervised
• Optimization – weakly supervised
techniques • Generative adversarial
networks
• Loss function
engineering • Robust NNs
• Training very deep NNs • Generative models
• Attention in NNs • Probabilistic models
• Insights into NNs • Training PGMs
Evaluation structure
• Assignment 1 – 15% • 90%  AA
• 80%  AB
• Assignment 2 – 15% • 70%  BB
• 60%  BC
• Exam 1 – 20% • 50%  CC
• 40%  CD
• Exam 2 – 30% • 30%  DD

• Project – 20% • 40%  Audit pass


Where would basic ML struggle?

Large number of dimensions on a grid

• What features
should we extract
from an image?
• Should pixels be
features?
• Is Euclidean
distance a good
metric?

Image source: Wikipedia


Where would basic ML struggle?

Learning from similar domains


• If we have lots of
labeled digits
• But only a few
labeled examples
of an ancient
script
• Can we transfer
learning?

Image source: Wikipedia


Where would basic ML struggle?

Scaling feature extraction with input

• More samples

• Higher
dimensional
samples

Image source: Wikipedia


Where would basic ML struggle?

Depth vs. width


y1 y2 … yn

… … …

h11 h12 … h1n1

x1 x2 … xd
How to optimize complex models

Image source: Unknown


How to regularize models

Image source: Wikipedia


Where would basic ML struggle?

Data dimensions form a graph


• What if variables are
related as graphs?

• What if different
variables are missing
in different samples?
Where would basic ML struggle?

How to use unlabeled data


Where would basic ML struggle?
What if labels are given to bags of
samples
Where would basic ML struggle?

How to generate more samples


Discriminative Generative
Questions and suggestions
Next lecture
• Revision of supervised machine learning

• Revision of neural networks

You might also like