Lecture Slides for
ARTIFICIAL INTELLIGENCE
AND MACHİNE LEARNİNG
MD ABDUL ALIM SHEIKH
Aliah University
Dept. of Electronics and Communication Engineering
[Link]@[Link]
CHAPTER 1:
INTRODUCTİON
Topics of this lecture
3
A brief review of AI history
What is artificial intelligence?
Related research fields
Scope of this course
Early work (Around 1900 ）
4
Early work(1930 ～）
5
The 1stwave of AI (1950 ～ )
6
The 2ndwave of AI (1980 ～ )
7
The 3rdwave of AI (2000 ～ )
8
Representative technologies
Internet: Tim Berners-Lee, WWW inventor, 1989
Internet of things: Kevin Ashton, MIT Auto-ID Center,
1999
Cloud computing
Main frame (1950s)
Virtual machine (1970s)
Cloud (1990s)
Big data: John R. Masey, SGI, 1998
Deep learning: Geoffrey Hinton, UoT, 2006
A brief summary
9
Current status of AI
10
In March 2016, Alpha-Go of DeepMind defeated
Lee Sedol, who was the strongest human GO
player at that time.
This is a big news that may have profound meaning
in the human history.
Cycle for AI, 2020
11
After all, what is intelligence?
12
Intelligence is an umbrella term used to describe a property of
the mind that encompasses many related abilities, such as the
capacities
to reason,
to plan,
to solve problems,
to think abstractly,
to comprehend ideas,
to use language, and
to learn.
[Link]: capacity for learning, reasoning, understanding, and similar
(from Wikipedia)
forms of mental activity
What is intelligence?
13
[Link]: capacity for learning, reasoning,
understanding, and similar forms of mental activity
Ability to perceive and act in the world
Reasoning: proving theorems, medical diagnosis
Planning: take decisions
Learning and Adaptation: recommend movies,
learn traffic patterns
Understanding: text, speech, visual scene
Artificial Intelligence
14
Subject matters in AI
Get machines to do what humans do but
machines can’t
Unlike any classical subject, the frontier of
what is AI is not static.
Intelligence can be defined as the ability for
solving problems
15
Problem solving is to find the “best” solution in the
problem space.
Reasoning is to interpret or justify solutions or sub-
solutions.
Planning is to find ways for solving the problem.
Thinking abstractly is to simulate the problem solving
process inside the system (brain).
Idea/language comprehension is a way (or means) for
data/problem/knowledge representation;
Learning is the process to find better ways for solving a
problem (or a class of problems).
What is AI ?
16
Textbooks often define artificial intelligence as “the
study and design of computing systems that
perceives its environment and takes actions like
human beings”.
The term was introduced by John McCarthy in
1956 in the well-known Dartmouth Conference.
In my study, an AI is defined as a system that
possesses at least one (not necessarily all) of the
abilities mentioned in the previous page.
As a research area, AI studies theories and technologies for
obtaining systems that are partially or fully intelligent.
What is intelligence
17
Acting humanly: Turing Test
Turing (1950) "Computing machinery and
intelligence":
"Can machines think?"
Imitation Game
Acting humanly: Turing Test
18
What is AI?
19
four views:
Think like a Think rationally
human
Act like a human Act rationally
Search
20
Brute-force search
Depth-first search
Breadth-first search
Heuristic Search
Hill climbing search
Best-first search
A* Algorithm
Intelligent search
Genetic algorithms
Meta-heuristics
Three MAPs for knowledge acquisition
21
What is the input?
Map from real world to the mind model
What is the output?
Map from the mind model to the real world
What is the relation between the input and the output?
Abstracted knowledge about the real world
Representation methods
22
Representation of the problem
State space representation
Vector representation
Representation of knowledge
Production (decision) rules
Semantic network and ontology
Predicate logic
Fuzzy logic
Neural network (for tacit knowledge)
Learning models and algorithms
23
Neural network learning
Including MLP, SVM, deep learning, etc.
Evolutionary learning
GA or meta-heuristics in general
Reinforcement learning
Artificial immune system
Fuzzy logic
Decision tree
Hybrid system
Scope of this course
24
Search
Problem formulation and basic search algorithms
Expert system-based reasoning
Production system, semantic network, and frame
Logic based-reasoning
Propositional logic and predicate logic
Soft computing based reasoning
Fuzzy logic and multilayer neural network
Scope of this course
25
Machine learning
Pattern recognition
Self-organization
Neural networks
Decision trees
Intelligent search (if we have time)
Genetic algorithm
Ant colony optimization
Purpose of this course
26
Learn how to use the basic search methods;
Understand the basic methods for problem
formulation and knowledge representation;
Understand the basic idea of automatic reasoning;
Know some basic concepts related to pattern
recognition and machine learning.
Be able to learn more by yourself to
follow the newest trends!
AI History
27
Timeline : Prehistory / Early AI
Pre-history: Pascal, Leibniz
hoaxes
Babbage
1943 McCulloch & Pitts:
Boolean circuit model
of neuron
1950 Turing's "Computing
Machinery and Intelligence”
1956 Dartmouth meeting: von kempelen’s chess-playing turk, 1769 (hoax)
“Artificial Intelligence” name
1955: coining the name “Artificial
28
Intelligence”
John McCarthy, Marvin
Minsky, N Rochester,
and Claude Shannon:
(1955 ) :
“the conjecture that every
aspect of learning or
any other feature of
intelligence can in
principle be so
precisely described that
a machine can be made
to simulate it.”
Timeline : AI – Logical Models
29
1943 McCulloch & Pitts: Boolean circuit
model of brain
1950 Turing's "Computing Machinery and
Intelligence"
1956 Dartmouth meeting: "Artificial
Intelligence" name
1956 Newell & Simon's Logic Theorist,
1959 Samuel's checkers program: learned by
playing itself
30
31
32
33
Rosenblatt-Perceptrons
34
35
36
37
38
39
AI-Learning
40
Big Data
41
Widespread use of personal computers and wireless
communication leads to “big data”
We are both producers and consumers of data
Data is not random, it has structure, e.g., customer
behavior
We need “big theory” to extract that structure from
data for
(a) Understanding the process
(b) Making predictions for the future
Why “Learn” ?
42
Machine learning is programming computers to
optimize a performance criterion using example data
or past experience.
There is no need to “learn” to calculate payroll
Learning is used when:
Human expertise does not exist (navigating on Mars),
Humans are unable to explain their expertise (speech
recognition)
Solution changes in time (routing on a computer network)
Solution needs to be adapted to particular cases (user
biometrics)
What We Talk About When We Talk About
“Learning”
43
Learning general models from a data of particular
examples
Data is cheap and abundant (data warehouses, data
marts); knowledge is expensive and scarce.
Example in retail: Customer transactions to
consumer behavior:
People who bought “Blink” also bought “Outliers”
([Link])
Build a model that is a good and useful
approximation to the data.
Data Mining
44
Retail: Market basket analysis, Customer
relationship management (CRM)
Finance: Credit scoring, fraud detection
Manufacturing: Control, robotics, troubleshooting
Medicine: Medical diagnosis
Telecommunications: Spam filters, intrusion
detection
Bioinformatics: Motifs, alignment
Web mining: Search engines
...
What is Machine Learning?
45
Optimize a performance criterion using example
data or past experience.
Role of Statistics: Inference from a sample
Role of Computer science: Efficient algorithms to
Solve the optimization problem
Representing and evaluating the model for inference
Applications
46
Association
Supervised Learning
Classification
Regression
Unsupervised Learning
Reinforcement Learning
Learning Associations
47
Basket analysis:
P (Y | X ) probability that somebody who buys X
also buys Y where X and Y are products/services.
Example: P ( chips | drink ) = 0.7
Classification
48
Example: Credit
scoring
Differentiating
between low-risk and
high-risk customers
from their income and
savings
Discriminant: IF income > θ1 AND savings > θ2
THEN low-risk ELSE high-risk
Classification: Applications
49
Aka Pattern recognition
Face recognition: Pose, lighting, occlusion (glasses,
beard), make-up, hair style
Character recognition: Different handwriting styles.
Speech recognition: Temporal dependency.
Medical diagnosis: From symptoms to illnesses
Biometrics: Recognition/authentication using
physical and/or behavioral characteristics: Face,
iris, signature, etc
Outlier/novelty detection:
Face Recognition
50
Training examples of a person
Test images
ORL dataset,
AT&T Laboratories, Cambridge UK
Regression
Example: Price of a
used car
y = wx+w0
x : car attributes
y : price
y = g (x | q )
g ( ) model,
q parameters
51
Regression Applications
52
Navigating a car: Angle of the steering
Kinematics of a robot arm
(x,y) α1= g1(x,y)
α2= g2(x,y)
α2
α1
Response surface design
Supervised Learning: Uses
53
Prediction of future cases: Use the rule to predict
the output for future inputs
Knowledge extraction: The rule is easy to
understand
Compression: The rule is simpler than the data it
explains
Outlier detection: Exceptions that are not covered
by the rule, e.g., fraud
Unsupervised Learning
54
Learning “what normally happens”
No output
Clustering: Grouping similar instances
Example applications
Customer segmentation in Customer Relationship
Management (CRM)
Image compression: Color quantization
Bioinformatics: Learning motifs
Reinforcement Learning
55
Learning a policy: A sequence of outputs
No supervised output but delayed reward
Credit assignment problem
Game playing
Robot in a maze
Multiple agents, partial observability, ...
Resources: Datasets
56
UCI Repository:
[Link]
Statlib: [Link]
Resources: Journals
57
Journal of Machine Learning Research [Link]
Machine Learning
Neural Computation
Neural Networks
IEEE Trans on Neural Networks and Learning Systems
IEEE Trans on Pattern Analysis and Machine
Intelligence
Journals on Statistics/Data Mining/Signal
Processing/Natural Language
Processing/Bioinformatics/...
Resources: Conferences
58
International Conference on Machine Learning (ICML)
European Conference on Machine Learning (ECML)
Neural Information Processing Systems (NIPS)
Uncertainty in Artificial Intelligence (UAI)
Computational Learning Theory (COLT)
International Conference on Artificial Neural Networks
(ICANN)
International Conference on AI & Statistics (AISTATS)
International Conference on Pattern Recognition (ICPR)
...
