SUMMER TERM 2025
INTRODUCTION TO THE LECTURE
FOUNDATIONS OF MACHINE LEARNING
Eyke Hüllermeier
Artificial Intelligence and Machine Learning
Institute of Informatics, LMU
[email protected]
L I T E R AT U R E
− Y.S. Abu-Mostafa, M. Magdon-Ismail, H.T. Lin. Learning from Data. AML Book, 2012.
− I. Goodfellow, Y. Begio, A. Courvill. Deep Learning. MIT Press, 2016.
− P. Flach. Machine Learning. Cambidge Univ. Press, 2012.
− E. Alpaydin. Machine Learning. Oldenbourg 2008.
− C.M. Bishop. Pattern Recognition and Machine Learning. Springer 2006.
− D.J. Hand, H. Mannila, P. Smyth. Principles of Data Mining. MIT Press 2000.
− T. Hastie, R. Tibshirani and J. Friedman. The Elements of Statistical Learning: Data Mining,
Inference, and Prediction. Springer-Verlag, 2001.
− T. Mitchell. Machine Learning. McGraw Hill, 1997.
− I.H. Witten, E. Frank. Data Mining: Practical Machine Learning Tools and
Techniques with Java Implementations. Morgan Kaufmann, 2000.
− E. Lehman, F.T. Leighton A.R. Meyer: Mathematics for Computer Science, 2017.
− M.P. Deisenroth, A.A. Faisal, C.S. Ong. Mathematics for Machine Learning. Cambridge University
Press, 2020. https://mml-book.com.
2 Intelligent Systems and Machine Learning
M I LE S T O N E S O F A I
ChatGPT (2022)
AlphaGo schlägt Lee Sedol (2016)
Watson gewinnt Jeopardy! (2011)
Deep Blue schlägt Garry Kasparov (1997)
3
COMPLEX PROBLEMS
G A M E P L AY I N G LANGUAGE MODELS
Input, e.g.,
context or state Output, e.g.,
describing the action or decision
environment
MALE
IM AG E R EC OG NI TIO N AUTONOMOUS CARS
4
COMPLEX PROBLEMS
G A M E P L AY I N G LANGUAGE MODELS
Complex problems, not amenable to classical algorithmic problem
solving, nor or traditional (knowledge-based) AI methodology.
MALE
IM AG E R EC OG NI TIO N AUTONOMOUS CARS
5
I M P L I C I T S K I L LS
Human skills are not always easy to explain!
MALE
OR
FEMALE
6
I M P L I C I T S K I L LS
Human skills are not always easy to explain!
For example, a Eine Beschränkung des
reduction of the Suchraums führt
search space does not beispielsweise nicht
immediately imply unmittelbar zu
better solutions. besseren Lösungen.
7
I M P L I C I T S K I L LS
Human skills are not always easy to explain!
8
L E A R N I N G F R O M D ATA
Instead of providing a complete and consistent description of domain knowledge, or
designing a model by hand, it is easier to ...
− give examples and let − let the system explore − demonstrate and let
the system generalize and provide feedback the system imitate
MALE
→ supervised learning → reinforcement learning → imitation learning
9
L E A R N I N G F R O M D ATA
computer scientist
“Machine learning is the science of getting computers
to act without being explicitly programmed.”
Andrew Ng, 2013
10
L E A R N I N G F R O M D ATA
computer scientist
DATA LEARNER
11
L E A R N I N G F R O M D ATA
computer scientist
DATA
LEARNER
PRIOR
KNOWLEDGE
Learning does not mean turning data into knowledge, but revising prior
knowledge in the light of observed data.
12
D ATA A N D K N O W L E D G E
Einfacher Zusammenhang,
hohe Messfehler
Unimodaler Zusammenhang,
mittlere Messfehler
100 Komplexer Zusammenhang,
geringe Messfehler
QUALITY
50
10 20 30 40 50
13
D ATA A N D K N O W L E D G E
Simple dependency,
high measurement error
Unimodaler Zusammenhang,
mittlere Messfehler
100 Komplexer Zusammenhang,
geringe Messfehler
QUALITY
50
10 20 30 40 50
14
D ATA A N D K N O W L E D G E
Einfacher Zusammenhang,
hohe Messfehler
Unimodular dependency,
medium errorKomplexer
Zusammenhang, geringe
100 Messfehler
QUALITY
50
10 20 30 40 50
15
D ATA A N D K N O W L E D G E
Einfacher Zusammenhang,
hohe Messfehler
Unimodaler Zusammenhang,
mittlere Messfehler
100 Complex dependency,
low error
QUALITY
50
10 20 30 40 50
16
D ATA A N D K N O W L E D G E
Simple dependency,
high measurement error
Unimodular dependency,
medium error
100 Complex dependency,
low error
QUALITY
50
10 20 30 40 50
17
A LOT HELPS A LOT …
Simple dependency,
high measurement error
Unimodular dependency,
medium error
100 Complex dependency,
low error
QUALITY
50
10 20 30 40 50
18
E N GI N E E R I N G A N M L S O L U T I O N
Modeling the learning problem Solving the model induction problem
− specify the type of problem and prediction − model selection
task to be solved − choice of the learning algorithm
− success criteria (accuracy/loss function, − estimation of generalization performance (e.g.,
model complexity, ...) cross-validation)
− ... − tuning of hyper-parameters
− interpreting and reacting to feedback gathered
Instantiating the learning problem from experiments
− postprocessing of models
− preprocessing, including feature selection,
− ...
normalization, etc.
− choice of the model class
− feature description
− kernel functions
− ...
19
M L PA R A D I G M S A N D M E T H O D O L O GI E S
ML Paradigms ML Methodologies
− Active learning − Deep learning
and experiment design − Gaussian processes
− Cost-sensitive learning − Graphical models
− Inverse reinforcement learning and Bayesian networks
− Meta learning − Inductive logic programming
− Multi-task learning − Kernel-based methods
− Online learning and support vector machines
− Reinforcement learning − Latent variable and topic models
− Semi-supervised learning − Markov networks
− Transductive learning − Preference learning and ranking
− Structured output prediction − Relational learning
− Transfer learning − Rule and decision tree learning
− … − Sparsity and compressed sensing
− …
20
M L - B A S E D P R O B L E M S O LV IN G
ML-SYSTEM
Induction principle Algorithm Model
Data
LEARNER
Prior
Knowledge
21
IMPRES SIVE P ERF ORMA NCE
ARS ELECTRONICA BLOG
https://ars.electronica.art/aeblog/
de/2018/08/20/style-transfer/
Der Standard, 3.4.2021
https://www.derstandard.de/story/2000125583234/kuenstlic
he-intelligenz-komponiert-neuen-nirvana-song
22
SU MMARY
◼ Machine learning is a subfield of artificial intelligence, which deals with the design and implementation of
adaptive systems that can learn from experience.
◼ With the help of learning algorithms, such systems manage to improve their performance in the course of
time.
◼ The field of machine learning is broad and covers diverse components and aspects:
− theoretical (learning theory),
− practical (design and implementation of algorithms),
− empirical/experimental (evaluation and performance analysis),
− application-oriented (use of learning algorithms in concrete applications).
◼ Therefore, it has connections to many other fields of science.
◼ It becomes more and more important in practical applications, both inside and outside computer science.
◼ As a consequence, the relevance of ML for society has increased a lot, a social aspects such as fairness,
transparency, explainability, etc. have become important topics of research.
23
